Calculating WAPE

WAPE stands for weighted absolute percentage error. It is a very simple calculation that is woefully under-represented on the internets…so here goes the most simple explanation.

1) You have a set of baseline measurements that are, in your professional opinion, the best (often actuals)
2) You have a set of experiment values for which you want to calculate the WAPE against the baseline.
3) For each value in baseline, calculate an absolute difference field (e.g., absolute value of Baseline – Experiment)
4) Sum all of the absolute differences
5) Divide that sum of absolute differences by the sum of all baseline values.

Boom. You have WAPE. The benefit for WAPE is that it lets you see your error on an item by item basis at an aggregate level. Replay. If you just calculated the error at an aggregate level, a lot of item level shifts (aka-errors) would cancel themselves out…and that is bad. If you just averaged the errors on the item level, you might have a nominal item with a high error rate seriously throwing off your results. WAPE is the way to get the aggregate error of your set of measurements.

Simple Example: What does a fruit shopper purchase at a farmer’s market?
Baseline: what we measured this weekend:
100 Apples
20 Pears
10 Bananas
Experiment: what we predicted they would buy
80 Apples
30 Pears
5 Bananas

The worst thing you can do is just say “expected purchases v. actual purchases.” With this, you would have 115 experiment v 130 actual, or 15/130 = 11.5% error. The problem is that some of your under-estimates cancel out some of your over-estimates.

Absolute Difference
|100-80|=20
|20-30|=10
|10-5|=5

Absolute Error
20/100=.2
10/20=.5
5/10=.5

If we were to average these values, we would end up with .4 or 40% error rate (or MAPE). . But this is not right…our biggest measurement was most accurate, so we should punish ourselves less than the average. This is where WAPE comes in: weighting your errors based on their significance.

20+10+5=35
100+20+10=130
35/130=27%

Here, our WAPE is 27%, Not as good as delta/actuals (11.5%), not as bad as MAPE (40%), but just right at a weight adjusted 27%.

Let me know if you have any questions.

Thanks.

Matt

Advertisements

Creating an LLC in Washington State

It is usually a good idea to set up a legal entity when you are embarking on a new business venture. But, you have a million things to do, so this gets tacked on to the list – usually a bit near the bottom because you think it is going to take a long time.

Instead of adding this to your to-do list, just spend ten minutes knocking it out. Here is how:

  1. Create an account at https://ccfs.sos.wa.gov/#/UserRegistration (chose User Account > Individual)
  2. Create or register a business https://ccfs.sos.wa.gov/#/businessFormationIndex
  3. Select WA LIMITED LIABILITY COMPANY (there are a lot of options…so spend weeks learning about those differences, or just chose LLC).
  4. Defer initial report for 120 days. Add a reminder to your calendar for 3 months out so you do not forget!
  5. Specify yourself as the executor and authorized person.
  6. Chose a name with LLC in it (e.g., RANDOM STUFF LLC – that is the LLC that Oliver and I set up to license PuttMate) – you may never even use this business name (you can operate under a “DBA” or doing business as, so do not spend time/energy worry about it).
  7. Pay $200
  8. Get a FEIN (Federal Employer ID Number…basically the SSN of your business) https://www.irs.gov/businesses/small-businesses-self-employed/employer-id-numbers
    1. In honor of the 1990s, the FEIN website is only open Monday – Friday, 7AM – 10PM eastern time, so tough luck you weekend warriors!
  9. Complete a Business License Application http://www.bls.dor.wa.gov/ > https://secure.dor.wa.gov/home/ (note: you will need your SSN as well as the SSN of any of your partners).
  10. Pay $20

GoDaddy, you broke my heart

For starters, you broke my website.

Recently, I have sporadically seen random characters being appended to my domain redirect URLs, so that when somebody types in this:

www.puttmate.com

They end up here:

https://inventoselive.azurewebsites.net/Prototypes/Details/35/UMheZ/

Instead of here:

https://inventoselive.azurewebsites.net/Prototypes/Details/35

Notice the random 6 characters that are being appended to the first URL: UMheZ. Since my server does not have a path/page/route that maps to these random characters, it does not render a page.

It seems like I am not the only one having this problem:

https://www.godaddy.com/community/Managing-Domains/My-domain-name-not-resolving-correctly-6-random-characters-are/td-p/60782

So why do I claim that GoDaddy broke my heart, and not just my website? Because of the absolute apathy, borderline antipathy, they have shown to their customers. I would expect them to fix this. But I get it-the solution may be complex, it may be caused by somebody else’s issue, the Internet is a complex place.

But there is simply no excuse to say “not my fault” and consider the case closed. We are your customers. Tell us “sorry this is broken, we are working to fix it and will let you know once we do; in the meantime, adding a ? to the end of your redirect domain usually solves the problem; I have sent you a screencast of me doing that if you want to see exactly how it is done.” The fact that they are so eerily quiet on this thread brings me to a sad realization: the love has gone from our relationship, and it has been for years now. We are only together because it is too expensive for us to break up.

But I think it is time for me to get back in the game, and find a domain partner who really likes me, who brings me unexpected surprises that make me happy, and works hard to make it right when something bad happens. Yep, we are breaking up.

If you have had a positive experience with GoDaddy that can offset this experience, I would love to hear about it. If GoDaddy broke your heart too, tell me in the comments below.

Have a great day.

Matt

Up and running with TensorFlow on Windows

I was having some problems installing TensorFlow the way that is described on the webiste https://www.tensorflow.org/install/install_windows

I tried both pip and Anaconda, and failed.

The pip installation gave me this error:

Importing the multiarray numpy extension module failed. Most
likely you are trying to import a failed build of numpy.
If you’re working with a numpy git repo, try `git clean -xdf` (removes all
files not under version control). Otherwise reinstall numpy.

The Anaconda gave me this error message:

tensorflow-1.2.1-cp35-cp35m-win_amd64.whl is not a supported wheel on this platform

and later

ImportError: No module named ‘tensorflow’

I then tried to pip using this:

pip install tensorflow

And, if I used an Anaconda command prompt, everything worked like a (sort of) charm. You can test it out by:

activate tensorflow

python

and then running this in the python console:

>>> import tensorflow as tf
>>> hello = tf.constant('Hello, TensorFlow!')
>>> sess = tf.Session()
>>> print(sess.run(hello))

Not sure if one of the original 15 attempts set this one up for success, but if you are having a hard time getting Tensorflow installed on your windows machine, this mine be the trick for you as well.

Regards,

Matt

New Product Development Process

There is no one-size-fits-all process for new product development, but I liked this one:
New-Product-Development

I saw it while reading CTL.SC1x Key Concepts􀀁MITx MicroMasters in Supply Chain Management (link), and I believe that it is extracted from “Cooper, Robert (2001) Winning at New Products.”

I am hoping to do a write up on how this conventional product development pipeline struggles to add value to an enterprise. The primary challenge is connecting an unpredictable, non-linear invention process with a corporate framework that requires predictability. More to follow soon.