Saturday, February 6, 2010

How to Measure Success

In Australia over the Christmas break I was asked by the CEO of a startup company how to tell if he had the right product strategy. Some companies believe (falsely) that you can analyze success into a product by doing greater and greater amounts of market research and analysis.

The true answer is extremely straightforward - you gain multiple paying customers.

We live in a capitalist economic system, and money is our store of value.  Therefore if a customer buys your product, they are clearly demonstrating that you are delivering them value at least equal to the purchase price. Selling the same product multiple times without customization proves you have built something repeatable.

All other ways of measuring product success, whether it be market share, market recognition, analyst awards etc, are irrelevant if you fail to sell the product. One large software company in particular sets goals around market share, but it does this because as a certain % of market share translates to a certain % of market revenue.

Therefore, any company that wishes to minimize product risk, needs to get to market rapidly, with a minimal viable product (MVP), and gain paying customers during the development process.  Finding 2-3 customer willing to commit to buy your product and engage during development dramatically reduces product risk.  If you can't get multiple customers to commit to buying your product before you start developing, this may be a signal that you are building the wrong thing.

As an example, while at Citrix we developed the Citrix Secure Gateway (CSG), that later morphed into the Access Gateway and quickly became number #1 in the SSL VPN market, and over $100M revenue. CSG was built by a small Citrix skunkworks team in only seven months, and very early in the process Lehman Brothers engaged as our development partner.  They actively helped during the development process by reviewing our specs, meeting with the team, detailing their requirements, and committing to using the product.

Compare this to a project at a different company, where after two years of defining requirements, development, testing, marketing, sales and SE training, the product didn't sell - it was essentially DOA.  There were a few main differences:
  • No customer(s) who had committed to buying the product on release.
  • No focus on delivering the minimum viable product (MVP).  Requirements blew out as "must haves" kept on being added.
  • No development customer partners.
We've all worked with development teams who get motivated by solving cool or difficult problems, and view financial goals as non-motivating.  As a Product person, you need to explain carefully to these people that the financial goal is simply a measurement of the success in meeting customer needs.  To meet the financial goal we still need them to do their magic and deliver great products.