The book consists of the learnings which the author had while working on his startup IMVU. The book focuses on the concept of validated learning and the build-measure-learn feedback loop. It tries to bring in a systematic approach to measuring the progress at a startup. A startup has a true north, its vision. It employs a strategy that includes a business model, a product road map, and a view of partners, competitors, and customers. The product is the result of the strategy. Products constantly change (engine tuning). Strategy changes occasionally (pivot). Vision rarely changes. In general management, failure to deliver results is caused by failure to plan or failure to execute. Both are frowned upon. But in the modern economy, both are useful tools for testing new ideas.
A startup is a human institution designed to create a new product or a service under conditions of extreme uncertainty. Success under such scenarios requires rapid experimentation.
There is a lot of learning involved in the process, and sometimes, when things go south, people resort to saying, “I learned a lot”. What’s more important is to figure out validated learning. The goal is to cut down to the absolute minimum effort required to learn what customers want and eliminate everything else. Eric Ries wrote 3-D avatars as IM add-ons for popular IMs in 2004. During the user testing phase, he realized that not only do users not understand what an add-on is, they also don’t mind installing a new IM software. A lot of heavy IM integration turned out to be a waste of effort. After this, Eric and his team launched several experiments regularly to test what works and what doesn’t.
Another counter-intuitive thing that IMVU experimented with is charging early. Many startups delay charging their customers. Not having any revenue is better than having low revenue since the former invites the imagination of overnight success once they start charging. The downside is that it can lead to the creation of a product no one is willing to pay. Conclusion: start with a low-quality prototype, charge customers from day one, and use low-volume revenue targets for accountability.
Nick Swinmurn had a hypothesis that people will buy shoes online. Rather than purchasing inventory upfront, he took photos of shoes at local shoe stores, and if the users bought it, he would buy and ship it to them. This minimum product tested customer demand as well as many other business issues like payments, returns, and customer interaction. Amazon acquired Zappos acquired for $1.2 Billion.
- An experiment starts with a hypothesis. A value hypothesis tests whether the product/service will deliver the value to its users. A growth hypothesis tests whether new customers will be able to discover the service.
- To test the value hypothesis, find some customers to experiment. Don’t go for an average customer but find an early adopter, whose needs are most accurately served by the product.
- Now, build a concierge minimum viable product. Such early adopters are more forgiving of the quality of the product, and their feedback is useful to know whether the product fulfilled their needs or not. If users complain about a missing feature and they’re on the roadmap, that’s a good thing, since it implies that the team understands their customer. If there is a feature that is on the roadmap, but the user does not complain about it being missing, then that’s an indication to remove that feature.
- The results will guide you to validate the hypothesis.
The Build-measure-learn feedback loop is at the core of the Lean Startup model. After building the MVP, the goal is to rapidly learn and iterate upon the product based on the user’s feedback.
All startups make some assumptions (leap of faith) about their viability. In the case of iPod, there were two assumptions, users would put earphones in a public place (“analogous to Walkman”), and they would pay for the music (“antilogous to Napster”). Only the second one was a leap of faith. Verifying these assumptions is paramount. Toyota does it via Genchi Gembutsu (“go and see for yourself”). Toyota’s minivan, Sienna’s chief engineer, drove through North America. He realized that kids are most appreciative of their environment and launched the new model with a particular focus on interior comfort for long trips. This lead to significant success for Toyota. Scott Cook, the founder of Intuit, believed that someday people would use a computer to pay bills and track expenses. He verified that the market for such a product by calling random people over the phone. One pitfall to avoid here is analysis paralysis. One can keep repeatedly talking to customers and whiteboarding over and over again. But many errors in such a strategy would go unnoticed since they depend on subtle interactions between the user and the product.
Groupon started as a “collective activism platform”. That assumption failed. Andrew Mason experimented with a WordPress blog and a mailing list to sell discount coupons. They had no fancy forms on the site. The idea took off, and slowly every aspect of it was automated. It is essential to get early results with a buggy product then to perfect about based on assumptions that might not hold in the future. It is counter-intuitive for entrepreneurs who want to build a high-quality product.
Drew Houston had a hard time convincing investors about DropBox. VCs thought that the market is crowded, no one made money, and the problem was not an important one. Drew believed that all that was because all the current products were of low-quality. Rather than spending years doing thorough integrations, he made an excellent video demonstrating the seamless behavior.
Wizard of Oz is a useful testing process for such situations. Rather than building an automated system, fake it with a human. It is faster for learning what users want.
Sometimes, a user’s quality metrics are very different. Users care about how much they enjoy the product, not how much time was spent building it. IMVU had no time to build a smooth movement of avatars from one place to another. So, they decided to and were ashamed of, cheat by making avatar re-appear at the destination instantly. Users rated this teleportation among the top three most liked features. Therefore, it is of paramount importance to remove any feature, process, or effort which does not contribute to the learning you seek.
Some entrepreneurs fear the competition that MVP will bring in, usually, from large companies. Most of the time, the Product Managers at the big companies are overwhelmed by good ideas. If not, they can still copy the product at a later stage. And the fear of being out-executed remains.
After building an MVP and putting it out for the early adopters, test the riskiest assumptions first. Now define a baseline metric, a hypothesis to improve the metric, and a set of experiments targeted towards the same. Once you have the results, decide whether to pivot or persevere. One of the biggest dangers is to get stuck with vanity metrics like total registered users. They paint a rosy picture but does not tell you whether the product improvements are making it better for the user or not. AAA (actionable-accessible-auditable) metric would measure the impact of a particular feature.
Grockit followed the Kanban model, where there are four buckets – backlog -> in progress -> built -> validated. Each was containing at most three features. After validation, either they made the feature or discarded it.
Pivot or Persevere
A startup’s runway is the number of pivots it can make. Votizen started as a social network for verified voters to discuss civic actions. That did not take off. Then it pivoted to @2gov, which allows users to recruit more verified voters for their petitions. This product has higher usage, but still, very few were willing to pay for it. They pivoted further to businesses as customers, who, despite signing the letter of intent, decide to eventually not buy the product. The final pivot was to use Google Adwords for acquiring users who want to pay to acquire more users. That worked out. Startup Visa Act was solely a result of that social lobbying.
Wealthfront pivoted from a virtual stock trading/gaming platform to an online service offering money management by professional money managers (ashishb’s note: and after the book was written, further to index-based investing).
Most entrepreneurs regret delaying the pivot. Vanity metrics, not having a clear success hypothesis, and being afraid of the failure, are the usual causes of delaying the pivot.
Types of pivots
- Zoom-in Pivot- A popular feature becomes the new product. Votizen moved from voter social network to a voter contact product.
- Zoom-out Pivot – The current product becomes a feature of the new product.
- Customer segment Pivot – The target customers change.
- Customer needs Pivot – where the customer base remains the same, but the product changes to suit them more. Potbelly Sandwich shop started as an antique store in 1977. It decided to sell sandwiches to bolster traffic. It eventually pivoted to become a sandwich shop.
- Platform Pivot – The product changes from a single-use product to a platform for the other products.
- Business Architecture Pivot – Geoffrey Moore observed that most companies follow either a high margin, low volume model; or a low margin, high volume product. Former is usually for B2B, and the latter is generally for B2C. A business architecture pivot is jumping from one to the other or vice-versa.
- Value Capture Pivot – where the way business makes money changes.
- The Growth engine Pivot – The business’ way of reaching new customers changes.
- Channel Pivot – The distribution channel for the product changes.
- Technology Pivot – The underlying technology to do task changes.
When pivoting to a strategy followed by a successful company, it is crucial to copy the essential and not just the superficial features.
It is counterintuitive, but smaller batches are much better for lean startups. They appear inefficient but allow faster turnaround for the product leading to a more rapid iterative cycle. It helps in earlier detection of a problem as well as quick feedback from the customers. Toyota used the small-batch approach to compete with its much more capitalized American counterparts whose batch sizes were relatively bigger.
New customers come from the actions of past customers. They inform others, end up showing the product to others, or end up purchasing the product again. Sabeer Bhatia grew Hotmail by adding a signature “Get your free e-mail at Hotmail” to every outgoing email. If you are asking whether your startup has achieved a product/market fit, then you are not there yet. When the product/market fit happens, it leaves no room for doubt.
As a startup grows, it has to adapt to the changing customer base. Early adopters are more forgiving of the quality; later ones are not. Five whys help one to diagnose the problems and build the right set of things that should go into an employee training manual.
Big companies can innovate, but for that to happen, they should secure resources for internal teams, provide an independent development authority, and the internal team should get a stake in the outcome. Toyota calls the manager in charge of running the development of a new vehicle, shusa (Chief Engineer). The parent organization must be protected to create this platform for experimentation. If the existing managers feel threatened, they will have an incentive to work against the new project. Also, if such a unit is kept hidden, it will attract more political battles since existing executives will be wondering what else could be hiding. Therefore, any team should complete ownership to run an experiment and see the end-to-end results. Every company has to deal with four types of works – launching a new product, scaling it for the broad adoption, combating its commoditization by incremental improvements, and maintenance of the product in the long run as a part of the company’s product line. All the steps are essential, but the last stage of becoming the status quo is a hard one to swallow as an entrepreneur.