With experiments to validated business ideas
How to systematically reduce the risk of new business ideas
No one needs to become a scientist to conduct experiments to test business ideas. However, there are a few basic principles to keep in mind. In this article, I discuss why validating business ideas is important and how experiments are most effectively implemented.
Index
Why should I be interested in experiments?
Companies live and die by their ability to discover new and profitable business opportunities, continually deliver value to customers, and adapt to changing market conditions. Given new technologies, such as Artificial Intelligence, Blockchain, and 5G networks, we can expect more change and therefore more new business opportunities in the next 10 years than in the last 40 years [1].
At the same time, however, not every new business idea automatically leads to a profitable business model. Too many ideas are implemented prematurely because they look good in presentations and business plans and are based on hip technologies. What is overlooked is that not everything that can be built should be built. Rather, it is crucial to build what the customer actually wants. The fact that this is not always taken into account is illustrated, for example, by flop rates of between 70 and 90% for new products [2].
At the beginning, it is impossible to tell whether a business idea is a good or a bad one. And despite a flood of available information, managers usually lack the right data to tell the difference. Instead, decisions are made on the basis of experience, convictions and gut feeling. However, especially in the case of truly innovative ideas, it becomes apparent that these often do not form a helpful basis, resulting in the wrong decisions being made. This is not least due to the fact that the final implementation of the ideas lies in the future and is conditioned by a multitude of influencing factors. Success is therefore always associated with uncertainty and the risk of failure. Therefore, it is necessary to be aware of the key influencing factors that are crucial for a business idea to be successful and what assumptions we make regarding these influencing factors. In other words, what would have to be true for the idea to become a success?
This is where business idea testing comes in. The goal is to test the key assumptions of a business idea in the form of hypotheses by conducting experiments and thereby systematically reduce the uncertainty and risk of the idea. Even though business experiments do not have to meet the same strict statistical requirements as scientific experiments, some basic principles must still be observed.
Basic principles of testing business ideas
Make the key assumptions transparent and test them
In many companies, no new business idea is approved unless a business plan has first been created. Regardless of how much time has been spent on creating the business plan, at the end of the day, every business plan is based on a large number of assumptions. As long as these are not made transparent, a business plan is no help in estimating the value and risk of the business idea. Rather, the business plan merely represents what we believe – but without providing information about how likely this case is.
To determine the key assumptions, I would like to present three approaches that complement each other:
- Storyboards
In storyboards, the sequence of events of the new business idea is visually represented. For example, the assumptions regarding a product innovation could be raised on the basis of the individual phases of the customer journey. This involves answering the question “What would have to happen for the customer to enter the next phase of the purchase or usage process?” - Assumptions workshop
In a team workshop, the key assumptions of a business idea are made explicit and then prioritized. The assumptions can be differentiated according to (1) whether the idea is desired by the customer, (2) whether the idea is feasible, and (3) whether the idea is economical. The prioritization of the assumptions is carried out with a so-called “Assumptions Map”. In this, the assumptions are positioned in a matrix based on the available evidence for an assumption as well as the respective importance of the assumption. The execution of the experiments will then focus on those assumptions that are of high importance and for which little evidence is available. If the experiments show that these assumptions are false, there is a very high probability that the entire business idea will not produce the hoped-for result [3]. - Business models with Monte Carlo simulation
Monte Carlo simulation is a method from stochastics used to estimate the possible effects of uncertainty. In the context of validating business ideas, they help identify the particularly sensitive and important elements of the model. Since Monte Carlo simulations can also be performed in spreadsheet programs (such as Microsoft Excel or Google Sheets), they provide a quick and straightforward way to simulate the financial impact of uncertainty [4].
Formulate assumptions as falsifiable hypotheses
In order to be able to test the most important assumptions, they are written down in the form of a hypothesis in which a prediction is made about future events. The hypothesis should be formulated precisely and include a clear measure of success. In addition, the principle of falsifiability applies. This means that a hypothesis can only be disproved, but not proven, since it cannot be ruled out that it will not turn out to be wrong after all. For example, if one has seen only white swans in one’s life so far, it could be claimed that all swans are white. However, a single black swan would immediately disprove this [5].
Own data is better than foreign data
The goal of conducting experiments is to collect data from which lessons can be learned about how to proceed with the business idea. In doing so, one should not rely solely on data from external sources, but instead collect one’s own data. The advantage of own data is, on the one hand, its high relevance and topicality, since it is obtained specifically for the business idea to be evaluated. On the other hand, the evidential value and trustworthiness of the data can be better assessed. The experiments to be conducted for this purpose can often even be carried out more easily and quickly than searching for suitable data from external data sources and evaluating them [6]. Moreover, in the case of truly novel ideas, external data are often not yet available.
Test quickly and cheaply at the beginning
If the degree of uncertainty in an assumption is very high, only very little information is needed to reduce this uncertainty significantly. Therefore, small experiments with small sample sizes are sufficient in the beginning [7]. Importantly, they can be performed quickly and inexpensively.
In the context of testing business ideas, numerous terms exist for the object that can be used to test the idea. For example, while Eric Ries speaks of a Minimal Viable Product (MVP) [8], Alberto Savoia introduces the term pretotyping, a word combination of “pretended” and “prototyping”, i.e., a “pretended prototype” [9]. Regardless of the choice of term, it is important to note that a working product and prototype do not need to exist for business idea testing. Rather, the smallest, fastest, and cheapest thing that can be used to test a hypothesis should be used. For example, this could be a data sheet, an informational brochure, or a product video.
Often, one experiment is not enough to sufficiently reduce the uncertainty regarding the relevant assumptions. Further experiments should reduce the uncertainty until there is sufficient certainty that the business idea actually works. In this context, more expensive experiments are required only as uncertainty decreases and at a later stage of implementation of the business idea [10].
The conduct of experiments is not limited to new products
Conducting experiments is usually discussed in connection with the development of product innovations and new business models. However, the underlying principles can also be applied to internal company ideas (e.g. digitization of internal processes or introduction of new IT tools), especially if these are associated with high financial and time investments. Thus, testing of business ideas represents a systematic and generally applicable approach in companies for reducing uncertainty and risk management [11].
Experiments are not an end in themselves, but support decisions
Experiments are necessary to reduce the uncertainty of new business ideas and to make decisions. For this purpose, the data and lessons learned from the experiment can, for example, be transferred to the business model already used to identify the most important assumptions.
Especially when multiple experiments are conducted on a variety of business ideas, it is a good idea to capture them in an Experiments Canvas and track progress here [12]. In these, not only the assumptions and the conducted experiments are recorded, but also the associated decisions: Will the idea be pursued further, will a change of course (pivot) take place or will the project be abandoned?
Conclusion on business idea experiments
The risk of new business ideas is radically reduced by conducting experiments. Instead of investing a lot of time and money in the implementation of an idea that no one will ultimately need or use, the first step is to quickly and cost-effectively investigate whether the idea should be implemented at all.
Many companies are still at the beginning of a rethinking process towards more experiment-based decisions. Finally, as an outlook on the potential and effectiveness of experiments, a comparison of the share prices of companies where the permanent implementation of experiments is an integral part of the corporate culture (e.g. Google, Amazon and Microsoft) and the S&P 500 share index is used. The share price performance of the experiment companies, which carry out several 1,000 experiments per year, outperformed the S&P 500 by more than 10 times [13].
Even though there is still a long way to go to reach these top experiment companies. The key is to get started. Sometimes the experiments are successful and confirm the assumptions. And sometimes they don’t. But from every experiment something can be learned to further develop the business idea.
References
[1] Steve Brown, The Innovation Ultimatum: How Six Strategic Technologies Will Reshape Every Business In The 2020s, 2020.
[2] Alberto Savoia, The Right It: Why So Many Ideas Fail and How to Make Sure Yours Succeed, 2019.
[3] David J. Bland / Alex Osterwalder, Testing Business Ideas, 2020.
[4] Jez Humble / Joanne Molesky / Barry O´Reilly, Lean Enterprise: How High Performance Organizations Innovate at Scale, 2015.
[5] Ash Maurya, Scaling Lean: Mastering the Key Metrics for Startup Growth, 2016.
[6] Alberto Savoia, The Right It: Why So Many Ideas Fail and How to Make Sure Yours Succeed, 2019.
[7] Douglas W. Hubbard, How to Measure Anything: Finding the Value of „Intangibles“ in Business, 3. Edition, 2014.
[8] Eric Ries, The Lean Startup: How Constant Innovation Creates Radically Successful Businesses, 2011.
[9] Alberto Savoia, The Right It: Why So Many Ideas Fail and How to Make Sure Yours Succeed, 2019.
[10] David J. Bland / Alex Osterwalder, Testing Business Ideas, 2020.
[11] Jez Humble / Joanne Molesky / Barry O´Reilly, Lean Enterprise: How High Performance Organizations Innovate at Scale, 2015.
[12] Bruno Pešec, Visual tools for experimentation and innovation accounting, https://www.pesec.no/visual-tools-for-experimentation-and-innovation-accounting, 2020.
[13] Stefan H. Thomke, Experimentation Works: The Surprising Power of Business Experiments, 2020.
Comments are closed.