How to run a successful pilot, even if your idea doesn’t work out
Okay. So you have a new product or service idea, you know your target market and you believe you have a strong proposition that has stickiness. To prove you’re right you need to run a pilot.
Pilots are a great way to test your new idea without breaking the bank or over investing in an idea that isn’t proven. However, there’s a lot of pilots that go awry. Many are poorly organised or don’t prove anything because the goal posts were continually moved. Others fail because people disagree what they were actually testing.
So how do you ensure you run a successful pilot?
- Define what you’re testing and what you’re not
This depends on your scope but it’s worth writing down and agreeing in your team what it is you want to test. It might be how well the proposition lands with users or how the service works end to end. It’s also wise to identify what you’re not testing.
In a recent pilot we decided that testing technology that was seen as industry standard wasn’t our ultimate goal. Of course we gathered learnings but wanted to focus more on how well the proposition was taken up, understood and liked by product and service users.
Make sure you have a set of success outcomes and metrics in mind. It’s a good idea to build this into a simple dashboard so you can monitor throughout the pilot period.
If you’re testing an app make sure you monitor usage behaviour and not just downloads. Downloads are vanity metrics and won’t really tell you how sticky a product or service might be or become in future.
Be prepared to carry out some qualitative research alongside any data assessments you might be doing. Speaking to users will help unearth a lot of hidden insight as to how well your pilot is working out.
2. Remember MVP is actually an MVPS
Think Minimal Viable Product/Service. A personal bug bear for me, but too many pilots seem to test a digital product rather than a digital product combined with an offline service and experience. Someone booking a service through an app will still have to engage with that service in the real world. Your pilot needs to consider operations, back-end and areas like customer support.
3. Give it time
You need to give your pilot time to be thoroughly tested. Anything shorter than 3 months won’t cut it. A suitable timeframe is anything from 3–6 months but it depends on what your proposition is offering. Have a clear timeframe for when you see the pilot coming to an end. This fieldwork cut-off date helps ensure your pilot doesn’t just creep along, endlessly sucking up cash.
4. Do not move the goal posts
Once you’re in pilot mode avoid the temptation to change elements of the design while the experiment is in-flight. It can be tempting, but doing this removes the control element of the pilot and will make it harder to deduce any firm learnings.
5. But… be willing to kill if necessary
You might be unlucky and your pilot might be causing damage to your business or indicating obvious failures. If this is the case, then don’t hold back from pulling the plug. Be willing to stop the pilot and report back on reasons for the decision being made. It is better to stop, review and potentially re-launch than continuing to support a pilot that is heading for disaster.
6. Be okay with failure
Start out with optimism and a positive attitude but don’t be blinded by the fact that in reality, your pilot might not be successful. Make sure you work for a company that accepts failure as part of learning. Sometimes a pilot’s failure might be down to those that lead it, but don’t accept a role in a company where pilot failures are often unfairly pinned on individuals. For every pilot that is successful, there are 10 that fail, some spectacularly.
7. Take learnings and extrapolate if your pilot was to scale
Reporting back on pilot is all great, but stakeholders and especially investors will want to know what the learnings tell you about scale. A pilot might work with a small group of users but how would it work at 500,000 or 1,200,000. You’ll also need to assess how your customer support or back-end operation might need to evolve or change at scale. This is critical if you’re going to be able to say an idea has legs.
Just remember that while a pilot might not look like a scalable bet all is not lost. Most pilot scale concerns can be overcome with careful focus on technology and operations. Thinking about your pilot at scale will help you avoid the common mistake of throwing blood and money (people and cash) at a proposition to make it work. Not everyone has access to the deep pockets of Amazon or Netflix.
8. Run a series of A/B tests
Your pilot is a chance to test and experiment so don’t waste it. Where possible run A/B tests on certain aspects of the product and service that will help accelerate your learnings. This will also feed into better decision making when you considering scaling.
At Strategy Activist we collaborate to run pilots that prove or disprove product and service propositions and business models. We can support the very smallest start-ups right through to large global enterprises. To learn more about what we do visit us at www.strategyactivist.com or call us on +44 7786063053.
