My experiment library unveiled
Cooks use knives. Carpenters use hammers. And experimenters use an experiment library.
Since childhood I have enjoyed libraries. I never thought I would create my own library. Not with books, but with experiments.
Experiment libraries are unusual outside science. During my academic tenure, I remember theoretical scientists with a shelf of notebooks full of experiment notes.
The power of experiment libraries
In an experiment library the most valuable thing is not the experiment. It is what you learn with the help of a well designed experiment. Remember that experiments do not pass or fail. We always learn.
When we think about a book library, we recognise that every book has been read by more than one person. It is of public knowledge. An experiment library is different. It is your unique record of learning towards a certain outcome. The chances that someone else has the same records is close to null. Thus, treat your experiment library as your intellectual property.
Imagine having an experiment library to run a business or a product. The wealth of knowledge accumulated is your competitive edge. Others will copy your business model or product. But they will never understand why you made certain decisions. They lack the full context and are operating in the dark.
Building an experiment library
For many years my experiment library was fragmented in my brain. I have designed experiments to simulate protostellar jets during star formation, to optimise gesture-based game controllers, to start a self-funded one man company, and to build digital products, customer journeys and digital platforms from zero to one.
When I started to build the experimentengines.com community, I decided to digitise experiments.
The first tool I used was excel. I wanted to start small, simple and avoid digital adoption friction.
In the second iteration, I created a new tool with no code app builder glideapps.com in an afternoon. It used the same core table columns from the first iteration plus new columns specific to a pretotype experiment.
This is the list of core table columns:
Name
Status (design, live, closed)
Start date
End date
Assumptions
Hypothesis
Design
Metric
Learning
Next step – proved hypothesis
Next step – unproved hypothesis
Result (proved, unproved)
This is the list of pretotype table columns:
Method
Market Engagement Hypothesis
XYZ Hypothesis
Target %
Hypozoom
Budget
In future iterations I want to add reporting to improve the efficiency of my main job – managing experiments from start to end. I would like to answer questions like “how many experiments did I run this year?”, “how many experiments passed/failed this month?”, and many more.
I am assuming that by now you are wondering how the library interface looks. What is more important is that you understand if you have the internal motivation to start experimenting. The right mindset matters more than the tool. Pen and paper can be a great starting point!
Using an experiment library
Building new habits was key. Can I create, update and review experiments over time? I increased the challenge of my deliberate practice by adding a two month break without experiments. Would I go back to the tool? Yes I did! Success.
These are habit building behaviours that I found helpful:
- Monday: design an experiment (a great accomplishment to start the week!)
- End of week: recap list of live experiments (an opportunity to remain engaged)
- End of month: celebrate wins, reflect ways to improve
Bonus: never peak into a live experiment!
If by now you feel an inner drive to start your own experiment library, start with these five steps. They should be agnostic to your current ways of working.
- define an assumption
- define a hypothesis using if-then format
- design a low cost experiment (can you leverage quantitative and/or qualitative data?
- define success using one metric
- run the experiment!
If you made it to the last step, congratulations! All that remains is to enter the above information plus the experiment results in your library. When you are ready, add more columns to your library.
If you want to create an experiment library or have one, I would love to hear from you!
