Learning and Experiments

Have you ever been in a post-activity meeting to discuss the outcomes of an event, and whether the plan in place was properly followed? If so, this a case of after-the-fact learning or rationalization that follows traditional business planning. Classical business planning calls for creating a plan, execution of the plan, evaluation of results, and looping back to minor adjustments of the initial plan. However, Eric Ries warns about “achieved failure” or successfully, faithfully, and rigorously executing a plan that turned out to have been utterly flawed” (2011, p. 22). In order to combat achieved failure, the Lean Startup method calls for validated learning, which is the process of demonstrating empirically that a team has discovered valuable truths about a startup present and future business prospects (Ries, 2011, p. 38). The reasoning behind finding these “valuable truths” is to carefully craft an experiment in which data-driven hypothesis are tested before vast amounts of financial capital, time, and efforts are spent on a given idea. While this theoretical reasoning based on the scientific method sounds complex, its application to business and startups is simpler than it looks.

Eric Ries’s book, The Lean Startup, extensively explores learning, experiments, and failures at several large organizations. Ries describes IMVU’s long journey to learning and accepting that their instant messaging (IM) product was fundamentally flawed. Also, he illustrates an experiment conducted by Nick Swinmurn which led to extremely valuable information and the eventual founding of Zappos. However, one of the most impressive illustrations of validating learning comes from the experiment the federal government conducted, based on the Lean Startup method, to successfully create and establish the Consumer Federal Protection Bureau (CFPB). Ries’ basic argument is to systematically figure out the right things to build given the extreme uncertainty of startups (2011, p. 52).

Looking back…

Reading about these large organizations and their experiences, made me consider the fact that they might not perfectly reflect small startups circumstances. Thus, I looked back at my prior experiences in the aviation industry looking for any trace of validated learning. In 2015, I was tasked with ‘doing something’ about our airport’s yearly required training program that seemed to be ineffective. The training had changed little to none over the years despite receiving negative feedback year after year. In Ries’ terms, the experiment had been conducted, yet the valuable truths had not been learned. My initial instinct was to review the feedback collected and assess how it matched with the current training program. Almost nothing matched. I grabbed a few ideas and toss out most of the old training program, incorporated the feedback suggestions, and accounted for the federal requirements. The result was a redesign program that met all the federal requirements, was extremely interactive and received close to 100% positive feedback upon its implementation.  

The fact that organizations continue to operate in a certain manner because ‘this is the way we have done it’ should make every business executive and entrepreneur cringe. Successful business and organizations evolve as needed. As Eric Ries points out in his book, The Lean Startup, even the federal government understands the importance of establishing a new government unit on solid and relevant data. Businesses can start small, use highly targeted advertising, online adds or whatever it takes to break from the past and obtain clear and quantifiable results. Achieving these results faster will save any organizations capital, time, and other valuable resources.  

Reference

Ries, E. (2011). The lean startup: How today’s entrepreneurs use continuous innovation to create radically successful businesses. New York, NY: Currency.

5 Replies to “Learning and Experiments”

  1. Hi Jose,

    I think you did a great job on this post. I hate the mindset of “this is how we have always done it.” My department is the exact same way. We have younger people joining the department every day but we will not change our ways, even when it comes to using less paper.
    People will say the department is not doing well or we need to update things but when we ask for input, no one gives it. A business cannot be successful if it is not willing to change. You have to look at where you are having problems and see what needs to be improved in order to be successful.

    Keep up the great work Jose.
    Warm Regards,
    Dani

  2. Jose,

    Rigorous review of actions is certainly a key part of building a solid foundation for success. My time in my high school marching band gives me something to relate to with your opening remarks about after-the-fact reviews. After every performance, the band gathered together to discuss the performance with the band director and staff. We would focus on areas that were already a concern and identify new ones to work on, and then end with a brief discussion about how well we did overall. A business should constantly validate itself from within, instead of relying on outside input on the matter. Something that I am curious about, however, is the concept of “valuable truths” in business. It’s my understanding that business is difficult to boil down to data-driven and tested rules, because of how unstable any such rules could be. The best I have seen are general guidelines that are too broad to be challenged by any specific thing. If you have the time, could you clear up what these experiments and objective, tested “valuable truths” generally mean?

    Patrik Gilley

  3. Hi Jose,

    Working in an old industry myself I hear the words “but this is how we’ve always done it” frequently. You are spot on where you mention that this should not be the case at all. It’s the mindset that stagnancy will create growth that actually kills businesses because they fall behind of their competition. Innovation is about iteration and continuous learning. If we do what has always been done, how can we ever do anything different?

    I really enjoyed how you pointed out that these examples from The Lean Startup were focused on larger more established companies. This does change the game, as this does seem more interpreneruial rather than a startup. I can see how these two environments would be drastically different in an approach for innovation because of important factors such as working capital, industry connections, brand recognition, and risk factor. An established company also has had more vetting opportunity to see which really are “valuable truths” but an established company also has history, politics, and sometimes differing management perspectives. However, the connection between the established company and startup does exist in its end goal: to validate a new product or service. Although slightly different obstacles, both still face challenges to discover their “valuable truths.”

    Carter Jones

  4. Hello Jose,

    I seem to encounter the philosophy of “this is how we have always done it” at each job I enter. It always seems to be a huge hurdle for any department or company that is stuck in that philosophy. My goal in my job has always been to bring in a fresh perspective even though the easy thing to do would be to adhere to the status quo. I like the idea of experimentation that the book above mentions as it allows for tests before springing a whole new idea or concept on a group of employees who are not used to change.

  5. First Jose please allow me to say WOW. I loved the opening of your website. That massive waterfall followed by a lady drinking a cup of coffee was amazing. While it was a short loop I had to watch it three times. Great effects.

    In your blog you absolutely hit the nail on the head when you said “The fact that organizations continue to operate in a certain manner because ‘this is the way we have done it’ should make every business executive and entrepreneur cringe”. You are so right. When building a successful organization of a reasonable since the entrepreneur may have great ideas but he/she needs help to implement those ideas. And with that comes the hiring of qualified people. Unfortunately many of those same people may come in with the same mindset “this is the way we have dont it”, and even worse, “this is the way we have ALWAYS done it”. I made changes in my organization chart recently where a couple of people did not like it. But I had to explain to them that if I want to continue getting the same results, I’ll just keep doing it the same way. But, I don’t want the same results. I needed new leadership and it is now in place but its still a slow process at times. Great work on the training reorganization and the near 100% positive feedback.

Comments are closed.