Agile, is it just a delivery mechanism?

As a Agile coach, I refer to a few tools to help me think about where my Scrum teams should go next on their path to Agility. One of these tools is the Agile subway map, a list of Agile practices grouped in different categories. It helps me think how a specific practice could help a team solve its actual problem. In the following picture, we can see how these practices are interconnected while the colors at the bottom indicate each category.

Agile Subway Map


Another tool that I find useful is the list of the most popular Agile techniques surveyed by the annual Agile survey. Each year, VersionOne surveys thousands of organizations to get an overview of Agile adoption across our industry. One data point that I find interesting in this survey is the list of techniques implemented by Agile teams. Just like the Agile subway map, it helps me think about how a technique could help solve an actual problem the team is facing.

VersionOne 2013 most used Agile techniques


Unfortunately, I find there is something wrong with these practices/techniques. They are all based around IT. None of them really address how to do customer development. Customer development, a term coined by Steve Blank, is used to describe the parallel process of building a continuous feedback loop with customers throughout the product development cycle. While the reader could be tempted to consider the “Product Development” line in the Agile subway map above, notice how these practices are all related to the software development team and not the actual customer. Personally, I do not find these practices useful to discover customer behaviors to then engage with them.

Advocates of the Scrum framework would be tempted to mention how the Product Owner role can cover the customer development area. While it is true that Scrum and XP, two of the Agile methods that have stood the test of time, address business themes like value and Return On Investment (ROI), they fall short on guidance as to how these themes should be implemented in software development.

In the last few years, what seems to have gained a lot of attention in the Agile world is frameworks/processes to scale Agile to the enterprise level. The most popular ones seem to be Scaled Agile Framework (SAFe) and Disciplined Agile Delivery (DAD). In my opinion, these are just mechanism to deliver software to the customer. They want to solve the problem of delivering software at the enterprise level but they don’t seem to focus on solving the problem of the customer.

If you simply look at their summarizing high-level view, notice how the customer development, or the “outside the organization” box is missing. Can you find these boxes in the Scaled Agile Framework big picture? I can’t. DAD even says on its home page that “The focus of DAD is delivery”. The picture below is the DAD high-level view of its hybrid framework. Where is the customer development area in this picture?

Disciplined Agile Delivery

I truly believe these are just delivery mechanisms to push features out the door faster.

With Agile, you make more shit. But it’s still shit.

At the 2012 Lean Software and Systems Conference 2012, I remember Jeff Patton saying how “Value is hypothetical. You just don’t know”. In his view, value is a bet. While the Scrum guide states on page 5 that the “Product Owner is responsible for maximizing the value of the product”, I find that Agile has been pretty silent at helping the Product Owner achieving this. At a gathering of Scrum experts last June, I ended up in a session where we discussed how to identify value. The best answers we came up with were techniques like A/B testing, feature toggling and impact mapping. We, ourselves as experts, were having a hard time naming techniques to measure and identify value in products.

Jeff Patton said another interesting thing during his presentation in 2012. He said “With Agile, you make more shit. But it’s still shit”. In other words, Agile has become a software delivery process where we make features faster. But we aren’t solving the problem of the customer. This seems to go in the same direction as Mary and Tom Poppendieck pointed out in their latest book “The Lean Mindset”. In the epilogue, they state how “agile methods often fail to deliver significantly improve business results”. It is now time “to stop thinking about software development as a delivery process and to start thinking of it as a problem-solving process, a creative process.”

Where to look next?

So this leads to an interesting conundrum: “How do I setup a problem-solving process?”. I have to say that I have found some of the answers in the Lean Startup movement. Lean Startup, a methodology first proposed by Eric Ries in 2011, aims at developing businesses and products using the scientific method. As it is gaining popularity, it has attracted the attention of many Agilists.

While I am not saying that we Agilists should switch to the values of Lean Startup, I have found some novel ideas when comparing Agile and Lean Startup techniques. In his blog post “Agile Vs. Lean Startup”, Joshua Kerievsky distill the differences between these techniques. You can also watch his presentation at GOTO conference to get a better understanding of his thoughts on the subject.

One technique that I find appealing is the AARRR metrics, humorously named the “Pirate metrics”. While velocity tells you how much features are going in production, it doesn’t tell you if customers are using these features. In fact, I am not aware of any popular Agile techniques (see Agile subway map or VersionOne survey mentioned above) to help us measure the value of features once deployed in the field. On the other hand, the AARRR metrics offer a very tangible way to measure this. To the reader who is interested to dive into more Lean Startup customer development techniques, I recommend the book “Running Lean”. It presents and describes in detail techniques related to customer development. While we ran short of answers at our Scrum expert meeting last June to measure value, Running Lean offers a breath of techniques like feature fake, scroll heat map and structured customer interviews to work with your customers throughout product development. is also providing some of the answers through an important initiative it is leading. Named “Evidence Based Management (EBM)”, it sets out on measuring business value metrics. Don McGreal has a great article on how we should set out to measure evidence about the value our customers see in the features we deliver to them.

While I feel that Lean Startup customer development techniques can help Agile deliver value in product development initiatives, I am a bit more skeptical about projects where we rewrite applications. For example, I work in a government town where public government agencies rewrite their existing applications because they are too costly to maintain. As these applications fit into a business process already in place, we find ourselves implementing the same features simply with a newer technology. In this context, I would be less warm at introducing customer development techniques as we are replacing an existing system over an existing business process. On the other hand, if we would extend the system to new platforms (Ex: mobiles and tablets), I would be tempted to include Lean Startup techniques to truly identify and measure what is value to the eyes of the customer.

Compared to Agile methodologies, where I feel we are working forward from the IT team, Lean Startup moves backward from business results that we are trying to achieve. This leads me to an interesting question. As I am an advocate of the “form follows function” principle, should we then design our software delivery process to serve the purpose of customer development? And if so, could fundamental practices like time-boxed iterations and TDD be dropped for the greater good of flow? As A/B testing, a customer development technique, would be used to challenge an existing feature with a new one, could we drop TDD until the right feature is discovered. As Agile is now in its second decade of existence, I am curious to see if the manifesto will survive an industry that is continuously maturing. Time can only tell.

Louis-Philippe first discovered Agile while toying with test-driven development and continuous integration in 2004. He then encountered Scrum and never looked back afterwards. Involved in dynamic, innovative and international environments, Louis-Philippe is an enthusiast natural leader and goal-oriented. Louis-Philippe enjoys working with people and brings a vision to an organization. He tries to do all that while preserving his strong software engineering skills best practices and methodologies. Louis-Philippe is a graduate in computer engineering (B. Eng) from the University of Sherbrooke and holds a business certificate from the same university.

  • David Starr

    Yes. :)

  • Marcos – Slipmp

    Good one!

  • Lare Lekman

    Thanks Louis for the great article. I also believe that value is hypothetical until we get money from buying customers. However, at least for myself, the empirical process framework itself is the most useful tool for identifying value. No bells or whistles needed. Like firing a heat-seeking rocket and watching it find its way. You could call me a Scrum advocate here, but after seeing the “heat-seeking” mechanism find value at tens of organisations, I believe in it. The paradox for myself is, why spend too much time on guesswork, as we can never be sure (until the customer buys our product)? Personally, I prefer to inspect and adapt, with minimum effort, until my MVP (Minimum Viable Product) value hypothesis is proven right or wrong.

    • Louis-Philippe

      I find both approaches use data/evidence to move forward. Scrum uses the keywords “inspect” & “adapt”. Lean Startup uses “measure” & “learn”.

      What’s bothering me with Scrum is that “Done” doesn’t help me identify value. It’s a quality assurance mechanism before the increment gets out the door. The Lean Startup guys put the Validated (qualitatively) and Verified (quantitatively) steps once the increment is in production. I believe this is how we can measure value.

  • scottprimeau

    Louis, thanks for the article. Whether Agile is just a delivery mechanism or not, finding ways to identify and meet customer needs is critical, whether that’s through surveys, focus groups, product owners, inviting actual customers to sprint reviews, etc. I’d also suggest that rewriting an application is a very good opportunity to address customer needs, improve processes, eliminate unneeded features, and make not just a rewritten product, but also a more effective/valuable product.

    • Louis-Philippe

      I couldn’t agree with you more Scott.

  • Kevin Fleischer

    Regarding your last question about dropping some practices to get the MVP out of the door:
    I don’t think there are many practices to strip from the agile dev toolkit. TDD ie.: it increases you time to customer, when you count buggy SW as “not reaching the customer, because he can not evaluate it due to the bug”. Adding tests later is a pain in the neck… And maybe a reason for the reduction in using UnitTests in the survey you showed.

    Btw: Thx for sharing the subway map. Haven’t seen it before.

    • Louis-Philippe

      I find there is value in the Agile dev toolkit but I’m starting to think we should first focus on customer development and pick dev. practices afterwards. I like how the Lean Startup guys validate qualitatively and quantitatively (see Running Lean from Ash Maurya) before determining a feature should stay in. Somehow, doing full TDD on something I’m not sure I’ll keep sounds right to me. But I haven’t had any experience in such a project yet so I’m only thinking outloud.

      • Kevin Fleischer

        I never said, that it makes sense to “achieve failure” as Eric Ries says in “The Lean Startup”. But its rather artificial to compare strategic tools and tactical tools. Scrum is a tactical tool. It is the preferred way of getting work done. It is not intended to determine what goals you should aim for.

        But I would definitely advise to use agile dev. practices when you facilitate your Lean Startup experiments. Else you run in the same problems that IMVU had, when they went “main stream”.

  • Samuel Cavalcante

    Very good post, led me to think about it.
    With the look you presented in the post, I agree with your opinion.
    But I see that the best way to agile is:
    1 – look at the Manifesto
    2 – focus on 12 principles
    3 – using methods that assist the production, focused on topics 1 and 2.
    4 – Feedback and continuous communication with the client and the team.

    PS. I’m not good at English, literal translation.

    • Louis-Philippe

      No worries Samuel. English is also not my first language.

  • Éric Sylvestre

    “With Agile, you make more shit. But it’s still shit.”

    Strong sentence, but still true for most projects.
    Hard to hit the “perfect” product for customers and keeping costs of development low.
    A/B testing and all others processes need time and money to be put in place.

    • Louis-Philippe

      Indeed, strong sentence. I recommend watching the presentation of Jeff. He’s a great presenter and the part about the underwear gnomes at 18:00 is hilarious.

  • Erik Buitenhuis

    Totally agree that we need a more structured way to measure and deliver value. There are some attempts to come up with these metrics (the Evidence Based Management article mentiones a few) but they are not (yet) part of the standard Agile toolsets and frameworks.
    But I don’t agree we are efficiently delivering more shit than with traditional methods. I think that delivering value is exactly what Agile is good at! The big gain from using Scrum is effectivity, not efficiency! Yes, higher velocity is good but delivering the right story points (with the least possible effort) is better. And though we don’t have the practices in place yet to measure and maximize value in a scientific, quantitative way, many Product Owners are somehow managing to have their teams working on more or less the right things.
    This is true for all those organisations I have worked with that are at least seriously trying to do Scrum (willing to adopt an agile culture). How do I know without measuring? I can’t give you the numbers of course but the gains were so big that it is totally obvious!
    So, many organisations are succesfull delivering value with Agile. But Product Owners need more help in maximizing value, they have little more than their intuition to guide them right now.

    • Louis-Philippe

      Thanks Erik for sharing your experience. I’m anxious to discover the techniques POs will use to measure the value delivered to their customers. Indeed, we have qualitative data by talking with customers but I would like to see more quantitative data to measure it.