Software Product Development Is Not A Production Line
How coding factories are born
“It’s possible for good people, in perversely designed systems, to casually perpetrate acts of great harm on strangers, sometimes without ever realizing it.“ — Ben Goldacre
Software products don’t roll off a conveyor belt. There are no physical parts to connect, no screws to tighten, and no bolts to torque. It is a trade that is more steeped in development than production. Yet the industry in aggregate conspires to convince us that Software Product Development must be contained in a manufacturing process, with the hope being we can build software like we build automobiles.
This line of thinking often germinates from a casual and misguided understanding of Lean Manufacturing and mass production.
Lean, which provides best practices, values, and principles that support software product development, is often unable to shake its manufacturing roots. With little thought on the part of adopters, the conflation of a Lean culture with the production line in a creative endeavor like software development does more harm than good.
Coming Of Age
Software is a young industry, approximately 50 years old when compared to enduring industries like automotive with a milestone of 120 years, electrical (a shocking 130 years), and construction, which built the world through the ages. Like all younglings, the software industry looked up to its elders and borrowed practices, values, and principles.
The industry, however, found itself tethered to a mold mostly consisting of upfront requirements, followed by design and then build processes, as all the traditional brick-and-mortar industries emphasized the manufacturability portion of the product creation process.
Development was decoupled, often only mentioned as an afterthought in support of production, value was seen to be created on the manufacturing floor.
Mass production was running a mock as the software industry was finding its legs and a great many of its practices, values, and principles made it into the lore of Software Product Development. The influence took hold early and has been a dead weight in an industry that thrives in agility.
Manufacturing Software
Around 2012, McKinsey published data in collaboration with the University of Oxford on software projects — these statistics have become well-known across the industry since. Their findings:
“…suggests that half of all large IT projects — defined as those with initial price tags exceeding $15 million — massively blow their budgets. On average, large IT projects run 45 percent over budget and 7 percent over time, while delivering 56 percent less value than predicted.”
In light of this data, emerging effective, then efficient methods to create software is a solemn responsibility of all software organizations.
However, to what can only be seen as a long-term detriment and a regression for the software industry, more and more companies and teams are knowingly or unknowingly setting up their software development organizations to operate more like production lines — complete with siloed teams, cycle time measures, and zero tolerance of variance — versus emerging systems that support effective development tailored to their culture.
In adopting “production line thinking”, organizations set up what they hope to be coding factories, where software moves down the line, from area to area — Development, User Experience, Testing, Quality Assurance, and so on — then eventually make it to the customer.
By casting development as a production line, other manufacturing practices are applied, such as measuring the time it takes to move the software product between areas and holding variance as close as possible to zero by requiring lots of upfront design and planning.
What teams eventually notice by taking a closer look at the supposed feature factory, is the high cognitive load, high communication overhead, low product quality, a litany of patches, incomplete work, little innovation, and employee burnout.
These issues arise due to the misguided but steady effort by the managers of the process to keep the line moving. The linear flow of work the production line expects is eventually compounded by the rapid cyclical nature of development, the system in turn collapses on itself, yielding only drips of value if any.
The result is a “production line” that is untenable and bursting at the seams that cobbled together a software product that is over budget, delayed, and missed the mark on promised value.
The production line is an ill-suited metaphor for the deep, creative, and technical work involved in developing software products.
Software Development Lifecycle Process
To the uninitiated, every time Lean comes up in software product development, TPS (Toyota Production System) is what comes to mind first — understandably so. In The Machine That Changed The World, Womack and the team reframed TPS as Lean and focused on manufacturing practices and values. Lean however has universal principles that apply equally well to development — continuous learning, continuous improvement, and respect for people.
Most Lean adopters in software development gravitate towards TPS, paying no attention to the fact that it is a Production system, not a Development system, and tailored to Toyota culture.
This oversight sends teams down a path rife with poor quality products and burnout due to unnatural asks of deep knowledge work that includes treating the software development process like a production line, expecting a linear flow of work, and having zero tolerance for variance among others.
TPS is not suited for Software Product Development.
In their book, Lean Software Development, Mary and Tom Poppendieck suggested an apt analogy for thinking about the difference between development and manufacturing, which is, development is the act of creating a recipe (deep work) and production or manufacturing is the set of steps that executes a recipe (shallow work).
In Software Product Development, the lion’s share of the effort rests in development, there is no ceremony in production. All the fanfare is in development. As the programmer conceives the logic and writes the code required to execute the sequence of operations, the software is “produced”.
Development is front and center; the means and the ends of creating a software product.
There is a process, one that is as lean as they come, a development process — ideation, prototyping, feedback. These are not a linear sequence of steps, but a loop that adds form to an intangible construct with each pass.
Developing software products is like creating a painting. With her brush in hand and reference in mind, the painter confronts a blank canvas. She starts by blocking in large areas of the image (idea) on the canvas, this gets her quick feedback on the overall composition. Brush stroke after brush stroke, layer after layer, details are added as the painter cycles through ideation, prototyping (a new stroke), and feedback.
From the first stroke, through the rest of the painting process, the product is visible. Each new choice is built atop a previous choice.
The idea is developed and brought into focus at each turn. Throughout the development process, the painter keeps as many avenues as possible open. There is no one “right” direction at the start; the variance is tolerated for as long as possible. The path emerges along the way, providing choices on where to stop.
Software is all virtual. And, only exists in the mind’s eye of the programmers and designers who in deep thought plucked it from their collective imagination, then with a combination of art and science lay the disjointed bits onto silicon to form the ephemeral construct we click about in.
Programming exacts a heavy toll on the mind as it works to keep a mental picture that can flee with the slightest break in concentration. It is deep work. Complicated work. It is development.
Rendering Software Product Development to operations on a production line is demeaning to the craft as well as the engineers and designers who ply the trade. This mindset leads to employee burnout, low-quality products, and inflated costs.
Instead, software organizations should use lean principles to emerge processes, practices, and values that support the deep and cyclical work required for the development. There is no one size fits.