Archive

Flow

How “Constant State of Ship” Drives Transformative Practices

Introduction

In the relentless pursuit of delivering value to customers, with unparalleled speed and reliability, the software development world has yet to widely embrace a revolutionary principle – the “Constant State of Ship”. This state, where software artefacts and products are perpetually poised for release into production environments within just 15 minutes’ notice, has emerged as a driving force behind best practices that enable true continuous deployment. Remarkably, this groundbreaking concept formed the foundation of the pioneering “Javelin” software development approach, a visionary approach conceived by FlowChainSensei (Bob Marshall) at Familiar circa 1996 and onwards, foreshadowing the industry’s even-now-yet-to-be-realised embrace of these practices.

The Power of “Constant State of Ship”

The “Constant State of Ship” serves us as an unyielding forcing function, inviting teams to adopt and adhere to a comprehensive set of best practices that catalyse the seamless flow of software into production. Let us explore how this principle reinforces each of thirteen fundamentals of Continuous Delivery (hat tip to Dave Farley):

The 13 Fundamentals Enabled

  1. A Repeatable, Reliable ProcessWith the ever-present possibility of an imminent release, teams may choose to establish a well-defined, automated pipeline for building, testing, and deploying their software. This process needs to be repeatable and reliable, minimising the risk of human error and ensuring consistency across releases.

    The “Constant State of Ship” mindset suggests that teams have a streamlined, automated release pipeline that can be triggered at any moment. Manual steps and ad-hoc and emergency exception procedures become liabilities, as they introduce variability and increase the chances of mistakes during deployment.

    To achieve this repeatability and reliability, teams are supported to invest in build automation tools, automated testing frameworks, and deployment automation pipelines. Every step of the release pipeline can be codified, documented, and thoroughly tested to ensure predictable outcomes each time.

    Moreover, the “Constant State of Ship” principle fosters an environment of continuous learning and improvement. Any failures or issues encountered during a release are promptly analysed, and the release process is refined to prevent future occurrences. This cycle of continuous feedback and optimisation ensures that the release pipeline remains reliable and efficient, even as the codebase and systems evolve over time.

    By operating in a “Constant State of Ship” mode, teams are invited to treat the release pipeline as a critical component of their software development lifecycle, investing the necessary resources and effort to make it repeatable, reliable, and capable of delivering changes to production environments at a moment’s notice.

  2. Automate All the ThingsIn a “Constant State of Ship” paradigm, manual interventions become significant bottlenecks and risks, hindering the required velocity and reliability. Automation becomes imperative, spanning every aspect of the delivery pipeline, from code compilation to infrastructure provisioning. The threat of an imminent release leaves no room for error-prone manual processes that could delay or derail a deployment. Teams must automate build processes, test execution, environment provisioning, deployment steps, and release orchestration to ensure consistency and minimise the risk of human error.
  3. Maintain a Releasable StateThe core tenet of “Constant State of Ship” requires that the codebase and associated artifacts remain in a perpetually releasable state. This principle invites teams to address issues promptly, maintain a high level of code quality, and vigilantly consider the accumulation of technical debt. Any defects, bugs, or instabilities in the codebase could potentially disrupt an imminent release, leading to costly delays or failures. Teams must adopt practices like continuous integration, automated testing, and ensemble programming to ensure that the codebase remains in a stable, deployable state at all times.
  4. Focus on Robust (Real) Quality Assurance

    In the “Constant State of Ship” paradigm, where the possibility of demand for an immediate release is ever-present, quality assurance cannot be treated as an afterthought. “Constant State of Ship” invites the integration of quality practices throughout the entire development lifecycle, ensuring that quality is baked into the software from inception to deployment.

    While testing plays a role, it is merely one facet of a comprehensive quality assurance strategy. Teams may choose to adopt a holistic approach that emphasises quality as a continuous, pervasive practice woven into every aspect of the development approach.

    This begins with cultivating a culture of quality-driven development, where every team member participates in collective ownership and responsibility for the quality of their work. Practices such as clarity of (quantified a la Gilb) requirements, ensemble programming, peer code reviews, adherence to coding standards, and continuous static code analysis can help identify and mitigate potential issues early in the development cycle.

    Furthermore, “Constant State of Ship” invites teams to embrace principles of iterative and incremental development. By breaking down complex features into smaller, manageable, well-bounded increments, teams can more effectively manage quality risks and ensure that each increment and subsystem meets the required quality criteria before progressing to the next.

    Continuous integration and deployment pipelines play a pivotal role in this quality assurance strategy, enabling teams to continuously validate and verify the software’s functionality, performance, and stability with each incremental change. These pipelines automate the execution of various quality checks, including unit tests, integration tests, and performance tests, providing real-time feedback and enabling teams to address issues promptly.

    However, quality assurance extends beyond mere testing alone. Teams have the opportunity to adopt a holistic approach that encompasses design practices, architectural decisions, and operational readiness. By considering quality implications at every stage of the software development lifecycle, teams can proactively identify and mitigate potential risks, ensuring that the software remains in a releasable state at all times.

    “Constant State of Ship” elevates quality assurance to a core discipline that permeates every aspect of the software development effort. By fostering a culture of quality-driven development and adopting continuous quality practices, teams can attend to the needs of all the Folks That Matter™, with confidence, knowing that their software meets the highest standards of reliability, stability, and performance.

  5. Implement Robust Deployment PipelinesAchieving a “Constant State of Ship” necessitates the implementation of robust deployment pipelines. These pipelines automate the entire process of building, testing, and deploying software changes, ensuring consistency and minimizing the risk of errors. With the ever-present possibility of an imminent release, teams cannot afford manual, error-prone deployment processes. Automated deployment pipelines provide a standardised, repeatable path to production, reducing the likelihood of failed or inconsistent deployments.
  6. Monitor the PipelineRegular smoke testing of the deployment pipeline is crucial in a “Constant State of Ship” mode. This practice helps catch issues early, before they can impact production environments, ensuring the pipeline’s reliability and preventing costly downtime. The possibility of an imminent release amplifies the importance of having a thoroughly validated deployment pipeline. Smoke tests act as a safety net, verifying the integrity of the pipeline and identifying any potential issues that could disrupt a deployment.
  7. Integrate ConstantlyThe “Constant State of Ship” mindset encourages teams to integrate their changes frequently, often multiple times per day. This practice surfaces issues early, reduces merge conflicts, and ensures that the codebase remains in a releasable state, ready for deployment at any given moment. Infrequent integration can lead to divergent codebases, making it harder to identify and resolve conflicts, which could potentially disrupt an imminent release. By integrating frequently, teams can maintain a stable, unified codebase that is always primed for deployment.
  8. Evolve the ArchitectureMaintaining a “Constant State of Ship” over time invites the continuous evolution of the system’s architecture (see also: Reverse Conway). Are teams prepared to refactor and adapt their architectures to accommodate new requirements, technologies, and scaling needs, without compromising the ability to release rapidly and reliably? As products grow and evolve, architectural decisions made early on may become hindrances to continuous deployment. The “Constant State of Ship” principle invites teams to proactively evaluate and evolve their architectures, ensuring that they remain flexible, scalable, and conducive to rapid releases.
  9. Leverage Data EnvironmentsWith the constant possibility of an imminent release, the ability to provision and manage data environments becomes critical. Teams may choose to adopt practices like database versioning, data seeding, and data masking to ensure consistent and reliable testing and deployment across environments, minimising the risk of data-related issues in production. The “Constant State of Ship” mindset invites a robust data management strategy that enables seamless and repeatable deployments, regardless of the data complexities involved.
  10. Mirror Production EnvironmentsTo minimise the risk of issues arising from environmental differences, teams operating in a “Constant State of Ship” mode may choose to ensure that their development, testing, and staging environments closely mirror production environments in terms of configuration, data, and infrastructure. This practice helps identify and address potential issues before they impact the live production system. The possibility of an imminent release heightens the importance of having production-like environments, as any discrepancies could lead to unexpected behavior or failures during deployment.
  11. Codify InfrastructureManually provisioning and configuring infrastructure for each release becomes a significant bottleneck when operating in a “Constant State of Ship” mode. Adopting Infrastructure as Code (IaC) practices, where infrastructure is defined and managed through code, enables teams to provision and tear down environments rapidly and consistently, minimising delays and reducing the risk of configuration drift. The “Constant State of Ship” principle invites a high degree of automation and repeatability in infrastructure management, making IaC a beneficial practice for ensuring rapid, reliable deployments.
  12. Foster Collaborative OwnershipAchieving a “Constant State of Ship” invites a high degree of collaboration and shared ownership among team members. Siloed responsibilities and knowledge become obstacles to rapid delivery. Teams may choose to adopt practices that promote collective code ownership, cross-functional collaboration, and shared understanding of the codebase and delivery processes. The “Constant State of Ship” mindset invites a culture of collective responsibility, where all team members are empowered to contribute to and understand the entire delivery process, enabling seamless and efficient releases.
  13. Continuous ImprovementOperating in a “Constant State of Ship” mode exposes inefficiencies and bottlenecks in the delivery pipeline and processes with uncompromising clarity. Teams may choose to embrace a culture of continuous improvement, regularly reviewing their practices, identifying areas for optimisation, and implementing changes to enhance their ability to deliver value rapidly and reliably. The constant presence of imminent releases acts as a driving force for continuous improvement, encouraging teams to continuously refine their processes, tools, and practices to achieve higher levels of velocity and quality. FlowChain was designed to systematise this very purpose.

The Visionary “Javelin” Approach

The “Javelin” approach (initally named “Jerid”) pioneered by me and my teams at Familiar from 1996 onward, was truly ahead of its time, recognising the transformative power of the “Constant State of Ship” mindset. By enshrining this principle as a cornerstone from its inception, “Javelin” has paved the way for the modern continuous deployment practices that have since become poised to gain industry standard status. This pioneering approach, along with FlowChain and e.g. Prod•gnosis, Flow•gnosis, Product Aikido, etc. exemplifies the spirit of continuous improvement intrinsic to the “Constant State of Ship” principle, ensuring its enduring relevance and impact.

Deep Cultural Implications

Reshaping the Culture and Mindset

Adopting the “Constant State of Ship” principle suggests a profound transformation that extends way beyond technical practices and processes – it hints at a seismic shift in the culture and mindset of software development teams and their parent organisations. This metamorphosis permeates every aspect of the organisation, reshaping shared assumptions, beliefs, and ways of working. However, navigating such a profound cultural shift can be a daunting challenge, often met with resistance and inertia.

This is where the discipline of organisational psychotherapy plays a pivotal role. By applying principles from psychotherapy, sociology, and group dynamics, organisational psychotherapy facilitates teams’ cultural and mindset shifts required to embrace the “Constant State of Ship” paradigm smoothly and effectively.

A Culture of Ownership and Accountability through Empowerment

The “Constant State of Ship” mindset fosters a culture of collective ownership and accountability. Organisational psychotherapy techniques, such as participative decision-making and fellowship, empower team members to take responsibility for the quality, stability, and deployability of the codebase and overall product. This sense of empowerment cultivates a culture of shared ownership, where individuals proactively address issues, collaborate across boundaries, and collectively strive for continuous improvement.

Embracing Transparency and Trust

Maintaining a “Constant State of Ship” requires a high degree of transparency and trust among team members. Organisational psychotherapy practices, such as surfacing shared assumptions and beliefs, encourage open communication and facilitate the identification of problems and risks early. By fostering an atmosphere where team members feel comfortable expressing concerns, sharing mistakes, and seeking help, a culture of transparency and trust emerges, enabling teams to collectively address challenges and ensure the software remains in a releasable state.

Prioritising Continuous Learning

The “Constant State of Ship” principle instills a mindset of continuous learning and improvement. With each release, teams gain valuable insights into their processes, tools, and practices. Embracing new shared assumptions becomes essential, as teams must continuously refine and adapt their approaches based on feedback and lessons learned. This culture of continuous learning fosters an environment of experimentation, where failures are embraced as opportunities for growth, and success is measured by the ability to deliver value rapidly and reliably.

Aligning Towards a Common Goal

Ultimately, the “Constant State of Ship” principle unifies teams around a common goal: meeting the needs of all the Folks That Matter™ with unparalleled speed and reliability. This shared mission transcends individual roles, responsibilities, and technical disciplines. It creates a sense of collective purpose, where every team member’s contribution, regardless of their specific function, is valued and recognised as essential to achieving this overarching objective.

By leveraging organisational psychotherapy techniques, organisations can accelerate and streamline the cultural and mindset shifts required to embrace the “Constant State of Ship” paradigm. This discipline not only makes the transition quicker and easier but also more cost-effective, as it addresses the root causes of resistance and inertia, facilitating a smoother and more sustainable transformation.

By reshaping the culture and mindset of software development teams, the “Constant State of Ship” principle cultivates an environment conducive to continuous deployment success. It fosters a sense of collective ownership, transparency, continuous learning, and shared purpose – traits that are indispensable in today’s rapidly evolving software landscape.

Embracing the Future

When the ability to swiftly adapt and innovate is paramount, the “Constant State of Ship” principle emerges as a beacon, guiding software development teams towards a future of quiet competence and competitiveness. By embracing this mindset, as exemplified by the visionary “Javelin” approach, teams can unlock the power to attend to folks’ needs with unprecedented speed, reliability, and quality – solidifying their organisation’s position as industry leaders in the software development arena.

The Why of FlowChain: Deliberate Continuous Improvement

In my career, working with hundreds of companies, I’ve almost never seen organisations* take a truly deliberate approach to continuous improvement. It’s nearly always treated as an afterthought or add-on to business-as-usual (BAU). But real transformation requires making continuous improvement an integral and core part of daily work. This is the “why” behind FlowChain – enabling deliberate, in-band continuous improvement.

In other words, applying the same disciplines from product development, delivery, etc. to the business (sic) of delivering continuous improvements  – continuously improving the way the work works.

What Is FlowChain?

So what is FlowChain? At its core, it is a system for managing flow – both the flow of outputs and the flow of improvements to the way the work works, concurrently and by the same means. And by “flow”, I mean the steady progress of work from request to completion through all steps in a process. Flow is optimised when the right work is happening at the right time by the right people. Roadblocks, delays, and waste are minimised or eliminated.

Flow

Optimising flow delivers the following benefits:

  • Increased productivity – less time wasted, more work completed
  • Improved quality – fewer defects, rework minimised
  • Better customer service – faster response times, reliability
  • Higher employee engagement – less frustration, more joy

But achieving flow requires continuous improvement. Problems must be made visible. Waste must be reduced iteratively. Roadblocks must be cleared continuously.

This is why FlowChain incorporates improvement into its regular rhythm. Each cycle follows a deliberate sequence:

  • Plan – Select and sequence the upcoming work.
  • Execute – Complete the work while tackling issues.
  • Review – Analyse completed work and identify improvements.
  • Adjust – Make changes to improve flow.

Unlike most continuous improvement efforts – that are separate from BAU – FlowChain makes improvement an integral in-band activity. The rapid cycles provide frequent opportunities to reflect, gain insights, and act.

Compounding Benefits

Over time, the compounding benefits are immense. Teams develop a “flow habit”, where improving flow becomes second nature. Powerful capabilities like root cause analysis, A3 problem-solving, improvement katas, and change management are honed.

In my experience, this deliberate approach is transformative. Teams gain tremendous agency to systematically improve their own flow. The organisation as a whole cultivates a culture of continuous improvement. And customers experience ever-better service and responsiveness.

The “why” of FlowChain is simple – create focus, visibility, accountability, and agency to drive continuous improvement. The results – ever better flow, reduced waste, and sustainable transformation. Deliberate, in-band continuous improvement stops being an aspiration and becomes a reality.

*Ask me about the exception.

Teams for the Flow Era

Teams that are capable of smoothly and swiftly reconfiguring themselves is becoming the norm in the software industry today. Rather than old-fashioned rigid structures and siloed employees, progressive organisations opt for a more fluid approach that allows for responsiveness to shifting conditions.

Team members work across multiple projects in these flexible arrangements, contributing versatile skills as needed and rearranging team composition to suit the task in hand. Situational leadership – a.k.a. Fellowship – emerges based on expertise instead of formal titles. With boundaries that allow copious and timely information sharing between interlocking teams, rapid coordination becomes commonplace.

Self-direction and mutual trust enable these dynamic teams. A strong, shared strategic purpose coupled with developmental training and organisational support gives members the confidence to take initiative without constant top-down control. Teammates understand how their piece connects, empowering local decision-making even as larger configurations reshape around evolving challenges.

This fluid team format creates adaptable organisations capable of smooth reprioritisation as demands change, unencumbered by bureaucratic processes. People, ideas and resources flow to each new focus area in turn, adjusting teaming patterns seamlessly to current priorities. Progress keeps pace with the opportunities and demands of the day.

Some continuity and purposeful guidance balances out the flux. Respected, experienced figures may anchor successive teaming waves, providing continuity of culture and knowledge transfer. As thought leaderBob Marshall describes in his acclaimed book ‘The Team Fruit Bowl’, fluid teams’ steadiness enables ongoing realignment around the organisation’s North Star.

Self-aligning, versatile teams represent the leading edge of organisational design today. Staying responsive without getting swept away, they ride currents of change toward transformative performance.

Electric Flow in Business

What is Electrical Continuity Testing?

Electrical continuity testing measures the flow of electric current through a circuit, checking for resistance or blockages. In electrical systems, an uninterrupted flow ensures that everything works as it should.

How Do Business Silos Work?

Business silos are isolated teams or departments within an organisation, where information rarely crosses the silo’s boundary. These divisions can cause hiccups in communication, stifle collaboration, and hinder overall performance. If a business must have silos (at least, for the present) might it not be useful to understand and regularly monitor the flow between different silos?

Can Electrical Continuity Testing Apply to Business Silos?

Applying the principles of electrical continuity testing to business silos can offer unique insights. Just as you’d test for impedance or resistance in an electrical circuit, you could evaluate the flow of information, resources, or processes between silos. This involves looking for ‘resistance’—such as lack of communication or structural barriers—that might impede this flow.

How to Test for ‘Continuity’?

  1. Identify Key Metrics: Before you start, establish what constitutes good ‘flow’ within your organisation. Is it speed of information sharing, quality of cross-departmental projects, or perhaps employee satisfaction?
  2. Conduct Audits: Regularly examine how well different departments interact. Surveys, interviews, and process mapping can all serve as diagnostic tools.
  3. Implement a Reporting System: Create a real-time dashboard that displays these key metrics. This will help you monitor the flow continuously, just as you would in an electrical system.
  4. Action Plans: Upon discovering a ‘short-circuit’ or ‘resistance’, develop a plan to restore optimal flow. This could mean reorganising teams, offering training, or implementing new communication tools.

What Benefits Can You Expect?

  1. Enhanced Collaboration: Better flow of information fosters a collaborative atmosphere.
  2. Operational Efficiency: With barriers removed, processes become streamlined and agile.
  3. Increased Innovation: Free flow of ideas can spur innovation as teams from diverse backgrounds contribute to a collective goal.

Is This Approach Worth It?

While applying a technical concept like electrical continuity testing to organisational dynamics might seem unconventional, it offers a structured way to diagnose and resolve issues around siloed working. By focusing on flow and reducing resistance, you set the stage for a more efficient, collaborative, and innovative work environment.

A Question of Flow

A New Take on Flow?

When we hear about flow, it’s often linked to a psychological state of focus and immersion. But flow isn’t just a psychological state popularised by Mihaly Csikszentmihalyi. There’s another kind of flow that plays a critical role in organisations. This type of flow refers to a well-coordinated series of actions, events, or processes that work in harmony – flow as a smooth succession of actions that take place within an organisational setting

Why Should You Bother?

Paying attention to flow in your organisation is not just an academic exercise; it’s a practical necessity. Improved flow means fewer bottlenecks, quicker turnarounds, and more satisfied employees. Given these benefits, can you afford to overlook the importance of achieving good flow?

Is Structure the Solution?

While structure might seem constraining, it offers a foundation upon which flow can thrive. Roles and responsibilities are clearer, and tasks follow a more predictable pattern. However, there’s a limit to the benefits of structure. Could too much structure curtail creativity and inhibit the capacity for innovative solutions? How much structure is “just enough”?

How Tech Can Contribute

Modern technology provides tools that make it easier to achieve flow. Automated systems can handle repetitive tasks, freeing humans to focus on complex challenges. Additionally, real-time data analytics can identify potential bottlenecks before they become major issues. But what’s the cost of getting your tech choices wrong?

Humans in the Equation

A finely tuned system isn’t just about technology or well-drafted processes. The human element—morale, motivation, and collaboration—also plays a pivotal role. When employees work well together, the flow is more natural and less forced. How can we foster a culture that encourages this kind of harmony?

What Metrics to Use

Choosing metrics for gauging flow involves more than just picking numbers to track. Do task completion times, takt times, employee engagement levels, or customer satisfaction rates resonate with your organisational goals? How do you know if you’re focusing on the right measures to sustain or enhance flow?

Ready to Make a Change?

Embarking on the journey to improve organisational flow is a complex but rewarding endeavour. It involves continuous assessment, involving multiple stakeholders, and making frequent adjustments. Are you willing to commit the necessary time and resources to achieve a state of optimal flow?

Flow, in this organisational sense, is a multifaceted concept. By asking the right questions and taking a balanced approach, you can create an environment where flow becomes a natural aspect of your operational landscape. So, are you prepared to rethink what flow means for you and your organisation?

A Question of Flow: Extending into the Software Realm

Does Software Development Redefine Flow?

In the complex landscape of software development, flow takes on nuanced dimensions. Beyond well-coordinated actions and processes, here it’s about managing interdependent systems, handling data, and synchronising multiple contributors. How does the concept of flow evolve when applied to software?

What Are the Stakes for Software Projects?

The relevance of flow extends far beyond general organisational theory when we delve into software projects. It’s not just about speed but also about code quality, user experience, and sustainable development. How does focusing on flow lead to more resilient and robust software systems?

How Does Architecture Influence Flow?

In software development, the architecture itself can facilitate or hinder flow. Modular designs and well-defined APIs can dramatically streamline the work process. How could architecture be leveraged to enhance flow, without compromising quality?

Are DevOps and CI/CD Flow Enablers?

DevOps culture and Continuous Integration/Continuous Deployment (CI/CD) pipelines are revolutionising the flow in software development. They merge development and operations, automating much of the workflow. Is the automation brought by these practices always beneficial, or could there be drawbacks?

Can Open Source Cultures Teach Us About Flow?

The open-source community often exhibits a unique kind of flow, built on collective intelligence and decentralized collaboration. Can traditional organisations learn from this paradigm?

What About Flow in Remote Teams?

The rise of remote work presents new challenges and opportunities for achieving flow in software development. How do time zones, cultural differences, and virtual communication affect flow? Can they be harnessed to enhance it instead?

Is Ethical Software Development Compatible with Flow?

Rapid development cycles and the pursuit of flow might conflict with the need for ethical considerations in software design, such as accessibility and data privacy. Can a balance be achieved where ethical imperatives don’t disrupt the flow?

Are We Ready for the Future of Software Development Flow?

As technologies like AI and quantum computing mature, they’ll inevitably impact the flow of software development. Will we adapt our notions of flow to these advancements, or will the concept of flow itself undergo a transformation?

By integrating these big-picture considerations, we can explore new frontiers of flow in both traditional organisations and software development landscapes. The question remains: Are we ready to adapt and evolve?

Unpacking Prod•gnosis

It’s been a while since I wrote about Prod•gnosis. I thought it might be a good time to mention it once more.

What is Prod•gnosis?

Prod•gnosis is a model for running a business, built on the modern principles of Flow, Systems thinking, and Value Streams.

Introduction: Foundations Terms

The terms “flow,” “value streams,” “systems thinking”, and “product development” often appear complicated and exclusive to business experts. However, understanding these ideas is vital for the effective operation of any organisation. This post aims to clarify these arcane terms and introduce you to Prod•gnosis, an approach to integrate these elements for business success.

Core Concepts Explained

Flow

Flow refers to the movement of work through an organisation. It starts from the initial concept and ends at the delivery of a finished product or service. Effective flow reduces delays, enhances quality, and makes best use of resources. Flow offers a number of business benefits.

Value Streams

A value stream is the sequence of activities an organisation performs to deliver a product or service to a customer. This set of activities starts from the moment a customer requests a product or service, and continues through to its delivery and cash collection. It covers everything from order-taking and production to logistics, distribution, billing, and even after-sales service.

The main goal of understanding any particular value stream is to identify ways to realise the benefits of improving flow. This means looking at how work moves through the organisation, determining where delays or bottlenecks occur, and finding ways to streamline those areas. By optimising its value streams, a company can produce and sell goods or offer services more efficiently, ultimately delivering better value to the customer.

Product Development Value Streams and Operational Value Streams

A Product Development Value Stream (PDVS) focuses on the series of activities that take a product from its conceptual stage through to market launch. Unlike a value stream that concerns itself with manufacturing or delivering a finished product, a PDVS involves steps like research, design, prototyping, testing, and development.

Product development is the complete process of bringing a new product or service to market. Operational value streams are the set of activities that continuously deliver existing products or services to the customer.

The goal is to manage this set of activities efficiently and cohesively so that new products can be brought to market more quickly, more cost-effectively, and with higher quality. In essence, it’s about aligning all the different departments and specialists involved in creating a new product—such as engineering, design, marketing, logistics, sales, and finance—to work towards a common objective.

Understanding and optimising the PDVS can enable an organisation to gain competitive advantage by accelerating time-to-market, reducing development costs, and ensuring that the final product better meets customer needs.

Systems Thinking

Systems thinking is an approach to problem-solving and understanding complex situations. Rather than breaking down a system into its component parts, for individual study, systems thinking aims to view the system as a cohesive whole. This approach recognises that the components of a system can interact with one another in ways that significantly impact the function and properties of the entire system.

In business and organisational contexts, systems thinking often involves considering how various departments, roles, processes, and external factors are interconnected. By understanding these relationships, one can better identify underlying issues, anticipate the consequences of specific actions, and generate more effective solutions for complex challenges.

It deviates from traditional linear thinking, which tends to focus on cause-and-effect relationships in isolation. Systems thinking acknowledges feedback loops, delays, and other complexities that are inherent in most systems, be they biological, organisational, or mechanical.

The benefits of applying systems thinking in an organisation include a better understanding of the big picture, improved collaboration among departments, and more effective problem-solving and decision-making processes. As Buckminster Fuller noted: a system when taken as a whole has behaviours that cannot be predicted by the behaviours of their parts taken separately.

Introducing Prod•gnosis

Prod•gnosis proposes a disciplined approach to managing the flow of new products into the market. In essence, it is a framework for creating and launching operational value streams dedicated to specific product lines. Each operational value stream is a custom-built organisational pathway for manufacturing, packaging, shipping, selling, billing, payment collection, and servicing a particular product.

The goal of Prod•gnosis is to manage the flow of new products into the market in a disciplined and evolving manner. It focuses on creating and launching operational value streams tailored to each new product or product line. Essentially, it strives to optimise an organisation’s capability to bring new products to market efficiently, effectively, and in a way that allows for continual improvement.

Where Does Product Development Fit?

Prod•gnosis challenges traditional models of product development, often – for software products at least – homed within in IT departments. Instead, it suggests the formation of a distinct “Product Development Value Stream” (PDVS). This PDVS value stream focuses solely on the process of creating new operational value streams for each new product or product line, thus preventing the inefficiency of constantly assembling and disassembling multi-departmental product development teams. By having a PVDS dedicated to creating operational value streams, the organisation can build a dedicated specvialist capability in this area. This contrasts with traditional approaches, where the capability for building operational value streams is fragmented across time and organisational departments

Summary: Why It Matters

Embracing Prod•gnosis is more than just tweaking a few business processes; it’s about a fundamental change in perspective informed by systems thinking. This approach enhances an organisation’s adaptability and positions it for accelerated growth. By focusing on flow and value streams, Prod•gnosis provides a robust framework for improving how an organisation introduces new products to its markets. It’s a concept accessible to everyone, not just business experts, and a game-changer for an organisation’s future. Are you prepared to think in these new terms?

Further Reading

Ward, A. C., & Sobek, D. K. (2014). Lean Product and Process Development (2nd ed.). Lean Enterprise Institute, Inc.

This book is a seminal work that delves into the principles and practices of lean thinking in the context of product development. The book aims to guide organisations through the complexities of creating valuable and desirable products. Using case studies, examples, and a wealth of expert insight, it explores how to establish sustainable value streams. The 2nd edition, published in 2014, offers updated content to reflect advancements in the field, making it a go-to resource for anyone interested in lean methodologies in product development.

Beyond the Agile Trap

Failure Writ Large

Have you ever questioned why Agile so often falls short of delivering on its impish promises? You’re not alone. Many find their Hail Mary agile initiatives underdelivering, leaving everyone perplexed, frustrated and embarrassed. The answer isn’t hidden within Agile, but it’s found in the overlooked, holistic mindset called “systems thinking”.

Cuckoos

Like a cuckoo, Agile’s brood parasite nature has strong-armed it a significant place in the field of software development, with its proponents’ knavish boasting of flexibility, adaptability, and rapid value delivery. However, despite its siren qualities, Agile frequently disappoints. But why?

The answer isn’t nestled in the details of Agile approaches, but rather within an organisation’s broader perspective.

Agile almost inevitably promotes mere local optimisations – approaches with immediate and tangible improvements that miss the context for organisation-wide benefits such as flow.

Systems Thinking

This is where systems thinking approaches like the Theory of Constraints (ToC) comes into play. This perspective urges us to look beyond individual silos and examine the system as a whole, identifying bottlenecks and developing comprehensive flow-oriented solutions. The move from local to system-wide optimisation isn’t a minor tweak; it’s a sea change. It requires a transition from concentrating on isolated elements to understanding the wider interdependencies of the entire system.

Rarely Seen

Regrettably, true systems thinking is rarely seen in organisations. It’s challenging to grasp, tough to implement, antithetical to the near-ubiquitous silo style of organisation, and calls for an open mind to coordinate – or better yet, merge – multiple silos.

However, it’s a crucial journey, a leap of faith necessary to build a robust, resilient, and more effective organisation.

Summary

When we ask, “why does Agile fail so often?”, the response isn’t found within Agile’s principles or practices. Instead, it lies in the broader organisational mindset that often overlooks the system-wide view. Moving from Agile’s local optimisation to the more comprehensive approach of system-wide optimisation isn’t a simple journey. Still, it’s a journey towards enlightenment. Let’s not settle for improving just the software development silo. Let’s strive to create a balanced, stable, and impressive flow chain. The essence of this conversation is about challenging our views, moving beyond our current practices, and welcoming the rewarding shift towards systems thinking. Here’s to building a future that doesn’t merely copy blindly, but optimises and truly excels.

Quintessential Product Development 

In my most recent book “Quintessence” I map out the details of what makes for highly effective software development organisations.

As fas as software development organisations are concerned, it’s a bit of a moot point – as software is generally something to be avoided, rather than sought (see also: #NoSoftware).

“The way you get programmer productivity is by eliminating lines of code you have to write. The line of code that’s the fastest to write, that never breaks, that doesn’t need maintenance, is the line you never had to write.”

~ Steve Jobs 

Foundational Concepts

There are just a few complementary concepts that mark out the quintessential product development company. These are:

  • Whole Product.
  • Systematic Product Management.
  • Whole Organisation (systems thinking).

Whole Product

The quintessential product development organisation embraces the concept of “whole product”. Which is to say, these organisations emphasise the need to have every element of a product i.e. core product elements plus a range of “intangibles” – everything that is needed for the customer to have a compelling reason to buy (Mckenna 1986).

Systematic Product Management

Quintessential product development organisations take a systematic approach to flowing new product ideas and features through a number of stages – often in parallel (Ward 1999) – to predictably arrive at a successful new product in the market:

  • Inception – spotting a gap in the market, a.k.a. some (potential customer) needs going unmet, interesting enough to do some discovery.
  • Discovery – uncovering and proving the real needs of customers, the things they value, the likely usability of possible solutions, the feasibility of meeting everyone’s needs, and the viability of a product as a means to these ends. In essence, the key risks facing the proposed product. 
  • Implementation – building a whole product solution, i.e. both core elements and “intangibles”.
  • Launch – Placing the product on sale (or otherwise making it available to customers).
  • Feedback – Seeing how the market responds.
  • Pivot or Augmentation – Acting on feedback to either reposition the solution (in response to unfavourable feedback) or to incrementally update / extend the “whole product” offering to continually strengthen the product’s value proposition and appeal.
  • Cash Cow – Reap the commercial rewards of a strong product and market share.
  • Sunsetting – Wind down the product in a way that meets the ongoing needs of all the Folks That Matter™️ (e.g. continued support, spare parts, etc.; easing customers’ transition to newer products; etc.). 

Whole Organisation

It’s common for organisations to think in terms of silos. A Product Management or Product Development silo being but one more silo in a long and ever-lengthening list. 

In the quintessential organisation, the whole organisation is geared around – amongst other things – the task of regularly and predictably getting new products and new product features/updates out the door and into the hands of customers. In the longer term, new products are the life blood of most organisations, especially in the technology industries.

We only have to look e.g. Toyota and their TPDS (Toyota Product Development System) to see both an example of how this works in practice, and the huge benefits of the whole-organisation approach.

Quintessential product development organisations embrace a range of progressive ideas such as Prod•gnosis and Flow•gnosis.

– Bob

Further Reading

Marshall, R.W. (2013). Product Aikido. [online] Think Different Available at: /wp-content/uploads/2013/04/productaikido041016.pdf [Accessed 13 Jan. 2022].

Mckenna, R. (1986). The Regis Touch: New Marketing Strategies for Uncertain Times. Reading, Mass.: Addison-Wesley Pub. Co.

Perri, M. (2019). Escaping The Build Trap: How Effective Product Management Creates Real Value. O’Reilly.

Ward, A.C. (1999). Toyota’s Principles of Set-Based Concurrent Engineering. [online] MIT Sloan Management Review. Available at: https://sloanreview.mit.edu/article/toyotas-principles-of-setbased-concurrent-engineering/. [Accessed 13 Jan. 2022].

Software Development at Scale

Scaled Agile – or agility at scale – seems like a hot potato at the moment. Here’s a few
thoughts:

Scaled Agile? Don’t. Just don’t. For GOD’S sake don’t.

Agile even at a small scale doesn’t work and scaling something that doesn’t work just leads to an even bigger mess that doesn’t work. Take a look at SAFe if you don’t believe it. Or just listen to Ackoff:

“The righter we do the wrong thing, the wronger we become. When we make a mistake doing the wrong thing and correct it, we become wronger. When we make a mistake doing the right thing and correct it, we become righter. Therefore, it is better to do the right thing wrong than the wrong thing right.”

Mindset (Obvs.)

Mindset (a.k.a. culture, memeplex) over methods, tools, etc.. What an organisation, collectively, believes and assumes has far more impact on e.g. productivity, quality, etc. than any methods or tools. See: Organisational Psychotherapy, the Marshall Model, and my new book, “Memeology“.

The only thing of real importance that leaders do is to create and manage culture. 

~ Edgar Schein

The thing I have learned at IBM is that culture is everything. It took me to age fifty-five to figure that out.

~ Lou Gerstner, CEO, IBM

If you get the culture right, most of the other stuff will just take care of itself.

~Tony Hsieh, CEO, Zappos.com

And Schein also provides us with a definition for “the culture of an organisation”:

Culture is the deeper level of basic assumptions and beliefs that are shared by members of an organisation, that operate unconsciously and define in a basic ‘taken for granted’ fashion an organisation’s view of its self and its environment…

It’s a pattern of shared basic assumptions invented, discovered, or developed by a given group.

~ Edgar Schein

Flow Efficiency

Goldratt wrote the book on Flow Efficiency (see “The Goal”, in case you’re interested).

Flow efficiency is miles better than resource efficiency (a.k.a. utilisation). What is flow efficiency? I’m sure you can look it up. But for the lazy: Flow efficiency suggest a focus on keeping work moving through its various value-adding steps, with minimal or zero queues and wait times, thereby attending to folks’ needs (value to the Folks That Matter) – and prompting feedback – as quickly as possible.

Formalise Innovation

Formalise innovation, evolve into continuous organisational transformation.

What do I mean by “innovation”?
In the context of organisations, here are a few possible definitions of “innovation”:

  •  A process by which a method, a product, or a service is renewed and refreshed by applying new ideas or introducing new techniques to create new value.
  • Turning an idea into a solution that adds value from the perspective of the folks that matter
  • A new strategy (means) for attending to the needs of the folks that matter.
  • The application of ideas that are novel and useful.
  • Staying relevant – keeping pace with changes to the (business) environment.
  • The implementation of something new – until it’s implemented it’s just an idea.
  • Stop talking about innovation. Focus on organisational transformation.

Here’s my preferred definition:

Innovation: Delivering against an idea which meets a specific set of needs, for all the Folks That Matter.

And “formalise”? What the hell does that mean, exactly? In effect, institute trainings, standard work, measures, etc., around the whole innovation (and, a.s.a.p., organisational transformation) piece.

I’ve been brief here to avoid a rant. Do get in touch if you’d like me to elaborate on some of these themes.

– Bob

Cognitive Function

How often do you get pissed off by interruptions and distractions? You know, when you’re zoned in on something, in a state of flow, and something happens to break the flow? Personally, when I’m writing code, I have to be in a quiet place, by myself or with my pair or mob, else I can’t get anything done for the continual distractions.

This is but one example of how easily cognitive function can be impaired.

Common sources of cognitive impairment:

  • Distractions and interruptions
  • Stress (specifically, negative stress a.k.a. distress) Cf Amygdala Hijack
  • Tiredness, fatigue, lack of sleep.
  • Multitasking
  • Poverty
  • Diet
  • Shift patterns
  • Noise and other forms of environmental stressors (lighting, odours, vibrations, exposure to particulates, elevated carbon dioxide, etc.)
  • Physiological issues (such as colds and flu, hypoglycemia, aphasia, depression, dehydration, hypertension, obesity, trauma, diabetes, Parkinson’s, POTS, dementia, hypoxia, atrial fibrillation)
  • Substance abuse (drink, drugs, etc. – short and long term effects, chronic and acute)

Wow. That’s quite a list. Seems like almost anything can impair cognitive function.

Why Does this Matter?

So why does cognitive function matter. What’s the connection with knowledge work? I’ll spell it out in case it’s not clear:

Knowledge work – such as software development – by definition involves working with our brains. If our brains are performing well (i.e. effective or relatively high cognitive functioning) then we can expect our work to go well, things to get done quicker, with fewer errors, and so on.

Conversely, when our cognitive function is impaired, our brains will take longer to accomplish tasks, come up with less effective solutions, commit more errors, and generally perform more ineffectively.

It’s also likely that with impaired cognitive function we’ll be less reflective, with less energy or capacity to spend on thinking about our work, our relationships, our behaviours, our practices, our customers, possible innovations, our needs and the needs of others, etc..

Does it sound to you like non-impaired cognitive function is something worth having? Something worth paying attention to?

Paying Attention?

So how many folks – managers, workers, organisations – pay any attention AT ALL to folks’ cognitive functioning in the workplace or whilst working? I’d suggest the answer is none, or as near none as makes no difference.

Which seems strange to me, if we truly seek our collaborative knowledge work (and workers) to be as effective as possible. Of course, that objective may be a false assumption. Maybe blissful ignorance and indifference is preferable to paying attention and taking action? Given the reluctance I’ve encountered when broaching this subject, I suspect blissful ignorance and/or indifference holds sway.

How does it go in your organisation?

– Bob

Solutions Demand Problems

I’m obliged to Ben Simo (@QualityFrog) for a couple of recent tweets that prompted me to write this post:

I very much concur that solutions disconnected from problems have little value or utility. It’s probably overdue to remind myself of the business problems which spurred me to create the various solutions I regularly blog about.

FlowChain

Problem

Continually managing projects (portfolios of projects, really) is a pain in the ass and a costly overhead (it doesn’t contribute to the work getting done, it causes continual scheduling and bottlenecking issues around key specialists, detracts from autonomy and shared purpose, and – from a flow-of-value-to-the-customer perspective – chops up the flow into mini-silos (not good for smooth flow). Typically, projects also leave little or no time, or infrastructure, for continually improving the way the work works. And the project approach is a bit like a lead overcoat, constraining management’s options, and making it difficult to make nimble re-adjustments to priorities on-the-fly.

Solution (in a Nutshell)

FlowChain proposes a single organisational backlog, to order all proposed new features and products, along with all proposed improvement actions (improvement to the way the work works). Guided by policies set by e.g. management, people in the pool of development specialists coalesce – in small groups, and in chunks of time of just a few days – around each suitable highest-priority work item to see it through to “done”.

Prod•gnosis

Problem

Speed to market for new products is held back and undermined by the conventional piecemeal, cross-silo approach to new product development. With multiple hands-offs, inter-silo queues, rework loops, and resource contentions, the conventional approach creates excessive delays (cf cost of delay), drives up the cost-of-quality (due to the propensity for errors), and the need for continual management  interventions (constant firefighting).

Solution (in a Nutshell)

Prod•gnosisproposes a holistic approach to New Product Development, seeing each product line or product family as an operational value stream (OVS), and the ongoing challenge as being the bringing of new operational value streams into existence. The Prod•gnosis approach stipulates an OVS-creating centre of excellence: a group of people with all the skills necessary to quickly and reliably creating new OVSs. Each new OVS, once created, is handed over to a dedicated OVS manager and team to run it under day-to-day BAU (Business as Usual).

Flow•gnosis

Problem

FlowChain was originally conceived as a solution for Analytic-minded organisations. In other words, an organisation with conventional functional silos, management, hierarchy, etc. In Synergistic-minded organisations, some adjustments can make FlowChain much more effective and better suited to that different kind of organisation.

Solution (in a Nutshell)

Flow•gnosis merges Prod•gnosis and FlowChain together, giving an organisation-wide, holistic solution which improves organisational effectiveness, reifies Continuous Improvement, speeds flowof new products into the market, provides an operational (value stream based) model for the whole business, and allows specialists from many functions to work together with a minimum of hand-offs, delays, mistakes and other wastes.

Rightshifting

Problem

Few organisations have a conscious idea of how relatively effective they are, and of the scope for them to become much more effective (and thus profitable, successful, etc.). Absent this awareness, there’s precious little incentive to lift one’s head up from the daily grind to imagine what could be.

Solution (in a Nutshell)

Rightshifting provides organisations with a context within which to consider their relative effectiveness, both with respect to other similar organisations, and more significantly, with respect to the organisation’s potential future self.

The Marshall Model

Problem

Few organisations have an explicit model for organisational effectiveness. Absence of such a model makes it difficult to have conversations around what actions the organisation needs to take to become more effective. And for change agents such as Consultants and Enterprise Coaches attempting to assist an organisation towards increased effectiveness, it can be difficult to choose the most effective kinds of interventions (these being contingent upon where the organisation is “at”, with regard to its set of collective assumptions and beliefs a.k.a. mindset).

Solution (in a Nutshell)

The Marshall Model provides an explanation of organisational effectiveness. The model provides a starting point for folks inside an organisation to begin discussing their own perspectives on what effectiveness means, what makes their own particular organisation effective, and what actions might be necessary to make the organisation more effective. Simultaneously, the Marshall Model (a.k.a. Dreyfus for Organisations) provides a framework for change agents to help select the kinds of interventions most likely to be successful.

Organisational Psychotherapy

Problem

Some organisations embrace the idea that the collective organisational mindset – what people, collectively believe about how organisations should work – is the prime determinant of organisational effectiveness, productivity, quality of life at work, profitability, and success. If so, how to “shift” the organisation’s mindset, its collective beliefs, assumptions and tropes, to a more healthy and effective place? Most organisations do not naturally have this skill set or capability. And it can take much time, and many costly missteps along the way, to acquire such a capability.

Solution (in a Nutshell)

Organisational Psychotherapy provides a means to accelerate the acquisition of the necessary skills and capabilities for an organisation to become competent in continually revising its collective set of assumptions and beliefs. Organisational Psychotherapists provide guidance and support to organisations in all stages of this journey.

Emotioneering

Problem

Research (cf Buy•ology ~ Martin Lindstrom) has shown conclusively that people buy things not on rational lines, but on emotional lines. Rationality, if it has a look-in at all, is reserved for post-hoc justification of buying decisions. However, most product development today is driven by rationality:

  • What are the customers’ pain points?
  • What are the user stories or customer journeys we need to address?
  • What features should we provide to ameliorate those pain points and meet those user needs?

Upshot: mediocre products which fail to appeal to the buyers’ emotions, excepting by accident. And thus less customer appeal, and so lower margins, lower demand, lower market share, and slower growth.

Solution (in a Nutshell)

Emotioneering proposes replacing the conventional requirements engineering process (whether that be big-design-up-front or incremental/iterative design) – focusing as it does on product features – with an *engineering* process focusing on ensuring our products creaate the emotional responses we wish to evoke in our customers and markets (and more broadly, in all the Folks That Matter).

The Antimatter Principle

Problem

How to create an environment where the relationships between people can thrive and flourish? An environment where engagement and morale is consistently through the roof? Where joy, passion and discretionary effort are palpable, ever-present and to-the-max?

Solution (in a Nutshell)

The Antimatter Principle proposes that putting the principle of “attending to folks’ needs” at front and centre of all of the organisation’s policies is by far the best way to create an environment where the relationships between people can thrive and flourish. Note: this includes policies governing the engineering disciplines of the organisation, i.e. attending to customers’ needs at least as much as to the needs of all the other Folks That Matter.

– Bob

The Simplest Thing That Could Possibly Work

Here’s an excerpt (pp 206) from “Birth Of the Chaordic Age” by Dee Hock, about “an odd project management scheme” they adopted in the early days – circa 1974:

“Swiftly, self-organisation emerged. An entire wall became a pinboard with every remaining day calendared across the top. Someone grabbed an unwashed coffee cup and suspended it on a long piece of string pinned to the current date. Every element of work to be done was listed on scraps of paper with the required completion date and the name of the person who had accepted the work. Anyone could revise the elements, adding tasks or revising dates, providing they coordinated with others affected. Everyone, at any time, could see the picture emerge and evolve. They could see how the whole depended on their work, and how their work was connected to every other part of the effort. Groups constantly assembled in front of the board as need and inclination arose, discussing and deciding in continuous flow; then dissolving as needs were met. As each task was completed its scrap of paper would be removed. Each day, the cup and string moved inexorably ahead.”

I’m struck by the similarities with FlowChain, and as with FlowChain, it seems an exemplar of simplicity and flow. I also note the implicit “Advice Process” vibe, and the emphasis on “making the work visible” (Cf Personal Kanban).

– Bob

Further Reading

Metaflow

If you’re in business for anything more than a one-hit-wonder, you may have given some thought to your next product. Albeit probably not much more than a few cursory thoughts, given the attention that you current product(s) demand of you.

Product Development And Delivery Flow

Some lucky few may have moved from the idea of economies of scale, maximising utilisation, etc., to the idea of flow. Flow of products from raw materials to finished and sold goods (or services). And flow of product ideas and new features into those products and product lines.

Prod•gnosis

Fewer again may have adopted something like Prod•gnosis, where ideas for each new product or service get deliberately “developed” into a new operational value stream:

(click for larger image)

With this kind of approach, people (in the green box) who specialise in creating new operational values streams can bring their talents, expertise and continuous improvements to bear on each new operational value stream (blue boxes) they “develop” for the organisation.

Aside: The Toyota Product Development System (TPDS) works much like this, with circa 100 people co-located in a Big Room, or Obeya, for the 15 months or so that it takes Toyota to transform a new product line (vehicle) idea into a new, running operational value stream (blue box).

Metaflow

If we consider the whole organisation, over time, then whether we use the above approach or no, there will a whole series of operational value streams (blue boxes) starting up, operating for a time, e.g. whilst profitable, and eventually shutting down:

(click for larger image)

We might like to see our blue boxes flow into the organisation (start up), with as smooth and effective a flow as possible. This is what I call metaflow: the effective flow of operational value streams “into” an organisation.

Ultimately, I guess it depends on how much you need new products and services to flow, and how much you need the benefits that can bring you.

– Bob

Further Reading

The Principles Of Product Development Flow ~ Don Reinertsen