Metadata increases feedback and ensures value stream integrity

In Continuous Delivery, Dave Farley and Jez Humble describe the Lean production principles that underpin Continuous Delivery, and how a pipeline encapsulates a value stream – the journey a customer feature undertakes from discovery to real world consumption.

In a pipeline each stage represents a step in the value stream, meaning that for application XYZ an example value stream of [Development -> Acceptance -> UAT -> Performance -> Production] could be defined as follows:

Pipeline with Metadata

In the above pipeline, each stage ends with a discrete piece of metadata (“created XYZ 2.1”, “XYZ 2.1 passed acceptance tests”, etc.) being written back to the binary repository, indicating that one or more new customer features have progressed in the value stream.

Unfortunately, pipelines are often constructed without metadata support:

Pipeline without Metadata

In this situation the lack of activity data reduces each stage to a fire-and-forget operation, constraining feedback and unnecessarily exposing the value stream to obtuse, time-consuming errors. For example, QA could mistakenly test new features that have not passed automated regression tests, or Operations could mistakenly release features that have not been signed off.

With metadata the following safeguards can be easily implemented:

  • Check if binary actually exists e.g. “can XYZ 1.2 be retrieved for deploy to Production”
  • Prevent binary re-entering a previous stage e.g. “once XYZ 1.2 has passed or failed Acceptance, the result is final”
  • Ensure binary has successfully passed sufficient dependencies to enter a stage e.g. “XYZ 1.2 can only enter Production once it has successfully passed UAT and Performance”
  • Introduce a manual sign-off process for critical environments e.g. “XYZ 1.2 can only pass UAT when exploratory testing is complete”
  • Visualise pipeline activity e.g. “I can see XYZ 1.2 was successfully released to Production a week ago, and that 1.4 is the next viable candidate as 1.3 failed Acceptance”

These features ensure fast feedback is always available and that the pipeline is an accurate representation of the underlying value stream. An absence of metadata unnecessarily hinders these goals and suggests a failure to understand the core values of Continuous Delivery.