Develop middleware to enable tool-agnostic end to end traceability within Continuous Delivery pipeline and allow creation of real-time visualization/statistical tools based on info provided by the middleware
- Ability to measure the lead time between events withing Continuous Delivery pipeline. For instance, lead time from source code change to the point when binary that includes this change receives a release candidate quality stamp.
- Ability identify all source code changes together with corresponding work items and 3pp components included into product release, i.e. ability to trace down from product release to source code change.
- Ability to identify all product releases that include source code change, i.e. ability to trace up from source code change to product release.
- Ability to trigger activities based on events within Continuous Delivery pipeline. For instance, trigger test when the new binary is available or trigger integration for the new source change
Proposed solution consists of several parts
- Database to store info about continuous delivery pipelines and events. We propose to use a graph database since events in Continuous Delivery pipeline represent nothing else than directed acyclic graph. We are about to evaluate few open source databases including Neo4j, ArangoDB, Apache TinkerPop.
- Middleware to deliver info from information producers to the database from where it will be available for the further processing by statistical and visualization tools. We propose to use RabbitMQ message broker. In this way, we can build a scalable distributed system that will be able to accommodate setups of all sizes from small one-repository products to the multi-stage multi-components enterprise grade products.
- Information aggregator. We need a service that can receive info about events happening in continuous delivery pipelines and then translate them into the data format used to store them in the database. At this point, we are not aware of open source solution that is capable of implementing such functionality so we will have to develop it ourselves.
- Adaptors for information sources. We need to provide a possibility for information producers to send info to the message broker. At this stage, we will implement core functionality and then wrap it to CLI tools and plugin for Jenkins that will allow sending messages to RabbitMQ so the information aggregator could process them and then store in the database. There is existing Jenkins plugin - RabbitMQ Consumer plugin, that we can use to build upon and save development time
- Trigger mechanism to trigger actions when the new message received. For this SoW we will limit triggering to triggering in Jenkins and will make use of the already existing plugin - RabbitMQ Build Trigger Plugin
POC of proposed solution: Watch this video on YouTube POC features message based triggering which is outside of scope for this SoW
- Information aggregator. We need to develop simple service capable of receiving messages and writing them to the graph database. Service should be stateless so it can be implemented as a pool of worker to ensure scalability. The messaging protocol should also support asynchronous writing if possible.
- Information producer. The intention is to develop core framework that would provide abstraction from input and output methods allowing further extendibility. The framework will provide extension point to define different types of input as well as extension points that can be used to implement support for different types of the middleware. In this way, we will be able to implement producer as command line tool and Jenkins plugin (using input extension point) that send messages using RabbitMQ (implementation using output extension point). In a probable case of need to support additional middleware the only development effort will be an implementation of corresponding adaptor using output extension point and no code changes for already written code. The same applies for protocol changes - messages definition is not included into the framework and provided as an extension using input extension points. That allows to adjust protocol definition for particular needs or create completely new protocol without source code changes for the framework.
Estimated workload is 160 hours
Alfa version of the framework library wrapped into Jenkins plugin and command line tool, information aggregator, documentation, tests, dockererized demo setup. The aim is to produce usable version of the plugin and command line tool, so the concept is accessible for others to try and contribute