Introduction
Continuous integration (CI) and continuous deployment/delivery (CD) pipelines are essential for software development teams looking to increase their agility, speed, and efficiency. CI/CD pipelines automate the process of building, testing, and deploying code, allowing teams to deliver changes quickly and reliably to production.
By using automated pipelines, teams can reduce human errors, ensure higher code quality, and easily roll back any changes that have unintended consequences. Additionally, CI/CD pipelines enable teams to rapidly iterate their products, quickly roll out bug fixes and easily deploy updates.
In short, CI/CD pipelines are a critical tool for modern software development, and this is also true for MuleSoft APIs.
So, let’s see how we can design and implement your winning CI/CD strategy for MuleSoft APIs.
Things to Consider
Before we start, it’s essential to understand that an efficient CI/CD Pipeline Strategy requires careful planning and consideration of the various steps involved.
These steps typically include the following:
- Source Control: To begin, all changes must be tracked in a source control system. This allows teams to keep track of changes, collaborate, and easily roll back any mistakes.
- Continuous Integration: After code is committed to the source control system, CI tools can automatically build and test the code. This allows the team to quickly detect any issues and ensure that the code works as expected.
- Continuous Delivery: After the code passes the automated unit tests, it can be deployed to staging environments.
- Automated Testing: Automated tests should be written to validate code changes, ensure that the code is functioning as expected and that there isn’t any regression in features.
- Monitoring: Should be implemented to track the application’s performance in the production environment. Allowing teams to quickly identify and address any issues that arise.
All the above steps must be addressed to implement a robust, stable and long-lasting CI/CD process. However, for this blog, we will focus on the CI/CD aspect, and details on how to set up source control, automated tests and monitoring will be covered in future blog posts.
Recommended Strategy
Our recommendation when it comes to CI/CD pipelines for MuleSoft APIs is to begin with 2 main pipelines,
- a Continuous Integration (CI) pipeline, which will be around building, unit testing, and publishing to an artefact repository, and
- a Continuous Delivery (CD) pipeline which will be used to deploy the APIs to a Mule Runtime and run some automated tests
On top of that, optionally, we can add another 2 pipelines,
- 1 for feature branches, so that developers can identify issues in the code as early as possible during the development phase and before merging to the main branch, and
- 1 for release branches, which can be useful for releases with longer testing cycles, so that the development of new features can be completed in parallel with the testing
With the four above pipelines in the picture, we will have the following flow:
Release Approach
The way the source code is managed during development and is packaged and prepared for release and deployment to staging and production environments is a key component of every DevOps process. This could not be different for MuleSoft APIs. To keep the packaged code separated by its intended use, we recommend using the standard maven versioning approach with snapshots and release versions.
A snapshot is typically a runnable version of an application that’s still in active development, meaning that it’s packaged and deployed for testing purposes and is not intended for production use.
On the other hand, a release version is a stable version of an application that has undergone a few testing cycles and is considered fit for purpose. This is the application’s version that is intended for production use and is also referred to as a release candidate.
Between snapshots and the final release version, we might have several release candidate versions, each one created to address an identified defect during the testing cycles. At the end of the testing phase, the last release candidate is promoted to production as the release version.
Feature Branch Pipeline
This pipeline’s main purpose is to ensure that the code is compliant with certain quality standards, is unit tested and can be compiled and deployed without any errors.
It is triggered automatically when code is committed to a feature branch.
The pipeline will:
- Checkout the code from the SCM Repository.
- Verify the dependencies using Maven.
- Analyse the code quality using SonarQube.
- Build a snapshot version using Maven.
- Unit test the code with MUnits and validate the coverage with Maven.
Main Branch Pipeline
This pipeline aims to build a Deployable Jar (Snapshot) and publish it to an artefact repository.
It is triggered automatically when code is merged to the Main branch.
In this case, and because some steps have already been executed in the feature pipeline, this pipeline will:
- Checkout the code from the SCM Repository.
- Build a snapshot version.
- Publish a deployable JAR to an Artefact Repository.
- Send a notification to a Slack channel.
Snapshots created from this pipeline will normally be deployed to a Development environment.
Release Preparation Pipeline
Similar to the previous one, this pipeline builds a Deployable Jar and publishes it to an artefact repository, but this time it’s a Release Version rather than a Snapshot.
It is triggered automatically when a release branch is created.
Like before, this pipeline will checkout the code from the SCM repository and then it will:
- Replace the snapshot version with a release candidate version.
- Build a release candidate version.
- Publish a deployable JAR file to an Artefact Repository.
- Send a notification to a Slack channel.
The same release version created from this pipeline will be deployed to all higher environments (i.e. QA, Staging, Production) if no defect is found.
Deployment and Testing Pipeline
The purpose of this pipeline is to retrieve a deployable Jar from the artefact repository, deploy that to a Mule Runtime Environment and then run some automated tests against the deployed Mule Application.
It is triggered manually, usually after acquiring approval from peers for the staging environment or from the business for production.
Note: Steps in grey boxes are optional and are not executed every time the pipeline runs.
This last pipeline will
- Retrieve a Deployable Jar from the Artefact Repo.
- Retrieve a Configuration Jar from the Artefact Repo.
- Retrieve Secrets from an external Secrets Management Service.
- Deploy the Jar to a Mule Runtime.
- Perform automated Integration and Functional Testing with the use of a BDD framework.
- Perform automated Performance Testing with JMeter.
- Send a notification to a Slack channel.
This pipeline can be used to deploy both snapshot and release versions.
Good Practices
Consider using the following good practices:
- Define a Branching Strategy and agree on a Versioning Convention for the Mule APIs before creating a CI/CD Strategy.
- To avoid repeating the same steps in the pipelines (eg. verify, unit tests etc), consider applying restrictions so that code merging from a source to a destination branch will only be possible if all the steps of the source pipeline are successful.
- Depending on how the team works, the code merging process can be fully automated (part of the pipeline) or manual (pull request). Merging must be a secure activity to ensure appropriate reviews and approvals are captured appropriately, especially in highly regulated industries.
- Use the same artefact with different configurations to deploy code to higher environments instead of building the code again for each environment.
- Externalise configuration for environments and third-party systems, retrieve and bundle with the deployable Mule Jar during deployment. Robust Configuration Management approaches separate API functionality (code) away from runtime configuration (behaviour) and open the door for feature toggles.
- Externalise passwords, clients secrets and passphrases to a secrets management service, retrieve and pass them to the Mule Runtime during deployment.
Conclusion
In this blog “Your Winning CI/CD Strategy for MuleSoft APIs” I have outlined a CI/CD Strategy for MuleSoft APIs and described in detail 4 pipelines, their purpose, how they are triggered and the actions they perform.
I have also described a release approach using Maven snapshot and release versions and finished with good practices to consider if you want to create a winning CI/CD Strategy for MuleSoft APIs.
All the above pipelines and steps are part of our DevOps Accelerators, so if you’d like to see a demo, please don’t hesitate to contact us. We’d be happy to tailor our accelerators to your needs or discuss how we can help your team create efficient CI/CD pipelines.
Note: Remember to check our blog in the future, as there will be follow-up posts about Branching Strategy, API Versioning and Monitoring.
