CI/CD: Continuous Integration & Delivery Explained
CI/CD enables the best tech companies to improve their products many times per day. Here’s what you need to know to do the same.
What is CI/CD?
CI/CD is a way of developing software in which you’re able to release updates at any time in a sustainable way. When changing code is routine, development cycles are more frequent, meaningful and faster.
“CI/CD” stands for the combined practices of Continuous Integration (CI) and Continuous Delivery (CD).
Continuous Integration is a prerequisite for CI/CD, and requires:
- Developers to merge their changes to the main code branch many times per day.
- Each code merge to trigger an automated code build and test sequence. Developers ideally receive results in less than 10 minutes, so that they can stay focused on their work.
The job of Continuous Integration is to produce an artifact that can be deployed. The role of automated tests in CI is to verify that the artifact for the given version of code is safe to be deployed.
In the practice of Continuous Delivery, code changes are also continuously deployed, although the deployments are triggered manually. If the entire process of moving code from source repository to production is fully automated, the process is called Continuous Deployment.
A long deployment process is error-prone and distracting. Developers get frustrated by a process that feels like an overkill. Teams trade small iterations for risky big-bang releases. They may also be led not to make some small improvements at all.
The goal of CI/CD is to make the deployment easy enough, safe and fast. In such context, the probability of introducing major bugs is low. When something bad does happen, it’s easy to deliver a fix or revert the change.
A litmus test for doing CI/CD
If any developer in your team can stop what they’re doing right now and ship the current development version of code to production in 20 minutes or less without anyone stressing about what could happen — congratulations, you’re doing CI/CD!
CI/CD principles
Continuous Delivery practices take CI further by describing principles for successful production deployments:
- Architect the system in a way that supports iterative releases. Avoid tight coupling between components. Implement metrics that help detect issues in real-time.
- Always keep the code in a deployable state. Maintain a comprehensive and healthy automated test suite. Build in monitoring, logging, and fault-tolerance by design.
- Work in small iterations. For example, if you develop in feature branches, they should live no longer than a day. When you need more time to develop new features, use feature flags.
- Developers can push the code into production-like staging environments. This ensures that the new version of the software will work when it gets in the hands of users.
- Anyone can deploy any version of the software to any environment on demand, at a push of a button. If you need to consult a wiki on how to deploy, it’s game over.
- If you build it, you run it. Autonomous engineering teams should be responsible for the quality and stability of the software they build. This breaks down the silos between traditional developers and operations groups, as they work together to achieve high-level goals.
To make CI/CD a reality, you need to automate everything that you can in the software delivery process and run it in a CI/CD pipeline.
Example CI/CD workflows
Here’s a simple example of fully automated CI/CD (Continuous Deployment) pipeline:
Each change on the master Git branch performs the following steps on Semaphore:
- Build code and web assets, while reusing dependency cache.
- Run an automated test suite. This is a JavaScript / Node.js application, so the tests are written with Jest.
- If the tests pass, a Deploy block updates the production code that runs in the cloud.
CI/CD with manual steps
Here’s a CI/CD workflow for Kubernetes which includes more steps:
In this example, each change automatically performs these steps:
- Build application from source code and dependencies.
- Run an automated test suite.
- If tests pass, automatically build a Docker container image and push it to a private registry.
At the end of the Docker build pipeline, we have a working artifact, a container image. The developer, or more formally, release manager, can decide to manually trigger:
- Deployment to staging or production, which may include smoke tests to verify no major problems have been introduced.
- Tagging of the container image as an artifact that was introduced to production, to enable audits and rollbacks.
DevOps and CI/CD
DevOps aims to create better communication and collaboration between “development” and “operations” groups. The end goal is to help companies build and release software at high velocity.
The DevOps model is a set of cultural traits, engineering practices, and tools. CI/CD is one of the DevOps practices: a process for effective software delivery.
What are the benefits of CI/CD?
CI/CD is much more than the automation of tasks to avoid human error. While CI today is mainstream, CD is still not. So if your company practices CI/CD, it has a competitive advantage which enables you to:
Deliver software with less risk. By describing your CI/CD pipelines as code in your Git repository, you standardize release processes across projects. By testing every change in source code, we reduce the chances of introducing bugs.
Release new features more frequently. A CI/CD pipeline can visualize your entire path from commit to production in a single screen. You can navigate across stages, spot inefficiencies, and optimize your process. Speed wins the market. By removing the roadblocks to productivity, you enable your company for success.
Deliver the product that users need. Delivering updates often leads to more user feedback. You can take advantage of that by A/B testing features, or testing early versions of products with real customers. This way you avoid investing too much in features that your customers don’t need, and focus on those that matter.
Improve developer productivity. Engineering teams that don’t practice CI/CD often work under stress. There are constant fires for bad deploys and hard-to-fix outages. They write a lot of code that never gets used. Long-lived feature branches are too big to get proper peer review, so code degrades in quality. CI/CD guides product management to optimize for user impact. Developers deploy code while it’s fresh in their minds. The result is a happy engineering team.
The benefits of CI/CD compound: fast deploy cycles lead to less risky and more frequent updates, which lead to more user feedback, which leads to happy developers building better products.
Learn more: 7 Ways Continuous Delivery Helps Build a Culture of Learning.
- Your customers don’t want continuous updates to their systems.
- Regulations restrict how software can be updated. For example, continuously updating software used in aerospace, telecom, and medical industries is not an option.
When is CI/CD not feasible?
For a more in-depth look into CI and its best practices, read Semaphore’s guide to Continuous Integration.
“ Continuous Delivery is great but won’t work with my project “ is a common remark. There are indeed some cases when CD may not be suitable:
Even in a CD-averse environment, teams can enjoy the benefits of easy deployment and keeping the system in a deployable state. They can also practice and reap the benefits of CI to the full extent.
A typical journey to CI/CD
If you’re used to long development cycles, you will need to change your mindset.
But if you are considering to adopt CI/CD, you’ve made an important first step!
Think of anything that gets in the way of practicing CI/CD as technical debt, and treat it as such. Estimate work that needs to be done. Set clear expectations with all stakeholders, and schedule a time to work on it.
There are two major milestones on your journey to CI/CD success. The first is to implement CI: fast, reliable and frequent integrations. The second is to implement CD: automate deployment, make it a push-button operation, and make it easy to test new code in production-like environments.
Getting to more frequent integrations
1/ The first thing is to start treating master as if you would deploy it at any time. Since you haven’t done this before, it will be hard. But it’s a start of doing something painful often until it stops being painful.
2/ Treat every test failure as a bug. This includes flaky tests. Extract the log, investigate and fix it. Regardless of what the test was for, you can’t close the issue until it’s fixed.
3/ Improve your test suite. You may need to rewrite some code to make tests more reliable. Have all developers commit to writing tests as carefully as they write production code. After a while, you’ll reach a point where a failed test means there’s a real bug. Eventually, you’ll hit 100% green CI builds. That’s your first mountain. Celebrate it!
4/ Stop using long-lived feature branches and start branching by abstraction using feature flags. This way everyone’s pushing to master and it’s easy to test features in development with the rest of the system. So you can detect issues right away instead of after merging months later.
5/ Optimize the feedback loop. Investigate slow tests and improve or rewrite them to be faster. Invest in a CI/CD tool that can separate your CI pipeline in stages and run tests in parallel. Semaphore customers often report drastic improvements such as reduced build time from an hour or even two to around ten minutes. This way you fully test every pull request before it’s merged.
Developing a fast and comprehensive automated test suite is a big upfront investment. However, the benefits are huge. Testers and QA can hunt for bugs and do much more useful checking. Developers are confident that they can push a change and CI will catch unforeseen issues. This confidence drives development velocity.
Getting to more frequent releases
6/ The final step is to automate the deployment.
Learn more here: Understanding the CI/CD Pipeline.
If your deployment process is manual, start by documenting it as a series of tasks. One by one, automate each task with a script (Bash, Makefile), or a configuration management tool such as Ansible, Chef or Puppet. Once you’re done, put all the tasks together in a CI/CD pipeline. You can start by setting up the transition in each stage to be triggered manually.
Learn more here: What’s the difference between Continuous Integration, Continuous Deployment and Continuous Delivery?
The details of release automation will depend a lot on your technology stack. The implementation when using Python on Docker will be different from using VMs to run Java application servers.
7/ If you’re deploying in the cloud, you can automate the creation of infrastructure by using “infrastructure as code” tools such as AWS Cloudformation and Terraform. If you’re deploying on-premises, you will likely not be able to create and destroy production-like environments on-demand. But you can still allocate them to be always available.
The final result is that after all this effort your team will start releasing monthly, then weekly, then daily. You can be Agile in the original sense of the term and deliver value incrementally, even for Enterprise customers.
Continuous Delivery vs Continuous Deployment
Continuous Deployment is a case of CI/CD in which developers deploy every change to production automatically. There are no manual steps. The only approval involved may be an approval of the related code pull request.
Continuous Deployment requires teams to have top-notch testing and monitoring practices. Otherwise quality will degrade over time. It also requires a communication process that keeps support, marketing and sales teams in the loop.
Get started with CI/CD
When you’re ready to set up a CI/CD process for your project, sign up for a free Semaphore account. Semaphore provides autoscaling, enterprise-grade CI/CD pipelines as a service.
Originally published at semaphoreci.com.
