CI CD Pipeline Patterns

CI/CD Pipeline Patterns

A comprehensive skill for designing, implementing, and optimizing CI/CD pipelines using GitHub Actions and modern DevOps practices. Master workflow automation, testing strategies, deployment patterns, and release management for continuous software delivery.

When to Use This Skill

Use this skill when:

• Setting up continuous integration and deployment pipelines for projects
• Automating build, test, and deployment workflows
• Implementing multi-environment deployment strategies (staging, production)
• Managing release automation and versioning
• Configuring matrix builds for multi-platform testing
• Securing CI/CD pipelines with secrets and OIDC
• Optimizing pipeline performance with caching and parallelization
• Building containerized applications with Docker in CI
• Deploying to cloud platforms (AWS, Azure, GCP, Vercel, Netlify)
• Implementing infrastructure as code with Terraform/CloudFormation
• Setting up monorepo CI/CD patterns
• Creating reusable workflow templates and custom actions
• Implementing deployment strategies (blue-green, canary, rolling)
• Automating changelog generation and semantic versioning
• Integrating quality gates and code coverage checks

Core Concepts

CI/CD Fundamentals

Continuous Integration (CI): Automatically building and testing code changes as developers commit to the repository.

Continuous Deployment (CD): Automatically deploying code changes to production after passing tests.

Continuous Delivery: Keeping code in a deployable state, with manual approval for production deployment.

GitHub Actions Architecture

GitHub Actions provides event-driven automation directly integrated with your repository.

Workflows

YAML files in .github/workflows/ that define automated processes:

name: CI Pipeline
on: [push, pull_request]
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Build project
        run: npm run build

Key Components:

• name: Human-readable workflow name
• on: Events that trigger the workflow (push, pull_request, schedule, workflow_dispatch)
• jobs: Collection of steps that run in sequence or parallel
• runs-on: The runner environment (ubuntu-latest, windows-latest, macos-latest)

Jobs

Groups of steps executed on the same runner:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm test

  deploy:
    needs: test  # Runs after 'test' job completes
    runs-on: ubuntu-latest
    steps:
      - run: npm run deploy

Job Features:

• needs: Define job dependencies (sequential execution)
• if: Conditional execution based on expressions
• strategy: Matrix builds for multiple configurations
• outputs: Share data between jobs
• environment: Deployment environments with protection rules

Steps

Individual tasks within a job:

steps:
  - name: Checkout code
    uses: actions/checkout@v4

  - name: Setup Node.js
    uses: actions/setup-node@v4
    with:
      node-version: '20'

  - name: Install dependencies
    run: npm ci

  - name: Run tests
    run: npm test

Step Types:

• uses: Run a pre-built action from marketplace or repository
• run: Execute shell commands
• with: Provide inputs to actions
• env: Set environment variables for the step

Actions

Reusable units of code that perform specific tasks:

Official Actions:

• actions/checkout@v4: Check out repository code
• actions/setup-node@v4: Setup Node.js environment
• actions/cache@v4: Cache dependencies
• actions/upload-artifact@v4: Upload build artifacts
• actions/download-artifact@v4: Download artifacts from previous jobs

Marketplace Actions:

• docker/build-push-action@v5: Build and push Docker images
• aws-actions/configure-aws-credentials@v4: Configure AWS credentials
• codecov/codecov-action@v4: Upload code coverage
• google-github-actions/auth@v2: Authenticate with Google Cloud

Secrets and Variables

Secrets: Encrypted sensitive data (API keys, credentials, tokens)

steps:
  - name: Deploy to production
    env:
      API_KEY: ${{ secrets.API_KEY }}
      DATABASE_URL: ${{ secrets.DATABASE_URL }}
    run: npm run deploy

Variables: Non-sensitive configuration data

env:
  NODE_ENV: ${{ vars.NODE_ENV }}
  API_ENDPOINT: ${{ vars.API_ENDPOINT }}

Secret Types:

• Repository secrets: Available to all workflows in a repository
• Environment secrets: Scoped to specific environments (production, staging)
• Organization secrets: Shared across repositories in an organization

Artifacts

Files produced by workflows that can be downloaded or used by other jobs:

- name: Upload build artifacts
  uses: actions/upload-artifact@v4
  with:
    name: dist-files
    path: dist/
    retention-days: 7

- name: Download artifacts
  uses: actions/download-artifact@v4
  with:
    name: dist-files
    path: ./dist

Workflow Triggers

Event Triggers

Push Events:

on:
  push:
    branches:
      - main
      - develop
      - 'release/**'
    paths:
      - 'src/**'
      - 'package.json'
    tags:
      - 'v*'

Pull Request Events:

on:
  pull_request:
    types: [opened, synchronize, reopened]
    branches:
      - main
    paths-ignore:
      - 'docs/**'
      - '**.md'

Schedule (Cron):

on:
  schedule:
    - cron: '0 0 * * *'  # Daily at midnight UTC
    - cron: '0 */6 * * *'  # Every 6 hours

Manual Triggers (workflow_dispatch):

on:
  workflow_dispatch:
    inputs:
      environment:
        description: 'Environment to deploy to'
        required: true
        type: choice
        options:
          - staging
          - production
      version:
        description: 'Version to deploy'
        required: true
        type: string

Release Events:

on:
  release:
    types: [published, created, released]

Workflow Call (Reusable Workflows):

on:
  workflow_call:
    inputs:
      environment:
        required: true
        type: string
    secrets:
      api-key:
        required: true

Matrix Builds

Run jobs across multiple configurations in parallel:

jobs:
  test:
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [ubuntu-latest, windows-latest, macos-latest]
        node-version: [18, 20, 22]
        include:
          - os: ubuntu-latest
            node-version: 20
            coverage: true
        exclude:
          - os: macos-latest
            node-version: 18
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node-version }}
      - run: npm test
      - if: matrix.coverage
        run: npm run coverage

Matrix Features:

• Parallel execution: All combinations run simultaneously
• include: Add specific configurations
• exclude: Remove specific combinations
• fail-fast: Stop all jobs if one fails (default: true)
• max-parallel: Limit concurrent jobs

Caching Strategies

Speed up workflows by caching dependencies:

Node.js Caching:

- uses: actions/setup-node@v4
  with:
    node-version: '20'
    cache: 'npm'  # Automatically caches npm dependencies

Custom Caching:

- uses: actions/cache@v4
  with:
    path: |
      ~/.npm
      ~/.cache
      node_modules
    key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
    restore-keys: |
      ${{ runner.os }}-node-

Docker Layer Caching:

- uses: docker/build-push-action@v5
  with:
    context: .
    cache-from: type=gha
    cache-to: type=gha,mode=max

Testing Strategies in CI

Unit Testing

Fast, isolated tests for individual components:

jobs:
  unit-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4