CI/CD Pipeline Setup

Setup continuous integration pipeline: $ARGUMENTS

Current Project Analysis

• Project type: @package.json or @setup.py or @go.mod or @pom.xml (detect language/framework)
• Existing workflows: !find .github/workflows -name "*.yml" 2>/dev/null | head -3
• Git branches: !git branch -r | head -5
• Dependencies: @package-lock.json or @requirements.txt or @go.sum (if exists)
• Build scripts: Check for build commands in package.json or Makefile

Task

Implement comprehensive CI/CD following best practices: $ARGUMENTS

• Project Analysis

- Identify the technology stack and deployment requirements

- Review existing build and test processes

- Understand deployment environments (dev, staging, prod)

- Assess current version control and branching strategy

• CI/CD Platform Selection

- Choose appropriate CI/CD platform based on requirements:

- GitHub Actions: Native GitHub integration, extensive marketplace

- GitLab CI: Built-in GitLab, comprehensive DevOps platform

- Jenkins: Self-hosted, highly customizable, extensive plugins

- CircleCI: Cloud-based, optimized for speed

- Azure DevOps: Microsoft ecosystem integration

- AWS CodePipeline: AWS-native solution

• Repository Setup

- Ensure proper .gitignore configuration

- Set up branch protection rules

- Configure merge requirements and reviews

- Establish semantic versioning strategy

• Build Pipeline Configuration

GitHub Actions Example:

name: CI/CD Pipeline
   
   on:
     push:
       branches: [ main, develop ]
     pull_request:
       branches: [ main ]
   
   jobs:
     test:
       runs-on: ubuntu-latest
       steps:
         - uses: actions/checkout@v3
         - name: Setup Node.js
           uses: actions/setup-node@v3
           with:
             node-version: '18'
             cache: 'npm'
         - run: npm ci
         - run: npm run test
         - run: npm run build

GitLab CI Example:

stages:
     - test
     - build
     - deploy
   
   test:
     stage: test
     script:
       - npm ci
       - npm run test
     cache:
       paths:
         - node_modules/

• Environment Configuration

- Set up environment variables and secrets

- Configure different environments (dev, staging, prod)

- Implement environment-specific configurations

- Set up secure secret management

• Automated Testing Integration

- Configure unit test execution

- Set up integration test running

- Implement E2E test execution

- Configure test reporting and coverage

Multi-stage Testing:

test:
     strategy:
       matrix:
         node-version: [16, 18, 20]
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v3
       - uses: actions/setup-node@v3
         with:
           node-version: ${{ matrix.node-version }}
       - run: npm ci
       - run: npm test

• Code Quality Gates

- Integrate linting and formatting checks

- Set up static code analysis (SonarQube, CodeClimate)

- Configure security vulnerability scanning

- Implement code coverage thresholds

• Build Optimization

- Configure build caching strategies

- Implement parallel job execution

- Optimize Docker image builds

- Set up artifact management

Caching Example:

- name: Cache node modules
     uses: actions/cache@v3
     with:
       path: ~/.npm
       key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
       restore-keys: |
         ${{ runner.os }}-node-

• Docker Integration

- Create optimized Dockerfiles

- Set up multi-stage builds

- Configure container registry integration

- Implement security scanning for images

Multi-stage Dockerfile:

FROM node:18-alpine AS builder
   WORKDIR /app
   COPY package*.json ./
   RUN npm ci --only=production
   
   FROM node:18-alpine AS runtime
   WORKDIR /app
   COPY --from=builder /app/node_modules ./node_modules
   COPY . .
   EXPOSE 3000
   CMD ["npm", "start"]

• Deployment Strategies

- Implement blue-green deployment

- Set up canary releases

- Configure rolling updates

- Implement feature flags integration

• Infrastructure as Code

- Use Terraform, CloudFormation, or similar tools

- Version control infrastructure definitions

- Implement infrastructure testing

- Set up automated infrastructure provisioning

• Monitoring and Observability

- Set up application performance monitoring

- Configure log aggregation and analysis

- Implement health checks and alerting

- Set up deployment notifications

• Security Integration

- Implement dependency vulnerability scanning

- Set up container security scanning

- Configure SAST (Static Application Security Testing)

- Implement secrets scanning

Security Scanning Example:

security:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v3
       - name: Run Snyk to check for vulnerabilities
         uses: snyk/actions/node@master
         env:
           SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}

• Database Migration Handling

- Automate database schema migrations

- Implement rollback strategies

- Set up database seeding for testing

- Configure backup and recovery procedures

• Performance Testing Integration

- Set up load testing in pipeline

- Configure performance benchmarks

- Implement performance regression detection

- Set up performance monitoring

• Multi-Environment Deployment

- Configure staging environment deployment

- Set up production deployment with approvals

- Implement environment promotion workflow

- Configure environment-specific configurations

Environment Deployment:

deploy-staging:
     needs: test
     if: github.ref == 'refs/heads/develop'
     runs-on: ubuntu-latest
     steps:
       - name: Deploy to staging
         run: |
           # Deploy to staging environment
   
   deploy-production:
     needs: test
     if: github.ref == 'refs/heads/main'
     runs-on: ubuntu-latest
     environment: production
     steps:
       - name: Deploy to production
         run: |
           # Deploy to production environment

• Rollback and Recovery

- Implement automated rollback procedures

- Set up deployment verification tests

- Configure failure detection and alerts

- Document manual recovery procedures

• Notification and Reporting

- Set up Slack/Teams integration for notifications

- Configure email alerts for failures

- Implement deployment status reporting

- Set up metrics dashboards

• Compliance and Auditing

- Implement deployment audit trails

- Set up compliance checks (SOC 2, HIPAA, etc.)

- Configure approval workflows for sensitive deployments

- Document change management processes

• Pipeline Optimization

- Monitor pipeline performance and costs

- Implement pipeline parallelization

- Optimize resource allocation

- Set up pipeline analytics and reporting

Best Practices:

• Fail Fast: Implement early failure detection
• Parallel Execution: Run independent jobs in parallel
• Caching: Cache dependencies and build artifacts
• Security: Never expose secrets in logs
• Documentation: Document pipeline processes and procedures
• Monitoring: Monitor pipeline health and performance
• Testing: Test pipeline changes in feature branches
• Rollback: Always have a rollback strategy

Sample Complete Pipeline:

name: Full CI/CD Pipeline

on:
  push:
    branches: [ main, develop ]
  pull_request:
    branches: [ main ]

jobs:
  lint-and-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
        with:
          node-version: '18'
          cache: 'npm'
      - run: npm ci
      - run: npm run lint
      - run: npm run test:coverage
      - run: npm run build

  security-scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Security scan
        run: npm audit --audit-level=high

  deploy-staging:
    needs: [lint-and-test, security-scan]
    if: github.ref == 'refs/heads/develop'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Deploy to staging
        run: echo "Deploying to staging"

  deploy-production:
    needs: [lint-and-test, security-scan]
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    environment: production
    steps:
      - uses: actions/checkout@v3
      - name: Deploy to production
        run: echo "Deploying to production"

Start with basic CI and gradually add more sophisticated features as your team and project mature.