labFusion/docs/CI_CD.md

LabFusion CI/CD Pipeline Documentation

Overview

LabFusion uses Gitea Actions for continuous integration and deployment. The pipeline is designed to handle a polyglot microservices architecture with Java Spring Boot, Python FastAPI, Node.js Express, and React.

Pipeline Structure

Main CI Pipeline (ci.yml)

• Trigger: Push to main/develop branches, pull requests
• Services: All services (API Gateway, Service Adapters, API Docs, Frontend)
• Features: Testing, code quality, security scanning, integration tests

Service-Specific Pipelines

API Gateway (api-gateway.yml)

• Language: Java Spring Boot
• Testing: Unit tests, code quality (SpotBugs, Checkstyle, PMD)
• Coverage: JaCoCo code coverage reporting
• Matrix: Java 17, 21
• Security: Trivy vulnerability scanning

Service Adapters (service-adapters.yml)

• Language: Python FastAPI
• Testing: Unit tests, code quality (Black, isort, flake8, mypy)
• Coverage: pytest-cov coverage reporting
• Matrix: Python 3.9, 3.10, 3.11, 3.12
• Security: Bandit, Safety, Trivy scanning

API Docs (api-docs.yml)

• Language: Node.js Express
• Testing: Jest unit tests
• Coverage: Jest coverage reporting
• Matrix: Node.js 16, 18, 20
• Security: npm audit, Trivy scanning

Frontend (frontend.yml)

• Language: React
• Testing: Jest unit tests, Lighthouse CI
• Coverage: Jest coverage reporting
• Matrix: Node.js 16, 18, 20
• Performance: Lighthouse performance testing
• Security: npm audit, Trivy scanning

Integration Tests (integration-tests.yml)

• Services: PostgreSQL, Redis
• Testing: End-to-end integration tests
• Performance: k6 performance testing
• Coverage: Complete service interaction testing

Configuration

Environment Variables

No External Dependencies

• Local Testing: All Docker images built locally
• No Registry: No external container registry required
• Self-Contained: Pipelines run without external dependencies

Service Configuration

API Gateway

• Port: 8080
• Health Check: /actuator/health
• Dependencies: Maven, JDK 17/21
• Quality Tools: SpotBugs, Checkstyle, PMD, JaCoCo

Service Adapters

• Port: 8000
• Health Check: /health
• Dependencies: Python 3.9-3.12, pip
• Quality Tools: Black, isort, flake8, mypy, Bandit, Safety

API Docs

• Port: 3000
• Health Check: /health
• Dependencies: Node.js 16-20, npm
• Quality Tools: ESLint, Jest

Frontend

• Port: 3001
• Health Check: Root path /
• Dependencies: Node.js 16-20, npm
• Quality Tools: ESLint, Jest, Lighthouse CI

Pipeline Features

Code Quality

• Linting: ESLint, flake8, Checkstyle
• Formatting: Black, isort, Prettier
• Type Checking: mypy, TypeScript
• Security: Bandit, Safety, npm audit, Trivy
• Coverage: Minimum 80% code coverage required

Testing

• Unit Tests: All services
• Integration Tests: End-to-end service communication
• Performance Tests: k6 load testing
• Lighthouse Tests: Frontend performance and accessibility

Security

• Vulnerability Scanning: Trivy for all Docker images
• Dependency Scanning: npm audit, Maven security checks
• Code Security: Bandit for Python, ESLint security rules

Caching

• Maven: ~/.m2 directory
• npm: node_modules and package-lock.json
• pip: ~/.cache/pip
• Docker: Multi-stage build caching

Multi-Platform Support

• Architectures: linux/amd64, linux/arm64
• Matrix Testing: Multiple language versions
• Cross-Platform: Docker multi-platform builds

Docker Images

Local Testing

• Build: Docker images built locally for testing
• Tags: Simple test tags (e.g., api-gateway:test)
• Purpose: Integration testing and validation
• No Registry: Images not pushed to external registry

Monitoring and Reporting

Test Reports

• Unit Tests: JUnit XML reports
• Coverage: Codecov integration
• Performance: Lighthouse CI reports
• Security: SARIF format vulnerability reports

Notifications

• Success: Deployment notifications
• Failure: Error notifications with details
• Security: Vulnerability alerts

Troubleshooting

Common Issues

Build Failures

1. Dependency Issues: Check package.json, requirements.txt, pom.xml
2. Version Conflicts: Update to compatible versions
3. Cache Issues: Clear GitHub Actions cache

Test Failures

1. Unit Tests: Check test files and assertions
2. Integration Tests: Verify service health endpoints
3. Performance Tests: Check resource limits and timeouts

Security Issues

1. Vulnerabilities: Update dependencies
2. Code Issues: Fix security warnings
3. Image Issues: Update base images

Debugging

• Logs: Check GitHub Actions logs
• Artifacts: Download test reports and coverage
• Local Testing: Run tests locally before pushing

Best Practices

Code Quality

• Write comprehensive tests
• Maintain high code coverage
• Follow linting rules
• Use meaningful commit messages

Security

• Keep dependencies updated
• Scan for vulnerabilities regularly
• Use secure base images
• Implement proper authentication

Performance

• Optimize Docker images
• Use multi-stage builds
• Implement proper caching
• Monitor resource usage

Deployment

• Test in staging first
• Use blue-green deployments
• Implement rollback procedures
• Monitor production health

Future Enhancements

Planned Features

• Kubernetes: Helm charts for deployment
• Monitoring: Prometheus and Grafana integration
• Logging: Centralized logging with ELK stack
• Database: Automated migration testing
• Notifications: Slack/Teams integration

Performance Improvements

• Parallel Testing: Run tests in parallel
• Selective Testing: Only test changed services
• Caching: Improved Docker layer caching
• Resource Optimization: Better resource allocation

Support

For issues with the CI/CD pipeline:

1. Check the GitHub Actions logs
2. Review the troubleshooting section
3. Check service-specific documentation
4. Contact the development team

References

• Gitea Actions Documentation
• Docker Multi-platform Builds
• Trivy Security Scanner
• Lighthouse CI