labFusion/docs/CI_CD.md

# LabFusion CI/CD Pipeline Documentation

## Overview

LabFusion uses Gitea Actions for continuous integration and deployment. The pipeline is designed to handle a polyglot microservices architecture with Java Spring Boot, Python FastAPI, Node.js Express, and React.

## Pipeline Structure

### Main CI Pipeline (`ci.yml`)
- **Trigger**: Push to `main`/`develop` branches, pull requests
- **Services**: All services (API Gateway, Service Adapters, API Docs, Frontend)
- **Features**: Testing, code quality, security scanning, integration tests

### Service-Specific Pipelines

#### API Gateway (`api-gateway.yml`)
- **Language**: Java Spring Boot
- **Testing**: Unit tests, code quality (SpotBugs, Checkstyle, PMD)
- **Coverage**: JaCoCo code coverage reporting
- **Matrix**: Java 17, 21
- **Security**: Trivy vulnerability scanning

#### Service Adapters (`service-adapters.yml`)
- **Language**: Python FastAPI
- **Testing**: Unit tests, code quality (Black, isort, flake8, mypy)
- **Coverage**: pytest-cov coverage reporting
- **Matrix**: Python 3.9, 3.10, 3.11, 3.12
- **Security**: Bandit, Safety, Trivy scanning

#### API Docs (`api-docs.yml`)
- **Language**: Node.js Express
- **Testing**: Jest unit tests
- **Coverage**: Jest coverage reporting
- **Matrix**: Node.js 16, 18, 20
- **Security**: npm audit, Trivy scanning

#### Frontend (`frontend.yml`)
- **Language**: React
- **Testing**: Jest unit tests, Lighthouse CI
- **Coverage**: Jest coverage reporting
- **Matrix**: Node.js 16, 18, 20
- **Performance**: Lighthouse performance testing
- **Security**: npm audit, Trivy scanning

### Integration Tests (`integration-tests.yml`)
- **Services**: PostgreSQL, Redis
- **Testing**: End-to-end integration tests
- **Performance**: k6 performance testing
- **Coverage**: Complete service interaction testing

## Configuration

### Environment Variables

#### No External Dependencies
- **Local Testing**: All Docker images built locally
- **No Registry**: No external container registry required
- **Self-Contained**: Pipelines run without external dependencies

### Service Configuration

#### API Gateway
- **Port**: 8080
- **Health Check**: `/actuator/health`
- **Dependencies**: Maven, JDK 17/21
- **Quality Tools**: SpotBugs, Checkstyle, PMD, JaCoCo

#### Service Adapters
- **Port**: 8000
- **Health Check**: `/health`
- **Dependencies**: Python 3.9-3.12, pip
- **Quality Tools**: Black, isort, flake8, mypy, Bandit, Safety

#### API Docs
- **Port**: 3000
- **Health Check**: `/health`
- **Dependencies**: Node.js 16-20, npm
- **Quality Tools**: ESLint, Jest

#### Frontend
- **Port**: 3001
- **Health Check**: Root path `/`
- **Dependencies**: Node.js 16-20, npm
- **Quality Tools**: ESLint, Jest, Lighthouse CI

## Pipeline Features

### Code Quality
- **Linting**: ESLint, flake8, Checkstyle
- **Formatting**: Black, isort, Prettier
- **Type Checking**: mypy, TypeScript
- **Security**: Bandit, Safety, npm audit, Trivy
- **Coverage**: Minimum 80% code coverage required

### Testing
- **Unit Tests**: All services
- **Integration Tests**: End-to-end service communication
- **Performance Tests**: k6 load testing
- **Lighthouse Tests**: Frontend performance and accessibility

### Security
- **Vulnerability Scanning**: Trivy for all Docker images
- **Dependency Scanning**: npm audit, Maven security checks
- **Code Security**: Bandit for Python, ESLint security rules

### Caching
- **Maven**: ~/.m2 directory
- **npm**: node_modules and package-lock.json
- **pip**: ~/.cache/pip
- **Docker**: Multi-stage build caching

### Multi-Platform Support
- **Architectures**: linux/amd64, linux/arm64
- **Matrix Testing**: Multiple language versions
- **Cross-Platform**: Docker multi-platform builds

## Docker Images

### Local Testing
- **Build**: Docker images built locally for testing
- **Tags**: Simple test tags (e.g., `api-gateway:test`)
- **Purpose**: Integration testing and validation
- **No Registry**: Images not pushed to external registry

## Monitoring and Reporting

### Test Reports
- **Unit Tests**: JUnit XML reports
- **Coverage**: Codecov integration
- **Performance**: Lighthouse CI reports
- **Security**: SARIF format vulnerability reports

### Notifications
- **Success**: Deployment notifications
- **Failure**: Error notifications with details
- **Security**: Vulnerability alerts

## Troubleshooting

### Common Issues

#### Build Failures
1. **Dependency Issues**: Check package.json, requirements.txt, pom.xml
2. **Version Conflicts**: Update to compatible versions
3. **Cache Issues**: Clear GitHub Actions cache

#### Test Failures
1. **Unit Tests**: Check test files and assertions
2. **Integration Tests**: Verify service health endpoints
3. **Performance Tests**: Check resource limits and timeouts

#### Security Issues
1. **Vulnerabilities**: Update dependencies
2. **Code Issues**: Fix security warnings
3. **Image Issues**: Update base images

### Debugging
- **Logs**: Check GitHub Actions logs
- **Artifacts**: Download test reports and coverage
- **Local Testing**: Run tests locally before pushing

## Best Practices

### Code Quality
- Write comprehensive tests
- Maintain high code coverage
- Follow linting rules
- Use meaningful commit messages

### Security
- Keep dependencies updated
- Scan for vulnerabilities regularly
- Use secure base images
- Implement proper authentication

### Performance
- Optimize Docker images
- Use multi-stage builds
- Implement proper caching
- Monitor resource usage

### Deployment
- Test in staging first
- Use blue-green deployments
- Implement rollback procedures
- Monitor production health

## Future Enhancements

### Planned Features
- **Kubernetes**: Helm charts for deployment
- **Monitoring**: Prometheus and Grafana integration
- **Logging**: Centralized logging with ELK stack
- **Database**: Automated migration testing
- **Notifications**: Slack/Teams integration

### Performance Improvements
- **Parallel Testing**: Run tests in parallel
- **Selective Testing**: Only test changed services
- **Caching**: Improved Docker layer caching
- **Resource Optimization**: Better resource allocation

## Support

For issues with the CI/CD pipeline:
1. Check the GitHub Actions logs
2. Review the troubleshooting section
3. Check service-specific documentation
4. Contact the development team

## References

- [Gitea Actions Documentation](https://docs.gitea.io/en-us/actions/)
- [Docker Multi-platform Builds](https://docs.docker.com/buildx/working-with-buildx/)
- [Trivy Security Scanner](https://trivy.dev/)
- [Lighthouse CI](https://github.com/GoogleChrome/lighthouse-ci)