labFusion/docs/OPTIMIZATION_RECOMMENDATIONS.md

LabFusion CI/CD Optimization Recommendations

Current State Analysis

What We're Currently Using:

• Full Ubuntu Latest images for all workloads (~2-3GB per container)
• Setup actions that install tools every time (Java, Python, Node.js)
• Generic labels that don't optimize for specific workloads

What Each Pipeline Actually Needs:

Heavy Runner (Java + Python):

• Java: JDK 17, Maven (for API Gateway)
• Python: Python 3.11, pip, pytest, flake8, black, isort, bandit, safety, mypy (for Service Adapters)
• Docker: For building images

Light Runner (Node.js + Frontend):

• Node.js: Node 16/18/20, npm (for API Docs and Frontend)
• Docker: For building images

Docker Runner:

• Docker: Docker-in-Docker capabilities
• Basic tools: curl, git

Security Runner:

• Security tools: Trivy, OWASP ZAP, etc.
• Docker: For scanning images

Optimization Strategy

1. Use Specialized Images

Instead of full Ubuntu latest, use optimized images from Gitea's runner-images:

# Heavy Runner - Java workloads
- "java:docker://docker.gitea.com/runner-images:ubuntu-22.04-java17-maven"
- "heavy:docker://docker.gitea.com/runner-images:ubuntu-22.04-java17-maven"

# Heavy Runner - Python workloads  
- "python:docker://docker.gitea.com/runner-images:ubuntu-22.04-python3.11"

# Light Runner - Node.js workloads
- "nodejs:docker://docker.gitea.com/runner-images:ubuntu-22.04-node20"
- "frontend:docker://docker.gitea.com/runner-images:ubuntu-22.04-node20"

# Docker Runner - Docker-in-Docker
- "docker:docker://docker.gitea.com/runner-images:ubuntu-22.04-docker"

# Security Runner - Security tools
- "security:docker://docker.gitea.com/runner-images:ubuntu-22.04-security"

2. Benefits of Specialized Images

• Faster startup: Pre-installed tools mean no setup time
• Smaller images: Only includes what's needed (~500MB vs 2-3GB)
• Better caching: Tools are already in the image layer
• More reliable: No network dependency for tool installation

3. Workflow Optimizations

Remove Redundant Setup Steps

Before (current):

- name: Set up JDK 17
  uses: actions/setup-java@v4
  with:
    java-version: '17'
    distribution: 'temurin'

After (optimized):

# Remove this step - Java 17 and Maven are pre-installed
# - name: Set up JDK 17
#   uses: actions/setup-java@v4

Use Matrix Strategy for Node.js Versions

Current approach: Multiple setup steps Optimized approach: Use matrix with specialized images

strategy:
  matrix:
    node-version: [16, 18, 20]
    image:
      - "ubuntu-22.04-node16"
      - "ubuntu-22.04-node18" 
      - "ubuntu-22.04-node20"

4. Fallback Strategy

Keep ubuntu-latest as fallback for:

• Complex builds that need many tools
• Debugging when specialized images fail
• New workloads not yet optimized

5. Performance Impact

Expected Improvements:

• Startup time: 30-60 seconds faster per job
• Image size: 60-70% smaller
• Cache efficiency: Better layer reuse
• Resource usage: Lower memory footprint

Example Timeline:

Current:  Checkout (10s) + Setup Java (30s) + Setup Maven (20s) + Run Tests (60s) = 120s
Optimized: Checkout (10s) + Run Tests (60s) = 70s
Savings: 50s per job (42% faster)

Implementation Plan

Phase 1: Update Config Files ✅

• Update config_heavy.yaml with specialized images
• Update config_light.yaml with Node.js images
• Update config_docker.yaml with Docker-in-Docker image
• Update config_security.yaml with security tools image

Phase 2: Test and Validate

• Test each runner with optimized images
• Verify all tools are available
• Check performance improvements
• Validate fallback works

Phase 3: Optimize Workflows

• Remove redundant setup steps
• Update matrix strategies
• Add performance monitoring
• Document changes

Phase 4: Monitor and Tune

• Monitor job execution times
• Track resource usage
• Fine-tune based on metrics
• Update documentation

Available Gitea Runner Images

Check Gitea's runner-images repository for available images:

Java Images:

• ubuntu-22.04-java17-maven
• ubuntu-22.04-java21-maven
• ubuntu-22.04-java17-gradle

Python Images:

• ubuntu-22.04-python3.11
• ubuntu-22.04-python3.12
• ubuntu-22.04-python3.11-pip

Node.js Images:

• ubuntu-22.04-node18
• ubuntu-22.04-node20
• ubuntu-22.04-node21

Docker Images:

• ubuntu-22.04-docker
• ubuntu-22.04-docker-compose

Security Images:

• ubuntu-22.04-security
• ubuntu-22.04-trivy

Next Steps

1. Test the updated config files with a simple job
2. Verify image availability on Gitea's registry
3. Update workflows to remove redundant setup steps
4. Monitor performance improvements
5. Document the changes for the team

References

• Gitea Runner Images
• Gitea Actions Design
• LabFusion Runner Labels