labFusion/docs/OPTIMIZATION_RECOMMENDATIONS.md

LabFusion CI/CD Optimization Recommendations

Current State Analysis

What We're Currently Using:

• Full Ubuntu Latest images for all workloads (~2-3GB per container)
• Setup actions that install tools every time (Java, Python, Node.js)
• Generic labels that don't optimize for specific workloads

What Each Pipeline Actually Needs:

Heavy Runner (Java + Python):

• Java: JDK 17, Maven (for API Gateway)
• Python: Python 3.11, pip, pytest, flake8, black, isort, bandit, safety, mypy (for Service Adapters)
• Docker: For building images

Light Runner (Node.js + Frontend):

• Node.js: Node 16/18/20, npm (for API Docs and Frontend)
• Docker: For building images

Docker Runner:

• Docker: Docker-in-Docker capabilities
• Basic tools: curl, git

Security Runner:

• Security tools: Trivy, OWASP ZAP, etc.
• Docker: For scanning images

Optimization Strategy

1. Use Specialized Images

Instead of full Ubuntu latest, use optimized images from Gitea's runner-images:

# Heavy Runner - Java workloads
- "java:docker://openjdk:17-jdk-slim"
- "heavy:docker://openjdk:17-jdk-slim"

# Heavy Runner - Python workloads  
- "python:docker://python:3.11-slim"

# Light Runner - Node.js workloads
- "nodejs:docker://node:20-slim"
- "frontend:docker://node:20-slim"

# Docker Runner - Docker-in-Docker
- "docker:docker://docker:24-dind"

# Security Runner - Security tools
- "security:docker://alpine:3.19"

2. Benefits of Specialized Images

• Faster startup: Pre-installed tools mean no setup time
• Smaller images: Only includes what's needed (~500MB vs 2-3GB)
• Better caching: Tools are already in the image layer
• More reliable: No network dependency for tool installation

3. Workflow Optimizations

Remove Redundant Setup Steps

Before (current):

- name: Set up JDK 17
  uses: actions/setup-java@v4
  with:
    java-version: '17'
    distribution: 'temurin'

After (optimized):

# Remove this step - Java 17 and Maven are pre-installed
# - name: Set up JDK 17
#   uses: actions/setup-java@v4

Use Matrix Strategy for Node.js Versions

Current approach: Multiple setup steps Optimized approach: Use matrix with specialized images

strategy:
  matrix:
    node-version: [16, 18, 20]
    image:
      - "ubuntu-22.04-node16"
      - "ubuntu-22.04-node18" 
      - "ubuntu-22.04-node20"

4. Fallback Strategy

Keep ubuntu-latest as fallback for:

• Complex builds that need many tools
• Debugging when specialized images fail
• New workloads not yet optimized

5. Performance Impact

Expected Improvements:

• Startup time: 30-60 seconds faster per job
• Image size: 60-70% smaller
• Cache efficiency: Better layer reuse
• Resource usage: Lower memory footprint

Example Timeline:

Current:  Checkout (10s) + Setup Java (30s) + Setup Maven (20s) + Run Tests (60s) = 120s
Optimized: Checkout (10s) + Run Tests (60s) = 70s
Savings: 50s per job (42% faster)

Implementation Plan

Phase 1: Update Config Files ✅

• Update config_heavy.yaml with specialized images
• Update config_light.yaml with Node.js images
• Update config_docker.yaml with Docker-in-Docker image
• Update config_security.yaml with security tools image

Phase 2: Test and Validate

• Test each runner with optimized images
• Verify all tools are available
• Check performance improvements
• Validate fallback works

Phase 3: Optimize Workflows

• Remove redundant setup steps
• Update matrix strategies
• Add performance monitoring
• Document changes

Phase 4: Monitor and Tune

• Monitor job execution times
• Track resource usage
• Fine-tune based on metrics
• Update documentation

Available Docker Images

Note: Gitea doesn't provide a comprehensive set of specialized runner images. We use official, well-maintained Docker images instead:

Official Docker Images Used:

Java Images:

• openjdk:17-jdk-slim - OpenJDK 17 with Debian slim base
• openjdk:21-jdk-slim - OpenJDK 21 with Debian slim base

Python Images:

• python:3.11-slim - Python 3.11 with Debian slim base
• python:3.12-slim - Python 3.12 with Debian slim base

Node.js Images:

• node:20-slim - Node.js 20 with Debian slim base
• node:18-slim - Node.js 18 with Debian slim base

Docker Images:

• docker:24-dind - Docker-in-Docker with Alpine base
• docker:24-dind-rootless - Rootless Docker-in-Docker

Security Images:

• alpine:3.19 - Minimal Alpine Linux for security tools
• alpine:latest - Latest Alpine Linux

Next Steps

1. Test the updated config files with a simple job
2. Verify image availability on Gitea's registry
3. Update workflows to remove redundant setup steps
4. Monitor performance improvements
5. Document the changes for the team

References

• Gitea Runner Images
• Gitea Actions Design
• LabFusion Runner Labels