labFusion/docs/OPTIMIZATION_RECOMMENDATIONS.md

# LabFusion CI/CD Optimization Recommendations

## Current State Analysis

### What We're Currently Using:
- **Full Ubuntu Latest images** for all workloads (~2-3GB per container)
- **Setup actions** that install tools every time (Java, Python, Node.js)
- **Generic labels** that don't optimize for specific workloads

### What Each Pipeline Actually Needs:

#### Heavy Runner (Java + Python):
- **Java**: JDK 17, Maven (for API Gateway)
- **Python**: Python 3.11, pip, pytest, flake8, black, isort, bandit, safety, mypy (for Service Adapters)
- **Docker**: For building images

#### Light Runner (Node.js + Frontend):
- **Node.js**: Node 16/18/20, npm (for API Docs and Frontend)
- **Docker**: For building images

#### Docker Runner:
- **Docker**: Docker-in-Docker capabilities
- **Basic tools**: curl, git

#### Security Runner:
- **Security tools**: Trivy, OWASP ZAP, etc.
- **Docker**: For scanning images

## Optimization Strategy

### 1. Use Specialized Images

Instead of full Ubuntu latest, use optimized images from [Gitea's runner-images](https://gitea.com/docker.gitea.com/runner-images):

```yaml
# Heavy Runner - Java workloads
- "java:docker://openjdk:17-jdk-slim"
- "heavy:docker://openjdk:17-jdk-slim"

# Heavy Runner - Python workloads  
- "python:docker://python:3.11-slim"

# Light Runner - Node.js workloads
- "nodejs:docker://node:20-slim"
- "frontend:docker://node:20-slim"

# Docker Runner - Docker-in-Docker
- "docker:docker://docker:24-dind"

# Security Runner - Security tools
- "security:docker://alpine:3.19"
```

### 2. Benefits of Specialized Images

- **Faster startup**: Pre-installed tools mean no setup time
- **Smaller images**: Only includes what's needed (~500MB vs 2-3GB)
- **Better caching**: Tools are already in the image layer
- **More reliable**: No network dependency for tool installation

### 3. Workflow Optimizations

#### Remove Redundant Setup Steps

**Before (current):**
```yaml
- name: Set up JDK 17
  uses: actions/setup-java@v4
  with:
    java-version: '17'
    distribution: 'temurin'
```

**After (optimized):**
```yaml
# Remove this step - Java 17 and Maven are pre-installed
# - name: Set up JDK 17
#   uses: actions/setup-java@v4
```

#### Use Matrix Strategy for Node.js Versions

**Current approach**: Multiple setup steps
**Optimized approach**: Use matrix with specialized images

```yaml
strategy:
  matrix:
    node-version: [16, 18, 20]
    image:
      - "ubuntu-22.04-node16"
      - "ubuntu-22.04-node18" 
      - "ubuntu-22.04-node20"
```

### 4. Fallback Strategy

Keep `ubuntu-latest` as fallback for:
- Complex builds that need many tools
- Debugging when specialized images fail
- New workloads not yet optimized

### 5. Performance Impact

#### Expected Improvements:
- **Startup time**: 30-60 seconds faster per job
- **Image size**: 60-70% smaller
- **Cache efficiency**: Better layer reuse
- **Resource usage**: Lower memory footprint

#### Example Timeline:
```
Current:  Checkout (10s) + Setup Java (30s) + Setup Maven (20s) + Run Tests (60s) = 120s
Optimized: Checkout (10s) + Run Tests (60s) = 70s
Savings: 50s per job (42% faster)
```

## Implementation Plan

### Phase 1: Update Config Files ✅
- [x] Update `config_heavy.yaml` with specialized images
- [x] Update `config_light.yaml` with Node.js images  
- [x] Update `config_docker.yaml` with Docker-in-Docker image
- [x] Update `config_security.yaml` with security tools image

### Phase 2: Test and Validate
- [ ] Test each runner with optimized images
- [ ] Verify all tools are available
- [ ] Check performance improvements
- [ ] Validate fallback works

### Phase 3: Optimize Workflows
- [ ] Remove redundant setup steps
- [ ] Update matrix strategies
- [ ] Add performance monitoring
- [ ] Document changes

### Phase 4: Monitor and Tune
- [ ] Monitor job execution times
- [ ] Track resource usage
- [ ] Fine-tune based on metrics
- [ ] Update documentation

## Available Docker Images

**Note**: Gitea doesn't provide a comprehensive set of specialized runner images. We use official, well-maintained Docker images instead:

### Official Docker Images Used:

#### Java Images:
- `openjdk:17-jdk-slim` - OpenJDK 17 with Debian slim base
- `openjdk:21-jdk-slim` - OpenJDK 21 with Debian slim base

#### Python Images:
- `python:3.11-slim` - Python 3.11 with Debian slim base
- `python:3.12-slim` - Python 3.12 with Debian slim base

#### Node.js Images:
- `node:20-slim` - Node.js 20 with Debian slim base
- `node:18-slim` - Node.js 18 with Debian slim base

#### Docker Images:
- `docker:24-dind` - Docker-in-Docker with Alpine base
- `docker:24-dind-rootless` - Rootless Docker-in-Docker

#### Security Images:
- `alpine:3.19` - Minimal Alpine Linux for security tools
- `alpine:latest` - Latest Alpine Linux

## Next Steps

1. **Test the updated config files** with a simple job
2. **Verify image availability** on Gitea's registry
3. **Update workflows** to remove redundant setup steps
4. **Monitor performance** improvements
5. **Document the changes** for the team

## References

- [Gitea Runner Images](https://gitea.com/docker.gitea.com/runner-images)
- [Gitea Actions Design](https://docs.gitea.com/usage/actions/design#act-runner)
- [LabFusion Runner Labels](RUNNER_LABELS.md)