labFusion/docs/RUNNERS.md

# LabFusion Gitea Runners Setup

This document explains how to set up and manage multiple Gitea Actions runners for the LabFusion project using Docker Compose.

## Overview

The LabFusion project uses multiple specialized runners to handle different types of CI/CD workloads:

- **Heavy Runner**: Java Spring Boot (API Gateway) and Python FastAPI (Service Adapters)
- **Light Runner**: Node.js (API Docs) and React (Frontend)
- **Docker Runner**: Integration tests and Docker builds
- **Security Runner**: Security scans and vulnerability assessments

**Note**: All runners use the official `gitea/act_runner:nightly` image with individual configuration files. Each runner has its own data directory and configuration for better isolation and management.

## Quick Start

### 1. Prerequisites

- Docker and Docker Compose installed
- Gitea instance running
- Runner registration token from Gitea

### 2. Configuration

**Windows PowerShell:**
```powershell
# Navigate to runners directory
cd runners

# Copy the environment template
Copy-Item env.runners.example .env.runners

# Edit the configuration
notepad .env.runners
```

**Linux/macOS:**
```bash
# Navigate to runners directory
cd runners

# Copy the environment template
cp env.runners.example .env.runners

# Edit the configuration
nano .env.runners
```

**Manual Creation (if copy fails):**
Create `runners/.env.runners` file with the following content:

```bash
# Gitea instance URL (adjust port if different)
GITEA_INSTANCE_URL=http://localhost:3000

# Runner registration token (get from Gitea Admin > Actions > Runners)
GITEA_RUNNER_REGISTRATION_TOKEN=your_runner_registration_token_here
```

**Important:** Replace `your_runner_registration_token_here` with your actual token from Gitea Admin > Actions > Runners.

### 2.1. Create Configuration Files

Each runner requires its own configuration file. Create the following files in the `runners/` directory:

**`config_heavy.yaml`:**
```yaml
log:
  level: info

runner:
  file: /data/.runner
  capacity: 2
  envs:
    - JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64
    - PATH=/usr/lib/jvm/java-17-openjdk-amd64/bin:$PATH
```

**`config_light.yaml`:**
```yaml
log:
  level: info

runner:
  file: /data/.runner
  capacity: 2
  envs:
    - NODE_VERSION=20
```

**`config_docker.yaml`:**
```yaml
log:
  level: info

runner:
  file: /data/.runner
  capacity: 1
```

**`config_security.yaml`:**
```yaml
log:
  level: info

runner:
  file: /data/.runner
  capacity: 1
```

### 3. Start Runners

**Linux/macOS:**
```bash
# Navigate to runners directory
cd runners

# Make script executable
chmod +x manage-runners.sh

# Start all runners
./manage-runners.sh start
```

**Windows PowerShell:**
```powershell
# Navigate to runners directory
cd runners

# Start all runners
.\manage-runners.ps1 start
```

**Docker Compose directly:**
```bash
# Navigate to runners directory
cd runners

# Start runners
docker-compose -f docker-compose.runners.yml --env-file .env.runners up -d
```

## Runner Configuration

### Resource Allocation

| Runner | CPU | Memory | Use Case |
|--------|-----|--------|----------|
| Heavy | 4 cores | 8GB | Java/Python builds |
| Light | 2 cores | 4GB | Node.js/Frontend |
| Docker | 6 cores | 12GB | Integration tests |
| Security | 2 cores | 4GB | Security scans |

### Labels

Each runner is configured with specific labels for workload routing:

- **Heavy Runner**: `ubuntu-latest,self-hosted,heavy,java,python`
- **Light Runner**: `ubuntu-latest,self-hosted,light,nodejs,frontend`
- **Docker Runner**: `ubuntu-latest,self-hosted,docker,integration`
- **Security Runner**: `ubuntu-latest,self-hosted,security,scan`

## Management Commands

### Using the Management Scripts

**Linux/macOS:**
```bash
# Navigate to runners directory
cd runners

# Start runners
./manage-runners.sh start

# Check status
./manage-runners.sh status

# View logs
./manage-runners.sh logs
./manage-runners.sh logs labfusion-runner-heavy

# Restart runners
./manage-runners.sh restart

# Stop runners
./manage-runners.sh stop

# Clean up (removes all data)
./manage-runners.sh clean
```

**Windows PowerShell:**
```powershell
# Navigate to runners directory
cd runners

# Start runners
.\manage-runners.ps1 start

# Check status
.\manage-runners.ps1 status

# View logs
.\manage-runners.ps1 logs
.\manage-runners.ps1 logs labfusion-runner-heavy

# Restart runners
.\manage-runners.ps1 restart

# Stop runners
.\manage-runners.ps1 stop

# Clean up (removes all data)
.\manage-runners.ps1 clean
```

### Direct Docker Compose Commands

```bash
# Navigate to runners directory
cd runners

# Start all runners
docker-compose -f docker-compose.runners.yml up -d

# Check status
docker-compose -f docker-compose.runners.yml ps

# View logs
docker-compose -f docker-compose.runners.yml logs -f

# Stop runners
docker-compose -f docker-compose.runners.yml down

# Remove everything (including volumes)
docker-compose -f docker-compose.runners.yml down -v
```

## Workflow Configuration

To use specific runners in your workflows, update the `runs-on` field:

```yaml
# .gitea/workflows/api-gateway.yml
jobs:
  test:
    runs-on: [self-hosted, heavy]  # Uses heavy runner

# .gitea/workflows/frontend.yml
jobs:
  test:
    runs-on: [self-hosted, light]  # Uses light runner

# .gitea/workflows/integration-tests.yml
jobs:
  integration:
    runs-on: [self-hosted, docker]  # Uses docker runner
```

## Monitoring

### Health Checks

Each runner includes health checks that monitor:
- Container status
- HTTP health endpoint
- Resource usage

### Logs

View logs for troubleshooting:

```bash
# All runners
docker-compose -f docker-compose.runners.yml logs -f

# Specific runner
docker-compose -f docker-compose.runners.yml logs -f labfusion-runner-heavy
```

### Resource Monitoring

```bash
# Container resource usage
docker stats

# Volume usage
docker system df -v
```

## Troubleshooting

### Common Issues

1. **"Cannot find: node in PATH" or similar tool errors**
   - **This is handled** by the official `gitea/act_runner:nightly` image which includes necessary tools
   - **Environment variables** can be set in the configuration files to ensure proper tool paths
   - **Check your configuration files** to ensure proper environment setup

2. **"registration file not found, please register the runner first" error**
   - **This is handled** by the official `gitea/act_runner:nightly` image with proper configuration
   - **The `.runner` file** is created in the `/data` directory and persists between restarts
   - **Ensure configuration files** are properly mounted and environment variables are set
   - **Restart the runners** if you see this error:
     ```bash
     cd runners
     docker-compose -f docker-compose.runners.yml down
     docker-compose -f docker-compose.runners.yml up -d
     ```

3. **Configuration file not found errors**
   - **Ensure all config files exist**: `config_heavy.yaml`, `config_light.yaml`, `config_docker.yaml`, `config_security.yaml`
   - **Check file permissions** and ensure they're readable by the container
   - **Verify volume mounts** in the Docker Compose file are correct

4. **Runners not registering / Token appears empty**
   - **Check if `.env.runners` file exists** in the `runners/` directory
   - **Verify the file contains your actual token** (not the placeholder)
   - **Make sure you're using the management scripts** (they include `--env-file` parameter)
   - **If running Docker Compose directly**, add `--env-file .env.runners`:
     ```bash
     cd runners
     docker-compose -f docker-compose.runners.yml --env-file .env.runners up -d
     ```

5. **Runners not registering**
   - Check Gitea instance URL and port
   - Verify runner token is correct
   - Ensure Gitea Actions is enabled

6. **High resource usage**
   - Adjust CPU/memory limits in docker-compose.runners.yml
   - Check for stuck jobs in Gitea Actions

7. **Docker socket issues**
   - Ensure Docker socket is accessible: `/var/run/docker.sock`
   - Check Docker daemon is running

8. **Network connectivity**
   - Verify runners can reach Gitea instance
   - Check firewall settings

### Debug Mode

Enable debug logging by adding to `.env.runners`:

```bash
# Enable debug logging
ACTIONS_RUNNER_DEBUG=1
```

### Reset Runners

If runners become unresponsive:

```bash
# Stop and remove all runners
docker-compose -f docker-compose.runners.yml down -v

# Clean up volumes
docker volume prune -f

# Restart
docker-compose -f docker-compose.runners.yml up -d
```

## Security Considerations

- Runners run with Docker socket access for container builds
- Each runner has isolated data volumes
- Shared cache volume for build optimization
- Resource limits prevent resource exhaustion
- Health checks ensure runner availability

## Scaling

To add more runners:

1. Copy a runner service in `docker-compose.runners.yml`
2. Update the runner name and labels
3. Adjust resource allocation
4. Restart the compose stack

Example additional runner:

```yaml
gitea-runner-additional:
  image: gitea/act_runner:latest
  container_name: labfusion-runner-additional
  environment:
    - GITEA_INSTANCE_URL=${GITEA_INSTANCE_URL}
    - GITEA_RUNNER_REGISTRATION_TOKEN=${GITEA_RUNNER_TOKEN}
    - GITEA_RUNNER_NAME=labfusion-runner-additional
    - GITEA_RUNNER_LABELS=ubuntu-latest,self-hosted,additional
  volumes:
    - /var/run/docker.sock:/var/run/docker.sock
    - runner-data-additional:/data
    - shared-cache:/cache
  restart: unless-stopped
```

## Support

For issues with the runners setup:

1. Check the troubleshooting section above
2. Review runner logs
3. Verify Gitea Actions configuration
4. Check Docker and Docker Compose installation