Enhance runner configurations by adding Docker registry authentication to avoid rate limits; update example environment file with Docker Hub credentials; include new documentation for Docker rate limit solutions in structure.txt.
Some checks failed
LabFusion CI/CD Pipeline / api-gateway (push) Failing after 3s
LabFusion CI/CD Pipeline / api-docs (push) Failing after 3s
LabFusion CI/CD Pipeline / service-adapters (push) Failing after 2s
LabFusion CI/CD Pipeline / frontend (push) Failing after 2s
LabFusion CI/CD Pipeline / security-scan (push) Has been skipped
LabFusion CI/CD Pipeline / integration-tests (push) Has been skipped
Docker Build and Push / build-and-push (push) Failing after 3m33s
Docker Build and Push / security-scan (push) Has been skipped
Docker Build and Push / deploy-staging (push) Has been skipped
Docker Build and Push / deploy-production (push) Has been skipped
Integration Tests / integration-tests (push) Failing after 2s
Integration Tests / performance-tests (push) Has been skipped
Some checks failed
LabFusion CI/CD Pipeline / api-gateway (push) Failing after 3s
LabFusion CI/CD Pipeline / api-docs (push) Failing after 3s
LabFusion CI/CD Pipeline / service-adapters (push) Failing after 2s
LabFusion CI/CD Pipeline / frontend (push) Failing after 2s
LabFusion CI/CD Pipeline / security-scan (push) Has been skipped
LabFusion CI/CD Pipeline / integration-tests (push) Has been skipped
Docker Build and Push / build-and-push (push) Failing after 3m33s
Docker Build and Push / security-scan (push) Has been skipped
Docker Build and Push / deploy-staging (push) Has been skipped
Docker Build and Push / deploy-production (push) Has been skipped
Integration Tests / integration-tests (push) Failing after 2s
Integration Tests / performance-tests (push) Has been skipped
This commit is contained in:
201
docs/DOCKER_RATE_LIMIT_FIX.md
Normal file
201
docs/DOCKER_RATE_LIMIT_FIX.md
Normal file
@@ -0,0 +1,201 @@
|
||||
# Docker Hub Rate Limit Fix
|
||||
|
||||
## Problem
|
||||
```
|
||||
Error response from daemon: toomanyrequests: You have reached your unauthenticated pull rate limit. https://www.docker.com/increase-rate-limit
|
||||
```
|
||||
|
||||
Docker Hub has strict rate limits:
|
||||
- **Unauthenticated**: 100 pulls per 6 hours per IP
|
||||
- **Authenticated (free)**: 200 pulls per 6 hours per user
|
||||
- **Pro/Team**: Higher limits
|
||||
|
||||
## Solutions
|
||||
|
||||
### Solution 1: Use Docker Hub Authentication (Recommended)
|
||||
|
||||
#### 1.1. Create Docker Hub Account
|
||||
1. Go to [Docker Hub](https://hub.docker.com)
|
||||
2. Create a free account
|
||||
3. Note your username and password
|
||||
|
||||
#### 1.2. Update Runner Configurations
|
||||
|
||||
Add Docker authentication to each runner config:
|
||||
|
||||
**`runners/config_heavy.yaml`:**
|
||||
```yaml
|
||||
container:
|
||||
# Docker registry authentication
|
||||
docker_username: "your_dockerhub_username"
|
||||
docker_password: "your_dockerhub_password"
|
||||
```
|
||||
|
||||
**`runners/config_light.yaml`:**
|
||||
```yaml
|
||||
container:
|
||||
# Docker registry authentication
|
||||
docker_username: "your_dockerhub_username"
|
||||
docker_password: "your_dockerhub_password"
|
||||
```
|
||||
|
||||
**`runners/config_docker.yaml`:**
|
||||
```yaml
|
||||
container:
|
||||
# Docker registry authentication
|
||||
docker_username: "your_dockerhub_username"
|
||||
docker_password: "your_dockerhub_password"
|
||||
```
|
||||
|
||||
**`runners/config_security.yaml`:**
|
||||
```yaml
|
||||
container:
|
||||
# Docker registry authentication
|
||||
docker_username: "your_dockerhub_username"
|
||||
docker_password: "your_dockerhub_password"
|
||||
```
|
||||
|
||||
#### 1.3. Alternative: Use Environment Variables
|
||||
|
||||
Instead of hardcoding credentials, use environment variables:
|
||||
|
||||
**Update `runners/.env.runners`:**
|
||||
```bash
|
||||
# Docker Hub credentials
|
||||
DOCKER_USERNAME=your_dockerhub_username
|
||||
DOCKER_PASSWORD=your_dockerhub_password
|
||||
```
|
||||
|
||||
**Update config files:**
|
||||
```yaml
|
||||
container:
|
||||
docker_username: ${DOCKER_USERNAME}
|
||||
docker_password: ${DOCKER_PASSWORD}
|
||||
```
|
||||
|
||||
### Solution 2: Use Alternative Registries
|
||||
|
||||
#### 2.1. Use GitHub Container Registry (ghcr.io)
|
||||
|
||||
Update image references to use GitHub's registry:
|
||||
|
||||
**Heavy Runner:**
|
||||
```yaml
|
||||
labels:
|
||||
- "java:docker://ghcr.io/openjdk/openjdk:17-jdk-slim"
|
||||
- "python:docker://ghcr.io/library/python:3.11-slim"
|
||||
```
|
||||
|
||||
**Light Runner:**
|
||||
```yaml
|
||||
labels:
|
||||
- "nodejs:docker://ghcr.io/library/node:20-slim"
|
||||
- "frontend:docker://ghcr.io/library/node:20-slim"
|
||||
```
|
||||
|
||||
#### 2.2. Use Quay.io Registry
|
||||
|
||||
```yaml
|
||||
labels:
|
||||
- "java:docker://quay.io/eclipse/alpine_jdk17:latest"
|
||||
- "python:docker://quay.io/python/python:3.11-slim"
|
||||
- "nodejs:docker://quay.io/node/node:20-slim"
|
||||
```
|
||||
|
||||
### Solution 3: Use Local Image Caching
|
||||
|
||||
#### 3.1. Pre-pull Images on Runner Host
|
||||
|
||||
```bash
|
||||
# On your runner host machine
|
||||
docker pull openjdk:17-jdk-slim
|
||||
docker pull python:3.11-slim
|
||||
docker pull node:20-slim
|
||||
docker pull docker:24-dind
|
||||
docker pull alpine:3.19
|
||||
|
||||
# Tag as local images
|
||||
docker tag openjdk:17-jdk-slim localhost:5000/openjdk:17-jdk-slim
|
||||
docker tag python:3.11-slim localhost:5000/python:3.11-slim
|
||||
docker tag node:20-slim localhost:5000/node:20-slim
|
||||
docker tag docker:24-dind localhost:5000/docker:24-dind
|
||||
docker tag alpine:3.19 localhost:5000/alpine:3.19
|
||||
```
|
||||
|
||||
#### 3.2. Update Config to Use Local Images
|
||||
|
||||
```yaml
|
||||
labels:
|
||||
- "java:docker://localhost:5000/openjdk:17-jdk-slim"
|
||||
- "python:docker://localhost:5000/python:3.11-slim"
|
||||
- "nodejs:docker://localhost:5000/node:20-slim"
|
||||
```
|
||||
|
||||
### Solution 4: Reduce Image Pulls
|
||||
|
||||
#### 4.1. Disable Force Pull
|
||||
|
||||
Update all config files:
|
||||
```yaml
|
||||
container:
|
||||
# Don't pull if image already exists
|
||||
force_pull: false
|
||||
```
|
||||
|
||||
#### 4.2. Use Image Caching
|
||||
|
||||
```yaml
|
||||
container:
|
||||
# Enable image caching
|
||||
force_pull: false
|
||||
force_rebuild: false
|
||||
```
|
||||
|
||||
### Solution 5: Use Self-Hosted Registry
|
||||
|
||||
#### 5.1. Set up Local Registry
|
||||
|
||||
```bash
|
||||
# Run local Docker registry
|
||||
docker run -d -p 5000:5000 --name registry registry:2
|
||||
|
||||
# Mirror images to local registry
|
||||
docker pull openjdk:17-jdk-slim
|
||||
docker tag openjdk:17-jdk-slim localhost:5000/openjdk:17-jdk-slim
|
||||
docker push localhost:5000/openjdk:17-jdk-slim
|
||||
```
|
||||
|
||||
#### 5.2. Update Configs to Use Local Registry
|
||||
|
||||
```yaml
|
||||
labels:
|
||||
- "java:docker://localhost:5000/openjdk:17-jdk-slim"
|
||||
```
|
||||
|
||||
## Recommended Approach
|
||||
|
||||
**For immediate fix**: Use Solution 1 (Docker Hub authentication)
|
||||
**For long-term**: Combine Solutions 1 + 4 (auth + caching)
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
1. **Create Docker Hub account** (if you don't have one)
|
||||
2. **Update `.env.runners`** with credentials
|
||||
3. **Update all config files** with authentication
|
||||
4. **Set `force_pull: false`** to reduce pulls
|
||||
5. **Test with a simple job**
|
||||
|
||||
## Verification
|
||||
|
||||
After implementing, test with:
|
||||
```bash
|
||||
# Check if authentication works
|
||||
docker login
|
||||
docker pull openjdk:17-jdk-slim
|
||||
```
|
||||
|
||||
## References
|
||||
|
||||
- [Docker Hub Rate Limits](https://www.docker.com/increase-rate-limit)
|
||||
- [Gitea Actions Documentation](https://docs.gitea.com/usage/actions/design#act-runner)
|
||||
- [Docker Registry Authentication](https://docs.docker.com/engine/reference/commandline/login/)
|
||||
@@ -118,4 +118,5 @@ runners/
|
||||
├── RUNNERS.md # Gitea runners setup and management
|
||||
├── RUNNER_LABELS.md # Runner labels technical documentation
|
||||
├── OPTIMIZATION_RECOMMENDATIONS.md # CI/CD optimization recommendations
|
||||
├── DOCKER_RATE_LIMIT_FIX.md # Docker Hub rate limit solutions
|
||||
└── CI_CD.md # CI/CD pipeline documentation
|
||||
Reference in New Issue
Block a user