initial project setup

2025-09-11 22:08:12 +02:00
parent 8cc588dc92
commit 21e4972ab1
46 changed files with 2755 additions and 1 deletions
--- a/docs/progress.md
+++ b/docs/progress.md
@@ -0,0 +1,161 @@
+# LabFusion Development Progress
+
+## Project Overview
+LabFusion is a unified dashboard and integration hub for homelab services, built with a polyglot microservices architecture.
+
+## Architecture Decisions
+- **Modular Services**: Each service is independently deployable
+- **Polyglot**: Services can use different languages (Java, Python, Go, Node.js)
+- **Message Bus**: Redis for inter-service communication
+- **Database**: PostgreSQL for persistent data
+- **Containerization**: Docker for all services
+
+## Completed Tasks ✅
+
+### Phase 1: Project Structure & Foundation
+- [x] **Project Structure Setup** (2024-11-09)
+  - Created modular `services/` directory structure
+  - Moved from `backend/` to `services/` for better modularity
+  - Each service has its own directory with independent Docker setup
+
+- [x] **API Gateway Service** (2024-11-09)
+  - Java Spring Boot application
+  - JPA entities for User, Dashboard, Widget, Event, DeviceState
+  - REST controllers for dashboard management
+  - PostgreSQL integration
+  - Redis message bus support
+  - JWT authentication framework
+
+- [x] **Service Adapters** (2024-11-09)
+  - Python FastAPI application
+  - Integration endpoints for Home Assistant, Frigate, Immich, n8n
+  - Redis event publishing
+  - Configurable service connections
+  - Mock data for development
+
+- [x] **Frontend Dashboard** (2024-11-09)
+  - React application with Ant Design
+  - System metrics visualization
+  - Service status monitoring
+  - Settings configuration
+  - Responsive design
+
+- [x] **Infrastructure Setup** (2024-11-09)
+  - Docker Compose for production and development
+  - PostgreSQL database configuration
+  - Redis message bus setup
+  - Environment configuration templates
+
+- [x] **Documentation** (2024-11-09)
+  - Comprehensive README with setup instructions
+  - Service-specific README files
+  - Progress tracking document
+  - Updated project structure documentation
+
+## Current Status 🚧
+
+### Services Directory Structure
+```
+services/
+├── api-gateway/          # Java Spring Boot (Port 8080) ✅
+├── service-adapters/     # Python FastAPI (Port 8000) ✅
+├── metrics-collector/    # Go service (Port 8081) 🚧
+└── notification-service/ # Node.js service (Port 8082) 🚧
+```
+
+### Infrastructure
+- **Database**: PostgreSQL (Port 5432) ✅
+- **Message Bus**: Redis (Port 6379) ✅
+- **Frontend**: React (Port 3000) ✅
+- **Containerization**: Docker Compose ✅
+
+## Next Steps 🎯
+
+### Phase 2: Service Implementation
+- [ ] **Metrics Collector Service** (Go)
+  - Implement Docker API integration
+  - Add Prometheus metrics collection
+  - Create Redis publishing mechanism
+  - Add configuration management
+
+- [ ] **Notification Service** (Node.js/TypeScript)
+  - Implement notification channels (email, webhook, push)
+  - Create alert rule engine
+  - Add notification preferences
+  - Integrate with Redis for event processing
+
+### Phase 3: Real Service Integrations
+- [ ] **Home Assistant Integration**
+  - Implement actual HA API calls
+  - Add WebSocket support for real-time updates
+  - Create entity state synchronization
+
+- [ ] **Frigate Integration**
+  - Implement Frigate API integration
+  - Add event processing and filtering
+  - Create detection timeline visualization
+
+- [ ] **Immich Integration**
+  - Implement Immich API calls
+  - Add photo metadata processing
+  - Create asset management features
+
+### Phase 4: Advanced Features
+- [ ] **Real-time Updates**
+  - WebSocket implementation for live data
+  - Event streaming from Redis
+  - Frontend real-time UI updates
+
+- [ ] **Data Correlation Engine**
+  - Cross-service event correlation
+  - Timeline analysis
+  - Pattern detection
+
+- [ ] **Authentication & Security**
+  - JWT token implementation
+  - User management
+  - Role-based access control
+
+### Phase 5: Production Readiness
+- [ ] **Monitoring & Logging**
+  - Centralized logging
+  - Health checks
+  - Performance monitoring
+
+- [ ] **CI/CD Pipeline**
+  - GitHub Actions setup
+  - Automated testing
+  - Deployment automation
+
+- [ ] **Kubernetes Migration**
+  - Helm charts
+  - Kubernetes manifests
+  - Production deployment
+
+## Development Notes
+
+### Service Independence
+Each service is designed to be:
+- Independently deployable
+- Language-agnostic
+- Self-contained with its own dependencies
+- Communicating via Redis message bus
+
+### Future Service Additions
+The modular structure allows for easy addition of new services:
+- **Data Processing Service** (Rust) - For heavy data processing
+- **ML Service** (Python) - For machine learning features
+- **API Gateway v2** (Go) - For high-performance routing
+- **Cache Service** (C++) - For high-performance caching
+
+## Technical Debt
+- [ ] Add comprehensive error handling
+- [ ] Implement proper logging across all services
+- [ ] Add unit and integration tests
+- [ ] Create API documentation with OpenAPI/Swagger
+- [ ] Add health check endpoints for all services
+
+## Resources
+- [Project Specifications](specs.md)
+- [Project Structure](structure.txt)
+- [Main README](../README.md)
--- a/docs/specs.md
+++ b/docs/specs.md
@@ -0,0 +1,126 @@
+# Project Specification: **LabFusion**
+
+## 1. Purpose & Goals
+
+* Provide a **unified application** that integrates data and metrics from multiple homelab services (Home Assistant, Frigate, Immich, n8n, etc.).
+* Enable creation of **dashboards** that visualize system and service performance (CPU, RAM, storage, uptime, etc.).
+* Support **data mashups** (cross-service insights, e.g., "Frigate detected a person → check HA presence → enrich with Immich images").
+* Serve as a **learning playground** for:
+
+  * Running **polyglot microservices** (Python, Java/Spring, modern frontend).
+  * Hosting everything with **Docker Compose** (extendable to Kubernetes later).
+  * Building **APIs and integrations** across services.
+
+---
+
+## 2. User Stories
+
+### Core / First Iteration
+
+* As a homelab owner, I want to see a **system dashboard** that shows CPU, RAM, and storage usage of all devices (Docker host, NAS, etc.).
+* As a homelab owner, I want to see the **status of each service** (running, uptime, logs, alerts).
+* As a homelab owner, I want to have a **dashboard builder** (predefined widgets like charts, tables, logs).
+
+### Advanced / Later Iterations
+
+* As a homelab owner, I want to **combine service data** (e.g., Frigate face detection + HA presence info).
+* As a homelab owner, I want **notifications** (e.g., service down, high CPU, suspicious camera activity).
+* As a homelab owner, I want the app to expose an **API layer** that other tools (like n8n) can consume.
+
+---
+
+## 3. Functional Requirements
+
+1. **System Monitoring**
+
+   * Collect metrics from Docker hosts & containers (CPU, memory, disk, network).
+   * Display service health (via APIs, logs, or container status).
+
+2. **Service Integrations**
+
+   * **Home Assistant**: read entity states via its REST/WebSocket API.
+   * **Frigate**: consume event data (detections, timelines).
+   * **Immich**: access image metadata (faces, tags, time).
+   * **n8n**: optional integration (trigger workflows from events).
+
+3. **Dashboards**
+
+   * Modular widget-based UI (charts, tables, cards).
+   * Configurable dashboards (JSON schema for layout).
+   * Examples:
+
+     * Host performance overview.
+     * Container uptime.
+     * HA device states.
+     * Frigate detections timeline.
+
+4. **Data Correlation (Phase 2)**
+
+   * Join data across services.
+   * Example: *Frigate detected “Alice” at 18:42 → HA shows Alice’s phone connected → Confirm Alice was home.*
+
+---
+
+## 4. Non-Functional Requirements
+
+* **Performance**: Handle \~1000 events/day without lag.
+* **Scalability**: Add more services without rewriting core.
+* **Security**: Local auth (JWT/OAuth2), role-based access (later).
+* **Resilience**: Auto-restart containers, graceful error handling.
+* **Extensibility**: Plugin-like integration architecture.
+
+---
+
+## 5. System Architecture
+
+* **Frontend (React, Vue, or Svelte)**
+
+  * Dashboard UI (charts, cards, widgets).
+  * Talks to backend via REST/WebSocket API.
+
+* **Backend (polyglot)**
+
+  * **Java/Spring Boot**: Core API gateway, auth, user management.
+  * **Python FastAPI**: Service integration adapters (HA, Frigate, Immich).
+  * **Shared Message Bus** (e.g., Redis, Kafka, or RabbitMQ): decouple services.
+  * **Database**: PostgreSQL (persistent config, dashboards, event history).
+
+* **Dockerized Deployment**
+
+  * Each component in its own container.
+  * Docker Compose orchestrates services.
+  * Future upgrade path: Kubernetes.
+
+---
+
+## 6. Data Model (simplified)
+
+* **User** (id, username, roles).
+* **Dashboard** (id, name, layout, widgets).
+* **Widget** (id, type, config, service binding).
+* **Event** (timestamp, service, type, metadata).
+* **Device/Service State** (timestamp, entity\_id, value).
+
+---
+
+## 7. Deployment & Operations
+
+* **Docker Compose stack** (frontend, Spring API, FastAPI adapters, DB, Redis).
+* CI/CD pipeline (GitHub Actions).
+* Monitoring (Prometheus + Grafana, optional since you’re building dashboards).
+* Config via `.env` or centralized config service.
+
+---
+
+## Roadmap
+
+* **Iteration 1**: Build basic dashboards with host & container metrics.
+* **Iteration 2**: Add service integrations (HA, Frigate).
+* **Iteration 3**: Implement cross-service correlations (the "Alice detection" scenario).
+* **Iteration 4**: Add notifications & automations.
+
+---
+
+## Project Title
+
+**LabFusion** – a unified dashboard and integration hub for your homelab.
--- a/docs/structure.txt
+++ b/docs/structure.txt
@@ -0,0 +1,55 @@
+labfusion/
+├── docker-compose.yml          # Production Docker setup
+├── docker-compose.dev.yml      # Development Docker setup
+├── env.example                 # Environment configuration template
+├── .gitignore                  # Git ignore rules
+├── README.md                   # Comprehensive documentation
+├── services/                   # Modular microservices
+│   ├── api-gateway/           # Java Spring Boot API Gateway (Port 8080)
+│   │   ├── src/main/java/com/labfusion/
+│   │   │   ├── model/         # JPA entities (User, Dashboard, Widget, Event, DeviceState)
+│   │   │   ├── repository/    # Data repositories
+│   │   │   ├── service/       # Business logic
+│   │   │   └── controller/    # REST controllers
+│   │   ├── src/main/resources/
+│   │   │   └── application.yml # Spring configuration
+│   │   ├── pom.xml            # Maven dependencies
+│   │   ├── Dockerfile         # Production container
+│   │   ├── Dockerfile.dev     # Development container
+│   │   └── README.md          # Service documentation
+│   ├── service-adapters/      # Python FastAPI Service Adapters (Port 8000)
+│   │   ├── main.py            # FastAPI application
+│   │   ├── requirements.txt   # Python dependencies
+│   │   ├── Dockerfile         # Production container
+│   │   ├── Dockerfile.dev     # Development container
+│   │   └── README.md          # Service documentation
+│   ├── metrics-collector/     # Go Metrics Collector (Port 8081) 🚧
+│   │   ├── main.go            # Go application (planned)
+│   │   ├── go.mod             # Go dependencies (planned)
+│   │   ├── Dockerfile         # Production container (planned)
+│   │   ├── Dockerfile.dev     # Development container (planned)
+│   │   └── README.md          # Service documentation
+│   └── notification-service/  # Node.js Notification Service (Port 8082) 🚧
+│       ├── src/               # TypeScript source (planned)
+│       ├── package.json       # Node.js dependencies (planned)
+│       ├── Dockerfile         # Production container (planned)
+│       ├── Dockerfile.dev     # Development container (planned)
+│       └── README.md          # Service documentation
+├── frontend/                  # React Frontend (Port 3000)
+│   ├── src/
+│   │   ├── components/        # React components
+│   │   │   ├── Dashboard.js   # Main dashboard
+│   │   │   ├── SystemMetrics.js # Metrics visualization
+│   │   │   └── Settings.js    # Configuration UI
+│   │   ├── App.js             # Main app component
+│   │   ├── index.js           # App entry point
+│   │   └── index.css          # Global styles
+│   ├── public/
+│   │   └── index.html         # HTML template
+│   ├── package.json           # Node.js dependencies
+│   ├── Dockerfile             # Production container
+│   └── Dockerfile.dev         # Development container
+└── docs/                      # Documentation
+    ├── specs.md              # Project specifications
+    ├── structure.txt         # Project structure
+    └── progress.md           # Development progress tracking