adebaumann/ESP-Home-Git-Synchroniser
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

initial commit

Browse Source
This commit is contained in:
Adrian A. Baumann
2026-01-13 15:45:44 +01:00
commit 818576c03a
6 changed files with 1049 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

291
CLAUDE.md Normal file
View File

@@ -0,0 +1,291 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
This is an ESPHome Gitea Sync Service - a Flask-based synchronization tool that bridges Gitea repositories with ESPHome device configurations. The service:
- Clones and syncs a Gitea repository containing ESPHome configs
- Watches for local file changes and can auto-push to Gitea
- Receives webhooks from Gitea to pull changes
- Provides manual sync endpoints
The ESPHome container remains immutable and handles all compilation/deployment operations. This service only manages git synchronization.
## Architecture
### Core Components
1. **Flask REST API** (app.py)
- Health check and device listing endpoints
- Gitea webhook endpoint (`/webhook/gitea`) for receiving push events
- Manual sync endpoints (`/sync/pull`, `/sync/push`)
- Thread-safe operation management using locks
2. **File Watcher System** (app.py)
- Uses watchdog library to monitor YAML file changes
- Implements debouncing (default 5 seconds) to prevent duplicate triggers
- Tracks last modification time per file in `ESPHomeFileHandler.last_modified`
- Monitors all `.yaml`/`.yml` files directly in the config directory
- Optionally triggers `git push` when `AUTO_PUSH=true`
3. **Git Operations** (app.py)
- `git_clone()` - Initial clone of Gitea repository on startup
- `git_pull()` - Pull changes from Gitea (triggered by webhook or manual)
- `git_push()` - Push local changes to Gitea (auto or manual)
- `initialize_git_repo()` - Called on startup to clone or sync repo
- Authentication via Gitea token injected into remote URL
4. **Synchronization Flow**
- **Gitea → ESPHome**: Gitea webhook triggers `/webhook/gitea` → `git pull` → ESPHome sees updated files
- **ESPHome → Gitea**: File change detected → debounced → optionally `git push` (if AUTO_PUSH enabled)
- Compilation/upload handled entirely by ESPHome container
### Expected Directory Structure
```
config/
├── device-name-1.yaml
├── device-name-2.yaml
├── device-name-3.yaml
└── .git/
```
The service expects a flat structure with each device having its own `.yaml` file directly in the config directory. The device name is the filename without the extension.
### Thread Safety
- `operation_lock` (Lock) protects the `pending_operations` dictionary
- Prevents concurrent operations on the same device
- Operations run in separate threads spawned from file watcher or webhook handlers
## Development Commands
### Local Development (Without Docker)
```bash
# Install dependencies
pip install -r requirements.txt
# Set environment variables
export ESPHOME_CONFIG_DIR=/path/to/config
export DEBOUNCE_SECONDS=5
export GITEA_URL=https://gitea.example.com
export GITEA_REPO=username/esphome-configs
export GITEA_TOKEN=your_gitea_token
export GITEA_BRANCH=main
export AUTO_PUSH=false
export GIT_USER_NAME="ESPHome Sync Service"
export GIT_USER_EMAIL="esphome-sync@localhost"
# Run the service directly
python app.py
```
### Docker Development
```bash
# Start services
docker-compose up -d
# View logs
docker-compose logs webhook
docker-compose logs esphome
# Stop services
docker-compose down
# Rebuild webhook service after code changes
docker-compose build webhook
docker-compose up -d webhook
```
### Building Custom Image
```bash
docker build -t esphome-webhook:custom .
```
## API Endpoints
- `GET /health` - Health check with configuration info
- `GET /devices` - List all devices (scans for `.yaml`/`.yml` files in config directory)
- Returns device name (filename without extension), config path, and last modified time
- Example: `bedroom-light.yaml` → device name: `bedroom-light`
- `POST /webhook/gitea` - Gitea webhook endpoint (handles push events, triggers git pull)
- `POST /sync/pull` - Manually trigger git pull from Gitea
- `POST /sync/push` - Manually trigger git push to Gitea (optional JSON body: `{"message": "commit message"}`)
## Configuration
Environment variables (set in docker-compose.yml or locally):
**Core Settings:**
- `ESPHOME_CONFIG_DIR` - Path to device configurations (default: `/config`)
- `DEBOUNCE_SECONDS` - Delay before triggering after file change (default: `5`)
**Gitea Configuration:**
- `GITEA_URL` - URL of Gitea instance (e.g., `https://gitea.example.com`)
- `GITEA_REPO` - Repository path (e.g., `username/esphome-configs`)
- `GITEA_TOKEN` - Authentication token for Gitea API (generate in Gitea user settings)
- `GITEA_BRANCH` - Git branch to use (default: `main`)
- `AUTO_PUSH` - Auto-push local changes to Gitea (default: `false`)
**Git User Configuration:**
- `GIT_USER_NAME` - Git commit author name (default: `ESPHome Sync Service`)
- `GIT_USER_EMAIL` - Git commit author email (default: `esphome-sync@localhost`)
## Setup Guide
### Initial Setup
1. **Create Gitea Repository**
- Create a new repository in Gitea for your ESPHome configs
- Generate an access token in Gitea: Settings → Applications → Generate New Token
- Give token appropriate permissions (read/write repository)
2. **Configure Environment Variables**
- Update `docker-compose.yml` with your Gitea details:
- `GITEA_URL`: Your Gitea instance URL
- `GITEA_REPO`: Your repository path (username/repo)
- `GITEA_TOKEN`: Your generated token
- Set `AUTO_PUSH=true` if you want local changes auto-pushed to Gitea
3. **Start Services**
```bash
docker-compose up -d
```
- On first startup, service will clone the Gitea repository to `/config`
- If `/config` already has files, manually initialize git first
4. **Configure Gitea Webhook** (Optional but recommended)
- Go to repository Settings → Webhooks → Add Webhook
- Set URL: `http://your-server:5000/webhook/gitea`
- Set Content Type: `application/json`
- Select events: `Push` events
- When you push to Gitea, service will automatically pull changes
### Migration from Old Structure
If you have existing configs in the old `device-name/main.yaml` structure, you need to migrate to the flat structure:
```bash
# Move all device configs to the flat structure
cd config
for dir in */; do
if [ -f "${dir}main.yaml" ]; then
device_name="${dir%/}"
mv "${dir}main.yaml" "${device_name}.yaml"
rmdir "${dir}" 2>/dev/null || echo "Directory ${dir} not empty, manual cleanup needed"
fi
done
# Commit the changes
git add -A
git commit -m "Migrate to flat YAML structure"
git push origin main
```
After migration, your structure should be:
```
config/
├── device-1.yaml (was device-1/main.yaml)
├── device-2.yaml (was device-2/main.yaml)
└── .git/
```
### Workflows
**Editing in ESPHome Dashboard:**
1. Edit YAML files in ESPHome dashboard
2. If `AUTO_PUSH=true`, changes automatically pushed to Gitea
3. If `AUTO_PUSH=false`, manually push: `curl -X POST http://localhost:5000/sync/push`
**Editing in Gitea (or git client):**
1. Commit and push changes to Gitea repository
2. Gitea webhook triggers `/webhook/gitea`
3. Service runs `git pull`
4. ESPHome sees updated files
**Manual Sync:**
- Pull from Gitea: `curl -X POST http://localhost:5000/sync/pull`
- Push to Gitea: `curl -X POST http://localhost:5000/sync/push -H "Content-Type: application/json" -d '{"message": "My commit message"}'`
## Key Implementation Details
### Device Config Resolution
The `find_device_config()` function looks for device configs at:
- `{ESPHOME_CONFIG_DIR}/{device_name}.yaml`
If the device name doesn't include an extension, `.yaml` is automatically appended.
### File Watching Behavior
The file watcher monitors all `.yaml` and `.yml` files directly in the config directory:
- **Watched**: Any `*.yaml` or `*.yml` file in the root of the config directory
- **Ignored**: Subdirectories, non-YAML files, and hidden files (like `.git`)
- **Debouncing**: Changes are debounced for `DEBOUNCE_SECONDS` (default 5) to prevent duplicate triggers
- **Auto-push**: If `AUTO_PUSH=true`, any watched file change triggers a git commit and push
- **Commit message format**: `Auto-sync: {device_name}.yaml changed`
Example watched files in `/config`:
- `bedroom-light.yaml` ✓
- `kitchen-sensor.yaml` ✓
- `garage-door.yml` ✓
- `README.md` ✗ (not YAML)
- `.gitignore` ✗ (hidden file)
- `backup/old-config.yaml` ✗ (in subdirectory)
### Git Authentication
Authentication is handled by injecting the Gitea token into the remote URL:
- Input: `GITEA_URL=https://gitea.com`, `GITEA_TOKEN=abc123`, `GITEA_REPO=user/repo`
- Remote URL: `https://abc123@gitea.com/user/repo.git`
- Token is only stored in environment variables, never committed to git
### Operation Flow
1. **Startup**: `initialize_git_repo()` → Clone if not exists OR Pull latest changes
2. **File Change Detected** → Debounced → Check if YAML file in config directory → Optionally `git push` (threaded)
3. **Gitea Webhook** → Parse payload → `git pull` → ESPHome sees updated files
4. **Thread Safety** → Operations use `operation_lock` to prevent concurrent access to `pending_operations` dict
### Git Repository Initialization
On startup, the service:
1. Checks if `/config/.git` exists
2. If not, clones the repository to a temp directory, then moves contents to `/config`
3. If yes, configures git user and pulls latest changes
4. All git operations have 60-second timeout
### Gunicorn Configuration (Dockerfile)
- 2 workers
- 600 second timeout
- Binds to 0.0.0.0:5000
## Docker Compose Architecture
Two services work together:
1. **esphome** - Official ESPHome dashboard (host network mode for mDNS/OTA)
- Dashboard: http://localhost:6052
- Handles compilation and device deployment
- Shares `./config` volume with sync service (read-only)
- Uses persistent `esphome-cache` volume
- **Immutable** - never modified, only reads configs
2. **webhook** (esphome-gitea-sync) - This service (bridged network)
- API: http://localhost:5000
- Synchronizes configurations between Gitea and ESPHome
- Read-write mount of `./config` for git operations
- Has git installed and manages the git repository
- ESPHome container handles all compilation/upload operations
### Volume Architecture
- `./config` is shared between both containers
- Sync service has read-write access (manages git repo)
- ESPHome has read access (consumes configs)
- On first startup, if `./config` is empty, sync service clones Gitea repo