adebaumann/ESP-Home-Git-Synchroniser

Fork 0

Files

Adrian A. Baumann e5ba624aa9 File changer never ran - changed.

2026-01-13 16:03:04 +01:00

11 KiB

Raw Blame History

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

This is an ESPHome Gitea Sync Service - a Flask-based synchronization tool that bridges Gitea repositories with ESPHome device configurations. The service:

Clones and syncs a Gitea repository containing ESPHome configs
Watches for local file changes and can auto-push to Gitea
Receives webhooks from Gitea to pull changes
Provides manual sync endpoints

The ESPHome container remains immutable and handles all compilation/deployment operations. This service only manages git synchronization.

Architecture

Core Components

Flask REST API (app.py)
- Health check and device listing endpoints
- Gitea webhook endpoint (/webhook/gitea) for receiving push events
- Manual sync endpoints (/sync/pull, /sync/push)
- Thread-safe operation management using locks
File Watcher System (app.py)
- Uses watchdog library to monitor YAML file changes
- Implements debouncing (default 5 seconds) to prevent duplicate triggers
- Tracks last modification time per file in ESPHomeFileHandler.last_modified
- Monitors all .yaml/.yml files directly in the config directory
- Optionally triggers git push when AUTO_PUSH=true
- In production (Gunicorn), started via post_fork hook in first worker only
Git Operations (app.py)
- git_clone() - Initial clone of Gitea repository on startup
- git_pull() - Pull changes from Gitea (triggered by webhook or manual)
- git_push() - Push local changes to Gitea (auto or manual)
- initialize_git_repo() - Called on startup to clone or sync repo
- Authentication via Gitea token injected into remote URL
Synchronization Flow
- Gitea → ESPHome: Gitea webhook triggers /webhook/gitea → git pull → ESPHome sees updated files
- ESPHome → Gitea: File change detected → debounced → optionally git push (if AUTO_PUSH enabled)
- Compilation/upload handled entirely by ESPHome container

Expected Directory Structure

config/
├── device-name-1.yaml
├── device-name-2.yaml
├── device-name-3.yaml
└── .git/

The service expects a flat structure with each device having its own .yaml file directly in the config directory. The device name is the filename without the extension.

Thread Safety

operation_lock (Lock) protects the pending_operations dictionary
Prevents concurrent operations on the same device
Operations run in separate threads spawned from file watcher or webhook handlers

Project Files

app.py - Main Flask application with file watcher, git operations, and API endpoints
gunicorn_config.py - Gunicorn configuration with hooks for file watcher initialization
requirements.txt - Python dependencies
Dockerfile - Container image definition
docker-compose.yml - Multi-container orchestration (webhook + esphome services)
CLAUDE.md - This documentation file

Development Commands

Local Development (Without Docker)

# Install dependencies
pip install -r requirements.txt

# Set environment variables
export ESPHOME_CONFIG_DIR=/path/to/config
export DEBOUNCE_SECONDS=5
export GITEA_URL=https://gitea.example.com
export GITEA_REPO=username/esphome-configs
export GITEA_TOKEN=your_gitea_token
export GITEA_BRANCH=main
export AUTO_PUSH=false
export GIT_USER_NAME="ESPHome Sync Service"
export GIT_USER_EMAIL="esphome-sync@localhost"

# Run the service directly
python app.py

Docker Development

# Start services
docker-compose up -d

# View logs
docker-compose logs webhook
docker-compose logs esphome

# Stop services
docker-compose down

# Rebuild webhook service after code changes
docker-compose build webhook
docker-compose up -d webhook

Building Custom Image

docker build -t esphome-webhook:custom .

API Endpoints

GET /health - Health check with configuration info
GET /devices - List all devices (scans for .yaml/.yml files in config directory)
- Returns device name (filename without extension), config path, and last modified time
- Example: bedroom-light.yaml → device name: bedroom-light
POST /webhook/gitea - Gitea webhook endpoint (handles push events, triggers git pull)
POST /sync/pull - Manually trigger git pull from Gitea
POST /sync/push - Manually trigger git push to Gitea (optional JSON body: {"message": "commit message"})

Configuration

Environment variables (set in docker-compose.yml or locally):

Core Settings:

ESPHOME_CONFIG_DIR - Path to device configurations (default: /config)
DEBOUNCE_SECONDS - Delay before triggering after file change (default: 5)

Gitea Configuration:

GITEA_URL - URL of Gitea instance (e.g., https://gitea.example.com)
GITEA_REPO - Repository path (e.g., username/esphome-configs)
GITEA_TOKEN - Authentication token for Gitea API (generate in Gitea user settings)
GITEA_BRANCH - Git branch to use (default: main)
AUTO_PUSH - Auto-push local changes to Gitea (default: false)

Git User Configuration:

GIT_USER_NAME - Git commit author name (default: ESPHome Sync Service)
GIT_USER_EMAIL - Git commit author email (default: esphome-sync@localhost)

Setup Guide

Initial Setup

Create Gitea Repository
- Create a new repository in Gitea for your ESPHome configs
- Generate an access token in Gitea: Settings → Applications → Generate New Token
- Give token appropriate permissions (read/write repository)
Configure Environment Variables
- Update docker-compose.yml with your Gitea details:
  - GITEA_URL: Your Gitea instance URL
  - GITEA_REPO: Your repository path (username/repo)
  - GITEA_TOKEN: Your generated token
- Set AUTO_PUSH=true if you want local changes auto-pushed to Gitea
Start Services
```
docker-compose up -d
```
- On first startup, service will clone the Gitea repository to /config
- If /config already has files, manually initialize git first
Configure Gitea Webhook (Optional but recommended)
- Go to repository Settings → Webhooks → Add Webhook
- Set URL: http://your-server:5000/webhook/gitea
- Set Content Type: application/json
- Select events: Push events
- When you push to Gitea, service will automatically pull changes

Migration from Old Structure

If you have existing configs in the old device-name/main.yaml structure, you need to migrate to the flat structure:

# Move all device configs to the flat structure
cd config
for dir in */; do
  if [ -f "${dir}main.yaml" ]; then
    device_name="${dir%/}"
    mv "${dir}main.yaml" "${device_name}.yaml"
    rmdir "${dir}" 2>/dev/null || echo "Directory ${dir} not empty, manual cleanup needed"
  fi
done

# Commit the changes
git add -A
git commit -m "Migrate to flat YAML structure"
git push origin main

After migration, your structure should be:

config/
├── device-1.yaml  (was device-1/main.yaml)
├── device-2.yaml  (was device-2/main.yaml)
└── .git/

Workflows

Editing in ESPHome Dashboard:

Edit YAML files in ESPHome dashboard
If AUTO_PUSH=true, changes automatically pushed to Gitea
If AUTO_PUSH=false, manually push: curl -X POST http://localhost:5000/sync/push

Editing in Gitea (or git client):

Commit and push changes to Gitea repository
Gitea webhook triggers /webhook/gitea
Service runs git pull
ESPHome sees updated files

Manual Sync:

Pull from Gitea: curl -X POST http://localhost:5000/sync/pull
Push to Gitea: curl -X POST http://localhost:5000/sync/push -H "Content-Type: application/json" -d '{"message": "My commit message"}'

Key Implementation Details

Device Config Resolution

The find_device_config() function looks for device configs at:

{ESPHOME_CONFIG_DIR}/{device_name}.yaml

If the device name doesn't include an extension, .yaml is automatically appended.

File Watching Behavior

The file watcher monitors all .yaml and .yml files directly in the config directory:

Watched: Any *.yaml or *.yml file in the root of the config directory
Ignored: Subdirectories, non-YAML files, and hidden files (like .git)
Debouncing: Changes are debounced for DEBOUNCE_SECONDS (default 5) to prevent duplicate triggers
Auto-push: If AUTO_PUSH=true, any watched file change triggers a git commit and push
Commit message format: Auto-sync: {device_name}.yaml changed

Example watched files in /config:

bedroom-light.yaml ✓
kitchen-sensor.yaml ✓
garage-door.yml ✓
README.md ✗ (not YAML)
.gitignore ✗ (hidden file)
backup/old-config.yaml ✗ (in subdirectory)

Git Authentication

Authentication is handled by injecting the Gitea token into the remote URL:

Input: GITEA_URL=https://gitea.com, GITEA_TOKEN=abc123, GITEA_REPO=user/repo
Remote URL: https://abc123@gitea.com/user/repo.git
Token is only stored in environment variables, never committed to git

Operation Flow

Startup: initialize_git_repo() → Clone if not exists OR Pull latest changes
File Change Detected → Debounced → Check if YAML file in config directory → Optionally git push (threaded)
Gitea Webhook → Parse payload → git pull → ESPHome sees updated files
Thread Safety → Operations use operation_lock to prevent concurrent access to pending_operations dict

Git Repository Initialization

On startup, the service:

Checks if /config/.git exists
If not, clones the repository to a temp directory, then moves contents to /config
If yes, configures git user and pulls latest changes
All git operations have 60-second timeout

Gunicorn Configuration

Configuration File: gunicorn_config.py

2 workers
600 second timeout
Binds to 0.0.0.0:5000

Hooks for File Watcher:

on_starting hook: Initializes git repository once before workers are forked
post_fork hook: Starts file watcher thread in the first worker only (worker.age == 0)
This ensures the file watcher runs correctly under Gunicorn without duplicates

Why hooks are needed: When running under Gunicorn, the if __name__ == '__main__': block in app.py is never executed. The hooks ensure that:

Git initialization happens once at startup
File watcher starts in exactly one worker process
Multiple workers don't create duplicate file watchers

Docker Compose Architecture

Two services work together:

esphome - Official ESPHome dashboard (host network mode for mDNS/OTA)
- Dashboard: http://localhost:6052
- Handles compilation and device deployment
- Shares ./config volume with sync service (read-only)
- Uses persistent esphome-cache volume
- Immutable - never modified, only reads configs
webhook (esphome-gitea-sync) - This service (bridged network)
- API: http://localhost:5000
- Synchronizes configurations between Gitea and ESPHome
- Read-write mount of ./config for git operations
- Has git installed and manages the git repository
- ESPHome container handles all compilation/upload operations

Volume Architecture

./config is shared between both containers
Sync service has read-write access (manages git repo)
ESPHome has read access (consumes configs)
On first startup, if ./config is empty, sync service clones Gitea repo

11 KiB Raw Blame History