ESP-Home-Git-Synchroniser/README.md

# ESPHome Gitea Sync Service

A Docker-based synchronization service that bridges Gitea repositories with ESPHome device configurations. Automatically syncs YAML configuration files between your Gitea git repository and your ESPHome installation.

## Features

- **Bidirectional Git Sync**: Sync configurations between Gitea and ESPHome
- **File Watching**: Automatically detect YAML changes and push to Gitea
- **Gitea Webhooks**: Receive push events and pull changes automatically
- **REST API**: Manual sync endpoints and device listing
- **Docker Polling**: Uses filesystem polling for reliable change detection on Docker bind mounts
- **Thread-Safe**: Concurrent operation handling with locks
- **Archive Support**: Detects when files are moved to archive folders

## Architecture

This service works alongside the official ESPHome container:
- **ESPHome Container**: Handles all compilation and device deployment (immutable)
- **Sync Service**: Manages git synchronization between Gitea and ESPHome (this service)

The ESPHome container remains unchanged and simply consumes the YAML configs. This service only handles git operations.

## Quick Start

### 1. Configure Environment Variables

Edit `docker-compose.yml` and set your Gitea details:

```yaml
environment:
  - GITEA_URL=https://your-gitea-instance.com
  - GITEA_REPO=username/esphome-configs
  - GITEA_TOKEN=your_gitea_access_token
  - GITEA_BRANCH=main
  - AUTO_PUSH=false  # Set to true for automatic push on file changes
```

### 2. Start the Services

```bash
docker-compose up -d
```

This starts two services:
- **ESPHome Dashboard**: `http://localhost:6052` (compilation and deployment)
- **Gitea Sync Service**: `http://localhost:5000` (git synchronization)

### 3. Access the Services

- ESPHome Dashboard: http://localhost:6052
- Sync API Health: http://localhost:5000/health
- Device List: http://localhost:5000/devices

### 4. Configure Gitea Webhook (Optional)

To automatically pull changes when you push to Gitea:

1. Go to your Gitea repository → Settings → Webhooks
2. Add Webhook → Gitea
3. Set Target URL: `http://your-server:5000/webhook/gitea`
4. Content Type: `application/json`
5. Trigger On: `Push Events`
6. Click "Add Webhook"

## Environment Variables

Configure the sync service in `docker-compose.yml`:

| Variable | Default | Description |
|----------|---------|-------------|
| `ESPHOME_CONFIG_DIR` | `/config` | Directory containing device configs |
| `DEBOUNCE_SECONDS` | `5` | Delay before triggering after file change |
| `USE_POLLING` | `true` | Use polling for Docker compatibility (required) |
| `POLLING_INTERVAL` | `1.0` | Seconds between filesystem polls |
| `GITEA_URL` | - | URL of your Gitea instance |
| `GITEA_REPO` | - | Repository path (username/repo) |
| `GITEA_TOKEN` | - | Gitea access token (generate in user settings) |
| `GITEA_BRANCH` | `main` | Git branch to sync |
| `AUTO_PUSH` | `false` | Automatically push local changes to Gitea |
| `GIT_USER_NAME` | `ESPHome Sync Service` | Git commit author name |
| `GIT_USER_EMAIL` | `esphome-sync@localhost` | Git commit author email |

## API Endpoints

### Health Check

```bash
curl http://localhost:5000/health
```

### List Devices

Lists all YAML files in the config directory:

```bash
curl http://localhost:5000/devices
```

Returns:
```json
{
  "devices": [
    {
      "name": "bedroom-light",
      "config_path": "/config/bedroom-light.yaml",
      "last_modified": "2026-01-13T10:30:00"
    }
  ]
}
```

### Gitea Webhook

Receives push events from Gitea and pulls changes:

```bash
curl -X POST http://localhost:5000/webhook/gitea \
  -H "Content-Type: application/json" \
  -H "X-Gitea-Event: push" \
  -d '{"repository": {"full_name": "user/repo"}}'
```

### Manual Pull from Gitea

Manually trigger a git pull:

```bash
curl -X POST http://localhost:5000/sync/pull
```

### Manual Push to Gitea

Manually trigger a git push:

```bash
curl -X POST http://localhost:5000/sync/push \
  -H "Content-Type: application/json" \
  -d '{"message": "Manual sync from API"}'
```

## File Watching

The service monitors all `.yaml` and `.yml` files in the root of the config directory.

### Detected Events

1. **File Modified** - Edit existing YAML file
2. **File Created** - Add new YAML file
3. **File Deleted** - Remove YAML file
4. **File Renamed** - Rename YAML file
5. **File Archived** - Move YAML file to subdirectory (e.g., `archive/`)
6. **File Restored** - Move YAML file from subdirectory back to root

When `AUTO_PUSH=true`, all these events automatically trigger a git commit and push to Gitea.

### Commit Messages

- Modified/Created: `Auto-sync: device-name.yaml changed`
- Deleted: `Auto-sync: device-name.yaml deleted`
- Renamed: `Auto-sync: Renamed old-name.yaml to new-name.yaml`
- Archived: `Auto-sync: device-name.yaml archived`
- Restored: `Auto-sync: device-name.yaml restored from archive`

## Directory Structure

```
config/
├── bedroom-light.yaml       # Device configs (flat structure)
├── kitchen-sensor.yaml
├── garage-door.yaml
├── .gitignore              # Optional: ignore archive folder
├── archive/                # Optional: archived configs (not monitored)
│   └── old-device.yaml
└── .git/                   # Git repository (managed by sync service)
```

## Workflows

### Editing in ESPHome Dashboard

1. Edit YAML files in ESPHome dashboard
2. File watcher detects the change
3. If `AUTO_PUSH=true`: Changes automatically committed and pushed to Gitea
4. If `AUTO_PUSH=false`: Manually trigger push via API

### Editing in Gitea (or git client)

1. Commit and push changes to Gitea repository
2. Gitea webhook triggers `/webhook/gitea` endpoint
3. Service runs `git pull` to fetch changes
4. ESPHome sees updated files and reloads dashboard

### Manual Sync

```bash
# Pull latest changes from Gitea
curl -X POST http://localhost:5000/sync/pull

# Push local changes to Gitea
curl -X POST http://localhost:5000/sync/push \
  -H "Content-Type: application/json" \
  -d '{"message": "My commit message"}'
```

## Troubleshooting

### Service Won't Start

Check logs:
```bash
docker-compose logs webhook
docker-compose logs esphome
```

### File Watcher Not Detecting Changes

**Cause**: inotify events don't propagate through Docker bind mounts on WSL2, Unraid, and macOS.

**Solution**: The service uses polling mode by default (`USE_POLLING=true`). Ensure this is enabled in docker-compose.yml.

Check logs for:
```
Using PollingObserver (interval: 1.0s) for Docker bind mount compatibility
```

### Git Operations Failing

1. Verify Gitea token has correct permissions (read/write repository)
2. Check `GITEA_URL`, `GITEA_REPO`, and `GITEA_TOKEN` are set correctly
3. Check logs for git command errors:
   ```bash
   docker-compose logs webhook | grep -i "git command"
   ```

### Webhook Not Receiving Events

1. Verify webhook URL is accessible from Gitea (firewall, network)
2. Check Gitea webhook delivery logs (Repository → Settings → Webhooks → Recent Deliveries)
3. Ensure `Content-Type` is `application/json` and trigger is `Push Events`

### Changes Not Syncing to Gitea

1. Check if `AUTO_PUSH=true` in docker-compose.yml
2. Verify file is in the root of `/config/` (not in subdirectory)
3. Check logs for file change detection:
   ```bash
   docker-compose logs webhook | grep -i "detected"
   ```

## Development

### Run Locally (Without Docker)

```bash
# Install dependencies
pip install -r requirements.txt

# Set environment variables
export ESPHOME_CONFIG_DIR=/path/to/config
export DEBOUNCE_SECONDS=5
export USE_POLLING=true
export GITEA_URL=https://gitea.example.com
export GITEA_REPO=username/esphome-configs
export GITEA_TOKEN=your_token
export GITEA_BRANCH=main
export AUTO_PUSH=false
export GIT_USER_NAME="Your Name"
export GIT_USER_EMAIL="your@email.com"

# Run the service
python app.py
```

### Build Custom Image

```bash
docker build -t esphome-gitea-sync:custom .
```

### Testing File Watcher

Create, modify, delete, or move files in the config directory and watch the logs:

```bash
docker-compose logs -f webhook
```

You should see:
```
Detected change in /config/test.yaml
Change detected in device: test
AUTO_PUSH enabled, pushing changes to Gitea
```

## Security Considerations

- The sync service has no authentication by default
- Gitea token is stored in environment variables (keep docker-compose.yml secure)
- Only expose port 5000 on trusted networks
- Use a reverse proxy (nginx, Traefik) with authentication for external access
- Keep `AUTO_PUSH=false` unless you trust all changes made in ESPHome dashboard
- Review the `.gitignore` file to avoid committing sensitive data

## Migration from Old Structure

If migrating from `device-name/main.yaml` to flat `device-name.yaml` structure:

```bash
cd config
for dir in */; do
  if [ -f "${dir}main.yaml" ]; then
    device_name="${dir%/}"
    mv "${dir}main.yaml" "${device_name}.yaml"
    rmdir "${dir}"
  fi
done
git add -A
git commit -m "Migrate to flat YAML structure"
git push origin main
```

## License

MIT License - See LICENSE file for details.