diff --git a/CLAUDE.md b/CLAUDE.md index 476ce60..7fe026b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -133,6 +133,8 @@ Environment variables (set in docker-compose.yml or locally): **Core Settings:** - `ESPHOME_CONFIG_DIR` - Path to device configurations (default: `/config`) - `DEBOUNCE_SECONDS` - Delay before triggering after file change (default: `5`) +- `USE_POLLING` - Use polling instead of inotify for file watching (default: `true`, required for Docker bind mounts) +- `POLLING_INTERVAL` - Seconds between filesystem polls when using polling mode (default: `1.0`) **Gitea Configuration:** - `GITEA_URL` - URL of Gitea instance (e.g., `https://gitea.example.com`) @@ -237,7 +239,34 @@ The file watcher monitors all `.yaml` and `.yml` files directly in the config di - **Ignored**: Subdirectories, non-YAML files, and hidden files (like `.git`) - **Debouncing**: Changes are debounced for `DEBOUNCE_SECONDS` (default 5) to prevent duplicate triggers - **Auto-push**: If `AUTO_PUSH=true`, any watched file change triggers a git commit and push -- **Commit message format**: `Auto-sync: {device_name}.yaml changed` +- **Polling Mode**: Uses filesystem polling (default) instead of inotify to ensure compatibility with Docker bind mounts on WSL2, macOS, Unraid, and other environments where inotify events don't propagate correctly through volume mounts + +#### Events Detected + +1. **File Modified** (`device.yaml` edited) + - Logs: "Detected change in /config/device.yaml" + - Commit message: `Auto-sync: device.yaml changed` + +2. **File Created** (new `device.yaml` added) + - Logs: "Detected change in /config/device.yaml" + - Commit message: `Auto-sync: device.yaml changed` + +3. **File Deleted** (`device.yaml` removed) + - Logs: "Detected deletion of /config/device.yaml" + - Commit message: `Auto-sync: device.yaml deleted` + +4. **File Renamed** (`old-device.yaml` → `new-device.yaml`) + - Logs: "Rename detected: old-device.yaml -> new-device.yaml" + - Commit message: `Auto-sync: Renamed old-device.yaml to new-device.yaml` + +5. **File Archived** (`device.yaml` → `archive/device.yaml`) + - Logs: "Device moved to archive/subdirectory: device" + - Commit message: `Auto-sync: device.yaml archived` + - The file deletion from `/config/` root is committed (archive folder typically in `.gitignore`) + +6. **File Restored** (`archive/device.yaml` → `device.yaml`) + - Logs: "Device moved from subdirectory to config root: device" + - Commit message: `Auto-sync: device.yaml restored from archive` Example watched files in `/config`: - `bedroom-light.yaml` ✓ @@ -245,7 +274,7 @@ Example watched files in `/config`: - `garage-door.yml` ✓ - `README.md` ✗ (not YAML) - `.gitignore` ✗ (hidden file) -- `backup/old-config.yaml` ✗ (in subdirectory) +- `archive/old-config.yaml` ✗ (in subdirectory, not watched but move events detected) ### Git Authentication diff --git a/README.md b/README.md index e10ce0c..1f25278 100644 --- a/README.md +++ b/README.md @@ -1,50 +1,84 @@ -# ESPHome Webhook Service +# ESPHome Gitea Sync Service -A Docker-based webhook service for automating ESPHome device compilation and deployment with file watching capabilities. +A Docker-based synchronization service that bridges Gitea repositories with ESPHome device configurations. Automatically syncs YAML configuration files between your Gitea git repository and your ESPHome installation. ## Features -- **Webhook Endpoints**: Trigger ESPHome operations via HTTP POST requests -- **File Watching**: Automatically detect YAML changes and trigger compilation/upload -- **REST API**: List devices, check health, and manage operations -- **Docker Compose**: Easy deployment with ESPHome dashboard -- **Debouncing**: Prevent duplicate operations from rapid file changes +- **Bidirectional Git Sync**: Sync configurations between Gitea and ESPHome +- **File Watching**: Automatically detect YAML changes and push to Gitea +- **Gitea Webhooks**: Receive push events and pull changes automatically +- **REST API**: Manual sync endpoints and device listing +- **Docker Polling**: Uses filesystem polling for reliable change detection on Docker bind mounts - **Thread-Safe**: Concurrent operation handling with locks +- **Archive Support**: Detects when files are moved to archive folders + +## Architecture + +This service works alongside the official ESPHome container: +- **ESPHome Container**: Handles all compilation and device deployment (immutable) +- **Sync Service**: Manages git synchronization between Gitea and ESPHome (this service) + +The ESPHome container remains unchanged and simply consumes the YAML configs. This service only handles git operations. ## Quick Start -### 1. Start the Services +### 1. Configure Environment Variables + +Edit `docker-compose.yml` and set your Gitea details: + +```yaml +environment: + - GITEA_URL=https://your-gitea-instance.com + - GITEA_REPO=username/esphome-configs + - GITEA_TOKEN=your_gitea_access_token + - GITEA_BRANCH=main + - AUTO_PUSH=false # Set to true for automatic push on file changes +``` + +### 2. Start the Services ```bash -cd esphome-webhook-service docker-compose up -d ``` This starts two services: -- **ESPHome Dashboard**: `http://localhost:6052` (host network mode) -- **Webhook Service**: `http://localhost:5000` +- **ESPHome Dashboard**: `http://localhost:6052` (compilation and deployment) +- **Gitea Sync Service**: `http://localhost:5000` (git synchronization) -### 2. Access the Services +### 3. Access the Services - ESPHome Dashboard: http://localhost:6052 -- Webhook API: http://localhost:5000/health +- Sync API Health: http://localhost:5000/health +- Device List: http://localhost:5000/devices -### 3. Stop the Services +### 4. Configure Gitea Webhook (Optional) -```bash -docker-compose down -``` +To automatically pull changes when you push to Gitea: + +1. Go to your Gitea repository → Settings → Webhooks +2. Add Webhook → Gitea +3. Set Target URL: `http://your-server:5000/webhook/gitea` +4. Content Type: `application/json` +5. Trigger On: `Push Events` +6. Click "Add Webhook" ## Environment Variables -Configure the webhook service behavior in `docker-compose.yml`: +Configure the sync service in `docker-compose.yml`: | Variable | Default | Description | |----------|---------|-------------| | `ESPHOME_CONFIG_DIR` | `/config` | Directory containing device configs | -| `AUTO_COMPILE` | `true` | Auto-compile on YAML file changes | -| `AUTO_UPLOAD` | `false` | Auto-upload on YAML file changes (⚠️ use with caution) | -| `DEBOUNCE_SECONDS` | `5` | Delay before triggering operations after file change | +| `DEBOUNCE_SECONDS` | `5` | Delay before triggering after file change | +| `USE_POLLING` | `true` | Use polling for Docker compatibility (required) | +| `POLLING_INTERVAL` | `1.0` | Seconds between filesystem polls | +| `GITEA_URL` | - | URL of your Gitea instance | +| `GITEA_REPO` | - | Repository path (username/repo) | +| `GITEA_TOKEN` | - | Gitea access token (generate in user settings) | +| `GITEA_BRANCH` | `main` | Git branch to sync | +| `AUTO_PUSH` | `false` | Automatically push local changes to Gitea | +| `GIT_USER_NAME` | `ESPHome Sync Service` | Git commit author name | +| `GIT_USER_EMAIL` | `esphome-sync@localhost` | Git commit author email | ## API Endpoints @@ -56,103 +90,116 @@ curl http://localhost:5000/health ### List Devices +Lists all YAML files in the config directory: + ```bash curl http://localhost:5000/devices ``` -### Validate Configuration - -```bash -curl -X POST http://localhost:5000/webhook/validate/ades-office-control-panel +Returns: +```json +{ + "devices": [ + { + "name": "bedroom-light", + "config_path": "/config/bedroom-light.yaml", + "last_modified": "2026-01-13T10:30:00" + } + ] +} ``` -### Compile Device +### Gitea Webhook + +Receives push events from Gitea and pulls changes: ```bash -curl -X POST http://localhost:5000/webhook/compile/ades-office-control-panel +curl -X POST http://localhost:5000/webhook/gitea \ + -H "Content-Type: application/json" \ + -H "X-Gitea-Event: push" \ + -d '{"repository": {"full_name": "user/repo"}}' ``` -### Upload Device (OTA) +### Manual Pull from Gitea + +Manually trigger a git pull: ```bash -curl -X POST http://localhost:5000/webhook/upload/ades-office-control-panel +curl -X POST http://localhost:5000/sync/pull ``` -### Compile and Upload +### Manual Push to Gitea + +Manually trigger a git push: ```bash -curl -X POST http://localhost:5000/webhook/run/ades-office-control-panel +curl -X POST http://localhost:5000/sync/push \ + -H "Content-Type: application/json" \ + -d '{"message": "Manual sync from API"}' ``` ## File Watching -The service automatically watches all YAML files in the config directory. When a `main.yaml` file is modified: +The service monitors all `.yaml` and `.yml` files in the root of the config directory. -1. **Auto-Compile Enabled**: Automatically compiles the device configuration -2. **Auto-Upload Enabled**: Automatically uploads firmware to the device (OTA) +### Detected Events -**⚠️ WARNING**: Only enable `AUTO_UPLOAD=true` if you're confident in your changes and have physical access to devices in case of failures. +1. **File Modified** - Edit existing YAML file +2. **File Created** - Add new YAML file +3. **File Deleted** - Remove YAML file +4. **File Renamed** - Rename YAML file +5. **File Archived** - Move YAML file to subdirectory (e.g., `archive/`) +6. **File Restored** - Move YAML file from subdirectory back to root -## Integration Examples +When `AUTO_PUSH=true`, all these events automatically trigger a git commit and push to Gitea. -### GitHub Actions Webhook +### Commit Messages -Trigger compilation after pushing changes: - -```yaml -name: ESPHome Compile - -on: - push: - paths: - - '**/*.yaml' - -jobs: - compile: - runs-on: ubuntu-latest - steps: - - name: Trigger Webhook - run: | - curl -X POST http://your-server:5000/webhook/compile/ades-office-control-panel -``` - -### Home Assistant Automation - -Create an automation to compile devices: - -```yaml -automation: - - alias: "Compile ESPHome Device" - trigger: - platform: webhook - webhook_id: esphome_compile - action: - service: rest_command.compile_device - data: - device: "{{ trigger.data.device }}" - -rest_command: - compile_device: - url: "http://localhost:5000/webhook/compile/{{ device }}" - method: POST -``` +- Modified/Created: `Auto-sync: device-name.yaml changed` +- Deleted: `Auto-sync: device-name.yaml deleted` +- Renamed: `Auto-sync: Renamed old-name.yaml to new-name.yaml` +- Archived: `Auto-sync: device-name.yaml archived` +- Restored: `Auto-sync: device-name.yaml restored from archive` ## Directory Structure ``` -esphome-webhook-service/ -├── app.py # Webhook service with file watching -├── Dockerfile # Container image definition -├── requirements.txt # Python dependencies -├── docker-compose.yml # Service orchestration -├── README.md # This file -└── config/ # Device configurations (mounted as /config) - ├── ades-office-control-panel/ - │ └── main.yaml - ├── Old_Phone_Doorbell/ - │ └── main.yaml - └── Oekoboiler/ - └── main.yaml +config/ +├── bedroom-light.yaml # Device configs (flat structure) +├── kitchen-sensor.yaml +├── garage-door.yaml +├── .gitignore # Optional: ignore archive folder +├── archive/ # Optional: archived configs (not monitored) +│ └── old-device.yaml +└── .git/ # Git repository (managed by sync service) +``` + +## Workflows + +### Editing in ESPHome Dashboard + +1. Edit YAML files in ESPHome dashboard +2. File watcher detects the change +3. If `AUTO_PUSH=true`: Changes automatically committed and pushed to Gitea +4. If `AUTO_PUSH=false`: Manually trigger push via API + +### Editing in Gitea (or git client) + +1. Commit and push changes to Gitea repository +2. Gitea webhook triggers `/webhook/gitea` endpoint +3. Service runs `git pull` to fetch changes +4. ESPHome sees updated files and reloads dashboard + +### Manual Sync + +```bash +# Pull latest changes from Gitea +curl -X POST http://localhost:5000/sync/pull + +# Push local changes to Gitea +curl -X POST http://localhost:5000/sync/push \ + -H "Content-Type: application/json" \ + -d '{"message": "My commit message"}' ``` ## Troubleshooting @@ -165,26 +212,40 @@ docker-compose logs webhook docker-compose logs esphome ``` -### Webhook Returns 404 +### File Watcher Not Detecting Changes -Ensure the device name matches the directory name: -```bash -curl http://localhost:5000/devices +**Cause**: inotify events don't propagate through Docker bind mounts on WSL2, Unraid, and macOS. + +**Solution**: The service uses polling mode by default (`USE_POLLING=true`). Ensure this is enabled in docker-compose.yml. + +Check logs for: +``` +Using PollingObserver (interval: 1.0s) for Docker bind mount compatibility ``` -### OTA Upload Fails +### Git Operations Failing -1. Verify device is on the same network -2. Check ESPHome logs: `docker-compose logs esphome` -3. Ensure OTA password is correct in secrets.yaml -4. Try using host network mode for webhook service (uncomment in docker-compose.yml) +1. Verify Gitea token has correct permissions (read/write repository) +2. Check `GITEA_URL`, `GITEA_REPO`, and `GITEA_TOKEN` are set correctly +3. Check logs for git command errors: + ```bash + docker-compose logs webhook | grep -i "git command" + ``` -### File Watcher Not Triggering +### Webhook Not Receiving Events -1. Check that `AUTO_COMPILE` is set to `true` -2. Verify the file path is correct -3. Check webhook logs for file change events -4. Ensure Docker has permission to watch the mounted volume +1. Verify webhook URL is accessible from Gitea (firewall, network) +2. Check Gitea webhook delivery logs (Repository → Settings → Webhooks → Recent Deliveries) +3. Ensure `Content-Type` is `application/json` and trigger is `Push Events` + +### Changes Not Syncing to Gitea + +1. Check if `AUTO_PUSH=true` in docker-compose.yml +2. Verify file is in the root of `/config/` (not in subdirectory) +3. Check logs for file change detection: + ```bash + docker-compose logs webhook | grep -i "detected" + ``` ## Development @@ -193,12 +254,18 @@ curl http://localhost:5000/devices ```bash # Install dependencies pip install -r requirements.txt -pip install esphome # Set environment variables -export ESPHOME_CONFIG_DIR=/path/to/esphome2/config -export AUTO_COMPILE=true -export AUTO_UPLOAD=false +export ESPHOME_CONFIG_DIR=/path/to/config +export DEBOUNCE_SECONDS=5 +export USE_POLLING=true +export GITEA_URL=https://gitea.example.com +export GITEA_REPO=username/esphome-configs +export GITEA_TOKEN=your_token +export GITEA_BRANCH=main +export AUTO_PUSH=false +export GIT_USER_NAME="Your Name" +export GIT_USER_EMAIL="your@email.com" # Run the service python app.py @@ -207,17 +274,51 @@ python app.py ### Build Custom Image ```bash -docker build -t esphome-webhook:custom . +docker build -t esphome-gitea-sync:custom . +``` + +### Testing File Watcher + +Create, modify, delete, or move files in the config directory and watch the logs: + +```bash +docker-compose logs -f webhook +``` + +You should see: +``` +Detected change in /config/test.yaml +Change detected in device: test +AUTO_PUSH enabled, pushing changes to Gitea ``` ## Security Considerations -- The webhook service has no authentication by default +- The sync service has no authentication by default +- Gitea token is stored in environment variables (keep docker-compose.yml secure) - Only expose port 5000 on trusted networks - Use a reverse proxy (nginx, Traefik) with authentication for external access -- Keep `AUTO_UPLOAD=false` unless absolutely necessary -- Review changes before enabling auto-upload +- Keep `AUTO_PUSH=false` unless you trust all changes made in ESPHome dashboard +- Review the `.gitignore` file to avoid committing sensitive data + +## Migration from Old Structure + +If migrating from `device-name/main.yaml` to flat `device-name.yaml` structure: + +```bash +cd config +for dir in */; do + if [ -f "${dir}main.yaml" ]; then + device_name="${dir%/}" + mv "${dir}main.yaml" "${device_name}.yaml" + rmdir "${dir}" + fi +done +git add -A +git commit -m "Migrate to flat YAML structure" +git push origin main +``` ## License -Same as parent repository (see LICENSE file). +MIT License - See LICENSE file for details. diff --git a/app.py b/app.py index 21bab05..211eb4b 100644 --- a/app.py +++ b/app.py @@ -21,6 +21,7 @@ from datetime import datetime from flask import Flask, request, jsonify from watchdog.observers import Observer +from watchdog.observers.polling import PollingObserver from watchdog.events import FileSystemEventHandler # Configure logging @@ -35,6 +36,8 @@ app = Flask(__name__) # Configuration ESPHOME_CONFIG_DIR = os.environ.get('ESPHOME_CONFIG_DIR', '/config') DEBOUNCE_SECONDS = int(os.environ.get('DEBOUNCE_SECONDS', '5')) +POLLING_INTERVAL = float(os.environ.get('POLLING_INTERVAL', '1.0')) # Seconds between polls +USE_POLLING = os.environ.get('USE_POLLING', 'true').lower() == 'true' # Use polling for Docker compatibility # Gitea configuration GITEA_URL = os.environ.get('GITEA_URL', '') @@ -58,7 +61,8 @@ class ESPHomeFileHandler(FileSystemEventHandler): def __init__(self): self.last_modified = {} - def on_modified(self, event): + def _handle_file_change(self, event): + """Common handler for file modifications and creations""" if event.is_directory: return @@ -70,6 +74,7 @@ class ESPHomeFileHandler(FileSystemEventHandler): now = time.time() if event.src_path in self.last_modified: if now - self.last_modified[event.src_path] < DEBOUNCE_SECONDS: + logger.debug(f"Debouncing {event.src_path} (too soon)") return self.last_modified[event.src_path] = now @@ -88,6 +93,93 @@ class ESPHomeFileHandler(FileSystemEventHandler): Thread(target=git_push, args=(f"Auto-sync: {device_name}.yaml changed",)).start() else: logger.info(f"AUTO_PUSH disabled, skipping push to Gitea") + else: + logger.debug(f"Ignoring file outside config directory: {event.src_path}") + + def on_modified(self, event): + """Handle file modification events""" + self._handle_file_change(event) + + def on_created(self, event): + """Handle file creation events""" + self._handle_file_change(event) + + def on_deleted(self, event): + """Handle file deletion events""" + if event.is_directory: + return + + # Only watch YAML files + if not event.src_path.endswith(('.yaml', '.yml')): + return + + logger.info(f"Detected deletion of {event.src_path}") + + # Trigger git push if AUTO_PUSH is enabled + device_path = Path(event.src_path) + # Check if this is a device config file directly in the config directory + if device_path.parent == Path(ESPHOME_CONFIG_DIR): + device_name = device_path.stem # Get filename without extension + logger.info(f"Deletion detected for device: {device_name}") + + if AUTO_PUSH: + logger.info(f"AUTO_PUSH enabled, pushing deletion to Gitea") + Thread(target=git_push, args=(f"Auto-sync: {device_name}.yaml deleted",)).start() + else: + logger.info(f"AUTO_PUSH disabled, skipping push to Gitea") + else: + logger.debug(f"Ignoring file outside config directory: {event.src_path}") + + def on_moved(self, event): + """Handle file move/rename events""" + if event.is_directory: + return + + # Only watch YAML files + if not event.src_path.endswith(('.yaml', '.yml')) and not event.dest_path.endswith(('.yaml', '.yml')): + return + + src_path = Path(event.src_path) + dest_path = Path(event.dest_path) + config_dir = Path(ESPHOME_CONFIG_DIR) + + src_in_config = src_path.parent == config_dir + dest_in_config = dest_path.parent == config_dir + + logger.info(f"Detected move from {event.src_path} to {event.dest_path}") + + if src_in_config and dest_in_config: + # Rename within config directory - treat as modification + device_name = dest_path.stem + logger.info(f"Rename detected: {src_path.name} -> {dest_path.name}") + + if AUTO_PUSH: + logger.info(f"AUTO_PUSH enabled, pushing rename to Gitea") + Thread(target=git_push, args=(f"Auto-sync: Renamed {src_path.name} to {dest_path.name}",)).start() + else: + logger.info(f"AUTO_PUSH disabled, skipping push to Gitea") + + elif src_in_config and not dest_in_config: + # Moved OUT of config directory (e.g., to archive folder) - treat as deletion + device_name = src_path.stem + logger.info(f"Device moved to archive/subdirectory: {device_name}") + + if AUTO_PUSH: + logger.info(f"AUTO_PUSH enabled, pushing archived file as deletion to Gitea") + Thread(target=git_push, args=(f"Auto-sync: {src_path.name} archived",)).start() + else: + logger.info(f"AUTO_PUSH disabled, skipping push to Gitea") + + elif not src_in_config and dest_in_config: + # Moved INTO config directory - treat as creation + device_name = dest_path.stem + logger.info(f"Device moved from subdirectory to config root: {device_name}") + + if AUTO_PUSH: + logger.info(f"AUTO_PUSH enabled, pushing new file to Gitea") + Thread(target=git_push, args=(f"Auto-sync: {dest_path.name} restored from archive",)).start() + else: + logger.info(f"AUTO_PUSH disabled, skipping push to Gitea") def find_device_config(device_name): @@ -403,18 +495,49 @@ def manual_push(): def start_file_watcher(): """Start the file system watcher""" + logger.info(f"Initializing file watcher for directory: {ESPHOME_CONFIG_DIR}") + logger.info(f"Watching for .yaml and .yml files with {DEBOUNCE_SECONDS}s debounce") + logger.info(f"AUTO_PUSH is {'enabled' if AUTO_PUSH else 'disabled'}") + + # Select observer type based on configuration + if USE_POLLING: + logger.info(f"Using PollingObserver (interval: {POLLING_INTERVAL}s) for Docker bind mount compatibility") + else: + logger.info("Using native filesystem observer (inotify)") + + # Verify the directory exists + config_path = Path(ESPHOME_CONFIG_DIR) + if not config_path.exists(): + logger.error(f"Config directory does not exist: {ESPHOME_CONFIG_DIR}") + return + if not config_path.is_dir(): + logger.error(f"Config path is not a directory: {ESPHOME_CONFIG_DIR}") + return + event_handler = ESPHomeFileHandler() - observer = Observer() + + # Use PollingObserver for Docker compatibility, or native Observer for local development + if USE_POLLING: + observer = PollingObserver(timeout=POLLING_INTERVAL) + else: + observer = Observer() + observer.schedule(event_handler, ESPHOME_CONFIG_DIR, recursive=True) observer.start() - logger.info(f"File watcher started on {ESPHOME_CONFIG_DIR}") + logger.info(f"File watcher started successfully on {ESPHOME_CONFIG_DIR}") + + # Log existing YAML files being watched + yaml_files = list(config_path.glob('*.yaml')) + list(config_path.glob('*.yml')) + logger.info(f"Currently watching {len(yaml_files)} YAML files: {[f.name for f in yaml_files]}") try: while True: time.sleep(1) except KeyboardInterrupt: + logger.info("File watcher interrupted, stopping...") observer.stop() observer.join() + logger.info("File watcher stopped") if __name__ == '__main__': diff --git a/docker-compose.yml b/docker-compose.yml index 84b7c86..23aefa2 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -36,6 +36,10 @@ services: # Debounce delay in seconds to prevent rapid repeated triggers - DEBOUNCE_SECONDS=5 + # File watcher settings (polling mode required for Docker bind mounts) + - USE_POLLING=true + - POLLING_INTERVAL=1.0 + # Gitea repository configuration - GITEA_URL=https://git.baumann.gr/ - GITEA_REPO=adebaumann/ESP-Home-Scripts diff --git a/gunicorn_config.py b/gunicorn_config.py index 09f3997..f7fef29 100644 --- a/gunicorn_config.py +++ b/gunicorn_config.py @@ -10,8 +10,11 @@ loglevel = "info" logger = logging.getLogger(__name__) -# Track if file watcher has been started -_watcher_started = False +# Track which worker has the file watcher (using a file-based flag) +import os +import tempfile + +WATCHER_LOCK_FILE = os.path.join(tempfile.gettempdir(), 'esphome_watcher.lock') def on_starting(server): @@ -21,6 +24,14 @@ def on_starting(server): """ logger.info("Gunicorn master process starting - initializing git repository") + # Clean up any stale lock file from previous runs + if os.path.exists(WATCHER_LOCK_FILE): + try: + os.remove(WATCHER_LOCK_FILE) + logger.info("Removed stale watcher lock file") + except Exception as e: + logger.warning(f"Could not remove stale lock file: {e}") + # Import here to avoid circular imports from app import initialize_git_repo @@ -36,12 +47,15 @@ def post_fork(server, worker): Called after a worker has been forked. Start the file watcher only in the first worker to avoid duplicates. """ - global _watcher_started + # Use a file-based lock to ensure only one worker starts the watcher + # This works across process boundaries unlike global variables + try: + # Try to create the lock file (exclusive creation) + fd = os.open(WATCHER_LOCK_FILE, os.O_CREAT | os.O_EXCL | os.O_WRONLY) + os.write(fd, str(worker.pid).encode()) + os.close(fd) - # Only start the file watcher in the first worker (age/number would be 0) - # Check worker.age which is the worker's sequential number - if worker.age == 0 and not _watcher_started: - _watcher_started = True + # This worker won the race, start the file watcher logger.info(f"Starting file watcher in worker {worker.pid}") # Import here to avoid circular imports @@ -54,5 +68,13 @@ def post_fork(server, worker): logger.info("File watcher started successfully") except Exception as e: logger.error(f"Failed to start file watcher: {e}") - else: + # Clean up lock file on failure + try: + os.remove(WATCHER_LOCK_FILE) + except: + pass + except FileExistsError: + # Another worker already started the watcher logger.info(f"Worker {worker.pid} started (file watcher already running in another worker)") + except Exception as e: + logger.error(f"Error in post_fork for worker {worker.pid}: {e}")