From 3c72ffc320e18c934486d65c2aae1b1d81cb5d54 Mon Sep 17 00:00:00 2001 From: "Adrian A. Baumann" Date: Mon, 13 Jul 2026 10:25:20 +0200 Subject: [PATCH] Add project README Co-Authored-By: Claude Opus 4.8 --- README.md | 219 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 219 insertions(+) create mode 100644 README.md diff --git a/README.md b/README.md new file mode 100644 index 0000000..240da86 --- /dev/null +++ b/README.md @@ -0,0 +1,219 @@ +# LabHelper + +A Django web app for keeping track of what's in the physical storage boxes in a lab +or workshop. Boxes hold *things*, things are described, pictured, tagged, and linked +to datasheets and resources, and everything is searchable from one box. Authentication +is handled by Keycloak (OIDC), and the app is packaged for Kubernetes deployment via +ArgoCD. + +## Features + +- **Boxes & box types** — define reusable box types (with physical dimensions) and + create boxes of those types. Box IDs are human-chosen strings (e.g. `A1-001`), not + auto-increment numbers. +- **Things** — items stored in a box, each with a name, Markdown description, picture, + file attachments (datasheets, etc.), and hyperlinks. +- **Faceted tagging** — organise things by facets (e.g. *Category*, *Priority*). Each + facet is single- or multi-valued and has its own colour. Tags belong to a facet. +- **Search** — live AJAX search across names, descriptions, tags, files, and links. + Supports `Facet:Value` queries (e.g. `Category:Resistors`). +- **Resources view** — a flat list of every file and link across all things. +- **"Fix me" view** — find things missing a tag for a given facet and bulk-tag them. +- **Bulk add** — add many things to a box at once via a formset. +- **SSO** — Keycloak/OIDC login with automatic Django group and `is_staff` mapping. + +## Tech stack + +| Area | Choice | +|------|--------| +| Framework | Django 5.2 (Python 3.13/3.14) | +| Database | SQLite (file-backed, persisted on a PVC in production) | +| Auth | `mozilla-django-oidc` against Keycloak | +| Static files | WhiteNoise (compressed manifest storage) | +| Images | Pillow + `sorl-thumbnail` | +| Markdown | `markdown` rendered then sanitised with `bleach` | +| Tree data | `django-mptt` (facet/type hierarchies) | +| WSGI server | Gunicorn (3 workers) | +| Deployment | Docker image → Gitea registry → ArgoCD → Kubernetes (Traefik ingress) | + +## Data model + +``` +BoxType ──1:N──> Box ──1:N──> Thing ──M:N──> Tag <──1:N── Facet + │ + ├──1:N──> ThingFile + └──1:N──> ThingLink +``` + +- `BoxType → Box` and `Box → Thing` use `PROTECT` (you can't delete a box type still + in use, or a box that still holds things). +- `Facet → Tag` and `Thing → ThingFile`/`ThingLink` use `CASCADE`. +- `Box.id` is a `CharField(max_length=10)` primary key. +- `Facet.cardinality` is `single` (0..1 tag per thing) or `multiple` (0..n). + +## Local development + +```bash +# 1. Activate the virtualenv +source .venv/bin/activate + +# 2. Apply migrations (SQLite DB lives at data/db.sqlite3) +python manage.py migrate + +# 3. Run the dev server +python manage.py runserver # http://localhost:8000 +python manage.py runserver 0.0.0.0:8000 # all interfaces +``` + +For local development you'll typically want `DEBUG=True` (the default) and to reach +the app from an IP inside `ALLOWED_CIDR_NETS`, which bypasses the login requirement +(see [Authentication](#authentication)). Otherwise you need a working Keycloak realm +configured through the environment variables below. + +### Common commands + +```bash +# Database +python manage.py makemigrations boxes # after editing boxes/models.py +python manage.py migrate +python manage.py showmigrations + +# Tests +python manage.py test # everything +python manage.py test boxes # the boxes app only + +# Static files (after CSS changes) +python manage.py collectstatic + +# Custom management commands +python manage.py list_things # print all things with box IDs +python manage.py clean_orphaned_files --dry-run # remove unreferenced files +python manage.py clean_orphaned_images --dry-run # remove unreferenced images +``` + +## Configuration + +All configuration is read from environment variables (see `labhelper/settings.py`). +Sensible defaults exist for local development; production values come from the +Kubernetes ConfigMap (`argocd/configmap.yaml`) and Secret. + +| Variable | Purpose | Default | +|----------|---------|---------| +| `DJANGO_SECRET_KEY` | Django secret key | dev key (do **not** use in prod) | +| `DEBUG` | Debug mode | `True` | +| `ALLOWED_HOSTS` | Comma-separated allowed hosts | `*` | +| `ALLOWED_CIDR_NETS` | CIDR ranges that skip login | `10.0.0.0/16,192.168.0.0/16` | +| `CSRF_TRUSTED_ORIGINS` | Trusted origins for CSRF | prod URL + `127.0.0.1:8000` | +| `STATIC_URL` / `MEDIA_URL` | Static/media URL prefixes | `/static/` / `/media/` | +| `TIME_ZONE`, `LANGUAGE_CODE`, `USE_I18N`, `USE_TZ` | Localisation | UTC / en-us / True | +| `LOGIN_URL` | Where unauthenticated users go | `oidc_authentication_init` | +| `LOGIN_REDIRECT_URL` / `LOGOUT_REDIRECT_URL` | Post-auth redirects | `index` / `login` | +| `OIDC_OP_BASE_URL` | Keycloak realm URL (endpoints derived from it) | — | +| `OIDC_RP_CLIENT_ID` / `OIDC_RP_CLIENT_SECRET` | OIDC client credentials | — | +| `GUNICORN_OPTS` | Extra gunicorn flags | — | + +`OIDC_OP_BASE_URL` should be the realm URL, e.g. +`https://sso.example.com/realms/homelab`. The individual OIDC endpoints +(auth, token, userinfo, JWKS, logout) are derived from it automatically but can each +be overridden with their own `OIDC_OP_*_ENDPOINT` variable. + +## Authentication + +Login is handled by Keycloak through `mozilla-django-oidc`. Two access paths exist: + +1. **OIDC login** — the normal path. `KeycloakOIDCBackend` + (`labhelper/auth_backend.py`) syncs the user's name, `is_staff` flag, and Django + group membership from Keycloak group claims on every login. The mapping: + + | Keycloak group | Django group | `is_staff` | + |----------------|--------------|-----------| + | `LabHelper Administrators` | `LabHelper Administrators` | yes | + | `LabHelper Staff` | `LabHelper Staff` | no | + | `LabHelper Viewers` | `LabHelper Viewers` | no | + +2. **Trusted networks** — every view is wrapped in + `conditional_login_required` (`boxes/decorators.py`), which **skips** authentication + entirely for clients whose IP falls within `ALLOWED_CIDR_NETS`. This is what lets + the app run open on a trusted LAN while still requiring SSO from outside. + +The `ModelBackend` is retained as a fallback so the Django admin +(`/admin/`) remains reachable for emergency access. + +See `Keycloak-installation.md` for notes on setting up the realm, client, and groups. + +## Deployment + +The app ships as two container images to a Gitea registry +(`git.baumann.gr/adebaumann/…`) and is reconciled onto Kubernetes by ArgoCD from the +manifests in `argocd/`: + +- **`web`** — the Django app (built from the root `Dockerfile`, served by gunicorn). +- **`loader`** — an init container (`data-loader/`) that seeds the SQLite database from + a preloaded copy on first boot, then preserves any existing DB on the PVC. + +Images are **not** built on every push. The Gitea Actions workflow +(`.gitea/workflows/build-containers-on-demand.yml`) triggers only when +`argocd/deployment.yaml`, the `Dockerfile`, or `data-loader/**` change, reads the exact +image tag out of the deployment manifest, and builds + pushes only if that tag isn't +already in the registry. So **deploying = bumping the image tag** in the manifest. + +Two helper scripts prepare a release: + +```bash +# Full deploy: bump BOTH image versions (+0.001) and snapshot the DB into the loader +./scripts/full_deploy.sh + +# Partial deploy: bump only the main web container (no DB snapshot) +./scripts/partial_deploy.sh +``` + +After running a script, commit and push the changed manifests — Gitea Actions builds +the images and ArgoCD rolls them out. + +### Kubernetes resources (`argocd/`) + +| File | Resource | +|------|----------| +| `deployment.yaml` | Deployment (web + loader init container) and Service | +| `ingress.yaml` | Traefik Ingress for `labhelper.adebaumann.com` | +| `configmap.yaml` | Non-secret environment configuration | +| `001_pvc.yaml` | PersistentVolumeClaim for the SQLite DB and media | +| `nfs-pv.yaml` | NFS-backed PersistentVolume | + +Secrets (`DJANGO_SECRET_KEY`, `oidc-client-secret`) are supplied via a Kubernetes +Secret; see `k8s-templates/secret.yaml` and `scripts/deploy_secret.sh`. Liveness and +readiness probes hit the `/health/` endpoint. + +## Project layout + +``` +labhelper/ +├── boxes/ # main app: models, views, forms, templates, static, commands +│ ├── models.py # BoxType, Box, Facet, Tag, Thing, ThingFile, ThingLink +│ ├── views.py # all views (function-based) +│ ├── decorators.py # conditional_login_required +│ └── management/commands # list_things, clean_orphaned_files, clean_orphaned_images +├── labhelper/ # project config +│ ├── settings.py # env-driven settings +│ ├── urls.py # URL routing +│ ├── auth_backend.py # Keycloak OIDC backend + group mapping +│ └── templates/ # base.html, login.html +├── data/ # SQLite DB + uploaded media (mounted from a PVC in prod) +├── data-loader/ # init-container image that seeds the DB +├── argocd/ # Kubernetes manifests (ArgoCD-managed) +├── k8s-templates/ # secret templates +├── scripts/ # deploy helpers +├── Dockerfile # web container build +├── gunicorn.conf.py # gunicorn configuration +└── manage.py +``` + +## Notes & gotchas + +- Templates live in `labhelper/templates/` (base) and `boxes/templates/boxes/` (app). +- Descriptions are Markdown, rendered via a `render_markdown` filter and sanitised with + `bleach` — don't trust raw HTML from descriptions. +- Media files are served by Django in all environments (WhiteNoise handles static). +- Uploaded pictures are renamed to `things/-.` after the thing gets + a primary key (see `Thing.save`). +- Deleting a thing also deletes its picture and attached files from disk.