Add project README
This commit is contained in:
@@ -0,0 +1,219 @@
|
||||
# LabHelper
|
||||
|
||||
A Django web app for keeping track of what's in the physical storage boxes in a lab
|
||||
or workshop. Boxes hold *things*, things are described, pictured, tagged, and linked
|
||||
to datasheets and resources, and everything is searchable from one box. Authentication
|
||||
is handled by Keycloak (OIDC), and the app is packaged for Kubernetes deployment via
|
||||
ArgoCD.
|
||||
|
||||
## Features
|
||||
|
||||
- **Boxes & box types** — define reusable box types (with physical dimensions) and
|
||||
create boxes of those types. Box IDs are human-chosen strings (e.g. `A1-001`), not
|
||||
auto-increment numbers.
|
||||
- **Things** — items stored in a box, each with a name, Markdown description, picture,
|
||||
file attachments (datasheets, etc.), and hyperlinks.
|
||||
- **Faceted tagging** — organise things by facets (e.g. *Category*, *Priority*). Each
|
||||
facet is single- or multi-valued and has its own colour. Tags belong to a facet.
|
||||
- **Search** — live AJAX search across names, descriptions, tags, files, and links.
|
||||
Supports `Facet:Value` queries (e.g. `Category:Resistors`).
|
||||
- **Resources view** — a flat list of every file and link across all things.
|
||||
- **"Fix me" view** — find things missing a tag for a given facet and bulk-tag them.
|
||||
- **Bulk add** — add many things to a box at once via a formset.
|
||||
- **SSO** — Keycloak/OIDC login with automatic Django group and `is_staff` mapping.
|
||||
|
||||
## Tech stack
|
||||
|
||||
| Area | Choice |
|
||||
|------|--------|
|
||||
| Framework | Django 5.2 (Python 3.13/3.14) |
|
||||
| Database | SQLite (file-backed, persisted on a PVC in production) |
|
||||
| Auth | `mozilla-django-oidc` against Keycloak |
|
||||
| Static files | WhiteNoise (compressed manifest storage) |
|
||||
| Images | Pillow + `sorl-thumbnail` |
|
||||
| Markdown | `markdown` rendered then sanitised with `bleach` |
|
||||
| Tree data | `django-mptt` (facet/type hierarchies) |
|
||||
| WSGI server | Gunicorn (3 workers) |
|
||||
| Deployment | Docker image → Gitea registry → ArgoCD → Kubernetes (Traefik ingress) |
|
||||
|
||||
## Data model
|
||||
|
||||
```
|
||||
BoxType ──1:N──> Box ──1:N──> Thing ──M:N──> Tag <──1:N── Facet
|
||||
│
|
||||
├──1:N──> ThingFile
|
||||
└──1:N──> ThingLink
|
||||
```
|
||||
|
||||
- `BoxType → Box` and `Box → Thing` use `PROTECT` (you can't delete a box type still
|
||||
in use, or a box that still holds things).
|
||||
- `Facet → Tag` and `Thing → ThingFile`/`ThingLink` use `CASCADE`.
|
||||
- `Box.id` is a `CharField(max_length=10)` primary key.
|
||||
- `Facet.cardinality` is `single` (0..1 tag per thing) or `multiple` (0..n).
|
||||
|
||||
## Local development
|
||||
|
||||
```bash
|
||||
# 1. Activate the virtualenv
|
||||
source .venv/bin/activate
|
||||
|
||||
# 2. Apply migrations (SQLite DB lives at data/db.sqlite3)
|
||||
python manage.py migrate
|
||||
|
||||
# 3. Run the dev server
|
||||
python manage.py runserver # http://localhost:8000
|
||||
python manage.py runserver 0.0.0.0:8000 # all interfaces
|
||||
```
|
||||
|
||||
For local development you'll typically want `DEBUG=True` (the default) and to reach
|
||||
the app from an IP inside `ALLOWED_CIDR_NETS`, which bypasses the login requirement
|
||||
(see [Authentication](#authentication)). Otherwise you need a working Keycloak realm
|
||||
configured through the environment variables below.
|
||||
|
||||
### Common commands
|
||||
|
||||
```bash
|
||||
# Database
|
||||
python manage.py makemigrations boxes # after editing boxes/models.py
|
||||
python manage.py migrate
|
||||
python manage.py showmigrations
|
||||
|
||||
# Tests
|
||||
python manage.py test # everything
|
||||
python manage.py test boxes # the boxes app only
|
||||
|
||||
# Static files (after CSS changes)
|
||||
python manage.py collectstatic
|
||||
|
||||
# Custom management commands
|
||||
python manage.py list_things # print all things with box IDs
|
||||
python manage.py clean_orphaned_files --dry-run # remove unreferenced files
|
||||
python manage.py clean_orphaned_images --dry-run # remove unreferenced images
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
All configuration is read from environment variables (see `labhelper/settings.py`).
|
||||
Sensible defaults exist for local development; production values come from the
|
||||
Kubernetes ConfigMap (`argocd/configmap.yaml`) and Secret.
|
||||
|
||||
| Variable | Purpose | Default |
|
||||
|----------|---------|---------|
|
||||
| `DJANGO_SECRET_KEY` | Django secret key | dev key (do **not** use in prod) |
|
||||
| `DEBUG` | Debug mode | `True` |
|
||||
| `ALLOWED_HOSTS` | Comma-separated allowed hosts | `*` |
|
||||
| `ALLOWED_CIDR_NETS` | CIDR ranges that skip login | `10.0.0.0/16,192.168.0.0/16` |
|
||||
| `CSRF_TRUSTED_ORIGINS` | Trusted origins for CSRF | prod URL + `127.0.0.1:8000` |
|
||||
| `STATIC_URL` / `MEDIA_URL` | Static/media URL prefixes | `/static/` / `/media/` |
|
||||
| `TIME_ZONE`, `LANGUAGE_CODE`, `USE_I18N`, `USE_TZ` | Localisation | UTC / en-us / True |
|
||||
| `LOGIN_URL` | Where unauthenticated users go | `oidc_authentication_init` |
|
||||
| `LOGIN_REDIRECT_URL` / `LOGOUT_REDIRECT_URL` | Post-auth redirects | `index` / `login` |
|
||||
| `OIDC_OP_BASE_URL` | Keycloak realm URL (endpoints derived from it) | — |
|
||||
| `OIDC_RP_CLIENT_ID` / `OIDC_RP_CLIENT_SECRET` | OIDC client credentials | — |
|
||||
| `GUNICORN_OPTS` | Extra gunicorn flags | — |
|
||||
|
||||
`OIDC_OP_BASE_URL` should be the realm URL, e.g.
|
||||
`https://sso.example.com/realms/homelab`. The individual OIDC endpoints
|
||||
(auth, token, userinfo, JWKS, logout) are derived from it automatically but can each
|
||||
be overridden with their own `OIDC_OP_*_ENDPOINT` variable.
|
||||
|
||||
## Authentication
|
||||
|
||||
Login is handled by Keycloak through `mozilla-django-oidc`. Two access paths exist:
|
||||
|
||||
1. **OIDC login** — the normal path. `KeycloakOIDCBackend`
|
||||
(`labhelper/auth_backend.py`) syncs the user's name, `is_staff` flag, and Django
|
||||
group membership from Keycloak group claims on every login. The mapping:
|
||||
|
||||
| Keycloak group | Django group | `is_staff` |
|
||||
|----------------|--------------|-----------|
|
||||
| `LabHelper Administrators` | `LabHelper Administrators` | yes |
|
||||
| `LabHelper Staff` | `LabHelper Staff` | no |
|
||||
| `LabHelper Viewers` | `LabHelper Viewers` | no |
|
||||
|
||||
2. **Trusted networks** — every view is wrapped in
|
||||
`conditional_login_required` (`boxes/decorators.py`), which **skips** authentication
|
||||
entirely for clients whose IP falls within `ALLOWED_CIDR_NETS`. This is what lets
|
||||
the app run open on a trusted LAN while still requiring SSO from outside.
|
||||
|
||||
The `ModelBackend` is retained as a fallback so the Django admin
|
||||
(`/admin/`) remains reachable for emergency access.
|
||||
|
||||
See `Keycloak-installation.md` for notes on setting up the realm, client, and groups.
|
||||
|
||||
## Deployment
|
||||
|
||||
The app ships as two container images to a Gitea registry
|
||||
(`git.baumann.gr/adebaumann/…`) and is reconciled onto Kubernetes by ArgoCD from the
|
||||
manifests in `argocd/`:
|
||||
|
||||
- **`web`** — the Django app (built from the root `Dockerfile`, served by gunicorn).
|
||||
- **`loader`** — an init container (`data-loader/`) that seeds the SQLite database from
|
||||
a preloaded copy on first boot, then preserves any existing DB on the PVC.
|
||||
|
||||
Images are **not** built on every push. The Gitea Actions workflow
|
||||
(`.gitea/workflows/build-containers-on-demand.yml`) triggers only when
|
||||
`argocd/deployment.yaml`, the `Dockerfile`, or `data-loader/**` change, reads the exact
|
||||
image tag out of the deployment manifest, and builds + pushes only if that tag isn't
|
||||
already in the registry. So **deploying = bumping the image tag** in the manifest.
|
||||
|
||||
Two helper scripts prepare a release:
|
||||
|
||||
```bash
|
||||
# Full deploy: bump BOTH image versions (+0.001) and snapshot the DB into the loader
|
||||
./scripts/full_deploy.sh
|
||||
|
||||
# Partial deploy: bump only the main web container (no DB snapshot)
|
||||
./scripts/partial_deploy.sh
|
||||
```
|
||||
|
||||
After running a script, commit and push the changed manifests — Gitea Actions builds
|
||||
the images and ArgoCD rolls them out.
|
||||
|
||||
### Kubernetes resources (`argocd/`)
|
||||
|
||||
| File | Resource |
|
||||
|------|----------|
|
||||
| `deployment.yaml` | Deployment (web + loader init container) and Service |
|
||||
| `ingress.yaml` | Traefik Ingress for `labhelper.adebaumann.com` |
|
||||
| `configmap.yaml` | Non-secret environment configuration |
|
||||
| `001_pvc.yaml` | PersistentVolumeClaim for the SQLite DB and media |
|
||||
| `nfs-pv.yaml` | NFS-backed PersistentVolume |
|
||||
|
||||
Secrets (`DJANGO_SECRET_KEY`, `oidc-client-secret`) are supplied via a Kubernetes
|
||||
Secret; see `k8s-templates/secret.yaml` and `scripts/deploy_secret.sh`. Liveness and
|
||||
readiness probes hit the `/health/` endpoint.
|
||||
|
||||
## Project layout
|
||||
|
||||
```
|
||||
labhelper/
|
||||
├── boxes/ # main app: models, views, forms, templates, static, commands
|
||||
│ ├── models.py # BoxType, Box, Facet, Tag, Thing, ThingFile, ThingLink
|
||||
│ ├── views.py # all views (function-based)
|
||||
│ ├── decorators.py # conditional_login_required
|
||||
│ └── management/commands # list_things, clean_orphaned_files, clean_orphaned_images
|
||||
├── labhelper/ # project config
|
||||
│ ├── settings.py # env-driven settings
|
||||
│ ├── urls.py # URL routing
|
||||
│ ├── auth_backend.py # Keycloak OIDC backend + group mapping
|
||||
│ └── templates/ # base.html, login.html
|
||||
├── data/ # SQLite DB + uploaded media (mounted from a PVC in prod)
|
||||
├── data-loader/ # init-container image that seeds the DB
|
||||
├── argocd/ # Kubernetes manifests (ArgoCD-managed)
|
||||
├── k8s-templates/ # secret templates
|
||||
├── scripts/ # deploy helpers
|
||||
├── Dockerfile # web container build
|
||||
├── gunicorn.conf.py # gunicorn configuration
|
||||
└── manage.py
|
||||
```
|
||||
|
||||
## Notes & gotchas
|
||||
|
||||
- Templates live in `labhelper/templates/` (base) and `boxes/templates/boxes/` (app).
|
||||
- Descriptions are Markdown, rendered via a `render_markdown` filter and sanitised with
|
||||
`bleach` — don't trust raw HTML from descriptions.
|
||||
- Media files are served by Django in all environments (WhiteNoise handles static).
|
||||
- Uploaded pictures are renamed to `things/<thing-id>-<slug>.<ext>` after the thing gets
|
||||
a primary key (see `Thing.save`).
|
||||
- Deleting a thing also deletes its picture and attached files from disk.
|
||||
Reference in New Issue
Block a user