Compare commits

...

35 Commits

Author SHA1 Message Date
957a1b9255 Changed tests to be more in line with our terms 2025-10-27 10:03:58 +01:00
afc07d4561 Fix tests: Update field names to match actual model structure (abschnitttyp field, inhalt field) 2025-10-24 17:58:54 +00:00
af06598172 Fix tests: Update Abschnitttyp to AbschnittTyp 2025-10-24 17:54:59 +00:00
4213ca60ac Add comprehensive unit tests for dokumente app 2025-10-24 17:48:08 +00:00
bf2f15fa5c removed initial test lines 2025-10-24 19:42:07 +02:00
c1eb2d7871 Deploy 0.933 2025-10-24 01:07:37 +02:00
b29e894b22 Merge branch 'feature/diagram-post-caching' into testing 2025-10-24 01:04:54 +02:00
0f096d18aa Add MEDIA_ROOT, MEDIA_URL and DIAGRAM_CACHE_DIR to docker settings 2025-10-23 23:00:14 +00:00
9b484787a4 Add media file serving for cached diagrams
- Configure MEDIA_URL/MEDIA_ROOT serving in DEBUG mode
- Separate static and media file configurations for clarity
2025-10-23 22:59:54 +00:00
8dd3b4e9af Add MEDIA_ROOT, MEDIA_URL and DIAGRAM_CACHE_DIR settings 2025-10-23 22:59:40 +00:00
0d0199ca62 Wrong branching point - corrected 2025-10-24 00:54:10 +02:00
5f58d660c0 Merge branch 'feature/diagram-post-caching' into testing 2025-10-24 00:47:18 +02:00
e84f25ca1d New DB 2025-10-24 00:37:40 +02:00
dfb8eeef97 Deploy 931 (with diagram branch in container) - readiness probes off temporarily 2025-10-24 00:30:22 +02:00
0225fb3396 Deploy 931 (with diagram branch in container) - 2nd attempt 2025-10-24 00:27:07 +02:00
7377ddaea3 Deploy 931 (with diagram branch in container) 2025-10-24 00:20:34 +02:00
67c393ecf1 Add documentation for diagram POST caching feature 2025-10-23 22:06:30 +00:00
dbb3ecd5bf Add diagram cache directory to gitignore 2025-10-23 22:06:07 +00:00
966cd46228 Add management command to clear diagram cache
Usage:
  python manage.py clear_diagram_cache
  python manage.py clear_diagram_cache --type plantuml
2025-10-23 22:05:36 +00:00
1ee9b3c46f Add commands package 2025-10-23 22:05:26 +00:00
8f57f5fc5b Add management module 2025-10-23 22:05:21 +00:00
cd7195b3aa Update diagram rendering to use POST with caching
- Replace URL-encoded GET approach with POST requests
- Use local filesystem cache for generated diagrams
- Add error handling with fallback message
- Serve diagrams from MEDIA_URL instead of proxy
2025-10-23 22:05:13 +00:00
020dff0871 Add diagram caching module with POST support
- Implement content-based hashing for cache keys
- POST diagram content to Kroki server instead of URL encoding
- Store generated SVGs in local filesystem cache
- Add cache clearing functionality
2025-10-23 22:04:19 +00:00
1dbdbc7f3c Initialize diagramm_proxy module 2025-10-23 22:03:39 +00:00
4d1232b764 Tests - not runnable yet. 2025-10-23 16:35:31 +02:00
fe2e02934a README added, first try at signing commits. 2025-10-23 09:27:25 +02:00
add1a88ce4 README added, first try at signing commits. 2025-10-23 09:26:44 +02:00
3c23918e1f No-clobber back on for database in ArgoCD 2025-10-23 09:01:45 +02:00
fa0a2a9df9 new data structure due to renaming - clobbering database temporarily in init-container 2025-10-23 08:25:12 +00:00
Adrian A. Baumann
9feaf6686f Deploy 930 2025-10-23 09:42:41 +02:00
7087be672a Added "Geltungsbereich" back into search function and corrected it; Changed "standards" page to "dokumente" internally 2025-10-23 09:35:23 +02:00
Adrian A. Baumann
969141601d Merge branch 'rename_standards' into development 2025-10-22 15:14:48 +02:00
Adrian A. Baumann
b391ab0ef6 >Renamed app "standards" to "dokumente" - finally working as expected. 2025-10-22 15:08:42 +02:00
4de2ad38c5 Readme added 2025-10-21 23:46:17 +02:00
d46d937e93 Geltungsbereich removed from search scope for now. Maybe check Haystack or other dedicated search engines 2025-10-21 16:16:42 +02:00
39 changed files with 814 additions and 58 deletions

3
.gitignore vendored
View File

@@ -10,3 +10,6 @@ keys/
.idea/
*.kate-swp
# Diagram cache directory
media/diagram_cache/

View File

@@ -0,0 +1,105 @@
# Diagram POST Caching Implementation
This feature replaces the URL-encoded GET approach for diagram generation with POST requests and local filesystem caching.
## Changes Overview
### New Files
- `diagramm_proxy/__init__.py` - Module initialization
- `diagramm_proxy/diagram_cache.py` - Caching logic and POST request handling
- `abschnitte/management/commands/clear_diagram_cache.py` - Management command for cache clearing
### Modified Files
- `abschnitte/utils.py` - Updated `render_textabschnitte()` to use caching
- `.gitignore` - Added cache directory exclusion
## Configuration Required
Add to your Django settings file (e.g., `VorgabenUI/settings.py`):
```python
# Diagram cache settings
DIAGRAM_CACHE_DIR = 'diagram_cache' # relative to MEDIA_ROOT
# Ensure MEDIA_ROOT and MEDIA_URL are configured
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
MEDIA_URL = '/media/'
```
### URL Configuration
Ensure media files are served in development. In your main `urls.py`:
```python
from django.conf import settings
from django.conf.urls.static import static
# ... existing urlpatterns ...
# Serve media files in development
if settings.DEBUG:
urlpatterns += static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)
```
## How It Works
1. When a diagram is rendered, the system computes a SHA256 hash of the diagram content
2. It checks if a cached SVG exists for that hash
3. If cached: serves the existing file
4. If not cached: POSTs content to Kroki server, saves the response, and serves it
5. Diagrams are served from `MEDIA_URL/diagram_cache/{type}/{hash}.svg`
## Benefits
- **No URL length limitations** - Content is POSTed instead of URL-encoded
- **Improved performance** - Cached diagrams are served directly from filesystem
- **Reduced server load** - Kroki server is only called once per unique diagram
- **Persistent cache** - Survives application restarts
- **Better error handling** - Graceful fallback on generation failures
## Usage
### Viewing Diagrams
No changes required - diagrams will be automatically cached on first render.
### Clearing Cache
Clear all cached diagrams:
```bash
python manage.py clear_diagram_cache
```
Clear diagrams of a specific type:
```bash
python manage.py clear_diagram_cache --type plantuml
python manage.py clear_diagram_cache --type mermaid
```
## Testing
1. Create or view a page with diagrams
2. Verify diagrams render correctly
3. Check that `media/diagram_cache/` directory is created with cached SVGs
4. Refresh the page - second load should be faster (cache hit)
5. Check logs for cache hit/miss messages
6. Test cache clearing command
## Migration Notes
- Existing diagrams will be regenerated on first view after deployment
- The old URL-based approach is completely replaced
- No database migrations needed
- Ensure `requests` library is installed (already in requirements.txt)
## Troubleshooting
### Diagrams not rendering
- Check that MEDIA_ROOT and MEDIA_URL are configured correctly
- Verify Kroki server is accessible at `http://svckroki:8000`
- Check application logs for error messages
- Ensure media directory is writable
### Cache not working
- Verify Django storage configuration
- Check file permissions on media/diagram_cache directory
- Review logs for cache-related errors

View File

@@ -1 +1,6 @@
# VDeployment2
# vgui-cicd
There are examples for importing text in the "Documentation"-directory. Actual documentation follows.
Documentation on Confluence so far.
This commit should be signed.

View File

@@ -38,7 +38,7 @@ INSTALLED_APPS = [
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'standards',
'dokumente',
'abschnitte',
'stichworte',
'mptt',
@@ -126,6 +126,13 @@ STATICFILES_DIRS= (
os.path.join(BASE_DIR,"static"),
)
# Media files (User-uploaded content)
MEDIA_URL = '/media/'
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
# Diagram cache settings
DIAGRAM_CACHE_DIR = 'diagram_cache' # relative to MEDIA_ROOT
# Default primary key field type
# https://docs.djangoproject.com/en/5.2/ref/settings/#default-auto-field

View File

@@ -43,7 +43,7 @@ INSTALLED_APPS = [
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'standards',
'dokumente',
'abschnitte',
'stichworte',
'referenzen',
@@ -139,6 +139,13 @@ STATICFILES_DIRS= (
os.path.join(BASE_DIR,"static"),
)
# Media files (User-uploaded content)
MEDIA_URL = '/media/'
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
# Diagram cache settings
DIAGRAM_CACHE_DIR = 'diagram_cache' # relative to MEDIA_ROOT
# Default primary key field type
# https://docs.djangoproject.com/en/5.2/ref/settings/#default-auto-field

View File

@@ -19,7 +19,7 @@ from django.urls import include, path, re_path
from django.conf import settings
from django.conf.urls.static import static
from diagramm_proxy.views import DiagrammProxyView
import standards.views
import dokumente.views
import pages.views
import referenzen.views
@@ -28,11 +28,17 @@ admin.site.site_header="Autorenumgebung"
urlpatterns = [
path('',pages.views.startseite),
path('search/',pages.views.search),
path('standards/', include("standards.urls")),
path('dokumente/', include("dokumente.urls")),
path('autorenumgebung/', admin.site.urls),
path('stichworte/', include("stichworte.urls")),
path('referenzen/', referenzen.views.tree, name="referenz_tree"),
path('referenzen/<str:refid>/', referenzen.views.detail, name="referenz_detail"),
re_path(r'^diagramm/(?P<path>.*)$', DiagrammProxyView.as_view()),
] + static(settings.STATIC_URL, document_root=settings.STATIC_ROOT)
]
# Serve static files
urlpatterns += static(settings.STATIC_URL, document_root=settings.STATIC_ROOT)
# Serve media files (including cached diagrams)
if settings.DEBUG:
urlpatterns += static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)

View File

@@ -0,0 +1 @@
# Management commands

View File

@@ -0,0 +1 @@
# Commands package

View File

@@ -0,0 +1,23 @@
from django.core.management.base import BaseCommand
from diagramm_proxy.diagram_cache import clear_cache
class Command(BaseCommand):
help = 'Clear cached diagrams'
def add_arguments(self, parser):
parser.add_argument(
'--type',
type=str,
help='Diagram type to clear (e.g., plantuml, mermaid)',
)
def handle(self, *args, **options):
diagram_type = options.get('type')
if diagram_type:
self.stdout.write(f'Clearing cache for {diagram_type}...')
clear_cache(diagram_type)
else:
self.stdout.write('Clearing all diagram caches...')
clear_cache()
self.stdout.write(self.style.SUCCESS('Cache cleared successfully'))

View File

@@ -3,6 +3,10 @@ import base64
import zlib
import re
from textwrap import dedent
from django.conf import settings
# Import the caching function
from diagramm_proxy.diagram_cache import get_cached_diagram
DIAGRAMMSERVER="/diagramm"
@@ -25,15 +29,23 @@ def render_textabschnitte(queryset):
elif typ == "tabelle":
html = md_table_to_html(inhalt)
elif typ == "diagramm":
temp=inhalt.splitlines()
diagramtype=temp.pop(0)
diagramoptions='width="100%"'
if temp[0][0:6].lower() == "option":
diagramoptions=temp.pop(0).split(":",1)[1]
rest="\n".join(temp)
html = '<p><img '+diagramoptions+' src="'+DIAGRAMMSERVER+"/"+diagramtype+"/svg/"
html += base64.urlsafe_b64encode(zlib.compress(rest.encode("utf-8"),9)).decode()
html += '"></p>'
temp = inhalt.splitlines()
diagramtype = temp.pop(0)
diagramoptions = 'width="100%"'
if temp and temp[0][0:6].lower() == "option":
diagramoptions = temp.pop(0).split(":", 1)[1]
rest = "\n".join(temp)
# Use caching instead of URL encoding
try:
cache_path = get_cached_diagram(diagramtype, rest)
# Generate URL to serve from media/static
diagram_url = settings.MEDIA_URL + cache_path
html = f'<p><img {diagramoptions} src="{diagram_url}"></p>'
except Exception as e:
# Fallback to error message
html = f'<p class="text-danger">Error generating diagram: {str(e)}</p>'
elif typ == "code":
html = "<pre><code>"
html += markdown(inhalt, extensions=['tables', 'attr_list'])

View File

@@ -18,14 +18,14 @@ spec:
fsGroupChangePolicy: "OnRootMismatch"
initContainers:
- name: loader
image: git.baumann.gr/adebaumann/vgui-data-loader:0.5
image: git.baumann.gr/adebaumann/vui-data-loader:0.8
command: [ "sh","-c","cp -n preload/preload.sqlite3 /data/db.sqlite3; chown -R 999:999 /data; ls -la /data; sleep 10; exit 0" ]
volumeMounts:
- name: data
mountPath: /data
containers:
- name: web
image: git.baumann.gr/adebaumann/vui:0.929
image: git.baumann.gr/adebaumann/vui:0.933
imagePullPolicy: Always
ports:
- containerPort: 8000

Binary file not shown.

Binary file not shown.

View File

@@ -0,0 +1 @@
# Diagram proxy module

View File

@@ -0,0 +1,91 @@
import hashlib
import os
import requests
from pathlib import Path
from django.conf import settings
from django.core.files.storage import default_storage
from django.core.files.base import ContentFile
import logging
logger = logging.getLogger(__name__)
# Configure cache directory
CACHE_DIR = getattr(settings, 'DIAGRAM_CACHE_DIR', 'diagram_cache')
KROKI_UPSTREAM = "http://svckroki:8000"
def get_cache_path(diagram_type, content_hash):
"""Generate cache file path for a diagram."""
return os.path.join(CACHE_DIR, diagram_type, f"{content_hash}.svg")
def compute_hash(content):
"""Compute SHA256 hash of diagram content."""
return hashlib.sha256(content.encode('utf-8')).hexdigest()
def get_cached_diagram(diagram_type, diagram_content):
"""
Retrieve diagram from cache or generate it via POST.
Args:
diagram_type: Type of diagram (e.g., 'plantuml', 'mermaid')
diagram_content: Raw diagram content
Returns:
Path to cached diagram file (relative to MEDIA_ROOT)
"""
content_hash = compute_hash(diagram_content)
cache_path = get_cache_path(diagram_type, content_hash)
# Check if diagram exists in cache
if default_storage.exists(cache_path):
logger.debug(f"Cache hit for {diagram_type} diagram: {content_hash[:8]}")
return cache_path
# Generate diagram via POST request
logger.info(f"Cache miss for {diagram_type} diagram: {content_hash[:8]}, generating...")
try:
url = f"{KROKI_UPSTREAM}/{diagram_type}/svg"
response = requests.post(
url,
data=diagram_content.encode('utf-8'),
headers={'Content-Type': 'text/plain'},
timeout=30
)
response.raise_for_status()
# Ensure cache directory exists
cache_dir = os.path.dirname(default_storage.path(cache_path))
os.makedirs(cache_dir, exist_ok=True)
# Save to cache
default_storage.save(cache_path, ContentFile(response.content))
logger.info(f"Diagram cached successfully: {cache_path}")
return cache_path
except requests.RequestException as e:
logger.error(f"Error generating diagram: {e}")
raise
def clear_cache(diagram_type=None):
"""
Clear cached diagrams.
Args:
diagram_type: If specified, only clear diagrams of this type
"""
if diagram_type:
cache_path = os.path.join(CACHE_DIR, diagram_type)
else:
cache_path = CACHE_DIR
if default_storage.exists(cache_path):
full_path = default_storage.path(cache_path)
# Walk through and delete files
for root, dirs, files in os.walk(full_path):
for file in files:
file_path = os.path.join(root, file)
try:
os.remove(file_path)
logger.info(f"Deleted cached diagram: {file_path}")
except OSError as e:
logger.error(f"Error deleting {file_path}: {e}")

View File

@@ -3,4 +3,4 @@ from django.apps import AppConfig
class standardsConfig(AppConfig):
default_auto_field = 'django.db.models.BigAutoField'
name = 'standards'
name = 'dokumente'

View File

@@ -4,7 +4,7 @@ from pathlib import Path
from django.core.management.base import BaseCommand, CommandError
from django.utils import timezone
from standards.models import (
from dokumente.models import (
Dokument,
Dokumententyp,
Thema,

View File

@@ -53,7 +53,7 @@ class Migration(migrations.Migration):
('rght', models.PositiveIntegerField(editable=False)),
('tree_id', models.PositiveIntegerField(db_index=True, editable=False)),
('level', models.PositiveIntegerField(editable=False)),
('oberreferenz', mptt.fields.TreeForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.SET_NULL, related_name='unterreferenzen', to='standards.referenz')),
('oberreferenz', mptt.fields.TreeForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.SET_NULL, related_name='unterreferenzen', to='dokumente.referenz')),
],
options={
'verbose_name_plural': 'Referenzen',
@@ -65,7 +65,7 @@ class Migration(migrations.Migration):
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('inhalt', models.TextField(blank=True, null=True)),
('abschnitttyp', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.PROTECT, to='abschnitte.abschnitttyp')),
('erklaerung', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='standards.referenz')),
('erklaerung', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='dokumente.referenz')),
],
options={
'verbose_name': 'Erklärung',
@@ -80,9 +80,9 @@ class Migration(migrations.Migration):
('gueltigkeit_bis', models.DateField(blank=True, null=True)),
('signatur_cso', models.CharField(blank=True, max_length=255)),
('anhaenge', models.TextField(blank=True)),
('autoren', models.ManyToManyField(related_name='verfasste_dokumente', to='standards.person')),
('dokumententyp', models.ForeignKey(on_delete=django.db.models.deletion.PROTECT, to='standards.dokumententyp')),
('pruefende', models.ManyToManyField(related_name='gepruefte_dokumente', to='standards.person')),
('autoren', models.ManyToManyField(related_name='verfasste_dokumente', to='dokumente.person')),
('dokumententyp', models.ForeignKey(on_delete=django.db.models.deletion.PROTECT, to='dokumente.dokumententyp')),
('pruefende', models.ManyToManyField(related_name='gepruefte_dokumente', to='dokumente.person')),
],
options={
'verbose_name': 'Standard',
@@ -95,7 +95,7 @@ class Migration(migrations.Migration):
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('inhalt', models.TextField(blank=True, null=True)),
('abschnitttyp', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.PROTECT, to='abschnitte.abschnitttyp')),
('geltungsbereich', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='standards.standard')),
('geltungsbereich', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='dokumente.standard')),
],
options={
'verbose_name': 'Geltungsbereichs-Abschnitt',
@@ -108,8 +108,8 @@ class Migration(migrations.Migration):
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('datum', models.DateField()),
('aenderung', models.TextField()),
('autoren', models.ManyToManyField(to='standards.person')),
('dokument', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='changelog', to='standards.standard')),
('autoren', models.ManyToManyField(to='dokumente.person')),
('dokument', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='changelog', to='dokumente.standard')),
],
),
migrations.CreateModel(
@@ -120,10 +120,10 @@ class Migration(migrations.Migration):
('titel', models.CharField(max_length=255)),
('gueltigkeit_von', models.DateField()),
('gueltigkeit_bis', models.DateField(blank=True, null=True)),
('dokument', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='vorgaben', to='standards.standard')),
('referenzen', models.ManyToManyField(blank=True, to='standards.referenz')),
('dokument', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='vorgaben', to='dokumente.standard')),
('referenzen', models.ManyToManyField(blank=True, to='dokumente.referenz')),
('stichworte', models.ManyToManyField(blank=True, to='stichworte.stichwort')),
('thema', models.ForeignKey(on_delete=django.db.models.deletion.PROTECT, to='standards.thema')),
('thema', models.ForeignKey(on_delete=django.db.models.deletion.PROTECT, to='dokumente.thema')),
],
options={
'verbose_name_plural': 'Vorgaben',
@@ -134,7 +134,7 @@ class Migration(migrations.Migration):
fields=[
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('frage', models.CharField(max_length=255)),
('vorgabe', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='checklistenfragen', to='standards.vorgabe')),
('vorgabe', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='checklistenfragen', to='dokumente.vorgabe')),
],
options={
'verbose_name_plural': 'Fragen für Checkliste',
@@ -145,7 +145,7 @@ class Migration(migrations.Migration):
fields=[
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('inhalt', models.TextField(blank=True, null=True)),
('abschnitt', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='standards.vorgabe')),
('abschnitt', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='dokumente.vorgabe')),
('abschnitttyp', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.PROTECT, to='abschnitte.abschnitttyp')),
],
options={
@@ -158,7 +158,7 @@ class Migration(migrations.Migration):
fields=[
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('inhalt', models.TextField(blank=True, null=True)),
('abschnitt', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='standards.vorgabe')),
('abschnitt', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='dokumente.vorgabe')),
('abschnitttyp', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.PROTECT, to='abschnitte.abschnitttyp')),
],
options={

View File

@@ -8,7 +8,7 @@ class Migration(migrations.Migration):
dependencies = [
('abschnitte', '0001_initial'),
('standards', '0001_initial'),
('dokumente', '0001_initial'),
]
operations = [
@@ -18,7 +18,7 @@ class Migration(migrations.Migration):
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('inhalt', models.TextField(blank=True, null=True)),
('abschnitttyp', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.PROTECT, to='abschnitte.abschnitttyp')),
('einleitung', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='standards.standard')),
('einleitung', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='dokumente.standard')),
],
options={
'verbose_name': 'Einleitungs-Abschnitt',

View File

@@ -6,7 +6,7 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('standards', '0002_einleitung'),
('dokumente', '0002_einleitung'),
]
operations = [

View File

@@ -7,7 +7,7 @@ class Migration(migrations.Migration):
dependencies = [
('referenzen', '0001_initial'),
('standards', '0003_einleitung_order_geltungsbereich_order_and_more'),
('dokumente', '0003_einleitung_order_geltungsbereich_order_and_more'),
]
operations = [

View File

@@ -7,7 +7,7 @@ class Migration(migrations.Migration):
dependencies = [
('rollen', '0001_initial'),
('standards', '0004_remove_referenzerklaerung_erklaerung_and_more'),
('dokumente', '0004_remove_referenzerklaerung_erklaerung_and_more'),
]
operations = [

View File

@@ -6,7 +6,7 @@ from django.db import migrations
class Migration(migrations.Migration):
dependencies = [
('standards', '0005_vorgabe_relevanz'),
('dokumente', '0005_vorgabe_relevanz'),
]
operations = [

View File

@@ -6,7 +6,7 @@ from django.db import migrations
class Migration(migrations.Migration):
dependencies = [
('standards', '0006_rename_standard_dokument_alter_dokument_options'),
('dokumente', '0006_rename_standard_dokument_alter_dokument_options'),
]
operations = [

View File

@@ -2,10 +2,10 @@
{% block content %}
<h1>Standards Informatiksicherheit</h1>
<ul>
{% for standard in standards %}
{% for dokument in dokumente %}
<li>
<a href="{% url 'standard_detail' nummer=standard.nummer %}">
{{ standard.nummer }} {{ standard.name }}
<a href="{% url 'standard_detail' nummer=dokument.nummer %}">
{{ dokument.nummer }} {{ dokument.name }}
</a>
</li>
{% endfor %}

497
dokumente/tests.py Normal file
View File

@@ -0,0 +1,497 @@
from django.test import TestCase, Client
from django.urls import reverse
from datetime import date, timedelta
from .models import (
Dokumententyp, Person, Thema, Dokument, Vorgabe,
VorgabeLangtext, VorgabeKurztext, Geltungsbereich,
Einleitung, Checklistenfrage, Changelog
)
from abschnitte.models import AbschnittTyp
from referenzen.models import Referenz
from stichworte.models import Stichwort
from rollen.models import Rolle
class DokumententypModelTest(TestCase):
"""Test cases for Dokumententyp model"""
def setUp(self):
self.dokumententyp = Dokumententyp.objects.create(
name="Standard IT-Sicherheit",
verantwortliche_ve="SR-SUR-SEC"
)
def test_dokumententyp_creation(self):
"""Test that Dokumententyp is created correctly"""
self.assertEqual(self.dokumententyp.name, "Standard IT-Sicherheit")
self.assertEqual(self.dokumententyp.verantwortliche_ve, "SR-SUR-SEC")
def test_dokumententyp_str(self):
"""Test string representation of Dokumententyp"""
self.assertEqual(str(self.dokumententyp), "Standard IT-Sicherheit")
def test_dokumententyp_verbose_name(self):
"""Test verbose name"""
self.assertEqual(
Dokumententyp._meta.verbose_name,
"Dokumententyp"
)
self.assertEqual(
Dokumententyp._meta.verbose_name_plural,
"Dokumententypen"
)
class PersonModelTest(TestCase):
"""Test cases for Person model"""
def setUp(self):
self.person = Person.objects.create(
name="Max Mustermann",
funktion="Manager"
)
def test_person_creation(self):
"""Test that Person is created correctly"""
self.assertEqual(self.person.name, "Max Mustermann")
self.assertEqual(self.person.funktion, "Manager")
def test_person_str(self):
"""Test string representation of Person"""
self.assertEqual(str(self.person), "Max Mustermann")
def test_person_verbose_name_plural(self):
"""Test verbose name plural"""
self.assertEqual(
Person._meta.verbose_name_plural,
"Personen"
)
class ThemaModelTest(TestCase):
"""Test cases for Thema model"""
def setUp(self):
self.thema = Thema.objects.create(
name="Security",
erklaerung="Security related topics"
)
def test_thema_creation(self):
"""Test that Thema is created correctly"""
self.assertEqual(self.thema.name, "Security")
self.assertEqual(self.thema.erklaerung, "Security related topics")
def test_thema_str(self):
"""Test string representation of Thema"""
self.assertEqual(str(self.thema), "Security")
def test_thema_blank_erklaerung(self):
"""Test that erklaerung can be blank"""
thema = Thema.objects.create(name="Testing")
self.assertEqual(thema.erklaerung, "")
class DokumentModelTest(TestCase):
"""Test cases for Dokument model"""
def setUp(self):
self.dokumententyp = Dokumententyp.objects.create(
name="Policy",
verantwortliche_ve="Legal"
)
self.autor = Person.objects.create(
name="John Doe",
funktion="Author"
)
self.pruefer = Person.objects.create(
name="Jane Smith",
funktion="Reviewer"
)
self.dokument = Dokument.objects.create(
nummer="DOC-001",
dokumententyp=self.dokumententyp,
name="Security Policy",
gueltigkeit_von=date.today(),
signatur_cso="CSO-123",
anhaenge="Appendix A, B"
)
self.dokument.autoren.add(self.autor)
self.dokument.pruefende.add(self.pruefer)
def test_dokument_creation(self):
"""Test that Dokument is created correctly"""
self.assertEqual(self.dokument.nummer, "DOC-001")
self.assertEqual(self.dokument.name, "Security Policy")
self.assertEqual(self.dokument.dokumententyp, self.dokumententyp)
def test_dokument_str(self):
"""Test string representation of Dokument"""
self.assertEqual(str(self.dokument), "DOC-001 Security Policy")
def test_dokument_many_to_many_relationships(self):
"""Test many-to-many relationships"""
self.assertIn(self.autor, self.dokument.autoren.all())
self.assertIn(self.pruefer, self.dokument.pruefende.all())
def test_dokument_optional_fields(self):
"""Test optional fields can be None or blank"""
dokument = Dokument.objects.create(
nummer="DOC-002",
dokumententyp=self.dokumententyp,
name="Test Document"
)
self.assertIsNone(dokument.gueltigkeit_von)
self.assertIsNone(dokument.gueltigkeit_bis)
self.assertEqual(dokument.signatur_cso, "")
self.assertEqual(dokument.anhaenge, "")
class VorgabeModelTest(TestCase):
"""Test cases for Vorgabe model"""
def setUp(self):
self.dokumententyp = Dokumententyp.objects.create(
name="Standard IT-Sicherheit",
verantwortliche_ve="SR-SUR-SEC"
)
self.dokument = Dokument.objects.create(
nummer="R01234",
dokumententyp=self.dokumententyp,
name="IT Standard"
)
self.thema = Thema.objects.create(name="Security")
self.vorgabe = Vorgabe.objects.create(
nummer=1,
dokument=self.dokument,
thema=self.thema,
titel="Password Requirements",
gueltigkeit_von=date.today() - timedelta(days=30)
)
def test_vorgabe_creation(self):
"""Test that Vorgabe is created correctly"""
self.assertEqual(self.vorgabe.nummer, 1)
self.assertEqual(self.vorgabe.dokument, self.dokument)
self.assertEqual(self.vorgabe.thema, self.thema)
def test_vorgabennummer(self):
"""Test Vorgabennummer generation"""
expected = "R01234.S.1"
self.assertEqual(self.vorgabe.Vorgabennummer(), expected)
def test_vorgabe_str(self):
"""Test string representation of Vorgabe"""
expected = "R01234.S.1: Password Requirements"
self.assertEqual(str(self.vorgabe), expected)
def test_get_status_active(self):
"""Test get_status returns 'active' for current vorgabe"""
status = self.vorgabe.get_status()
self.assertEqual(status, "active")
def test_get_status_future(self):
"""Test get_status returns 'future' for future vorgabe"""
future_vorgabe = Vorgabe.objects.create(
nummer=2,
dokument=self.dokument,
thema=self.thema,
titel="Future Requirement",
gueltigkeit_von=date.today() + timedelta(days=30)
)
status = future_vorgabe.get_status()
self.assertEqual(status, "future")
def test_get_status_expired(self):
"""Test get_status returns 'expired' for expired vorgabe"""
expired_vorgabe = Vorgabe.objects.create(
nummer=3,
dokument=self.dokument,
thema=self.thema,
titel="Old Requirement",
gueltigkeit_von=date.today() - timedelta(days=60),
gueltigkeit_bis=date.today() - timedelta(days=10)
)
status = expired_vorgabe.get_status()
self.assertEqual(status, "expired")
def test_get_status_verbose(self):
"""Test get_status with verbose=True"""
future_vorgabe = Vorgabe.objects.create(
nummer=4,
dokument=self.dokument,
thema=self.thema,
titel="Future Test",
gueltigkeit_von=date.today() + timedelta(days=10)
)
status = future_vorgabe.get_status(verbose=True)
self.assertIn("Ist erst ab dem", status)
self.assertIn("in Kraft", status)
def test_get_status_with_custom_check_date(self):
"""Test get_status with custom check_date"""
vorgabe = Vorgabe.objects.create(
nummer=5,
dokument=self.dokument,
thema=self.thema,
titel="Test Requirement",
gueltigkeit_von=date.today() - timedelta(days=60),
gueltigkeit_bis=date.today() - timedelta(days=10)
)
check_date = date.today() - timedelta(days=30)
status = vorgabe.get_status(check_date=check_date)
self.assertEqual(status, "active")
class VorgabeTextAbschnitteTest(TestCase):
"""Test cases for VorgabeLangtext and VorgabeKurztext"""
def setUp(self):
self.dokumententyp = Dokumententyp.objects.create(
name="Standard IT-Sicherheit",
verantwortliche_ve="SR-SUR-SEC"
)
self.dokument = Dokument.objects.create(
nummer="R01234",
dokumententyp=self.dokumententyp,
name="Test Standard"
)
self.thema = Thema.objects.create(name="Testing")
self.vorgabe = Vorgabe.objects.create(
nummer=1,
dokument=self.dokument,
thema=self.thema,
titel="Test Vorgabe",
gueltigkeit_von=date.today()
)
self.abschnitttyp = AbschnittTyp.objects.create(
abschnitttyp="Paragraph"
)
def test_vorgabe_langtext_creation(self):
"""Test VorgabeLangtext creation"""
langtext = VorgabeLangtext.objects.create(
abschnitt=self.vorgabe,
abschnitttyp=self.abschnitttyp,
inhalt="This is a long text description",
order=1
)
self.assertEqual(langtext.abschnitt, self.vorgabe)
self.assertEqual(langtext.inhalt, "This is a long text description")
def test_vorgabe_kurztext_creation(self):
"""Test VorgabeKurztext creation"""
kurztext = VorgabeKurztext.objects.create(
abschnitt=self.vorgabe,
abschnitttyp=self.abschnitttyp,
inhalt="Short summary",
order=1
)
self.assertEqual(kurztext.abschnitt, self.vorgabe)
self.assertEqual(kurztext.inhalt, "Short summary")
class DokumentTextAbschnitteTest(TestCase):
"""Test cases for Geltungsbereich and Einleitung"""
def setUp(self):
self.dokumententyp = Dokumententyp.objects.create(
name="Policy",
verantwortliche_ve="Legal"
)
self.dokument = Dokument.objects.create(
nummer="POL-001",
dokumententyp=self.dokumententyp,
name="Test Policy"
)
self.abschnitttyp = AbschnittTyp.objects.create(
abschnitttyp="Paragraph"
)
def test_geltungsbereich_creation(self):
"""Test Geltungsbereich creation"""
geltungsbereich = Geltungsbereich.objects.create(
geltungsbereich=self.dokument,
abschnitttyp=self.abschnitttyp,
inhalt="Applies to all employees",
order=1
)
self.assertEqual(geltungsbereich.geltungsbereich, self.dokument)
self.assertEqual(geltungsbereich.inhalt, "Applies to all employees")
def test_einleitung_creation(self):
"""Test Einleitung creation"""
einleitung = Einleitung.objects.create(
einleitung=self.dokument,
abschnitttyp=self.abschnitttyp,
inhalt="This document defines...",
order=1
)
self.assertEqual(einleitung.einleitung, self.dokument)
self.assertEqual(einleitung.inhalt, "This document defines...")
class ChecklistenfrageModelTest(TestCase):
"""Test cases for Checklistenfrage model"""
def setUp(self):
self.dokumententyp = Dokumententyp.objects.create(
name="Standard IT-Sicherheit",
verantwortliche_ve="QA"
)
self.dokument = Dokument.objects.create(
nummer="QA-001",
dokumententyp=self.dokumententyp,
name="QA Standard"
)
self.thema = Thema.objects.create(name="Quality")
self.vorgabe = Vorgabe.objects.create(
nummer=1,
dokument=self.dokument,
thema=self.thema,
titel="Quality Check",
gueltigkeit_von=date.today()
)
self.frage = Checklistenfrage.objects.create(
vorgabe=self.vorgabe,
frage="Have all tests passed?"
)
def test_checklistenfrage_creation(self):
"""Test Checklistenfrage creation"""
self.assertEqual(self.frage.vorgabe, self.vorgabe)
self.assertEqual(self.frage.frage, "Have all tests passed?")
def test_checklistenfrage_str(self):
"""Test string representation"""
self.assertEqual(str(self.frage), "Have all tests passed?")
def test_checklistenfrage_related_name(self):
"""Test related name works correctly"""
self.assertIn(self.frage, self.vorgabe.checklistenfragen.all())
class ChangelogModelTest(TestCase):
"""Test cases for Changelog model"""
def setUp(self):
self.dokumententyp = Dokumententyp.objects.create(
name="Standard IT-Sicherheit",
verantwortliche_ve="SR-SUR-SEC"
)
self.dokument = Dokument.objects.create(
nummer="R01234",
dokumententyp=self.dokumententyp,
name="IT Standard"
)
self.autor = Person.objects.create(
name="John Doe",
funktion="Developer"
)
self.changelog = Changelog.objects.create(
dokument=self.dokument,
datum=date.today(),
aenderung="Initial version"
)
self.changelog.autoren.add(self.autor)
def test_changelog_creation(self):
"""Test Changelog creation"""
self.assertEqual(self.changelog.dokument, self.dokument)
self.assertEqual(self.changelog.aenderung, "Initial version")
self.assertIn(self.autor, self.changelog.autoren.all())
def test_changelog_str(self):
"""Test string representation"""
expected = f"{date.today()} R01234"
self.assertEqual(str(self.changelog), expected)
class ViewsTestCase(TestCase):
"""Test cases for views"""
def setUp(self):
self.client = Client()
self.dokumententyp = Dokumententyp.objects.create(
name="Standard IT-Sicherheit",
verantwortliche_ve="SR-SUR-SEC"
)
self.dokument = Dokument.objects.create(
nummer="R01234",
dokumententyp=self.dokumententyp,
name="Test Standard"
)
self.thema = Thema.objects.create(name="Testing")
self.vorgabe = Vorgabe.objects.create(
nummer=1,
dokument=self.dokument,
thema=self.thema,
titel="Test Requirement",
gueltigkeit_von=date.today()
)
self.abschnitttyp = AbschnittTyp.objects.create(
abschnitttyp="Paragraph"
)
def test_standard_list_view(self):
"""Test standard_list view"""
response = self.client.get(reverse('standard_list'))
self.assertEqual(response.status_code, 200)
self.assertContains(response, "R01234")
self.assertIn('dokumente', response.context)
def test_standard_detail_view(self):
"""Test standard_detail view"""
response = self.client.get(
reverse('standard_detail', kwargs={'nummer': 'R01234'})
)
self.assertEqual(response.status_code, 200)
self.assertIn('standard', response.context)
self.assertIn('vorgaben', response.context)
self.assertEqual(response.context['standard'], self.dokument)
def test_standard_detail_view_404(self):
"""Test standard_detail view returns 404 for non-existent document"""
response = self.client.get(
reverse('standard_detail', kwargs={'nummer': 'NONEXISTENT'})
)
self.assertEqual(response.status_code, 404)
def test_standard_checkliste_view(self):
"""Test standard_checkliste view"""
response = self.client.get(
reverse('standard_checkliste', kwargs={'nummer': 'R01234'})
)
self.assertEqual(response.status_code, 200)
self.assertIn('standard', response.context)
self.assertIn('vorgaben', response.context)
def test_standard_history_view(self):
"""Test standard_detail with history (check_date)"""
url = reverse('standard_history', kwargs={'nummer': 'R01234'})
response = self.client.get(url)
self.assertEqual(response.status_code, 200)
class URLPatternsTest(TestCase):
"""Test URL patterns"""
def test_standard_list_url_resolves(self):
"""Test that standard_list URL resolves correctly"""
url = reverse('standard_list')
self.assertEqual(url, '/dokumente/')
def test_standard_detail_url_resolves(self):
"""Test that standard_detail URL resolves correctly"""
url = reverse('standard_detail', kwargs={'nummer': 'TEST-001'})
self.assertEqual(url, '/dokumente/TEST-001/')
def test_standard_checkliste_url_resolves(self):
"""Test that standard_checkliste URL resolves correctly"""
url = reverse('standard_checkliste', kwargs={'nummer': 'TEST-001'})
self.assertEqual(url, '/dokumente/TEST-001/checkliste/')
def test_standard_history_url_resolves(self):
"""Test that standard_history URL resolves correctly"""
url = reverse('standard_history', kwargs={'nummer': 'TEST-001'})
self.assertEqual(url, '/dokumente/TEST-001/history/')

View File

@@ -9,9 +9,9 @@ calendar=parsedatetime.Calendar()
def standard_list(request):
standards = Dokument.objects.all()
dokumente = Dokument.objects.all()
return render(request, 'standards/standard_list.html',
{'standards': standards}
{'dokumente': dokumente}
)

View File

@@ -16,7 +16,7 @@
</button>
<div class="collapse navbar-collapse" id="navbarNavAltMarkup">
<div class="navbar-nav">
<a class="nav-item nav-link active" href="/standards">Standards</a>
<a class="nav-item nav-link active" href="/dokumente">Standards</a>
<a class="nav-item nav-link" href="/referenzen">Referenzen</a>
<a class="nav-item nav-link" href="/stichworte">Stichworte</a>
<a class="nav-item nav-link" href="/search">Suche</a>
@@ -28,6 +28,6 @@
<div class="flex-fill">{% block content %}Main Content{% endblock %}</div>
<div class="col-md-2">{% block sidebar_right %}{% endblock %}</div>
</div>
<div>VorgabenUI v0.930</div>
<div>VorgabenUI v0.931</div>
</body>
</html>

View File

@@ -3,7 +3,7 @@
<h1>Vorgaben Informatiksicherheit BIT</h1>
<h2>Aktuell erfasste Standards</h2>
<ul>
{% for standard in standards %}
{% for standard in dokumente %}
<li><a href="{% url 'standard_detail' nummer=standard.nummer %}">{{ standard }}</a></li>
{% endfor %}
</ul>

View File

@@ -1,31 +1,31 @@
from django.shortcuts import render
from abschnitte.utils import render_textabschnitte
from standards.models import Dokument, VorgabeLangtext, VorgabeKurztext, Geltungsbereich
from dokumente.models import Dokument, VorgabeLangtext, VorgabeKurztext, Geltungsbereich
from itertools import groupby
import datetime
import pprint
def startseite(request):
standards=list(Dokument.objects.all())
return render(request, 'startseite.html', {"standards":standards,})
return render(request, 'startseite.html', {"dokumente":standards,})
def search(request):
if request.method == "GET":
return render(request, 'search.html')
elif request.method == "POST":
suchbegriff=request.POST.get("q")
areas=request.POST.getlist("suchbereich[]")
result= {"all": {}}
qs = VorgabeKurztext.objects.filter(inhalt__contains=suchbegriff).exclude(abschnitt__gueltigkeit_bis__lt=datetime.date.today())
result["kurztext"] = {k: [o.abschnitt for o in g] for k, g in groupby(qs, key=lambda o: o.abschnitt.dokument)}
qs = VorgabeLangtext.objects.filter(inhalt__contains=suchbegriff).exclude(abschnitt__gueltigkeit_bis__lt=datetime.date.today())
result['langtext']= {k: [o.abschnitt for o in g] for k, g in groupby(qs, key=lambda o: o.abschnitt.dokument)}
for r in result.keys():
for s in result[r].keys():
result["all"][s] = set(result[r][s])
result["geltungsbereich"]={}
geltungsbereich=set(list([x.geltungsbereich for x in Geltungsbereich.objects.filter(inhalt__contains=suchbegriff)]))
for s in geltungsbereich:
result["geltungsbereich"][s]=render_textabschnitte(s.geltungsbereich_set.order_by("order"))
for r in result.keys():
for s in result[r].keys():
result["all"][s] = set(result[r][s])
print (result)
pprint.pp (result)
return render(request,"results.html",{"suchbegriff":suchbegriff,"resultat":result})

View File

@@ -1,3 +0,0 @@
from django.test import TestCase
# Create your tests here.