{% extends 'workflows/base_shell.html' %}
{% load static %}

{% block title %}Developer Handbook{% endblock %}



{% block extra_css %}
<link rel="stylesheet" href="{% static 'workflows/css/docs_pages.css' %}" />
{% endblock %}

{% block shell_body %}
      {% include 'workflows/includes/app_header.html' with header_show_home=1 header_inside_shell=1 %}
    <div class="page-stack">
    <div class="top">
      <h1>Developer Handbook</h1>
      <div style="display:flex; gap:8px; flex-wrap:wrap;">
        <a class="btn btn-secondary" href="/admin-tools/wiki/">Project Wiki</a>
      </div>
    </div>
    <p class="sub">Engineering runbook for development, deployment, maintenance, and extension of the current company portal deployment.</p>

    <div class="toc">
      <a href="#overview">Overview</a>
      <a href="#structure">Structure</a>
      <a href="#workflow">Workflow</a>
      <a href="#local">Local Dev</a>
      <a href="#docker">Docker</a>
      <a href="#db">Database</a>
      <a href="#guidelines">Guidelines</a>
      <a href="#translations">Translations</a>
      <a href="#pdf">PDF Pipeline</a>
      <a href="#email">Email Pipeline</a>
      <a href="#nextcloud">Nextcloud</a>
      <a href="#builders">Builders</a>
      <a href="#testing">Testing</a>
      <a href="#backup">Backup</a>
      <a href="#hosts">Hosts & Domains</a>
      <a href="#cicd">CI/CD</a>
      <a href="#deploy">Deployment</a>
      <a href="#commands">Commands</a>
      <a href="#troubleshooting">Troubleshooting</a>
      <a href="#security">Security</a>
    </div>

    <h2 id="overview">1) Overview</h2>
    <div class="box">
      <p>This handbook is for developers and maintainers. It documents the actual engineering workflow of the standalone product repository.</p>
      <ul>
        <li>Repository: <code>workdock-platform</code> (current local path; legacy compose project name may still be retained for runtime continuity)</li>
        <li>Main stack: Django + Celery + PostgreSQL + Redis + MailHog</li>
        <li>Runtime mode: Docker Compose for local development and staging-style operation</li>
        <li>Repository-level quick-start guide for future coders: <code>CONTRIBUTING.md</code></li>
      </ul>
    </div>

    <h2 id="structure">2) Repository Structure</h2>
    <ul>
      <li><code>/backend/config/</code>: Django settings, WSGI, URL config</li>
      <li><code>/backend/workflows/</code>: application logic, views, models, tasks, templates, static assets</li>
      <li><code>/backend/workflows/views.py</code>: thin route wrapper layer; most view logic now lives in split modules by domain</li>
      <li><code>/backend/workflows/models.py</code>: stable model import surface over split model modules</li>
      <li><code>/backend/workflows/tasks.py</code>: async task entrypoints; PDF and notification logic were moved into dedicated modules</li>
      <li><code>/backend/workflows/templates/workflows/base_shell.html</code>: standard page shell for new staff-facing pages</li>
      <li><code>/backend/workflows/roles.py</code>: centralized role names, capability matrix, and template permission helpers</li>
      <li>Rule: all interactive app pages should extend <code>base_shell.html</code>; do not rebuild topbar/frame logic in page-local templates.</li>
      <li><code>/backend/media/templates/</code>: PDF HTML templates and letterhead source files</li>
      <li><code>/backend/media/pdfs/</code>: generated PDF outputs on host volume</li>
      <li><code>/backend/locale/</code>: translation catalogs</li>
      <li><code>/docker-compose.yml</code>: local runtime orchestration</li>
      <li><code>/Makefile</code>: repeatable translation commands</li>
      <li><code>/.github/workflows/i18n.yml</code>: translation compile validation in CI</li>
    </ul>

    <h2 id="workflow">3) Working Model</h2>
    <div class="box">
      <h3>Branch strategy</h3>
      <ul>
        <li><code>develop</code> is the active integration branch.</li>
        <li><code>main</code> is the stable branch intended for production promotion.</li>
        <li>Feature work should start from <code>develop</code> and merge back into <code>develop</code>.</li>
      </ul>
    </div>
    <div class="box">
      <h3>Normal change flow</h3>
      <ol>
        <li>start from <code>develop</code></li>
        <li>implement the change</li>
        <li>run validation</li>
        <li>update docs if architecture or workflow changed</li>
        <li>push to GitHub and let CI run</li>
        <li>deploy from the Mac if test-server verification is needed</li>
        <li>promote <code>develop</code> into <code>main</code> when stable</li>
      </ol>
    </div>

    <h2 id="local">4) Local Development Workflow</h2>
    <h3>Start</h3>
    <pre><code>cd /path/to/workdock-platform
docker compose up -d --build</code></pre>
    <h3>Main URLs</h3>
    <ul>
      <li>App: <code>http://127.0.0.1:8088/</code></li>
      <li>MailHog: <code>http://127.0.0.1:8025/</code></li>
      <li>Health check: <code>http://127.0.0.1:8088/healthz/</code></li>
    </ul>
    <h3>Bootstrap users</h3>
    <ul>
      <li>Admin: <code>admin_test</code> / <code>admin12345</code></li>
      <li>User: <code>user_test</code> / <code>user12345</code></li>
    </ul>

    <h2 id="docker">5) Docker Operations</h2>
    <pre><code>docker compose up -d --build
docker compose restart web
docker compose restart worker
docker compose logs --no-color --tail=120 web
docker compose logs --no-color --tail=120 worker
docker compose down
docker compose down -v</code></pre>
    <div class="note">
      The source code is bind-mounted into the container. Most template/view/static changes only require a web restart, not a full rebuild. Image changes such as system packages require <code>docker compose up -d --build</code>.
    </div>

    <h2 id="db">6) Database and Migrations</h2>
    <pre><code>docker compose exec -T web python manage.py makemigrations
docker compose exec -T web python manage.py migrate
docker compose exec -T web python manage.py check</code></pre>
    <ul>
      <li>Never edit or remove historical migrations casually.</li>
      <li>When adding fields to builder-driven models, preserve fallback behavior for existing rows.</li>
      <li>Fresh boot sequence runs migrations automatically in <code>entrypoint-web.sh</code>.</li>
      <li>Do not assume database-backed admin config matches between local and deployed environments.</li>
    </ul>

    <h3>Role and Permission Model</h3>
    <ul>
      <li>Stable Django group names: <code>Platform Owner</code>, <code>Super Admin</code>, <code>Admin</code>, <code>IT Staff</code>, <code>Staff</code>.</li>
      <li>Groups are created automatically through a <code>post_migrate</code> hook in <code>workflows.signals</code>.</li>
      <li>Capability checks are centralized in <code>workflows.roles.CAPABILITIES</code>.</li>
      <li>Use <code>_require_capability(...)</code> in views instead of flat <code>is_staff</code> checks.</li>
      <li>Templates receive permission flags from <code>workflows.context_processors.role_context</code>.</li>
      <li><code>Platform Owner</code> is the product-level role. Company roles remain <code>Super Admin</code>, <code>Admin</code>, <code>IT Staff</code>, and <code>Staff</code>.</li>
      <li>User management lives at <code>/admin-tools/users/</code> and is the preferred path for normal role assignment, account activation, invitation mail dispatch, password-reset mail dispatch, and controlled user deletion.</li>
      <li>Backward-compatibility rule: authenticated legacy users with <code>is_staff=True</code> but no explicit role group currently fall back to the <code>Admin</code> capability set.</li>
      <li><code>superuser</code> accounts resolve to <code>Super Admin</code>.</li>
      <li>When adding a new operational page or action, define the capability in <code>roles.py</code>, gate the view, and hide the UI affordance when the capability is absent.</li>
    </ul>

    <h2 id="guidelines">7) Engineering Guidelines</h2>
    <div class="box">
      <h3>Core rules</h3>
      <ol>
        <li>Preserve behavior while refactoring.</li>
        <li>Prefer shared components over page-local special cases.</li>
        <li>Do not overwrite environment-specific runtime config as a side effect of code deploys.</li>
        <li>Keep code-driven behavior and data-driven behavior mentally separate.</li>
        <li>Update documentation in the same branch when operational workflow changes.</li>
      </ol>
    </div>
    <div class="box">
      <h3>Code vs data</h3>
      <ul>
        <li><strong>Code-driven:</strong> routing, permissions, deployment scripts, rendering, tasks, layout.</li>
        <li><strong>Data-driven:</strong> app registry order, branding, company config, builder config, intro checklist items, notification templates.</li>
        <li>If local UI and server UI differ, inspect runtime configuration before assuming code drift.</li>
      </ul>
    </div>

    <h2 id="translations">8) Translation Workflow</h2>
    <h3>Standard Django i18n path</h3>
    <pre><code>make i18n-update-en
make i18n-compile</code></pre>
    <p>Equivalent raw commands:</p>
    <pre><code>docker compose exec -T web django-admin makemessages -l en
docker compose exec -T web django-admin compilemessages</code></pre>
    <ul>
      <li><code>gettext</code> is installed in the Docker image.</li>
      <li>Do not use custom ad hoc <code>.mo</code> compilation anymore.</li>
      <li>Phase 1 bilingual support covers fixed UI, including the main admin-tool labels/buttons.</li>
      <li>Phase 2 covers builder-driven labels and checklist items with explicit German and English fields.</li>
      <li>Email phase covers <code>NotificationTemplate</code>, <code>NotificationRule</code>, and the welcome-email settings UI with explicit DE/EN subject/body fields.</li>
      <li>Request records now persist <code>preferred_language</code> so workflow emails can render in the submitter's active UI language with German fallback.</li>
      <li><code>preferred_language</code> is normalized in model <code>save()</code> and also has a DB default of <code>de</code>, so alternate creation paths cannot insert null values.</li>
    </ul>

    <h2 id="pdf">9) PDF Pipeline</h2>
    <ul>
      <li>PDF generation is HTML-to-PDF using <code>xhtml2pdf</code>.</li>
      <li>Letterhead overlay defaults to <code>templates.pdf</code>, but can now be replaced from Admin Apps → <code>Branding</code>.</li>
      <li>Task entrypoints live in <code>backend/workflows/tasks.py</code>.</li>
      <li>Rendering and section-building now live in <code>pdf_rendering.py</code> and <code>pdf_sections.py</code>.</li>
      <li>Fixed PDF labels/headings are rendered from task-level DE/EN text dictionaries, not hard-coded directly in request processing logic.</li>
      <li>PDF language follows the normalized request <code>preferred_language</code>, with German fallback.</li>
      <li>Key templates:
        <ul>
          <li><code>onboarding_template.html</code></li>
          <li><code>offboarding_template.html</code></li>
          <li><code>onboarding_intro_template.html</code></li>
          <li><code>onboarding_intro_session_pdf.html</code></li>
        </ul>
      </li>
    </ul>
    <div class="note">
      xhtml2pdf is sensitive to layout complexity. Keep print templates conservative and verify every structural change with a real generated PDF.
    </div>

    <h2 id="email">10) Email Pipeline</h2>
    <ul>
      <li>Notification defaults are defined in the workflow/email orchestration layer.</li>
      <li>Admin-configured overrides live in <code>NotificationTemplate</code> and <code>NotificationRule</code>.</li>
      <li>Both models now support explicit DE/EN content fields. Prefer filling both languages in the frontend integrations UI instead of embedding mixed-language copy into a single template.</li>
      <li>Welcome-email configuration under Admin Apps has the same DE/EN split and follows the same runtime language selection.</li>
      <li>The request objects passed into email tasks always carry a normalized <code>preferred_language</code> value.</li>
      <li>Mail sending uses Celery tasks and supports test mode redirection.</li>
      <li>Related helper modules include <code>email_workflows.py</code> and <code>notification_dispatch.py</code>.</li>
      <li>MailHog is the local verification path when test mode is active.</li>
    </ul>

    <h2 id="nextcloud">11) Nextcloud Integration</h2>
    <ul>
      <li>Configured from Admin Apps → Integrations.</li>
      <li>Upload logic lives in <code>backend/workflows/services.py</code>.</li>
      <li>Feature can be globally toggled without changing environment variables.</li>
      <li>Failures should degrade gracefully and not block request persistence.</li>
      <li>Remote backup currently reuses the configured Nextcloud connection, but writes into a separate backup directory configured under <code>Integrationen → Backup-Ziel</code>.</li>
      <li>Do not point remote backup at the same Nextcloud directory used for normal onboarding/offboarding document uploads.</li>
    </ul>

    <h2 id="branding">12) Branding</h2>
    <ul>
      <li>Portal-level branding is stored in the singleton model <code>PortalBranding</code>.</li>
      <li>Configured from Admin Apps → <code>Branding</code>.</li>
      <li>Current scope: portal title, company name, company domain, support email, sender display name, login subtitle, footer/legal text, logo, favicon, PDF letterhead, and primary/secondary colors.</li>
      <li>Shared header/logo rendering now uses the branding context processor instead of hardcoded customer-specific asset references.</li>
      <li>The company domain now drives onboarding/offboarding email autofill and domain validation, so new customer deployments no longer require <code>@tub.co</code> code changes.</li>
      <li>Outgoing system mail sender names are now branded through the same layer.</li>
      <li>User invitation emails and welcome-template fallbacks also use the configured branding defaults.</li>
      <li>Environment sync for text/color/company metadata is explicit:
        <pre><code>docker compose exec -T web python manage.py export_portal_deployment_config --output /tmp/portal-deployment-config.json
docker compose exec -T web python manage.py import_portal_deployment_config /tmp/portal-deployment-config.json --dry-run
docker compose exec -T web python manage.py import_portal_deployment_config /tmp/portal-deployment-config.json</code></pre>
      </li>
      <li>Uploaded assets such as logo, favicon, and PDF letterhead are intentionally not included in this sync payload.</li>
    </ul>

    <h2 id="app-registry">12b) App Registry</h2>
    <ul>
      <li>Registry definitions live in <code>workflows/app_registry.py</code>.</li>
      <li>DB overrides live in <code>PortalAppConfig</code>.</li>
      <li>The landing page now renders from registry data instead of hardcoded cards.</li>
      <li>Security remains code-based: app visibility/order is configurable, but access still depends on role capabilities in <code>roles.py</code>.</li>
      <li>Management UI: <code>/admin-tools/apps/</code> for <code>Platform Owner</code>.</li>
      <li>Environment sync is explicit, not automatic. Use:
        <pre><code>docker compose exec -T web python manage.py export_portal_app_config --output /tmp/portal-app-config.json
docker compose exec -T web python manage.py import_portal_app_config /tmp/portal-app-config.json --dry-run
docker compose exec -T web python manage.py import_portal_app_config /tmp/portal-app-config.json</code></pre>
      </li>
      <li>This is the correct path when local and server app ordering differ.</li>
    </ul>

    <h2 id="trial">12c) Trial Lifecycle</h2>
    <ul>
      <li>Deployment-level trial settings are stored in the singleton model <code>PortalTrialConfig</code>.</li>
      <li>Management UI: <code>/admin-tools/trial/</code> for <code>Platform Owner</code>.</li>
      <li>Current scope: trial enable/disable, start/end timestamps, bilingual shell banner text, production-integration restriction, and cleanup readiness.</li>
      <li>Enforcement lives in <code>workflows.middleware.TrialModeMiddleware</code>, so expiry is handled centrally instead of per-view.</li>
      <li>While trial restriction is active, service-level integration checks force Nextcloud off and E-Mail into test mode.</li>
      <li>Expired-trial cleanup is intentionally CLI-only:
        <pre><code>docker compose exec -T web python manage.py cleanup_expired_trial_workspace --yes-delete</code></pre>
      </li>
    </ul>

    <h2 id="builders">13) Builder Architecture</h2>
    <h3>Form Builder</h3>
    <ul>
      <li>Model: <code>FormFieldConfig</code> + <code>FormOption</code></li>
      <li>Controls field order, visibility, required flags, option sets, and bilingual label/help-text overrides</li>
      <li>Static definitions live in <code>form_builder_config.py</code>.</li>
      <li>Runtime behavior lives in <code>form_builder_runtime.py</code>.</li>
      <li><code>form_builder.py</code> is the stable facade import path.</li>
    </ul>
    <h3>Intro Builder</h3>
    <ul>
      <li>Model: <code>IntroChecklistItem</code></li>
      <li>Controls additional checklist items used by intro PDF and live intro checklist</li>
      <li>Now supports bilingual DE/EN labels</li>
    </ul>
    <div class="note">
      Dynamic content should use explicit DE/EN fields with German fallback, not machine translation at runtime.
    </div>
    <h3>Audit Trail</h3>
    <ul>
      <li>Model: <code>AdminAuditLog</code></li>
      <li>Purpose: record staff-side mutations that affect operations or configuration.</li>
      <li>Current hooks include builder edits, PDF generation, welcome-email actions, integration changes, mode toggles, tests, and request deletions.</li>
      <li>Staff UI page: <code>/admin-tools/audit-log/</code></li>
      <li>The current UI supports filtering by action, user, and date range. Keep filters server-side to avoid loading unbounded audit rows into the browser.</li>
      <li>Backup UI page: <code>/admin-tools/backups/</code> for create, verify, and delete actions. Keep real restore CLI-only.</li>
    </ul>

    <h2 id="testing">14) Testing and Validation</h2>
    <pre><code>docker compose exec -T web python manage.py check
docker compose exec -T web python manage.py test
docker compose exec -T web python manage.py run_staging_e2e_check</code></pre>
    <ul>
      <li>Use <code>check</code> after model/view/template changes.</li>
      <li>Use targeted shell checks for render validation when changing templates or routes.</li>
      <li>New pages should extend <code>base_shell.html</code> and keep header/frame logic out of page-local templates.</li>
      <li>Use real PDF generation tests when changing PDF templates or intro/offboarding document logic.</li>
      <li>Use the dedicated Release Checklist page as the final go/no-go runbook before shipping changes.</li>
      <li>The automated bilingual smoke tests now cover DE/EN request language capture and English email-template rendering.</li>
      <li>Onboarding and offboarding request objects now expose explicit processing state and last-error fields. Async tasks are responsible for transitioning <code>submitted → processing → completed/failed</code>.</li>
      <li>The Requests Dashboard includes a retry action for failed requests. Retries reset the error text, set the request back to <code>submitted</code>, and enqueue the appropriate Celery task again.</li>
    </ul>

    <h2 id="backup">15) Backup and Restore</h2>
    <pre><code>make backup-create
make backup-verify BACKUP_DIR=backups/backup_YYYYmmdd_HHMMSS</code></pre>
    <ul>
      <li>Backups are stored under <code>backend/backups/</code> and ignored by git.</li>
      <li>Each backup contains a PostgreSQL custom-format dump, a compressed <code>media/</code> archive, metadata, and SHA256 checksums.</li>
      <li>Backup metadata records both local availability and remote backup status.</li>
      <li>Remote backup target configuration is managed in <code>Integrationen → Backup-Ziel</code>.</li>
      <li>Current remote target support: <code>nextcloud</code> implemented, <code>s3</code> and <code>nfs</code> config-ready but not yet implemented.</li>
      <li>Verification is non-destructive: it restores into a temporary verification database and extracts media into a temporary directory.</li>
      <li>For scheduled operational hygiene, verify the newest bundle directly from the app container:
        <pre><code>docker compose exec -T web python manage.py verify_latest_backup --create-if-missing</code></pre>
      </li>
      <li>The Backup &amp; Recovery page now shows whether the latest verification is current, stale, missing, or still unverified.</li>
      <li>Real restore is explicit and destructive by design:
        <pre><code>./scripts/backup_restore.sh --yes-restore backend/backups/backup_YYYYmmdd_HHMMSS</code></pre>
      </li>
      <li>Do not run the restore script casually against a live working dataset.</li>
      <li>The staff UI uses the shared action-progress overlay for backup creation and verification so long-running actions present one standard app behavior.</li>
    </ul>

    <h2 id="hosts">16) Host and Domain Configuration</h2>
    <ul>
      <li>Primary env variables:
        <ul>
          <li><code>APP_DOMAIN</code>: canonical hostname without scheme</li>
          <li><code>APP_BASE_URL</code>: canonical external URL including scheme</li>
          <li><code>DJANGO_ALLOWED_HOSTS</code>: explicit host/IP allow-list</li>
          <li><code>DJANGO_CSRF_TRUSTED_ORIGINS</code>: explicit origin allow-list with scheme</li>
        </ul>
      </li>
      <li>The settings layer now folds <code>APP_DOMAIN</code> and the hostname from <code>APP_BASE_URL</code> into the effective allowed-host configuration automatically.</li>
      <li><code>APP_BASE_URL</code> is also appended to trusted CSRF origins automatically.</li>
      <li>Use <code>APP_DOMAIN</code> and <code>APP_BASE_URL</code> as the primary deployment-facing values instead of repeatedly editing long host/origin strings.</li>
      <li>If you also reach the app by IP address, keep the IP in <code>DJANGO_ALLOWED_HOSTS</code> and, if needed, in <code>DJANGO_CSRF_TRUSTED_ORIGINS</code>.</li>
      <li>The dedicated runbook page is available at <code>/admin-tools/deployment-hosts/</code>.</li>
      <li>An <code>Invalid HTTP_HOST header</code> failure happens before normal page routing, so a broken hostname cannot render a custom error page on that same broken host. Use a working host or IP to access the runbook and fix the env file.</li>
    </ul>

    <h2 id="cicd">17) CI/CD</h2>
    <div class="box">
      <h3>Current operating model</h3>
      <ul>
        <li>Repository model: one private GitHub repository.</li>
        <li>Working branch: <code>develop</code>.</li>
        <li>Stable branch: <code>main</code>.</li>
        <li>GitHub Actions is used for CI.</li>
        <li>Current test deployment is done manually from a LAN-connected machine, not from GitHub-hosted runners.</li>
      </ul>
    </div>

    <div class="box">
      <h3>Why deploy is manual right now</h3>
      <p>The test server is inside the local network and uses a private IP address <code>192.168.2.55</code>. GitHub-hosted runners on the public internet cannot reliably reach that target. Because of that, the correct deployment path today is:</p>
      <ol>
        <li>push code to GitHub</li>
        <li>let GitHub run CI</li>
        <li>deploy from the Mac on the same LAN</li>
      </ol>
      <p>Automatic CD from GitHub becomes appropriate only after moving to a public server or using a self-hosted runner inside the LAN.</p>
    </div>

    <div class="box">
      <h3>What to do for normal work</h3>
      <ol>
        <li>Start from <code>develop</code>.</li>
        <li>Do the implementation work.</li>
        <li>Push to GitHub.</li>
        <li>Let CI finish.</li>
        <li>Run the local test deployment helper from the Mac.</li>
        <li>Verify the updated version in the browser.</li>
        <li>When the integration line is stable, merge <code>develop</code> into <code>main</code>.</li>
      </ol>
    </div>

    <div class="box">
      <h3>One-command test deployment</h3>
      <p>From the Mac on the same network:</p>
      <pre><code>git checkout develop
./scripts/deploy_test_from_mac.sh</code></pre>
      <p>This helper script does all of the following:</p>
      <ol>
        <li>checks that the current branch is <code>develop</code></li>
        <li>fast-forwards from <code>origin/develop</code></li>
        <li>checks that the server env file exists</li>
        <li>syncs the repository to <code>/opt/workdock</code> with <code>rsync</code></li>
        <li>preserves server-local env files like <code>.env.test</code> and <code>.env.prod</code></li>
        <li>runs the remote deployment script</li>
        <li>waits for the health endpoint</li>
        <li>prints the deployed commit and branch</li>
      </ol>
    </div>

    <div class="box">
      <h3>Current test server values</h3>
      <ul>
        <li>Host: <code>192.168.2.55</code></li>
        <li>SSH user: <code>root</code></li>
        <li>Deploy path: <code>/opt/workdock</code></li>
        <li>Env file: <code>.env.test</code></li>
        <li>Health URL: <code>http://192.168.2.55:8088/healthz/</code></li>
        <li>Public app URL: <code>https://workdock.bostame.de/</code></li>
      </ul>
    </div>

    <div class="box">
      <h3>GitHub Actions status</h3>
      <ul>
        <li><code>.github/workflows/deploy-test.yml</code> exists, but GitHub-hosted deploy to the LAN server is not the recommended path right now.</li>
        <li><code>.github/workflows/deploy-prod.yml</code> exists for later production use.</li>
        <li>The practical use of GitHub today is CI, code review, PRs, and branch history.</li>
      </ul>
    </div>

    <div class="box">
      <h3>If the local deploy helper fails</h3>
      <ol>
        <li>Check whether <code>/opt/workdock/.env.test</code> still exists on the server.</li>
        <li>Check SSH access from the Mac:
          <pre><code>ssh -4 root@192.168.2.55</code></pre>
        </li>
        <li>Check server health directly:
          <pre><code>curl -I http://192.168.2.55:8088/healthz/</code></pre>
        </li>
        <li>Check container status:
          <pre><code>ssh root@192.168.2.55 "cd /opt/workdock && docker compose --env-file .env.test -f docker-compose.prod.yml ps"</code></pre>
        </li>
      </ol>
    </div>

    <div class="note">
      The current LAN test deployment intentionally uses <code>DJANGO_DEBUG=1</code> in <code>.env.test</code> because the security checks correctly reject insecure cookie settings when <code>DEBUG=0</code> and the deployment is still plain HTTP behind a local test topology. This is acceptable for the test box only. Production must run with HTTPS and <code>DEBUG=0</code>.
    </div>

    <h2 id="deploy">18) Deployment</h2>
    <h3>Test server stack</h3>
    <ul>
      <li>Stack file: <code>docker-compose.prod.yml</code></li>
      <li>Entrypoints:
        <ul>
          <li><code>backend/entrypoint-web-prod.sh</code></li>
          <li><code>backend/entrypoint-worker-prod.sh</code></li>
        </ul>
      </li>
      <li>Reverse proxy/static/media: <code>deploy/Caddyfile</code></li>
      <li>Main deploy script: <code>scripts/deploy_stack.sh</code></li>
    </ul>
    <h3>What the deploy script does</h3>
    <ol>
      <li>Validate env file presence</li>
      <li>Build <code>web</code>, <code>worker</code>, and <code>caddy</code></li>
      <li>Start <code>db</code> and <code>redis</code></li>
      <li>Initialize writable volume ownership for media/static/backups</li>
      <li>Run migrations</li>
      <li>Run <code>bootstrap_initial_users</code></li>
      <li>Run <code>collectstatic</code></li>
      <li>Optionally run <code>manage.py check</code></li>
      <li>Start <code>web</code>, <code>worker</code>, and <code>caddy</code></li>
      <li>Wait until <code>/healthz/</code> becomes healthy</li>
    </ol>
    <h3>Manual deploy</h3>
    <p>The preferred current test-deployment path is the local helper script from a Mac or another LAN-connected workstation:</p>
    <pre><code>./scripts/deploy_test_from_mac.sh</code></pre>
    <p>This script fast-forwards <code>develop</code>, checks that the remote env file exists, syncs the repo to the server with <code>rsync</code>, runs the remote deployment, verifies the health endpoint, and prints the deployed commit hash.</p>
    <p>The script explicitly preserves server-local env files such as <code>.env.test</code> and <code>.env.prod</code> so deployment does not wipe machine-specific secrets.</p>
    <p>Direct server-side deploy is still available if the code is already on the server:</p>
    <pre><code>cd /opt/workdock
RUN_DJANGO_CHECK=0 DEPLOY_HEALTH_URL="http://127.0.0.1:8088/healthz/" ./scripts/deploy_stack.sh .env.test docker-compose.prod.yml</code></pre>
    <h3>Validation after deploy</h3>
    <pre><code>curl -I http://192.168.2.55:8088/healthz/
ssh root@192.168.2.55 "cd /opt/workdock && docker compose --env-file .env.test -f docker-compose.prod.yml ps"</code></pre>
    <h3>Runtime config sync</h3>
    <p>Deployment updates code. It does not automatically overwrite runtime database configuration. Use explicit sync when you want local configuration compared or applied to the server.</p>
    <p>Supported sync scopes:</p>
    <ul>
      <li><code>PortalAppConfig</code></li>
      <li><code>PortalBranding</code></li>
      <li><code>PortalCompanyConfig</code></li>
    </ul>
    <p>Export locally:</p>
    <pre><code>docker compose exec -T web python manage.py export_portal_app_config --output /tmp/portal-app-config.json
docker compose exec -T web python manage.py export_portal_deployment_config --output /tmp/portal-deployment-config.json
docker compose cp web:/tmp/portal-app-config.json /tmp/portal-app-config.json
docker compose cp web:/tmp/portal-deployment-config.json /tmp/portal-deployment-config.json</code></pre>
    <p>Copy the JSON files to the server host:</p>
    <pre><code>scp -4 /tmp/portal-app-config.json /tmp/portal-deployment-config.json root@192.168.2.55:/opt/workdock/</code></pre>
    <p>Because the server runs baked container images instead of a bind-mounted app tree, copy the files into the running web container before importing:</p>
    <pre><code>ssh -4 root@192.168.2.55 '
docker cp /opt/workdock/portal-app-config.json workdock-web-1:/tmp/portal-app-config.json &amp;&amp;
docker cp /opt/workdock/portal-deployment-config.json workdock-web-1:/tmp/portal-deployment-config.json
'</code></pre>
    <p>Dry-run the import first:</p>
    <pre><code>ssh -4 root@192.168.2.55 '
docker exec workdock-web-1 python manage.py import_portal_app_config /tmp/portal-app-config.json --dry-run &amp;&amp;
docker exec workdock-web-1 python manage.py import_portal_deployment_config /tmp/portal-deployment-config.json --dry-run
'</code></pre>
    <p>Only apply the import after the dry run looks correct:</p>
    <pre><code>ssh -4 root@192.168.2.55 '
docker exec workdock-web-1 python manage.py import_portal_app_config /tmp/portal-app-config.json &amp;&amp;
docker exec workdock-web-1 python manage.py import_portal_deployment_config /tmp/portal-deployment-config.json
'</code></pre>
    <div class="note">
      Uploaded branding assets such as logo, favicon, and PDF letterhead are intentionally not included in deployment-config sync. They remain explicit media assets.
    </div>
    <h3>Proxmox / LXC requirement</h3>
    <p>The current server is an Ubuntu CT on Proxmox running Docker inside the container. The CT required Proxmox-side configuration before Docker containers could start correctly.</p>
    <pre><code>features: nesting=1,keyctl=1
lxc.apparmor.profile: unconfined
lxc.mount.entry: /dev/null sys/module/apparmor/parameters/enabled none bind 0 0</code></pre>
    <p>Those lines belong in <code>/etc/pve/lxc/&lt;CTID&gt;.conf</code> on the Proxmox host, followed by <code>pct restart &lt;CTID&gt;</code>.</p>
    <h3>Production expectations</h3>
    <ul>
      <li>Use a real hostname</li>
      <li>Use HTTPS</li>
      <li>Use <code>DJANGO_DEBUG=0</code></li>
      <li>Use secure cookies and SSL redirect</li>
      <li>Run deployment with <code>RUN_DJANGO_CHECK=1</code></li>
    </ul>
    <h3>Release checklist</h3>
    <ol>
      <li>Run <code>manage.py check</code></li>
      <li>Run tests or targeted verification</li>
      <li>Run translation compile step</li>
      <li>Rebuild containers if Python dependencies changed, then verify <code>python -c "import requests"</code> does not emit a compatibility warning</li>
      <li>Generate at least one onboarding/offboarding PDF if PDF templates changed</li>
      <li>Verify MailHog or SMTP path if email behavior changed</li>
      <li>Verify Nextcloud upload if integration behavior changed</li>
      <li>Update Project Wiki and Developer Handbook if architecture or operational workflow changed</li>
      <li>Take a snapshot commit before major next-phase work</li>
    </ol>

    <h2 id="commands">19) Command Reference</h2>
    <div class="box">
      <h3>Local development</h3>
      <pre><code>docker compose up -d --build</code></pre>
      <p>Start or rebuild the local stack.</p>
      <pre><code>docker compose restart web
docker compose restart worker</code></pre>
      <p>Restart app services after code or template changes.</p>
    </div>
    <div class="box">
      <h3>Validation</h3>
      <pre><code>docker compose exec -T web python manage.py check</code></pre>
      <p>Run Django system checks.</p>
      <pre><code>docker compose exec -T web python manage.py test</code></pre>
      <p>Run the full test suite.</p>
    </div>
    <div class="box">
      <h3>Local test deployment</h3>
      <pre><code>./scripts/deploy_test_from_mac.sh</code></pre>
      <p>Sync the current <code>develop</code> checkout to the LAN test server and deploy it.</p>
    </div>
    <div class="box">
      <h3>Direct server deployment</h3>
      <pre><code>cd /opt/workdock
RUN_DJANGO_CHECK=0 DEPLOY_HEALTH_URL="http://127.0.0.1:8088/healthz/" ./scripts/deploy_stack.sh .env.test docker-compose.prod.yml</code></pre>
      <p>Deploy when code is already present on the server.</p>
    </div>
    <div class="box">
      <h3>Config sync</h3>
      <pre><code>docker compose exec -T web python manage.py export_portal_app_config --output /tmp/portal-app-config.json
docker compose exec -T web python manage.py export_portal_deployment_config --output /tmp/portal-deployment-config.json</code></pre>
      <p>Export runtime configuration from local.</p>
      <pre><code>docker exec workdock-web-1 python manage.py import_portal_app_config /tmp/portal-app-config.json --dry-run
docker exec workdock-web-1 python manage.py import_portal_deployment_config /tmp/portal-deployment-config.json --dry-run</code></pre>
      <p>Validate server-side config import before applying it.</p>
    </div>
    <div class="box">
      <h3>Backup</h3>
      <pre><code>make backup-create
make backup-verify BACKUP_DIR=backups/backup_YYYYmmdd_HHMMSS</code></pre>
      <p>Create and verify backup bundles.</p>
    </div>

    <h2 id="troubleshooting">20) Troubleshooting</h2>
    <ul>
      <li><strong>Page looks stale:</strong> restart <code>web</code> and hard-refresh browser</li>
      <li><strong>Second request hangs:</strong> inspect web logs and verify health endpoint</li>
      <li><strong>PDF looks unchanged:</strong> regenerate the PDF; browser may cache old file names unless the path changes</li>
      <li><strong>Language switch not visible:</strong> verify translation catalog compiled and restart web</li>
      <li><strong>Mail not visible:</strong> check MailHog on port <code>8025</code> and test/production mode toggle</li>
      <li><strong>Nextcloud sync unclear:</strong> verify config in Integrations page and inspect service logs</li>
      <li><strong>Requests dependency warning appears:</strong> verify <code>chardet==5.2.0</code> is installed in the rebuilt image and restart <code>web</code>/<code>worker</code></li>
    </ul>

    <h2 id="security">21) Security and Maintenance Notes</h2>
    <ul>
      <li>Containers run as non-root <code>app</code> user.</li>
      <li>Keep secrets in <code>.env</code>, not in tracked files.</li>
      <li>Avoid destructive git operations in a dirty repo.</li>
      <li>Prefer standard framework workflows over custom one-off maintenance scripts.</li>
      <li>When adding new features, document them in both the Project Wiki and this handbook if they change engineering workflow.</li>
    </ul>
    <div class="box">
      <h3>For the next coder</h3>
      <ul>
        <li>Read this page first, then the Project Wiki, then the release checklist.</li>
        <li>Assume some visible behavior is runtime configuration, not only template or view code.</li>
        <li>Keep stable import surfaces intact where possible when modularizing.</li>
        <li>Prefer explicit, documented behavior over hidden automation.</li>
      </ul>
    </div>
    </div>
  
{% endblock %}