{% extends 'workflows/base_shell.html' %}
{% load static %}

{% block title %}Developer Handbook{% endblock %}



{% block extra_css %}
<link rel="stylesheet" href="{% static 'workflows/css/docs_pages.css' %}" />
{% endblock %}

{% block shell_body %}
      {% include 'workflows/includes/app_header.html' with header_show_home=1 header_inside_shell=1 %}
    <div class="top">
      <h1>Developer Handbook</h1>
      <div style="display:flex; gap:8px; flex-wrap:wrap;">
        <a class="btn btn-secondary" href="/admin-tools/wiki/">Project Wiki</a>
      </div>
    </div>
    <p class="sub">Engineering runbook for development, deployment, maintenance, and extension of the TUBCO Onboarding & Offboarding Portal.</p>

    <div class="toc">
      <a href="#overview">Overview</a>
      <a href="#structure">Structure</a>
      <a href="#local">Local Dev</a>
      <a href="#docker">Docker</a>
      <a href="#db">Database</a>
      <a href="#translations">Translations</a>
      <a href="#pdf">PDF Pipeline</a>
      <a href="#email">Email Pipeline</a>
      <a href="#nextcloud">Nextcloud</a>
      <a href="#builders">Builders</a>
      <a href="#testing">Testing</a>
      <a href="#deploy">Deployment</a>
      <a href="#troubleshooting">Troubleshooting</a>
      <a href="#security">Security</a>
    </div>

    <h2 id="overview">1) Overview</h2>
    <div class="box">
      <p>This handbook is for developers and maintainers. It documents the actual engineering workflow of the standalone product repository.</p>
      <ul>
        <li>Repository: <code>tubco-onboarding-offboarding-portal</code></li>
        <li>Main stack: Django + Celery + PostgreSQL + Redis + MailHog</li>
        <li>Runtime mode: Docker Compose for local development and staging-style operation</li>
      </ul>
    </div>

    <h2 id="structure">2) Repository Structure</h2>
    <ul>
      <li><code>/backend/config/</code>: Django settings, WSGI, URL config</li>
      <li><code>/backend/workflows/</code>: application logic, views, models, tasks, templates, static assets</li>
      <li><code>/backend/workflows/templates/workflows/base_shell.html</code>: standard page shell for new staff-facing pages</li>
      <li>Rule: all interactive app pages should extend <code>base_shell.html</code>; do not rebuild topbar/frame logic in page-local templates.</li>
      <li><code>/backend/media/templates/</code>: PDF HTML templates and letterhead source files</li>
      <li><code>/backend/media/pdfs/</code>: generated PDF outputs on host volume</li>
      <li><code>/backend/locale/</code>: translation catalogs</li>
      <li><code>/docker-compose.yml</code>: local runtime orchestration</li>
      <li><code>/Makefile</code>: repeatable translation commands</li>
      <li><code>/.github/workflows/i18n.yml</code>: translation compile validation in CI</li>
    </ul>

    <h2 id="local">3) Local Development Workflow</h2>
    <h3>Start</h3>
    <pre><code>cd /Users/bostame/Documents/tubco-onboarding-offboarding-portal
docker compose up -d --build</code></pre>
    <h3>Main URLs</h3>
    <ul>
      <li>App: <code>http://127.0.0.1:8088/</code></li>
      <li>MailHog: <code>http://127.0.0.1:8025/</code></li>
      <li>Health check: <code>http://127.0.0.1:8088/healthz/</code></li>
    </ul>
    <h3>Bootstrap users</h3>
    <ul>
      <li>Admin: <code>admin_test</code> / <code>admin12345</code></li>
      <li>User: <code>user_test</code> / <code>user12345</code></li>
    </ul>

    <h2 id="docker">4) Docker Operations</h2>
    <pre><code>docker compose up -d --build
docker compose restart web
docker compose restart worker
docker compose logs --no-color --tail=120 web
docker compose logs --no-color --tail=120 worker
docker compose down
docker compose down -v</code></pre>
    <div class="note">
      The source code is bind-mounted into the container. Most template/view/static changes only require a web restart, not a full rebuild. Image changes such as system packages require <code>docker compose up -d --build</code>.
    </div>

    <h2 id="db">5) Database and Migrations</h2>
    <pre><code>docker compose exec -T web python manage.py makemigrations
docker compose exec -T web python manage.py migrate
docker compose exec -T web python manage.py check</code></pre>
    <ul>
      <li>Never edit or remove historical migrations casually.</li>
      <li>When adding fields to builder-driven models, preserve fallback behavior for existing rows.</li>
      <li>Fresh boot sequence runs migrations automatically in <code>entrypoint-web.sh</code>.</li>
    </ul>

    <h2 id="translations">6) Translation Workflow</h2>
    <h3>Standard Django i18n path</h3>
    <pre><code>make i18n-update-en
make i18n-compile</code></pre>
    <p>Equivalent raw commands:</p>
    <pre><code>docker compose exec -T web django-admin makemessages -l en
docker compose exec -T web django-admin compilemessages</code></pre>
    <ul>
      <li><code>gettext</code> is installed in the Docker image.</li>
      <li>Do not use custom ad hoc <code>.mo</code> compilation anymore.</li>
      <li>Phase 1 bilingual support covers fixed UI, including the main admin-tool labels/buttons.</li>
      <li>Phase 2 covers builder-driven labels and checklist items with explicit German and English fields.</li>
      <li>Email phase covers <code>NotificationTemplate</code>, <code>NotificationRule</code>, and the welcome-email settings UI with explicit DE/EN subject/body fields.</li>
      <li>Request records now persist <code>preferred_language</code> so workflow emails can render in the submitter's active UI language with German fallback.</li>
      <li><code>preferred_language</code> is normalized in model <code>save()</code> and also has a DB default of <code>de</code>, so alternate creation paths cannot insert null values.</li>
    </ul>

    <h2 id="pdf">7) PDF Pipeline</h2>
    <ul>
      <li>PDF generation is HTML-to-PDF using <code>xhtml2pdf</code>.</li>
      <li>Letterhead overlay is applied from <code>templates.pdf</code>.</li>
      <li>Main logic lives in <code>backend/workflows/tasks.py</code>.</li>
      <li>Fixed PDF labels/headings are rendered from task-level DE/EN text dictionaries, not hard-coded directly in request processing logic.</li>
      <li>PDF language follows the normalized request <code>preferred_language</code>, with German fallback.</li>
      <li>Key templates:
        <ul>
          <li><code>onboarding_template.html</code></li>
          <li><code>offboarding_template.html</code></li>
          <li><code>onboarding_intro_template.html</code></li>
          <li><code>onboarding_intro_session_pdf.html</code></li>
        </ul>
      </li>
    </ul>
    <div class="note">
      xhtml2pdf is sensitive to layout complexity. Keep print templates conservative and verify every structural change with a real generated PDF.
    </div>

    <h2 id="email">8) Email Pipeline</h2>
    <ul>
      <li>Notification defaults are defined in <code>DEFAULT_NOTIFICATION_TEMPLATES</code> in <code>tasks.py</code>.</li>
      <li>Admin-configured overrides live in <code>NotificationTemplate</code> and <code>NotificationRule</code>.</li>
      <li>Both models now support explicit DE/EN content fields. Prefer filling both languages in the frontend integrations UI instead of embedding mixed-language copy into a single template.</li>
      <li>Welcome-email configuration under Admin Apps has the same DE/EN split and follows the same runtime language selection.</li>
      <li>The request objects passed into email tasks always carry a normalized <code>preferred_language</code> value.</li>
      <li>Mail sending uses Celery tasks and supports test mode redirection.</li>
      <li>MailHog is the local verification path when test mode is active.</li>
    </ul>

    <h2 id="nextcloud">9) Nextcloud Integration</h2>
    <ul>
      <li>Configured from Admin Apps → Integrations.</li>
      <li>Upload logic lives in <code>backend/workflows/services.py</code>.</li>
      <li>Feature can be globally toggled without changing environment variables.</li>
      <li>Failures should degrade gracefully and not block request persistence.</li>
    </ul>

    <h2 id="builders">10) Builder Architecture</h2>
    <h3>Form Builder</h3>
    <ul>
      <li>Model: <code>FormFieldConfig</code> + <code>FormOption</code></li>
      <li>Controls field order, visibility, required flags, option sets, and bilingual label/help-text overrides</li>
    </ul>
    <h3>Intro Builder</h3>
    <ul>
      <li>Model: <code>IntroChecklistItem</code></li>
      <li>Controls additional checklist items used by intro PDF and live intro checklist</li>
      <li>Now supports bilingual DE/EN labels</li>
    </ul>
    <div class="note">
      Dynamic content should use explicit DE/EN fields with German fallback, not machine translation at runtime.
    </div>
    <h3>Audit Trail</h3>
    <ul>
      <li>Model: <code>AdminAuditLog</code></li>
      <li>Purpose: record staff-side mutations that affect operations or configuration.</li>
      <li>Current hooks include builder edits, PDF generation, welcome-email actions, integration changes, mode toggles, tests, and request deletions.</li>
      <li>Staff UI page: <code>/admin-tools/audit-log/</code></li>
      <li>The current UI supports filtering by action, user, and date range. Keep filters server-side to avoid loading unbounded audit rows into the browser.</li>
    </ul>

    <h2 id="testing">11) Testing and Validation</h2>
    <pre><code>docker compose exec -T web python manage.py check
docker compose exec -T web python manage.py test
docker compose exec -T web python manage.py run_staging_e2e_check</code></pre>
    <ul>
      <li>Use <code>check</code> after model/view/template changes.</li>
      <li>Use targeted shell checks for render validation when changing templates or routes.</li>
      <li>New pages should extend <code>base_shell.html</code> and keep header/frame logic out of page-local templates.</li>
      <li>Use real PDF generation tests when changing PDF templates or intro/offboarding document logic.</li>
      <li>Use the dedicated Release Checklist page as the final go/no-go runbook before shipping changes.</li>
      <li>The automated bilingual smoke tests now cover DE/EN request language capture and English email-template rendering.</li>
      <li>Onboarding and offboarding request objects now expose explicit processing state and last-error fields. Async tasks are responsible for transitioning <code>submitted → processing → completed/failed</code>.</li>
      <li>The Requests Dashboard includes a retry action for failed requests. Retries reset the error text, set the request back to <code>submitted</code>, and enqueue the appropriate Celery task again.</li>
    </ul>

    <h2 id="deploy">12) Deployment and Release Checklist</h2>
    <ol>
      <li>Run <code>manage.py check</code></li>
      <li>Run tests or targeted verification</li>
      <li>Run translation compile step</li>
      <li>Rebuild containers if Python dependencies changed, then verify <code>python -c "import requests"</code> does not emit a compatibility warning</li>
      <li>Generate at least one onboarding/offboarding PDF if PDF templates changed</li>
      <li>Verify MailHog or SMTP path if email behavior changed</li>
      <li>Verify Nextcloud upload if integration behavior changed</li>
      <li>Update Project Wiki and Developer Handbook if architecture/workflow changed</li>
      <li>Take a snapshot commit before major next-phase work</li>
    </ol>

    <h2 id="troubleshooting">13) Troubleshooting</h2>
    <ul>
      <li><strong>Page looks stale:</strong> restart <code>web</code> and hard-refresh browser</li>
      <li><strong>Second request hangs:</strong> inspect web logs and verify health endpoint</li>
      <li><strong>PDF looks unchanged:</strong> regenerate the PDF; browser may cache old file names unless the path changes</li>
      <li><strong>Language switch not visible:</strong> verify translation catalog compiled and restart web</li>
      <li><strong>Mail not visible:</strong> check MailHog on port <code>8025</code> and test/production mode toggle</li>
      <li><strong>Nextcloud sync unclear:</strong> verify config in Integrations page and inspect service logs</li>
      <li><strong>Requests dependency warning appears:</strong> verify <code>chardet==5.2.0</code> is installed in the rebuilt image and restart <code>web</code>/<code>worker</code></li>
    </ul>

    <h2 id="security">14) Security and Maintenance Notes</h2>
    <ul>
      <li>Containers run as non-root <code>app</code> user.</li>
      <li>Keep secrets in <code>.env</code>, not in tracked files.</li>
      <li>Avoid destructive git operations in a dirty repo.</li>
      <li>Prefer standard framework workflows over custom one-off maintenance scripts.</li>
      <li>When adding new features, document them in both the Project Wiki and this handbook if they change engineering workflow.</li>
    </ul>
  
{% endblock %}