Radix-Skills/skills/data-science/erp-data-reconciliation/references/auditor-pack-and-dedup.md

Auditor-ready reconciliation packs and duplicate-import cleanup

Use this when a reconciliation webapp must produce material that can be sent to an accountant/revisor, not just a green dashboard.

Revisor-grade status language

Do not collapse every successful bank reconciliation into “fully reconciled”. Use explicit statuses:

• AFSTEMT: all required sources loaded, relevant bank movements explained, and no open invoice/credit-note residuals.
• AFSTEMT_MED_ÅBNE_POSTER: bank layer is explained, but invoice/credit-note residuals remain and must be reviewed by accounting/revisor.
• BANK_DIFFERENCER: required sources loaded, but relevant bank postings are unexplained.
• MANGLER_DATA: one or more required sources are missing.

This prevents the dangerous interpretation “0 unexplained bank postings = no accounting work remains”.

Duplicate import cleanup pattern

Historical accounting imports are often rerun. Before trusting totals, check for duplicate SourceFile rows by source/file name/parsed count and repeated downstream row counts.

Safe cleanup sequence:

1. Create a database backup first.
2. Identify duplicate import runs, keeping the canonical/original SourceFile rows.
3. Delete duplicate SourceFile rows through the ORM so related imported rows cascade consistently.
4. Record an audit log entry with reason and deleted source file IDs.
5. Recompute counts and reports after cleanup.

Do not modify original source files. The goal is to remove duplicate imported rows, not change evidence.

Auditor pack contents

Generate a deterministic export package with at least:

• Markdown executive summary in Danish.
• Excel workbook with sheets:
  • Resume: data completeness, bank coverage, open balances, status.
  • Kildefiler: imported files, parsed/error counts, timestamps.
  • Måneder: month-level totals across Coop, bank, invoice list, and payment allocation.
  • Åbne poster: invoice/credit-note residuals with invoice number, date, customer/account, total, paid, residual, vouchers, explanation.
  • Uforklaret bank: relevant bank movements that could not be explained.
  • Ignoreret bank: bank movements outside the debtor reconciliation scope, with reason.
  • Payment advice: uploaded settlement/advice records.
  • Rå posteringer: normalized postings for traceability.

Use Danish number formatting in human-facing Markdown and UI (1.234.567,89). Excel cells may stay numeric where useful, but headings and sheet names should be accountant-readable.

Implementation notes

• Keep the analysis/export layer read-only; it should not mutate match status.
• Convert timezone-aware datetimes to ISO strings before writing with pandas/openpyxl; Excel rejects timezone-aware datetimes.
• Include both gross open amount and net open balance. Accountants often need both:
  • gross open review amount = sum of absolute residuals;
  • positive residuals = potential receivables;
  • negative residuals = open credits/modregninger;
  • net residual = positive + negative.
• Add tests that read the generated workbook back and assert key sheets/rows exist.
• If visual browser automation is unavailable, verify dashboard output with Django’s test client/HTTP response text and keep a separate browser/UI check when the environment supports it.

Minimum verification

• Framework check passes.
• Full reconciliation/core tests pass.
• Auditor export command runs idempotently.
• Workbook exists and contains all expected sheets.
• Summary status and open totals match the recomputed analysis function.
• Dashboard shows bank coverage and open-poster status consistently with the exported package.