Initial import: 10 Radix skills across 3 categories

Kategoriserer danske regnskabs-/ERP-skills, ERP-data-afstemnings- skill og to devops-workflows (gitea-issue-agent, kanban-workflows) som Radix-medlemmers AI-agenter kan dele. Struktur: skills/<kategori>/<skill-navn>/SKILL.md skills/<kategori>/<skill-navn>/{references,templates,scripts}/ Indekseret af index.json (genereret af scripts/rebuild_index.py). Kør `python3 scripts/rebuild_index.py --check` i CI for at fange synkroniseringsfejl.
2026-06-15 14:41:04 +02:00
commit 236053cd7f
30 changed files with 4460 additions and 0 deletions
@@ -0,0 +1,314 @@
+---
+name: erp-data-reconciliation
+description: |
+  Analyze and reconcile financial/accounting data between two ERP systems
+  (e.g. SAP/Coop and Uniconta) using Excel exports. Covers file inspection,
+  column mapping, Danish-format normalization, match-engine design, and
+  iterative build workflow.
+trigger: |
+  User asks to reconcile, match, compare, or afstemme data between two
+  accounting/ERP systems, typically via Excel files. Also relevant when
+  importing/exporting financial ledgers, kontoudtog, vendor/customer
+  statements, or bonus calculations across systems.
+---
+
+# ERP Data Reconciliation Skill
+
+## Overview
+
+Reconciling data between two ERP/accounting systems is a multi-phase task.
+The typical deliverable is a webapp or script that can:
+1. Import Excel files from both systems
+2. Normalize dates, amounts, and text
+3. Match postings between systems
+4. Show unmatched, partial matches, and differences
+5. Support manual approval/rejection
+6. Generate reports
+
+## Phase 1: File Analysis (CRITICAL — do not skip)
+
+Before writing any import code, run a structured analysis of the Excel files.
+
+### Steps
+1. List all files in both source directories
+2. For each file, identify:
+   - Sheet names
+   - Actual header row location (may NOT be row 1!)
+   - Column names and data types
+   - Date columns and their formats
+   - Amount columns and their formats
+   - Text columns
+   - Reference/invoice/document columns
+   - Null percentages
+   - Duplicate rows
+   - Data range (min/max dates)
+
+### Key Pitfalls
+- **Header row is NOT always row 1.** Uniconta exports often have account
+  info in rows 1-2, blank row 3, headers in row 4.
+- **Amounts mixed formats.** Danish Excel files often have amounts as both
+  Excel floats AND as Danish text strings like `-2.543.803,93`.
+  Always normalize using a robust parser:
+  ```python
+  def parse_danish_number(val):
+      if pd.isna(val): return float('nan')
+      if isinstance(val, (int, float)): return float(val)
+      s = str(val).strip()
+      if not s: return float('nan')
+      if '.' in s and ',' in s:
+          s = s.replace('.', '').replace(',', '.')
+      elif ',' in s: s = s.replace(',', '.')
+      return float(s)
+  ```
+- **Invoice/reference numbers may NOT match between systems.** This is the
+  #1 cause of failed reconciliation. Always check overlap BEFORE assuming
+  IDs can be used as join keys. Use amount overlap as a quick viability check.
+
+### Analysis Script Template
+Use or adapt `scripts/analyze_excel.py` from the Coop-Uniconta project:
+- Reads all sheets in all files
+- Prints column stats, null counts, sample values
+- Detects Danish number formats
+- Identifies empty/duplicate rows
+
+## Phase 2: Determine Match Strategy
+
+After analysis, document the match strategy BEFORE coding the engine.
+
+### Decision Tree
+1. **Do invoice/reference numbers overlap?** 
+   - If yes → use `Faktura + abs(Amount) + Date` as PRIMARY signal (two-phase).
+     First phase: Reference==Faktura + abs(Amount) within tolerance (±5 DKK) + Date±21d.
+     Second phase: abs(Amount) + Date±21d for remaining rows.
+   - If no → use `Amount + Date + Text similarity`
+2. **Are dates exact or within tolerance?**
+   - Same day → exact
+   - ±1-3 days → high confidence partial
+   - ±7 days → medium confidence
+   - ±21 days → general match tolerance
+   - >21 days → fuzzy/manual only
+3. **Is text available on both sides?**
+   - Both have text → Levenshtein/rapidfuzz similarity score
+   - Only one side has text → amount+date only
+4. **Do signs differ?**
+   - Same sign → same direction transaction
+   - Opposite sign → same transaction viewed from creditor vs debtor side.
+     Match on `abs(amount)` but label "MODSAT" in output.  
+     *Example: Coop RE (payment) is negative; Uniconta Faktura is positive.*
+
+### Tolerance Matching (V6 innovation)
+
+Instead of exact amount match (±0.01), use a tolerance (±5 DKK):
+- Catches rounding differences between systems
+- Catches partial payments (one system may record 440.000, the other 430.732)
+- Uses bucket-based indexing for fast lookup:
+  ```python
+  AMT_TOL = 5.0
+  bucket = round(abs_amount / AMT_TOL) * AMT_TOL
+  # Search adjacent buckets: [bucket-AMT_TOL, bucket, bucket+AMT_TOL]
+  ```
+
+### Coop Fakturaliste + Uniconta Betalingsfordeling as the Simple Forward Workflow
+
+When the user wants the process to be as simple as possible, prefer a forward-looking workflow based on:
+1. `Payment_advice`
+2. latest Uniconta Coop debtor/payment allocation export
+3. the complete Coop invoice list
+
+Model the invoice list and payment allocation as separate import sources. Use invoice number as the primary join between Coop invoices and Uniconta allocation lines, and calculate paid/open/partial status from payment/settlement lines rather than only from original invoice postings. Treat the payment allocation as an explanation layer between a bank deposit and many invoice/credit lines.
+
+See `references/coop-invoice-payment-allocation.md` for the concrete Django model/import/status pattern, UI shape, and tests.
+
+### Coop Payment Advice / Settlement Files as a Fourth Source
+
+When the user provides Coop betalingsadviseringer/opgørelser for individual bank
+payments, treat them as a separate reconciliation source, not just attachments.
+They are often the missing bridge:
+
+`Bank deposit → Coop payment advice total → advice invoice/credit lines → Coop ledger / Uniconta invoices`
+
+For Coop-Uniconta style systems, the import UI should expose the full operational set, not just "Coop" and "Uniconta":
+1. Uniconta Coop Debitor / payment allocation
+2. Uniconta/ERP invoice list: all Coop invoices/credit notes for the year
+3. Coop account statement / kontoudtog
+4. Bank statement containing all Coop postings
+5. Coop Payment_advice documents, uploadable continuously as they are collected
+
+Recommended source/model names:
+- `coop_settlement` or `coop_payment_advice`
+- `PaymentAdvice` / `PaymentAdviceDocument` / `SettlementFile`: file name, unique id/bilag number, file type
+  (PDF/XLSX), document date, payment date, total amount, period, parse status, matched bank transaction, raw
+  text/raw JSON.
+- `PaymentAdviceLine` / `SettlementLine`: invoice number, invoice date, company
+  code, amount, currency, line type (invoice/credit/bonus/modregning/unknown),
+  links to bank/Coop/Uniconta transactions, match status, explanation.
+
+If the user does not yet have all payment advice documents, still allow upload now. Store raw PDF text or Excel row JSON with best-effort date/amount/reference/classification so later parsers can be tightened without re-requesting files.
+
+Parser patterns observed:
+- XLSX files may have header row 3 with columns `Fakturanummer`, `Fakturadato`,
+  `Firmakode`, `Beløb`, `Valuta` and supplier metadata in rows 1-2.
+- Text PDFs often extract as repeated triples:
+  invoice number line, `dd.mm.yyyy firmakode` line, amount/`DKK` line.
+- Some PDFs split sign into a separate `-` column/line; treat `123,45`, `-`,
+  `DKK` as `-123,45`.
+- Some PDFs are bank statement screenshots, not payment advice. Classify and do
+  not parse them as settlement lines.
+- Some files include manual middle calculations or special layouts. Keep raw text
+  and mark parse confidence/status instead of silently forcing a schema.
+
+Matching/reporting logic:
+1. Match payment-advice total to bank deposit using ±5 DKK tolerance.
+2. Then match advice lines to Uniconta by invoice number first, then amount/date.
+3. Match advice lines to Coop ledger by reference/doc number; in the Coop-Uniconta
+   project this was a very strong signal.
+4. Treat payment-advice matches as an evidence/explanation layer, not as automatic
+   ledger settlement. An invoice can remain open in Uniconta while Payment_advice
+   proves Coop included it in a settlement, credit, bonus, modregning, or manual
+   middle calculation.
+5. Report per bank deposit/open item: bank date/amount, matched advice file,
+   difference, line count, Uniconta hit count, Coop hit count, credit/bonus/
+   modregning total, advice documents/vouchers/classifications, and status OK /
+   partial / needs review.
+6. Keep 2026 or out-of-period advice files in the database, but exclude or label
+   them separately when current bank/Uniconta data only covers 2024-2025.
+
+### Invoice Status: Net-based + Deduplication
+
+**Critical pitfall:** The same payment may exist in BOTH systems (Coop RE and Uniconta Betaling). If you sum both, you get double the actual paid amount.
+
+**Solution:** Calculate net payments per invoice:
+```python
+# Net Coop payments (NOT abs-sum!)
+coop_net = sum(c['amount'] for c in coop 
+               if c['reference'] == fnr and c['doc_type'] in ('RE','ZP'))
+coop_paid = abs(coop_net) if coop_net < 0 else 0
+
+# Net Uniconta payments
+uni_net = sum(u['amount'] for u in uni 
+            if u['faktura'] == fnr and u['konteringstype'] == 'Betaling')
+uni_paid = abs(uni_net) if uni_net < 0 else 0
+
+# Deduplicate: if both exist, use only one
+if coop_paid > 0 and uni_paid > 0:
+    if abs(coop_paid - uni_paid) <= AMT_TOL:
+        betalt = coop_paid  # Same payment recorded twice
+    else:
+        betalt = max(coop_paid, uni_paid)  # Partial, take largest
+```
+
+**Also handle:** Positive Coop RE amounts are credit memos (modregninger) that should be SUBTRACTED from the net, not added with abs().
+
+### Match Scoring Formula (default)
+```
+Score = 0
+# Amount match (40%)
+if abs(a-b) < AMT_TOL: Score += 40
+
+# Date match (35%)
+days = abs(d1 - d2).days
+if days == 0: Score += 35
+elif days <= 3: Score += 25
+elif days <= 7: Score += 15
+elif days <= 14: Score += 5
+
+# Text match (25%)
+if both_have_text:
+    similarity = fuzzy_ratio(text1, text2)  # 0-100
+    if similarity > 80: Score += 25
+    elif similarity > 50: Score += 12
+
+# Reference bonus (extra)
+if reference_match: Score += 40
+
+# Thresholds
+if Score >= 85: exact_match, auto_approvable
+if Score >= 60: partial_match, requires_manual
+if Score >= 40: fuzzy_match, requires_manual
+```
+
+| Table | Key Fields |
+|-------|-----------|
+| `ImportBatch` | source (coop/uniconta), filename, imported_at, raw_row_count |
+| `Posting` | batch_id, source, fiscal_year, posting_date, amount, original_text, normalized_text, reference, document_number, original_data (JSON) |
+| `Match` | posting_a_id, posting_b_id, score, match_type (exact/partial/fuzzy/manual), status (proposed/approved/rejected), matched_by (auto/user), matched_at |
+| `AuditLog` | user, action, match_id, old_status, new_status, comment, timestamp |
+
+## Phase 4: Build Iteratively
+
+Follow this order strictly:
+1. Analysis scripts and documentation → `ANALYSE.md`
+2. Import parser with preview → validate before commit
+3. Normalization layer (dates, amounts, text)
+4. Match engine (exact → partial → fuzzy)
+5. Simple web UI (dashboard → import → matching table)
+6. Manual approve/reject + audit log
+7. Reports and exports
+8. Bonus calculation (if needed)
+9. Tests + README + Docker
+
+### Revisor/Auditor Package Pattern
+
+When the user asks for output that can be sent to an accountant/revisor, produce a deterministic evidence package rather than relying on dashboard screenshots. First verify and remove duplicate import runs after a DB backup; then export a Markdown summary plus an Excel workbook with source files, month totals, open invoice/credit residuals, unexplained bank items, ignored bank items, payment advice rows, and raw normalized postings. Use explicit status language such as `AFSTEMT_MED_ÅBNE_POSTER` when the bank layer reconciles but invoice/credit-note residuals remain.
+
+See `references/auditor-pack-and-dedup.md` for the duplicate-import cleanup sequence, auditor workbook sheet shape, status semantics, and verification checklist.
+
+### Idempotent SourceFile Imports
+
+For accounting import UIs, especially Coop `Payment_advice` bulk uploads, make uploads idempotent. Add a SHA256 `content_hash` on the source-file model, compute it before creating a new row, skip identical reuploads, and update/reparse an existing same-named source file when the content changed. Wrap child-row replacement in a database transaction and test both duplicate-skip and update-replace paths so repeated user uploads do not inflate reconciliation totals.
+
+See `references/idempotent-source-file-imports.md` for the Django model/view/parser pattern, UI counters, bulk upload setting, and regression checklist.
+
+## Danish-specific Conventions
+
+- Dates: use ISO format internally (`YYYY-MM-DD`), but display in Danish
+  (`dd-MM-yyyy`) per user preference. Parse ambiguous Danish text dates with day-first semantics (`dayfirst=True`).
+- Amounts: always normalize to float/Decimal; display with 2 decimals, Danish comma.
+- Fiscal year: calendar year (1/1 – 31/12) unless specified otherwise.
+- For Coop workflows, model invoice year/month separately from payment year/month: Coop can have ~3 months of credit, so December invoices may be paid in March of the following fiscal year. Reports must not treat that as an error by default.
+- Text: normalize by lowercasing, stripping extra spaces, removing punctuation before fuzzy comparison.
+- Regulation/bonus/fee/fine classification should be explicit and reviewable. Start with keyword classification (`bonus`, `rabat`, `gebyr`, `bøde`, `regulering`, `korrektion`, `modregning`) but keep raw data and manual review because wording/layout varies.
+
+## Django Accounting UI Display Formatting
+
+When the reconciliation deliverable is a Django webapp, treat amount display as part of the reconciliation contract rather than a cosmetic detail. Use one shared template filter/localization utility for money columns in dashboards, match tables, invoice/payment status pages, and reports.
+
+For Danish accounting UIs, monetary amounts should normally render with `.` thousands separators, `,` decimal separators, exactly two decimals, and a leading minus sign for negative values (for example `1.234.567,89`, `-1.234.567,89`, `0,00`). Prefer `Decimal`-based formatting over Python floats. Replace monetary `|floatformat:2` usage, but do not reformat percentages, match scores, counts, or raw imported/audit JSON unless explicitly requested.
+
+Minimum verification for UI formatting changes:
+- Add a unit/template test for the formatting helper.
+- Add at least one rendered-page regression test proving formatted money appears and raw `1234567.89`-style output is absent.
+- Confirm templates using the filter load its tag module, and restart the web container if needed.
+
+See `references/django-accounting-ui-formatting.md` for the concrete Django template-filter implementation, search targets, Docker test commands, and pitfalls.
+
+## References
+- See `references/coop-uniconta-analysis.md` for the concrete discovery
+  patterns from the Coop ↔ Uniconta project (header row locations,
+  column mappings, SAP document type codes, etc.).
+- See `references/coop-uniconta-match-engine.md` for the 5-phase match engine
+  implementation, tolerance matching, net-based invoice status, and score formulas.
+- See `references/bank-reconciliation.md` for three-source reconciliation extending
+  Coop ↔ Uniconta with bank CSV — month-level validation, bonus reconciliation,
+  and many-to-one (bank payment = sum of Coop invoices) matching strategy.
+- See `references/coop-payment-advice.md` for using Coop betalingsadviseringer
+  (PDF/XLSX settlement files) as the bridge between bank deposits and invoice
+  lines, including parser patterns, coverage metrics, and reporting shape.
+- See `references/three-way-django-analysis-engine.md` for Django implementation
+  structure: separate analysis module, batch import command, three-way report
+  outputs, interpretation pitfalls, and minimum tests.
+- See `references/coop-invoice-payment-allocation.md` for the simplified forward
+  workflow using Coop invoice lists + Uniconta payment allocation + Payment_advice,
+  including separate models, invoice paid/open status logic, employee-expense
+  exceptions, UI badges, and minimum tests.
+- See `references/coop-uniconta-import-ui-and-payment-advice.md` for the five-source
+  import UI pattern, flexible PDF/Excel Payment_advice raw import, monthly dashboard
+  fields, 3-month Coop credit handling, and regression tests.
+- See `references/auditor-pack-and-dedup.md` for revisor-ready exports, duplicate
+  import cleanup after DB backup, status semantics such as `AFSTEMT_MED_ÅBNE_POSTER`,
+  and Excel/Markdown verification.
+- See `references/idempotent-source-file-imports.md` for SHA256-based idempotent
+  accounting source-file uploads, update-vs-skip semantics, Django transaction
+  boundaries, bulk upload limits, and regression tests.
+- See `references/django-accounting-ui-formatting.md` for Django money-formatting
+  implementation details, template search targets, regression tests, and pitfalls.