# CLAUDE.md — SparKURS / Process-ADObjects

Briefing for Claude sessions on this repository. Read fully before touching code.

## Purpose

PowerShell system processing Active Directory group memberships for Sparkassen
banking institutes (currently R125, Y015). Reads AD export CSVs, transforms them
per institute, and writes CSV files consumed by (a) the DAW GUI import baskets
("Basket.Import.ADGroup.*") and (b) KNIME workflows. Runtime target: ~3 minutes
per institute (down from 10 h; do not regress this).

## File map

| File | Role |
|---|---|
| `Process-ADObjects.ps1` | Main entry, loads config, dot-sources the rest |
| `Process-ADObjects-Configuration.ps1` | `$Config` hashtable: institutes, GroupManagementMappings, flags (`TestMode`, `Verbose`, ...). Pure data, no logic |
| `Process-ADObjects-Base.ps1` | Core lookups: membership index (`newObjectMembershipsIndex`, `getObjectMemberships`, `getUsersByMemberGroup`), `getRoleType`, prefix matcher (`buildPrefixMatcher`, `testKnownPrefixes`) |
| `Process-ADObjects-Utility.ps1` | Logging (`openLog`/`closeLog`/`outLog`/`outLogVerbose`), input checks, exception helpers, `removeDuplicatesFromSortedList` |
| `Process-ADObjects-Work.ps1` | Institute dispatch |
| `Process-ADObjects-Work-R125.ps1` / `-Y015.ps1` | Per-institute processing. Structurally identical copies (deduplication planned). Sections: FileSystemAccess, AppAccess, AppConfiguration, Organisation, Security, Anwenderrollen; then post-processing (sort → dedup → Neu derivation → file writes) |
| `Process-ADObjects-Analyze.ps1` | AD object analysis. Known incomplete; cleanup planned |
| `Process-ADObjects-Test.ps1` | Test scaffolding |

## Conventions (mandatory)

- **English** for all code, comments, identifiers, log messages. German only in chat.
- **Allman** brace style (opening brace on its own line).
- **`utf8NoBom`** for every file output; LF is fine (runs on macOS/pwsh 7).
- No `Write-Host` in processing code — use `outLog` / `outLogVerbose`.
  Exceptions: the output helpers themselves (`outLog`, `outScreen`, `dumpVar`,
  `writeExceptionInfo*`).
- Institute code is parameterized via `$SPKID` (e.g. "R125") and `$SPKNO`
  (e.g. "125"). No hardcoded institute literals in Work files.
- Code is currently intentionally comment-free (old comments stripped 2026-07;
  re-commenting planned). Bug history lives in git and in this file.

## Logging system

- `outLog <args...>` — always-on log line; args are joined with **newline**.
  For a single-line message concatenate in parentheses:
  `outLog ("found " + $n + " entries")`.
- `outLog ... -ToScreen $false` — file only. Named parameter only; positional
  args all go to the message (`PositionalBinding=$false` +
  `ValueFromRemainingArguments`).
- `outLogVerbose <args...>` — trace layer, gated by `$Config.Verbose`, never to
  screen. ~440 call sites instrument every loop iteration and if/elseif/else
  branch. With `Verbose = $true` a full run writes millions of lines — use
  `TestMode` for verbose runs.
- Writer: persistent `StreamWriter` (UTF-8 no BOM, append, `AutoFlush=$true`),
  auto-opens on first `outLog` and auto-reopens when `$LogFileName` changes.
  `closeLog` at end is good style, not required. `tail -f` on the log works.
- Timestamp format is `HH:mm:ss.fff`. (`".mmm"` was a historic bug: minutes,
  not milliseconds. Do not reintroduce.)

## Output post-processing (per Work file, end of run)

1. `Sort($ordinalComparer)` on all four collections
   (`DAWGruppenAnlegenOutList`, `DAWGruppenverwaltungOutList`,
   `KURSSollProfileOutList`, `KURSIstMitarbeiterProfileOutList`).
2. `removeDuplicatesFromSortedList` on each (logs removed count per list).
3. `DAWGruppenverwaltungNeuOutList` is **derived** from the sorted Out list —
   never written inline. Rule: rows with empty `Accounts.Add` pass through;
   first row per `GUIADGroup.LDAP` block keeps all columns; subsequent rows
   carry only the account (`;;;;;<acct>;;;;`).
   Open point: same account via multiple Quellgruppen still yields duplicate
   reduced rows within a block (3-line fix pending DAW import behavior).

## Verification workflow

1. Syntax check before any run (parser API, no execution):
   `[System.Management.Automation.Language.Parser]::ParseFile(...)` per file.
   On error lists, fix the **bottom-most concrete** error (e.g. "Unexpected
   token 'X'"); "Missing closing brace" messages above it are usually cascade.
2. Output comparison is always a **sorted multiset compare**, never a join:
   `diff <(tail -n +3 REF.csv | sort -u) <(tail -n +3 NEW.csv | sort)`
   (`+3`: skip Basket preamble + header; `-u` on the reference because current
   code dedups and old references contain exact duplicates, e.g. 362 in the
   R125 reference).
3. Reference files written under Windows PowerShell 5.1 have **UTF-8 BOM +
   CRLF** and a semicolon-padded preamble (`ManageMembers;;;;;;;;;`); current
   output has no BOM, LF, unpadded preamble. Normalize before comparing.

## Trap registry (hard-won; check before "optimizing")

- **PS 5.1 vs pwsh 7 divergences:**
  - `Sort-Object -Property "Order"` on hashtables silently does NOT sort on
    5.1. Always use scriptblock form: `Sort-Object { [int]$_.Order }`.
  - `-Encoding utf8` writes BOM on 5.1, no BOM on 7. Always specify
    `utf8NoBom` (or `utf8BOM` deliberately).
  - Reference outputs originate from 5.1 → see verification §3.
- **R125 Proxy rules have `Order = 130/140`** (were 0/1). Reason: reference
  behavior classifies `GGX-Proxy*` as `GPO`; on pwsh 7 a working sort would let
  the (non-FI-Doku) Proxy rules win. Other institutes still have the 0/1
  pattern — same trap waits there when validating against 5.1-era references.
- **`getUsersByMemberGroup` must index `$ADObjectMembershipsIndexed`** (the
  hashtable). Indexing the flat array with a string key fails silently
  ($null → all indirect memberships vanish). Cost us 157k rows once.
- **Both membership expansion branches need their `$users = ...` assignment.**
  A deleted line in the AppAccess group branch silently dropped 83k rows
  ($users was stale from the FS section).
- **Out vs Neu list Add targets:** adding a line to the wrong collection
  duplicated every FS creation row. The Neu list is derived now, but the same
  copy-paste class applies to the four remaining lists.
- **Prefix filter (Anwenderrollen):** `R125GUX*`/`<SPKID>GUX*` objects fall
  THROUGH (original cascade had an empty first branch). The explicit
  `StartsWith` check preserves this regardless of the prefix CSV content.
- **`+` in argument mode is a literal argument**, not concatenation:
  `outLog "a" + $b` passes three args. Parenthesize: `outLog ("a" + $b)`.
- **Adding named parameters to `$args`-style functions** breaks positional
  callers. Pattern: `[CmdletBinding(PositionalBinding=$false)]` +
  `[Parameter(ValueFromRemainingArguments)]`.
- **Institute porting:** check for bare `SPKNO`/`SPKID` (missing `$`) —
  happened twice in Y015. `grep -P '[^$\w]SPK(NO|ID)\b'`.
- **Editor hazards:** regex replace across guard lines and format-on-save have
  each corrupted a file (interleaved fragments, brace salad). Prefer scripted,
  validated transforms; always `git diff` after merging delivered files;
  replace whole files rather than hand-merging.
- **KNIME comparisons are fragile:** BOM breaks column autodetection; the
  `DOMAIN\user` backslash can be eaten as escape char; join column pairing
  drifts (Add vs Rem columns); duplicate keys break join semantics. Use the
  shell/PowerShell multiset compare instead.
- **`Start-ThreadJob` scriptblocks don't write parent variables** — the
  `$startParallel` path in Analyze never worked. Needs `Receive-Job` if ever
  activated.
- **Empty `catch {}` blocks** exist in both Work files (Write-Host removed) —
  exceptions are currently swallowed there; add log lines when instrumenting.

## Backlog

- Re-comment the codebase (fresh, English). Document at minimum: scriptblock
  sort rationale, GUX fall-through, Proxy Order renumbering, Neu derivation.
- Deduplicate R125/Y015 into a shared parameterized core.
- TestMode: implement consistently across all sections.
- Analyze: rework (incomplete; parallel path broken by design).
- Decide: preamble semicolon padding for DAW import; Neu per-block account
  dedup.
- Y015: first verification run against its 5.1-era reference still pending.