152 lines
9.2 KiB
Markdown
152 lines
9.2 KiB
Markdown
# CLAUDE.md — SparKURS / Process-ADObjects
|
|
|
|
Briefing for Claude sessions on this repository. Read fully before touching code.
|
|
|
|
## Purpose
|
|
|
|
PowerShell system processing Active Directory group memberships for Sparkassen
|
|
banking institutes (currently R125, Y015). Reads AD export CSVs, transforms them
|
|
per institute, and writes CSV files consumed by (a) the DAW GUI import baskets
|
|
("Basket.Import.ADGroup.*") and (b) KNIME workflows. Runtime target: ~3 minutes
|
|
per institute (down from 10 h; do not regress this).
|
|
|
|
## File map
|
|
|
|
| File | Role |
|
|
|---|---|
|
|
| `Process-ADObjects.ps1` | Main entry, loads config, dot-sources the rest |
|
|
| `Process-ADObjects-Configuration.ps1` | `$Config` hashtable: institutes, GroupManagementMappings, flags (`TestMode`, `Verbose`, ...). Pure data, no logic |
|
|
| `Process-ADObjects-Base.ps1` | Core lookups: membership index (`newObjectMembershipsIndex`, `getObjectMemberships`, `getUsersByMemberGroup`), `getRoleType`, prefix matcher (`buildPrefixMatcher`, `testKnownPrefixes`) |
|
|
| `Process-ADObjects-Utility.ps1` | Logging (`openLog`/`closeLog`/`outLog`/`### outLogVerbose`), input checks, exception helpers, `removeDuplicatesFromSortedList` |
|
|
| `Process-ADObjects-Work.ps1` | Institute dispatch |
|
|
| `Process-ADObjects-Work-R125.ps1` / `-Y015.ps1` | Per-institute processing. Structurally identical copies (deduplication planned). Sections: FileSystemAccess, AppAccess, AppConfiguration, Organisation, Security, Anwenderrollen; then post-processing (sort → dedup → Neu derivation → file writes) |
|
|
| `Process-ADObjects-Analyze.ps1` | AD object analysis. Known incomplete; cleanup planned |
|
|
| `Process-ADObjects-Test.ps1` | Test scaffolding |
|
|
|
|
## Conventions (mandatory)
|
|
|
|
- **English** for all code, comments, identifiers, log messages. German only in chat.
|
|
- **Allman** brace style (opening brace on its own line).
|
|
- **`utf8NoBom`** for every file output; LF is fine (runs on macOS/pwsh 7).
|
|
- No `Write-Host` in processing code — use `outLog` / `### outLogVerbose`.
|
|
Exceptions: the output helpers themselves (`outLog`, `outScreen`, `dumpVar`,
|
|
`writeExceptionInfo*`).
|
|
- Institute code is parameterized via `$SPKID` (e.g. "R125") and `$SPKNO`
|
|
(e.g. "125"). No hardcoded institute literals in Work files.
|
|
- Code is currently intentionally comment-free (old comments stripped 2026-07;
|
|
re-commenting planned). Bug history lives in git and in this file.
|
|
|
|
## Logging system
|
|
|
|
- `outLog <args...>` — always-on log line; args are joined with **newline**.
|
|
For a single-line message concatenate in parentheses:
|
|
`outLog ("found " + $n + " entries")`.
|
|
- `outLog ... -ToScreen $false` — file only. Named parameter only; positional
|
|
args all go to the message (`PositionalBinding=$false` +
|
|
`ValueFromRemainingArguments`).
|
|
- `### outLogVerbose <args...>` — trace layer, gated by `$Config.Verbose`, never to
|
|
screen. ~440 call sites instrument every loop iteration and if/elseif/else
|
|
branch. With `Verbose = $true` a full run writes millions of lines — use
|
|
`TestMode` for verbose runs.
|
|
- Writer: persistent `StreamWriter` (UTF-8 no BOM, append, `AutoFlush=$true`),
|
|
auto-opens on first `outLog` and auto-reopens when `$LogFileName` changes.
|
|
`closeLog` at end is good style, not required. `tail -f` on the log works.
|
|
- Timestamp format is `HH:mm:ss.fff`. (`".mmm"` was a historic bug: minutes,
|
|
not milliseconds. Do not reintroduce.)
|
|
|
|
## Output post-processing (per Work file, end of run)
|
|
|
|
1. `Sort($ordinalComparer)` on all four collections
|
|
(`DAWGruppenAnlegenOutList`, `DAWGruppenverwaltungOutList`,
|
|
`KURSSollProfileOutList`, `KURSIstMitarbeiterProfileOutList`).
|
|
2. `removeDuplicatesFromSortedList` on each (logs removed count per list).
|
|
3. `DAWGruppenverwaltungNeuOutList` is **derived** from the sorted Out list —
|
|
never written inline. Rule: rows with empty `Accounts.Add` pass through;
|
|
first row per `GUIADGroup.LDAP` block keeps all columns; subsequent rows
|
|
carry only the account (`;;;;;<acct>;;;;`).
|
|
Open point: same account via multiple Quellgruppen still yields duplicate
|
|
reduced rows within a block (3-line fix pending DAW import behavior).
|
|
|
|
## Verification workflow
|
|
|
|
1. Syntax check before any run (parser API, no execution):
|
|
`[System.Management.Automation.Language.Parser]::ParseFile(...)` per file.
|
|
On error lists, fix the **bottom-most concrete** error (e.g. "Unexpected
|
|
token 'X'"); "Missing closing brace" messages above it are usually cascade.
|
|
2. Output comparison is always a **sorted multiset compare**, never a join:
|
|
`diff <(tail -n +3 REF.csv | sort -u) <(tail -n +3 NEW.csv | sort)`
|
|
(`+3`: skip Basket preamble + header; `-u` on the reference because current
|
|
code dedups and old references contain exact duplicates, e.g. 362 in the
|
|
R125 reference).
|
|
3. Reference files written under Windows PowerShell 5.1 have **UTF-8 BOM +
|
|
CRLF** and a semicolon-padded preamble (`ManageMembers;;;;;;;;;`); current
|
|
output has no BOM, LF, unpadded preamble. Normalize before comparing.
|
|
|
|
## Trap registry (hard-won; check before "optimizing")
|
|
|
|
- **PS 5.1 vs pwsh 7 divergences:**
|
|
- `Sort-Object -Property "Order"` on hashtables silently does NOT sort on
|
|
5.1. Always use scriptblock form: `Sort-Object { [int]$_.Order }`.
|
|
- `-Encoding utf8` writes BOM on 5.1, no BOM on 7. Always specify
|
|
`utf8NoBom` (or `utf8BOM` deliberately).
|
|
- Reference outputs originate from 5.1 → see verification §3.
|
|
- **R125 Proxy rules have `Order = 130/140`** (were 0/1). Reason: reference
|
|
behavior classifies `GGX-Proxy*` as `GPO`; on pwsh 7 a working sort would let
|
|
the (non-FI-Doku) Proxy rules win. Other institutes still have the 0/1
|
|
pattern — same trap waits there when validating against 5.1-era references.
|
|
- **`getUsersByMemberGroup` must index `$ADObjectMembershipsIndexed`** (the
|
|
hashtable). Indexing the flat array with a string key fails silently
|
|
($null → all indirect memberships vanish). Cost us 157k rows once.
|
|
- **Both membership expansion branches need their `$users = ...` assignment.**
|
|
A deleted line in the AppAccess group branch silently dropped 83k rows
|
|
($users was stale from the FS section).
|
|
- **Out vs Neu list Add targets:** adding a line to the wrong collection
|
|
duplicated every FS creation row. The Neu list is derived now, but the same
|
|
copy-paste class applies to the four remaining lists.
|
|
- **Verify ALL FOUR outputs after any Work-file change — not just one.** A sorted
|
|
multiset compare of a single output (e.g. Ist-MitarbeiterProfile) can be
|
|
byte-identical while a sibling (Soll-Profile) silently diverges. Seen: the
|
|
AppAccess `ProfilName` flipped from `"Anwendungszugriff <name>"` to the AD
|
|
description because a source-column read changed `$item.'Beschreibung'` (a
|
|
column that does NOT exist in `*_Anwendungen.csv` → always `$null` → empty →
|
|
fell through to the intended default) to `$item.'Beschreibung (256 Zeichen)'`
|
|
(the real, populated column). Ist / GruppenAnlegen / Gruppenverwaltung matched
|
|
exactly; only Soll-Profile differed — cols 3+6 of the 638 AppAccess rows.
|
|
Corollary: **a read of an absent column is load-bearing "always-empty"
|
|
behavior.** Do not "fix" a phantom column name to the real one without checking
|
|
the downstream `if (-not $x)` default it feeds; set the value to `""` explicitly
|
|
instead so a future rename can't resurrect the bug.
|
|
- **Prefix filter (Anwenderrollen):** `R125GUX*`/`<SPKID>GUX*` objects fall
|
|
THROUGH (original cascade had an empty first branch). The explicit
|
|
`StartsWith` check preserves this regardless of the prefix CSV content.
|
|
- **`+` in argument mode is a literal argument**, not concatenation:
|
|
`outLog "a" + $b` passes three args. Parenthesize: `outLog ("a" + $b)`.
|
|
- **Adding named parameters to `$args`-style functions** breaks positional
|
|
callers. Pattern: `[CmdletBinding(PositionalBinding=$false)]` +
|
|
`[Parameter(ValueFromRemainingArguments)]`.
|
|
- **Institute porting:** check for bare `SPKNO`/`SPKID` (missing `$`) —
|
|
happened twice in Y015. `grep -P '[^$\w]SPK(NO|ID)\b'`.
|
|
- **Editor hazards:** regex replace across guard lines and format-on-save have
|
|
each corrupted a file (interleaved fragments, brace salad). Prefer scripted,
|
|
validated transforms; always `git diff` after merging delivered files;
|
|
replace whole files rather than hand-merging.
|
|
- **KNIME comparisons are fragile:** BOM breaks column autodetection; the
|
|
`DOMAIN\user` backslash can be eaten as escape char; join column pairing
|
|
drifts (Add vs Rem columns); duplicate keys break join semantics. Use the
|
|
shell/PowerShell multiset compare instead.
|
|
- **`Start-ThreadJob` scriptblocks don't write parent variables** — the
|
|
`$startParallel` path in Analyze never worked. Needs `Receive-Job` if ever
|
|
activated.
|
|
- **Empty `catch {}` blocks** exist in both Work files (Write-Host removed) —
|
|
exceptions are currently swallowed there; add log lines when instrumenting.
|
|
|
|
## Backlog
|
|
|
|
- Re-comment the codebase (fresh, English). Document at minimum: scriptblock
|
|
sort rationale, GUX fall-through, Proxy Order renumbering, Neu derivation.
|
|
- Deduplicate R125/Y015 into a shared parameterized core.
|
|
- TestMode: implement consistently across all sections.
|
|
- Analyze: rework (incomplete; parallel path broken by design).
|
|
- Decide: preamble semicolon padding for DAW import; Neu per-block account
|
|
dedup.
|
|
- Y015: first verification run against its 5.1-era reference still pending.
|