Data retention: purge expired auth rows, audit retention and client deletion #8
Labels
No labels
priority/high
priority/low
priority/medium
type/feature
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
ben/career-values-cards#8
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Define and implement data retention once the practice confirms its retention policy.
Today nothing is purged automatically:
auth_sessionsandlogin_tokensrows;The mail outbox is the exception: it already purges after 30 days.
Scope (to be confirmed with the practice)
docs/self-hosting.mdand the privacy checklist.Constraints
Acceptance
Depends on: the practice's retention policy, for everything except the auth-row purge.
The auth-row purge landed in
16705ebonmain. Not deployed yet; deploying is a normal rebuild with no migration.server/retention.js). Session and link validity come from shared predicates inserver/auth-validity.js, which authentication uses too.server/test/retention.test.jscover time boundaries, equivalence with real requests, refresh/consume/issuance ordering, SSE closure after commit, the token × outbox matrix, dashboard invite status, the scheduler lifecycle and shutdown.docs/plans/2026-10-14-21-34_purge-expired-auth-rows.md(done).docs/self-hosting.md.Left on this issue:
outbox(token_hash), deferred because it needs a migration.The auth-row purge (
16705eb) is deployed, live at13541d6, schema v5, no migration. On startup the live server purged 1 dead sign-in session and 2 spent sign-in links. Smoke checks passed: healthz ok, signed-out requests get 401, the container is healthy, and the mail tunnel is active.Still open on this issue:
outbox(token_hash).The first two wait on the practice's retention policy.
Decisions (maintainer, 2026-10-15):
Constraint still in force: content-mutation audit rows drive dashboard progress, so a purge must keep them, or progress must move to its own store first. A hard delete removes those rows only together with the client's sessions.
Index: the
outbox(token_hash)index is filed separately.Access-audit retention research. This is not legal advice; counsel should confirm.
Finding: no Alberta or federal statute, regulation or regulator guidance sets a retention period for staff access logs. The period rests on PIPA s.35(1): keep information "only for as long as the organization reasonably requires" it for legal or business purposes, then destroy it or make it non-identifying within a reasonable time (s.35(2)).
What applies:
Recommended defaults (configurable):
For the practice to confirm with counsel:
Sources (primary text unless noted):
Next: the plan for audit retention plus hard client deletion, using these defaults. Nothing has been committed for it yet.
Decisions (maintainer, 2026-10-15):
The plan for the rest of this issue landed in
a0f5627:docs/plans/2026-10-15-10-58_audit-retention-hold-client-deletion.md(status: draft). It was reviewed in a High consult with codex and glm (2/2); its Review notes record every point and how it was resolved. No implementation code has been committed yet.Approach:
activity.finishedis also retained. The settings areAUDIT_RETENTION_DAYS,AUDIT_IP_RETENTION_DAYSandRETENTION_PAUSED.legal_holdstable, with owner-onlyPUTandDELETE /api/users/:id/hold. For a held client, the hold suspends the audit purge, IP blanking, and the auth-row and outbox purges. While any hold exists, unattributable audit rows are kept. A held client can't be deleted or have their role changed. The docs will ask for an evidence snapshot when a hold is set.POST /api/users/:id/delete {email}(owner only), pluscli.js delete-clientfor re-applying a deletion after a restore. It runs in one immediate transaction, with an FK check before commit. Staff access rows are kept, withsession_idset to NULL so they refer only to UUIDs that no longer resolve. The deletion writes aclient.deleteaudit row and closes the affected streams. Accounts with staff history are refused.secure_deleteis turned on, and the migration runs a one-timeVACUUM. There is no automatic backup before a deletion; the docs cover backup age and the restore procedure.legal_holds, plus the indexesoutbox_token_hash(#13),audit_atandaudit_actor.Left: implementing the seven slices, then a High code review, a migration dry run on a copy of the live database, and deploying when the maintainer asks. The counsel questions are listed in the plan and don't block the work.
Implemented in
8754704onmain. Not deployed yet. Deploying needs migration v6, which takes an automatic backup and runs a one-time VACUUM.Audit retention:
PURGEABLE_ACTIONSallowlist (server/audit-classes.js) are deleted 730 days after the event (AUDIT_RETENTION_DAYS).AUDIT_IP_RETENTION_DAYS).activity.finishedrows stay with their session.RETENTION_PAUSED=truesuspends every retention purge.server/retention.jscreateRetentionPurger), and ends each run with a WAL checkpoint.Legal hold:
PUTandDELETE /api/users/:id/hold. Each change is audited; the reason is stored only inlegal_holds.Hard client deletion:
POST /api/users/:id/deletewith the typed email, orcli.js delete-clientfor re-applying deletions after a restore.cli.js release-holdsupports the same procedure.client.deleterow is written, and the affected streams close.secure_deleteandjournal_size_limit = 0on every connection, plus checkpoints, keep deleted data out of the file and the WAL.Docs:
docs/self-hosting.md§2, §5, §9, §10 and §11: the privacy checklist, settings, deleting and holds, and backups with the restore procedure;docs/api.md;docs/deployment.md.Verification:
tscis clean, 643 unit tests pass, coverage is at least 85% per file, and Playwright passes 10 with 1 skipped, including a new People journey (hold, release, delete).docs/plans/2026-10-15-10-58_audit-retention-hold-client-deletion.md. 7/7 slices and 21/22 acceptance items have tests.Left:
docs/self-hosting.md§2. They don't block the work.This issue closes once v6 is deployed and verified.
Live: deployed at
ec6f0cb, schema v6. A pre-deploy backup was taken first. After the migration:integrity_checkreturned ok,foreign_key_checkwas clean, and the free list was 0;legal_holdsand the indexesoutbox_token_hash,audit_atandaudit_actorexist;RETENTION_PAUSED=false, with the defaults of 730 and 90 days;Dry run (before the deploy): run on a copy of the live database on a dev host. It passed 47 of 47 checks.
pre-v5backup was written with mode 600;integrity_checkreturned ok andforeign_key_checkwas clean;docs/self-hosting.md§10 restore procedure: a synthetic client was deleted with a hold set (refused, nothing changed), then afterrelease-hold(deleted). Afterwards:The details are in the plan's completion notes (
ec6f0cb).Acceptance:
16705eb, live since13541d6).8754704; tests inserver/test/client-deletion.test.js, plus the dry run above).Still open, outside this issue: the counsel questions in
docs/self-hosting.md§2 (the retention period, the backup age limit, when to release a hold). Neither the periods norRETENTION_PAUSEDneed a code change.