DATU Artifact Retention Policy¶
Revision: 2025-10-12
This policy defines how long DATU teams must retain artefacts produced by the consensus network foundation workstream. It balances auditability, regulatory obligations, and storage cost so Phase 1 operations remain compliant.
1. Scope¶
The policy applies to artefacts generated by:
- Infrastructure automation (Docker image builds, container logs).
- Validator operations (SCP envelopes, history archive snapshots, quorum diagnostics).
- Incident response and compliance reviews (meeting minutes, evidence bundles).
Artefacts excluded from this policy follow the broader project data governance handbook.
2. Retention Schedule¶
| Artefact | Retention Period | Storage Location | Notes |
|---|---|---|---|
Docker images (stellar/quickstart fork builds) |
90 days | DATU container registry (registry.datu.foundation/stellar-fork) |
Latest successful build is pinned indefinitely. |
| Docker build logs & CI console output | 180 days | GitHub Actions log retention bucket | Export critical logs to incident evidence locker when rotations occur. |
| Stellar Core history archives | 365 days | Validator-specific S3 buckets (see config/quorumsets.toml) |
Apply Glacier transition after 120 days for cost optimisation. |
| Soroban RPC request traces | 30 days | Observability stack (Prometheus + Loki) | Aggregate metrics kept for 180 days. |
| Validator syslogs | 60 days | Centralised logging cluster | Redact sensitive data per security policy. |
| Incident evidence bundles | 7 years | Encrypted compliance vault | Includes key rotation artefacts, approvals, and retrospectives. |
| Governance meeting minutes related to validator operations | 4 years | Documentation portal (docs/operations/) |
Reference incident IDs in filenames. |
3. Roles & Responsibilities¶
| Role | Responsibilities |
|---|---|
| Infrastructure engineering | Configure CI pipelines to enforce log and artefact expiration; ensure registry cleanup jobs run weekly. |
| Validator operators | Validate history archives replicate to S3 with lifecycle policies, and document evidence transfers in the incident tracker. |
| Compliance liaison | Audit retention compliance quarterly, attest to regulators, and approve exceptions. |
| Security engineering | Review storage encryption settings annually and during any tooling change. |
4. Enforcement Controls¶
- Automation – GitHub Actions workflows must call the registry cleanup script after each daily build to prune images older than 90 days.
- S3 Lifecycle Rules – Apply bucket policies that transition objects to Glacier after 120 days and delete them after 365 days.
- Observability Stack – Configure retention via Helm charts (
values.yaml) to enforce log and metric deletion. - Incident Vault – Store evidence bundles in the dedicated compliance vault with envelope encryption. Access reviews occur quarterly.
5. Exception Handling¶
- Exceptions require written approval from the compliance liaison and security engineering.
- Approved exceptions must specify artefact type, extended retention duration, and expiry date.
- Log exceptions in the compliance tracker and revisit during quarterly audits.
6. Review Cadence¶
This policy is reviewed every 6 months or after any regulatory update impacting artefact storage. Proposed edits should be submitted via pull request with compliance sign-off before merging.
For operational procedures linked to this policy, see the Validator Key Rotation Playbook and the upcoming validator onboarding packet.