Persistence

Vega stores two categories of data: structured domain records (projects, repositories, scans, findings, events, users, billing) and large objects (source code archives, scan reports, debug bundles). These use different storage systems.

Structured domain records

Structured records are stored either as JSON files (local development) or in Postgres (production).

All domain entities extend DomainModel (a Pydantic v2 base class). Records are stored as JSONB documents in Postgres — there is no ORM and no per-entity SQL schema beyond a common payload jsonb column. This is a document store pattern layered on top of a relational database:

-- Example: all scan records in one table
CREATE TABLE scans (
    record_key   TEXT PRIMARY KEY,
    payload      JSONB NOT NULL,
    created_at   TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    updated_at   TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

Local (JSON)Production (Postgres)

JSON files under data/ — easy to inspect, easy to reset, no database required. Controlled by VEGA_PERSISTENCE_BACKEND=json.

Each entity type gets its own JSON file (e.g., data/scans.json). The JSON adapter (adapters/storage/json/) reads and writes these files with record-level locking.

A managed Postgres database via AWS RDS/Aurora. All records are stored as JSONB documents in typed tables. Controlled by VEGA_PERSISTENCE_BACKEND=postgres.

The Postgres adapter (adapters/storage/postgres/) uses psycopg directly (no ORM). It serializes DomainModel instances to payload JSON on write and deserializes them on read.

Storage ports

The application layer never calls storage directly. It uses storage ports (interfaces) that are implemented by the adapters:

Port	Description
`ProjectStore`	`WorkspaceProject` CRUD
`RepositoryStore`	`Repository` CRUD
`SnapshotStore`	`SourceSnapshot` CRUD
`IngestJobStore`	`IngestJob` CRUD
`ScanStore`	`ScanRecord` CRUD
`FindingStore`	`FindingRecord` CRUD
`ArtifactStore`	`ArtifactRecord` CRUD
`EventStore`	Append-only `DomainEvent` log
`GenericRecordStore`	Document store for auxiliary records (users, API keys, billing, etc.)

Large objects

Large files are stored either in local directories (development) or in S3 (production).

Local (files)Production (S3)

Files under subdirectories of data/ — simple, no AWS credentials required. Controlled by VEGA_FILE_STORAGE_BACKEND=local.

The local object storage adapter (adapters/objects/local/) reads and writes files in data/snapshots/ and data/artifacts/.

S3 object storage across multiple named buckets. Durable, scalable, and accessible from any ECS task. Controlled by VEGA_FILE_STORAGE_BACKEND=s3.

In production, scan runners use scan-scoped STS credentials (ScanScopedS3Credentials) to access S3. This limits each runner to its own scan's artifacts.

Domain events

Domain events use a separate EventStore port. In Postgres, events are stored in the domain_events table (not in the generic JSONB pattern) with a monotonically increasing sequence number per aggregate:

CREATE TABLE domain_events (
    event_id     UUID PRIMARY KEY,
    event_type   TEXT NOT NULL,
    aggregate_id TEXT NOT NULL,
    payload      JSONB NOT NULL,
    sequence     BIGINT NOT NULL,
    occurred_at  TIMESTAMPTZ NOT NULL,
    UNIQUE (aggregate_id, sequence)
);

This ordering enables the frontend to efficiently poll for new events since a known sequence position.

Key files

File	What it does
`app/domain/base.py`	`DomainModel` — Pydantic v2 base for all records
`app/adapters/storage/postgres/`	`PostgresRecordStore` — all Postgres JSONB operations
`app/adapters/storage/json/`	`JsonRecordStore` — local JSON file store
`app/adapters/events/postgres/`	`PostgresEventStore` — domain_events table
`app/adapters/events/json/`	`JsonEventStore` — local JSON event log
`app/adapters/objects/s3/`	`S3ObjectStorage` — S3 upload/download/presigned URLs
`app/adapters/objects/local/`	`LocalObjectStorage` — local file storage
`app/storage/migrations/`	SQL migration files (001–028)
`scripts/run-db-migrations.py`	Applies pending migrations locally
`scripts/aws/run-migrations.sh`	Runs migrations via the ECS maintenance task

Pages in this section

Postgres — JSONB schema, the GenericRecordStore, and debugging
S3 and Artifacts — buckets, object keys, and what gets stored
Migrations — how to apply schema changes and the full migration list