Skip to content

Persistence

Vega stores two categories of data: structured metadata (projects, repositories, scans, findings, users) and large objects (source code archives, scan reports, debug bundles). These use different storage systems.

Structured metadata

Structured metadata is stored either as JSON files (local development) or in Postgres (production).

JSON files under data/ — easy to inspect, easy to reset, no database required. Controlled by VEGA_PERSISTENCE_BACKEND=json.

A managed Postgres database via AWS RDS/Aurora. All records have proper types, foreign keys, and transactional guarantees. Controlled by VEGA_PERSISTENCE_BACKEND=postgres.

The application code calls PostgresStore (or the JSON equivalent) for all metadata operations. Route handlers and domain services never write raw SQL.

Large objects

Large files are stored either in local directories (development) or in S3 (production).

Files under subdirectories of data/ — simple, no AWS credentials required. Controlled by VEGA_FILE_STORAGE_BACKEND=local.

S3 object storage across multiple named buckets. Durable, scalable, and accessible from any ECS task. Controlled by VEGA_FILE_STORAGE_BACKEND=s3.

Key files

File What it does
app/storage/postgres.py PostgresStore — all Postgres operations
app/storage/s3.py S3 upload, download, and presigned URLs
app/storage/archive.py Safe archive extraction (size + path limits)
app/storage/migrations/ SQL files that evolve the schema over time
scripts/run-db-migrations.py Applies pending migrations locally
scripts/aws/run-migrations.sh Runs migrations via the ECS maintenance task

Pages in this section