Skip to content

Core Architecture

Vega Backend is structured as a set of cooperating processes built on a hexagonal (ports-and-adapters) architecture. Understanding the layers and how the processes fit together makes it much easier to debug issues, extend behavior, or deploy the system.

The big picture

flowchart TD
    subgraph clients["Clients"]
        FE[React dashboard]
        CLI[API clients]
    end

    subgraph api_layer["API layer (ECS Fargate)"]
        API[vega-api\nFastAPI on port 8000]
    end

    subgraph workers["Background processes (ECS Fargate)"]
        SW[vega-scan-worker]
        IW[vega-repo-ingest-worker]
        Proxy[vega-llm-proxy]
        MT[vega-maintenance]
    end

    subgraph runners["Ephemeral runners (ECS RunTask)"]
        SR[vega-scan-runner\none per scan phase]
        IR[vega-repo-ingest-runner\none per ingest job]
    end

    subgraph storage["Storage"]
        PG[(Postgres JSONB)]
        S3[(S3)]
        SQS_S[SQS scan queue]
        SQS_I[SQS ingest queue]
    end

    FE --> API
    CLI --> API
    API --> PG
    API --> S3
    API --> SQS_S
    API --> SQS_I

    SQS_S --> SW
    SQS_I --> IW
    SW -->|ECS RunTask| SR
    IW -->|ECS RunTask| IR
    SR --> PG
    SR --> S3
    SR --> Proxy
    IR --> PG
    IR --> S3
    Proxy --> Provider[AI provider]

Architecture layers

The codebase follows hexagonal architecture with four explicit layers:

app/
├── entrypoints/    ← process entry points (API, workers, runners, maintenance)
├── api/            ← HTTP routes (thin — delegate to use cases via container)
├── application/    ← use-case classes: all business logic lives here
├── domain/         ← pure domain models (Pydantic), state machines, value objects
├── ports/          ← Python Protocol interfaces (no implementations)
├── adapters/       ← implementations: Postgres, S3, SQS, ECS, Cognito, GitHub, Stripe…
└── composition/    ← wiring: builds RuntimeContainer from settings + adapters

How a request flows:

HTTP request
  → api/routers/*.py         (validate input, extract CurrentUser)
  → application/*UseCase     (business logic, state transitions)
  → ports/*Port              (interface boundary)
  → adapters/*               (Postgres / S3 / SQS / ECS / …)

Application code depends only on ports/ interfaces. The composition/wiring/ layer selects the correct adapter implementations at startup based on settings.

The domain model

Everything in Vega is organized around a hierarchy of entities:

WorkspaceProject
└── Repository
    ├── SourceSnapshot   (immutable source capture)
    ├── IngestJob        (tracks the ingest pipeline)
    └── ScanRecord
        ├── DomainEvent[]   (append-only event log)
        ├── FindingRecord[] (security issues)
        ├── ArtifactRecord[] (scan output files)
        └── RunnerJob       (ECS task record)

A WorkspaceProject groups related repositories. A Repository is the source code target. When a repository is added, the system runs an ingest pipeline (clone → extract → snapshot) tracked by an IngestJob. A ScanRecord is one audit run against a snapshot; it produces DomainEvents (live progress), FindingRecords (security issues), and ArtifactRecords (output files).

See Data Model for the full entity reference.

Process separation

Vega separates concerns across seven named service roles:

Role What it does
vega-api Handles all HTTP traffic. Never runs scans or ingest directly in production.
vega-scan-worker Polls SQS for scan jobs, claims them, launches scan runner ECS tasks.
vega-scan-runner Runs exactly one scan phase (plan/audit/verify) and exits.
vega-repo-ingest-worker Polls SQS for ingest jobs, claims them, launches ingest runner ECS tasks.
vega-repo-ingest-runner Clones / materializes one repository and exits.
vega-llm-proxy Proxies AI requests with per-scan usage limits and credential isolation.
vega-maintenance One-off tasks: migrations, stale scan cleanup, artifact retention.

This separation means scan failures don't crash the API, runaway AI spend is capped per scan, ingest failures don't block scan workers, and sensitive credentials are isolated.

Two runtime shapes

Vega intentionally supports two ways of running:

Everything runs on one machine with no cloud dependencies. State lives in JSON files (or a local Postgres database if you prefer). Scans run inside the API process (thread mode) or in a separate worker process. Ingest runs in-process. The scan engine runs Codex in a local Docker container.

# Terminal 1 — API (scan execution in-process via thread mode)
uvicorn app.main:app --reload --reload-dir app

# Terminal 2 — React frontend
cd frontend && npm run dev

Everything is separated and durable. State lives in Postgres (JSONB) and S3. Scans and ingest jobs are queued via SQS, claimed by Fargate workers, and executed in isolated Fargate runner tasks.

CloudFront → ALB → vega-api (ECS)
                      ↓
                   SQS ← scan created
                      ↓
              vega-scan-worker (ECS) ← polls SQS
                      ↓
        vega-scan-runner (ECS RunTask) ← one per scan phase

SQS ← repository added
  ↓
vega-repo-ingest-worker (ECS) ← polls SQS
  ↓
vega-repo-ingest-runner (ECS RunTask) ← one per repo ingest

Code organization

The codebase has six layers:

Entrypoints (app/entrypoints/) — Process entry points: create_app() for the API, worker polling loops, runner main functions, maintenance tasks, and the LLM proxy app.

API layer (app/api/) — Route handlers. They validate input, resolve CurrentUser via auth dependency, retrieve use cases from the RuntimeContainer, and return response models. No business logic lives here.

Application layer (app/application/) — Use-case classes. Each operation is a *UseCase class with *Command input and *Result output. Business rules, state transitions, and orchestration all live here.

Domain layer (app/domain/) — Pure Pydantic models. State machines, value objects, and domain event types. No I/O, no framework dependencies.

Ports layer (app/ports/) — Python Protocol interfaces that the application layer depends on. No implementations.

Adapters layer (app/adapters/) — Implementations of ports: Postgres, S3, SQS, ECS, Cognito, GitHub App, Stripe, Sub2API, and local/in-memory variants for development.

Composition (app/composition/) — RuntimeContainer that wires adapters to ports based on RuntimeSettings. The entrypoints call build_*_runtime() to get a fully-wired container.

Next steps