Skip to content

SkyPurza — Design v1 (M0)

Status: M0 scaffold implemented (synth-only, no AWS deployment). Mirrors PLAN.md; decisions in prompts.md (generated copies in the docs site; sources live at the project root).

1. What SkyPurza is

An open-core multi-cloud GPU broker: the OSS broker follows the SkyPilot open standard (task YAML accepted verbatim, CLI verb parity), while closed-source platform services (accounting, automatic dispatching, observability, console) attach through explicit extension points and are the monetization surface.

2. Repo topology (hybrid monorepo, D1/D2/D10)

skypurza/
├── packages/oss/                 # Apache-2.0 intent — Python 3.11+, uv workspace
│   ├── shared/                   #   contracts: JSON Schema + pydantic + naming (+ zod, TS side)
│   ├── broker-core/              #   optimizer, provisioner state machine, job queue
│   ├── adapters/aws/             #   EC2 backend (boto3) behind the Backend seam
│   ├── broker-cli/               #   `spz` CLI: launch / status / down / check
│   └── api-server/               #   FastAPI /v1/clusters + extension points
├── packages/platform/            # closed-source — TypeScript, pnpm workspace
│   ├── accounting-service/       #   implements AccountingHook        (placeholder)
│   ├── dispatch-service/         #   implements DispatchPolicy        (placeholder)
│   ├── observability-service/    #   implements MetricsSink           (placeholder)
│   └── frontend/                 #   consumes api-server REST/OpenAPI (placeholder)
├── packages/infra/               # TypeScript CDK: naming ratchet, DNS, pipeline
├── scripts/                      # ci.sh, quality gates
└── docs/
  • Python side: one uv workspace rooted at pyproject.toml (shared uv.lock), hatchling src-layout packages, pytest.
  • TypeScript side: pnpm workspace (pnpm-workspace.yaml), per-package tsconfig extending tsconfig.base.json, vitest.
  • packages/oss/shared is deliberately hybrid (pyproject + package.json): the contract has exactly one home.

3. Contracts: JSON-Schema-first (D10)

packages/oss/shared/src/skypurza_shared/schemas/task.schema.json is the canonical SkyPilot-compatible task contract (resources.accelerators/infra/ cpus/memory, num_nodes, setup, run, envs, file_mounts; unknown fields allowed for forward compatibility). Two mirrors, both tested against it:

  • pydantic (skypurza_shared.task.Task) — runtime validation in the broker; Task.from_yaml validates against the JSON Schema first, then parses into models.
  • zod (@skypurza/shared, src/task.ts) — platform/frontend typing.

Anti-drift: conformance tests assert the pydantic/zod field sets equal the schema's properties exactly, and the same three fixture YAMLs (taken from SkyPilot docs patterns) must validate under JSON Schema, pydantic, and zod. The api-server's openapi.json is committed and drift-checked in CI.

4. Broker data flow

task.yaml ──> spz CLI ─────────────┐
              (or) POST /v1/clusters│
                                   ▼
                        Task (schema + pydantic)
                                   ▼
          Optimizer.plan(task) ──> [RegionPlan...]   (price-ranked, cheapest first)
                                   ▼
          DispatchPolicy.choose()  (extension point; passthrough in OSS)
                                   ▼
          Provisioner.launch() ──> Backend.launch(region, ...)
             PENDING→PROVISIONING     │ CapacityError → next region (failover)
                      →RUNNING ◄──────┘
                                   ▼
          AccountingHook / MetricsSink notified (extension points)
  • Optimizer (broker-core/optimizer.py): static per-region price tables (on-demand and spot) over a supported instance matrix (t3/t4g CPU, g4dn/g5/g6/p3/p4d GPU) across us-east-1, us-east-2, us-west-2, eu-west-1; SkyPilot-style accelerator resolution keyed on (name, count) (A10G:1→g5.xlarge, V100:4→p3.8xlarge, …), cpus/memory fit on the t3 ladder, instance_type pin wins. Region allowlist narrows via resources.infra (aws or aws/us-east-1). Candidates sorted (price, region) per market; with use_spot: true every spot candidate precedes every on-demand candidate (any_of spot→on-demand fallback). The plan order is the failover order.
  • Provisioner (broker-core/provisioner.py): validated state machine PENDING→PROVISIONING→RUNNING→STOPPING→TERMINATED (failed launches go to TERMINATED); records keep state history plus region/market/IPs/price; reconcile() syncs a RUNNING record against live backend state.
  • Backend seam (broker-core/backend.py): launch/describe/terminate protocol (market + region-hint aware). Implementations: FakeBackend (in-memory, deterministic — drives core tests and SKYPURZA_BACKEND=fake) and the real Ec2Backend (adapters/aws): AMI via SSM public parameters (AL2023, arch-aware incl. arm64), a create-or-reuse stage-scoped managed SG (skypurza-<stage>-cluster-sg, SSH-only ingress, tag-discoverable), per-cluster EC2 key pairs (PEM under ~/.skypurza/keys, deleted on terminate), spot via InstanceMarketOptions, wait-until-running/terminated, and capacity-error mapping (InsufficientInstanceCapacity, Unsupported, MaxSpotInstanceCountExceeded, …) to CapacityError for failover. Runtime resources carry Project/Stage + ManagedBy=skypurza-broker tags.
  • Execution (broker-core/ssh.py + support.py): a paramiko SshRunner streams setup/run on node 0 with SkyPilot-compatible env (SKYPILOT_NODE_RANK/NODE_IPS/NUM_NODES/NUM_GPUS_PER_NODE + task envs). Parsed-but-unexecutable M1 features (file_mounts, workdir, multi-node execution, image_id, ports, disk_tier, zone pins) raise explicit NotYetSupportedErrors at the CLI — never silent drops.
  • State store (broker-cli/statestore.py): SQLite at ~/.skypurza/state.db (SKYPURZA_STATE_DB override) — clusters, nodes (instance ids + IPs), status_history tables; spz status --refresh reconciles records against live EC2 and persists the result.
  • JobQueue (broker-core/queue.py): in-memory FIFO with validated job status transitions (single-process).

5. Extension-point seam (the open-core boundary, D2/D3)

Defined in OSS (skypurza_api_server.extensions), implemented by platform:

Protocol OSS default Platform implementor
AccountingHook (cluster launched/terminated) no-op accounting-service
DispatchPolicy (reorder/filter region plan) passthrough dispatch-service
MetricsSink (operational metrics) no-op observability-service

The frontend consumes the api-server REST contract (openapi.json) and the shared zod types. Dependency direction is one-way (platform → oss); scripts/quality/boundary-gate.sh fails CI on any OSS→platform reference.

6. Naming & tagging (shared AWS account, D6)

Format: resources skypurza-<stage>-<component>-<purpose> (≤63, kebab, charset-validated), stacks Skypurza<Stage><Id> (≤128), paths /skypurza/<stage>/..., tags Project=skypurza + Stage=<stage>.

Three enforcement layers (kwedra-roko ratchet, from day one):

  1. Two mirrored namers — skypurza_shared.naming (Python) and packages/infra/lib/naming.ts (CDK) — conformance-tested against the same fixture file (packages/oss/shared/conformance/naming-fixtures.json).
  2. Synth-time NamingAspect — every explicitly named CFN resource must carry the stage's namer prefix; violations fail cdk synth. The aspect is stage-boundary-aware (each cdk.Stage enforces its own namer).
  3. scripts/quality/naming-gate.sh — greps source for hardcoded stage-qualified brand literals outside the namers/config/tests/fixtures.

7. Infra & delivery (packages/infra)

  • Config: config/{beta,prod}.ts (account 355390957370, us-west-2), config/pipeline.ts (CodeCommit deploy-conduit source: repo skypurza, branch main — D17/D19).
  • DnsStack: imports the existing hosted zone skypurza.ranarhi.people.aws.dev (Z04475113SDZIHOOKV41I) by id — never creates it (D7). Beta creates + delegates beta.<parent>; prod uses the parent zone bare. Domains come from namer.domainName().
  • BrokerStack: stage log group /skypurza/<stage>/broker + SSM scaffold marker, plus the deployed api-server: FastAPI via Mangum on Lambda (python3.12, bundle from scripts/build-api-lambda.sh) behind an API Gateway HTTP API with an ACM DNS-validated custom domain on the stage domain, aliased in the stage zone. The public endpoint runs the in-memory FakeBackend runtime by design — the real EC2 backend is never exposed unauthenticated.
  • OpsStack: per-stage operational monitoring (kwedra canary pattern). A python3.12 canary Lambda (skypurza-<stage>-canary, code under packages/infra/canary/) runs on an EventBridge rate(5 minutes) schedule and probes the public endpoint end to end: landing page, /openapi.json, and a /v1/clusters create→list→get→delete round-trip with a unique canary-<ms> name (tolerant of cross-Lambda-container 404s on the FakeBackend). It emits Skypurza/<Stage> metrics (Success, LatencyMs) and raises on failure so the function's Errors metric fires too. Alarms: canary-failure (missing data = breaching), api-errors (api Lambda Errors), and the composite skypurza-<stage>-release-health (OR of both) notifying an unencrypted SNS topic — the aws/sns managed key is unusable by CloudWatch alarms (kwedra lesson). Intended to gate pipeline bakes later.
  • Beta deployment (live): SkypurzaBetaDns + SkypurzaBetaBroker + SkypurzaBetaOps + SkypurzaBetaDocs are deployed directly (make deploy-beta / make deploy-ops / make docs-deploy) to account 355390957370 / us-west-2 (docs in us-east-1), reusing the account's existing default-qualifier CDK bootstrap. The api-server is served at https://beta.skypurza.ranarhi.people.aws.dev (/, /docs, /openapi.json); scripts/beta-smoke.sh is the repeatable post-deploy check (API + docs site).
  • DocsStack (D16): the interactive docs site at https://docs.<stage domain> — an MkDocs Material build (architecture with mermaid diagrams, alarm runbooks and manual processes, testing guide, a Make-targets page generated from make help, embedded live Swagger UI) in a private S3 bucket (skypurza-<stage>-docs-site) behind CloudFront with origin access control, a namer-named CloudFront function rewriting directory URIs to index.html, and an ACM DNS-validated cert. The stack lives in us-east-1 (CloudFront certificate requirement) and references the stage zone cross-region (crossRegionReferences). Built by scripts/docs-build.sh (also a --strict offline step in ci.sh); tests inject an inline docsSource the same way apiCode is injected.
  • PipelineStack (SkypurzaPipeline, deployed): self-mutating CDK Pipelines fed by the CodeCommit deploy conduit skypurza (snapshot-synced from hpc.git by make push-pipeline, D19) — Source → Synth (scripts/ci.sh) → SkypurzaBeta → BetaSmoke gate → SkypurzaBakeUsw2 (placeholder) → PromoteToProd manual approval → SkypurzaProd → SkypurzaDrUse1 (placeholder, us-east-1). Beta auto-deploys on every push and is gated post-deploy by scripts/beta-smoke.sh; prod never deploys without the approval (it publishes real public resources).

8. Testing strategy (D4)

  • Hermetic by construction: core tests run on FakeBackend; AWS adapter tests on moto (mock_aws) — SSM AMI resolution, SG/key lifecycle, spot lifecycle, ICE region failover, spot→on-demand fallback, capacity-code mapping, teardown hygiene — via injected failing clients; the SSH runner against a REAL in-process paramiko server (full handshake + exec); CLI via typer's CliRunner on moto + temp SQLite store; API via FastAPI TestClient; infra via vitest + Template.fromStack assertions and full app.synth(). CI needs no AWS credentials.
  • SkyPilot parity corpus: 10 real-world task YAMLs under packages/oss/shared/fixtures/ (minimal, gpu, multi-node, spot, envs, file_mounts, region/instance pins) — all must parse; unsupported features surface as NotYetSupportedError.
  • Contract tests keep schema/pydantic/zod/namers from drifting (section 3/6).
  • scripts/ci.sh (= make ci) is the single gate: uv sync + pytest → OpenAPI drift → pnpm build + vitest → boundary gate → naming gate → docs site build (MkDocs --strict, offline) → cdk synth --all.
  • scripts/live-smoke.sh (gated, optional, NOT in ci.sh): one real t3.micro in the cheapest allowed region (stage=beta tags), verify status, terminate, assert zero residue (no live tagged instances anywhere, key pair
  • PEM deleted; the managed SG persists by design and is tag-discoverable). Skips with an explicit message when credentials are unavailable.
  • Test pyramid above the hermetic tiers (D14) — online lanes, never in ci.sh:
  • E2E beta lane (packages/e2e-tests, scripts/e2e.sh beta): the live beta endpoint over plain HTTPS — landing, Swagger docs, served OpenAPI deep-equals the committed spec, full cluster CRUD lifecycle with a unique name and retry tolerance for cross-container FakeBackend state.
  • E2E live-aws lane (scripts/e2e.sh live-aws, cred-gated like live-smoke): real spz launch → exec → status --refresh → down on a t3.micro plus zero-residue assertions; pytest-skips with a reason when credentials are missing.
  • UI tests (packages/ui-tests, Playwright/chromium, Node ≥ 20): landing page content, Swagger UI operation list, openapi.json validity against the beta website; skip gracefully when unreachable.
  • Canary (OpsStack, section 7): continuous 5-minute probe of the beta endpoint feeding the skypurza-beta-release-health composite alarm.

9. Conventions

  • The root Makefile is the single entry point (D15): self-documenting make help (default target), every target wrapping a script under scripts/; docs/reference/make-targets.md is generated from it on every docs build.
  • Shared git repo: every commit prefixed skypurza: — enforced by the repo-root versioned hook (core.hooksPath → scripts/hooks/commit-msg, installer: sh scripts/install-hooks.sh from the repo root; D9, D20).
  • CLI surface follows SkyPilot verbs: spz launch|exec|status|down|check in M1 (launch -c/--cluster names the cluster; status --refresh reconciles); jobs/logs/gpus arrive with later milestones.
  • CLI runtime knobs via env: SKYPURZA_BACKEND (ec2|fake), SKYPURZA_STAGE, SKYPURZA_STATE_DB, SKYPURZA_KEY_DIR, SKYPURZA_WAITER_DELAY/ATTEMPTS, SKYPURZA_SSH_HOST/PORT/USER/KEY (test seam / overrides).
  • Cluster records persist in SQLite at ~/.skypurza/state.db (SKYPURZA_STATE_DB override); per-cluster SSH keys under ~/.skypurza/keys/.

10. Next milestones (from PLAN.md)

  • M1 (done, D12): real EC2 provisioning behind spz launch (SSH setup/run execution, spz exec), multi-region capacity failover, spot→on-demand fallback, SQLite state store, SkyPilot parity corpus.
  • M2: api-server persistence, multi-node execution, file_mounts/workdir, spz logs.
  • M3: platform services v1 (accounting/dispatch/observability on the seam).
  • M4+: Kubernetes/Slurm backends (D5).