Storage backends and scaling

The reactor persists every operation through a Kysely-backed storage layer. You pick the backend and the execution topology on the ReactorBuilder before you call build(). This page covers the default in-memory backend, switching to Postgres, and the two scaling paths: an executor worker pool and sharded projection workers.

Both scaling features are recent and design-doc-tracked. Where a path is not yet stable, this page says so.

Default: in-memory PGlite

Out of the box the builder gives you an in-memory PGlite database. When you build a reactor with only document models registered, the builder calls an internal createDefaultDatabase() that constructs new Kysely({ dialect: new PGliteDialect(new PGlite()) }).

import { ReactorBuilder } from "@powerhousedao/reactor";

const reactor = await new ReactorBuilder()
  .withDocumentModelSources([myDocumentModel])
  .build();

new PGlite() with no arguments is in-memory. Nothing is written to disk and nothing survives the process. This is the right backend for unit tests and local development, where you want a fresh, isolated database every run with no external service.

What you get in this path:

Migrations run automatically. The migration strategy defaults to "auto", so the builder runs runMigrations(db, REACTOR_SCHEMA) to create the reactor schema and apply the current migration set.
All storage operates under the "reactor" Postgres schema (REACTOR_SCHEMA === "reactor").
A single in-process executor runs jobs (maxConcurrency defaults to 1).
An in-process read-model coordinator keeps the document view and indexer up to date.
module.pools is [] — there is no pg.Pool to instrument.

State is ephemeral. Move to Postgres when you need data to survive a restart, when more than one process must share storage, or when you scale to a worker pool or projection shards (neither can run on PGlite — see below).

Choosing a backend

The builder picks the base database in this precedence order, inside buildModule():

The instance you passed to withKysely(...), if any.
A Postgres pool, if the worker pool is enabled and a worker DB config is set.
The in-memory PGlite default.

Bring your own Kysely instance

withKysely injects a pre-built Kysely instance and overrides default database creation.

withKysely(kysely: Kysely<Database>): this

Database is the combined schema the reactor operates over:

type Database = StorageDatabase & DocumentViewDatabase & DocumentIndexerDatabase;

StorageDatabase holds the operation log and sync tables (Operation, Keyframe, document_collections, operation_index_operations, sync_remotes, sync_cursors, sync_dead_letters); the other two cover the document view and the document indexer. Database, StorageDatabase, DocumentIndexerDatabase, and OperationTable are exported from the package root.

A persistent backend needs a Postgres-compatible dialect and the reactor schema. Pass a real Postgres Kysely here when you want to control pool construction yourself; otherwise use the worker pool's db (next section), which builds the pool for you. If you build the pool yourself, register its instrumentation with withInstrumentedPool(instrumentation) so pool stats surface through module.pools.

Postgres via the worker pool's db

When the worker pool is configured with a db (below), the builder builds the parent reactor's Postgres pool for you from that same DbConfig. The internal createPostgresDatabase constructs a pg.Pool and wraps it in a PostgresDialect:

new Pool({
  host: config.host,
  port: config.port,
  database: config.database,
  user: config.user,
  password: config.password,
  ssl: config.ssl ? { rejectUnauthorized: false } : undefined,
  application_name: config.applicationName,
  max: config.poolSize,
  connectionTimeoutMillis: config.connectionTimeoutMillis,
  idleTimeoutMillis: config.idleTimeoutMillis,
});

The pool is instrumented with instrumentPgPool(pool, config.applicationName ?? "reactor-host") and exposed via module.pools.

Migrations

The migration strategy is set with withMigrationStrategy:

withMigrationStrategy(strategy: MigrationStrategy): this
// MigrationStrategy = "auto" | "manual" | "none"

Default is "auto". Only "auto" runs migrations during buildModule(). A migration failure throws `Database migration failed: ${error.message}`. For "manual" or "none", run migrations yourself with the exported helpers:

import { runMigrations, getMigrationStatus, REACTOR_SCHEMA } from "@powerhousedao/reactor";

const result = await runMigrations(db, REACTOR_SCHEMA);
if (!result.success && result.error) {
  throw result.error;
}

runMigrations returns { success, migrationsExecuted, error? } and never throws on a migration error — it reports the error in the result. Use getMigrationStatus(db, REACTOR_SCHEMA) to inspect applied versus pending migrations without running them.

Scaling with a worker pool

The executor worker pool moves job execution out of the main thread into N node:worker_threads workers. Each worker opens its own Postgres pool and runs document models in isolation. Jobs route to a worker stickily by document id, so all work for one document lands on the same worker.

Enable it with withWorkerPool — calling it enables the pool; there is no enabled flag — and register document models as importable sources ({ filePath } or { packageName } — workers re-import them; a live module cannot cross the thread boundary):

import { ReactorBuilder } from "@powerhousedao/reactor";

const reactor = await new ReactorBuilder()
  .withDocumentModelSources([
    { packageName: "@my-org/account-document-model", subpath: "document-models" },
  ])
  .withWorkerPool({
    numWorkers: 4,
    db: {
      host: "localhost",
      port: 5432,
      database: "reactor",
      user: "reactor",
      password: process.env.PGPASSWORD!,
    },
  })
  .build();

Config shapes

WorkerPoolOptions:

type WorkerPoolOptions =
  | { numWorkers: number; db: DbConfig; factory?: WorkerFactory }
  | { numWorkers: number; factory: WorkerFactory; db?: DbConfig };

numWorkers is the number of workers the manager spawns at start() and the modulus used for sticky routing. Either db (the default thread transport; each worker opens its own Postgres pool) or a custom factory is required by construction — an enabled pool without connection info is unrepresentable rather than a runtime error.

DbConfig:

type DbConfig = {
  host: string;
  port: number;
  database: string;
  user: string;
  password: string;
  ssl?: boolean;
  applicationName?: string;
  poolSize?: number;
  connectionTimeoutMillis?: number; // pg defaults to 0 (unlimited wait) if omitted
  idleTimeoutMillis?: number;       // pg defaults to 10000 if omitted
};

DbConfig is sent across worker IPC, so it must be JSON-clonable. ssl: true maps to { rejectUnauthorized: false }; poolSize maps to pg's max; applicationName maps to application_name.

Workers verify signatures the same way the in-process executor does, with the same signatureVerification mode from withExecutorConfig. A refusal inside a worker is re-emitted on the host event bus. See Signature Verification.

How sources resolve

buildModule() resolves every registered source in one pass: file and package sources are imported host-side and their exports scanned for DocumentModelModule values; the resulting modules are registered on the host registry (identically in both executor modes), and the importable sources form the manifest each worker imports at boot. The host and workers are derived from the same list, so they cannot diverge.

Build-time constraints that throw

When a worker pool is configured, buildModule() enforces the model wiring up front. Each of these throws:

No importable sources: "withWorkerPool requires at least one worker-importable document-model source ({ filePath } or { packageName })."
A model registered only as a live module: "withWorkerPool requires worker-importable sources, but these models were registered only as live modules: ..." — provide a { filePath } or { packageName } source for each (a live module for the same documentType@version may coexist; the importable source covers it).

Connection info is not a runtime check: the WorkerPoolOptions type requires db or a custom factory, so a misconfigured pool fails at compile time.

The worker pool requires a real Postgres server. PGlite cannot be shared across threads, so the parent and the workers must point at the same Postgres instance. When the pool carries a db, the builder builds the parent's database from that same config (unless withKysely overrides it).

Custom transports

Passing factory in the pool options injects a custom WorkerFactory, skipping the default thread-transport wiring:

type WorkerFactory = (index: number) => IExecutorWorker;

Use this for tests or an alternative transport (e.g. a child-process adapter); db is then optional, since the builder is not constructing the default transport.

Projection shards

Projection shards move read-model indexing out of the main thread. Instead of the in-process read-model coordinator, the builder installs a shard manager that fans JOB_WRITE_READY events to N projection workers, sharded by document id. Each worker materializes the built-in read models against its own Postgres pool and re-emits JOB_READ_READY and read-model events back to the host bus, so synchronization, awaiters, and observers see them exactly once per job.

Configure shards with withProjectionShards. Projection workers open their own Postgres pools from the config's db; when the executor worker pool is also configured with a db, it may be omitted here and is reused.

import { ReactorBuilder } from "@powerhousedao/reactor";

const reactor = await new ReactorBuilder()
  .withDocumentModelSources([
    { packageName: "@my-org/account-document-model", subpath: "document-models" },
  ])
  .withProjectionShards({
    db: {
      host: "localhost",
      port: 5432,
      database: "reactor",
      user: "reactor",
      password: process.env.PGPASSWORD!,
    },
    shardCount: 4,
    preReadyKinds: ["document-view"],
    postReadyKinds: ["document-indexer"],
  })
  .build();

Config shape

ProjectionShardBuilderConfig:

type ProjectionShardBuilderConfig = {
  shardCount: number;
  preReadyKinds: BuiltInReadModelKind[];
  postReadyKinds: BuiltInReadModelKind[];
  poolSize?: number;
  initTimeoutMs?: number;
  shutdownGraceMs?: number;
  drainTimeoutMs?: number;
  chainDepthReportIntervalMs?: number;
};
// BuiltInReadModelKind = "document-view" | "document-indexer"

preReadyKinds run before a job reaches READ_READY; postReadyKinds run after. Between them they must name each BuiltInReadModelKind exactly once: under sharding the host's own copies of the built-in read models never index an operation, so a kind named in neither list is indexed by nobody — its reads stay stale and any read carrying a consistency token for it waits forever — while a kind named in both is indexed twice per operation. poolSize overrides the reused worker DB pool size for the shard pools only; their application_name is db.applicationName when set, otherwise "reactor-projection-shard".

Defaults applied when the optional fields are omitted: initTimeoutMs 30000, shutdownGraceMs 5000, drainTimeoutMs 30000, chainDepthReportIntervalMs 250.

initTimeoutMs bounds only a worker that never answers. A worker whose init throws reports the failure instead: build() rejects with that error — name, message, and stack intact — and the worker thread is terminated, rather than the build waiting out the timeout and reporting it as the cause.

Build-time constraints that throw

withProjectionShards without a db anywhere: "withProjectionShards requires a db (or an executor worker pool configured with one); projection workers need connection info to open their own pools."
shardCount < 1: `ProjectionShardManager: shardCount must be >= 1 (got ${shardCount})`.
preReadyKinds and postReadyKinds that do not name each built-in read model exactly once: "withProjectionShards requires preReadyKinds and postReadyKinds to name each built-in read model (document-view, document-indexer) exactly once between them; ...", with the missing and duplicated kinds listed.
withReadModel or withReadModelFactory alongside withProjectionShards: the workers build their own read models from the shard config, so host-registered instances and factories would be silently omitted. Rejected rather than dropped.
A projection worker db that is not the parent reactor's database, when the parent is not withKysely-provided: either nothing configures the parent (it would fall back to the default embedded database) or the two name different Postgres hosts, ports, or databases. The parent serves reads from the tables the worker writes, so a split would leave every built-in read model empty with nothing raised.
withProjectionShards alongside withReadModelCoordinatorFactory: two ways of owning the coordinator. Use one; the factory reaches the projection worker through its createProjectionShardManager dependency (see below).
A document model registered only as a live module when a projection worker is configured: the worker rebuilds its registry from the importable manifest, so a live-module-only model would be missing from it. Provide a { filePath } or { packageName } source for each.

Shards can run on their own or alongside the worker pool. They replace the read-model coordinator wholesale; the executor side is independent. Because the replacement is wholesale, the reactor's own subscription-notification read model and processor manager — built unconditionally by buildModule() — are never fed an operation under bare withProjectionShards: GraphQL subscriptions do not fire and package-installed processors see nothing. Hosts that rely on either, or that register their own read models, use the hybrid coordinator instead. As with the worker pool, projection shards require real Postgres — the workers open their own pools and PGlite cannot be shared across threads.

Hybrid projection worker

withReadModelCoordinatorFactory hands the coordinator's construction to a factory that runs after the reactor's internal read models exist. createHybridProjectionCoordinatorFactory is the shipped factory: one projection worker owns document-view and document-indexer, and the host keeps everything else, in the same order the in-process coordinator runs today — host pre-ready read models (withReadModel, withReadModelFactory), then JOB_READ_READY on the host bus, then subscription notifications and the processor manager post-ready. Reads carrying a consistency token stay correct because the host's trackers advance from the worker's indexing reports.

import {
  ReactorBuilder,
  createHybridProjectionCoordinatorFactory,
} from "@powerhousedao/reactor";

const reactor = await new ReactorBuilder()
  .withKysely(postgresKysely)
  .withDocumentModelSources([
    { packageName: "@my-org/account-document-model", subpath: "document-models" },
  ])
  .withReadModelFactory(async (deps) => buildMyReadModel(deps))
  .withReadModelCoordinatorFactory(
    createHybridProjectionCoordinatorFactory({
      db: {
        host: "localhost",
        port: 5432,
        database: "reactor",
        user: "reactor",
        password: process.env.PGPASSWORD!,
        applicationName: "my-host-projection",
      },
      poolSize: 8,
      onFatal: (shardId, reason) => {
        logger.error(`projection worker ${shardId} died`, reason);
        process.kill(process.pid, "SIGTERM");
      },
    }),
  )
  .build();

type HybridProjectionOptions = {
  shardCount?: number; // default 1
  poolSize?: number;
  db?: DbConfig; // falls back to the worker pool's db
  onFatal?: (shardId: string, reason: Error) => void;
  initTimeoutMs?: number;
  shutdownGraceMs?: number;
  drainTimeoutMs?: number;
  chainDepthReportIntervalMs?: number;
};

The parent database must be the same Postgres the worker reads, and this is enforced: the builder cannot derive the parent from a db that reaches it through the factory, because the parent database is already open by the time the factory runs. Pass the parent connection with withKysely, or configure withWorkerPool with the same db; a db with no parent configured at all throws rather than leaving the parent on the default embedded database while the worker projects into Postgres. onFatal fires when the worker errors or exits after it was ready, and for every write it then has to drop; nothing respawns the worker, so a host that cares should restart the process. The hook may fire repeatedly, so latch it. withReadModelCoordinatorFactory is mutually exclusive with withReadModelCoordinator and with withProjectionShards.

Each job produces two READMODEL_BATCH_COMPLETED events under this coordinator, because two chains ran: the worker relays its own, covering the built-in read models, and the host reports the chain it ran afterwards. Stage durations therefore stay attributable to the side that spent them — a merged event could not tell you whether the flag moved work off the host loop — while READMODEL_INDEXED continues to name one read model per event on both sides.

Switchboard exposes this as REACTOR_PROJECTION_WORKER=1 with REACTOR_DB_POOL_SIZE_PROJECTION (default 8) for the worker's pool. It requires a Postgres reactor database and is rejected in dev mode, like the executor worker pool.

withProjectionWorkerFactory injects a custom ProjectionWorkerFactory and skips the default thread-transport wiring, mirroring the pool options' factory:

type ProjectionWorkerFactory = (shardIndex: number, shardId: string) => IProjectionTransport;

:::warning Not yet stable Projection shards and the hybrid projection worker are recent, design-doc-tracked features. The shard-manager class itself is internal; you configure it through withProjectionShards or createHybridProjectionCoordinatorFactory and the public ProjectionShardBuilderConfig, HybridProjectionOptions, BuiltInReadModelKind, ProjectionWorkerFactory, and IProjectionTransport types. :::

Decision guide

In-memory PGlite (default). Dev and tests. Zero setup, ephemeral, single process. Use it unless you need one of the below.
Postgres (withKysely or the worker pool's db). When state must survive a restart or be shared across processes.
Worker pool (withWorkerPool). When job execution is CPU-bound and one thread is the bottleneck. Requires Postgres and importable document-model sources.
Hybrid projection worker (createHybridProjectionCoordinatorFactory). When the host event loop is saturated by indexing the built-in read models and you still need subscriptions, processors, or host read models. Requires Postgres. One worker; the host keeps everything else.
Projection shards (withProjectionShards). When only the built-in read models matter and you want N workers sharded by document id. Requires Postgres. Disables subscriptions and processors. Combine with the worker pool when both write and read paths need to scale.

All three scaling paths require real Postgres and trade simplicity for throughput. Start in-memory, move to Postgres for persistence, and reach for the worker pool, the hybrid projection worker, or shards only when a single thread is the proven bottleneck.

For the full list of builder methods, see the builder table on Advanced Reactor Usage. For the read-model and processor side of indexing, see Processors and Document model registry. For the client surface that talks to a built reactor, see IReactorClient.

Default: in-memory PGlite​

Choosing a backend​

Bring your own Kysely instance​

Postgres via the worker pool's db​

Migrations​

Scaling with a worker pool​

Config shapes​

How sources resolve​

Build-time constraints that throw​

Custom transports​

Projection shards​

Config shape​

Build-time constraints that throw​

Hybrid projection worker​

Decision guide​

Default: in-memory PGlite

Choosing a backend

Bring your own Kysely instance

Postgres via the worker pool's db

Migrations

Scaling with a worker pool

Config shapes

How sources resolve

Build-time constraints that throw

Custom transports

Projection shards

Config shape

Build-time constraints that throw

Hybrid projection worker

Decision guide