> ## Documentation Index
> Fetch the complete documentation index at: https://claritylabs.inc/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# CL Pipelines Core Concepts: Jobs, Phases, Checkpoints

> Understand Jobs, Phases, Checkpoints, PipelineStatus, StorageAdapter, SchedulerAdapter, RetryMode, and PhaseResult before writing your first pipeline.

## Job

A **job** is a single pipeline run identified by a `jobId` string you provide. Every piece of state — status, checkpoint, and log entries — is scoped to that ID. Two pipelines with different `jobId` values are completely independent, even if they share the same phase definitions and adapters.

```typescript theme={"system"}
await runPipeline({
  jobId: "invoice-7829",  // your ID — use a database row ID, UUID, etc.
  phases: [...],
  storage,
  scheduler,
  initialState: { ...},
});
```

## Phase

A **phase** is the smallest unit of work. It is a plain TypeScript object with two fields:

* `name: string` — unique within the phases array; used to look up the phase by name from the checkpoint
* `run: (ctx: PhaseContext<TState>) => Promise<PhaseResult<TState>>` — your business logic

Phases are stateless objects — all mutable state flows through the `Checkpoint<TState>`. This makes them easy to test in isolation: construct a fake `PhaseContext` and call `run` directly.

## Checkpoint

A **checkpoint** is a serialized snapshot of in-progress state. The storage layer persists it after every successful phase transition and whenever your phase calls `ctx.saveState()`.

```typescript theme={"system"}
type Checkpoint<TState = unknown> = {
  nextPhase: string;  // name of the next phase to execute
  state: TState;      // arbitrary state passed between phases
  createdAt: number;  // Unix timestamp in milliseconds
};
```

The checkpoint is what makes crash recovery possible. If your process dies mid-job, the next `runPipeline` call loads the saved state and re-enters the checkpoint's phase. That phase may repeat work, including work before a mid-phase save, unless it uses the saved state to skip completed steps.

## PipelineStatus

Every job has one of five statuses:

| Status | Meaning |
| - | - |
| `"idle"` | Job record exists but has not started yet |
| `"running"` | A phase is scheduled or currently executing |
| `"paused"` | Execution intentionally halted (e.g. awaiting human approval) |
| `"complete"` | Final phase returned `{ kind: "done" }` |
| `"error"` | A phase threw or returned `{ kind: "error" }` |

## StorageAdapter

The **StorageAdapter** is the interface between `cl-pipelines` and your database or key-value store. You implement five methods:

```typescript theme={"system"}
type StorageAdapter<TState> = {
  getJob(jobId: string): Promise<{
    status: PipelineStatus;
    checkpoint: Checkpoint<TState> | null;
    error?: string;
  } | null>;
  setStatus(jobId: string, status: PipelineStatus, error?: string): Promise<void>;
  setCheckpoint(jobId: string, checkpoint: Checkpoint<TState> | null): Promise<void>;
  appendLog(jobId: string, entry: LogEntry): Promise<void>;
  clearLog(jobId: string): Promise<void>;
};
```

The library never accesses your database directly — it only calls these methods. This means you can back it with Convex, PostgreSQL, Redis, or the built-in in-memory adapter without changing any pipeline logic.

## SchedulerAdapter

The **SchedulerAdapter** is the interface between `cl-pipelines` and your job queue or task scheduler. It has a single method:

```typescript theme={"system"}
type SchedulerAdapter = {
  scheduleAdvance(jobId: string, delayMs: number): Promise<void>;
};
```

After each successful phase, the library calls `scheduleAdvance(jobId, 0)` to trigger the next advance. You implement this by enqueuing a call to `advancePhase` in your scheduler (e.g. Convex's `ctx.scheduler.runAfter`).

## RetryMode

**RetryMode** controls what happens when you call `runPipeline` on a job that already has a checkpoint:

| Mode | Behavior |
| - | - |
| `"resume"` | Starts from the last saved checkpoint — only failed work re-runs |
| `"full"` | Discards the checkpoint and restarts from `initialPhase` with `initialState` |

When no `retryMode` is specified and a checkpoint exists, the library behaves as `"resume"`. See the [Retry Modes](https://claritylabs.inc/docs/cl-pipelines/phase-runner/retry-modes.md) page for guidance on when to choose each.

## PhaseResult

Every `run` function must return one of three result shapes:

```typescript theme={"system"}
type PhaseResult<TState> =
  | { kind: "next"; nextPhase: string; state: TState }  // advance to named phase
  | { kind: "done" }                                    // pipeline complete
  | { kind: "error"; error: string };                   // set status: "error"
```

> [!NOTE]
>
> Both `{ kind: "error" }` returns **and** thrown exceptions preserve the checkpoint. This means `retryMode: "resume"` works correctly for all error scenarios — the job can always resume from the last safe state.

## How the pieces connect

```
runPipeline()
  └─ writes initial Checkpoint<TState> to StorageAdapter
  └─ calls SchedulerAdapter.scheduleAdvance(jobId, 0)

SchedulerAdapter fires
  └─ calls advancePhase()
       └─ reads Checkpoint via StorageAdapter.getJob()
       └─ finds Phase by checkpoint.nextPhase
       └─ calls phase.run(ctx)
            └─ ctx.log()       → StorageAdapter.appendLog()
            └─ ctx.saveState() → StorageAdapter.setCheckpoint()
       └─ on "next"  → setCheckpoint() + scheduleAdvance()
       └─ on "done"  → setCheckpoint(null) + setStatus("complete")
       └─ on "error" → setStatus("error") [checkpoint unchanged]
```


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.