Proposal workflow
Formerly design.md §8’s execution model, human-in-the-loop rule and decision traces. This doc
covers how an Intent is executed durably, what the requester sees, how dates in a request are
interpreted, and the traces that explain a decision. What the Suggester does inside one run is in
suggester.md.
Approval
Proposals are never auto-executed (decision 0005). Every
approval for one requester, manual or automatic, enters one store-owned, requester-scoped ordering:
an in-process keyed semaphore on SQLite, and a transaction-scoped advisory lock on PostgreSQL whose
quota reads and approval commit share that transaction and connection, so a lost session rolls back
instead of releasing the ordering early. Ordering ends at the commit or rollback, before best-effort
channel reconciliation. Any ordering or quota-read failure leaves the Proposal submitted.
Proposal Jobs and the Journey
A Proposal Job is the caller-owned durable execution of an Intent. internal/proposals/workflow
owns its lifecycle, attempt recovery, caller visibility and the First-channel Journey projection.
Its interface is small: submit an Intent (fresh, or for an existing Channel), claim bounded work, complete or fail one claimed
Attempt, and read or list Journeys. The workflow’s one Repository (backed by store.ProposalRepository,
the single persistence surface for Proposals and their Jobs) and the model runner are private ports; API, worker
and frontend never rebuild lifecycle rules from raw rows.
The Intent, the Proposal and the evidence they carry (Scores, Decision Trace) are the value types of the
leaf package internal/proposals. The store is the one place a Proposal’s JSON is encoded and
decoded: store.Proposal.Body is the typed value, an empty column reads as the zero Proposal, and
a column the store cannot decode is an error rather than an empty Proposal. Everything above the
store passes the typed value.
The Job id is the correlation spine, not a state machine that takes over domain ownership. The
Proposal stays the grounded artifact and approval audit; the Channel owns building/live. The
Journey composes them into one milestone:
queued/running -> generatingdone + Proposal submitted -> awaiting_approvaldone + Proposal denied -> denieddone + Proposal approved + Channel building -> buildingdone + Proposal approved + Channel live -> livefailed -> failedA later pause does not erase that the Journey reached live; current Channel status is returned
separately. An impossible combination fails the read closed and is recorded for operators.
Attempts and recovery
- Each lease creates a numbered Attempt with its own times and outcome (
succeeded,failed,interrupted). Claiming is one transaction that takes duequeuedJobs and expiredrunningones, marks the expired Attemptinterrupted, bumps the attempt token and renews the lease (FOR UPDATE SKIP LOCKEDon PostgreSQL). A crash causes bounded re-execution, never a stranded row. - Completion and failure compare-and-swap on
(job_id, attempt), so a stale worker cannot publish. Success inserts the Proposal and marks Attempt and Job terminal in one transaction; a failed execution materialises neither a Proposal nor a Channel. The suggest worker has no other job lifecycle:suggest.NewServicerequires the workflow, a stale Attempt’s result is dropped without a failure write, and approvers are told a request is waiting only when a person still has to act: a Proposal an automatic grant approved sends no waiting notice. A claimed Job whose stored Intent cannot decode is failed on its own and never aborts the rest of the claim batch. - Retrying a failed first request creates a fresh Job. Refine and re-curate also create a fresh Job,
seeded from the intent the Channel owns (
channels.intent_json); the Job names the Channel (proposal_jobs.channel_id), so approving its Proposal patches that Channel in place. It never reuses or edits the Job the Channel was made from, and it never reads the cache. A refine is the refining admin’s own request; a re-curation is ownerless system work (kind=recurate, empty owner) and is in nobody’s request list. A refine never takes the requester auto-approve grant (an admin reviews it in the refine panel; its acquisitions are charged to the refining admin on approval), and a first approval refuses to create a Channel when its Job row is gone. A Channel has one unfinished channel Job at a time (queued, running, or done with a Proposal still awaiting a decision): refine answers 409 and the re-curation run skips it with an Info line. The Channel’sintent_refstays the Job that created it. A Channel also keeps itsrequested_by, set once at approval and never changed by a refine or re-curation, so housekeeping can purge Jobs without taking the intent or the requester with them. A Channel with no stored intent (its Job was purged before this existed) is neither refined nor re-curated from a guess: refine asks the admin to describe it again and re-curation skips it with a logged reason. A cache hit (hash of normalised Intent and constraints, 24 h TTL) copies grounded content into a fresh caller-owned Job and Proposal; it never shares another caller’s ids, approval or history. Only a successful job is cached, and zero grounded titles fails the job rather than persisting an empty Proposal. - Persisted Jobs and Attempts carry a workflow-schema version. Readers accept every version an upgrade still supports and fail closed on unknown, future or corrupt state; a release may add a transition path but never reinterpret stored state. Upgrade fixtures prove it.
Proposal Jobs keep their own leased executor because their history is business state. River runs
named recurring operator tasks (deployment.md).
| Activity | Contract |
|---|---|
| Catalog tool | Read-only, bounded, safe to retry within the tool-round ceiling. |
| Model turn | Bounded by the Job timeout; may re-run after worker loss; only the current Attempt token commits. |
| Proposal completion | One local transaction; no Proposal is visible for a Job that did not become done. |
| Approval | The atomic submitted -> approved decision; authorization and quota ordering stay outside model control. |
| Acquisition and publication | Idempotent reconciliation after approval; repair by converging again. |
What the requester sees
GET /v1/proposal-jobs/{jobId} is authoritative after reload, restart, replica change or event loss.
It returns the Intent, bounded Attempt history, safe failure code and message, the newest Proposal,
the intent-bound Channel, the milestone and the server-derived permitted actions (wait, review,
retry, edit, check_ai, open_channel). The frontend never reconstructs authorization or retry
policy. /v1/events streams suggestion frames ({jobId, phase, round}) that only trigger a
refetch, and the Web client reads the Journey on them rather than polling it. Phases are emitted
inside the tool loop at the transition they describe and may repeat; done follows the stored
proposal and failed the stored failure. A stream that misses a frame is reconnected, and the
reconnect reads the Journey again. An empty proposal frame follows each stored submission,
approval and denial, whether a person or an automatic grant decided, so the requester sees a denial
and every admin’s approvals stay current without a reload.
The builder shows one calm status message: no rounds, timers, percentages or checklists. A failure replaces progress with one plain explanation and the useful authorised action, and keeps the saved Intent. Revisions keep the current list visible and read-only until the replacement arrives. The active Job and its approval edits are restored after reload until an explicit discard.
Failures. A failed Journey keeps its stable outer code and adds a closed reason and
recoveryAction with fixed server-owned copy:
| Reason | Recovery | Selected by |
|---|---|---|
reference_unreadable | edit_reference | a typed failure from the reference-read stage only |
retrieval_unavailable | retry_later | any other reference or catalog retrieval failure |
no_catalog_match | broaden_request | no grounded titles |
named_set_unproven | provide_examples | selection empty only because members lacked membership evidence |
constraints_conflict | resolve_constraints | identical include and exclude values, or an empty same-axis date intersection |
date_semantics_unclear | clarify_dates | validated, anchored date ambiguity |
invalid_tool_calls | retry_later | every consumed tool round was invalid |
provider_response_invalid | retry_later | malformed final JSON, or exhaustion after candidates were surfaced |
discovery_budget_exhausted | retry_later | exhaustion with nothing surfaced; copy stays neutral |
provider_timeout, provider_unavailable | retry_later | an actual provider-turn deadline or outage |
generation_failed | retry_later | anything else |
The mapping and its copy live in internal/proposals/workflow/workflow.go. Only allowlisted typed
evidence captured at the rejecting branch selects a specific reason. check_ai is offered only for an AI-stage failure, and a member is
never told to change an admin-only setting. No projection exposes raw provider errors, prompts,
responses, fetched text, credentials, private titles or candidate identities.
/metrics reports Jobs by status, the oldest queued or running Job’s age, Attempts by outcome, and
failed Jobs by safe code. Unknown values collapse to other; ids and text are never labels.
Dates in a request
Every catalog_search call and final response carries dateMeaning (#1034): kind is none,
constraints or ambiguous; anchors point at submitted text (field, optional array index,
half-open rune offsets); axes are one to three of movie_release, series_premiere and
series_airing, each with combine: any|all and one to four 1900–2099 intervals.
- Canonicalisation intersects
all, coalescesany, and only an empty same-axis intersection is a conflict. ”90s and 2000s” is a union. Different axes stay independent. kind: noneis malformed when the request has unmistakable date syntax (anErafield, a decade, a year range, orfrom/before/since… a year); it goes through the bounded repair path. A year inside a title is not a filter.- Tool meanings and the final meaning must canonicalise identically. Sources are initialised once per invocation, only after a valid non-ambiguous interpretation, so ambiguity, conflict and exhausted malformed output dispatch nothing.
scope.datesstoresmovieRelease,seriesPremiereandseriesAiringranges: union within a list, conjunction across axes. Airing is checked per episode, so an older series can supply in-window episodes. Nothing widens explicit windows.scope.datesis the only stored date scope: anerais read as an alias for the same range on all three axes and never written (#1877, programming design).- Filler follows through
filler.eraWindows(at most eight ranges, exclusive withfiller.era), inherited from the airing or release windows when neither is set. Unknown filler years do not satisfy a selected era. - A union is one retrieval: each window is one scalar query, results are deduplicated and truncated once to 24 candidates, and a failed window fails the whole union.
- Network epochs (“like a documentary network in the 1990s”) are editorial references, not episode filters. The network is resolved through Catalog discovery, the older title pool comes from a premiere upper bound at the end of the decade, and the network-only tool interface is exposed. Title or generic genre lookups without a network identity are corrected before dispatch.
Credits. A generation has six work credits; one Suggest invocation has
ProductionBounds().MaxToolCalls in total (currently 24), shared across repairs. A discovery that
expands into N queries costs N credits, reserved before dispatch.
Model requests
- Prefix caching. On a provider with a prompt-prefix cache (
llm.PrefixCacher, such as llama.cpp), every turn of one suggestion sends the same tools array and finalisation usestool_choice: "none", because removing tools changes the prompt from its first tokens and forces a full re-prefill. Static content leads. Hosted providers still drop tools at finalisation. - Reference requests start with no tools: the model interprets the Intent and returns empty picks, then Loomarr resolves the reference and supplies its evidence within the same six-turn generation. Source lookups are cached across repairs.
- A request centred on a named franchise stays within its members unless the Intent invites adjacent discoveries; fewer supported members beat padding.
- Grounded selection uses the LLM adapter’s request profile (
grounded-selection-v3), which adapts optional parameters per exact model and enforces strict routing and data-collection controls on hosted aggregators. It never selects a model or qualifies one. - Structured filler calls use JSON mode, low temperature and an explicit
max_tokens; self-hosted endpoints also getenable_thinking=false. - Prompt, tool-schema and certification manifests are versioned and immutable; a model or route change needs fresh qualification.
Decision traces
Proposal trace (#496). The ranking emits one immutable, Proposal-scoped DecisionTrace:
original evidence that edits, availability changes, cache clones and re-curation never recompute.
Each surfaced candidate has one closed disposition and reason (selected, alternate, not-selected,
policy-refused, validation-dropped, acquisition-cap, plus terminal outcomes). The published rank is
the integer tuple (relevance, preference, novelty, canonical key). Constraint evidence is the
relevance count and closed booleans for request, tone, era, include, exclude and refine.
Scheduler trace (#496). Every schedule.ComputeDesiredAt run emits its own trace of current
cycle evidence, exposed by the cycle and programming-preview endpoints from the same computation.
Stages are hard_filter, availability, episode_selection and placement; metadata records the
ordering, decimal-string shuffle seed and window. 256 of the 1,024 facts are reserved for placement.
Both traces are versioned, stably ordered, bounded to 1,024 facts with factTotal, recordedTotal
and truncated counting past the bound, and carry only canonical identity, safe display names and
closed codes. Prompts, rationale, credentials, provider payloads, paths and locations are redacted.
Evaluators consume the same typed evidence and fail closed on mismatch.
Tests that pin this
Formerly design.md §19.
- Interface-level tests cover every legal Journey milestone and permitted action without reading raw tables. SQLite and Postgres conformance proves atomic claim, expired-running recovery, monotonic Attempt tokens, stale-worker rejection, success and failure rollback, cache cloning and bounded history. Crash tests stop after claim, model return, Proposal insert and approval commit; restart converges to exactly one visible outcome. Old-version fixtures stay readable; unknown versions and approved-without-Channel fail closed. Dropping every SSE frame does not change the result.