ADR-0138: Studio backend attachment, daemon ownership, and extraction from the wheels¶

• Status: Proposed
• Kind: Aspirational
• Implementation-status: not-started
• Area: studio
• Date: 2026-10-25
• Relations: decides the Studio backend form that ADR-0136 D7 deferred and that ADR-0136 Amendment 1 A4 item 2 requires before dependent changes; adopted together with ADR-0136 Amendment 2, which moves the application's extraction to 2.0

This record decides how the Studio application's backend relates to the daemon: which component owns each process resource, route, and credential, which code is shared, and when the application leaves the library wheels.

Context¶

Source references (path:line) in this record refer to development-line commit b4a3bebf. ADR-0136 denotes docs/adr/ADR-0136-moving-non-core-packages-into-lionhub.md throughout. Proposed paths, interfaces, configuration names, and verification tests describe the decision, not existing implementation.

Adoption of this record requires the accompanying Amendment 2 to ADR-0136. During 1.x, the application remains at its current lionagi.studio paths inside the single distribution. Wave 2 establishes the daemon, route, identity, and shared-service boundaries. At 2.0, the application moves to apps/studio/backend/src/, outside both library wheels. The core never gains a dependency on application code. The ADR-0136 D2 exempt set is {cli, studio} until Amendment 2 step 6 moves the CLI, {studio} from that step through the last 1.x release, and empty from 2.0, when the application leaves the wheel (ADR-0136 D1, ADR-0136 D2; tests/test_moved_modules.py:669).

Amendment 1 already requires the backend record to allocate every scheduler and CLI dependency on Studio, puts shared command logic and parsing in lionhub.orchestration, and requires fresh-install local schedule validation and export without the backend. Those are premises of this decision (ADR-0136 A4 items 2 to 5). It also leaves automatic plugin discovery deferred; this record selects one explicitly activated application, without changing that decision (ADR-0136 Amendment 1, What this amendment does not decide).

The following current behaviors constrain the boundary:

• A directory boundary is insufficient. The scheduler imports services.scheduler_state, project resolution, lifecycle reapers, and database maintenance. Reconciliation imports helpers from both admin and shows. These dependencies must be allocated with the scheduler (lionagi/studio/scheduler/engine.py:43-50, :384, :1135, :1542-1556; lionagi/studio/services/lifecycle.py:15-23).
• The scheduler-state dependency is not a Python module import cycle. Its chain is engine to scheduler state to coordination; coordination imports only standard-library modules. It needs relocation, not a new inversion framework (lionagi/studio/scheduler/engine.py:43-50; lionagi/studio/services/scheduler_state.py:18-26; lionagi/studio/scheduler/coordination.py:16-19).
• Pairing is already process-local. Codes, capabilities, and the epoch live in module state; the launcher explicitly uses one worker. A second process would require a new verification protocol, not merely another server command (lionagi/studio/launch_access.py:75-112, :141-166, :324-353; lionagi/studio/cli.py:497-503).
• A decision capability currently supplies attribution, not API admission. Invalid, absent, expired, or wrongly bound capabilities produce no principal; the decision still reaches the coordinator. Turning this into a mandatory credential would change behavior (lionagi/studio/services/operator.py:311-327, :335-359, :370-394).
• Neither schedule CLI HTTP client currently supplies Authorization. Both construct only a JSON content-type header. A token-protected daemon consequently refuses these clients; splitting packages does not fix that defect (lionagi/studio/cli.py:1224-1245; lionagi/cli/machine_schedule.py:43-71; lionagi/studio/app.py:613-615).
• _auto.py is not a usable HTTP host. Its list omits five modules, and its compiler consumes a marker that the active Studio decorator does not attach. Correcting the list alone would not convert the active routes (lionagi/_auto.py:111-141, :314-328, :355-371; lionagi/studio/registry.py:95-122).

Decision¶

The Basis column says where each decision comes from: a direction already set for the project, this record's own judgment, or a question this record leaves open.

1. F1: Attach one explicitly enabled backend in the daemon process (proposed)¶

F1 is proposed and awaits a project decision between this in-process attachment and a separate local application process beside the daemon; F4, and the parts of F2 that place application startup inside the daemon's lifespan, follow whichever form is chosen. The rest of this section describes the proposed form.

Choose an explicitly configured, in-process backend with one fixed attachment interface. The daemon owns the listener, root HTTP application, admission middleware, scheduler, and process lifecycle. The Studio backend owns its application routers, Operator, static frontend mounting, and application startup and shutdown hooks. Its implementation remains under lionagi/studio through 1.x and moves to apps/studio/backend/src/ at 2.0, with Operator at apps/studio/backend/src/operator/. It never moves into lionhub.

In wave 2, lionagi.studio.app provides a fixed attach_studio(application, services) function and retains create_app, app, and the existing launch surfaces. The daemon factory accepts that attachment callable or no callable. Attachment adds the application routes and returns explicit startup and shutdown hooks. The services argument contains the existing daemon-owned launch and access services; it creates no second scheduler, launch supervisor, or capability store. The standalone daemon imports no application module. The Studio launch path imports the retained application only when selected and passes its attachment to the daemon factory. Application handlers call public wheel services rather than CLI command wrappers. This explicit composition preserves the existing app export and launch contracts (lionagi/studio/app.py:550-557, lionagi/studio/app.py:689; lionagi/studio/__main__.py:1-7).

At 2.0 the same attachment moves with the application to apps/studio/backend/src/. The Studio launcher imports that fixed attachment from the installed application and passes it to the same daemon factory. No arbitrary module name is accepted. How the extracted application is installed, and how the launcher locates it, is decided with its distribution, outside this record; that locating step is the only change to activation required by extraction, and the callable and lifecycle remain the wave-2 contract.

The interface is deliberately fixed. There is no entry-point enumeration, plugin registry, provider ranking, discovery scan, hot replacement, or backend management API. Compose the routers into the root application so that existing paths, route ordering, OpenAPI, exception translation, and CORS method discovery can remain intact. A nested ASGI mount that silently changes these surfaces is not an equivalent implementation. The current application registers routes and derives CORS methods from that resulting application (lionagi/studio/app.py:151-160, :451-478, :627-662).

The daemon factory with no attachment starts the daemon API alone and imports no application module. Its application endpoints return their normal missing-route response. In 1.x, li studio explicitly selects the bundled application without requiring a new configuration value or installation step. At 2.0 it requires the extracted application to be installed. An explicitly selected but missing, incompatible, or broken backend fails startup with an actionable installation error. --no-frontend continues to mean that the frontend is supplied elsewhere, not that the application backend is absent. Daemon-only startup uses the daemon factory without an attachment in both packaging eras. The present desktop launcher depends on this distinction: it launches li studio --no-frontend and supplies its own frontend (apps/studio/desktop/src-tauri/src/backend.rs:155-173).

Request failure and cancellation are isolated by ownership of tasks, not by operating-system isolation. A failed request or closed event stream must not cancel the scheduler or unrelated accepted work. The backend owns Operator tasks; the daemon owns scheduler and detached launch tasks. No task group may make every request and both subsystems siblings whose failure cancels the whole group. Process termination, memory exhaustion, blocking code, and native crashes remain shared failures. No claim of sandboxing or independent backend restart is made. Enabling, disabling, upgrading, or recovering a failed configured backend requires restarting the process and therefore its epoch.

This preserves the present single-worker pairing arrangement and one supervised desktop child. The current container image also exposes one listener and needs no inter-service proxy or second supervisor (lionagi/studio/cli.py:497-503; apps/studio/desktop/src-tauri/src/backend.rs:164-188; apps/studio/Dockerfile:75-80).

A general daemon plugin host is rejected because one known attachment needs no discovery or selection machinery, and Amendment 1 leaves automatic bootstrap deferred (ADR-0136 Amendment 1, What this amendment does not decide). A separate local application process beside the daemon is the open alternative. It provides stronger failure isolation and lets the application restart without restarting the daemon. It requires pairing state to move out of process memory into a store both processes verify against, authenticated cross-process capability verification, two lifecycles, coordination of state writers, and a second supervised desktop child. The present process-local capability checks and single-child launcher are why this record proposes the in-process form; they are not an assertion that process isolation has no value. If the separate process is chosen, F1 and F4 are replaced by an amendment that specifies that verification, and the daemon-side decisions in F2, F3, F5, and F6 stand.

2. F2: The daemon owns process resources and the backend owns Operator recovery¶

Allocate every operation in the existing lifespan as follows. Shared modules supply functions and values; they never independently start a lifecycle.

Construct and validate the selected application before starting process resources. The successful startup order remains: snapshot, traceback and warnings, notification registration, logging context, scheduler start, daemon reconciliation, optional backend recovery, begin_serving, reconnect channel, mirror, deferred warmup, and readiness to serve. Without the backend, omit only backend recovery. Do not rely on a framework to automatically execute the lifespan of a nested mounted application; the root lifecycle explicitly enters the returned backend hook at the designated position.

Readiness to accept requests follows completion of startup, not the deferred checkpoint. /health remains the lightweight liveness and reconnect-proof surface. /api/admin/readiness retains its store and scheduler assessment and reports the result in the response body. Do not silently change it to a 503 endpoint or add an Operator-store requirement. The current Operator startup intentionally skips recovery when the configured store has no local file; its routes then report the existing unsupported-store behavior (lionagi/studio/app.py:321-329, :651-668; lionagi/studio/services/admin.py:1455-1481; lionagi/studio/services/operator.py:65-86). With a selected backend, completion of its supported startup path is a prerequisite to opening the listener, not a second readiness server.

On a normal stop, withdraw the host from serving new requests and preserve the current cleanup order: close the session live-tail registry, disarm traceback capture, clear the reconnect key and stop its channel, finish warmup, stop the mirror, shut down the backend coordinator when present, shut down detached launches, stop the scheduler, and exit the log context. The current code does not implement a literal reversal of startup (lionagi/studio/app.py:408-424).

The new attachment seam must unwind each resource that successfully started if a later startup stage fails or is cancelled. Shutdown must attempt remaining cleanup even when an earlier hook raises; the current lifespan does: its cleanup runs in a finally around the yield, shielded from cancellation, and records each step's failure and raises them together after the last step (lionagi/studio/app.py:393-426). Startup is not covered: a stage that fails before the yield leaves the resources started before it running. Implement and verify that startup correction as a separate preparatory change. A selected backend startup failure ends this launch after cleanup. Backend absence is a configured mode, not an error recovery policy.

Do not describe scheduler startup as waiting for reconciliation. start() creates a running task before the next lifespan statement, and the loop performs recovery and then ticks. The reconciliation docstring's stronger sequencing claim is not enforced by this call order. Introducing a scheduler admission barrier or moving reconciliation earlier is a separate behavior change and is not bundled with relocation (lionagi/studio/scheduler/engine.py:537-557, :1092-1097; lionagi/studio/services/lifecycle.py:810-815).

Starting a scheduler, reapers, or mirror on each side is rejected because it duplicates ownership. Moving the whole lifespan into the application is rejected because it makes scheduling and run repair depend on installing Studio. The daemon-only requirement and the existing scheduler maintenance calls decide this allocation (lionagi/studio/scheduler/engine.py:1538-1561).

3. F3: Separate execution API routes from application routes¶

In wave 2, daemon API routes move under lionhub.serve and scheduler domain services move under lionhub.orchestration. Application routes stay at their existing lionagi.studio paths through 1.x and move to apps/studio/backend/src/ at 2.0. The HTTP ownership split is fully enforced in wave 2, even while all application files remain installed. Execution control and inspection belong to the daemon; Studio application management belongs to the backend.

The following table allocates all 29 registered modules. Each source is under lionagi/studio/services/. The cited line identifies a representative route; ownership covers every decorated route in that module. Helpers used across the boundary are allocated separately in F5.

There are ten daemon modules and nineteen application modules. The original registry is the coverage source (lionagi/studio/registry.py:44-74). The host also owns /health and the security wrapping of schema and documentation routes. The backend owns frontend assets and SPA fallback. An unknown /api path must remain a JSON 404 even when the SPA is installed (lionagi/studio/app.py:41-54, :514-547, :651-668).

Backend absence does not imply that every existing CLI command becomes an HTTP client. Keep local schedule validation and export local. The machine validator resolves a document without a daemon, and export reads stored rows and converts them locally; the human export does the same (lionagi/cli/machine_schedule.py:312-357, :360-415; lionagi/studio/cli.py:1951-2023). No application module may be imported transitively by those paths. This implements A4 item 5 in wave 2 even while the compatibility application is still installed (ADR-0136 A4 item 5).

The required new proof is named tests/packaging/test_studio_backend_boundary.py::test_fresh_wheel_schedule_validate_and_export_without_backend. During 1.x it installs the normal built lionagi[studio] wheel in a fresh environment and runs outside the checkout with the application disabled. Because D1 still requires the application files to be installed, the test also installs a test-only import blocker for the precise retained application modules and asserts that none load. It does not label a normal 1.x wheel physically backend-free. The deny list excludes empty parent packages needed by old compatibility names. At 2.0 the same test installs the released core and hub wheels with required scheduling dependencies and no application installed, so physical backend absence is tested. Both arms use temporary configuration and state. It exercises installed li schedule validate and li schedule export, plus the installed machine schedule entry points. It covers valid and invalid cron, interval, command, flow, and playbook declarations, populated managed and legacy export fixtures, and partial-export results. It asserts that validation creates no database and that export preserves the input rows. It checks that no backend, FastAPI route module, or running daemon is needed by the local conversion paths. An empty export and a mocked converter are insufficient evidence.

The 1.x test uses lionagi[studio] because cron validation imports croniter, which is currently in that extra (lionagi/studio/services/schedule_declaration.py:361-366; pyproject.toml:141-146). The 2.0 test uses the installation declared by the core/hub distribution record with the same required scheduling dependencies. The HTTP dependencies may be installed but the local validators must not import the HTTP adapters. Do not add a new packaging extra merely to rename this existing dependency set.

Also verify the complete enabled route contract, including methods, paths, static-before-parameter ordering, trailing-slash behavior, status codes, schemas, and middleware. Check the daemon-only route set against this table. This installation test is a proposed acceptance requirement, not an existing passing test.

Putting every route in the backend is rejected because schedule control, identity, pairing, execution inspection, and daemon maintenance would then disappear when the application is absent. Putting all routes in the daemon is rejected because Operator and application editors would remain wheel dependencies contrary to the ADR-0136 D7 boundary. The schedule client's concrete HTTP dependency and the desktop's identity handshake establish the minimum daemon surfaces; retaining execution inspection is the explicit product decision (lionagi/cli/machine_schedule.py:56; apps/studio/desktop/src-tauri/src/backend.rs:203-211).

4. F4: Keep one daemon-owned identity and capability store¶

The daemon owns the configured bearer policy, launch bearer, pairing codes, epoch, decision-capability hashes, revocation, reconnect channel, and port proof. These move together into lionhub.serve. Application code calls the same daemon-owned capability resolver in the same process. There is no second secret store, signed-token format, introspection endpoint, or capability replication.

Preserve the existing launch and reconnect flow. A launcher obtains a code in the daemon process or through the same-user reconnect channel, verifies the listener's proof when reconnecting, and opens the browser with the code in the fragment. The browser redeems it at /api/studio/pair for the admission token and decision token. Preserve single use, origin binding, the 120-second pairing lifetime, the eight-hour decision lifetime, revocation, and invalidation on a new daemon epoch (lionagi/studio/cli.py:319-401; lionagi/studio/launch_access.py:59-68, :141-166, :248-276, :301-361; lionagi/studio/services/pairing.py:63-100). The retired code-issuance endpoint continues to return 410; do not restore issuance over HTTP (lionagi/studio/services/pairing.py:35-59).

Keep the host's admission middleware outside both route sets. Preserve Host checking, configured-token admission, no-token Origin and Fetch Metadata checks, JSON requirements, and the existing pairing exemption. Static page loading remains distinct from API admission (lionagi/studio/app.py:452-470, :570-625, :653-662). Browser decisions pass the original request origin and decision header into the one resolver. A resolved principal is not a verified human identity, and a missing or invalid decision token remains unattributed under the existing API policy (lionagi/studio/launch_access.py:93-101; lionagi/studio/services/operator.py:311-327).

The desktop keeps one child, its generated configured token, its cross-origin frontend setting, and the authenticated /api/identity handshake. That endpoint must continue to reject an open daemon even though other no-token requests may be admitted (apps/studio/desktop/src-tauri/src/backend.rs:136-171, :203-258; lionagi/studio/services/identity.py:26-42). Desktop setup continues to use the bundled application during 1.x; at 2.0 it installs the extracted application as its distribution specifies. It must not treat daemon liveness alone as proof that the required application was installed; the launcher validates the selected application before its listener opens.

The CLI uses the same daemon address resolution, including LIONAGI_STUDIO_URL, and retains local validation and export. On an open daemon, current requests without browser headers are admitted. Against a configured-token daemon, both current schedule clients are refused because they omit the token (lionagi/studio/cli.py:1186-1205, :1224-1245; lionagi/cli/machine_schedule.py:56-71; lionagi/studio/app.py:123-137, :591-603). Preserve and document that result during the mechanical move. The proposed correction is a separate additive client change that sends the explicitly configured daemon token in Authorization to the configured daemon origin, preserves the no-token mode, and never forwards the token to a different origin. It must cover both clients and their timeout behavior. It is not an implicit consequence of relocating _base_url.

Application-owned pairing state is rejected because it would become the authority for a daemon launch and could diverge from its epoch. Self-contained signed capabilities and per-request network verification are rejected because neither is required within one process. If F1 resolves to a separate application process, the amendment that replaces this section must specify online verification by the issuing daemon, revocation and epoch failure behavior, authenticated transport, and which component commits the decision; this record does not specify that design.

5. F5: Put shared execution code in the wheel and keep application ownership explicit¶

The core never names lionhub. New shared helpers required only by CLI, daemon, and Studio belong to lionhub, not to a new core facade. A core-facing interface is warranted only by an actual lower-layer caller, and must not import a higher layer. Amendment 1 A4 item 3 already assigns shared command logic and parsing to lionhub.orchestration; item 6 names lionhub.serve.cli, subject to the CLI record. The CLI record still owns the public CLI contract and ownership of li at 2.0 (ADR-0136 A4 items 3 to 6, ADR-0136 D7).

Keeping a route in the backend does not mean moving every storage helper in its current file there. Conversely, extracting a helper does not authorize putting an application route back in the daemon. The final dependency direction is application to wheel services to existing core services. Daemon-only execution and local CLI commands must work without importing the application. The explicit Studio launch adapter is the sole CLI path permitted to select it. In 1.x this is enforced with the application installed but disabled and blocked from import; at 2.0 it is enforced with the application absent. The loaded backend can use in-process services without adding HTTP calls between components in the same process.

Retain existing stores and identifiers. The daemon owns execution and scheduler task handles; the backend owns Operator task handles and application state semantics. Physical storage need not be split to establish those owners. Existing persisted paths, status values, notification registration, and database layouts are not rewritten because code ownership changes. Apply the existing deprecation and persisted-name obligations when a compatible move occurs (ADR-0136 D4 items 1 and 2, D5).

Perform helper extraction, dependency correction, and lifecycle failure cleanup as explicit preparatory changes before package moves. The new attachment seam requires that cleanup; it is a separately specified behavior correction, not part of a file rename. This follows the existing requirement that actual moves contain import rewrites rather than hidden behavioral refactoring (ADR-0136 D6). The table above does not authorize relocating the whole Operator package into wheel services to satisfy a transitive import.

A broad shared package containing copied application modules is rejected because it hides ownership and retains route imports. Moving shared application-derived policy into core merely to break an import cycle is rejected because the demonstrated callers are higher-layer clients. The existing scheduler service interface and the concrete CLI imports favor moving coherent services and parser declarations instead.

6. F6: Delete the unused HTTP list and keep one route inventory per owner¶

Delete _auto.py's private _HTTP_MODULES list and its unused HTTP loading path in A4 step 5. Deletion satisfies the requirement that the list leave the core. The populated CLI catalog and its selection and loading entry points have already left the core in step 3; preserve their behavior at the new home. Generic declaration types can remain only if they name no hub modules. No new HTTP host is built from the compiler (ADR-0136 A4 items 3 to 5). The import path lionagi._auto itself remains available through 1.x; removing unused private HTTP machinery does not remove that module.

Use the active registry's route metadata as the migration source. Split its registrations between daemon-owned and backend-owned inventories according to F3, and compose them deterministically when the application is selected. The host never imports an application inventory in daemon-only mode. Avoid two independently maintained lists for the same routes. Cross-inventory method-and-path collisions fail application construction, and relative order within each existing route family remains explicit.

The evidence is decisive: the active registry lists 29 modules; the unused list has 24 and omits identity, pairing, hooks_library, operator_hooks, and attention (lionagi/studio/registry.py:44-74; lionagi/_auto.py:116-141). Moreover, the latter loader compiles _lionagi_auto_marker attributes, whereas studio_route appends StudioRoute records to its own registry (lionagi/_auto.py:111, :321-328, :355-363; lionagi/studio/registry.py:95-122). Tests of synthetic compiler markers are not evidence that the real application uses it.

Correcting the five omissions and adopting the compiler is rejected because the decorator mismatch still requires a second route migration with no benefit to this boundary. Leaving the misleading list dormant is rejected because it remains an incorrect starting point for a daemon host. Deleting the unused path is smaller than maintaining a shadow HTTP registry.

7. F7: Keep the application in the wheel through 1.x and extract it from the library wheels at 2.0¶

The location follows the project's direction. The timing is this record's judgment: the application leaves the wheel in the same release that removes every other compatibility path (ADR-0136 D4).

The 1.x lionagi distribution continues to contain both import roots and the retained Studio application. The studio extra supplies the current HTTP and scheduling dependencies. The frontend remains a separate static build. A 1.x Studio installation needs no new backend distribution. The current wheel roots and extra already support this arrangement (pyproject.toml:141-146, pyproject.toml:276-277; ADR-0136 D1). The ADR-0136 D2 exempt set is {cli, studio} until Amendment 2 step 6 moves the CLI, {studio} from that step through the last 1.x release, and empty from 2.0.

At 2.0, the core lionagi wheel contains neither hub nor application code. The separate lionhub distribution contains the daemon and scheduler, under ADR-0136 D1's distribution split. The application leaves both library wheels. Its code lives at apps/studio/backend/src/, with Operator at apps/studio/backend/src/operator/, and never moves into lionhub. How the extracted application and the frontend are packaged, distributed, and released is outside this record. Console-script ownership and the core/hub file-ownership transition remain the obligations of the CLI and distribution records (ADR-0136 D1, ADR-0136 D7).

Whatever distribution is chosen must keep the boundary this record fixes:

The 2.0 transition creates a new environment instead of overlaying a separately installed hub onto a bundled 1.x hub. This avoids overlapping file ownership while allowing the prior complete 1.x environment to remain available for rollback. It must pass the distribution record's fresh-install, upgrade, uninstall, and rollback checks before release (ADR-0136 D1). No upgrade or rollback runs two scheduler processes on the same state. Backend relocation does not itself change the state schema or require a new migration framework.

Extracting the application during 1.x is rejected. A separate application distribution or extra would remove lionagi.studio imports that an installed 1.x wheel otherwise satisfies, before the release that removes every compatibility path (pyproject.toml:141-146; ADR-0136 D1, ADR-0136 D4).

Consequences¶

Under F1's proposed form, the design keeps the existing one-process pairing and desktop supervision shape, and the daemon remains exposed to a selected backend's process-level failures; the open alternative trades that exposure for a cross-process verification design. Either way the daemon API runs without the Studio application, and shared services must be separated from route-bearing files before packages move.

Configuration names and successful request contracts remain stable through mechanical moves; authentication corrections and lifecycle failure cleanup are separate changes with their own verification. A code-only rollback does not prove that persisted state can be rolled back.

ADR-0136 Amendment 2 preserves its D1 and extends the existing Studio exemption until 2.0. Wave 2 removes daemon and shared dependencies on application code but retains the application files for compatibility. The costs are temporary application weight in the wheel and explicit packaging work at 2.0. The final extraction changes file placement, imports, and activation source; it does not postpone the route, identity, or lifecycle design. The application's distribution after extraction is decided separately and must meet the installation behavior in F7.

Alternatives considered¶