Governance Direction: lionagi Governed Orchestration¶

Version: 1.0 | Phase: 2 — Synthesis and Standards Status: Accepted — blocking all Phase 3-7 implementation

Note on phase and play numbering: This document uses P-numbers (P11, P12–P24) to refer to sequential implementation phases in the governed orchestration build-out. P11 is the synthesis phase that produced this document; P12–P24 are the 13 implementation phases described in Section 7. Each P-number corresponds to a bounded scope of work (ADR revision, substrate implementation, adapters, or docs) that feeds into the next.

This document is the master blueprint for lionagi's governed orchestration build-out (implementation phases P12–P24). It records the strategic direction, architecture decisions, ADR verdicts, the full phase list, integration plan, and risk register. Developers beginning implementation must read this document in full before writing any code.

1. Strategic Direction¶

1.1 Thesis¶

Governed orchestration is the product. lionagi's differentiator is not model integration or orchestration convenience — it is the ability to certify that an automated run followed a declared charter: enforceable permissions, tamper-evident evidence, task certificates, and segregation-of-duties constraints. Competitors (LangGraph, LlamaIndex, CrewAI, AutoGen) provide orchestration. None provide orchestration-time governance with verifiable compliance artifacts.

The build strategy is build-over-adopt for governance primitives (hash chains, policy resolution, log tiers, certificates) and wrap-over-rewrite for external frameworks (LangGraph, LlamaIndex, CrewAI). lionagi governs at the boundary; it does not replace the user's existing tooling.

1.2 What We Are Building¶

The smallest coherent governed orchestration surface, end-to-end. Every component listed below must integrate before any adapter is meaningful.

Charter DSL v0 — YAML-shaped governance contract language. Strict schema, no wildcards, no prose-only rules, deterministic policy resolution, exact registry entries, deny-by-default permissions. Compiled to runtime targets; ratified with a reproducible SHA-256 hash. IDE hints generated from the same Pydantic schema used by the CLI validator. See: docs/governance/standards/dsl-style.md

Runtime Substrate — the engine that Charter contracts compile into and execute against:

• Immutable evidence nodes (SHA-256 hash chains, append-only audit storage)
• OperationContext — per-operation actor identity, policy pin, trace IDs, propagated explicitly
• Log tier governance — MUTABLE / PROTECTED / IMMUTABLE tiers, application-layer backends
• Gate framework — hard, soft, advisory gate results; fail-closed exceptions; single canonical GateResult
• Policy release pinning — session-bound snapshot, most-specific-wins resolver, deny-on-tie
• Task certificates — minted at run completion, consuming evidence chain head, gate outcomes, break-glass state, and policy version

Orchestration Integration — governance wired into existing lionagi execution paths:

• Session.flow() enforces role allowlists, op limits, artifact contracts, and charter checks
• Per-operation evidence sidecars (Branch.operate(middle=...))
• CLI FlowPlan validation: charter activation, op budgets, artifact contracts
• Run-end certificate minting after artifact verification

Governance Layer Objects — the safety control surface:

• Break-glass protocol: emergency elevated mode with immutable justification evidence and lifecycle spans
• JIT tool grants: no-standing-capability, single-use permit semantics resolved through the registry
• Tool registry allowlists: exact-match, compiled DSL targets, policy-resolved privileged tier
• Segregation of duties: assignment-time conflict matrix, bidirectional role independence

OTel Governance Tracing — projection over typed governance records:

• Typed records are created first; trace/span IDs are embedded in evidence
• OTel spans carry evidence hashes for enterprise correlation
• Span names follow the governance span taxonomy (see docs/governance/standards/trace-naming.md)
• Enterprise readiness path: retention tiers, redaction, SIEM export, cost tracking, audit/ops separation

Provider Adapters — Two Waves:

• G1 SDK-native: PydanticAI, OpenAI Agents SDK, Anthropic Agent SDK (~200 LOC each, native event hooks)
• G2 Framework: LangGraph, LlamaIndex, CrewAI (zero-rewrite, coarse boundary governance)

1.3 What We Are Not Building¶

The following scope is explicitly cut from the current implementation phases. These items may reopen in a future revision.

1.4 Market Positioning¶

lionagi enters a crowded orchestration market and must carve a distinct position. The governance layer is that position: users who need auditable, certifiable automated runs (regulated industries, enterprise compliance teams, high-stakes automation) cannot get this from existing frameworks. The charter model (declare constraints, compile them, enforce at runtime, certify outcomes) is the product. The adapters are the distribution mechanism.

2. Architecture Overview¶

Governance is an additive runtime layer — it does not replace existing lionagi execution infrastructure.

┌──────────────────────────────────────────────────────────────────┐
│  Charter DSL v0                                                  │
│  YAML-shaped source → CLI validate → compile → ratification hash │
└────────────────────────────┬─────────────────────────────────────┘
                             │ compilation targets
        ┌────────────────────┼──────────────────────────────┐
        ▼                    ▼                              ▼
┌──────────────┐  ┌──────────────────────┐  ┌─────────────────────┐
│ Gate         │  │ Registry + Policy     │  │ SoD + Charter       │
│ Registry     │  │ Release Pin          │  │ Runtime Objects      │
│ (ADR-0051)   │  │ (ADR-0052)           │  │ (ADR-0047/0048)     │
└──────┬───────┘  └──────────┬───────────┘  └─────────┬───────────┘
       │                     │                         │
       └─────────────────────┼─────────────────────────┘
                             ▼
┌──────────────────────────────────────────────────────────────────┐
│  ActionManager.execute_governed()                                │
│  Phase: pre-gate → execution → post-gate → evidence sidecar      │
│  (ADR-0043/0044 resolved: ActionManager owns pipeline)           │
└────────────────────────────┬─────────────────────────────────────┘
                             │ emits
       ┌─────────────────────┼────────────────────────────┐
       ▼                     ▼                            ▼
┌────────────┐  ┌──────────────────────┐  ┌──────────────────────┐
│ Evidence   │  │ OperationContext      │  │ Log Tiers            │
│ Chain      │  │ (actor, policy pin,  │  │ MUTABLE /            │
│ (ADR-0041) │  │  trace IDs)          │  │ PROTECTED /          │
│            │  │ (ADR-0050)           │  │ IMMUTABLE            │
│            │  │                      │  │ (ADR-0049)           │
└─────┬──────┘  └──────────┬───────────┘  └──────────┬───────────┘
      │                    │                          │
      └────────────────────┼──────────────────────────┘
                           ▼
┌──────────────────────────────────────────────────────────────────┐
│  OTel Projection                                                 │
│  Typed records → span with evidence hash → SIEM / collector      │
│  Spans are projection; evidence chain is authoritative storage   │
└──────────────────────────────────────────────────────────────────┘
                           │ consumed by
┌──────────────────────────┼───────────────────────────────────────┐
│  Session.flow() / FlowPlan                                       │
│  Charter activation → op limits → artifact contracts             │
│  → run-end certificate minting (ADR-0042)                        │
└──────────────────────────┬───────────────────────────────────────┘
                           │ wraps
┌──────────────────────────┼───────────────────────────────────────┐
│  Provider Adapters                                               │
│  G1: PydanticAI | OpenAI Agents SDK | Anthropic Agent SDK        │
│  G2: LangGraph | LlamaIndex | CrewAI                             │
│  Zero-rewrite: accept user's existing objects at the boundary    │
└──────────────────────────────────────────────────────────────────┘

Layering rules:

1. Evidence chain is the source of truth — OTel spans are projections, not the record.
2. ActionManager owns the single governed execution pipeline — gates plug into phases, not around it.
3. OperationContext is created and propagated explicitly — contextvars is a bridge only.
4. Charter compilation must resolve all targets before activation; unresolved targets fail closed.
5. Adapters claim only what they govern — coarse boundary wrappers emit boundary evidence only.

3. Provider Adapter Strategy¶

3.1 Zero-Rewrite Principle¶

Users bring their existing objects. lionagi wraps the user's existing object and governs its execution boundary. The adapter does not rewrite the user's framework code. A governed adapter has the shape:

GovernedAdapter(user_object=their_existing_thing, charter=loaded_charter)

The adapter intercepts invocation, enforces the charter at the boundary, emits evidence, and returns results. Internal framework calls (LangGraph node transitions, LlamaIndex retrieval steps, CrewAI task delegation) are observed evidence, not governed lionagi tool calls, unless the user explicitly translates framework tools into lionagi Tool objects.

3.2 G1 — SDK-Native Adapters¶

These adapters receive first implementation because they expose typed event streams and native hooks.

Each G1 adapter is approximately 200 LOC. Fine-grained claims require the framework tool to be translated into a registered lionagi Tool object. Coarse-only claims govern invocation boundaries.

3.3 G2 — Framework Adapters¶

These adapters accept existing user objects and apply coarse governance at the boundary.

Internal framework state transitions (LangGraph node routing, LlamaIndex retrieval pipelines, CrewAI task delegation chains) are observed boundary evidence, not governed lionagi actions, in coarse mode. Any adapter claiming fine governance for internal steps must register those steps as translated lionagi Tool objects with registry entries.

3.4 G3 — Edge Adapters (Not In Scope for Current Implementation)¶

smolagents, OpenCode, and HuggingFace inference suites remain optional. They reopen only if earlier implementation phases complete ahead of schedule with clean adversarial test results.

3.5 Adapter Claim Standards¶

Every adapter ships with an explicit claim matrix in its docstring and test fixture. The matrix declares: which events are governed (gate + evidence), which events are observed-only (boundary evidence, no gate enforcement), and which events are not captured. Overclaiming governance for internal framework calls is a defect that fails the adversarial test suite.

4. DSL Strategy¶

4.1 Charter DSL v0 Overview¶

Charter DSL v0 is YAML-shaped, not YAML-arbitrary. It has a fixed top-level block set, strict Pydantic schema, and deterministic compilation. It is not a general-purpose policy language. Unsupported features are rejected at parse time with actionable error messages.

Canonical top-level blocks (required unless marked optional):

charter_dsl      — version pin ("0.1")
kind             — agent_charter | session_charter
metadata         — identity, release, ratification
agents           — actor bindings (one per agent_charter, two or more per session_charter)
registry         — ratified tool/model/path snapshot with evidence
constraints      — executable gate or hook bindings with enforcement levels
sod              — segregation-of-duties conflict matrix (required, rules: [] if inactive)
permissions      — explicit allow/deny rules with specificity resolution
break_glass      — (optional) emergency elevated mode with attestation and time bounds
trace            — span and evidence requirements for runtime verification

Full syntax specification: docs/governance/standards/dsl-style.md

Three concrete charter examples are in: docs/governance/charter-dsl-v0.md

4.2 Compilation Pipeline¶

Charter compilation follows six ordered phases. Each phase is a gate — failure fails closed.

Phase 1: Parse
  YAML source → AST nodes
  Validates: top-level block presence, key spelling, value types
  Rejects: unknown keys, wrong types, missing required blocks

Phase 2: Validate
  Schema validation via Pydantic models
  Checks: constraint binding exclusivity (exactly one of gate_id/hook_name),
          SoD rule structure, permission specificity, trace span name set

Phase 3: Bind
  Resolves gate_id → registered GateExecutor
  Resolves hook_name + hook_phase → registered hook
  Resolves registry entry values → canonical identifiers
  Fails closed on any unresolved target

Phase 4: Normalize
  Produces canonical JSON representation
  Comments stripped; field order fixed; enum values lowercased
  Output is deterministic given the same input

Phase 5: Emit
  Produces typed runtime target objects:
    - GateRegistration objects (one per constraint)
    - RegistryEntry objects (one per registry entry)
    - SoDRule objects (bidirectional, one per sod.rules entry)
    - EvidenceRequirement objects (from constraints and trace)
    - TraceExpectation objects (from trace.require_spans)
    - PermissionPolicy object (from permissions block)
    - PolicyPin record (from metadata.policy_release)

Phase 6: Activate
  Computes SHA-256 hash of normalized JSON
  Compares against metadata.ratification.hash (must match for accepted charters)
  Records activation evidence (charter_id, hash, policy_pin, activated_at)
  Registers all emitted targets with the runtime
  Returns typed AgentCharter or SessionCharter runtime object

4.3 Runtime Binding¶

At runtime, the compiled charter binding is the source of authority for:

• Gate enforcement: ActionManager.execute_governed() consults registered gates by constraint_id
• Registry enforcement: tool calls check against ratified registry snapshots
• Policy resolution: ADR-0052 resolver uses the policy pin from the charter
• SoD checks: role conflict matrix is evaluated at assignment time
• Trace verification: required spans are validated post-run against the OTel export

4.4 IDE Hints and CLI Integration¶

The same Pydantic schema that validates charters at runtime is exported as JSON Schema for IDE integration. Editor hints cannot drift from runtime validation because they share the same source.

CLI commands (planned — not yet implemented; implementation is tracked in the charter parser and flow governance phases):

• li charter validate <file> — parse + validate, report errors with line numbers
• li charter compile <file> — full compilation through Phase 5, output runtime target summary
• li charter activate <file> — full compilation through Phase 6, register targets in session

5. Tracing Strategy¶

5.1 Architecture: Records First, Spans Second¶

The governance tracing architecture has a strict hierarchy:

1. The runtime creates typed governance records first (evidence nodes, gate results, certificates)
2. trace_id and span_id are embedded into evidence at record creation time
3. OTel spans are projected from the same records, carrying evidence hashes for correlation
4. Spans are never the authoritative storage for governance decisions

This means: an auditor verifying a run reads the evidence chain, not the OTel export. OTel is the enterprise observability interface (SIEM, alerting, dashboards), not the compliance record.

5.2 Span Taxonomy Reference¶

The canonical span registry is defined in docs/governance/standards/trace-naming.md (section 4). That file is the authoritative source for span names, required attributes, and retention tiers. The summary below uses the same attribute names as trace-naming.md.

Base spans:

Additional spans (permit, certificate, and break-glass lifecycle — implemented in the OTel tracing phase):

See trace-naming.md section 4 for the complete attribute list for every span type.

5.3 Enterprise Readiness Path¶

The span schema is designed for enterprise readiness from the start. Retrofitting observability attributes onto opaque spans is expensive — declaring them in the schema creates the migration path.

6. ADR Slate Verdict¶

All 12 governance ADRs are reviewed. The verdict is five SURVIVE, seven REVISE, zero KILL. No ADR is correct-as-is across the entire governance design; revisions resolve overlaps and bind the ADRs to the Charter DSL compilation pipeline and shared runtime types.

Revision convergence groups:

• ADR-0043 + ADR-0044 + ADR-0050 → one governed execution pipeline in ActionManager
• ADR-0047 + ADR-0051 + ADR-0052 → unified under Charter DSL compilation
• ADR-0045 + ADR-0046 → explicit lifecycle evidence and OTel spans for emergency/elevated paths

7. Implementation Phase List¶

13 phases (P12–P24). No phase may be added without explicit scope cut elsewhere. G3 adapters, REST evidence API, dashboards, and fine-grained framework internals are the first cut targets if budget is exceeded. P-numbers are sequential identifiers; each number corresponds to one bounded scope of work described below.

P12 — ADR Slate Consolidation (the governance consolidation phase)¶

P13 — Charter DSL Parser and Schema¶

P14 — Charter Compiler and Runtime Targets¶

P15 — Evidence Chain and Log Tiers¶

P16 — OperationContext and Policy Release Pinning¶

P17 — Governed ActionManager Gates¶

P18 — Flow Governance Integration¶

P19 — Certificates, Break-Glass, JIT, Registry, SoD¶

P20 — OTel Governance Tracing¶

P21 — SDK-Native Governed Endpoints (G1)¶

P22 — Framework Governed Endpoints (G2)¶

P23 — Governance Test and Adversarial Fixture Pack¶

P24 — Docs, Migration Guide, and Public API Polish¶

8. Integration Plan¶

The governance build is additive. After implementation, existing lionagi code will be modified at well-defined extension points only. New modules will live in lionagi/protocols/governance/ (proposed — not yet created). Existing modules will receive narrow additions.

8.1 lionagi/protocols/¶

New package: lionagi/protocols/governance/

Modified: lionagi/protocols/generic/

Modified: lionagi/protocols/action/

8.2 lionagi/session/¶

Branch gains governance state that is initialized when a charter is activated. Ungoverned branches remain unchanged — governance is opt-in at the Session.flow() or Branch.activate_charter() call.

8.3 lionagi/cli/orchestrate/¶

8.4 lionagi/adapters/¶

New adapter modules (P21, P22). The base class is GovernedAdapter in lionagi/adapters/governed_base.py (ADR-0068). Each concrete adapter module contains the framework-specific GovernedAdapter subclass and its claim matrix.

Each adapter module exports exactly one concrete GovernedAdapter subclass. The claim matrix is documented in the class docstring and mirrored in the test fixture (tests/adapters/).

No adapter module is added until the substrate (P15-P17) and flow governance (P18) are complete and verified. An adapter claiming governance before the substrate exists is a false claim.

8.5 lionagi/tools/¶

8.6 Public API (lionagi/governance/)¶

After P19, expose ergonomic imports over the full protocol path:

from lionagi.governance import (
    AgentCharter,
    SessionCharter,
    TaskCertificate,
    EvidenceChain,
    OperationContext,
    ToolRegistry,
    PolicyResolver,
    SoDMatrix,
    BreakGlassProtocol,
    JITGrant,
    governed_tool,
    GovernanceViolationError,
)

The public API is the migration entry point. v1 migration attaches governance to existing branches via branch.activate_charter(charter) without requiring Session-level flow rewrites.

9. Risk Register¶

Cross-References¶

This document is authoritative for implementation phases P12–P24. Changes require a governance ADR revision or an explicit direction amendment with project maintainers' approval.