ADR-0001: Element identity and polymorphic serialization envelope¶

• Status: Accepted
• Kind: Retrospective
• Area: core-data-model
• Date: 2026-07-09
• Relations: none

Context¶

LionAGI needs persisted and externally addressable model objects to retain identity across collections, graph relationships, messages, logs, and serialization boundaries. Five concrete problems shape the shipped model.

P1 — Addressable objects need stable identity independent of their current values. A node can change content or metadata while references held by a Pile, Graph, log, or operation still need to identify the same object. Value equality would make identity depend on mutable fields; object identity would be lost at a serialization boundary.

P2 — A serialized base type must recover the concrete model. Callers frequently hold Element, Node, or heterogeneous Pile references while the payload contains an Instruction, Graph, Event, or extension subclass. Reconstructing only the statically requested base class would either reject subtype fields or discard subtype behavior.

P3 — Several persisted shapes and naming generations must remain readable. Normal model dumps use metadata; database adapters use node_metadata. Current writers store a fully qualified class name, while older built-in payloads may contain a short name such as Instruction. The reader therefore cannot be a single direct registry lookup.

P4 — The envelope has a trust and extension boundary. Node subclasses register automatically, other Element subclasses do not, and a fully qualified discriminator may cause a Python module import. That behavior makes third-party recovery possible, but it also means deserialization is not a passive data-only operation.

P5 — Graph nodes need a stable content shape without making every value object a node. Node adds arbitrary content and a nullable embedding to the Element envelope. Configuration and query objects do not become addressable merely because they are Pydantic models, and can remain plain models.

The current implementation is defined by lionagi/protocols/generic/element.py, lionagi/_class_registry.py, and lionagi/protocols/graph/node.py.

This ADR deliberately does not decide:

• node lifecycle policy such as versioning, soft deletion, or content hashing; those features are activated by per-class node configuration and are not required by the identity envelope;
• adapter-specific database schemas; adapters consume database-mode dictionaries but own their table and transport contracts;
• vector indexing or retrieval behavior; Node.embedding is a stored compatibility field, not a declaration that every Node is searchable by vector; or
• a future trusted plugin-registration policy; the current asymmetric registry/import behavior is recorded here, while its replacement remains an explicit delta.

Decision¶

D1 — Element supplies identity, creation time, and metadata¶

Element is the common identity envelope for addressable or persisted LionAGI models. It is a Pydantic model and the nominal Observable used by generic collections.

The contract (lionagi/protocols/generic/element.py):

class Element(BaseModel, Observable):
    model_config = ConfigDict(
        arbitrary_types_allowed=True,
        use_enum_values=True,
        populate_by_name=True,
        extra="forbid",
    )

    id: UUID = Field(default_factory=uuid4, frozen=True)
    created_at: float = Field(
        default_factory=lambda: now_utc().timestamp(),
        frozen=True,
    )
    metadata: dict = Field(default_factory=dict)

    @property
    def created_datetime(self) -> datetime: ...

    @classmethod
    def class_name(cls, full: bool = False) -> str: ...

Exact semantics:

• With no input, id is a UUID4, created_at is the current UTC timestamp expressed as a float, and metadata is a new empty dictionary.
• id accepts a UUID or a string accepted by UUID(str(value)). Invalid values fail Pydantic validation. The field cannot be assigned after construction because it is frozen.
• created_at=None produces a new current timestamp. A float is retained; a datetime uses its .timestamp(); a string is first parsed as ISO text after replacing a space with T, and a timezone-naive parsed string is treated as UTC. If ISO parsing fails, a numeric string is converted with float; failure of both string conversions becomes ValueError("Invalid datetime string: ..."). Other numeric-like inputs use float(value); failures become ValueError("Invalid created_at: ..."). The field is frozen after construction.
• created_datetime always projects the stored float as a timezone-aware UTC datetime.
• Falsey metadata becomes {}. A non-dictionary input is passed through the recursive to_dict utility; failure to obtain a dictionary raises ValueError("Invalid metadata.").
• If input metadata already contains lion_class, it must equal the concrete validating class's fully qualified name. A mismatch raises ValueError("Metadata class mismatch.").
• Unknown top-level model fields are rejected (extra="forbid"); arbitrary values remain allowed inside declared fields such as metadata and Node content.
• Equality between two Elements compares only UUIDs. Comparing to a non-Element returns NotImplemented. Hashing uses the UUID, and every Element is truthy even when a subclass also represents an empty collection. Collection subclasses may deliberately override truthiness.

The negation matters: without one identity field, Pile keys and Graph endpoints would each need their own identity convention, and equality of mutable objects would be unsuitable for references.

D2 — Serialization carries a concrete-class discriminator in metadata¶

Element has three dictionary modes and one JSON form. The discriminator is not a separate top-level field; it is injected into the serialized metadata copy.

The contract (lionagi/protocols/generic/element.py):

def to_dict(
    self,
    mode: Literal["python", "json", "db"] = "python",
    db_meta_key: str | None = None,
    **kw,
) -> dict: ...

def to_json(self, decode: bool = True, **kw) -> str: ...

@classmethod
def from_dict(cls, data: dict) -> Element: ...

@classmethod
def from_json(cls, json_str: str) -> Element: ...

The canonical Python shape is:

{
    "id": "00000000-0000-0000-0000-000000000001",
    "created_at": 1.0,
    "metadata": {
        "source": "example",
        "lion_class": "lionagi.protocols.generic.element.Element",
    },
}

Database mode changes only the metadata key by default:

{
    "id": "00000000-0000-0000-0000-000000000001",
    "created_at": 1.0,
    "node_metadata": {
        "source": "example",
        "lion_class": "lionagi.protocols.generic.element.Element",
    },
}

Exact semantics:

• Python mode uses Pydantic model_dump(**kw), adds the concrete fully qualified class name to the dumped metadata, and removes sentinel-valued fields. It does not intentionally add lion_class to the live object's metadata.
• JSON mode serializes through orjson and parses the bytes back to JSON-compatible Python values; UUIDs therefore appear as strings.
• Database mode follows JSON mode, removes metadata, and installs it under db_meta_key or the default node_metadata.
• Any mode other than python, json, or db raises ValueError("Unsupported mode: ...").
• to_json(decode=True) returns text. decode=False returns the underlying JSON bytes even though the public annotation says str; callers in to_dict rely on those bytes.
• from_json parses with orjson and delegates all recovery behavior to from_dict.
• from_dict shallow-copies the top-level dictionary and separately copies the chosen metadata dictionary before removing lion_class; the caller's input dictionaries are not mutated.
• When both metadata keys are present, node_metadata is selected first. The recovered in-memory metadata does not retain lion_class; the discriminator is regenerated on the next write.
• If no concrete subclass delegation occurs, reconstruction ends with cls.model_validate(data).

The discriminator is fully qualified because short class names are ambiguous across extension modules. Keeping it inside metadata preserves the stable three-field base envelope and lets older database shapes move metadata as one unit.

D3 — Class recovery is registry-first, import-capable, and legacy-aware¶

The resolver is deliberately compatibility-oriented rather than a uniform extension registry.

The contract (lionagi/_class_registry.py):

LION_CLASS_REGISTRY: dict[str, type] = {}

def get_class(class_name: str) -> type: ...

Resolution proceeds in this exact order:

1. Return an exact LION_CLASS_REGISTRY[class_name] hit.
2. If the string contains ., split it at the last dot and attempt to import that module attribute; return it only when it is a class.
3. Import each module in the fixed _BUILTIN_MODULES tuple and look for an attribute matching the short name.
4. Re-scan registry keys for an exact short-name or .<short-name> suffix match, because importing built-ins may have registered Node subclasses.
5. Raise ValueError("Unable to find class ...").

Element.from_dict removes the discriminator, asks get_class, restores the remaining metadata, and delegates to the resolved type's from_dict whenever the resolved type differs from the class currently reconstructing or supplies a different implementation. Resolver/import errors are caught for a final direct dotted import attempt; an unresolved or malformed discriminator then propagates an import, attribute, type, or value error rather than silently constructing the wrong base type.

Node provides the only automatic registration hook:

class Node(Element, Relational, AsyncAdaptable, Adaptable):
    @classmethod
    def __pydantic_init_subclass__(cls, **kwargs: Any) -> None:
        super().__pydantic_init_subclass__(**kwargs)
        LION_CLASS_REGISTRY[cls.class_name(full=True)] = cls

Exact semantics:

• The Node base itself is not inserted by its own subclass hook; each concrete Node subclass is.
• Registration is ordinary dictionary assignment. A second class with the same fully qualified key silently replaces the first; last writer wins.
• Non-Node Element subclasses do not auto-register. Built-in classes remain recoverable through module attribute lookup; importable third-party classes remain recoverable through dotted paths.
• A short-name suffix collision is resolved by registry insertion order, not by an explicit ambiguity error.
• Dotted-path recovery executes Python import machinery. Payloads must therefore come from a trust context permitted to name importable classes; the code does not maintain an allow-list.

This shape grew from two compatibility needs: qualified names for extensions and continued reads of older built-in short names. It buys recovery breadth at the cost of an asymmetric registration and trust model, which is retained as a delta rather than described as ideal.

D4 — Node adds content and a compatibility embedding field¶

Node is the graph-addressable specialization of Element.

The contract (lionagi/protocols/graph/node.py):

class Node(Element, Relational, AsyncAdaptable, Adaptable):
    node_config: ClassVar[Any] = None

    content: Any = None
    embedding: list[float] | None = None

The resulting constructor fields are id, created_at, metadata, content=None, and embedding=None.

Exact semantics:

• content accepts any Python value. On serialization, an Element becomes its to_dict() shape, a Pydantic model becomes model_dump(), a LionAGI DataClass uses its configured to_dict, and any other value passes through unchanged.
• On input, a dictionary whose metadata contains lion_class is reconstructed with Element.from_dict; other dictionaries remain ordinary content.
• embedding=None remains null. A list is converted element-by-element with float; invalid list contents raise ValueError("Invalid embedding list.").
• A string must decode as a JSON list and every member must convert to float; malformed JSON or a non-list JSON value raises ValueError("Invalid embedding string."). Other input types raise an invalid-embedding-type error. An empty list is valid.
• Sync and async adapters force to_dict(mode="db") on writes and from_dict on reads. JSON and TOML adapters are registered at module import; the optional PostgreSQL adapter is checked and registered lazily on first PostgreSQL adaptation, with missing extras degrading to no adapter. The PostgreSQL availability check is one-shot per process: the checked marker is installed even when dependencies are missing or registration raises, so later calls do not retry after the environment changes.

The embedding field remains on the broad Node base because it is part of the persisted shape. Its presence is not a vector-capability interface, and removing it safely requires an inventory and migration rather than a local field deletion.

Consequences¶

UUID references, heterogeneous collections, graph relationships, and persisted model payloads can share one base envelope. Most concrete model types recover their own schema and behavior through the discriminator, and legacy built-in short names remain readable.

Contributors extending the model must understand three non-obvious costs:

• subclass construction can violate the envelope if a custom __init__ does not forward inherited fields; Edge currently demonstrates this failure and is captured in the delta table;
• the resolver's ability to import a dotted path is also a code-loading trust boundary; and
• automatic registration is a Node behavior, not an Element behavior, with silent overwrite and ambiguous short-suffix recovery.

Reversing D1 would invalidate UUID-keyed stores and graph endpoints. Reversing D2 or D3 requires a persisted-data migration and a compatibility reader. Moving Node.embedding behind a capability is less structurally invasive but still requires a persisted-shape migration.

The inherited timestamp and metadata appear on descendants that may not independently need them. This is accepted for addressable framework models, while configuration and query value objects stay outside the inheritance tree.

Current-vs-ideal delta¶

Alternatives considered¶

No shared model envelope¶

Each model family could define its own identifier, timestamp, metadata, and serializer. That would let small value objects carry only what they need, but Pile, Graph, adapters, and polymorphic collections would need parallel identity and reconstruction rules. The repeated reference boundary is concrete and already shared, so duplicated envelopes lost to the common Element contract.

Value equality or Python object identity¶

Value equality would change identity when mutable content changes and can merge distinct records with equal fields. Python object identity cannot survive JSON or database storage. UUID equality is stable across both mutation and process boundaries, so both alternatives lost.

A top-level type field¶

A dedicated top-level discriminator would make the type marker visually prominent. It would also change the stable base record shape and require database adapters to map another reserved column. Keeping lion_class inside metadata lets metadata/node_metadata move as one payload and remains compatible with already persisted data.

Short class names only¶

Short names are compact and matched early built-in data, but they cannot distinguish two extension modules defining the same class name. Fully qualified names are written now; fixed built-in search remains only as a compatibility reader.

A closed registry with no dotted imports¶

A closed registry would make the deserialization trust boundary explicit and would detect unknown extension classes before import. It would also make current importable third-party discriminators unreadable unless every class was registered before the payload arrived. The shipped code chose compatibility and extension recovery; a trusted registration policy is deferred as delta 1 rather than retroactively asserted.

Automatic registration for every Element subclass¶

Moving the subclass hook to Element would make extension behavior uniform. It also changes global registration and collision behavior for every value in the hierarchy and does not by itself solve trusted import or legacy short-name ambiguity. The current Node-only hook is recorded as-is pending the explicit policy decision.

Make every Pydantic model an Element¶

Uniform inheritance would simplify a few generic annotations, but it would assign UUIDs, timestamps, and mutable metadata to configuration and query values that are compared and moved by value. The boundary remains addressable or persisted framework objects, not all structured data.

Remove Node.embedding immediately¶

An opt-in vector capability would narrow the base node contract. Immediate removal loses the persisted compatibility field and provides no migration for stored embeddings. The capability split remains a delta until the stored-data inventory and migration are defined.

Notes¶

The Edge reconstruction delta was verified from lionagi/protocols/graph/edge.py: the custom constructor forwards only head, tail, and a newly assembled properties dictionary to Element, so inherited envelope keys arriving through **kwargs become nested properties while a new Element UUID and timestamp are generated. ADR-0004 records the separate Graph-level reconstruction failures without changing ownership of the base Edge serialization decision.