`lattice-processor` (Processing Node)

The lattice-processor node inspects upstream payloads and produces a single JSON summary describing detected data types, lightweight metadata, and per-node/per-item observations.

Use this node when you need a broad, defensive inspection of mixed inputs (JSON, tabular, text, archives, and other binary formats) before downstream processing decisions.

Configuration Schema

Property	Type	Required	Description
`dataSourceName`	`string`	Yes*	Name of the data source used when writing the summary output artifact.
`mode`	`string`	No	Processing mode label. Currently recorded in output and reserved for future behavior tuning.

Notes:

Manifest schema marks dataSourceName as required.
Runtime also supports fallback to runner default_store when dataSourceName is omitted.
Additional configuration properties are currently allowed by the manifest.

How it works

At runtime, the processor:

Fetches all upstream inputs (fetch_all_inputs) across parent nodes.
Resolves target data source from dataSourceName (or fallback runner default_store).
Iterates each input blob and detects broad format category.
Produces lightweight summaries:
- JSON: key/schema summary
- Tabular: headers/type samples
- Text: line count and sample excerpt
- Archives: container/decompression metadata and inner detection where possible
Builds a single summary payload with:
- lattice_version
- mode
- nodes (per parent node item reports)
- overview (total_nodes, total_items, by_category)
Submits the summary JSON as node output.

The processor is defensive by design and attempts to continue even when individual items are malformed.

Output shape (high level)


{
  "lattice_version": 1,
  "mode": "standard",
  "nodes": {
    "parent-node-a": [
      {
        "index": 0,
        "size_bytes": 1234,
        "sha256": "abc...",
        "detected": {
          "category": "json",
          "format": "json"
        }
      }
    ]
  },
  "overview": {
    "total_nodes": 1,
    "total_items": 1,
    "by_category": {
      "json": 1
    }
  }
}

Runtime failure behavior

The node fails when:

No upstream inputs are available.
No usable data source can be resolved (dataSourceName missing and no default_store fallback).
The resolved data source cannot be found.

Per-item analysis errors are captured into item notes where possible; they do not automatically fail the whole node.

Example configuration


{
  "dataSourceName": "default-store",
  "mode": "standard"
}

Integration in a session template

Place lattice-processor after one or more upstream nodes that emit file/blob outputs.
Configure dataSourceName (or ensure runner default_store exists).
Optionally set mode.
Connect downstream nodes that consume the generated JSON summary.

lattice-processor (Processing Node)