`filter-payload` (Processing Node)

The filter-payload node checks an upstream payload against allowed file types and forwards the binary data only when it matches.

Use this node as a gate before type-specific processors so downstream nodes only receive known payload formats.

Configuration Schema

Property	Type	Required	Description
`dataSourceName`	`string`	Yes	Data source used when writing the filtered binary output.
`allowedTypes`	`array<string>`	Yes	Allowed logical type keys (for example: `json`, `csv`, `txt`, `xml`, `html`, `dis`, `protobuf`, `bin`).

At runtime, the processor:

Fetches the latest full input payload (fetch_latest_full_input_data).
Reads config values dataSourceName and allowedTypes.
Runs type detection (payload_filter_utils.orchestrator.guess) on the input bytes with allowedTypes.
If detection matches, writes a binary output artifact with extension set to the detected type.
Uses a deterministic outputIdentifier to avoid re-processing the same input/type/size combination.

If no input is available, the node fails.
If input does not match any configured allowed type, the node fails.
If the same input was already filtered previously (same detected type, source path, and size), processing is skipped.
On successful match, output is written as binary via submit_binary_output.

Validation behavior:


{
  "dataSourceName": "default-store",
  "allowedTypes": ["json", "csv"]
}

Supported type keys based on detector mapping:


{
  "dataSourceName": "default-store",
  "allowedTypes": ["json", "csv", "txt", "xml", "html", "dis", "protobuf", "bin"]
}