On data arrival
Start a flow when a dataset is uploaded, synced, changed or its schema drifts.
Starts a flow when a dataset or connector this application watches receives data.
topics narrows which arrivals count — an upload, a connector sync, any change, or a schema drift. debounceS collapses a burst of arrivals into one run. Keeps state between runs so it knows what it has already seen, which makes it durable only.
At a glance
Module type
OnDataArrival
Category
Events
Default tier
durable
Runs on
durable
Starts a flow
yes
Holds a body
no
Keeps state between runs
yes
Available in
every application type
Parameters
concurrency
"replace" | "skip" | "queue" · required · defaults to "queue"
What to do when the previous run is still going.
datasetIds
string[] · required · defaults to []
The datasets to watch.
connectorIds
string[] · required · defaults to []
The connectors to watch.
topics
("dataset.uploaded" | "dataset.synced" | "dataset.changed" | "dataset.schemaDrift")[] · required · defaults to ["dataset.uploaded","dataset.synced"]
Which kinds of arrival count: an upload, a sync, any change, a schema drift.
debounceS
number · required · defaults to 0
Collapse arrivals within this many seconds into one run.
Inputs
None.
Outputs
event
event
event, json
The event envelope: {kind, at, payload}.
datasetId
scalar<string>
scalar, json, text, signal
Which dataset arrived.
data
dataframe
dataframe, signal
The rows that arrived, ready to work on.
Last updated