Workspaces and data

Workspaces and what lives in them: sources, data views, connectors.

Source

An ingested data source — raw rows as imported.

Properties

• id (str)

• name (str)

• schema (pd.DataFrame)

Source.delete

Source.delete() -> None

Delete this source permanently. Requires the sources:delete scope.

Twins and datasets built on it keep their ids but lose their data — delete or retrain those first.

Source.link

Source.link() -> PlatformLink

The source's detail page on the platform, as a clickable URL.

Source.to_frame

Source.to_frame() -> pd.DataFrame

Undocumented; the signature above is the contract.

Source.extend

Source.extend(frame: pd.DataFrame) -> None

Append new rows to this source. Blocks until the rows are ingested.

Parameter	Type	Default	Description
frame	pd.DataFrame	required	Rows to append. The schema must match the source.

Raises

• InvalidArgumentError: frame is not a usable, non-empty DataFrame.

DataView

A derived, queryable dataset built from one or more sources.

Properties

• id (str)

• name (str)

• schema (pd.DataFrame)

DataView.delete

DataView.delete() -> None

Delete this dataset permanently. Requires the datasets:delete scope.

Twins trained on it keep their fitted models but cannot retrain until repointed at another dataset.

DataView.link

DataView.link() -> PlatformLink

The dataset's page on the platform, as a clickable URL.

DataView.to_frame

DataView.to_frame() -> pd.DataFrame

Undocumented; the signature above is the contract.

DataView.records

DataView.records(limit: int = 100, cursor: str | None = None) -> list[dict[str, Any]]

One page of rows as dicts.

Parameter	Type	Default	Description
limit	int	100	Rows per page.
cursor	str | None	None	The previous page's cursor, to continue from it.

Returns (list[dict[str, Any]]): The page's rows.

Connector

An organisation-level connector to an external system (Snowflake, S3, …).

Properties

• id (str)

• name (str)

Connector.test

Connector.test() -> dict[str, Any]

Validate that the stored credentials can reach the external system.

Connector.browse

Connector.browse(level: str, context: str = {}) -> Any

Walk the external system's hierarchy one level at a time.

Parameter	Type	Default	Description
level	str	required	Which level to list, for example databases or tables.
**context	str	{}	The levels already chosen, narrowing the listing.

Returns (Any): The listing for that level.

Connector.query

Connector.query(query: str, *, limit: int = 100, config: Any = {}) -> pd.DataFrame

Run a custom query against the external system and return sample rows.

The authoring loop for custom SQL: nothing is stored, database errors come back verbatim.

Parameter	Type	Default	Description
query	str	required	The SQL to run.
limit	int	100	Row cap on the sample that comes back.
**config	Any	{}	Connector config overrides, for example database=, warehouse=, schema=.

Returns (pd.DataFrame): The sample rows as a DataFrame.

Raises

• RootCauseError: The external system rejected the query. Its error is quoted verbatim.

Connector.import_table

Connector.import_table(
    table: str,
    *,
    name: str | None = None,
    timeout: float = 3600.0,
    config: Any = {},
) -> Source

Import one table into the workspace as a new source.

Parameter	Type	Default	Description
table	str	required	Table to import.
name	str | None	None	Name for the new source. Derived from the table when omitted.
timeout	float	3600.0	Seconds to wait for the import job.
**config	Any	{}	Connector config overrides, for example database=, schema=.

Returns (Source): The new Source.

Connector.import_query

Connector.import_query(
    query: str,
    *,
    name: str | None = None,
    timeout: float = 3600.0,
    config: Any = {},
) -> Source

Import the result of a custom query into the workspace as a new source.

Parameter	Type	Default	Description
query	str	required	The SQL whose result becomes the source.
name	str | None	None	Name for the new source.
timeout	float	3600.0	Seconds to wait for the import job.
**config	Any	{}	Connector config overrides.

Returns (Source): The new Source.

Connector.run_import

Connector.run_import(
    config: dict[str, Any],
    *,
    dataset_name: str | None = None,
    timeout: float = 3600.0,
) -> Source

Import with a raw, connector-specific payload.

The escape hatch under import_table and import_query.

Parameter	Type	Default	Description
config	dict[str, Any]	required	The connector's own import payload.
dataset_name	str | None	None	Name for the new source.
timeout	float	3600.0	Seconds to wait for the import job.

Returns (Source): The new Source.

Workspace

A workspace handle: sources, datasets (views), twins, connectors, ontology.

Properties

• id (str)

• name (str)

• sources (_Collection)

• datasets (_Collection)

• twins (_Collection)

• connectors (_Collection)

Workspace.link

Workspace.link() -> PlatformLink

The workspace's home page on the platform, as a clickable URL.

Workspace.add_connector

Workspace.add_connector(name: str, type: str, credentials: Any = {}) -> Connector

Register a connector to an external system (credentials are stored encrypted).

Parameter	Type	Default	Description
name	str	required	Name for the connector.
type	str	required	Connector type, for example postgresql or snowflake.
**credentials	Any	{}	The connector's credentials. Stored encrypted, and never returned by the API.

Returns (Connector): The registered Connector.

Workspace.dataset

Workspace.dataset(needle: str) -> DataView

Undocumented; the signature above is the contract.

Workspace.source

Workspace.source(needle: str) -> Source

Undocumented; the signature above is the contract.

Workspace.twin

Workspace.twin(needle: str) -> Twin

Undocumented; the signature above is the contract.

Workspace.upload

Workspace.upload(
    frame: pd.DataFrame,
    name: str,
    *,
    wait: bool = True,
    timeout: float = 600.0,
) -> Source

Upload a DataFrame as a new source (parquet on the wire, full ingest server-side).

Parameter	Type	Default	Description
frame	pd.DataFrame	required	The data to upload.
name	str	required	Name for the new source.
wait	bool	True	Block until the schema materialises server side.
timeout	float	600.0	Seconds to wait for ingest, when wait is True.

Returns (Source): The new Source.

Raises

• InvalidArgumentError: frame is not a usable, non-empty DataFrame, or name is blank.

Workspace.create_twin

Workspace.create_twin(
    name: str,
    *,
    kind: str = 'static',
    dataset_id: str | None = None,
    source_id: str | None = None,
    time_column: str | None = None,
    environment_columns: list[str] | None = None,
    tags: list[str] | None = None,
) -> Twin

Create a twin over a dataset, or directly over a raw source.

Parameter	Type	Default	Description
name	str	required	Name for the twin.
kind	str	'static'	One of static, temporal, multi-environment-static, multi-environment-temporal.
dataset_id	str | None	None	Dataset to train on. Pass this or source_id.
source_id	str | None	None	Raw source to train on, skipping the dataset step.
time_column	str | None	None	Time column, for temporal kinds.
environment_columns	list[str] | None	None	Columns that identify an environment, for panel kinds.
tags	list[str] | None	None	Tags to file the twin under.

Returns (Twin): The new Twin, untrained.

Raises

• InvalidArgumentError: Both, or neither, of dataset_id and source_id were given, or kind is not a known twin kind.