Orchestrator

*Note: This page will be updated soon. *

Bringing systems for data loading, manipulation and writing together is what makes Keboola so powerful and easy to use. With source connectors, you can fetch data from data sources into Storage. With transformations and applications, you can modify the data. With destination connectors, you can send it to destination systems.

The Orchestrator component has two important features:

Organizing the configured components into logical building blocks or full data pipelines.
Scheduling automated and repeated execution of those blocks or pipelines.

The former feature of the Orchestrator allows you to define dependencies between different configured components. For example, it allows you to specify that a transformation must be run after an extraction which provides the source data for it. You can also specify that two extractions may run in parallel because they are independent of each other. This way it is possible to organize large and complex projects and ensure you always work with the latest data available.

The latter feature of the Orchestrator allows you to fully automate the entire data pipeline. That guaranties that the destination systems (e.g. Tableau Business Analytics) always contain the latest available data, without any user intervention. Every change to your entry data will automatically propagate up to your dashboard or visualisation project.

See the corresponding part of our tutorial to quickly set up an example Orchestration.

Single Task Orchestrations

In case you want to automate a single configuration, the quickest option is to use the Automate button at the configuration page:

Specify the orchestration name and schedule:

If any of the predefined scheduled options do not fit your needs, select Custom schedule to set up an individual schedule.

When done, click the Automate button to create a new Orchestration containing the configuration as its only task. The orchestration has no special properties, so you can further modify it as any other orchestration.