Reference - Workflow editor


Cargo's workflow editor is the primary place to build, edit, and inspect any workflow. It consists of a canvas where nodes can be combined and linked together to create a workflow's logic.

It gives access to workflow settings (including each workflow node configuration) as well as the ability to make test node executions with sample data to test as you build the workflow logic.

The editor also gives access to workflow versions which allow tracking the history of workflow edits by all users on the users of the workspace. The editor also gives access to a runs view that permits the enrolment of new runs as well as managing the history of previous runs with details on the input data and outcome of each run created.

TOP BAR

Top bar

Manage workflow settings directly from the top bar of the orchestration panel.

Cover image
  • Name
    Return to workspace
    Description

    Navigate back to the main workflows panel using the dedicated button on the top bar. The editor always autosaves your latest changes in a draft, so whenever you exit it you can get back to the editor and continue where you left off.

  • Name
    Enable workflow
    Description

    Toggling the workflow enablement button on the left side of the top bar enables the workflow to automatically enrol records as a function of changes in the underlying data model. You can only enable a workflow once it is published.

  • Name
    Data model connector
    Description

    The tiny icon next to the workflow's name in the top bar represents the connector used by the data model to feed the workflow, example 'HTTP'.

  • Name
    Rename a workflow
    Description

    Click inside the workflow's display name in the top bar to rename the workflow.

  • Name
    Version preview
    Description

    View past drafts and published versions of the workflow, including the date and time of the last edit. Selecting a past version or draft will preview its workflow configuration and display a prompt asking you to rebase to the selected version.

  • Name
    Editor tab
    Description

    Switch to the editor tab to inspect the primary workflow canvas.

  • Name
    Runs tab
    Description

    Switch to the runs tab to analyze past executions, filter by run status, retry/cancel specific runs, setup enrolment conditions and batch health alerts.

  • Name
    Undo/redo actions
    Description

    Use the undo/redo buttons on the top bar to reverse or re-apply changes in your worklfow draft.

  • Name
    Share workflow access
    Description

    Use the share button on the top bar to manage access to the workflow with other users in the workspace.

  • Name
    Publish workflow
    Description

    Publish your workflow draft to your workspace with the publish button. A published workflow version will be used for future enrolment and execution of new and retried runs. The button automatically becomes available when a change is detected in your local draft versus the currently published version. Moreoever, it is only clickable when no errors are present in the workflow logic.

  • Name
    Export workflow
    Description

    Clicking the ... on the far right side of the top bar reveals Export JSON button, which enables the user to download a config file containing the current workflow draft's configuration.

  • Name
    Import workflow
    Description

    Inversely to the Export JSON button, clicking Import JSON will open a modal allowing the upload of a workflow's config in JSON format. Improperly formatted JSON will result in a failure to import the workflow configuration.

  • Name
    Edit workflow settings
    Description

    Clicking the ... on the far right side of the top bar reveals the Edit button which allows the user to edit the name, description, folder and rate limit settings of the workflow.

  • Name
    Duplicate workflow
    Description

    Click on the ... on the far right side of the of the top bar and select Duplicate to create a copy of the current workflow, setting the current draft as the initial draft of newly created workflow.

  • Name
    Remove a Workflow
    Description

    Click on the ... on the far right side of the top bar and select Remove to permanently delete a workflow from the workspace.


RIGHT BAR

Right bar

Access node catalog and manage node positioning within the workflow editor.

Cover image
  • Name
    Add a node
    Description

    In order to add a new node, click the + icon in the top right to open the Nodes Catalog and drag-and-drop the selected node to the workflow canvas.

  • Name
    Autolayout
    Description

    Automatically repositions nodes neatly across the workflow editor.

  • Name
    Run next nodes
    Description

    Automatically run all remaining nodes present in the current workflow draft. This is useful to quickly populate test data for a preview of all nodes in the workflow.

  • Name
    Node Configuration
    Description

    Select any node in the workflow canvas by clicking on it, this will open up a sidebar with details of node configuration. Learn more about this sidebar in the Nodes reference.

  • Name
    Node Catalog
    Description

    After clicking the "Add a node" button the right sidebar becomes "Node Catalog" which lists all available node types. Drag-and-drop any selected done to add it to the workflow canvas.

  • Name
    Note
    Description

    At any moment you can drag and drop a yellow box from the bottom right of the editor to create a sticky note in your workflow to add extra commentary to annotate your workflow. Notes do not impact the behavior or logic of the workflow and cannot be linked to nodes in the workflow.


RUNS

Runs

Inspect details of historical workflow runs.

Cover image
  • Name
    Search
    Description

    Filter previous runs using a key search term.

  • Name
    Filters
    Description
    • Nodes - select which nodes to display in the runs table, by default nodes from archived versions are not displayed in the table.
    • Statuses - filter historical runs based on the run status, e.g. success, failed, skipped, etc.
    • Versions - filter runs by workflow version it was created by.
    • Batches - filter runs by the batch they were a part of.
  • Name
    Run details
    Description

    Click on the expand button on the left hand of any run in the table opens a detailed view about the run in a diagram including input data, output data, and node execution details. Clicking on an individual cell in the runs table displays a modal containing the configuration, input and output data of the selected node for the run in question.

  • Name
    Data model
    Description

    View details of the data model set as the base for the given workflow.

  • Name
    Schedule
    Description

    Select how the workflow will be triggered. By default, it is set to "Realtime" or every "Every 12 hours" depending on the type of the data model set as the base of the given workflow. The available options depend on the capabilities of the relevant connector.

  • Name
    Enroll
    Description

    Manually select and enroll selected records from a related data model to run in a batch. Set filters to limits the number of records enrolled.

  • Name
    Status
    Description

    Allows to configure two parameters defining workflow health status:

    • Batch Health threshold - what is the success rate threshold at which the workflow is considered healthy, e.g. 90% of runs in a batch need to be successful, before a health alert is emitted.
    • Health alert - configure a Slack message alert that will be sent when the batch health threshold is not met. Ensure Cargo has access to the destination slack channel in order to successfully receive the alert.

HEALTH STATUS

Health Status

Define when workflow is considered healthy and optionally set up a Slack alert when error rate spikes above defined threshold.

Cover image
  • Name
    Batch Health threshold
    Description

    Define percentage of runs in a given batch that needs to be successful in order to mark whole batch as healthy.

  • Name
    Health alert
    Description

    Optionally configure a Slack Write action that will be triggered in case of unhealthy batch.

    • Slack Connector - dropdown to select existing Slack connector or create a new one
    • Action - only supports "Post message"
    • Slack Channel - select existing Slack channel or user to send a Direct Message. It can be a static value or an expression
    • Bot Username - name of bot that will send the configured message
    • Bot Icon Emoji - additional bot emoji icon
    • Format - defines how Slack altert message will be configured, either in Markdown or using Bocks
    • Message content - available when Format is set Markdown
    • Blocks - available with Format is set to "Block Kit"

VERSIONS

Versions

Each workflow keeps the history of all published versions and the current draft. Switch between a draft mode that allows edits and a preview mode that allows you to see different versions of the workflow.

  • Name
    Deployed
    Description

    Preview the currently deployed workflow version.

  • Name
    Draft
    Description

    Current editable version of the workflow. Since the editor autosaves each change there is always only one draft at a time.

  • Name
    Archived
    Description

    View all the previously published versions of the workflow.

  • Name
    Use this version
    Description

    While previewing any version of the workflow other than the current draft a button appears below the top bar: "Use this version", it clones the selected version and sets it as a new draft.


SYNC

Sync

A sync is an enrolment of records from the underlying data model to create runs through the workflow logic. Syncs can be manual or automatic (based on a predefined interval). Not all schedule types are available for in case of all data models, check the data storage documentation details on this concept.

  • Name
    Schedule type
    Description

    This is the main setting that controls how workflow enrols records.

    • Manual - Enroll records manually.
    • Real-time - Enroll records when new records are added to the model, e.g. by calling a HTTP webhook. If selected, no other option is available to manage the sync settings.
    • Interval - Schedule to run on a set interval (e.g. once per hour). This is a default setting for many CRM connectors
    • Cron - More precise than the interval schedule, allows defining at what time of the day and days of the week exactly the workflow will run
    • Dependency - Triggers a workflow run post the completion of another selected workflow.
  • Name
    Schedule settings
    Description

    In case of interval, cron and dependency schedule types more options are required:

    • Interval - Select from a predefined list of intervals provided by the connector, e.g. 30 minutes.
    • Cron expression - Raw cron expression to control when the workflow will run. Cron format is following: minute hour day-of-the-month month day-of-the-week. E.g. 0 9 * * 1-5 will run the workflow every working day (Mon-Fri) at 9am UTC, on the other hand 0 */3 * * * will run the workflow every 3rd hour all the week.
    • Workflow - Select another workflow, which when completed will trigger the current workflow
  • Name
    Conditions filter
    Description

    Set filters to determine which data model records will be enroled into the workflow.

  • Name
    Conditions sort
    Description

    Set the order in which records will be enroled.

  • Name
    Conditions limit
    Description

    Maxium number of records to be considered for each run of the workflow. Leave empty to select all elgible records.

  • Name
    Conditions changes
    Description

    Determines which type of change in a given record should enrol it into the workflow.

    • Added - only newly added records to the current filter will be considered for the workflow
    • Updated - any update in any of the columns in the record will allow it to be enrolled for workflow processing
    • Removed - will case workflow process records that have been removed form the current filter
    • Unchanged - will enrol records that have not changed since the last run. This is useful when the workflow logic should retry applying certain changes on records
  • Name
    Run creation rule
    Description

    Decides if any given record is able to be enrolled multiple times over the workflow's lifetime

    • Always enroll - the workflow will process the record multiple times depending only on other conditions
    • Enroll once - each record will be processed by this workflow only once, regardless of the other conditions
    • If is not currently enrolled - the workflow will process the record multiple times unless the record is currently in the process of being enrolled
  • Name
    Sync now
    Description

    Runs the workflow with saved conditions immediately.

  • Name
    Reset CDC
    Description

    Resetting change data capture will cause the workflow to only consider future changes in the data model. Even if there were existing records that should match the workflow sync conditions, reset CDC will mark them as skipped. To achieve that, clicking Reset CDC will trigger a new "schedule" batch to capture a current snapshot of your model. This special batch will be marked as skipped and will not send any data to the worfklow. Thereafter, future scheduled batches will track changes normally.