Orchestration Best Practices

Cargo's modular structure provides a versatile and powerful platform for building workflows, similar to assembling LEGO pieces. This flexibility enables users to create a broad range of processes, but it also introduces the risk of bugs and building workflows that are difficult to maintain.

This explanation seeks to impart some best practices that will assist the user in minimizing errors and building workflows that are scalable and easier to operate.

Workflow conception

Workflows need to be unidirectional in their logic. This means that each node in the workflow only moves forward, building upon the data processed by the nodes that came before it.

When planning your workflow, start by mapping out the entire process from beginning to end. Place the starting and ending nodes first, using placeholder values, then incrementally introduce and test additional components to the execution flow.

Testing is crucial. After adding a new node, immediately test it with a few records. Utilize autocomplete features to avoid errors in data mapping. Create observability by organizing your data being mapped across different nodes by declaring explicit variables. Maximize abstraction in the workflow by consolidating repetitive actions at the top of the workflow, so that subsequent nodes can call upon that logic. Use clear and meaningful names for nodes and segments to maintain readability.

Node naming

When incorporating a node from the node catalog into your workflow, it's initially assigned a unique, albeit generic, name. This name typically includes a reference to both the connector name and the action type it performs. However, for the sake of clarity and efficiency within the workflow, renaming these nodes to more accurately reflect their specific function is not just recommended—it's considered a best practice. You can also use the node description to provide better context about the node's function.

For example, consider a scenario where your workflow includes two read nodes from Apollo. The default naming convention might label them as apolloio_read, which doesn't offer much insight into their distinct roles within the workflow. If one node is tasked with searching through the accounts objects and the other through contacts objects, renaming them to apollo_accounts and apollo_contacts respectively clarifies their functions at a glance. This hugely simplifies future modifications or troubleshooting as the workflows become bigger and more complex.

Moreover, if there are multiple apollo nodes being used, then use the node description to specify the input and desired output, e.g. retrieving marketing executives belonging to the domain

Building in phases

Effective Cargo users are adept at both process design and issue identification. The best practices include:

• Mocking the workflow logic: Beginning with a minimum version of the workflow. This approach allows for early identification of core business logic flaws.

• Adding complexity in layers: Gradually add nodes to the workflow and test their execution individually, reviewing the logic or data mappings within for errors or inconsistencies.

Testing in phases

Cargo's deployment process reflects common software development practices:

1. Node testing: Create and test the workflow using a sample record from the start node. The start node is present in every workflow to encourage users to design processes in their simplest form, using a small set of test values.

2. Workflow testing: Test the initial release by importing a batch of records from the data model. The Runs panel in the workflow editor includes an option to Import records meeting a filter condition to allow testing for edge cases across a smaller sample on the latest published version. This facilitates efficient testing at a small scale but is only available once a version is published.

3. Trigger testing: Trigger-based enrolment stage, automatically enrol records from the segment whenever the trigger conditions are met. Once a workflow is published and enabled and linked to a segment, automatic enrolment will take place.