Understanding system of records in Cargo

Introduction

Cargo workflows are always powered by 'data models' present behind them. These models, or tables of data, contain data organized in columns which can be imported into a workflow to create runs.

To manage this data efficiently, Cargo uses a data warehouse infrastructure to host these data models. The data warehouse collects, manages, and stores data for each workspace in Cargo ensuring that data is accurately and consistently accessible for all data models present on that workspace.

These models can created on a range of external and internal data sources using data loaders from API endpoints, webhook events, or SQL queries on tables in the data warehouse. This integration facilitates a comprehensive view of your data by merging various sources into a single, cohesive model.

By allowing all these possibilities of creating models data outside of Cargo, or on other existing models, Cargo allows users to consolidate different data sources under a common schema.

Models support several advanced features, such as:

Creating segments: Apply filters to data models to form segments, allowing for targeted workflows based on specific criteria.
Change-based triggers: Enable workflows to respond in real-time to updates, additions, or deletions in the data, ensuring action based on the most pertient signals.
Avoiding duplicate runs: Track records to prevent workflows from processing the same data multiple times.
Model relationships: Allow models to connect to each other using common identifiers, providing a unified view.
Custom columns: Use dynamic functions to create new fields that transform and enrich the data, adding calculated values or tags that enhance it.

This framework ensures that Cargo users can effectively manage their data, offering a dynamic set of options for feeding data into workflows.

Data Connectors

Data loaders

Among all of the integrations available in Cargo, those available as storage connectors allow the user to bring external data to a data model using a data loader. This capability enables you to integrate data from various sources, such as CRMs, marketing automation platforms, and e-commerce databases, into a single, unified model.

In Go-To-Market (GTM) functions, where fragmented data about prospects, customers, and their behaviors is often a limit on effective workflow creation, this is a game-changing feature. See the section on integration for more information on how this works.

There are three types of data loaders available in Cargo:

API data loader: This loader allows you to connect to an external API endpoint to fetch data and import it into a model.
Webhook data loader: This loader allows you to listen for events from an external source and import the data into a model.
SQL data loader: This loader allows you to run SQL queries on tables in the data warehouse and import the results into a model.

As webhook-based data loaders create non-standard data structure, they have their own set of rules. For instance, once a webhook-based model is created, its columns are strongly typed. For instance, if column A was configured as an integer, it cannot be changed to a string for any subsequent records.

Relationships

Model relationships

Relationships between models in Cargo are useful for creating a comprehensive view of your data. By connecting different models using common identifiers, you can create merged views of your customer data from different sources.

Defining relationships involves specifying how records in one model relate to records in another model. This includes determining the type of relationship (such as one-to-one, one-to-many, or many-to-many), and mapping the identifier columns between the models. Relating models to one another can provide richer context for decision-making and workflow triggers.

For instance, consider the example of connecting Salesforce data with Stripe invoice data. By linking the 'SalesforceAccounts' model with the 'StripeInvoices' model using a common identifier such as 'CustomerID', you can create a relationship that combines CRM data with financial data. This integrated view allows you to see how customer interactions in Salesforce relate to their invoicing history in Stripe.

Filters and triggers

Filters and change-based triggers

By applying filters to data models, users can create focused subsets of data within the larger model. For example, a filter could be created to narrow down prospects who were participants in a webinar event.

Filters can also be applied to specific activities that records have undertaken within a workflow or segment, such as whether a record has entered a workflow or segment.

Enabling a workflow allows it to respond to real-time changes in the underlying model's data. When an appropriate change is detected, the workflow can import records from that model to create runs. A workflow will periodically resync for changes according to a schedule or a 'cron job' (except for webhook-based models, which resync automatically when a new record is added). An appropriate change is triggered whenever a record's data matches the criteria set in the filter, whether due to a record's update, addition, or removal.

Importing records into workflows based on these changes allows users to build workflows that operate autonomously. For example, in a model containing customer data, records can be imported into a workflow whenever a new customer is added or when existing customer data is updated.

Another advantage of this system is the ability to track records that enter a workflow. By maintaining a history of imported records, Cargo prevents workflows from creating multiple runs for the same record.

Search and upsert in models

Search and upsert model data

Within workflows, users have the ability to query data from any model in Cargo, ensuring that the data is accessible and actionable when needed.

This capability allows for one-off data retrieval operations without needing to create any merged views.

In a similar fashion, upserts to a model enable users to to push data to custom columns for existing records for transactional data, as it allows for synchronization of data across different systems.

Custom columns

Cargo allows users to create custom columns. These columns let you transform data within your models by adding calculated fields or tags, enhancing the utility of your data. For example, you can categorize customers based on their purchase frequency or calculate their lifetime value.

You can use this to apply dynamic functions to your data models, creating new fields that aren't part of the original dataset. These columns can be used to add calculated values or to append tags that categorize data dynamically based on specific criteria. For instance, you might create a custom column that categorizes customers based on their purchase frequency.

There are three types of custom columns that you can create in Cargo:

Custom columns: These columns are free-form columns that you can upsert data into from within a workflow. For example, you could create a custom column
Computed columns: These columns are created using storage that calculate values based on other columns in the model. For example, you could create a computed column that classifies customers based on their job title and years of experience whether they are senior or not
Metrics columns: These columns are used to store aggregated data, such as sums, averages, or counts, based on the values in other columns in other models. For example, you could create a metrics column inside the Salesforce data model that calculates the total revenue generated by each customer from the Stripe data model.