Data Module

This section provides detailed instructions and insights on managing data within the Cargo workspace through data models and segments.

MODEL

Managing Data Models in Storage

Create and manage data models to structure and store data effectively. A data model represents a type of object or database table, where individual fields are defined as columns.

Cover image
  • Name
    Create a connector
    Description

    Integrate data loaders to bring external data into the workspace. Connectors allow importing data from external APIs, connected data warehouses, or through generic file uploads and HTTP endpoints. You can create multiple instances of the same connector and define multiple data models for each connector.

  • Name
    Add model
    Description

    Define a new data model based on the connector's capabilities. For external APIs, the model maps to an object type available in the source system (e.g., contacts or deals). Generic connectors like SQL, HTTP, or File uploads allow more customization of data models.

    Options for API connectors:

    • Name - human readable name of the data model
    • Slug - unique key of the data model
    • Object type - list of built-in data types provided by the connector
    • Column selection mode - you can decide to bring all properties or select a subset
  • Name
    Edit model
    Description

    Modify a model's settings to update its name or configuration based on the connector's capabilities.

  • Name
    Refresh model
    Description

    Trigger a manual refresh of your model. By default, your model is refreshed daily or according to your play sync schedule.

    • Name
      Full refresh
      Description

      This model uses incremental sync, meaning it refreshes data from the last sync point. Enable this option if you prefer to perform a full refresh.

  • Name
    Preview Button
    Description

    Review a model's data output before finalizing its creation or changes.

  • Name
    Columns
    Description

    Edit data model column configurations. Columns represent properties of records in a data model. Columns can be mapped from a connected service (e.g., CRM) or created within Cargo.

    Once created, type of a column cannot be changed at all. If a change is required a new column or data model needs to be created.
    • Name
      Create a Computed Column
      Description

      Define and compute values dynamically from existing columns in the model. Computed values can be of the following data types:

      • String
      • Number
      • Boolean (yes or no)
      • Date
      • Array
      • Object
      • Any

      Each computed value is evaluated using a provided expression. This expression is javascript code that can reference other columns in the data model to produce the output.

      Example: concatenate first and last name of a record

      {{ record.firstName }} {{ record.lastName }}
    • Name
      Create a Custom Column
      Description

      Add specialized data fields to enhance the model's data structure. Custom columns create dedicated properties that store additional data for a model. These columns can be updated using the model's custom columns node in any workflow.

      Internally those columns are prefixed with custom__ (notice double underscore). For instance a custom column with slug name will be accessible in storage nodes as custom__name. That allows to create a custom column of the same name as the built-in column.

      Example: adding new tag column that can accept extra information about the record that can be written in a workflow

    • Name
      Create a Metrics Column
      Description

      Establish columns that calculate and display aggregated data metrics. It means running calculations such as sum or average across related data models. For instance we can define metric columns on company level to surface key information about employees. Such as number of employees, average tenure etc.

      Example: Leads count metric that counts all leads from a single company, so that information can be used for scoring.

DATA TYPES

Data type

Cargo's storage system works with 6 data types modeled after JavaScript types. For API connectors, the data type will be automatically derived from API information.

When designing your data models, carefully consider which type best suits your data before creation. If you need to change a column's type later, you'll need to create a new column with the correct type and migrate your data.

For nested columns such as arrays and objects, only first-level parameters are type-checked. This means that a column of type object can contain any JavaScript object regardless of its internal structure.
  • Name
    String
    Description

    String columns can contain any Unicode text. For example, job titles or descriptions.

    Best for: Text-based data of any length, IDs that include non-numeric characters, or data that might begin with numbers but isn't numeric in nature (like postal codes).

  • Name
    Number
    Description

    Number columns can contain any numeric value, whether an integer (e.g., 1, 9) or decimal (e.g., 3.14, 9.99).

    Best for: Quantities, measurements, calculations, and statistical data.

  • Name
    Boolean
    Description

    Boolean columns can contain true or false values only.

    Best for: Binary states, flags, or yes/no conditions. In filters and expressions, Booleans can simplify complex logic and improve workflow readability.

  • Name
    Date
    Description

    Date columns contain full date, time, and timezone information.

    Best for: Timestamps, event scheduling, time-based triggers, and any temporal data.

  • Name
    Array
    Description

    Array columns contain elements of any type. For example: [1, "abc", 3.14].

    Best for: Lists of related items, multiple selections, or collections that need to be processed as a group.

  • Name
    Object
    Description

    Object columns can store any nested JavaScript object. The schema of nested parameters is not validated, which means this column type can accept different object structures.

    Best for: Complex, structured data that has multiple properties or when working with JSON responses from external systems.

NODES

Interacting with storage using nodes

Store or access structured data.

Cover image
  • Name
    Memory
    Description

    Use storage to save, modify and retrieve any data as in a key/value store. Options:

    • Scope: whether the key/value should be stored on workspace or workflow level
    • Key: the unique indentifier of the stored value

    Actions:

    • Get - read value from the storage
    • Set - write value to the storage
    • Get or set - read the value from the storage if available if not set the provided value for the future use
    • Increment - increase value of a counter
    • Decrement - decrease value of a counter
    • Remove - remove key and its value
  • Name
    Model custom column
    Description

    Update custom columns on a selected data model. This node allow writing data to custom columns created on a particular data model. It allows extending the data model that is synchronized from an external system with additional data points. For instance we may want to add extra metadata about particular records or flag them.

    Options:

    • Model: data model to work with
    • Action: upsert which means record will be created when not found
    • Record ID: primary identifier of the data model record to update
    • Mappings: list of pairs of column name and value (e.g. indstry: automotive)

    The ID required by this node needs to be the primary identifier of the data model. When the workflow is using the same data model, the record ID can be set to nodes.start._id. In case no primary ID is available in the workflow, consider using the model search node.

  • Name
    Model record
    Description

    Add record to the compatible data model. Some data model, such as HTTP, allow creating new records from within Cargo's workflows. It can be useful to store new data and trigger other workflows.

    Data models coming from external serivces such as CRM doesn't support writing records using this node. In order to create new objects use dedicated write node for the service in question (e.g. HubSpot write node).

    Options:

    • Model: data model to add a new record to (must be a compatible data record)
    • Action: insert
    • Data: JSON object with data for the new record
    {{
JSON.parse(`{
  "_id": "${nodes.start._id}",
  "email": "${nodes.start.email}",
  "is_business_email": true
}`)
}}

    Ensure all parameters are carefully passed with correct data types. If a new column is created using this node, the first value will determine the data type, which cannot be changed later.

  • Name
    Model search
    Description

    Search and retrieve model or dataset records.

    Options:

    • Model: data model to work with
    • Filter: filter to match records to return
    • Sort: define how records will be sorted
    • Limit: number of the records to return, default: 1