Data Module

This section provides detailed instructions and insights on managing data within the Cargo workspace through data models and segments.

MODEL

Managing Data Models in Storage

Create and manage data models to structure and store data effectively. A data model represents a type of object or database table, where individual fields are defined as columns.

Cover image
  • Name
    Create a connector
    Description

    Integrate data loaders to bring external data into the workspace. Connectors allow importing data from external APIs, connected data warehouses, or through generic file uploads and HTTP endpoints. You can create multiple instances of the same connector and define multiple data models for each connector.

  • Name
    Add model
    Description

    Define a new data model based on the connector's capabilities. For external APIs, the model maps to an object type available in the source system (e.g., contacts or deals). Generic connectors like SQL, HTTP, or File uploads allow more customization of data models.

    Options for API connectors:

    • Name - human readable name of the data model
    • Slug - unique key of the data model
    • Object type - list of built-in data types provided by the connector
    • Column selection mode - you can decide to bring all properties or select a subset
  • Name
    Edit model
    Description

    Modify a model's settings to update its name or configuration based on the connector's capabilities.

  • Name
    Refresh model
    Description

    Trigger a manual refresh of your model. By default, your model is refreshed daily or according to your play sync schedule.

    • Name
      Full refresh
      Description

      This model uses incremental sync, meaning it refreshes data from the last sync point. Enable this option if you prefer to perform a full refresh.

  • Name
    Preview Button
    Description

    Review a model's data output before finalizing its creation or changes.

  • Name
    Columns
    Description

    Edit data model column configurations. Columns represent properties of records in a data model. Columns can be mapped from a connected service (e.g., CRM) or created within Cargo.

    • Name
      Create a Computed Column
      Description

      Define and compute values dynamically from existing columns in the model. Computed values can be of the following data types:

      • String
      • Number
      • Boolean (yes or no)
      • Date
      • Array
      • Object
      • Any

      Each computed value is evaluated using a provided expression. This expression is javascript code that can reference other columns in the data model to produce the output.

      Example: concatenate first and last name of a record

      {{ record.firstName }} {{ record.lastName }}
    • Name
      Create a Custom Column
      Description

      Add specialized data fields to enhance the model's data structure. Custom columns create dedicated properties that store additional data for a model. These columns can be updated using the model's custom columns node in any workflow.

      Example: adding new tag column that can accept extra information about the record that can be written in a workflow

    • Name
      Create a Metrics Column
      Description

      Establish columns that calculate and display aggregated data metrics. It means running calculations such as sum or average across related data models. For instance we can define metric columns on company level to surface key information about employees. Such as number of employees, average tenure etc.

      Example: Leads count metric that counts all leads from a single company, so that information can be used for scoring.

NODES

Interacting with storage using nodes

Store or access structured data.

Cover image
  • Name
    Memory
    Description

    Use storage to save, modify and retrieve any data as in a key/value store. Options:

    • Scope: whether the key/value should be stored on workspace or workflow level
    • Key: the unique indentifier of the stored value

    Actions:

    • Get - read value from the storage
    • Set - write value to the storage
    • Get or set - read the value from the storage if available if not set the provided value for the future use
    • Increment - increase value of a counter
    • Decrement - decrease value of a counter
    • Remove - remove key and its value
  • Name
    Model custom column
    Description

    Update custom columns on a selected data model. This node allow writing data to custom columns created on a particular data model. It allows extending the data model that is synchronized from an external system with additional data points. For instance we may want to add extra metadata about particular records or flag them.

    Options:

    • Model: data model to work with
    • Action: upsert
    • ID: unique identifier of the data model record to update
    • Mappings: list of pairs of column name and value (e.g. indstry: automotive)
  • Name
    Model record
    Description

    Add record to the compatible data model. Some data model, such as HTTP, allow creating new records from within Cargo's workflows. It can be useful to store new data and trigger other workflows.

    Data models coming from external serivces such as CRM doesn't support writing records using this node. In order to create new objects use dedicated write node for the service in question (e.g. HubSpot write node).

    Options:

    • Model: data model to add a new record to (must be a compatible data record)
    • Action: insert
    • Data: JSON object with data for the new record
    {{
JSON.parse(`{
  "_id": "${nodes.start._id}",
  "email": "${nodes.start.email}",
  "is_business_email": "TRUE",
}`)
}}
  • Name
    Model search
    Description

    Search and retrieve model or dataset records.

    Options:

    • Model: data model to work with
    • Filter: filter to match records to return
    • Sort: define how records will be sorted
    • Limit: number of the records to return, default: 1