Skip to main content
Version: Next

Mode

Certified

Important Capabilities

CapabilityStatusNotes
Asset ContainersEnabled by default
Column-level LineageSupported by default
DescriptionsEnabled by default
Detect Deleted EntitiesOptionally enabled via stateful_ingestion.remove_stale_metadata
Extract OwnershipEnabled by default
Platform InstanceEnabled by default
Table-Level LineageSupported by default

This plugin extracts Charts, Reports, and associated metadata from a given Mode workspace. This plugin is in beta and has only been tested on PostgreSQL database.

Report

/api/{account}/reports/{report} endpoint is used to retrieve the following report information.

  • Title and description
  • Last edited by
  • Owner
  • Link to the Report in Mode for exploration
  • Associated charts within the report

Chart

/api/{workspace}/reports/{report}/queries/{query}/charts' endpoint is used to retrieve the following information.

  • Title and description
  • Last edited by
  • Owner
  • Link to the chart in Metabase
  • Datasource and lineage information from Report queries.

The following properties for a chart are ingested in DataHub.

Chart Information

NameDescription
FiltersFilters applied to the chart
MetricsFields or columns used for aggregation
XFields used in X-axis
X2Fields used in second X-axis
YFields used in Y-axis
Y2Fields used in second Y-axis

Table Information

NameDescription
ColumnsColumn names in a table
FiltersFilters applied to the table

Pivot Table Information

NameDescription
ColumnsColumn names in a table
FiltersFilters applied to the table
MetricsFields or columns used for aggregation
RowsRow names in a table

CLI based Ingestion

Install the Plugin

The mode source works out of the box with acryl-datahub.

Starter Recipe

Check out the following recipe to get started with ingestion! See below for full configuration options.

For general pointers on writing and running a recipe, see our main recipe guide.

source:
type: mode
config:
# Coordinates
connect_uri: http://app.mode.com

# Credentials
token: token
password: pass

# Options
workspace: "datahub"
default_schema: "public"
owner_username_instead_of_email: False
api_options:
retry_backoff_multiplier: 2
max_retry_interval: 10
max_attempts: 5

sink:
# sink configs

Config Details

Note that a . is used to denote nested fields in the YAML recipe.

FieldDescription
password 
string(password)
When creating workspace API key this is the 'Secret'.
token 
string
When creating workspace API key this is the 'Key ID'.
workspace 
string
The Mode workspace name. Find it in Settings > Workspace > Details.
connect_uri
string
Mode host URL.
default_schema
string
Default schema to use when schema is not provided in an SQL query
Default: public
exclude_restricted
boolean
Exclude restricted collections
Default: False
ingest_embed_url
boolean
Whether to Ingest embed URL for Reports
Default: True
owner_username_instead_of_email
boolean
Use username for owner URN instead of Email
Default: True
platform_instance_map
map(str,string)
tag_measures_and_dimensions
boolean
Tag measures and dimensions in the schema
Default: True
env
string
The environment that all assets produced by this connector belong to
Default: PROD
api_options
ModeAPIConfig
Retry/Wait settings for Mode API to avoid "Too many Requests" error. See Mode API Options below
Default: {'retry_backoff_multiplier': 2, 'max_retry_interva...
api_options.max_attempts
integer
Maximum number of attempts to retry before failing
Default: 5
api_options.max_retry_interval
One of integer, number
Maximum interval to wait when retrying
Default: 10
api_options.retry_backoff_multiplier
One of integer, number
Multiplier for exponential backoff when waiting to retry
Default: 2
space_pattern
AllowDenyPattern
Regex patterns for mode spaces to filter in ingestion (Spaces named as 'Personal' are filtered by default.) Specify regex to only match the space name. e.g. to only ingest space named analytics, use the regex 'analytics'
Default: {'allow': ['.*'], 'deny': ['^Personal$'], 'ignoreC...
space_pattern.ignoreCase
boolean
Whether to ignore case sensitivity during pattern matching.
Default: True
space_pattern.allow
array
List of regex patterns to include in ingestion
Default: ['.*']
space_pattern.allow.string
string
space_pattern.deny
array
List of regex patterns to exclude from ingestion.
Default: []
space_pattern.deny.string
string
stateful_ingestion
StatefulStaleMetadataRemovalConfig
Base specialized config for Stateful Ingestion with stale metadata removal capability.
stateful_ingestion.enabled
boolean
Whether or not to enable stateful ingest. Default: True if a pipeline_name is set and either a datahub-rest sink or datahub_api is specified, otherwise False
Default: False
stateful_ingestion.remove_stale_metadata
boolean
Soft-deletes the entities present in the last successful run but missing in the current run with stateful_ingestion enabled.
Default: True

See Mode's Authentication documentation on how to generate token and password.

Code Coordinates

  • Class Name: datahub.ingestion.source.mode.ModeSource
  • Browse on GitHub

Questions

If you've got any questions on configuring ingestion for Mode, feel free to ping us on our Slack.