Skip to main content

Other SQLAlchemy databases

For context on getting started with ingestion, check out our metadata ingestion guide.


To install this plugin, run pip install 'acryl-datahub[sqlalchemy]'.

The sqlalchemy source is useful if we don't have a pre-built source for your chosen database system, but there is an SQLAlchemy dialect defined elsewhere. In order to use this, you must pip install the required dialect packages yourself.


This plugin extracts the following:

  • Metadata for databases, schemas, views, and tables
  • Column types associated with each table
  • Table, row, and column statistics via optional SQL profiling

Quickstart recipe#

Check out the following recipe to get started with ingestion! See below for full configuration options.

For general pointers on writing and running a recipe, see our main recipe guide.

source:  type: sqlalchemy  config:    # Coordinates    connect_uri: "dialect+driver://username:password@host:port/database"
sink:  # sink configs

Config details#

Note that a . is used to denote nested fields in the YAML recipe.

As a SQL-based service, the Athena integration is also supported by our SQL profiler. See here for more details on configuration.

platformโœ…Name of platform being ingested, used in constructing URNs.
connect_uriโœ…URI of database to connect to. See
env"PROD"Environment to use in namespace when constructing URNs.
options.<option>Any options specified here will be passed to SQLAlchemy's create_engine as kwargs.
See for details.
table_pattern.allowList of regex patterns for tables to include in ingestion.
table_pattern.denyList of regex patterns for tables to exclude from ingestion.
table_pattern.ignoreCaseTrueWhether to ignore case sensitivity during pattern matching.
schema_pattern.allowList of regex patterns for schemas to include in ingestion.
schema_pattern.denyList of regex patterns for schemas to exclude from ingestion.
schema_pattern.ignoreCaseTrueWhether to ignore case sensitivity during pattern matching.
view_pattern.allowList of regex patterns for views to include in ingestion.
view_pattern.denyList of regex patterns for views to exclude from ingestion.
view_pattern.ignoreCaseTrueWhether to ignore case sensitivity during pattern matching.
include_tablesTrueWhether tables should be ingested.
include_viewsTrueWhether views should be ingested.


Coming soon!


If you've got any questions on configuring this source, feel free to ping us on our Slack!