Tableau
Module tableau
Important Capabilities
Capability | Status | Notes |
---|---|---|
Data Profiling | ❌ | |
Dataset Usage | ❌ | |
Descriptions | ✅ | Enabled by default |
Detect Deleted Entities | ❌ | |
Domains | ❌ | Requires transformer |
Extract Ownership | ✅ | Requires recipe configuration |
Extract Tags | ✅ | Requires recipe configuration |
Partition Support | ❌ | Not applicable to source |
Platform Instance | ❌ | Not applicable to source |
Table-Level Lineage | ✅ | Enabled by default |
Install the Plugin
pip install 'acryl-datahub[tableau]'
Quickstart Recipe
Check out the following recipe to get started with ingestion! See below for full configuration options.
For general pointers on writing and running a recipe, see our main recipe guide
source:
type: tableau
config:
# Coordinates
connect_uri: https://prod-ca-a.online.tableau.com
site: acryl
projects: ["default", "Project 2"]
# Credentials
username: "${TABLEAU_USER}"
password: "${TABLEAU_PASSWORD}"
# Options
ingest_tags: True
ingest_owner: True
default_schema_map:
mydatabase: public
anotherdatabase: anotherschema
sink:
# sink configs
Config Details
- Options
- Schema
Note that a .
is used to denote nested fields in the YAML recipe.
View All Configuration Options
Field | Required | Type | Description | Default |
---|---|---|---|---|
connect_uri | ✅ | string | Tableau host URL. | None |
username | string | Tableau username, must be set if authenticating using username/password. | None | |
password | string | Tableau password, must be set if authenticating using username/password. | None | |
token_name | string | Tableau token name, must be set if authenticating using a personal access token. | None | |
token_value | string | Tableau token value, must be set if authenticating using a personal access token. | None | |
site | string | Tableau Site. Always required for Tableau Online. Use emptystring to connect with Default site on Tableau Server. | ||
projects | Array of string | List of projects | ['default'] | |
default_schema_map | Dict | Default schema to use when schema is not found. | {} | |
ingest_tags | boolean | Ingest Tags from source. This will override Tags entered from UI | False | |
ingest_owner | boolean | Ingest Owner from source. This will override Owner info entered from UI | False | |
ingest_tables_external | boolean | Ingest details for tables external to (not embedded in) tableau as entities. | False | |
workbooks_page_size | integer | @deprecated(use page_size instead) Number of workbooks to query at a time using Tableau api. | None | |
page_size | integer | Number of metadata objects (e.g. CustomSQLTable, PublishedDatasource, etc) to query at a time using Tableau api. | 10 | |
env | string | Environment to use in namespace when constructing URNs. | PROD |
The JSONSchema for this configuration is inlined below.
{
"title": "TableauConfig",
"type": "object",
"properties": {
"connect_uri": {
"title": "Connect Uri",
"description": "Tableau host URL.",
"type": "string"
},
"username": {
"title": "Username",
"description": "Tableau username, must be set if authenticating using username/password.",
"type": "string"
},
"password": {
"title": "Password",
"description": "Tableau password, must be set if authenticating using username/password.",
"type": "string"
},
"token_name": {
"title": "Token Name",
"description": "Tableau token name, must be set if authenticating using a personal access token.",
"type": "string"
},
"token_value": {
"title": "Token Value",
"description": "Tableau token value, must be set if authenticating using a personal access token.",
"type": "string"
},
"site": {
"title": "Site",
"description": "Tableau Site. Always required for Tableau Online. Use emptystring to connect with Default site on Tableau Server.",
"default": "",
"type": "string"
},
"projects": {
"title": "Projects",
"description": "List of projects",
"default": [
"default"
],
"type": "array",
"items": {
"type": "string"
}
},
"default_schema_map": {
"title": "Default Schema Map",
"description": "Default schema to use when schema is not found.",
"default": {},
"type": "object"
},
"ingest_tags": {
"title": "Ingest Tags",
"description": "Ingest Tags from source. This will override Tags entered from UI",
"default": false,
"type": "boolean"
},
"ingest_owner": {
"title": "Ingest Owner",
"description": "Ingest Owner from source. This will override Owner info entered from UI",
"default": false,
"type": "boolean"
},
"ingest_tables_external": {
"title": "Ingest Tables External",
"description": "Ingest details for tables external to (not embedded in) tableau as entities.",
"default": false,
"type": "boolean"
},
"workbooks_page_size": {
"title": "Workbooks Page Size",
"description": "@deprecated(use page_size instead) Number of workbooks to query at a time using Tableau api.",
"type": "integer"
},
"page_size": {
"title": "Page Size",
"description": "Number of metadata objects (e.g. CustomSQLTable, PublishedDatasource, etc) to query at a time using Tableau api.",
"default": 10,
"type": "integer"
},
"env": {
"title": "Env",
"description": "Environment to use in namespace when constructing URNs.",
"default": "PROD",
"type": "string"
}
},
"required": [
"connect_uri"
],
"additionalProperties": false
}
Prerequisites
In order to ingest metadata from tableau, you will need:
- Python 3.6+
- Tableau Server Version 2021.1.10 and above. It may also work for older versions.
- Enable the Tableau Metadata API for Tableau Server, if its not already enabled.
- Tableau Credentials (Username/Password or Personal Access Token)
Integration Details
This plugin extracts Sheets, Dashboards, Embedded and Published Data sources metadata within Workbooks in a given project
on a Tableau site. This plugin is in beta and has only been tested on PostgreSQL database and sample workbooks
on Tableau online. Tableau's GraphQL interface is used to extract metadata information. Queries used to extract metadata are located
in metadata-ingestion/src/datahub/ingestion/source/tableau_common.py
Concept Mapping
This ingestion source maps the following Source System Concepts to DataHub Concepts:
Source Concept | DataHub Concept | Notes |
---|---|---|
"Tableau" | Data Platform | |
Embedded DataSource | Dataset | SubType "Embedded Data Source" |
Published DataSource | Dataset | SubType "Published Data Source" |
Custom SQL Table | Dataset | SubTypes "View" , "Custom SQL" |
Embedded or External Tables | Dataset | |
Sheet | Chart | |
Dashboard | Dashboard | |
User | User (a.k.a CorpUser) | |
Workbook | Container | SubType "Workbook" |
Tag | Tag |
Workbook
Workbooks from Tableau are ingested as Container in datahub.
- GraphQL query
{
workbooksConnection(first: 10, offset: 0, filter: {projectNameWithin: ["default", "Project 2"]}) {
nodes {
id
name
luid
uri
projectName
owner {
username
}
description
uri
createdAt
updatedAt
}
pageInfo {
hasNextPage
endCursor
}
totalCount
}
}
Dashboard
Dashboards from Tableau are ingested as Dashboard in datahub.
- GraphQL query
{
workbooksConnection(first: 10, offset: 0, filter: {projectNameWithin: ["default", "Project 2"]}) {
nodes {
.....
dashboards {
id
name
path
createdAt
updatedAt
sheets {
id
name
}
}
}
pageInfo {
hasNextPage
endCursor
}
totalCount
}
}
Sheet
Sheets from Tableau are ingested as charts in datahub.
- GraphQL query
{
workbooksConnection(first: 10, offset: 0, filter: {projectNameWithin: ["default"]}) {
.....
sheets {
id
name
path
createdAt
updatedAt
tags {
name
}
containedInDashboards {
name
path
}
upstreamDatasources {
id
name
}
datasourceFields {
__typename
id
name
description
upstreamColumns {
name
}
... on ColumnField {
dataCategory
role
dataType
aggregation
}
... on CalculatedField {
role
dataType
aggregation
formula
}
... on GroupField {
role
dataType
}
... on DatasourceField {
remoteField {
__typename
id
name
description
folderName
... on ColumnField {
dataCategory
role
dataType
aggregation
}
... on CalculatedField {
role
dataType
aggregation
formula
}
... on GroupField {
role
dataType
}
}
}
}
}
}
.....
}
}
Embedded Data Source
Embedded Data source from Tableau is ingested as a Dataset in datahub.
- GraphQL query
{
workbooksConnection(first: 10, offset: 0, filter: {projectNameWithin: ["default"]}) {
nodes {
....
embeddedDatasources {
__typename
id
name
hasExtracts
extractLastRefreshTime
extractLastIncrementalUpdateTime
extractLastUpdateTime
upstreamDatabases {
id
name
connectionType
isEmbedded
}
upstreamTables {
name
schema
columns {
name
remoteType
}
}
fields {
__typename
id
name
description
isHidden
folderName
... on ColumnField {
dataCategory
role
dataType
defaultFormat
aggregation
columns {
table {
... on CustomSQLTable {
id
name
}
}
}
}
... on CalculatedField {
role
dataType
defaultFormat
aggregation
formula
}
... on GroupField {
role
dataType
}
}
upstreamDatasources {
id
name
}
workbook {
name
projectName
}
}
}
....
}
}
Published Data Source
Published Data source from Tableau is ingested as a Dataset in datahub.
- GraphQL query
{
publishedDatasourcesConnection(first: 10, offset: 0, filter: {idWithin: ["00cce29f-b561-bb41-3557-8e19660bb5dd", "618c87db-5959-338b-bcc7-6f5f4cc0b6c6"]}) {
nodes {
__typename
id
name
hasExtracts
extractLastRefreshTime
extractLastIncrementalUpdateTime
extractLastUpdateTime
downstreamSheets {
id
name
}
upstreamTables {
name
schema
fullName
connectionType
description
contact {
name
}
}
fields {
__typename
id
name
description
isHidden
folderName
... on ColumnField {
dataCategory
role
dataType
defaultFormat
aggregation
columns {
table {
... on CustomSQLTable {
id
name
}
}
}
}
... on CalculatedField {
role
dataType
defaultFormat
aggregation
formula
}
... on GroupField {
role
dataType
}
}
owner {
username
}
description
uri
projectName
}
pageInfo {
hasNextPage
endCursor
}
totalCount
}
}
Custom SQL Data Source
For custom sql data sources, the query is viewable in UI under View Definition tab.
- GraphQL query
{
customSQLTablesConnection(first: 10, offset: 0, filter: {idWithin: ["22b0b4c3-6b85-713d-a161-5a87fdd78f40"]}) {
nodes {
id
name
query
columns {
id
name
remoteType
description
referencedByFields {
datasource {
id
name
upstreamDatabases {
id
name
}
upstreamTables {
id
name
schema
connectionType
columns {
id
}
}
... on PublishedDatasource {
projectName
}
... on EmbeddedDatasource {
workbook {
name
projectName
}
}
}
}
}
tables {
id
name
schema
connectionType
}
}
}
}
Lineage
Lineage is emitted as received from Tableau's metadata API for
- Sheets contained in Dashboard
- Embedded or Published datasources upstream to Sheet
- Published datasources upstream to Embedded datasource
- Tables upstream to Embedded or Published datasource
- Custom SQL datasources upstream to Embedded or Published datasource
- Tables upstream to Custom SQL datasource
Caveats
- Tableau metadata API might return incorrect schema name for tables for some databases, leading to incorrect metadata in DataHub. This source attempts to extract correct schema from databaseTable's fully qualified name, wherever possible. Read Using the databaseTable object in query for caveats in using schema attribute.
Troubleshooting
Why are only some workbooks/custom SQLs/published datasources ingested from the specified project?
This may happen when the Tableau API returns NODE_LIMIT_EXCEEDED error in response to metadata query and returns partial results with message "Showing partial results. , The request exceeded the ‘n’ node limit. Use pagination, additional filtering, or both in the query to adjust results." To resolve this, consider
- reducing the page size using the
page_size
config param in datahub recipe (Defaults to 10). - increasing tableau configuration metadata query node limit to higher value.
Code Coordinates
- Class Name:
datahub.ingestion.source.tableau.TableauSource
- Browse on GitHub
Questions
If you've got any questions on configuring ingestion for Tableau, feel free to ping us on our Slack