PowerBI
Module powerbi
Important Capabilities
Capability | Status | Notes |
---|---|---|
Extract Ownership | ✅ | Enabled by default |
This plugin extracts the following:
- Power BI dashboards, tiles, datasets
- Names, descriptions and URLs of dashboard and tile
- Owners of dashboards
Configuration Notes
See the
- Microsoft AD App Creation doc for the steps to create a app client ID and secret.
- Login to Power BI as Admin and from
Tenant settings
allow below permissions.- Allow service principles to use Power BI APIs
- Allow service principals to use read-only Power BI admin APIs
- Enhance admin APIs responses with detailed metadata
Install the Plugin
pip install 'acryl-datahub[powerbi]'
Quickstart Recipe
Check out the following recipe to get started with ingestion! See below for full configuration options.
For general pointers on writing and running a recipe, see our main recipe guide
source:
type: "powerbi"
config:
# Your Power BI tenant identifier
tenant_id: a949d688-67c0-4bf1-a344-e939411c6c0a
# Ingest elements of below PowerBi Workspace into Datahub
workspace_id: 4bd10256-e999-45dd-8e56-571c77153a5f
# Workspace's dataset environments (PROD, DEV, QA, STAGE)
env: DEV
# Azure AD App client identifier
client_id: foo
# Azure AD App client secret
client_secret: bar
# dataset_type_mapping is fixed mapping of Power BI datasources type to equivalent Datahub "data platform" dataset
dataset_type_mapping:
PostgreSql: postgres
Oracle: oracle
sink:
# sink configs
Config Details
- Options
- Schema
Note that a .
is used to denote nested fields in the YAML recipe.
View All Configuration Options
Field | Required | Type | Description | Default |
---|---|---|---|---|
env | string | The environment that all assets produced by this connector belong to | PROD | |
tenant_id | ✅ | string | Power BI tenant identifier. | None |
workspace_id | ✅ | string | Power BI workspace identifier. | None |
dataset_type_mapping | ✅ | Dict[str,string] | Mapping of Power BI datasource type to Datahub dataset. | |
client_id | ✅ | string | Azure AD App client identifier. | None |
client_secret | ✅ | string | Azure AD App client secret. | None |
scan_timeout | integer | time in seconds to wait for Power BI metadata scan result. | 60 | |
scope | string | https://analysis.windows.net/powerbi/api/.default | ||
base_url | string | https://api.powerbi.com/v1.0/myorg/groups | ||
admin_base_url | string | https://api.powerbi.com/v1.0/myorg/admin | ||
authority | string | https://login.microsoftonline.com/ | ||
platform_name | string | powerbi | ||
platform_urn | string | urn:li:dataPlatform:powerbi | ||
dashboard_pattern | AllowDenyPattern (see below for fields) | {'allow': ['.*'], 'deny': [], 'ignoreCase': True, 'alphabet': '[A-Za-z0-9 _.-]'} | ||
dashboard_pattern.allow | Array of string | List of regex patterns for process groups to include in ingestion | ['.*'] | |
dashboard_pattern.deny | Array of string | List of regex patterns for process groups to exclude from ingestion. | [] | |
dashboard_pattern.ignoreCase | boolean | Whether to ignore case sensitivity during pattern matching. | True | |
dashboard_pattern.alphabet | string | Allowed alphabets pattern | [A-Za-z0-9 _.-] | |
chart_pattern | AllowDenyPattern (see below for fields) | {'allow': ['.*'], 'deny': [], 'ignoreCase': True, 'alphabet': '[A-Za-z0-9 _.-]'} | ||
chart_pattern.allow | Array of string | List of regex patterns for process groups to include in ingestion | ['.*'] | |
chart_pattern.deny | Array of string | List of regex patterns for process groups to exclude from ingestion. | [] | |
chart_pattern.ignoreCase | boolean | Whether to ignore case sensitivity during pattern matching. | True | |
chart_pattern.alphabet | string | Allowed alphabets pattern | [A-Za-z0-9 _.-] |
The JSONSchema for this configuration is inlined below.
{
"title": "PowerBiDashboardSourceConfig",
"description": "Any source that produces dataset urns in a single environment should inherit this class",
"type": "object",
"properties": {
"env": {
"title": "Env",
"description": "The environment that all assets produced by this connector belong to",
"default": "PROD",
"type": "string"
},
"tenant_id": {
"title": "Tenant Id",
"description": "Power BI tenant identifier.",
"type": "string"
},
"workspace_id": {
"title": "Workspace Id",
"description": "Power BI workspace identifier.",
"type": "string"
},
"dataset_type_mapping": {
"title": "Dataset Type Mapping",
"description": "Mapping of Power BI datasource type to Datahub dataset.",
"type": "object",
"additionalProperties": {
"type": "string"
}
},
"client_id": {
"title": "Client Id",
"description": "Azure AD App client identifier.",
"type": "string"
},
"client_secret": {
"title": "Client Secret",
"description": "Azure AD App client secret.",
"type": "string"
},
"scan_timeout": {
"title": "Scan Timeout",
"description": "time in seconds to wait for Power BI metadata scan result.",
"default": 60,
"type": "integer"
},
"scope": {
"title": "Scope",
"default": "https://analysis.windows.net/powerbi/api/.default",
"type": "string"
},
"base_url": {
"title": "Base Url",
"default": "https://api.powerbi.com/v1.0/myorg/groups",
"type": "string"
},
"admin_base_url": {
"title": "Admin Base Url",
"default": "https://api.powerbi.com/v1.0/myorg/admin",
"type": "string"
},
"authority": {
"title": "Authority",
"default": "https://login.microsoftonline.com/",
"type": "string"
},
"platform_name": {
"title": "Platform Name",
"default": "powerbi",
"type": "string"
},
"platform_urn": {
"title": "Platform Urn",
"default": "urn:li:dataPlatform:powerbi",
"type": "string"
},
"dashboard_pattern": {
"title": "Dashboard Pattern",
"default": {
"allow": [
".*"
],
"deny": [],
"ignoreCase": true,
"alphabet": "[A-Za-z0-9 _.-]"
},
"allOf": [
{
"$ref": "#/definitions/AllowDenyPattern"
}
]
},
"chart_pattern": {
"title": "Chart Pattern",
"default": {
"allow": [
".*"
],
"deny": [],
"ignoreCase": true,
"alphabet": "[A-Za-z0-9 _.-]"
},
"allOf": [
{
"$ref": "#/definitions/AllowDenyPattern"
}
]
}
},
"required": [
"tenant_id",
"workspace_id",
"dataset_type_mapping",
"client_id",
"client_secret"
],
"additionalProperties": false,
"definitions": {
"AllowDenyPattern": {
"title": "AllowDenyPattern",
"description": "A class to store allow deny regexes",
"type": "object",
"properties": {
"allow": {
"title": "Allow",
"description": "List of regex patterns for process groups to include in ingestion",
"default": [
".*"
],
"type": "array",
"items": {
"type": "string"
}
},
"deny": {
"title": "Deny",
"description": "List of regex patterns for process groups to exclude from ingestion.",
"default": [],
"type": "array",
"items": {
"type": "string"
}
},
"ignoreCase": {
"title": "Ignorecase",
"description": "Whether to ignore case sensitivity during pattern matching.",
"default": true,
"type": "boolean"
},
"alphabet": {
"title": "Alphabet",
"description": "Allowed alphabets pattern",
"default": "[A-Za-z0-9 _.-]",
"type": "string"
}
},
"additionalProperties": false
}
}
}
Concept mapping
Power BI | Datahub |
---|---|
Dashboard | Dashboard |
Dataset, Datasource | Dataset |
Tile | Chart |
Report.webUrl | Chart.externalUrl |
Workspace | N/A |
Report | N/A |
If Tile is created from report then Chart.externalUrl is set to Report.webUrl.
Code Coordinates
- Class Name:
datahub.ingestion.source.powerbi.PowerBiDashboardSource
- Browse on GitHub
Questions
If you've got any questions on configuring ingestion for PowerBI, feel free to ping us on our Slack