Skip to main content


Module elasticsearch


Important Capabilities

Platform InstanceEnabled by default

This plugin extracts the following:

  • Metadata for indexes
  • Column types associated with each index field

CLI based Ingestion

Install the Plugin

pip install 'acryl-datahub[elasticsearch]'

Config Details

Note that a . is used to denote nested fields in the YAML recipe.

View All Configuration Options
Field [Required]TypeDescriptionDefaultNotes
ca_certs [✅]stringPath to a certificate authority (CA) certificate.None
client_cert [✅]stringPath to the file containing the private key and the certificate, or cert only if using client_key.None
client_key [✅]stringPath to the file containing the private key if using separate cert and key files.None
host [✅]stringThe elastic search host URI.localhost:9200
ingest_index_templates [✅]booleanIngests ES index templates if enabled.None
password [✅]stringThe password credential.None
platform_instance [✅]stringThe instance of the platform that all assets produced by this recipe belong toNone
ssl_assert_fingerprint [✅]stringVerify the supplied certificate fingerprint if not None.None
ssl_assert_hostname [✅]booleanUse hostname verification if not False.None
url_prefix [✅]stringThere are cases where an enterprise would have multiple elastic search clusters. One way for them to manage is to have a single endpoint for all the elastic search clusters and use url_prefix for routing requests to different clusters.None
use_ssl [✅]booleanWhether to use SSL for the connection or not.None
username [✅]stringThe username credential.None
verify_certs [✅]booleanWhether to verify SSL certificates.None
env [✅]stringThe environment that all assets produced by this connector belong toPROD
index_pattern [✅]AllowDenyPatternregex patterns for indexes to filter in ingestion.{'allow': ['.*'], 'deny': ['^_.', '^ilm-history.'], 'ignoreCase': True}
index_pattern.allow [❓ (required if index_pattern is set)]array(string)None
index_pattern.deny [❓ (required if index_pattern is set)]array(string)None
index_pattern.ignoreCase [❓ (required if index_pattern is set)]booleanWhether to ignore case sensitivity during pattern matching.True
index_template_pattern [✅]AllowDenyPatternThe regex patterns for filtering index templates to ingest.{'allow': ['.*'], 'deny': ['^_.*'], 'ignoreCase': True}
index_template_pattern.allow [❓ (required if index_template_pattern is set)]array(string)None
index_template_pattern.deny [❓ (required if index_template_pattern is set)]array(string)None
index_template_pattern.ignoreCase [❓ (required if index_template_pattern is set)]booleanWhether to ignore case sensitivity during pattern matching.True

Code Coordinates

  • Class Name: datahub.ingestion.source.elastic_search.ElasticsearchSource
  • Browse on GitHub


If you've got any questions on configuring ingestion for Elasticsearch, feel free to ping us on our Slack