File
Module file
This plugin pulls metadata from a previously generated file. The file sink can produce such files, and a number of samples are included in the examples/mce_files directory.
Install the Plugin
The file
source works out of the box with acryl-datahub
.
Quickstart Recipe
Check out the following recipe to get started with ingestion! See below for full configuration options.
For general pointers on writing and running a recipe, see our main recipe guide
source:
type: file
config:
# Coordinates
filename: ./path/to/mce/file.json
sink:
# sink configs
Config Details
- Options
- Schema
Note that a .
is used to denote nested fields in the YAML recipe.
View All Configuration Options
Field | Required | Type | Description | Default |
---|---|---|---|---|
filename | ✅ | string | Path to file to ingest. | None |
The JSONSchema for this configuration is inlined below.
{
"title": "FileSourceConfig",
"type": "object",
"properties": {
"filename": {
"title": "Filename",
"description": "Path to file to ingest.",
"type": "string"
}
},
"required": [
"filename"
],
"additionalProperties": false
}
Code Coordinates
- Class Name:
datahub.ingestion.source.file.GenericFileSource
- Browse on GitHub
Questions
If you've got any questions on configuring ingestion for File, feel free to ping us on our Slack