In this video tutorial, you will be quickly introduced to onboarding RDBMS data, whether using full or incremental ingestion. The demo uses an Oracle orders database. If you replicate this demo, you will use your own database.
Steps
Ingest Training_RDBMS data source : Setup Source
Fetch Data Using: JDBC (or other driver appropriate to your source)
Connection URL: (provide connection URL to your source)
Username: (provide username to your source)
Password: (password)
Source Schema: (provide source schema name)
Enable Log Based CDC: (unchecked)
Save & Test Connection
Save & Crawl Metadata
Ingest Training_RDBMS data source : Configure Tables
Table Name: (select small lookup table from source) (review Details > Edit Schema)
Ingest Type: Full Refresh (note as default)
(remaining settings): (review without change)
Table Name: (select large table from source) (review Details > Sample Data, Audits)
Ingest Type: Incremental
Natural Keys: (select primary key column, or set of natural keys)
Incremental Mode: Merge
Watermark Column: (select update timestamp column, or similar column)
Add Partition: (select grouping column by which future queries will be most commonly filtered)
Split By Column: (select highly cardinal column (e.g., auto-increment key) by which DF will assign worker threads during crawling and ingestion)
Gen. History View: (checked)
Table Name: (select large table from source) (review Details > Sample Data, Audits)
Ingest Type: Incremental
Natural Keys: (select primary key column, or set of natural keys)
Incremental Mode: Merge
Watermark Column: (select update timestamp column, or similar column)
Split By Column: (select highly cardinal column (e.g., auto-increment key) by which DF will assign worker threads during crawling and ingestion)