Onboard RDBMS data

In this video tutorial, you will be quickly introduced to onboarding RDBMS data, whether using full or incremental ingestion. The demo uses an Oracle orders database. If you replicate this demo, you will use your own database.

Steps

1. Ingest Training_RDBMS data source : Setup Source
   • Fetch Data Using: JDBC (or other driver appropriate to your source)
   • Connection URL: (provide connection URL to your source)
   • Username: (provide username to your source)
   • Password: (password)
   • Source Schema: (provide source schema name)
   • Enable Log Based CDC: (unchecked)
   • Save & Test Connection
   • Save & Crawl Metadata
2. Ingest Training_RDBMS data source : Configure Tables
   • Table Name: (select small lookup table from source) (review Details > Edit Schema)
     • Ingest Type: Full Refresh (note as default)
     • (remaining settings): (review without change)
   • Table Name: (select large table from source) (review Details > Sample Data, Audits)
     • Ingest Type: Incremental
     • Natural Keys: (select primary key column, or set of natural keys)
     • Incremental Mode: Merge
     • Watermark Column: (select update timestamp column, or similar column)
     • Add Partition: (select grouping column by which future queries will be most commonly filtered)
     • Split By Column: (select highly cardinal column (e.g., auto-increment key) by which DF will assign worker threads during crawling and ingestion)
     • Gen. History View: (checked)
   • Table Name: (select large table from source) (review Details > Sample Data, Audits)
     • Ingest Type: Incremental
     • Natural Keys: (select primary key column, or set of natural keys)
     • Incremental Mode: Merge
     • Watermark Column: (select update timestamp column, or similar column)
     • Split By Column: (select highly cardinal column (e.g., auto-increment key) by which DF will assign worker threads during crawling and ingestion)
     • Gen. History View: (checked)
3. Ingest Data > (Select and ingest all tables)
   • Job Name: All Tables
   • (remaining settings): (review without change)
   • Save as Table Group: (checked)
   • Ingest
   • Click “here” to track progress
   • Click “Tables” to review results
4. Assign metadata to Training_RDBMS source
   • Favorite: (checked)
   • Description: (assign relevant description)
   • Tags: (assign relevant tags)
5. Assign metadata to Training_RDBMS > Orders
   • Favorite: (checked)
   • Description: (assign relevant description)
   • Tags: (assign relevant tags)