Description

Design, develop and implement ingestion framework from Oracle source to Azure Data Lake - initial load and incremental ETL.

Tools

• Oracle Golden Gate (knowledge and experience are an asset but not required) for data ingestion and change data capture (currently in final stages of proof of concept)
• Azure Data Factory (expert knowledge) to maintain pipeline from Oracle to Azure Data Lake
• Azure Databricks/PySpark (good Python knowledge required) to build transformations of raw data into curated zone in the data lake
• Azure Databricks/Python/SQL (good SQL knowledge required) to develop and/or troubleshoot transformations of curated data into datamart model

General Responsibilities

• Review the requirements, database tables, and database relationships. Identify gaps and inefficiencies in current production reporting environment and provide ...