Role Summary

Lead the migration from Glue/Redshift to Databricks.

Qualifications

• 10+ years of professional data engineering experience, with at least 2+ years in a tech‑lead or lead engineer capacity.
• 5–6 years of experience in each of the following areas:
  • Deep, hands‑on experience with Apache Spark (PySpark and SparkSQL), including performance tuning (partitioning, shuffles, skew, caching, file sizing).
  • Hands‑on production experience with Databricks— DeltaLake, UnityCatalog, Workflows, and either DeltaLiveTables or a comparable declarative pipeline framework.
  • Strong production experience with AWSGlue (jobs, crawlers, DataCatalog, triggers) and AmazonRedshift (including Redshift Spectrum, workload management, distribution and sort keys, performance tuning, and RedshiftSQL/ stored procedures).
  • Proven experience leading at least one substantial data platform migration or re‑platforming effort end‑to‑end.
  ...