About the Role

We are looking for a talented and motivated Data Engineer to join our growing engineering team at Will Sparrow Technologies. In this role, you will design, build, and maintain scalable data pipelines and infrastructure that power analytics and machine learning workloads. You will collaborate closely with data analysts, data scientists, and software engineers to ensure reliable, high-quality data delivery across the organisation.

Key Responsibilities

1. Design, develop, and maintain robust Apache Airflow DAGs for data orchestration on Astronomer Airflow (managed platform).
2. Build and optimise ETL/ELT pipelines ingesting data from diverse sources into AWS Redshift and AWS RDS.
3. Develop and manage data cataloguing, transformation, and integration jobs using AWS Glue (Glue Studio, Glue Crawlers, Glue Data Catalog).
4. Write efficient, well-tested Pyt...