Join to apply for the Data Engineer role at Kelsus, Inc.
Responsibilities - Design and develop efficient data pipelines using Python, AWS Glue/pyspark, AWS Lambda, Kafka, and related technologies.
- Implement and optimize ETL processes for data extraction, transformation, and loading into relational databases, especially Postgres.
- Collaborate on data warehouse architectures, ensuring proper data modeling, storage, and access.
- Utilize tools like StitchData and Apache Hudi for data integration and incremental management, improving efficiency and enabling complex operations.
- Identify and resolve data quality and consistency issues, implementing monitoring processes across pipelines and storage systems.
Requirements - Strong Python skills and proficiency in associated libraries for data processing and manipulation.
- Expertise in AWS Glue/pyspark, AWS Lambda, and Kafka for ETL workflow development and stre...