Data Engineer

2 days ago

Apply Now

Receive Emails with Similar Jobs

Aboitiz Group

WebsiteLinkedInAll Job Openings

Power • Banking • Food • Real Estate • Infrastructure

10,000+

Description

• Move data from various sources to AWS S3 (data landing zone) using tools such as AWS Database Management Service (DMS), Python, or PySpark. • Jobs will be orchestrated using Airflow or run in Databricks. • Ingest raw data into the Data Lake (RAW zone) for processing. • Prepare data in the Data Lake through the TRUSTED and CURATED zones using DBT and SQL to ensure data quality and readiness for analysis. • Design and implement data models that meet business requirements and optimize performance. • Work closely with data scientists, analysts, and other stakeholders to identify data requirements and provide solutions. • Monitor data pipelines and workflows, troubleshoot issues, and optimize performance for scalability and reliability.

Requirements

• Bachelor’s degree in Computer Science, Engineering, or a related field. • Proven experience as a Data Engineer or in a similar role. • Strong analytical and problem-solving skills. • Proficient in Python and SQL. • Experience with PySpark, DBT, and PyDeequ. • Familiarity with Databricks. • Strong knowledge of AWS services, including Database Management Service and S3. • Experience with Apache Airflow for workflow management. • Understanding of Data Lake, Data Warehouse, Data Modeling, and Medallion Architecture. • Experience with APIs/Web Services, API Gateway, and AWS Lambda. • Knowledge of Docker and EKS (Elastic Kubernetes Service). • Familiarity with Poetry for Python dependency management. • Excellent communication skills and ability to work collaboratively in a team environment.