Senior Data Engineer

July 6

Apply Now

Receive Emails with Similar Jobs

Trafilea Tech E-commerce Group

WebsiteLinkedInAll Job Openings

Life-Changing Brands

eCommerce • technology • retail • marketing • apparel

201 - 500

Description

• Analysis, Design, implementation, and maintenance pipelines that produce business-critical data reliably and efficiently using cloud technologies • Develop new ETLs, using the current Apache Airflow • Propose new initiatives to improve performance, scalability, reliability, and overall robustness • Collect, process, and clean data from different sources using Python & SQL • Work side by side with the main Architects and Developers to create and ensure best practices and guidelines are being used properly by all projects • Assess and communicate effectively the effort for required developments • Discover new data sources to improve new and existing data pipelines • Be in charge of building and maintaining data pipelines and data models for new and existing projects • Maintain detailed documentation of your work and changes to support data quality and governance • Provide feedback and expert points of view as needed to help thrive all data initiatives in the company • Improve the quality of existing and new data processes (ETL), incorporating statistical process control, creating alerts when anomalies are received from data sources on every step of the pipeline • Create benchmark control of execution times for every pipeline, to control and identify potential availability issues

Requirements

• 2-3 years of experience as a Data Engineer, ML Engineer, or similar • 2+ years of experience using Python object-oriented programming and scripting • Strong experience creating data pipelines and ETL processes in modern technologies using Apache Airflow as the main workflow management tool • 2+ years of experience with the AWS data ecosystem (S3, Glue, Athena, Redshift, Lambda, EC2, RDS ...) • Experience with large-scale data and query optimization techniques using SQL • Experience managing and monitoring data lakes systems • Strong experience using source control management and CI/CD (GitHub actions/Gitlab pipelines)

Benefits

• Collaborate with world-class talents in a data-driven, dynamic, energetic work environment • Opportunity to grow and develop both professionally and personally • Safe space to be who you truly are, with a commitment to diversity, equity, and inclusion • Openness to new ideas and initiatives • Great benefits package including remote work, 15 working days of paid holidays, Learning subsidy, and more!