Data Engineer

November 10

Apply Now

Receive Emails with Similar Jobs

DataChef

WebsiteLinkedInAll Job Openings

AWS • Data Lake • Data Mesh • Amazon Web Services • Predictive Analytics

11 - 50 employees

🤖 Artificial Intelligence

Description

• Looking for an experienced data engineer to help design and build a green field data mesh platform on AWS. • Building a scalable data foundation for one of the largest distribution companies in the world; creating a unique foundation and stream processing backend using AWS services (Lambda, Glue, Lake formation,...), Spark (Scala/Python) and Kafka. • Must have excellent communication skills to simplify and present concepts to other people. • Demonstrating solid technical skills in Scala and/or Python is required, with 2+ years of hands-on experience with AWS services.

Requirements

• Have 2+ years of hands-on experience with AWS services like Lambda functions, EMR, ElasticSearch, Lake formation and Glue. • Can demonstrate solid technical skills in Scala and/or Python (deep understanding of language internals, profiling, testing methods). • Have a solid understanding (and preferably experience) of building pub-sub and asynchronous systems using Apache Kafka or any other messaging API like SQS, Kinesis, Celery, RabbitMQ, ActiveMQ. • Can analyze algorithms complexity and know data structures beyond “List and Stack” and pros/cons of using each for a problem. • Have been using Linux/macOS, Docker and git in collaborative workflows for 3+ years.