Senior Spark Data Engineer

October 31

Apply Now

Receive Emails with Similar Jobs

Valtech

WebsiteLinkedInAll Job Openings

Digital strategy • Experience & Commerce Platforms • Connected Experiences • Content & Campaigns • Data Science

5001 - 10000

Description

• Ideal candidates will have strong quantitative backgrounds and analytical discipline. • Valtech is looking to hire Senior Spark Data Engineer to join our growing capabilities team. If you are innovative, passionate about data and AI technologies, and look to continually learn and enjoy sharing expertise, read on! • Demonstrate deep knowledge of the data engineering domain to build and support non-interactive (batch, distributed) & real-time, highly available data, data pipeline, and technology capabilities. • Build fault-tolerant, self-healing, adaptive, and highly accurate data computational pipelines. • Provide consultation and lead the implementation of complex programs. • Develop and maintain documentation relating to all assigned systems and projects. • Tune queries running over billions of rows of data running in a distributed query engine. • Perform root cause analysis to identify permanent resolutions to software or business process issues.

Requirements

• Proven industry experience executing data engineering, analytics and/or data science projects or Bachelors/Masters degree in quantitative studies including Engineering, Mathematics, Statistics, Computer Science or computation-intensive Sciences and Humanities. • More than 2 years of experience working with: 1. Strong/expert Spark (PySpark) Using Jupyter Notebooks, Colab or DataBricks 2. Hands-on data pipeline development, ingest patterns in Azure 3. Orchestration tools , ADF or Airflow 4. SQL 5. Denormalized Data modeling for big data systems • Collaborative, able to work remotely, and still be an engaging team member. • Strong analytical and design skills.