Senior Spark Data Engineer

October 31

Apply Now

Receive Emails with Similar Jobs

Valtech

WebsiteLinkedInAll Job Openings

Digital strategy • Experience & Commerce Platforms • Connected Experiences • Content & Campaigns • Data Science

5001 - 10000

Description

• Demonstrate deep knowledge of the data engineering domain to build and support non-interactive (batch, distributed) & real-time, highly available data, data pipeline, and technology capabilities. • Build fault-tolerant, self-healing, adaptive, and highly accurate data computational pipelines. • Provide consultation and lead the implementation of complex programs. • Develop and maintain documentation relating to all assigned systems and projects. • Tune queries running over billions of rows of data running in a distributed query engine. • Perform root cause analysis to identify permanent resolutions to software or business process issues.

Requirements

• Proven industry experience executing data engineering, analytics and/or data science projects or Bachelors/Masters degree in quantitative studies including Engineering, Mathematics, Statistics, Computer Science or computation-intensive Sciences and Humanities. • More than 2 years of experience working with: 1. Strong/expert Spark (PySpark) Using Jupyter Notebooks, Colab or DataBricks 2. Hands-on data pipeline development, ingest patterns in Azure 3. Orchestration tools , ADF or Airflow 4. SQL 5. Denormalized Data modeling for big data systems • Collaborative, able to work remotely, and still be an engaging team member. • Strong analytical and design skills.