DataBricks Engineer

December 7

Apply Now

Receive Emails with Similar Jobs

Sales Consulting

WebsiteLinkedInAll Job Openings

Recruitment&Selection • Assessment Center • HR Consultancy • HR Market Mapping

51 - 200 employees

Founded 1998

👥 HR Tech

🎯 Recruitment

Description

• Develop and provide solutions on Big Data; • Develop high-traffic, flawless web applications using Python, Cloud platform, and PySpark; • Code with performance, scalability, and usability in mind; • Work on new tools in leading industry trends, with new and emerging technologies, prototypes and engineering process improvements; • Migrate existing pipelines developed in ADF and python scripts from On-premise to Cloud using ADF and Databricks; • Develop new pipelines using ADF for data extraction from multiple sources (MySQL databases; Oracle databases, CSV files) and Databricks (PySpark & SparkSQL) for curation, validation and applying required business transformation logic; • Work on client projects to deliver Microsoft Azure based Data engineering & Analytics solutions; • Engineer and implement scalable analytics solutions on Azure.

Requirements

• Bachelor's degree in computer science or related field • 5+ years of experience in a similar role; leading fellow data engineers within the data engineering area; • Extensive experience working in agile project environments; • Working experience with Python or PySpark is required; • Working experience and understanding of Azure Data-bricks, Azure Data Factory, Azure Data Lake, Azure SQL DW, and Azure SQL is required; • The ability to apply such methods to solve business problems using one or more Azure Data and Analytics services in combination with building data pipelines, data streams, and system integration; • Experience in driving new data engineering developments (e.g. applying new cutting-edge data engineering methods to improve the performance of data integration, using new tools to improve.