Big Data Engineer

March 27

Apply Now

Receive Emails with Similar Jobs

MGID

WebsiteLinkedInAll Job Openings

MGID is an innovative global pioneer in native advertising, driving revenue growth for participants in the advertising ecosystem. The company provides solutions for monetization, audience acquisition, and engagement for premium publishers. MGID aims to foster a new era in digital media, promoting meaningful and positive interactions among consumers, content creators, and brands.

Native advertising • paid content distribution • audience development • website traffic management • advertising

501 - 1000 employees

📱 Media

☁️ SaaS

💰 Venture Round on 2009-11

📋 Description

• Collaborate with Data Scientists, Data Analysts, and other stakeholders to understand data needs and develop solutions. • Design, develop, and optimize PySpark applications for processing and analyzing large sets of structured and unstructured data. • Monitor and evaluate data to ensure accuracy and integrity, troubleshoot and debug PySpark code. • Build and maintain data pipelines for ingesting, processing, and storing data, optimizing for performance and scalability. • Develop and maintain data visualization dashboards and reports to enable insights and decision-making. • Create and maintain tools and libraries for efficient data processing. • Stay up-to-date with industry trends and new technologies to continuously improve data processing capabilities.

🎯 Requirements

• Proven experience in developing and optimizing PySpark applications. • Strong knowledge of distributed computing principles and concepts. • Practical experience working with large datasets using technologies such as Hadoop, Spark, ClickHouse. • Proficiency in programming languages such as Python, SQL. • Experience with Linux/Unix command-line interface. • Familiarity with data visualization and dashboarding tools. • Strong communication skills and ability to work effectively in a remote team environment. • Excellent problem-solving skills and attention to detail. • Will be a plus: Bachelor’s or Master’s degree in Computer Science or a related field. • Practical experience with ClickHouse. • Practical experience with stream processing and messaging systems such as Kafka. • Practical experience with NoSQL databases (for example MongoDB), especially Aerospike. • Knowledge of AdTech domain — understanding of online advertising, RTB. • Familiarity with containerization technologies such as Docker and Kubernetes, cloud computing platforms. • Familiarity with data governance and security best practices. • Knowledge of machine learning concepts and frameworks.