QA Engineer - Data Engineering

3 days ago

Apply Now

Receive Emails with Similar Jobs

Mediastream

WebsiteLinkedInAll Job Openings

CDN • Media • Broadcast • OTT • Online Video Platform

51 - 200 employees

Founded 2007

🤝 B2B

📱 Media

🛍️ eCommerce

Description

• The hire will be responsible for defining and implementing the most appropriate test strategy to validate our data pipeline architecture and data flow and collection for cross-functional teams. • Develop and maintain testing strategies for big data pipelines, machine learning models, and real-time data systems • Ensure these strategies align with the overall data architecture and business goals • Design, document, and execute test plans for data ingestion, transformation, and analytics tools in distributed environments • Include edge cases and stress testing scenarios • Automate test cases to ensure continuous integration and delivery (CI/CD) for data pipelines and infrastructure • Collaborate with the Data Engineer and DevOps team to troubleshoot issues and ensure high-quality data and system performance • Participate in code reviews and provide feedback on data quality • Conduct performance testing for real-time streaming and distributed systems to ensure scalability and reliability • Use performance monitoring tools to identify bottlenecks • Validate data quality and integrity across different stages of data pipelines and machine learning workflows • Document test cases, results, and issue reports while providing feedback to stakeholders

Requirements

• 3+ years of professional experience in QA • Strong knowledge of testing big data environments (e.g., Hadoop, Spark) and stream-processing systems (e.g., Kafka, Storm) • Familiarity with cloud-based big data solutions (e.g., AWS EMR, Google BigQuery) • Experience with automated testing frameworks and tools (e.g., Selenium, JUnit, PyTest, or similar) • Knowledge of behavior-driven development (BDD) frameworks like Cucumber • Familiarity with CI/CD practices and tools like Jenkins, GitLab, or Bitbucket Pipelines • Experience with containerization tools like Docker and orchestration tools like Kubernetes • Experience working with data warehouses, data lakes, and cloud storage (e.g., AWS S3, Hive, HBase) • Understanding of data governance and metadata management • Proficiency in scripting languages (e.g., Python) to automate test cases for data-related processes • Knowledge of other programming languages like Java or Scala is a plus • Upper-intermediate or advanced spoken and written English

Benefits

• Company and team events • Ongoing training and further education • An energetic atmosphere and opportunity to grow within and with the company