Senior Data Engineer

5 days ago

Apply Now

Receive Emails with Similar Jobs

Printed.com

WebsiteLinkedInAll Job Openings

Printing • Partnerships • Digital Print • Printing for creatives • Charity

51 - 200

Description

•You will play a crucial role in designing, building, and maintaining our data platform, with a strong emphasis on streaming data, cloud infrastructure, and machine learning operations. •Architect and Implement Data Pipelines: •Design, develop, and maintain scalable and efficient data pipelines. •Optimize ETL processes to ensure seamless data ingestion, processing, and integration across various systems. •Streaming Data Platform Development: •Lead the development and maintenance of a real-time data streaming platform using tools like Apache Kafka, Databricks, Kinesis. •Ensure the integration of streaming data with batch processing systems for comprehensive data management. •Cloud Infrastructure Management: •Utilize AWS data engineering services (including S3, Redshift, Glue, Kinesis, Lambda, etc.) to build and manage our data infrastructure. •Continuously optimize the platform for performance, scalability, and cost-effectiveness. •Collaborations: •Collaborate with cross-functional teams, including data scientists and BI developers, to understand data needs and deliver solutions. •Leverage the project management team to coordinate project, requirements, timelines and deliverables, allowing you to concentrate on technical excellence. •ML Ops and Advanced Data Engineering: •Establish ML Ops practices within the data engineering framework, focusing on automation, monitoring, and optimization of machine learning pipelines. •Data Quality and Governance: •Implement and maintain data quality frameworks, ensuring the accuracy, consistency, and reliability of data across the platform. •Drive data governance initiatives, including data cataloguing, lineage tracking, and adherence to security and compliance standards.

Requirements

•3+ years of experience in data engineering, with a proven track record in building and maintaining data platforms, preferably on AWS. •Strong proficiency in Python, experience in SQL and PostgreSQL. PySpark, Scala or Java is a plus. •Familiarity with Databricks and the Delta Lakehouse concept. •Experience mentoring or leading junior engineers is highly desirable. •Deep understanding of cloud-based data architectures and best practices. •Proficiency in designing, implementing, and optimizing ETL/ELT workflows. •Strong database and data lake management skills. •Familiarity with ML Ops practices and tools, with a desire to expand skills in this area. •Excellent problem-solving abilities and a collaborative mindset. •Familiarity with containerization and orchestration tools (e.g., Docker, Kubernetes). •Knowledge of machine learning pipelines and their integration with data platforms.

Benefits

•22 days holiday Plus 8 days bank holidays •Staff discounts & Friends and Family discounts •Cycle to work scheme and Tech Scheme •Breakfast and drinks provided •Charity day per annum supported •Summer and Christmas Parties •Street food days •Perkbox membership •Enhanced Maternity & Paternity leave •Employee referral scheme