Data Engineer - Senior Data Engineer

November 1

Apply Now

Receive Emails with Similar Jobs

Nucleus

WebsiteLinkedInAll Job Openings

Entity Setup • HR • Payroll • Accounting • Tax

51 - 200

Description

• setup, support, interact with and maintain AI components including generative and machine learning models • build scalable and efficient data pipelines • design the Data Architecture to process complex data from a variety of data sources • build and maintain the CI/CD pipeline and infrastructure to host and run data pipelines • build and maintain data APIs • build mechanisms for monitoring the data quality accuracy • evaluate and make technical decisions on suitable data technology based on business needs • collaborate with Data Scientists, Data Analysts, Software development and other stakeholders to understand data requirements • work closely with System Admins and Infrastructure teams to integrate data engineering platforms • monitor and optimise performance of data systems, troubleshoot issues, and implement solutions to improve efficiency and reliability

Requirements

• strong proficiency in Python for data processing • experience or strong understanding of Generative AI models, their deployment and orchestration • solid understanding of database technologies and modelling techniques, including relational databases, NoSQL databases • experience managing Databricks environments • competent working with Spark • solid understanding of data modelling techniques • competent with setting up CI/CD/Devops pipelines • experience with setting up infrastructure securely in cloud platforms (Azure and AWS) • experience developing and deploying containerised applications • experience of GCP desirable • experience of Scala desirable

Benefits

• great compensation and benefits package including private medical insurance • company pension • dynamic working environment • friendly and informal working environment • opportunities for career and personal development