Data Engineer

5 days ago

Apply Now

Receive Emails with Similar Jobs

Nucleus

WebsiteLinkedInAll Job Openings

Entity Setup • HR • Payroll • Accounting • Tax

51 - 200

Description

• Build scalable and efficient data pipelines. • Design the Data Architecture to process complex data from various sources. • Build and maintain the CI/CD infrastructure to host and run data pipelines. • Build and maintain data APIs. • Setup, support, and maintain AI components including generative and machine learning models. • Build mechanisms for monitoring data quality accuracy to ensure reliability and integrity of data. • Evaluate and make technical decisions on the most suitable data technology based on business needs. • Collaborate with Data Scientists, Data Analysts, Software development, and other stakeholders to understand data requirements. • Monitor and optimise performance of data systems, troubleshoot issues, and implement solutions to improve efficiency and reliability.

Requirements

• A strong proficiency in Python • Experience working with Generative AI models, their deployment and orchestration. • A solid understanding of database technologies and modelling techniques, including relational databases, NoSQL databases • Experience with setting and managing Databricks environments • Competent working with Spark • Solid understanding of data warehousing modelling techniques • Competent with setting up CI/CD / Devops pipelines. • Experience with the cloud platforms Azure and AWS and their associated data technologies • Experience and understand of graph technologies and modelling techniques is desirable • Experience of GCP, Scala desirable • Excellent communication skills, capable of explaining complex data/technical concepts to stakeholders with varying levels of technical awareness. • Ability to work collaboratively.

Benefits

• Private medical insurance • Company pension • Dynamic working • Friendly and informal working environment • Excellent opportunities for career and personal development