Sharesource
Artificial Intelligence • Robotic Process Automation • Distributed Teams • Engineering • Tech
Data Engineer - Databricks
November 22
Sharesource
Artificial Intelligence • Robotic Process Automation • Distributed Teams • Engineering • Tech
Description
• Design and implement data pipelines for extracting and loading data into a central data store • Collaborate with stakeholders to understand business requirements and design data models • Optimise and enhance existing data pipelines for high performance and reliability • Develop and maintain data quality checks to ensure accuracy and completeness • Monitor and maintain data pipelines, identifying areas for improvement • Participate in code reviews and collaborate with other engineers to maintain high-quality code • Create visualisations using tools like Power BI • Stay updated with industry trends, recommending new tools and technologies for data infrastructure improvement • Participate in discovery workshops and solutions design activities with customers
Requirements
• At least 5 years of related experience • Strong experience with Databricks, Snowflake, Redshift, or Synapse • Demonstrated proficiency in AWS and/or Azure • Extensive knowledge of ETL processes and data integration • Expertise in data warehousing, modelling, and lake house architecture • Strong programming skills in Python, SQL, and Spark • Hands-on experience with Python (Pyspark, Lambda, Batch, Glue, DataFlow, etc) • Proficiency in data transformation and migration • Understanding of Data Governance concepts (MDM, Metadata management, Data Security, Data Quality, etc)
Benefits
• Competitive salary and professional development • Collaborative client partnerships and a fun, inclusive work environment • Opportunities for leadership, expertise, and global client exposure • Monthly coaching, training, and career development • Remote work flexibility and a flexible hybrid model • Engage in social impact activities and industry impact • Achieve work-life balance and flexibility to support personal commitments • Comprehensive HMO coverage with one free dependent
Apply NowSimilar Jobs
November 10
Support data engineering needs for analytics projects at Arch.
October 20
Data Engineer for building scalable data pipelines at A-VNG.
October 1
Data engineer transforming raw data for SAVii's wellness services.
July 15
Manage data migration for Xero cloud-based practice and tax tools, supporting ABs & SMBs.