Lead Data Engineer

March 28

Apply Now

Receive Emails with Similar Jobs

Lunavi

WebsiteLinkedInAll Job Openings

Lunavi is a leading provider of digital transformation consulting and managed IT services, focused on empowering organizations to achieve their digital goals through innovative technology solutions. With over 15 years of expertise, Lunavi specializes in cloud services, modern application development, and managed services, leveraging its deep knowledge of platforms like Microsoft Azure and VMware. The company is dedicated to delivering business value through technology by offering customized solutions, performance-based contracts, and exceptional customer support.

Cloud Hosting • DevOps • Colocation • Managed Services • Managed Azure

51 - 200 employees

Founded 2007

☁️ SaaS

💰 $535k Venture Round on 2016-12

📋 Description

• Lead data engagements including defining architecture, creating databases structures and diagrams, optimizing speed and resiliency, and intelligent data strategies • Implement and manage data solutions using Microsoft Fabric, Databricks and other data platforms, ensuring optimal performance and scalability • Implement machine learning models for automated data classification and tagging, automated data quality validation, predictive insights and intelligent anomaly detection in data streams • Create and deploy resources using Infrastructure as Code and DevOps pipelines • Design, develop, and maintain ETL pipelines and notebooks to ensure efficient data flow and integration across various systems and incorporate predictive analytics and automated insights • Apply architecture frameworks and methodologies (Medallion, Lambda, data mesh, etc.) to organize data into layers for efficient processing and analysis • Design and implement resilient disaster recovery and high availability strategies that optimize system performance, minimize downtime, and balance operational efficiency with cost-effectiveness • Develop and maintain reports and dashboards and reports to provide actionable insights and key performance indicators to stakeholders using PowerBI, PowerApps and other data visualization platforms • Communicate effectively and translate complex technical concepts to technical and non-technical stakeholders and within cross-functional internal and client teams • Provide leadership and mentoring for developers, data engineers and data analysts on your project teams • Assist with scoping, estimating timelines, determining level of effort, and defining team roles required for projects

🎯 Requirements

• 5+ years of experience working with enterprise data warehouses/lakehouses with a recent focus on modern cloud native data architecture and development • 4+ years of experience working with a diverse range of data platforms including Fabric, Databricks, Microsoft SQL, Oracle, Postgres, or other data platforms • 4+ years of experience with PowerBI Pro or Premium datasets and developing strong visualizations • Provide leadership on implementation of engineering practices such as automated CI/CD processes, ETL pipelines, notebooks and other modern data transformation processes and modern data architecture patterns • Proficiency in programming languages commonly used in data analytics and AI/ML projects (e.g., Python, R, SQL) • Experience with AI and ML frameworks, tools, and libraries such as TensorFlow, PyTorch, and scikit-learn • Proficiency in data governance and security strategies including row level security, access controls using Purview, Unity Catalog, Active Directory, etc • 4+ years of experience in customer-facing leadership roles • 3+ years experience providing technical leadership on Agile data projects • Desire to learn, investigate, and provide guidance on new technology in a rapidly changing environment • The desire to bring out the best of the people working on the project to make a positive, shared impact