Lead Data Engineer

5 days ago

Apply Now

Receive Emails with Similar Jobs

The Baldwin Group

WebsiteLinkedInAll Job Openings

Commercial Risk Management • Private Risk Management • Personal Insurance • Employee Benefits • Asset and Income Protection

1001 - 5000

Description

• Oversee the design, building and optimization of data orchestration and pipeline architecture. • Optimize data collection and flow for cross functional teams. • Mentor junior engineers and provide technical guidance. • Maintain and update Data Engineering Architecture. • Design and implement ELT and ETL processes. • Create and maintain optimal data orchestration architecture to support data initiatives. • Provide thought leadership to improve implementation approaches. • Assemble large, complex data sets that meet business requirements. • Identify and implement internal process improvements. • Build infrastructure for optimal extraction, transformation, and ingestion of data. • Support analytics tools using the data pipeline. • Work with stakeholders to assist with data-related technical issues. • Create data tools for analytics teams. • Collaborate with colleagues for data collection and structuring. • Communicate with clients and collect/validate data. • Recommend new practices, processes, and procedures.

Requirements

• Bachelor’s degree in related field preferred, equivalent years’ experience considered. • At least seven to ten years of data related or analytical work experience in a Data Engineer role, preferably three of those within the Azure ecosystem. • Experience building and optimizing ‘big data’ data pipelines, architectures and data sets. • Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement. • Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of data platforms. • Advanced understanding of and experience implementing data lakes, lake houses. • Advanced understanding and experience with file storage layer management in data lake environment, including parquet and delta file formats. • Solid experience with SPARK (PySpark) language, and data processing techniques. • Solid Understanding of and experience with AZURE SYNAPSE tools and services. • Some knowledge of Python preferred. • Strong analytic skills related to working with structured, semi-structured, and unstructured datasets and blob storage. • Build processes supporting data transformation, data structures, metadata, dependency and workload management. • A successful history of manipulating, processing and extracting value from large disconnected datasets. • Strong project management and organizational skills. • Experience supporting and working with cross-functional teams in a dynamic environment. • Insurance industry experience preferred.