Senior Data Engineer

June 22, 2024

Apply Now

Receive Emails with Similar Jobs

Pierce Professional Resources

WebsiteLinkedInAll Job Openings

Pierce Professional Resources is a company specializing in technical and workforce solutions aimed at helping businesses evolve. They offer a range of services including technical staffing, automation and AI, cloud solutions, data modernization, digital transformation, and UI/UX consulting. Additionally, they provide agile leadership coaching, PMO services, and assist with transactional mergers and acquisitions. With over 30 years of experience, Pierce Professional Resources emphasizes delivering exceptional technical solutions and personalized services for their clients, aiming to optimize operations and drive innovation. They focus on building partnerships by deeply understanding client needs and providing customized solutions to achieve success.

Temporary Staffing • Contract Staffing • Permanent Staffing • Finance & Accounting Staffing • IT Staffing & Recruiting

11 - 50 employees

Founded 2003

🏢 Enterprise

🎯 Recruitment

📋 Description

• Strategy Creation: Collaborate with cross-functional teams to define the data engineering strategy aligned to business objectives, including data modeling that unifies data assets across a range of source systems used to manage the operations of our partnering hospitals. • Pipeline Development: Define and execute processes needed to develop, test, deploy, and maintain high quality data pipelines. Oversee the end-to-end development of data pipelines from source data extraction through to production-grade analytical dataset delivery, ensuring data quality and security throughout the pipeline. • Performance Optimization: Continuously monitor and optimize data processing performance and efficiency. Identify and address bottlenecks, optimize query performance, and improve overall system stability. • Data Governance: Establish and enforce data quality management policies, data access controls, and data privacy standards. • Technical Leadership: Stay abreast of the latest developments in engineering tools and best practices. Provide guidance to the team about technical challenges. • Documentation: Maintain clear and comprehensive documentation of data pipelines, architecture, and processes to ensure knowledge sharing and team continuity. • Third-party Management: Evaluate and manage relationships with third-party vendors and tools, making informed decisions about when to leverage external solutions.

🎯 Requirements

• Experience with the Azure cloud ecosystem • Experience developing production-ready, real-time machine learning model serving pipelines • Comfort developing in the Apache Spark Structured Streaming paradigm • Experience working in a private equity-backed services company • Experience deploying machine learning models with MLFlow or equivalent • Experience developing CI/CD pipelines

🏖️ Benefits

• 3+ years in data engineering roles in a production environment • Advanced proficiency in Python and SQL for data engineering • Up-to-date knowledge of and 1+ years of experience using Databricks for Lakehouse management • Deep understanding of data modeling, data architecture, and data integration best practices • Strong hands-on experience with Apache Spark • Familiarity with data governance, security, and privacy principles • Comfort using git or equivalent to manage the software development life cycle • Exceptional ability to learn and use new software development techniques and tools • Ability to manage multiple projects simultaneously • High energy, humble team player with "get it done" attitude, seeking collaboration with colleagues