Senior Data Engineer

5 days ago

Apply Now

Receive Emails with Similar Jobs

Allstate

WebsiteLinkedInAll Job Openings

Auto Insurance • Life Insurance • Retirement Planning • Homeowners Insurance • Motorcycle Insurance

10,000+ employees

Founded 1931

💰 Post-IPO Equity on 2014-01

Description

•Design, build, and maintain end-to-end data and machine learning pipelines to support analytics, reporting, and AI-driven applications. •Develop and optimize scalable ETL/ELT processes to extract, transform, and load data from diverse sources into a cloud-based platform. •Architect and manage data storage solutions within the data platform (e.g., data lakes, warehouses, and marts) to enable advanced analytics and machine learning. •Implement and manage ML pipelines, building feature pipelines and deploying models. •Collaborate with data scientists, analysts, and software engineering to integrate data products into business workflows. •Ensure data quality, consistency, and governance by implementing robust monitoring, validation, and alerting mechanisms. •Lead the adoption of new cloud-native technologies to streamline and enhance data and ML operations. •Mentor junior data engineers, fostering a culture of innovation and knowledge-sharing within the team.

Requirements

•Bachelor’s degree in Computer Science, Data Science, Software Engineering, Mathematics, Statistics, or a related field. •A Master’s degree is preferred. •5+ years of professional experience in data engineering, including end-to-end pipeline development and cloud integration. •Proven experience with machine learning workflows, including data preparation, feature engineering, and model deployment. •Proficiency in programming languages such as Python, Scala, or Java, with an emphasis on ML libraries like TensorFlow, PyTorch, or Scikit-learn. •Strong knowledge of data processing frameworks (e.g., Apache Spark, Flink, Beam) and real-time data streaming (e.g., Kafka, Kinesis). •Hands-on expertise with AWS or GCP ecosystems, including tools like: AWS: SageMaker, Redshift, Glue, Athena, EMR GCP: BigQuery, Vertex AI, Dataflow, Dataproc •Solid understanding of relational and non-relational database systems (SQL, NoSQL). •Experience with data orchestration tools (e.g., Airflow, Prefect, dbt) and CI/CD practices. •Knowledge of containerization and orchestration tools (e.g., Docker, Kubernetes) is a plus.