Software Engineer - Data Engineering and Machine Learning

August 26

Apply Now

Receive Emails with Similar Jobs

Rhizome

WebsiteLinkedInAll Job Openings

2 - 10

Description

• Design, construct, and maintain data pipelines to combine large volumes of geospatial, climate + weather, and electric utility datasets • Work with a cross-functional team to deliver data in support of analytic and ML pipelines • Develop deep familiarity with electric utility datasets and take ownership of integration of new datasets into our existing environments • Contribute to ML model development in the context of understanding future extreme weather impacts on the power grid • Optimize storage and ETL pipelines • Develop versioned, scalable, repeatable and reliable pipelines for utility data that is in GIS and Tabular format to Delta Lake format • Scale & automate data pipelines for statistical analysis for internal and external use-cases • Standardize and scale multi-tenant data storage • Exceptional ability to diagnose data issues and discrepancies • Ability to modularize different stages of data ingestion and verification • Ability to write algorithms for data sanity checks and classification of different data elements • Ability to develop heuristics and suggestions for missing data items • Ability to validate and test pipelines and write functional test to validate the pipelines

Requirements

• Exceptional Python programming skills • Exceptional programming skills with NumPy, SciPy, Xarrays • Exceptional programming skills with frameworks like Dagster or Airflow or Prefect • Exceptional programming skills with Databricks or Apache Spark or Amazon EMR or Cloudera • Deep expertise in storage optimization and partitioning on RDS, Postgres, PostGIS, Delta Lake • Hands on with GIS dataset and QGIS or ESRI • Hands on experience with multi-dimensional Climate or Weather data • Familiarity or hands on experience with Secure Cloud Development • Exposure or experience with electric utility tech stacks (AMS, OMS, GIS, etc.) • Exposure to applied ML and Data Engineering in the context of electric utilities

Benefits

• Unlimited time off • Stock options • Excellent health, dental, and vision • 401k