Join our Facebook group
👉 Remote Jobs Network
Software Engineer - Data Engineering and Machine Learning
August 26
Description
• Design, construct, and maintain data pipelines to combine large volumes of geospatial, climate + weather, and electric utility datasets • Work with a cross-functional team to deliver data in support of analytic and ML pipelines • Develop deep familiarity with electric utility datasets and take ownership of integration of new datasets into our existing environments • Contribute to ML model development in the context of understanding future extreme weather impacts on the power grid • Optimize storage and ETL pipelines • Develop versioned, scalable, repeatable and reliable pipelines for utility data that is in GIS and Tabular format to Delta Lake format • Scale & automate data pipelines for statistical analysis for internal and external use-cases • Standardize and scale multi-tenant data storage • Exceptional ability to diagnose data issues and discrepancies • Ability to modularize different stages of data ingestion and verification • Ability to write algorithms for data sanity checks and classification of different data elements • Ability to develop heuristics and suggestions for missing data items • Ability to validate and test pipelines and write functional test to validate the pipelines
Requirements
• Exceptional Python programming skills • Exceptional programming skills with NumPy, SciPy, Xarrays • Exceptional programming skills with frameworks like Dagster or Airflow or Prefect • Exceptional programming skills with Databricks or Apache Spark or Amazon EMR or Cloudera • Deep expertise in storage optimization and partitioning on RDS, Postgres, PostGIS, Delta Lake • Hands on with GIS dataset and QGIS or ESRI • Hands on experience with multi-dimensional Climate or Weather data • Familiarity or hands on experience with Secure Cloud Development • Exposure or experience with electric utility tech stacks (AMS, OMS, GIS, etc.) • Exposure to applied ML and Data Engineering in the context of electric utilities
Benefits
• Unlimited time off • Stock options • Excellent health, dental, and vision • 401k
Apply NowSimilar Jobs
August 17
MissionWired
51 - 200
Transform data into opportunities supporting progressive causes and social good.
August 17
OneLocal
51 - 200
Embed AI technology in tools for small and medium-sized business marketing solutions.
August 16
Age of Learning
501 - 1000
Support finance and digital marketing with scalable data warehouses and infrastructure.
August 8
Floodbase
11 - 50
Process large volumes of geospatial data for businesses adapting to climate change impacts.
July 31
Biofourmis
201 - 500
Manage data flows and enhance analysis for Biofourmis' AI-driven healthcare solutions.