Senior Data Engineer

October 31

Apply Now

Receive Emails with Similar Jobs

ICF

WebsiteLinkedInAll Job Openings

Aviation • Climate • Cybersecurity • Digital and Marketing • Education

5001 - 10000

💰 Grant on 2023-02

Description

• Design, develop, and maintain scalable data pipelines using Spark, Hive, and Airflow • Develop and deploy data processing workflows on the Databricks platform • Develop API services to facilitate data access and integration • Create interactive data visualizations and reports using AWS QuickSight • Builds required infrastructure for optimal extraction, transformation and loading of data from various data sources using AWS and SQL technologies • Monitor and optimize the performance of data infrastructure and processes • Develop data quality and validation jobs • Assembles large, complex sets of data that meet non-functional and functional business requirements • Write unit and integration tests for all data processing code • Work with DevOps engineers on CI, CD, and IaC • Read specs and translate them into code and design documents • Perform code reviews and develop processes for improving code quality • Improve data availability and timeliness by implementing more frequent refreshes, tiered data storage, and optimizations of existing datasets • Maintain security and privacy for data at rest and while in transit • Other duties as assigned

Requirements

• Bachelor's degree in computer science, engineering or related field or comparable experience. • 5+ years of hands-on software or data development experience (3 years with Masters) • 4+ years of data pipeline experience using Python, Java and cloud technologies • Candidate must be able to obtain and maintain a Public Trust clearance • Candidate must reside in the US, be authorized to work in the US, and work must be performed in the US • Must have lived in the US 3 full years out of the last 5 years