Staff Data Engineer

4 days ago

🇺🇸 United States – Remote

🦀 Maryland – Remote

Although this job is listed in Maryland, this company may be open to hiring remote in other states.

💵 $149.2k - $160k / year

⏰ Full Time

🔴 Lead

🚰 Data Engineer

Apply Now

Receive Emails with Similar Jobs

eSimplicity

WebsiteLinkedInAll Job Openings

Cybersecurity • Information Technology • Telecommunication • Healthcare IT • Spectrum Management

51 - 200 employees

Founded 2016

⚕️ Healthcare Insurance

📡 Telecommunications

🤖 Artificial Intelligence

Description

• Responsible for expanding and optimizing our data and data pipeline architecture • Support software developers, database architects, data analysts and data scientists on data initiatives and ensure optimal data delivery architecture is consistent throughout ongoing projects • Creates new pipeline development and maintains existing pipeline, updates Extract, Transfer, Load (ETL) process • Creates new ETL feature development, builds PoCs with Redshift Spectrum, Databricks, etc • Implements, with support of project data specialists, large dataset engineering: data augmentation, data quality analysis, data analytics (anomalies and trends), data profiling, data algorithms, and (measure/develop) data maturity models and develop data strategy recommendations • Assemble large, complex sets of data that meet non-functional and functional business requirements • Identify, design, and implement internal process improvements including re-designing data infrastructure for greater scalability, optimizing data delivery, and automating manual processes • Building required infrastructure for optimal extraction, transformation and loading of data from various data sources using AWS and SQL technologies • Building analytical tools to utilize the data pipeline, providing actionable insight into key business performance metrics including operational efficiency and customer acquisition • Working with stakeholders including data, design, product and government stakeholders and assisting them with data-related technical issues • Write unit and integration tests for all data processing code • Work with DevOps engineers on CI, CD, and IaC • Read specs and translate them into code and design documents • Perform code reviews and develop processes for improving code quality

Requirements

• Minimum of 8 years related experience • 7 + years of hands-on software development experience • 4+ years of Data pipeline experience using Python, Java and cloud technologies • A Bachelor’s degree in Computer Science, Information Systems, Engineering, Business, or other related scientific or technical discipline • Expert data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up • Self-sufficient and comfortable supporting the data needs of multiple teams, systems, and products • Experienced in designing data architecture for shared services, scalability, and performance • Experienced in designing data services including API, meta data, and data catalogue • Experienced in data governance process to ingest (batch, stream), curate, and share data with upstream and downstream data users • Ability to build and optimize data sets, ‘big data’ data pipelines and architectures • Ability to perform root cause analysis on external and internal processes and data to identify opportunities for improvement and answer questions • Excellent analytic skills associated with working on unstructured datasets • Ability to build processes that support data transformation, workload management, data structures, dependency and metadata • Demonstrated understanding and experience using software and tools including big data tools like Kafka, Spark and Hadoop; relational NoSQL and SQL databases including Cassandra and Postgres; workflow management and pipeline tools such as Airflow, Luigi and Azkaban; AWS cloud services including Redshift, RDS, EMR and EC2; stream-processing systems like Spark-Streaming and Storm; and object function/object-oriented scripting languages including Scala, C++, Java and Python • Flexible and willing to accept a change in priorities as necessary • Ability to work in a fast-paced, team-oriented environment • Experience with Agile methodology, using test-driven development • Experience with Atlassian Jira/Confluence • Excellent command of written and spoken English • Ability to obtain and maintain a Public Trust; residing in the United States

Benefits

• Highly competitive salary • Full healthcare benefits