Senior Data Engineer

February 15

Apply Now

Receive Emails with Similar Jobs

xScion Solutions

WebsiteLinkedInAll Job Openings

xScion Solutions (xScion) is a technology consulting firm that helps organizations in highly regulated industries Turn Change Into Value® through Business Agility, Cloud Transformation and Change Management expertise. As a certified Woman-Owned Small Business (WOSB) with more than 20 years of experience supporting Regulatory Technology (RegTech), xScion provides both domain experts and tailored solutions to help organizations navigate complex compliance and technology requirements efficiently in Financial Services, Healthcare, Nonprofits and Public Sector industries.

Technical Management Consulting • Product Management • Agile Transformation • Identity Lifecycle Management • Cloud

51 - 200 employees

📋 Description

• Design, develop, and maintain robust and scalable data pipelines using AWS Glue, Redshift, Python, and PySpark. • Migrate existing data processes from Informatica PowerCenter to AWS Glue, optimizing for performance and efficiency. • Develop and optimize Redshift stored procedures for data transformation and loading. • Write Python and PySpark scripts for data pre-processing, transformation, and analysis within Glue jobs. • Orchestrate and schedule data pipelines using AWS Glue workflows, ensuring reliable and timely execution. • Monitor and troubleshoot data pipelines, identifying and resolving performance bottlenecks and data quality issues. • Collaborate with data scientists and analysts to understand data requirements and translate them into efficient data solutions. • Implement data quality checks and monitoring to ensure data accuracy and integrity. • Stay up-to-date with the latest data engineering technologies and best practices. • Contribute to the continuous improvement of our data infrastructure and processes

🎯 Requirements

• Bachelor's degree in Computer Science, Engineering, or a related field. • Proven experience (8+ years) as a Data Engineer, with a focus on data warehousing and ETL processes. • Strong proficiency in SQL, including experience with Redshift and its specific features (e.g., distribution keys, window functions). • Hands-on experience developing and optimizing Redshift stored procedures. • Solid understanding of AWS Glue and its components (crawlers, ETL jobs, data catalog). • Expertise in Python and PySpark, with the ability to write efficient and scalable data processing scripts. • Experience with data orchestration and scheduling tools, preferably AWS Glue workflows. • Familiarity with data modeling concepts and best practices. • Excellent problem-solving and analytical skills. • Strong communication and collaboration skills. • Experience with other AWS services (e.g., S3, Lambda, EC2) is a plus. • US Citizenship is required for this position.

🏖️ Benefits

• Medical • Dental • 401(k) match • Flexible spending • Up to 27 days off a year (including your birthday!) • Remote work opportunities • Parental leave • Wellness benefits