Senior Data Engineer - Big Data

March 26

🗽 New York – Remote

Although this job is listed in New York, this company may be open to hiring remote in other states.

💵 $95k - $166k / year

⏰ Full Time

🟠 Senior

🚰 Data Engineer

Apply Now

Receive Emails with Similar Jobs

S&P Global

WebsiteLinkedInAll Job Openings

S&P Global is a leading provider of data, analytics, and insights for critical business decisions, offering a wide range of services including market intelligence, ratings, commodity insights, and indices like the S&P Dow Jones Indices. The company focuses on several key areas such as sustainability, private markets, energy transition, and artificial intelligence. It provides tools and platforms like S&P Capital IQ Pro and Ratings360, as well as ESG scores and indices that guide investment decisions. S&P Global’s offerings are integral to sectors such as finance, investment, and corporate governance, aiming to enhance transparency and efficiency in the global economy.

Analytics • Credit ratings • Finance • Energy & commodities information • Intelligence

10,000+ employees

Founded 1888

💸 Finance

⚡ Energy

📋 Description

• You will be an expert contributor and part of the Rating Organization’s Ingestion Pipelines Engineering Team. • The Data Engineer will support our data department on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects. • Design & Develop “Transformations” aspects using ELT framework to modernize the Ingestion pipelines and build data transformations at scale. • Build processes supporting data transformation, data structures, metadata, dependency and workload management. • Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources into SQL Server, MongoDB, and others. • Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc. • Be in tune with emerging trends Big data and cloud technologies and participate in evaluation of new technologies. • Ensure compliance through the adoption of enterprise standards and promotion of best practice / guiding principles aligned with organization standards.

🎯 Requirements

• BE, MCA or MS degree in Computer Science or Information Technology • 3+ years of hands-on experience in implementing data lake systems using AWS/Azure cloud technologies such as S3, Databricks, Hive. • 1+ years of Expertise in building application using Data stream processing tool, APIs and DBMS for building ingestion pipeline for Bulk and incremental data loads. • Experience with development frameworks as well as data and integration technologies such Python, Scala • Experience in microservices and API design and implementation, with service-oriented architectures, SOAP and RESTful APIs • Hands-on experience in developing scalable data pipeline using technologies like Data stream processing tools, Databricks, Distributed data processing framework and Scala applying ETL and ELT concepts • Deep Experience with three or more technologies of Java/J2EE, C#, AWS, Distributed data processing framework, Python, Scala, any RDBMS, Data stream processing tools, Informatica, Angular/ReactJS, Databricks, Cloud-native orchestration tools • Experience with Continuous integration and deployment tools like Jenkins and Azure DevOps • Experience working in UNIX/Linux environment including shell scripting • Strong understanding of cloud native architectures, design patterns and best practices • Should be in position to articulate and convert requirements into solution. • Knowledgeable in technology and industry trends with ability to develop and present substantive technical solutions • Knowledge of Agile approaches to software development and able to put key Agile principles into practice to deliver solutions incrementally • Quality first mindset with a strong background and experience developing products for a global audience at scale • Excellent analytical thinking, interpersonal, oral, and written communication skills with strong ability to influence both IT and business partners

🏖️ Benefits

• Health care coverage designed for the mind and body. • Generous time off helps keep you energized for your time on. • Access a wealth of resources to grow your career and learn valuable new skills. • Secure your financial future through competitive pay, retirement planning, a continuing education program with a company-matched student loan contribution, and financial wellness programs. • It’s not just about you. S&P Global has perks for your partners and little ones, too, with some best-in class benefits for families. • From retail discounts to referral incentive awards—small perks can make a big difference.