Senior Data Engineer

3 days ago

Apply Now

Receive Emails with Similar Jobs

CBTW

WebsiteLinkedInAll Job Openings

CBTW is a global company that embraces collaboration to deliver innovative technology and business solutions. With over 3000 professionals worldwide, CBTW specializes in various domains including Strategy & Governance, Software Engineering, Data Analytics, Cyber Security, and Banking Technology. They cater to a diverse range of sectors such as Life Sciences, Manufacturing, and Retail, driving success through tailored solutions for clients ranging from startups to Fortune 500 companies.

Software Engineering • Governance • Strategy • Design • Technology

1001 - 5000 employees

🤝 B2B

🔒 Cybersecurity

💳 Fintech

📋 Description

• The Sr. Data Engineer will be responsible for delivering best-in-class data integration capabilities to WELL’s next-generation platform. • This role involves developing bi-directional, stream-based, near real-time synchronization frameworks, enabling EMR teams to implement streaming data pipelines, and developing a data sync chaos monkey to identify data flow issues proactively. • What you will be doing: Develop bi-directional, stream-based, near real-time synchronization framework: Work closely with the Principal Architect to design and implement a system that synchronizes data bi-directionally from EMR systems to a FHIR repository, ensuring resilience against network outages, monitoring production systems, detecting and resolving data conflicts, and auditing data access. • Enable EMR teams to implement streaming data pipelines: Provide comprehensive guidance, tools, and training to EMR teams, develop clear documentation, and establish support sessions to ensure effective application of the synchronization framework. • Develop a data sync chaos monkey: Implement a system that actively tests the bi-directional sync system to identify and resolve data flow issues.

🎯 Requirements

• Change Data Capture (CDC): Expertise in implementing CDC solutions with tools like Debezium, and familiarity with alternatives such as GoldenGate, SQL Server CDC, and Qlik Replicate. • Data Pipeline Development: Experience in designing and maintaining data pipelines for batch and streaming data, with knowledge of stream processing frameworks like Apache Flink and Azure Stream Analytics. • Azure Cloud Technologies: Proficiency in Azure services including Azure Kubernetes Service (AKS), Azure Functions, and Azure Event Hubs. • Database Management: Advanced knowledge of MySQL, SQL Server/Azure SQL Database, PostgreSQL, and experience with CDC and replication mechanisms. • FHIR and Healthcare Interoperability: Understanding of FHIR standards and integration with existing systems. • Programming and Scripting: Proficiency in languages such as Java, Python, Go, C#, and scripting skills for automation tasks. • Streaming and Messaging Systems: Experience with platforms like Apache Kafka or Azure Event Hubs. • Version Control and CI/CD: Expertise in using Git and CI/CD pipelines. • Monitoring and Troubleshooting: Experience with tools like Prometheus, Grafana, and Kibana for diagnosing integration issues.