Senior Streaming Engineer - GCP

November 12

Apply Now

Receive Emails with Similar Jobs

Rackspace Technology

WebsiteLinkedInAll Job Openings

IT as a Service • Multi-Cloud • Managed Hosting • Managed AWS/Azure/Google Cloud Platform/OpenStack/Alibaba • Managed Private Cloud for VMware/Microsoft/OpenStack

5001 - 10000

Description

• About the Role: • We are seeking a highly skilled and experienced Senior Big Data Engineer to join our dynamic team. • The ideal candidate will have a strong background in developing and scaling both stream and batch processing systems, and a solid understanding of public cloud technologies, especially GCP. • This role involves working in a remote environment, requiring excellent communication skills and the ability to solve complex problems independently and creatively. • What you will be doing • Build a reusable, and reliable code for stream and batch processing systems at scale. This includes working with technologies like Pub/Sub, Kafka, Kinesis, DataFlow, Flink, Hadoop, Pig, Hive, and Spark. • Implementing automation/DevOps best practices for CI/CD, IaC, Containerization, etc.

Requirements

• Experience in building a reusable, and reliable code for stream and batch processing systems at scale. • Expertise in public cloud services, particularly in GCP. • Experience with GCP managed services and understanding of cloud-based messaging/stream processing systems are critical. Data Proc, Cloud Composer, GCS, DataFlow. • Experienced in Infrastructure and Applied DevOps principles in daily work. Utilize tools for continuous integration and continuous deployment (CI/CD), and Infrastructure as Code (IaC) like Terraform to automate and improve development and release processes. • Has knowledge in containerization technologies such as Docker and Kubernetes to enhance the scalability and efficiency of applications. • Ability to tackle complex challenges and devise effective solutions. Use critical thinking to approach problems from various angles and propose innovative solutions. • Worked effectively in a remote setting, maintaining strong written and verbal communication skills. Collaborate with team members and stakeholders, ensuring clear understanding of technical requirements and project goals. • Proven experience in engineering stream/batch processing systems at scale. • Strong programming abilities in Java and Python. • Hands-on experience in public cloud platforms, particularly GCP. Additional experience with other cloud technologies is advantageous.