Kafka Lead Engineer

5 days ago

Apply Now

Receive Emails with Similar Jobs

i4DM

WebsiteLinkedInAll Job Openings

business intelligence • application development • database & data warehouses • enterprise portals • identity management & compliance

51 - 200 employees

Founded 2002

🤝 B2B

☁️ SaaS

🏢 Enterprise

Description

• Design and develop Kafka-based data pipelines and streaming solutions • Develop connectors, transformations, and integration patterns for real-time data access • Collaborate with cross-functional teams to deliver adaptable solutions • Engage with Army personnel to tailor Kafka solutions to operational needs • Lead and mentor associate Kafka developers • Monitor Kafka systems for performance and troubleshoot issues • Create documentation of Kafka architectures and best practices

Requirements

• Proven experience as a Kafka Engineer or Kafka Lead • Expertise in Kafka architecture , streaming technologies , and data pipeline development (including Kafka Streams, Kafka Connect, and related tools) • Expertise in Kafka Performance tuning, data tagging and data aggregation • Expertise in building DataOps practices , primarily as it relates to automating testing and deployments of Kafka based data products through integrated CI/CD pipelines • Agile methodology experience, with the ability to adapt to fast-paced, changing environments and work closely with teams using agile frameworks (Scrum, Kanban, etc.) • Strong customer focus with the ability to engage directly with clients, understand their needs, and deliver technical solutions that align with their business goals • Experience working in government or defense sectors is a plus, with a strong understanding of how to work in highly regulated and mission-critical environments • Proficiency in Java , Scala , Python or other relevant programming languages for Kafka development • Experience with Kafka-related tools (e.g., Kafka Streams, Kafka Connect, Schema Registry) • Strong understanding of data integration , ETL processes , and real-time analytics • Excellent problem-solving, communication, and collaboration skills • Security clearance or the ability to obtain one is preferred (depending on project requirements) • Familiarity with Army-specific systems, workflows, and organizational culture • Experience integrating Kafka with Appian Data fabric • Experience using Kafka to automate data tagging • Experience with cloud-based solutions (e.g., AWS, Azure) and deploying Kafka on cloud platforms • Strong experience in data modeling , distributed systems , and microservices architecture