Sr. Engineer, Software Kafka

Sr. Engineer Software designs develops and delivers scalable software solutions that enable highly available and secure systems across the enterprise. Collaborating closely with data engineers this role architects and implements complex data pipelines real-time event-driven messaging frameworks and distributed microservices communication using Apache Kafka and messaging queue technologies.

The Engineer brings deep expertise in Kafka Python and Java object-oriented programming UNIX environments and Bash shell scripting along with hands-on experience building RESTful API endpoints open-source messaging platforms and end-to-end features spanning real-time API and data layers.

Success is measured by the quality scalability and reliability of software delivered and the ability to drive technical outcomes across teams.

What Youll Do

• Build real-time data processing pipelines that support low-latency high-throughput and fault-tolerant distributed systems.
• Develop and maintain event-driven messaging frameworks using Apache Kafka or similar technologies to enable microservices to communicate data through events.
• Automate manual processes and operational workflows using Bash shell scripting in UNIX environments.
• Design develop test and deploy high-quality software solutions using Python and Java.
• Build scalable high-performing RESTful APIs and microservices aligned with fault-tolerant distributed systems.
• Implement and support event-driven and message-based architectures using Kafka or similar technologies.
• Build scalable ETL frameworks using object-oriented Python and Java programming languages.
• Contribute to system and application architecture including service design data flows and integration patterns.
• Participate in code reviews agile ceremonies and technical design discussions.
• Experience deploying and managing Kafka in containerized environments using Docker and Kubernetes.
• Proficiency with Kafka Connect for data integration and Schema Registry for Avro/Protobuf/JSON schema management.
• Troubleshoot and resolve complex software issues across development and production environments.
• Experience developing Kafka Producer and Consumer APIs using Spring Boot and Spring Kafka.
• Proficiency building and exposing RESTful API endpoints for seamless service integration.
• Experience contributing to or building open-source messaging platforms.
• Develop and maintain CI/CD pipelines automation and deployment workflows.
• Write and maintain clear technical documentation including design documents and system specifications.
• Continuously evaluate and adopt modern technologies to improve system performance reliability and maintainability.
• Daily use of AI productivity tools (Claude Cursor) is required across the software development lifecycle including design API and microservice development code generation code review testing debugging deployment documentation and production support.
• Design build and operate AI agents that automate software engineering tasks such as generating boilerplate code for new microservices creating unit and API tests analyzing logs triaging incidents and supporting CI/CD pipelines.
• Build and integrate AI-enabled capabilities into backend services and APIs using foundation models prompt engineering and retrieval-augmented generation (RAG) patterns.
• Implement audit logging observability and human-in-the-loop controls for AI agents and AI-assisted workflows running in Tier-0 production environments.

What Youll Bring

• Bachelors degree in Computer Science Engineering or a related field or equivalent practical experience.
• 6 years of software engineering experience building and supporting enterprise-grade applications.
• Strong experience working in Agile or Scrum-based development environments.
• Demonstrated ability to work independently on complex technical problems while collaborating across teams.
• Strong analytical problem-solving and communication skills