Senior Data Engineer (GenAI AWS)

Our client is seeking a Senior Data Engineer with 6-10 years of experience to design and build advanced data pipelines and AI-powered document processing solutions on AWS. This role focuses on developing end-to-end workflows that integrate generative AI vector databases and RAG architectures to support intelligent agent systems.

Responsibilities & Qualifications

• Design and develop scalable data pipelines using AWS Step Functions Lambda S3 Textract and Comprehend for extracting and processing structured and unstructured data
• Build automated document-processing workflows with document chunking embeddings vector indexing and semantic search capabilities
• Develop and maintain vector databases and knowledge bases to support RAG (Retrieval-Augmented Generation) systems and integrate them with AI agents for contextual information retrieval
• Build and deploy AI agents using Claude Agent SDK and Amazon Bedrock AgentCore Runtime
• Create and maintain CI/CD pipelines using GitHub Actions to automate builds testing and deployments
• Optimize data pipelines for scalability reliability security and performance across production environments
• Implement infrastructure-as-code solutions using Terraform and AWS CDK for repeatable maintainable deployments

Requirements

• 6-10 years of professional experience as a Data Engineer or similar role
• Strong proficiency in Python SQL and TypeScript
• Hands-on experience with AWS services: Lambda S3 Step Functions Glue IAM and Bedrock
• Demonstrated expertise with vector databases(Pinecone pgvector OpenSearch) and embedding technologies
• Experience building RAG systems knowledge bases and semantic search solutions
• Proficiency with AI/ML frameworks and libraries: LangChain LlamaIndex and Claude Agent SDK
• Experience with Document AI OCR technologies (Textract) and NLP services (Comprehend)
• Familiarity with infrastructure-as-code tools (Terraform AWS CDK) and CI/CD automation (GitHub Actions)