Enter a job title or keyword
Gen AI Testing
Job Location:
Monthly Salary:
Posted:
19 May 2026 (30+ days ago)
Posted:
Application Deadline:
16 August 2026
Application Deadline:
Vacancies:
1 Vacancy
Vacancies:
Job Summary
Seeking a Lead Software Engineer Automation Testing with 6 8 years of experience in designing developing and executing automation frameworks for AI/ML-driven applications.
The ideal candidate brings hands-on experience with GenAI Testing LangSmith PyTest Agentic frameworks and strong AI/ML evaluation expertise.
- Strong programming expertise in Python.
- Hands-on expertise with LangSmith Agentic frameworks PyTest BDD/V&V frameworks.
AI / GenAI Testing
- Design and execute V&V strategies for classical ML and GenAI/ agentic systems.
- Define test plans tests cases and execute them
- Evaluate AI models for accuracy performance safety robustness hallucinations bias and correctness.
- For GenAI rely on LangSmith for LLM observability tracing monitoring and evaluation for offline and oline evaluation
- For classical ML UC ensure V&V tests are done during the whole life cycle of the model (ML monitoring)
Test Automation
- Build scalable automation frameworks using PyTest integrated with CI/CD pipelines.
- Develop automated test suites for APIs microservices and AI pipelines.
- Perform sanity integration functional regression and UAT testing across QA UAT and Prod environments.
- Implement automation strategies: Keyword-driven Data-driven BDD Hybrid frameworks.
Good understanding of Data Science related metrics (for both classical ML and GenAI UC)Experience with AI/Chatbot products and testing AI applications.
Experience with Git GitHub Copilot and CI/CD pipelines.
Knowledge of Azure/AWS cloud environments.
Familiarity with MySQL PostgreSQL Oracle SQL Server NoSQL databases.
