Let's begin! Senior QA AI Engineer

At Moodys we unite the brightest minds to turn todays risks into tomorrows opportunities. We do this by striving to create an inclusive environment where everyone feels welcome to be who they arewith the freedom to exchange ideas think innovatively and listen to each other and customers in meaningful ways. Moodys is transforming how the world sees risk. As a global leader in ratings and integrated risk assessment were advancing AI to move from insight to actionenabling intelligence that not only understands complexity but responds to it. We decode risk to unlock opportunity helping our clients navigate uncertainty with clarity speed and confidence.

If you are excited about this opportunity but do not meet every single requirement please apply! You still may be a great fit for this role or other open roles. We are seeking candidates who model our values: invest in every relationship lead with curiosity champion diverse perspectives turn inputs into actions and uphold trust through integrity.

Skills and Competencies

• 5-7 years of experience in software quality engineering with hands-on expertise testing Java-based backend services and RESTful application programming interfaces
• Proficiency with test automation frameworks such as Selenium TestNG JUnit RestAssured or Cucumber for user interface and application programming interface testing
• Strong understanding of relational databases including PostgreSQL and Microsoft SQL Server to perform data validation and verify data integrity across test scenarios
• Hands-on experience testing Anthropic Claude integrated features including large language model output validation semantic evaluation techniques hallucination detection and streaming application programming interface behavior
• Familiarity with Model Context Protocol and agentic artificial intelligence testing patterns including tool-use accuracy context management and deterministic versus non-deterministic output validation
• Experience with artificial intelligence security testing including prompt injection testing jailbreak validation sensitive data leakage checks and application reliability and performance assessments
• Strong understanding of software design principles and microservices architectures with the ability to develop effective test strategies and identify system-level edge cases
• Deep expertise in artificial intelligence with a track record of implementing advanced AI solutions to drive strategic transformation and operational efficiency. Strong experience using AI tools to lead innovation initiatives. Demonstrated leadership in managing AI-related risks ensuring ethical governance and fostering a culture of responsible AI adoption across the organization.

Education

• Bachelor or Master of Science in Computer Science Mathematics Physics or a related field

Responsibilities

• Design develop and maintain automated test suites for Java-based backend services and RESTful application programming interfaces to ensure high software quality across releases
• Own the test strategy for Anthropic Claude integrated features including large language model output validation regression baselines tool-call accuracy verification context management validation and end-to-end integration coverage
• Build and maintain continuous integration integrated evaluation frameworks for non-deterministic artificial intelligence outputs including rubric-based scoring automated prompt regression suites and behavioral drift detection
• Collaborate with product owners software engineers and machine learning engineers to define testable requirements review acceptance criteria and ensure high-quality delivery across agile releases
• Perform database validation testing against PostgreSQL and Microsoft SQL Server environments to verify data integrity transformation logic and backend workflow accuracy
• Conduct artificial intelligence specific security testing including prompt injection adversarial input validation jailbreak testing and sensitive data leakage assessments aligned with Moodys governance standards
• Define and enforce synthetic or masked test data standards for artificial intelligence feature testing to ensure appropriate handling of confidential and regulated information
• Monitor and report quality performance and artificial intelligence observability metrics including token consumption streaming latency tool-call success rates model error rates and reliability indicators
• Integrate automated testing into continuous integration and continuous delivery pipelines ensuring quality gates and deployment controls are effective and actionable
• Conduct test code reviews mentor quality engineering team members perform exploratory testing and drive continuous improvement of testing practices and processes
• Support performance and load testing of cloud-native software-as-a-service platforms on Amazon Web Services and Microsoft Azure to validate scalability availability and reliability requirements
• Maintain clear test documentation traceability matrices defect analysis reports and quality metrics to ensure transparency and alignment across stakeholders

About the Team

Moodys is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race color religion sex national origin disability protected veteran status sexual orientation gender expression gender identity or any other characteristic protected by law.

Candidates for Moodys Corporation may be asked to disclose securities holdings pursuant to Moodys Policy for Securities Trading and the requirements of the position. Employment is contingent upon compliance with the Policy including remediation of positions in those holdings as necessary.