Member of Technical Staff (Evals)

About Jarmin

Jarmin builds AI Employees for human-level roles. An AI Employee learns its responsibilities understands your data embeds into your systems and becomes part of your teams day-to-day workflow. Companies hire AI ML Engineers Account Executives Paralegals Financial Analysts Compliance Analysts and more.

Our founding team is from Meta Superintelligence Labs (Staff Engineers) Apple (Machine Learning) AWS (scaling infrastructure) Lockheed Martin (Research) and JPMorganChase with experience building AI and ML systems and automating the complex work of researchers data scientists and ML engineers.

We are a Y Combinator Fall 2025 company in San Francisco.

About the role

As a Member of Technical Staff on Evals youll own how we measure AI Employee performance on the job.

Areas you may work in

• Evaluation systems: datasets offline replay scorers and regression alerts

• Feedback loops for models and agents

• Autonomous failure analysis

• Defining what good degraded and failed runs look like and alerting on them

You may be a good fit if you

• Have built evaluation or measurement systems such as AI evals experimentation ranking or search quality

• Can turn ambiguous quality questions into concrete metrics and decisions

• Have strong software engineering fundamentals and ship production systems

• Are comfortable where there are few established patterns

• Want to work in person in San Francisco

Even better

• Youve shipped LLM systems to production and learned where they break

• You have strong data instincts and work well with researchers

• You contribute to open source projects

How we work

• Our core team is in person resourceful and building the future we want to live in.

• We give feedback early and often so you always know where you stand and where youre headed.

Interested

If you have a question before applying write to .

Required Experience:

Staff IC