Data Scientist, AIML Model Quality

Would you like to contribute to Machine Learning and Generative AI technologies Are you passionate about the integrity of the data that powers AI systems at scale Do you believe that trustworthy data is the foundation of every great model We truly believe it is!nnWe are defining what exceptional data quality looks like for machine learning across Wallet Payments and Commerce. As a Data Scientist AI/ML Model Quality you will build and maintain intelligent systems validation frameworks and monitoring pipelines that keep our data ecosystem healthy ensuring that every model we build is trained evaluated and deployed on data we can trust. Your work sits at the foundation of every ML feature that reaches hundreds of millions of work at the intersection of statistical rigor and production systems collaborating closely with ML Engineering Data Engineering Privacy and Legal teams. This unique opportunity puts you at the center of ML and AI quality owning the health of training and validation datasets defining and analyzing observability metrics to surface actionable product insights and leading telemetry analysis across GenAI workflows ensuring Apples financial features are built on the highest-quality data whether powering conventional ML models or the latest generative AI systems.n

The ideal candidate is a detail-obsessed data scientist who understands that model quality starts long before training it starts with the data. You have strong statistical instincts know how silent degradation and data drift manifest in production systems and can translate raw quality signals into insights that drive real decisions. nnYou will own the health of the data ecosystem that underpins ML and GenAI features across Wallet Payments and Commerce building validation frameworks defining observability metrics and leading telemetry analysis that keeps every model trained evaluated and monitored on data teams can trust. Your work sits at the foundation of every ML feature that reaches hundreds of millions of users.n

Curate analyze and maintain gold-standard ground-truth datasets for model evaluation and continuous validation across both ML and GenAI training data for systemic bias and fairness gaps prior to model deployment; establish ongoing analytical checks to catch bias introduced by data drift over track and report key data quality metrics completeness accuracy timeliness validity for engineering and leadership and define automated data quality rules and thresholds partnering with Data Engineering to ensure these checks are integrated into model development and CI/CD workflowsnnDefine and own ML observability metrics model performance output distributions training-serving skew silent degradation and feature drift translating raw production signals into actionable insights for engineering and product and develop observability dashboards and reporting workflows that give stakeholders a consistent real-time view of model health across both conventional ML and GenAI and analyze telemetry across GenAI workflows tracking quality signals such as output coherence latency task completion rates and regression degradation patterns and domain-specific failure modes in GenAI systems through systematic telemetry analysis translating findings into concrete recommendations for model and data teams.n

A Bachelors degree with exceptional hands-on experience in ML/AI model quality or applied research or a M.S or Ph.D in Machine Learning Computer Science Data Science Statistics Mathematics Engineering or a related quantitative field is strongly 3 years of experience in data science or a closely related analytical role with a strong focus on data quality model evaluation or ML observability in production in Python (Pandas NumPy Scikit-learn) and SQL for complex data analysis metric creation and querying and analyzing large-scale datasets using distributed computing frameworks (e.g. PySpark Spark or distributed SQL).nnSolid understanding of statistical methods hypothesis testing distribution analysis data drift detection and statistical process in defining and tracking ML model health metrics in production model performance monitoring feature drift detection and observability with GenAI or LLM systems including common quality failure modes output evaluation approaches and telemetry communication skills ability to translate complex data quality findings and model health risks into clear actionable insights for both engineering and non-technical stakeholde

Experience with data visualization and dashboarding tools (e.g. Tableau Apache Superset Databricks) to present complex ML with LLM evaluation frameworks (e.g. LangSmith) or techniques like with Bayesian or causal graph-based approaches to synthetic data with confidence calibration techniques and uncertainty with ML monitoring or observability platforms (e.g. MLflow Weights u0026 Biases or equivalent).nnExperience working with privacy-constrained data or under regulatory compliance frameworks (GDPR DMA).nnBackground in financial services fintech or consumer payment products.n