Software Engineer ML Infrastructure

San Francisco CA

Series A2
Total Funding: Approximately $28M

Fully on-site 5 days per week in the San Francisco office on Market Street.

$250000$300000 USD annually.

Competitive equity.

Open to visa transfers and new visa sponsorships including OPT H-1B transfers new H-1B and TN.

Full-time

1

The company combines AI technology with a radiology practice and works with major medical institutions across the United States. It has served more than 300000 patients and is working toward reading 1% of all daily X-rays in the US by year-end.

The platform is built around multimodal medical imaging including X-rays MRI scans CT scans echocardiograms and ultrasounds. The company has access to more than 250000 unique radiology images for model training and has secured its first customer through a signed $300000 annual contract.

Backed by Bedrock Capital Trust Ventures and Jack Altman the company has raised approximately $28M and is operating as a small fast-moving team.

The engineering and research teams work closely together with a strong emphasis on building foundational AI systems and infrastructure that allow researchers to experiment train evaluate and deploy models efficiently.

• Design and build the core infrastructure required to train large foundation models at scale.

• Own distributed training systems for large-scale medical imaging datasets.

• Build infrastructure that supports long-context parallelism and checkpointing for volumetric CT and MRI training.

• Ensure training workloads are scalable efficient and reliable.

• Identify and remove system bottlenecks that slow down research and experimentation.

• Design high-throughput data loading and preprocessing systems for large volumetric and multimodal datasets.

• Optimize data pipelines to keep GPUs fully utilized during model training.

• Build efficient workflows for handling large-scale medical imaging data.

• Improve data throughput and preprocessing performance to support faster experimentation.

• Build the reinforcement learning training stack for foundation-model and post-training workflows.

• Develop high-throughput rollout generation systems.

• Build and manage reward-model serving infrastructure.

• Implement experience collection systems to support large-scale training.

• Enable researchers to run online multi-reward reinforcement learning at scale.

• Own inference systems that take models from experimentation into production.

• Build efficient model-serving infrastructure.

• Contribute to production serving and deployment pipelines.

• Work on systems that support reliable and scalable model inference.

• Help ensure models can move from research environments into production applications.

• Develop evaluation infrastructure to support model experimentation and research.

• Build reliable systems for testing and evaluating models throughout the development lifecycle.

• Work closely with researchers to understand their workflows and anticipate evolving infrastructure needs.

• Make experimentation and training faster and more reliable so researchers can focus on scientific work.

• Work directly with researchers to understand their training and experimentation workflows.

• Prototype new ideas alongside researchers and translate them into production-ready systems.

• Own end-to-end delivery from experimentation through deployment and monitoring.

• Collaborate with the backend team on model rollout canary deployments and monitoring.

• Help establish the infrastructure foundations as an early member of the engineering team.

• Strong software engineering background with experience in ML infrastructure distributed systems or large-scale model training.

• Experience building infrastructure for training serving or evaluating machine learning models.

• Ability to work closely with research teams and translate research requirements into production-ready systems.

• Experience with distributed training and high-performance computing environments.

• Comfortable owning infrastructure end to end in a small fast-moving startup.

• Able to operate autonomously and make technical decisions with limited management overhead.

• Strong problem-solving skills and ability to identify and remove infrastructure bottlenecks.

The intake discussion indicates flexibility on years of experience for ML Infrastructure candidates. Prioritize technical depth relevant infrastructure ownership and ability to execute independently over a rigid years-of-experience requirement.

Programming and ML Frameworks

• Strong Python experience.

• Hands-on experience with PyTorch and/or JAX.

• Ability to work with research-oriented machine learning code and production systems.

Distributed Training

• Experience with distributed model training.

• Familiarity with FSDP DeepSpeed and/or Megatron.

• Understanding of distributed training architecture parallelism and checkpointing.

• Ability to optimize training efficiency across large-scale workloads.

Data Infrastructure

• Experience building high-throughput data loading and preprocessing systems.

• Understanding of GPU utilization and performance bottlenecks.

• Experience handling large datasets and complex data pipelines.

• Familiarity with multimodal or volumetric data is valuable.

Inference and Model Serving

• Experience with vLLM and/or SGLang.

• Familiarity with TensorRT and Triton.

• Understanding of model inference serving and deployment.

• Ability to build efficient production model-serving systems.

Infrastructure and Systems

• Experience with Kubernetes and Docker.

• Strong understanding of distributed infrastructure and production deployment environments.

• Familiarity with CUDA and GPU-based workloads.

• Ability to build reliable infrastructure for experimentation training and production.

• Experience building or supporting reinforcement learning infrastructure.

• Familiarity with high-throughput rollout generation.

• Experience with reward-model serving or experience collection.

• Understanding of online reinforcement learning workflows.

• Ability to support multi-reward RL training at scale.

• Can own infrastructure projects from initial design through deployment and monitoring.

• Comfortable working directly with researchers to understand and solve infrastructure challenges.

• Able to prototype quickly and turn experimental systems into reliable production code.

• Takes initiative in identifying performance and scalability bottlenecks.

• Comfortable operating in a small engineering team with minimal management layers.

• Strong ability to prioritize and execute independently.

• Strong communication skills when working with research and engineering teams.

• Comfortable translating research requirements into infrastructure solutions.

• Able to explain technical decisions and performance trade-offs.

• Collaborative and proactive when debugging complex systems.

• Comfortable working in a high-ownership startup environment.

• Experience training large foundation models.

• Experience with multimodal models.

• Experience working with medical imaging data including CT MRI X-rays or other imaging modalities.

• Experience with volumetric data processing.

• Experience with long-context parallelism and distributed checkpointing.

• Experience building online reinforcement learning infrastructure.

• Experience with GPU optimization and CUDA.

• Background at a strong AI research lab infrastructure company or technically demanding startup.

• Experience across both Big Tech and early-stage startup environments.

• Base salary of $250000$300000 USD annually.

• Competitive equity.

• Full-time position.

• On-site work 5 days per week in San Francisco.

• Visa transfers and new sponsorships supported including OPT H-1B and TN.

• Work on foundational ML infrastructure for a multimodal AI platform focused on healthcare and medical imaging.

• Own distributed training reinforcement learning inference and evaluation systems end to end.

• Work directly with researchers and influence how large models are trained and deployed.

• Build infrastructure for large-scale medical imaging datasets including CT MRI X-rays and other modalities.

• Join a small engineering team with significant ownership and minimal management layers.

• Help establish core ML infrastructure at an early stage with direct impact on research velocity and production capabilities.

• Work on a product with an existing customer contract and access to a substantial medical imaging dataset.