Data Engineer, CPU & Storage

About the Team

OpenAIs Industrial Compute organization is building the worlds most advanced AI infrastructure ecosystem. Through a combination of strategic partnerships and self-built campuses we are scaling the compute storage and networking platforms that power frontier AI training and inference.

The Scaling Analytics team builds the data and software systems that help Industrial Compute understand plan and operate infrastructure at global scale. We work across capacity hardware storage infrastructure software and operational systems to connect fragmented sources of infrastructure data and make that information reliable and usable for engineering and planning.

As OpenAIs infrastructure footprint grows CPU and storage data increasingly spans internal platforms vendor systems APIs databases object storage capacity management systems and operational tooling. Building reliable connections across these environments is critical to understanding available capacity utilization fleet state and infrastructure growth.

About the Role

We are seeking a Data Engineer to build the data systems and integrations that connect OpenAIs CPU storage and supporting infrastructure platforms.

This role sits at the intersection of data engineering and backend software engineering. Rather than focusing primarily on traditional analytical pipelines you will build the software and integrations required to collect normalize and make infrastructure data available across a heterogeneous set of systems.

CPU and storage data may originate from internal infrastructure platforms vendor APIs databases object storage capacity systems and operational services. You will determine how to reliably connect these systems and where those integrations should livewhether within an existing infrastructure service an orchestration framework a scheduled workload or a purpose-built application.

You will work closely with Infrastructure Engineering Capacity Engineering Storage Hardware Operations and Infrastructure Software to create a reliable data foundation for understanding CPU and storage capacity utilization inventory and operational state.

Success in this role requires strong software engineering fundamentals comfort working across unfamiliar systems and the ability to design pragmatic solutions for moving and reconciling data across infrastructure environments.

Responsibilities

• Build and maintain software integrations that collect CPU storage capacity inventory and operational data from internal systems and external vendors.

• Connect heterogeneous data sources including APIs relational databases object storage infrastructure services capacity management platforms and vendor systems.

• Design reliable ingestion mechanisms for both structured and semi-structured infrastructure data including batch scheduled and API-driven workflows.

• Determine the appropriate architecture and execution environment for new data integrations including existing backend services capacity systems orchestration frameworks scheduled workloads or purpose-built services.

• Build backend services jobs and tooling that normalize infrastructure data and make it available through databases offline tables and analytical datasets.

• Develop reliable interfaces between operational systems and downstream analytics planning and engineering workflows.

• Partner with infrastructure and software engineering teams to understand source systems APIs data ownership schemas and operational constraints.

• Design data models that reconcile information across multiple systems and establish consistent representations of CPU capacity storage capacity inventory utilization and infrastructure state.

• Build mechanisms for detecting missing stale inconsistent or incorrect infrastructure data and resolving discrepancies between source systems.

• Improve the reliability observability and maintainability of data integrations through testing monitoring logging and automated validation.

• Support infrastructure planning and operational analysis by ensuring critical CPU and storage datasets are accurate timely and accessible.

• Develop reusable patterns for onboarding new infrastructure and vendor data sources as OpenAIs compute footprint and partner ecosystem expand.

Qualifications

• Bachelors degree in Computer Science Engineering or a related technical field or equivalent practical experience.

• 4 years of experience in software engineering data engineering backend engineering infrastructure engineering or a related technical discipline.

• Strong Python programming skills and experience building production software services automation or data integrations.

• Strong SQL skills and experience working directly with relational databases and large operational datasets.

• Experience integrating systems through REST APIs SDKs databases object storage messaging systems or other programmatic interfaces.

• Experience building reliable batch scheduled or asynchronous workloads in production environments.

• Strong understanding of software engineering fundamentals including testing debugging version control observability and maintainable system design.

• Experience reasoning about data schemas system boundaries data ownership consistency and failure modes across distributed systems.

• Ability to navigate unfamiliar codebases and infrastructure environments and determine how new functionality should integrate with existing systems.

• Experience partnering closely with infrastructure backend platform or systems engineering teams.

Preferred Skills

• Experience building backend or data systems that integrate information across multiple internal services databases APIs and external platforms.

• Experience working with infrastructure capacity compute storage fleet management inventory or hardware lifecycle data.

• Familiarity with CPU platforms storage systems distributed systems cloud infrastructure or large-scale hardware environments.

• Experience building integrations with third-party or vendor APIs where schemas interfaces data quality and availability may vary across providers.

• Experience working with workflow orchestration or scheduling systems such as Airflow or comparable frameworks while understanding when orchestration is appropriate versus building functionality into an existing service.

• Experience with relational databases object storage offline tables and analytical data systems with the ability to move comfortably between operational and analytical environments.

• Experience designing reconciliation and data-quality mechanisms across systems that may contain overlapping or conflicting representations of infrastructure state.

• Strong backend engineering instincts including API design service integration reliability observability and production debugging.

• Ability to evaluate multiple implementation approaches and choose pragmatic architectures based on reliability maintainability ownership and operational complexity.

• Experience working in large-scale cloud hyperscale infrastructure AI infrastructure or similarly complex distributed environments.

About OpenAI

OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core and to achieve our mission we must encompass and value the many different perspectives voices and experiences that form the full spectrum of humanity.

We are an equal opportunity employer and we do not discriminate on the basis of race religion color national origin sex sexual orientation age veteran status disability genetic information or other applicable legally protected characteristic.

For additional information please see OpenAIs Affirmative Action and Equal Employment Opportunity Policy Statement.

Background checks for applicants will be administered in accordance with applicable law and qualified applicants with arrest or conviction records will be considered for employment consistent with those laws including the San Francisco Fair Chance Ordinance the Los Angeles County Fair Chance Ordinance for Employers and the California Fair Chance Act for US-based candidates. For unincorporated Los Angeles County workers: we reasonably believe that criminal history may have a direct adverse and negative relationship with the following job duties potentially resulting in the withdrawal of a conditional offer of employment: protect computer hardware entrusted to you from theft loss or damage; return all computer hardware in your possession (including the data contained therein) upon termination of employment or end of assignment; and maintain the confidentiality of proprietary confidential and non-public addition job duties require access to secure and protected information technology systems and related data security obligations.

To notify OpenAI that you believe this job posting is non-compliant please submit a report through this form. No response will be provided to inquiries unrelated to job posting compliance.

We are committed to providing reasonable accommodations to applicants with disabilities and requests can be made via this link.

OpenAI Global Applicant Privacy Policy

At OpenAI we believe artificial intelligence has the potential to help people solve immense global challenges and we want the upside of AI to be widely shared. Join us in shaping the future of technology.

Required Experience:

IC