HPC Operations Engineer

We have an opening for an HPC Operations Engineer to develop deploy and maintain the tools automation and processes that keep Livermore Computing high-performance computing systems available and performing at high levels of reliability and performance. You will work in a challenging team-oriented environment applying technical knowledge of HPC systems and software development to engineer monitoring alerting and diagnostic solutions for Livermore Computings (LC) high-performance computing clusters storage systems networks and data center facilities. You will contribute to technical projects as a member of a technical team and collaborate with system administrators software developers and facilities engineers to improve the reliability and efficiency of LC operations. This position is in the Livermore Computing Division within the Computing Directorate.

This position will be filled at either level based on knowledge and related experience as assessed by the hiring team. Additional job responsibilities (outlined below) will be assigned if hired at the higher level.

This position requires full-time on-site presence due to the nature of the work.

You will 

• Develop deploy configure and maintain monitoring alerting and observability tools for LC HPC clusters storage systems high-speed networks and data center facility infrastructure.
• Create and maintain software tools and processes that streamline alarm response diagnostics and routine operational tasks using established methods and procedures.
• Configure and integrate data from building management and facility systems such as power and cooling with HPC system monitoring to provide a unified operational view under established technical direction.
• Share alarm response and monitoring duties with operations staff and analyze diagnose and resolve hardware software network and facility problems of limited complexity determine root cause and implement corrective actions in collaboration with system administrators vendors and facilities staff.
• Participate in the deployment testing and validation of new HPC systems and components and the decommissioning of retired systems.
• Develop and maintain technical documentation procedures and runbooks for operational tools and processes.
• Respond to customer problems and questions working with Hotline system administration and other LC groups to improve the quality of service for end users.
• Perform other duties as assigned.

Additional job responsibilities at the SES.2 level

• Design and develop moderately complex software tools and automations that improve the operation reliability and efficiency of LC computing and facility systems.
• Analyze system alarm and incident data to identify trends and implement moderately complex strategies to improve monitoring coverage alarm accuracy and operational processes.
• Lead discrete technical projects or work packages such as the deployment validation and decommissioning of HPC systems coordinating tasks with system administration facilities and other LC groups.
• Diagnose and resolve moderately complex to complex system and facility problems and serve as a technical escalation resource.
• Develop and deliver training and provide guidance and informal mentoring to less experienced staff.
• Independently prioritize assigned work across multiple concurrent operational and project tasks.

Qualifications :

• Ability to secure and maintain a U.S. DOE Q-level security clearance which requires U.S. citizenship.
• Bachelors degree in computer science computer engineering or related field or the equivalent combination of education and related experience.
• Fundamental knowledge of and experience with Linux/UNIX systems and enterprise data storage including installation configuration networking RAID concepts system logs and troubleshooting.
• Understanding of programming and scripting languages such as C/C Rust Go Python or bash/csh/ksh.
• Experience with version control and configuration management systems such as Git Ansible or CFEngine.
• Fundamental knowledge of system monitoring alerting and incident management practices and tools.
• Sufficient verbal and written communication and interpersonal skills necessary to collaborate effectively with system administrators software developers facilities staff LC staff and end users and to present and explain technical information.
• Ability to work all shifts including Owl shift (12am 8am) weekends and holidays.

Additional qualifications at the SES.2 level

• Comprehensive knowledge of HPC environments and technologies such as Linux clusters InfiniBand Slurm Lustre and archival storage systems.
• Proficient experience developing software tools and automation in C/C Python or similar languages and broad application of technical principles to resolve moderately complex system network and facility issues.
• Experience leading technical work packages or discrete projects and mentoring less-experienced staff and ability to work with limited direction in a dynamic environment with competing priorities.
• Relevant previous experience and demonstrated ability to perform independent technical work.

Qualifications We Desire

• Experience with monitoring and observability tools such as Grafana Prometheus Splunk or Nagios.
• Experience with data center utility infrastructure and building management systems including power distribution water distribution and cooling distribution units supporting HPC environments.
• Experience with virtual machines container technologies and orchestration such as Docker Podman or Kubernetes.
• Experience working in a Closed Area or other secure computing environment.

Pay Range

$121830 - $185544 annually

$121830 - $154500 annually for SES.1 level

$146340 - $185544 annually for SES.2 level

This is the lowest to highest salary we in good faith believe we would pay for this role at the time of this posting; pay will not be below any applicable local minimum wage.  An employees position within the salary range will be based on several factors including but not limited to specific competencies relevant education qualifications certifications experience skills seniority geographic location performance and business or organizational needs.

Additional Information :

#LI-Onsite

Position Information

This is a Career Indefinite position open to Lab employees and external candidates.

Why Lawrence Livermore National Laboratory

• Included in 2026 Best Places to Work by Glassdoor!
• Flexible Benefits Package
• 401(k)
• Relocation Assistance
• Education Reimbursement Program
• Flexible schedules (*depending on project needs)
• Our values - visit  Clearance

  This position requires a Department of Energy (DOE) Q-level clearance.  If you are selected we will initiate a Federal background investigation to determine if you meet eligibility requirements for access to classified information or matter. Also all L or Q cleared employees are subject to random drug testing.  Q-level clearance requires U.S. citizenship. 

  Pre-Employment Drug Test

  External applicant(s) selected for this position must pass a post-offer pre-employment drug test. This includes testing for use of marijuana as Federal Law applies to us as a Federal Contractor.

  Wireless and Medical Devices

  Per the Department of Energy (DOE) Lawrence Livermore National Laboratory must meet certain restrictions with the use and/or possession of mobile devices in Limited Areas. Depending on your job duties you may be required to work in a Limited Area where you are not permitted to have a personal and/or laboratory mobile device in your possession.  This includes but not limited to cell phones tablets fitness devices wireless headphones and other Bluetooth/wireless enabled devices.  

  If you use a medical device which pairs with a mobile device you must still follow the rules concerning the mobile device in individual sections within Limited Areas.  Sensitive Compartmented Information Facilities require separate approval. Hearing aids without wireless capabilities or wireless that has been disabled are allowed in Limited Areas Secure Space and Transit/Buffer Space within buildings.

  How to identify fake job advertisements

  Please be aware of recruitment scams where people or entities are misusing the name of Lawrence Livermore National Laboratory (LLNL) to post fake job advertisements. LLNL never extends an offer without a personal interview and will never charge a fee for joining our company. All current job openings are displayed on the Career Page under Find Your Job of our website. If you have encountered a job posting or have been approached with a job offer that you suspect may be fraudulent we strongly recommend you do not respond.

  To learn more about recruitment scams:  Employment Opportunity

  We are an equal opportunity employer that is committed to providing all with a work environment free of discrimination and harassment. All qualified applicants will receive consideration for employment without regard to race color religion marital status national origin ancestry sex sexual orientation gender identity disability medical condition pregnancy protected veteran status age citizenship or any other characteristic protected by applicable laws.

  Reasonable Accommodation

  Our goal is to create an accessible and inclusive experience for all candidates applying and interviewing at the Laboratory.  If you need a reasonable accommodation during the application or the recruiting process please use our online form to submit a request. 

  California Privacy Notice

  The California Consumer Privacy Act (CCPA) grants privacy rights to all California residents. The law also entitles job applicants employees and non-employee workers to be notified of what personal information LLNL collects and for what purpose. The Employee Privacy Notice can be accessed here.

  
  

  Remote Work :

  No

  
  

  Employment Type :

  Full-time