All roles

NEUTRAL / CAREERS

HPC / GPU Systems Engineer

Investigate difficult GPU and HPC incidents, test the evidence and help teams recover productive compute time.

Apply for this role

Why this role exists

Large AI workloads depend on GPU nodes, fabrics, storage and schedulers working together. This role turns ambiguous failures into an evidence-backed assessment and a clear next action for the responsible team.

The work

  • Investigate GPU node faults across drivers, firmware, hardware and runtime signals, using raw logs and tools such as nvidia-smi and DCGM when summaries are incomplete.
  • Isolate InfiniBand or RoCE fabric faults at the link, port, transceiver or cabling level and prepare useful evidence for the provider or vendor.
  • Trace storage data-path, Linux and scheduler issues that affect multi-GPU workloads.
  • Review diagnostic findings critically, correct false leads and improve rules and runbooks from real incidents.
  • Plan changes in live environments with explicit risk assessment, validation and rollback steps.
  • Take part in incident escalation and an on-call rotation when the scope and schedule are agreed.

What you’ll bring

  • Hands-on investigation of GPU, HPC or large-scale Linux infrastructure incidents.
  • Working knowledge of NVIDIA drivers and runtime signals, and RDMA fabrics such as InfiniBand or RoCE.
  • Experience operating Slurm or Kubernetes workloads and reading telemetry without treating every graph as ground truth.
  • Clear written and spoken communication across engineers, customers and infrastructure partners.
  • Sound judgment when changing business-critical systems.

Useful experience

  • Python or Bash operational tooling
  • Infrastructure automation
  • Ceph or parallel file systems
  • Data centre operations

How we work

We take responsibility from first signal to outcome, build for scale and act when compute hours are at risk. Read more about how we work.

Interested?

Tell us what you have worked on and what you want to learn next.

Apply for this role

Explore more roles

See all open roles →