Lead Engineer

Your Key Responsibilities:HPC Operations Leadership: Oversee daily operations of HPC infrastructure, including compute, GPU, storage, networking, and scheduler platforms (e.g., Slurm, Kubernetes).Performance & Optimization: Drive continuous optimization of system performance, availability, and resource utilization, minimizing downtime and operational risk.Technical Escalation: Serve as the primary escalation point for L2 support teams, ensuring rapid diagnosis and resolution of complex incidents and service requests.Monitoring & Observability: Continuously monitor system health and performance using observability platforms such as Prometheus, Grafana, and DCGM, proactively identifying issues.AI/ML Workload Enablement: Manage user environments for AI and ML workloads, including container orchestration (Docker, Kubernetes) and workflow platforms such as MLflow and Kubeflow.Scheduling & Resource Governance: Define and enforce scheduling policies, priorities, partitions, and quotas within Slurm and Kubernetes to ensure fairness, efficiency, and workload optimization.Incident Management & RCA: Lead root cause analysis (RCA) activities, produce post-mortem documentation, and implement preventive and continuous improvement actions.Automation & Tooling: Drive automation initiatives using scripting and infrastructure-as-code tools to improve reliability, repeatability, and operational efficiency.Mentorship & Knowledge Sharing: Provide technical leadership, mentorship, and guidance to junior engineers; contribute to skills development and operational best practices. Participate in on-call rotations as required.Security & Compliance: Ensure adherence to security, operational, and compliance policies; support audits and maintain documentation for change, incident, and access management processes.Desired Candidate ProfileLead Engineer HPC Operations, Core42, Abu Dhabi - UAEAbout UsCore42, a leader in AI-powered cloud and digital infrastructure, is driving transformative technology solutions globally. Leveraging advanced resources and partnerships, Core42 empowers clients to harness sovereign AI infrastructure, especially in sectors with stringent regulatory needs. With a mission to redefine digital transformation, we combine sovereign capabilities with scalable, high-performance compute infrastructure, positioning itself at the forefront of AI innovation in the Middle East and ; br>The OpportunityThe Lead Engineer HPC Operations plays a critical role in ensuring the stability, performance, and scalability of Core42 s high-performance computing platforms that power large-scale AI and machine learning workloads.This role is responsible for overseeing day-to-day HPC operations across compute, storage, networking, and scheduling layers, while driving automation, performance optimization, and operational excellence. You will act as a senior technical authority, collaborating closely with engineering, AI platform, security, and compliance teams, and mentoring operations engineers in a highly complex, globally distributed environment.What We re Looking For(a) Required Skills & QualificationsBachelor s or Master s degree in Computer Science, Engineering, or a related technical field.Minimum of 8 years experience in HPC operations, systems engineering, or DevOps roles, with at least 2 years in a leadership or ownership capacity.Advanced expertise in designing, configuring, operating, and optimizing complex HPC environments, including hardware, software, and storage systems.Hands-on experience managing Slurm clusters and/or Kubernetes-based platforms supporting AI/ML workloads.Deep knowledge of GPU resource management, workload scheduling, and performance tuning for AI and machine learning use cases.Strong proficiency with monitoring and observability tools such as Prometheus, Grafana, and DCGM.Advanced scripting and automation skills using Python, Bash, Ansible, and Terraform.Strong Linux administration skills (RHEL, CentOS, Ubuntu) and solid understanding of high-speed networking (RDMA, InfiniBand, RoCE) and storage technologies (NFS, Lustre, Ceph).(b) Preferred Skills & QualificationsExperience operating large-scale, multi-tenant AI or research computing platforms.Familiarity with MLOps frameworks and production ML pipelines.Strong documentation, communication, and cross-functional collaboration skills.Experience working in regulated or sovereign cloud environments.