AI Engineer

Solumn AI helps frontier AI labs understand and improve how their models behave in challenging, real-world environments. We build the evaluation environments, research infrastructure and methods that test AI systems against frontier labs’ safety and alignment standards, and we turn that research into evaluations and training data that measurably improve model behaviour.
About the jobWe’re looking for AI Engineers who combine solid engineering with research judgement. You’ll investigate how frontier models behave, form hypotheses, design experiments, and build the environments, benchmarks and datasets that produce reliable evidence. This goes beyond implementing existing benchmarks: you’ll help decide what to measure, how to test it rigorously, and how to turn open-ended questions into reproducible systems.
How you will contributeResearch and experimental design: turn safety questions into hypotheses, controlled experiments, baselines and clear measurement criteriaEvaluation environments: build realistic agentic and tool-use environments, execution harnesses, trajectory collection and scalable experiment workflowsBenchmarks and evaluation quality: create benchmark datasets, robust graders and verifiers, and QA that catches shortcuts and misleading scoresModel behaviour analysis: investigate safety-relevant behaviours and failure modes, and tell genuine model limitations apart from environment, rubric or verifier defectsEngineering standards: improve reproducibility, test coverage, documentation and internal tooling across research projectsWho you areYou write strong Python and have built and debugged APIs, pipelines, tools or other complex systemsYou understand LLMs, transformers, inference, tool use and agentic architectures; post-training or RL experience is a plusYou can read technical work critically, design experiments and interpret results; a publication record is not requiredYou care about evaluation validity, confounders, reproducibility and verifier robustnessYou’re comfortable with ambiguity, learn fast, work independently, and are genuinely interested in AI safety and alignmentBonus if you’ve worked onLLM evaluations, benchmark development or model behaviour analysisAgentic, multi-agent or tool-use systemsAI safety, red teaming, adversarial testing or security researchML research engineering, experiment infrastructure or automated gradingFull-time
· 6-day work week
· 100% Remote