Jobs

Callosum is seeking a Research Engineer, Benchmarking to build and own a critical system for evaluating algorithmic strategies in multi-step LLM work. This role involves designing and implementing a reproducible, contamination-controlled benchmarking harness that measures task success, quality, and robustness based on real execution. The ideal candidate will have a strong research background in LLM/agent evaluation, demonstrated by publications, and possess the engineering skills to build and curate these complex systems. You will play a pivotal role in determining which approaches ship, influencing customer engagements, and leading external benchmark co-publications.