Find your next AI role

2 roles/ search · ↑ ↓ browse · ↵ open

T
Senior Software Engineer, AI Evaluation Infrastructure
Turing Remote (US)
Not specifiedFound by AICareersOnly. Our team found this opening on the company's official careers page. You apply directly with the employer.New
Engineeringdjangobenchmarkspostgresql4 hours ago
C
AI-Native Software Engineer, New Grad
Clara Ciudad de México, México
Not specifiedFound by AICareersOnly. Our team found this opening on the company's official careers page. You apply directly with the employer.
Trainee programsjavanew-gradpython2 weeks ago

Senior Software Engineer, AI Evaluation Infrastructure

at TuringRemote (US)

Apply now

Not specifiedFound by AICareersOnlyRemoteFull-timeSeniorEngineering

AI Signal

Job overview

About the role

Turing is hiring a senior backend engineer to build the infrastructure frontier AI labs use to evaluate coding agents and language models, including benchmark datasets, large-scale agent evaluation runs and automated task quality checks. You would own features end to end, from data model and background jobs to the user-facing screens.

What you'll do

Design and ship features across the stack: data model, API, background workers and UI when needed.
Run agent trials in sandboxed environments, score results and aggregate thousands of runs into trustworthy outputs.
Strengthen benchmark quality with verifiers, rubric-based grading and LLM-as-a-judge approaches.
Integrate new models, agents and benchmarks in a generalizable way, avoiding special cases.
Keep the system fast and reliable with efficient queries, safe migrations and solid concurrency; review teammates' pull requests.

Requirements

Typically 5+ years building and operating production backend systems, owning a service from design through on-call.
Strong Python, plus experience with Django or similar, PostgreSQL and a task queue such as Celery.
Hands-on LLM evaluation experience: building benchmark tasks, scoring agent trials, comparing results (Harbor, Terminal-Bench, SWE-bench or Inspect a plus).
Judgment to spot ambiguous instructions, gameable verifiers and leaked answers.
Comfort touching a TypeScript/React frontend and clear written communication.

djangobenchmarkspostgresqldockerllm-evaluationreactpythoncelery

Posted 4 hours ago

Find your next AI role

Find your next AI role

Senior Software Engineer, AI Evaluation Infrastructure

AI-Native Software Engineer, New Grad