Mindrift
Freelance Agent Evaluation Engineer
LATAMRemotePosted Today
Full TimeMidRemote
See how this job matches your profile
Sign in for an AI-powered fit score, breakdown, and a tailored resume.
Job Description
Please submit your CV in English and indicate your level of English proficiency.Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evalua
Key Highlights
- Build realistic developer environments – a virtual company with codebase, infrastructure, and context (tickets, docs, conversations) that forms a believable development history
- Design tasks from intermediate states of these environments – craft the prompt, define what “solved” means, and ensure the task is solvable by an AI agent
- Write tests that verify agent solutions – accept all valid approaches and reject incorrect ones, neither too strict nor too lenient
- Iterate on tasks and tests based on QA feedback – review agent solutions, analyze failures, and refine until the evaluation is fair and robust
- Not data labeling
Qualifications
Required Qualifications
- 5+ years in software development
- Core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, Redis
- Experience writing tests (functional, integration)
- English proficiency – B2+
Skills & Technologies
PythonFastAPIJavaScriptTypeScriptReactDockerKafka
About the Company
Mindrift
View company profile →
Interested in this role?
Sign in or create a free account to see how this job matches your skills, apply with one click, and let our AI tailor your resume.
Sign in to applyAI-powered resume optimization
Save and track your applications
Job Details
Employment Type
Full Time
Experience Level
Mid
Location
LATAM
Work Mode
Remote
Posted
Today