Software Engineer — AI Benchmark & Test Developer (Remote, EST)

Experienced Software Engineer needed to develop AI coding benchmarks: analyze real GitHub PRs, implement robust fixes, author rigorous tests, and craft clear LLM evaluation prompts within reproducible Docker environments. Follow a disciplined 7-step workflow—review PRs, run and validate repos in Docker, implement golden solutions on a dedicated branch, write fail-to-pass tests, compose metadata and prompt variants, and open a three-commit PR. Must be available during EST business hours, proficient in multiple languages (e.g., Python, JavaScript/TypeScript, Go, Ruby, Java), skilled with Git, testing frameworks, Docker, and CI. Ideal candidates practice meticulous TDD, contribute to open source, and communicate proactively, producing high-quality, reproducible deliverables validated by automated checks.