Foundation Model Evaluation — Benchmark Models for Your Use Case

I will design and run a systematic evaluation of multiple foundation models against your specific use case — building a task-specific evaluation dataset, running all candidate models against the same test cases, scoring outputs on your defined quality dimensions, and delivering a comparative evaluation report with a clear model recommendation. Generic benchmarks (MMLU, HumanEval, etc.) are poor predictors of model performance on your specific task — a task-specific evaluation using your actual use case data and quality criteria is the only reliable way to know which foundation model will perform best in your production application.

The evaluation covers test case dataset creation from your use case, prompt template standardisation across models, automated and human scoring on your quality dimensions, cost-per-quality analysis, latency comparison, and a recommendation report with quantitative evidence.