This task can be performed using Autoagent
Autoagent: autonomous harness engineering for smarter, faster testing
Best product for this task
Autoagent
oss
AutoAgent is an open-source framework for autonomous harness engineering where a meta-agent rewrites an LLM agent’s harness, runs Harbor benchmarks, and hill-climbs on scores. You define the loop in program.md and let it iteratively optimize prompts, tools, and orchestration.
What to expect from an ideal product
- AutoAgent creates a meta-agent that automatically rewrites and improves your testing setup without manual intervention
- The framework runs Harbor benchmarks continuously and uses hill-climbing algorithms to boost performance scores over multiple iterations
- You simply define your testing loop in a program.md file and the system handles all the behind-the-scenes optimization work
- The tool fine-tunes prompts, adjusts available tools, and reorganizes workflow steps based on real performance data
- Each testing cycle feeds results back into the system, creating a self-improving harness that gets smarter with every run
