Tag: Self-Correction

Sep 20
Reflection and Verification Accuracy: Benchmarking Critic Agents Against Real Implementation Errors

In the effort to overcome the brittleness of single-pass generation, multi-agent frameworks rapidly embraced iterative self-correction. Frameworks like Reflexion, Actor-Critic topologies, and debate loops introduced secondary evaluator agents—commonly designated as Critic Agents, Verifiers, or Self-Reflection modules. The design intent was straightforward: when a primary worker agent produces code, alters a database, or drafts a response, […]

Sep 20
InterCode: Evaluating Interactive Code Execution and Self-Correction Across Bash, Python, and SQL

In early code generation evaluations, programming capability was treated as a static, single-turn translation problem. Benchmarks like HumanEval, MBPP, and Spider provided a model with a clean natural-language docstring or schema, collected an unbroken script or SQL string in a single forward pass, and tested the output against static unit tests. The evaluation assumed that […]