Skip to content

Add submission: 20260601_gongent_gpt-5.5-high - #13

Open
qucheng-gong wants to merge 1 commit into
ProgramBench:mainfrom
qucheng-gong:add-20260601_gongent_gpt-5.5-high
Open

Add submission: 20260601_gongent_gpt-5.5-high#13
qucheng-gong wants to merge 1 commit into
ProgramBench:mainfrom
qucheng-gong:add-20260601_gongent_gpt-5.5-high

Conversation

@qucheng-gong

Copy link
Copy Markdown

Registers GPT-5.5 (high) (OpenAI) + Gongent.

  • instances attempted: 200

Source: https://github.com/qucheng-gong/20260601_gongent_gpt-5.5-high
Commit: 899099297bdb63699e282d78b490fc85ffe43855

Tier-0 verified (programbench submit verify .). Leaderboard scores are recomputed from _stats/score.json.

@john-b-yang

Copy link
Copy Markdown
Collaborator

Hi @qucheng-gong thanks so much for your submission, have been auditing it and it looks great!

Would you be interested in writing a short blog post to be hosted at our blog page: https://programbench.com/blog/, that describes your submission?

I was reading your description of gongent, it seems very interesting, but i had a bit of trouble understanding. Would you mind providing some sort of visualization of your method? For instance, i didn't quite understand if the builder or adversary (a) are both agents? (b) are in direction communication? or are they passing messages via a queue / share dfile?

@qucheng-gong

Copy link
Copy Markdown
Author

Thanks for the audit — and yes, I'd be happy to write the blog post.

Short answer to the architecture questions:

(a) Yes, both are agents — two separate instances of the same mini-swe-agent class, differing only in prompt. The builder is stock mini-swe-agent with its vanilla prompt, unmodified: given the black-box binary, write a reimplementation. The adversary is a separate loop whose only job is to probe the gold binary and turn what it observes into a test suite.

(b) No — they never talk to each other. Separate agents, separate histories, no shared context. The builder never sees the adversary's reasoning, and the adversary never sees the builder's code.

(c) They're coupled only through the workspace and the test suite, with the orchestrator in between. The adversary runs once up front to setup the test suites; after that the loop is just builder → compare → builder until the reimplementation matches the suite (or a hard cap). So it's less "two agents collaborating" and more "one agent writes the exam from the reference binary, the other agent sits it, repeatedly."

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants