Add Human-SL KGS-rank ladder (gtp_human<rank>.cfg, 9d→20k) + tunehuman subcommand#1209
Add Human-SL KGS-rank ladder (gtp_human<rank>.cfg, 9d→20k) + tunehuman subcommand#1209ChinChangYang wants to merge 1 commit into
Conversation
…n subcommand
Adds a `tunehuman` subcommand and a complete set of GTP configs that make KataGo
(with the Human-SL net) play at a chosen amateur rank from 9d down to 20k, where
each consecutive rank is exactly 1 KGS rank (1 stone) apart.
tunehuman subcommand (cpp/command/tunehuman.cpp, cpp/program/humansltuner.{cpp,h}):
- Plays in-process candidate-vs-baseline games and calibrates
humanSLChosenMovePiklLambda (the strength dial) to a target winrate at fixed
visits, reading the raw winrate (robust to the steep, ceiling-biased λ curve).
- -komi / -cand-color flags for the KGS 1-rank handicap (weaker rank as Black,
komi 0.5, no color alternation); inherits the baseline config's ruleset so
tuning is scored exactly like deployed play.
- Resumable per-round checkpointing (-resume-file) to survive long runs.
- Unit tests in cpp/tests/testhumansltuner.cpp (run via `katago runtests`).
The ladder (cpp/configs/gtp_human{9d..1d,1k..20k}.cfg):
- 29 configs. The 9d anchor is even-game parity vs the modern rank_9d profile;
every weaker rank is tuned so it (Black, komi 0.5) is an even game (50%) vs the
rung above it, with the 95% Wilson CI inside [40%, 60%]. Japanese rules.
- λ rises 0.045 (9d) → 1.223 (20k); the deep-kyu rungs need near-pure-human play
because the Human-SL net's rank profiles compress at the weakest end.
docs/HumanSL_Rank_Ladder.md documents the method, ruleset rationale, reproduction
commands, the full results table (λ / win rate / 95% CI / games per rung), and
findings. ladder_step.sh + tune_decide.py + tune_{lambda,maxvisits}.sh are the
automated sequential root-finder harness that produced the ladder.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L2nqY5X9rSVpH65nWHCPaF
…<year>) Consolidate rank_*/preaz_* into unified <rank> keys driven by the empirically-tuned KGS-rank ladder from lightvector/KataGo PR lightvector#1209 (preaz_<rank> + tuned humanSLChosenMovePiklLambda), and rename proyear_<year> to "Pro <year>" derived from the 9d config with lambda 0.06. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WoQvc49FJf5btyZsHLawb3
TDD plan: rewrite HumanSLModel around clean menu keys (lightvector#1209 preaz_<rank> ladder + Pro <year>), legacy input normalization, picker wiring, and a 3-platform verification pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WoQvc49FJf5btyZsHLawb3
Consolidate rank_*/preaz_* into single <rank> keys driven by PR lightvector#1209's tuned KGS-rank ladder (preaz_<rank> + per-rank humanSLChosenMovePiklLambda), rename proyear_<year> to 'Pro <year>' (9d-derived, lambda 0.06), and add legacy input normalization. AI profile behavior preserved. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WoQvc49FJf5btyZsHLawb3
Human-profile engine moves search a fixed 400 visits (lightvector#1209 calibration point) instead of the time budget, so a rank plays at its calibrated strength; "Time per move" applies only to the AI profile (human profiles get an Engine/Human toggle). Continuous analysis stays unbounded. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WoQvc49FJf5btyZsHLawb3
Add GtpCommandBuilder.searchBudgetCommands: AI profile stays time-bounded with unbounded visits; human rank/pro profiles use a fixed 400 visits (lightvector#1209 calibration point) + a safety time cap, ignoring Time per move. genMoveAnalyzeCommands now takes the effective profile. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WoQvc49FJf5btyZsHLawb3
The redesigned human-SL parameters played too strong and slow. Revert the engine parameters in HumanSLModel to the pre-redesign (4be0ad7~1) level-based formulas while keeping the new clean-key UI/structure (allProfiles, canonicalProfile, legacy normalization, preaz_/proyear_ mapping): - winLossUtilityFactor for human profiles 1.0 -> 0.0 (imitate, not try-to-win) - humanSLRootExploreProbWeightless 0.8 -> 0.5 - restore level-based temperatures + quadratic lambda 0.06+(level-9)^2*0.03; delete the lightvector#1209 rankLambda/proLambda ladder - reintroduce `level`, derived from the clean keys (AI/pro->9, Nd->N-1, Nk->-N) - AI profile unchanged Make the human play-move visit budget rank-dependent so weaker ranks play fast: 9d and all Pro -> 400 visits; ranks 8d..20k -> 40 (humanSLPlayVisitBudget(for:) in GtpCommandBuilder). Continuous-analysis reset-to-unbounded sites unchanged. Tests updated to the old expected values via a tolerance-based value helper. Full iOS suite 439/439; iOS/visionOS/macOS build green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WoQvc49FJf5btyZsHLawb3
Isn't that only half a stone? KGS's match system by default matches players 1 rank apart with no-komi games, but that's to conform to convention. I'm pretty sure the ranking system understands this too. |
|
i.e. I'm pretty sure the ranking system understands and is calibrated for the fact that for players exactly 1 rank apart (in internal full-precision ratings) who play a no-komi game, white is expected to win more often than 50%. The true 1-rank difference would be a reverse-komi game. |
|
I’m retuning Lambda with The retuning aims to address the following issues:
|
Summary
Adds a
tunehumansubcommand and a set of GTP configs (gtp_human<rank>.cfg) that makeKataGo + the Human-SL net play at a chosen amateur KGS rank, from 9d down to 20k,
where each consecutive rank is exactly 1 KGS rank (1 stone) apart.
tunehumansubcommandCalibrates
humanSLChosenMovePiklLambda(the Human-SL strength dial) to a target winrate byplaying in-process candidate-vs-baseline games at fixed visits:
-komi/-cand-colorflags realize the KGS 1-rank handicap (weaker rank as Black,komi 0.5, no color alternation).
-resume-file).cpp/tests/testhumansltuner.cpp(run viakatago runtests).The ladder
29 configs,
gtp_human{9d…1d,1k…20k}.cfg. The Human-SL net is conditioned on KGS rank, so a1-rank gap is well-defined: per KGS, 1 rank = 1 stone, realized as an even game where the
stronger side (White) gets no komi compensation (komi 0.5). Each rung is tuned so the weaker
rank (Black, komi 0.5) is an even game (50%) vs the rung above, with the 95% CI ⊂
[40%, 60%], under Japanese rules.
because the net's rank profiles compress at the weakest end.
docs/HumanSL_Rank_Ladder.mddocuments the method, ruleset rationale, reproduction commands,the full results table (λ / win rate / 95% CI / games per rung), and findings.
ladder_step.sh+tune_decide.py+tune_{lambda,maxvisits}.share the automatedsequential root-finder harness that produced the ladder.
Results
Experimental calibration results — every rung's 95% Wilson CI of win rate is ⊂ [40%, 60%].
The 9d row is even-game parity vs the modern
rank_9dprofile; every other row is thekomi-0.5 handicap win rate (weaker rank as Black) vs the rung above it.
humanSLChosenMovePiklLambda)Notes
humanSLProfile = preaz_<rank>(pre-AlphaZero KGS-rank profiles) at 400 visits. The subcommand is backend-agnostic.
🤖 Generated with Claude Code
https://claude.ai/code/session_01L2nqY5X9rSVpH65nWHCPaF