kto

A library with extensible implementations of DPO, KTO, PPO, ORPO, and other human-aware loss functions (HALOs).

Auditable human-feedback annotation, review, provenance, and frozen training-data export.

Advanced LLM fine-tuning techniques: SFT (LoRA, QLoRA, DoRA, P-/Prefix-Tuning), GRPO, DPO, ORPO, KTO & PPO; composable correctness/format rewards + LLM-as-a-Judge evals (DeepEval, Evidently AI) across math, multi-hop, medical & general QA on Llama 3, Mistral, Phi-4, Gemma & Qwen3. Built on TRL, PEFT & Unsloth.

GenPark AI Agent Skill - Pairwise Bradley-Terry and Elo rating tournament engine for ranking multi-agent strategies, prompt mutations, and generated tool solutions.

GenPark AI Agent Skill - Iterative self-rewarding LLM judge scoring multi-turn agent outputs and synthesizing contrastive hard-negative prompt variations.

GenPark AI Agent Skill - Direct Preference Optimization (DPO) implicit reward calculation, reference policy log-ratio tracking, and pairwise preference loss evaluation.

GenPark AI Agent Skill - Pairwise Bradley-Terry and Elo rating tournament engine for ranking multi-agent strategies, prompt mutations, and generated tool solutions.

GenPark AI Agent Skill - Direct Preference Optimization (DPO) implicit reward calculation, reference policy log-ratio tracking, and pairwise preference loss evaluation.

GenPark AI Agent Skill - Prioritized experience replay buffer for agent reinforcement fine-tuning (RFT / GRPO) with advantage estimation and importance sampling weights.

GenPark AI Agent Skill - Kahneman-Tversky Optimization (KTO) loss evaluator aligning agents directly on unpaired binary feedback (thumbs-up / thumbs-down) using prospect theory loss aversion.

GenPark AI Agent Skill - Kahneman-Tversky Optimization (KTO) loss evaluator aligning agents directly on unpaired binary feedback (thumbs-up / thumbs-down) using prospect theory loss aversion.

GenPark AI Agent Skill - Iterative self-rewarding LLM judge scoring multi-turn agent outputs and synthesizing contrastive hard-negative prompt variations.

GenPark AI Agent Skill - Prioritized experience replay buffer for agent reinforcement fine-tuning (RFT / GRPO) with advantage estimation and importance sampling weights.

Open, MCP-native training-data foundry: turn permitted documents into traceable, quality-gated SFT and preference datasets.

Headless LLM fine-tuning in 3 lines — smart defaults, VRAM-aware batch sizing, multi-run SLAO, GGUF export for Ollama.

Systematic comparison of DPO, IPO, KTO, and ORPO alignment methods

Preference optimization framework for text classification (DPO/ORPO/KTO), with SFT, encoder, and XGBoost baselines plus unified run pipeline and reproducible outputs.

🚀 Optimize preferences effectively with ORPO, a framework for monolithic preference optimization without a reference model.

Structured game-agent distillation: distilling frontier-LLM gameplay reasoning into a small open student model (Qwen) through a typed MCP tool API, in a persistent 2D MMORPG. SFT + KTO data collection across 4 LLM harnesses, measured against a quest benchmark. (Joint work with Niral Patel.)

RLHF experiments with Unsloth: DPO, ORPO, SimPO, KTO. Training scripts, notebooks, and quick evaluation utilities.