I build foundation models that can work—not just answer.
|
Research Scientist Intern @ Alibaba M.S. Computer Science @ UIUC |
Open-source engineering across NeMo, Megatron-LM, vLLM/Vime, SGLang, and ModelScope. Training reliability · long-context kernels · CUDA · RL infrastructure · inference serving |
Occamy-1.0 is a 35B-A3B co-work model continued from Qwen3.6-35B-A3B through full-parameter SFT, uniform model soup, and GRPO/SAO reinforcement learning.
I built verifier-gated data and training infrastructure for token-exact replay, state reconstruction, episode credit across context rewrites, immutable provenance, and quarantine gates. On the same frozen ClawEval harness, the resulting system raised Combined T/C Strict Pass@1/3 from 65.16/73.87% → 77.39/85.93% while reducing tokens per trajectory by 38.6%.
| Area | Selected contribution |
|---|---|
| Optimizer stability | Scale-invariant Newton–Schulz for small-norm Muon inputs in NVIDIA NeMo Emerging Optimizers #230 |
| Hybrid-model training | Recompute propagation and Mamba + attention + MoE runtime fixes in vLLM/Vime #337 |
| Training throughput | Order-preserving sequence packing, NCCL warmup, and Muon correctness across ModelScope ms-swift #9598, #9602, #9599, and #9591 |
| Long-context kernels | Fused GatedDeltaNet Q/K normalization for 128K SFT in Megatron-LM #5396 and selective Mamba recompute in #5463 |
| RL and inference reliability | Non-finite rollout-logprob sanitization in NeMo RL #2962 and safer hybrid-model weight reloads in SGLang #31621 |
- CineFlow — dependency-driven parallel video generation with semantic DAG compilation, trajectory-aware fusion, and critical-path scheduling; 1.7–5.5× speedup and 17.3% higher visual quality.
- Dynamic Prefill Optimization — AIMD control with p95 TTFT feedback and greedy/DP prompt packing; up to 20% lower TTFT on production-style traces.
- FlashAttention-style CUDA Optimization — tiled online softmax and kernel fusion for GPT-2; roughly 10× lower HBM traffic and up to 9% end-to-end speedup.
- RL for Legal Reasoning — Zero-RL → distilled-CoT SFT → GRPO, reaching 57.6% accuracy.
- Seismic Phase Picking — traditional and learning-based automatic phase picking for earthquake monitoring.
To an unceasing future. 致永无止境的明天。