Building an ML evaluation set without data leakage
Separate groups, duplicates, and data uses to build a held-out ML evaluation set that is controllable and suited to your decision.
KV cache: calculating memory by context, GQA and batch
Calculate the KV cache using KV heads, context and concurrent sequences. Distinguish GQA, precision and static cache before a real trial.
Planning ML ablations: control, effects and budget
Build an ML ablation plan with a control, factors, interactions and a trial budget. A worked example to prioritize variants and reach a conclusion.
Analyze ML errors to choose the next experiment
Analyze classification and extraction errors: confusion matrix, references, taxonomy, regressions and verification of the next fix.