Methods to build your next proof.
Fifteen references to size, evaluate, compare and keep your experiments. Pick a question, work through an example, then come back to the protocol and the resources you need for your decision.
Your next decision
has a starting point.
- Compute a budget
Set the parameters, precision, and an explicit reserve.
- Observe the phases
Link the peak to the inputs and the work actually executed.
- Compare the results
Fix quality before the cost of a useful result.
At your bench: IteraGPU Lab v1
Estimate weights → instrument a small network → prepare a comparison at fixed quality. The notebook and the table are blank; the memo folder specifies the scope of the script's validation run.
Handling the load
Separate weight computation, the cache and the phases that create the peak.
Sizing GPU memory: computation, reservations and validation
Compute weights in GiB, build an explicit reserve and validate the peak phase by phase. Worked examples, quantization pitfalls and per-card selection.
Read the methodGPU memory: explaining the gap between estimate and peak
Distinguish weights, cache and activations, measure allocated and reserved by phase, then choose a margin with a notebook and a PyTorch protocol.
Read the methodKV cache: calculating memory by context, GQA and batch
Calculate the KV cache using KV heads, context and concurrent sequences. Distinguish GQA, precision and static cache before a real trial.
Read the methodMicrobatch and accumulation: computing the effective batch
Compute the effective batch, normalize the loss and verify gradient accumulation across one or more cards, along with its equivalence limits.
Read the methodDefine an acceptable quality bar
Prepare the examples, read the errors and check what a compression changes.
Building an ML evaluation set without data leakage
Separate groups, duplicates, and data uses to build a held-out ML evaluation set that is controllable and suited to your decision.
Read the methodAnalyze ML errors to choose the next experiment
Analyze classification and extraction errors: confusion matrix, references, taxonomy, regressions and verification of the next fix.
Read the methodComparing quantizations: memory, quality, and useful cost
Compare reference, calibration, weight format, and KV cache with the same data. Measure memory and quality before deciding on the GPU budget.
Read the methodGPU inference: choosing based on memory, context and quality
Size inference with the context, the concurrency, the memory peak and the output quality. A concrete protocol before choosing your GPU package.
Read the methodAttribute the differences
Choose a control, organize the variants and distinguish an effect from variability.
Planning ML ablations: control, effects and budget
Build an ML ablation plan with a control, factors, interactions and a trial budget. A worked example to prioritize variants and reach a conclusion.
Read the methodVariability across seeds: is an ML gap convincing?
Interpret ML repetitions with paired seeds, dispersion, and a minimum effect threshold. Worked example and limitations of a small sample.
Read the methodGPU training: validate the loop before the campaign
Prepare the data, the batch and the resumption of a training run. One protocol per phase to control memory, gradients, validation and output files.
Read the methodFine-tuning: choosing an adaptation and verifying its contribution
Prepare a fine-tuning with a reference, separate data, and a reload check. LoRA, quantized base, budget, and regression analysis.
Read the methodPreserve and fund the evidence
Link the observations to the files, the committed budget and the next decision.
Tracking your ML experiments: from run to decision
Connect data, code, parameters, predictions and decision with an ML experiment manifest. An example of run and artifact tracking without a mandated tool.
Read the methodML benchmark: comparing cost at equivalent quality
Fix the quality, measure the same corpus and compare the full cost of 3-, 7- or 30-day GPU plans. Protocol and raw table available for download.
Read the methodBudgeting an ML campaign: runs, decisions, and the full package
Break down an ML campaign, prioritize variants, and tie the GPU package to useful results, with an example schedule and budget in USD.
Read the methodIteraGPU Lab v1
Memory calculation, small synthetic network, quality protocol, and blank results table: the two folders explain how to use the pack, then what remains to be measured on your workload.
- Download the IteraGPU Lab v1 pack
Original resources under MIT license. Results remain blank; the small local instrument check does not qualify the GPUs in the catalog.