Which GPU costs the least for the same ML result?

Define what will count as an accepted result

Write the decision before the tests: "Which configuration processes all my documents, meets my minimum quality and finishes before my deadline, for the lowest committed budget?" Fix the scenario: here, an offline processed corpus. An interactive application would also require defining request arrival, concurrency and acceptable latency; its ranking cannot be inferred from this single test.

Call a complete set of outputs that passes all your checks a validated corpus. A created file is not yet an accepted result. Check the identifiers, the format, the coverage and a metric tied to your use case. Record the threshold and the tolerance relative to a reference before looking at the times. Two configurations can thus be equivalent for the decision without producing bit-for-bit identical numbers.

This separation between dataset, quality objective and scenario also exists in the principles of MLPerf Inference. The protocol below is our working method to adapt to your project; it constitutes neither a run nor an MLPerf certification.

Technical sources: MLCommons — scenarios, metrics and quality objectives

Prepare the inputs, the reference and the manifest

You need a corpus you have the right to use, reference answers or an evaluation procedure, the inference code and an environment capable of running the chosen model. Separate the data used to tune the configuration from the final comparison corpus. If you adjust thresholds after seeing the latter, prepare a new independent evaluation to support the conclusion.

Give each input a stable identifier. Keep a corpus fingerprint, the order of passage, the model and tokenizer revision, and the code and dependency versions. The manifest also describes the GPUs used, precision, quantization, compilation, attention backend, padding policy and maximum length. A different truncation would change the work to be compared.

Record the driver and backend actually present. For an AMD variant, check the combination of system, GPU, ROCm and framework in the official matrix. A documentation page you consulted or the name of a card does not prove that this environment is installed. If the software stacks differ between two runs, the conclusion will cover the complete configurations tested.

Technical sources: AMD — ROCm compatibility matrix

Example: classifying the same 1,000 texts

Here is an experiment to build with your own data, with no presumed performance result. You want to classify 1,000 texts into your project's categories. Set aside 600 short entries, 300 medium ones and 100 long ones; define the boundaries with the tokenizer you chose and keep a class distribution that is representative. This split is an example of a protocol, not a provided corpus nor a universal recommendation of proportions.

Compare batches of 1, 4 and 8 with the same model, the same precision, the same order and the same padding rule. The only variable in this first series is the batch size. A second series can change the precision or the hardware, keeping the other choices explicitly fixed. Grouping by length is a new variant that must be declared, because it changes how the work is organized.

For each run, export the 1,000 predictions with their identifiers. The check must find exactly the expected identifiers, with no duplicates or omissions. Keep the incorrect predictions: they are used for the quality calculation. Removing the difficult examples would artificially improve the score and reduce the corpus actually processed.

Separate startup, warm-up and the measured run

Record the necessary download, installation, loading and compilation separately from the stabilized processing. They can be excluded from a run's timer while still taking up part of the rental. Set an identical warm-up rule for all variants: inputs covered, number of runs and handling of recompilations. Do not adjust this rule after seeing which variant benefits from it.

Define the boundaries of the main time. For this offline example, measure reading the corpus, tokenization, transfers, inference and materialization of the predictions on the host. Then time the evaluation and writing of the deliverables separately to establish the complete campaign. A duration limited to GPU compute is not directly comparable to this processing time.

CUDA operations are asynchronous: a host timer must wait for prior operations to finish before it starts and for the measured work to finish before it stops. CUDA events are suitable for a properly defined GPU scope. For an isolated operation, torch.utils.benchmark.Timer handles warm-up and synchronization. Keep the same measurement boundaries between variants.

Technical sources: PyTorch 2.14 — asynchronous CUDA execution · PyTorch — measuring with torch.utils.benchmark

Repeat and keep the failures

Plan for five complete runs per variant for this first comparison. Alternate their order, for example 1–4–8 then 4–8–1 then 8–1–4, so that a single variant is not always first. Keep the process, cache and warm-up policy. These five runs describe your small series; on their own they do not demonstrate stability over a long period.

A row in the raw table represents one attempted run, including a halt. It links the variant and the corpus to the parameters, the duration and the quality verdict. The expected_ids and observed_ids fields record the numbers of entries; ids_match confirms that the sets are equal, checked in the prediction files kept separately. corpus_accepted holds the verdict for the run. Record errors and the output path in notes. If a measurement is missing, leave its cell empty and state why. The absence of a GPU is not a measurement of zero seconds or zero bytes.

If batch 8 exceeds memory on the long inputs, keep the failure row and the number of entries completed. Do not silently replace that run with a smaller batch. The resume setting becomes a distinct variant; its time and attempts are part of the tally. An interruption or a format error is never an economic success just because it was fast.

Reading durations without overinterpreting five trials

Present the five raw durations, their median, their minimum and their maximum, along with the number of successes and failures. The median describes the center of these observations; it does not remove the incidents. If a run is excluded for a documented external cause, keep a record of it and apply the same exclusion rule to all variants.

Do not present a p95 computed from five runs as a robust estimate of slow cases. To study request latency, collect a suitable set of individual times along with the arrival scenario and the concurrency. The five corpus durations and the latencies of 1,000 requests are not the same population.

If the observed dispersion is comparable to the gap between the medians, the series does not yet separate the options. Add repetitions under a common protocol or examine a specific cause: loading, input shapes, compilation, concurrent activity. Avoid keeping only the best run from each card.

Checking quality after a change of precision

To compare FP32, BF16 or a quantization, start again from the same inputs and the same reference. Evaluate the format, the primary metric and the planned subgroups. A variant that exceeds your tolerance may be interesting for another objective; it does not join the equal-quality comparison by lowering the threshold after the fact.

Record the seeds and deterministic options used. PyTorch does not guarantee full reproducibility across versions, platforms or CPU and GPU runs, even with the same seed. So specify what you are trying to reproduce: identical outputs, a bounded numerical deviation or an acceptable business quality. For a stochastic protocol, plan several common seeds and keep their results separate.

Technical sources: PyTorch 2.14 — scope and limits of reproducibility

Using memory as a feasibility criterion

An option must finish the corpus with its long inputs before its cost is compared. Note the peak memory per device and the scope of the counter. With PyTorch, memory_allocated tracks tensors and memory_reserved the memory managed by the allocator: these values do not add up. The corresponding peaks may occur at different times.

The notebook in the memory folder helps distinguish estimation from observation. Its exercise does not replace measuring your own model: load your environment, keep the parameters and rerun your workload. A capacity advertised per card and a lot price do not allow you to infer a throughput, an interconnect or automatic model distribution across several GPUs.

Technical sources: PyTorch 2.14 — memory counters and allocator

Choosing the duration with the complete schedule

The necessary duration is not just the sum of the GPU kernels. Build an access window running from preparation on the rental to retrieval of the deliverables: installation, checks, warm-up, comparisons, planned resumptions, evaluation and export. Add the waiting periods during which you still need to keep the rental. When tasks overlap, reason on the actual schedule rather than adding their durations twice.

Example schedule, with no assumption about speed: you want to keep access from Monday at 9 a.m. to Friday at 9 a.m., in the same time zone and outside any daylight saving change. This window covers 96 hours. It exceeds the 72 hours of a 3-day package and fits within the 168 hours of a 7-day package. That does not prove your processing will finish on time: their durations still need to be measured.

The calculator lets you enter your total window and displays the 3-, 7- and 30-day packages. A package that covers the calendar becomes a candidate. If the campaign exceeds the chosen period, change the program or explicitly budget for the additional periods required; do not assume an automatic extension.

Calculate the committed cost with our packages

The budget for a rental is the package price for one lot, multiplied by the number of lots. The cost per validated corpus then divides that whole amount by the useful corpora actually accepted within the stated scope. If no corpus is accepted, the ratio is undefined. The committed spend, however, stays in the final tally.

The prices below come from our catalog, version of September 24, 2026. They illustrate the calculation rule, without establishing which GPU finishes your workload fastest. A B200 lot already includes two cards: multiplying its price by two a second time would count those cards twice. Two lots of RTX 4090 for 7 days therefore cost USD 220; one lot of two B200 for 7 days costs USD 2,071.

A short run does not turn the package into an hourly bill. The calculator uses the package total, even if part of the period goes unused. For a broader project budget, record separately the other expenses that actually apply and their justification. Do not mix costs measured for one variant with line items overlooked for the other.

Technical sources: IteraGPU — catalog packages

Count useful deliverables, not repeated measurements

The five repetitions of the same benchmark serve to observe dispersion. They do not become five useful production corpora simply because five files were written. Define the expected deliverables before the campaign and count each accepted deliverable only once. If the actual work covers several corpora, each configuration must process the same corpora and apply the same quality rule.

In the calculator, leave the number of corpora empty until the useful deliverables are actually finished and validated. The quality checkbox confirms your own check; the tool reads neither your predictions nor your metrics. Enter only the observed quantity. The campaign window can be a planning assumption, but a capacity forecast does not replace accepted results.

The final tally brings together, for each admissible configuration, the quality verdicts, the raw durations, the campaign window, the committed package and the number of accepted deliverables. A fast option may be useful for a tight deadline without being the cheapest. Two options that fit within the same calendar can be separated by their committed cost, their failures or the uncertainty that remains.

Download the protocol and keep reusable evidence

The IteraGPU Lab v1 folder gathers the materials for this comparison and for the memory measurement. Start with the README and the quality protocol, then fill in the raw table with your runs. The performance cells stay blank before an execution; the rates are catalog data, kept separate from the measurements.

To estimate two batches of RTX 4090 for 7 days, place calcul_forfaits.py and tarifs-forfaits.csv in the same folder, open a terminal in that folder and run the command below with Python 3.10 or newer. It outputs a flat rate of USD 220.00 for two GPUs. The optional --accepted-results flag takes your integer count of distinct useful corpora actually completed and validated. Leave it out as long as that finding doesn't exist: the script then computes only the flat rate, without inventing a cost per result.

Keep the manifest, input fingerprints, predictions, verdicts and the run table together. The decision note states the chosen setting, the workload covered and the reason for the choice. You can then match it against your rental in the IteraGPU ledger and rerun the exact experimental question when a model or version changes.

This offline protocol does not by itself qualify an interactive service, training to convergence or another dataset. For those uses, redefine the useful unit and the checks before comparing. The files provided are meant to prepare and record your trials; this folder presents no measured comparison between the configurations in the catalog and no observed savings.

python calcul_forfaits.py --gpu rtx-4090 --days 7 --lots 2

• IteraGPU Lab v1 — complete folder ↓

  Archive of the scripts, notebook, tables and instructions.

• Quality protocol ↓

  Inputs, acceptance criteria and comparison rules to set before the trials.

• Raw results table ↓

  Blank sheet for keeping parameters, measurements, verdicts and failures.

• Flat-rate calculation ↓

  Budget calculation with price per batch, durations of 3, 7 and 30 days and validated corpora.

• Flat rates ↓

  Snapshot of the prices in our catalog used by the calculation provided.

• Memory measurement notebook ↓

  Calculations and measurement exercise to be interpreted with the memory folder.

• Memory measurement script ↓

  Python version of the exercise, with environment check.

• Instructions and prerequisites ↓

  Scope of the tools and procedure for running them and keeping the outputs.

• Resource license ↓

  Terms for reusing the original resources in the folder.