Timing what your application really does
A test on tensors already present on the GPU describes the computation alone. A service or a training job must also decode, prepare and transfer its inputs. So build two measurements: the isolated compute loop and a loop that starts from your usual data. The gap tells you where to look for improvement. Wait for GPU work to actually finish before recording a time; CUDA launches are often asynchronous.
Using batch size as a controlled variable
For offline inference, compare several batch sizes while keeping the input length and type the same. For an interactive application, also measure the latency of a single request and of the slowest requests. The batch that maximizes throughput can make waiting less acceptable. On 80 GB, leave room for the cache or activations rather than stopping sizing at model load.
Log weight and compute precision separately. A model stored in a reduced format may still use higher-precision buffers or operations during execution.
Comparing H100 variants without conflating them
This page is about the H100 PCIe 80 GB. Results from an H100 SXM or from a multi-card setup do not automatically describe this configuration. Check CUDA, PyTorch and specialized libraries for your architecture, then choose a recipe common to the GPUs being compared. If 80 GB is not enough, look at the H200; if your need fits within 48 GB, add the L40S to your cost comparison.
Preparing a pipeline-oriented order
Three days can be enough to identify the share of compute and transfers. Seven days allow you to test fixes and repeat measurements. Thirty days support an inference campaign or a planned training run. Select duration and quantity, enter your environment, then your tracking details. The order's crypto instructions specify the network and amount; the "I've paid" button signals your transfer.