Distinguish campaign, configuration and run
A campaign carries a question, for example comparing a baseline to an adaptation. A configuration describes the technical choices. A run is an attempt to execute that configuration, with a seed, a start, an end and a status. Two identical attempts therefore keep two identifiers, even if the second replaces an interrupted trial.
Add an evaluation identifier when the same artifact is evaluated on several sets or with a new metric. This avoids confusing a new model with a new reading of the existing model. Link the resumption attempt to the one that preceded it and to the checkpoint that was loaded.
MLflow also organizes tracking around runs, parameters, metrics and artifacts. This distinction is useful even in a simple file folder. You can apply it with your usual tool; no particular tracking platform is required to get started.
Technical sources: MLflow — runs, parameters, metrics and artifacts
Write the manifest that was actually executed
Keep the resolved parameters after defaults and launch arguments have been applied. The original configuration file may omit a default batch size or an option changed at runtime. Record what was actually used, with a copy of the code or an immutable revision and the state of uncommitted changes.
The manifest also links model and tokenizer versions, software environment, the GPU actually used, precision, data, seeds and the definition of metrics. It describes the trial without becoming an installation tutorial. Write down the units: seconds, bytes, tokens, points or a proportion from 0 to 1 depending on the measurement.
At startup, the status is running; at the end, it becomes completed, failed or interrupted depending on what you observed. Do not retroactively fill in a forgotten version with the one currently installed. Mark the information as unknown and limit the conclusion that depends on it.
| Block | Items to keep | Question answered |
|---|---|---|
| Identity | campaign_id, config_id, run_id, possible parent_run_id | Which attempt produced this result? |
| Code and model | Exact revisions, local modifications, base model and tokenizer | Which computation was actually launched? |
| Data | Version, split, preprocessing, identifiers and relevant order | On which inputs? |
| Parameters | Effective values, seeds and units | With which settings? |
| Evaluation | Evaluated artifact, metric/version, threshold and population | What does the score mean? |
| Closure | Status, error, produced files and decision | Is the trial usable? |
Identify data beyond a folder name
A path like donnees/final does not designate a stable version. Keep the inventory of files or examples, the splits and the transformation procedure. If you correct labels or filter rows, create a new version and keep the relationship with the previous one. The old score must continue to point to its old data.
Hugging Face Datasets associates fingerprints with the state of a dataset and its transformations to manage the cache. This mechanism is useful, but you must also keep the origin of the data and the preprocessing. A non-hashable transformation can for instance lead to a random fingerprint: the cache identifier alone does not replace your provenance record.
For frozen artifacts, add file size and fingerprint. A SHA-256 digest computed before and after a copy makes it possible to verify that the bytes match the retained reference. It proves neither the quality of the labels, nor the rights of use, nor the absence of leakage between splits.
Technical sources: Hugging Face Datasets — fingerprints and transformations · Python 3.14 — file fingerprints with hashlib
Connect metrics, predictions and artifacts
A metric line should identify the run, the artifact evaluated, the evaluation set, the metric version, and its scope. Specify whether the value concerns an intermediate checkpoint, the final model, or a subgroup. A curve is not enough if you can no longer tell which file corresponds to the selected point.
Keep predictions with their input ID, their status, and the information needed for evaluation. The expected references can remain in a separate versioned file. For structured output, distinguish raw result, parsed result, and verdict: fixing the parsing must not overwrite the initial output.
MLflow lets you link metrics to models and data. With files, apply the same principle using explicit identifiers. Keep a readable inventory of artifacts: weights or adapter, generation parameters, outputs, evaluation report, and decision note. Do not assume a screenshot replaces these files.
Technical sources: MLflow — linking metrics, models, and datasets
Example: six runs and a duplicate that hides a gap
Let's consider an example of ranking with two configurations and three seeds. It produces six planned runs. Each must predict the same 300 evaluation IDs: the complete folder therefore expects 6 × 300 = 1,800 unique pairs (run_id, input_id). This calculation describes an expected inventory, not an experiment actually run.
Suppose a file contains 300 lines, but the ID doc-042 appears twice and doc-117 is missing. The total number of lines looks correct; yet there are only 299 unique IDs. This run fails the coverage check until the anomaly is explained and corrected.
If each run is then evaluated on the full corpus and on its subgroup of long texts, you obtain twelve metric lines for a given metric. That is still six runs, not twelve independent trainings. The evaluation key must include the scope to preserve this distinction.
| Check | Expected | Illustrative anomaly |
|---|---|---|
| Runs | 2 configurations × 3 seeds = 6 | A rerun receives a new identifier |
| Predictions per run | 300 unique IDs expected | 300 lines but only 299 unique IDs |
| Run/input pairs | 6 × 300 = 1 800 | The overall count alone does not detect all duplicates |
| Evaluations | 6 runs × 2 scopes = 12 | Twelve scores do not create twelve runs |
Closing the folder with a readable decision
Before declaring a run finished, check that files are present and openable, that identifiers match, that metrics are recalculable, and the status of each error. A technically finished attempt may remain rejected for insufficient quality. Keep these two states separate.
The decision note brings together the question, the variants compared, the announced criterion, the results retained, and the reasons for exclusion. Cite the run_id and artifact paths rather than "the latest model." Add the limitations: few repetitions, insufficient subgroup, version not found, or comparison that has become impossible.
A later correction must leave a trace: new evaluation, new report, and the reason for the change. Keep the old conclusion as an identified historical version, without letting it appear as the current decision. Finally, check that the exported copy opens from its destination folder.
Avoiding unnecessary collection and reproduction promises
Collect the fields needed for the proof, not all environment variables or the terminal history. A configuration or a URL may contain a token; prepare a shareable version without secrets and keep the data that must remain private in its authorized location. Technical test identifiers do not need to include a person's name.
A complete folder improves the ability to redo and understand an experiment. It does not guarantee numerical equality across platforms or versions: PyTorch documents these reproducibility limits. Distinguish between recovering the protocol, reloading the artifact, and reproducing the numbers exactly.
The IteraGPU notebook can keep your goals, parameters, and decisions, along with useful references. It does not launch runs or automatically collect files or telemetry. Use it as an index of your reasoning and keep the artifact folder in your own backups.
Technical sources: PyTorch 2.14 — reproducibility limits across environments
Practical questions
Is a Git commit enough to reproduce an experiment?
A commit identifies a version of the code, but not necessarily the data, the weights, the effective parameters or uncommitted changes. Pair it with a manifest and the artifacts produced. Without these links, two runs of the same commit can correspond to different experiments.
Should all predictions be kept?
Keep the outputs needed to verify the conclusions and recompute the evaluation, within the limits of your rights and retention constraints. For a bounded comparison corpus, the identifiers and full predictions make errors auditable. A simple aggregate score generally does not allow you to trace back the missing examples.
Should a resume reuse the same run_id?
The proposed schema gives a new identifier to each attempt and links the resume to the previous run and to the checkpoint loaded. You can group these attempts under a single logical experiment. This separation makes the interruption, the costs and the files actually produced at each step visible.