diff --git a/book/HPC.md b/book/HPC.md index 39aee51..a813875 100644 --- a/book/HPC.md +++ b/book/HPC.md @@ -421,6 +421,8 @@ Python 3.14.2 We could also set this as a default using `module save`, and the next time we log in we would see that this module is loaded. If there is a package that you need that isn't in the current list of packages, a friendly email to the help desk is usually sufficient to get it installed as a module. +Loading explicit module versions with `module load python/3.14.2`, rather than relying on whatever default happens to be installed, is one form of what the STAMPED framework [@Macdonald:2026aa] calls **T**racking: what the job runs against is spelled out, not implicit. + ### Virtual environments Throughout the book I have talked about the utility of virtual environments, and they are commonly used on HPC systems to can access to packages or package versions that are not available as modules on the system. There is, however, one issue that should be kept in mind when using virtual environments in the HPC context. When we install a virtual environment, the environment folder contains all of the dependencies that are installed in the environment. For some projects this can end up being quite large, to the degree that one can run into disk quota issues if they are stored in the home directory. For example, the full Anaconda installation is almost 10GB, which would largely fill the 15 GB quota for my home directory on the local HPC system; for this reason, I always recommend using miniconda which is a more minimal installation. `uv` does a better job of caching but its local cache directory can also get very large over many projects. For this reason, I we generally install Conda-based environments outside of the home directory, on a filesystem that has a larger quota. When using `uv`, we generally set the `$UV_CACHE_DIR` environment variable to a location with a larger quota as well. @@ -429,7 +431,9 @@ Throughout the book I have talked about the utility of virtual environments, and Containers solve at least two important problems for HPC users. First, as I noted in the earlier discussion, they provide a strong platform for reproducible computing; given that the HPC user has no control over operating system upgrades, containers help ensure that software dependencies will remain consistent over time. Second, they allow users to install software that may not be runnable on the HPC system; for example, the user may require a version of a package that is too old to run on the HPC's current operating system. For these reasons, containers have become very popular on HPC systems. -However, there is a rub: Because Docker requires that the user have root access to the system, it can't be run by users on HPC systems (which restrict root access to sysadmnins). Instead, most HPC systems support Apptainer (formerly called Singularity) as a platform for running containers. However, Apptainer can't actually build the container; it can only use existing containers, defined through an image file or downloaded from an image registry such as Dockerhub. In many cases, there are existing containers on Dockerhub that have the appropriate software installed, which can be called directly from Apptainer. However, if no suitable pre-existing container image is available, the user will need to first generate a container using Docker on a system where they have administrative access (usually their own personal computer), and then convert the Docker image into an Apptainer image for use on the HPC system. This can lead to long debugging cycles if there are problems with the container that require rebuilding, but once it's working it should continue working in the long term. +However, there is a rub: Because Docker requires that the user have root access to the system, it can't be run by users on HPC systems (which restrict root access to sysadmnins). Instead, most HPC systems support Apptainer (formerly called Singularity) as a platform for running containers. However, Apptainer can't actually build the container; it can only use existing containers, defined through an image file or downloaded from an image registry such as Dockerhub. In many cases, there are existing containers on Dockerhub that have the appropriate software installed, which can be called directly from Apptainer. However, if no suitable pre-existing container image is available, the user will need to first generate a container using Docker on a system where they have administrative access (usually their own personal computer), and then convert the Docker image into an Apptainer image for use on the HPC system. This can lead to long debugging cycles if there are problems with the container that require rebuilding, but once it's working it should continue working in the long term. + +Running each HPC job from a container image, and against a scratch directory that will be cleared after the job, also exercises the STAMPED **E**phemerality principle: the environment is built from an explicit specification, used, and then discarded, so a successful run confirms the specification was complete rather than depending on hidden state accumulated on a long-lived machine. ### Interactive access to HPC systems @@ -469,4 +473,4 @@ In addition to the HPC resources available at many research institutions, anothe Benefits of commercial cloud computing are that the resources are generally available immediately, and that the user has complete control and root access to the resources. The downside of commercial cloud computing is that it can become very expensive very quickly. Every component costs money: computing time, data storage, and data egress, as well as other features (e.g. database hosting). There are ways to reduce costs, such as using pre-emptible resources (which may be interrupted), but these systems can still be very costly for large data/compute needs. -We have used cloud computing for various projects, and it can be very useful to obtain specific kinds of compute resources quickly. However, I generally feel that pay-to-play is a bad model for scientific computing, because it inhibits the kind of creativity and exploration that I think scientists need to be able to engage in. When every computing cycle costs money, researchers are less likely to play and possible discover something new. That said, they are an important tool in the computational toolbox, and can often help solve problems that would be intractable using on-prem HPC resources. \ No newline at end of file +We have used cloud computing for various projects, and it can be very useful to obtain specific kinds of compute resources quickly. However, I generally feel that pay-to-play is a bad model for scientific computing, because it inhibits the kind of creativity and exploration that I think scientists need to be able to engage in. When every computing cycle costs money, researchers are less likely to play and possible discover something new. That said, they are an important tool in the computational toolbox, and can often help solve problems that would be intractable using on-prem HPC resources. diff --git a/book/data_management.md b/book/data_management.md index 21bef03..06affe4 100644 --- a/book/data_management.md +++ b/book/data_management.md @@ -47,6 +47,23 @@ Data are reusable if the requirements for reuse are clearly specified. Making d The FAIR principles are relatively abstract, in the sense that they don't provide specific guidance about what FAIR means in any particular domain. However, there are numerous resources that can help implement these principles, such as [RDMKit](https://rdmkit.elixir-europe.org/) and the [FAIR Cookbook](https://faircookbook.elixir-europe.org/content/home.html), both generated by the European ELIXIR organization. +### STAMPED: an operational companion to FAIR + +Where FAIR focuses on the discovery and governance of digital objects, the STAMPED principles {cite:p}`Macdonald:2026aa` address a complementary layer: the day-to-day practices that determine whether a research object can actually be sensibly shared, re-executed, and potentially extended by others. Seven principles are defined: + +- **S**elf-containment: a research object is a complete retrieval unit; every component needed to reproduce its computational results is reachable from a single top-level object (the "don't look up" rule — no reliance on implicit external state). +- **T**racking: the state and provenance of every component are recorded, typically via versioning manifests, content-addressed identifiers (checksums), and code-driven provenance records. +- **A**ctionability: the object contains machine-actionable instructions to reproduce its results — executable specifications, not just documentation (albeit with AI, good documentation becomes actionable as well). +- **M**odularity: components are organized as independent, composable modules, each carrying its own license, so they can be reused, replaced, or evolved on independent schedules (code often has a different life cycle than the data it processes). +- **P**ortability: computational environments are explicitly specified and version-controlled; procedures do not depend on undocumented host state. +- **E**phemerality: computational results are produced in throwaway environments — staged in a temporary location, run there, then discarded — so that a successful run confirms the environment was fully specified. +- **D**istributability: all referenced modules are persistently retrievable by others, with resolvable license identifiers, across repositories and portals used in scientific practice. + +Each principle is expressed as a graduated set of requirements (MUST for the practical minimum, SHOULD and MAY for progressively more aspirational practices), and the framework is accompanied by an [interactive checklist](https://checklist.stamped-principles.org/) that a project can use to self-assess and communicate its operational maturity. Much of the rest of this book puts STAMPED into practice: the *DataLad*-based tracking discussed later in this chapter, the containerized environments and structured project layouts in later chapters, and the workflow and CI patterns for reproducible re-execution. + +Full disclosure: STAMPED was co-developed by Yaroslav O. Halchenko, who also contributed to this book. + + ## The data lifecycle An important concept in research data management is the *data lifecycle*, which describes the role of data management in each of the different stages of a research project. [](#lifecycle-fig) shows an example of how the [RDMkit project](https://rdmkit.elixir-europe.org/data_life_cycle) outlines the stages of the data lifecycle. This figure highlights the fact that data management should be part of the discussion at each stage in a project. In this chapter I will discuss several of the stages in the data lifecycle in detail, though I leave in-depth discussion of data processing and analysis workflows to a later chapter. @@ -953,7 +970,7 @@ However,*git* does not work well for version control on larger datasets using bi ### Using *DataLad* for version control on larger datasets -A solution to this problem is to use a version control tool that is specifically designed for large data. There are several tools that address this problem; we will focus on [Dat*aLad](https://www.datalad.org/), which is a data management system that functions very similarly to git. It is based on a tool called [*git-annex*](https://git-annex.branchable.com/), but provides much greater ease of use for researchers. (Full disclosure: Our group collaborates with the *DataLad* group and our grants have supported some of their development work.) +A solution to this problem is to use a version control tool that is specifically designed for large data. There are several tools that address this problem; we will focus on [*DataLad*](https://www.datalad.org/) [@Halchenko:2021aa], which is a data management system that functions very similarly to git. It is based on a tool called [*git-annex*](https://git-annex.branchable.com/), but provides much greater ease of use for researchers. (Full disclosure: Our group collaborates with the *DataLad* group and our grants have supported some of their development work.) What *DataLad* provides — content-addressed identification of every component, plus code-driven provenance of every modification — is what the STAMPED framework calls the **T**racking principle in practice: two datasets can only be called "identical" if their content hashes match, not merely their labels. An important note: *DataLad* is quite powerful but has a significant learning curve, and takes a bit of time to get accustomed to. In particular, its use of symbolic links can sometimes confuse new users. Having said that, let's look at some simple examples. diff --git a/book/extras.md b/book/extras.md index 9d97379..8a3772a 100644 --- a/book/extras.md +++ b/book/extras.md @@ -573,4 +573,21 @@ It’s a perfect example: Serial: Too slow for 10^ 6 iterations. Vectorized: Might run out of memory. Parallel: The "Goldilocks" solution where you send chunks of iterations to different cores. -This provides a narrative arc: You start with the math, realize the serial version is too slow, try vectorizing but hit a memory limit, and finally arrive at parallelization as the engineering solution. \ No newline at end of file +This provides a narrative arc: You start with the math, realize the serial version is too slow, try vectorizing but hit a memory limit, and finally arrive at parallelization as the engineering solution. + + +## STAMPED principles ↔ book chapters mapping + +For readers who want to use the STAMPED framework [@Macdonald:2026aa] to self-assess their own projects, the table below indicates where each of the seven principles is put into practice in this book, together with representative items from the [interactive checklist](https://checklist.stamped-principles.org/). Checklist item IDs (`must/NNN`, `should/NNN`, `may/NNN`) come from `stamped-checklist` v0.1.0. + +| Principle | Where in this book | Representative checklist items | +|---|---|---| +| **S**elf-containment | Project structure chapter (§ "Should code and data live alongside one another?"); Data management chapter (§ *DataLad*) | `must/001`–`must/003` (all files under a common root; datasets and external software reachable from that root) | +| **T**racking | Data management chapter (§ *DataLad* for version control on larger datasets); Software engineering chapter (Git workflow) | `must/004`, `must/005` (Git for code and text; `git-annex` / *DataLad* / Git-LFS for large data); `must/006` (env specs in provenance records); `should/001` (one VCS across components) | +| **A**ctionability | Workflow chapter (§ Using a workflow engine); Sharing chapter (§ Software metadata — `CITATION.cff`, `codemeta.json`) | `must/007`, `must/008` (`README.md` or `Makefile` with usage; a clear starting point); `should/002` (workflow tested regularly) | +| **M**odularity | Project structure chapter (§ project structure, BIDS folder-naming); data-vs-code separation in the data management and sharing chapters | `should/003` (raw data / processed data / code / env separated into distinct modules); `may/001`–`may/005` (submodules, modular boundaries, composition mechanisms) | +| **P**ortability | Project structure chapter (§ "It needs to be portable", § Containers); HPC chapter (§ Modules, § Containers); Sharing chapter (env-sharing sections) | `must/009`–`must/011` (relative paths; deps in env spec; host assumptions documented); `must/012` (system requirements in `README`); `must/013`, `must/014` (env specs in VCS, updated as deps change) | +| **E**phemerality | Workflow chapter (§ Reproducible environments with containers); HPC chapter (§ Containers, per-job scratch directories) | `should/004`, `should/005` (pipeline tested in a fresh container / disposable env per execution) | +| **D**istributability | Sharing chapter (§ Persistent identifiers for code; § Choosing a data repository); Data management chapter (*DataLad* remotes and portals) | `must/015`–`must/017` (env specs persistent, shareable, documented); `should/006`–`should/008` (env artifacts regularly re-tested and archived) | + +Item numbers reflect the checklist at time of writing; verify current numbering at or before printing or citing. diff --git a/book/introduction.md b/book/introduction.md index 88c1464..b7cc952 100644 --- a/book/introduction.md +++ b/book/introduction.md @@ -176,6 +176,11 @@ It is increasingly common for researchers to share both code and data from their However, in our experience, it can be very difficult to actually run the shared code, due to various problems that limit the portability of the code. Throughout this book I will discuss the tools and techniques that can help improve the portability of shared code and thus increase the reproducibility of published results. +Two frameworks give shape to this discussion. +The FAIR principles (Findable, Accessible, Interoperable, Reusable) [@Wilkinson:2016aa] structure the discovery and governance layer of research-object sharing; we return to them in the data management chapter. +STAMPED [@Macdonald:2026aa] is a recent operational companion: research objects should be **S**elf-contained, **T**racked, **A**ctionable, **M**odular, **P**ortable, **E**phemeral, and **D**istributable, with an [interactive checklist](https://checklist.stamped-principles.org/) for self-assessment. +Much of what this book teaches — version control, containers, structured project layouts, and reproducible workflows — facilitate FAIR and STAMPED practices. + ### Bug-hacking A particular concern is that not all software errors are created equal. diff --git a/book/project_organization.md b/book/project_organization.md index 796c172..fb99b9c 100644 --- a/book/project_organization.md +++ b/book/project_organization.md @@ -26,7 +26,7 @@ As I discussed in the section in Chapter 3 on clean coding, one of the most impo ### It needs to be portable -It's rare for one to perform analyses that are only meant to run on one specific computer system. Coding portably (as discussed in Chapter 3) makes it easy to run the code on other machines. This can be useful, for example, when one replaces one's laptop, or when one needs to scale their code to run on a high-performance computing system. It also helps ensure that the code can be tested using automated testing tools, like those discussed in Chapter 4. +It's rare for one to perform analyses that are only meant to run on one specific computer system. Coding portably (as discussed in Chapter 3) makes it easy to run the code on other machines. This can be useful, for example, when one replaces one's laptop, or when one needs to scale their code to run on a high-performance computing system. It also helps ensure that the code can be tested using automated testing tools, like those discussed in Chapter 4. Portability, expressed as explicitly specifying environments and avoiding hidden dependencies on the host, is what the STAMPED framework [@Macdonald:2026aa] calls the **P**ortability principle; the container-based approach discussed later in this chapter is one of the strongest ways to achieve it. ## Project structure @@ -45,6 +45,8 @@ For my projects with datasets larger than a few gigabytes, I tend to keep data s In general, for portability it's also nice to have the data location parameterized in the code (e.g., via a .env file or local config file) rather than hardcoded through the use of a local directory name. Thus, even if you decide to put the data within the code directory, it's good to write the code in a way that can allow the data to live in an arbitrary location. +Keeping code and data as separately versioned modules is an instance of STAMPED-**M**odularity: code often changes on a different schedule than the curated data it processes, so pinning them to independent identifiers pays off later. At the same time, ensuring that everything needed to reproduce the project is reachable from a single top-level object — even when the data physically live elsewhere on the filesystem — is what STAMPED calls **S**elf-containment (the "don't look up" rule: no reliance on implicit external state). Tools such as *DataLad* (discussed in the data management chapter) make it practical to hold code and large data together in one logical project while keeping their storage separate. + ### Folder structure A consistent and rational folder structure is key to good project organization. For a simple Python project, I recommend starting using the package organization provided by `uv`. diff --git a/book/references.bib b/book/references.bib index 5d1d4e2..cea58a4 100644 --- a/book/references.bib +++ b/book/references.bib @@ -1337,3 +1337,27 @@ @article{Open-Science-Collaboration:2015aa volume = {349}, year = {2015}, bdsk-url-1 = {https://doi.org/10.1126/science.aac4716}} + +@article{Halchenko:2021aa, + author = {Halchenko, Yaroslav O. and Meyer, Kyle and Poldrack, Benjamin and Solanky, Debanjum Singh and Wagner, Adina S. and Gors, Jason and MacFarlane, Dave and Pustina, Dorian and Sochat, Vanessa and Ghosh, Satrajit S. and M{\"o}nch, Christian and Markiewicz, Christopher J. and Waite, Laura and Shlyakhter, Ilya and de la Vega, Alejandro and Hayashi, Soichi and H{\"a}usler, Christian Olaf and Poline, Jean-Baptiste and Kadelka, Tobias and Skyt{\'e}n, Kusti and Jarecka, Dorota and Kennedy, David and Strauss, Ted and Cieslak, Matt and Vavra, Peter and Ioanas, Horea-Ioan and Schneider, Robin and Pfl{\"u}ger, Mika and Haxby, James V. and Eickhoff, Simon B. and Hanke, Michael}, + doi = {10.21105/joss.03262}, + journal = {Journal of Open Source Software}, + number = {63}, + pages = {3262}, + publisher = {The Open Journal}, + title = {DataLad: distributed system for joint management of code, data, and their relationship}, + url = {https://doi.org/10.21105/joss.03262}, + volume = {6}, + year = {2021}, + bdsk-url-1 = {https://doi.org/10.21105/joss.03262}} + +@misc{Macdonald:2026aa, + author = {Macdonald, Austin and Baker, Cody and To, Isaac and Halchenko, Yaroslav O.}, + doi = {10.31222/osf.io/f3h82_v1}, + month = may, + note = {Preprint}, + publisher = {Center for Open Science}, + title = {{STAMPED} principles for reproducible research objects}, + url = {https://doi.org/10.31222/osf.io/f3h82_v1}, + year = {2026}, + bdsk-url-1 = {https://doi.org/10.31222/osf.io/f3h82_v1}} diff --git a/book/sharing.md b/book/sharing.md index 586c1db..1070338 100644 --- a/book/sharing.md +++ b/book/sharing.md @@ -63,6 +63,8 @@ At present, the easiest way to achieve this is to use the direct connection from There is another emerging standard PID for code known as the Software Hash Identifier (SWHID), which is being developed by the [Software Heritage](https://www.softwareheritage.org) organization that also runs an archive for software preservation. Unlike most PIDs, which are *extrinsic* in the sense that they have no direct relation to the content of the objects that they refer to, the SWHID is an *intrinsic* identifier that is based on a hash of the content (similar to the hashes that are used for commits in *git*). This has the benefit that one can directly validate whether code matches the SWHID, and may become more prevalent in the future. Saving code to the Software Heritage archive is as easy as submitting a [Save Code Now](https://archive.softwareheritage.org/save/) request. +Depositing releases on Zenodo, Software Heritage, container registries (Docker Hub, Quay), and — for large data — the wide range of portals supported by *git-annex* and *DataLad* is the essence of the STAMPED [@Macdonald:2026aa] **D**istributability principle: every referenced module of a research object should be persistently retrievable by others, with resolvable license identifiers. + ### Software citation As software resources are increasingly recognized as legitimate scientific contributions, it is increasingly common for them to be cited in research papers and included on *curricula vitae* for academic advancement and hiring. [@Smith:2016aa] laid out a set of principles for the citation of software: @@ -118,6 +120,8 @@ For researchers who use software in their research, it's important to cite the s In addition to citation information there are a number of other metadata that are important in order to make the code FAIR. A set of guidelines regarding software metadata have been laid out in the [RSMD](https://fair-impact.github.io/RSMD-guidelines/) (Research Software MetaData Guidelines for End-Users) project [@Gruenpeter:2024aa]. An emerging standard for the specification of software metadata is the `codemeta.json` file, which provides a standard vocabulary for the specification of software metadata. This file uses the *JSON-LD* format that I mentioned in a previous chapter, which links the terms in the dictionary to a format vocabulary. The [codemeta-generator](https://codemeta.github.io/codemeta-generator/) tool provides an easy interface for generating of these files. GitHub itself doesn't do anything special with the `codemeta.json` contents, but if the software is archived in Software Heritage then the project will be searchable by the specified metadata. Because this is becoming the standard, generating metadata now will also help ensure that your project remains findable in the future. +Machine-readable metadata files such as `CITATION.cff` and `codemeta.json` are the citation-and-identity end of what the STAMPED framework calls the **A**ctionability principle: rather than describing in prose how the software should be cited and identified, they let humans and tools (and, increasingly, AI agents) act on the information directly. + Because many systems (e.g. the *PyPI* package archive) do not use `codemeta.json`, it's also important to put relevant information in other files that may be used. For Python code, the `pyproject.toml` file allows specification of a number of metadata elements; in particular, it's important to specify the name, version, description, license, authors, and keywords under the `[project]` section, and project URLs under the `[project.urls]` section, since these are used by PyPI for searching packages in the index. ### Software versioning diff --git a/book/workflows.md b/book/workflows.md index 22ce741..3e6a544 100644 --- a/book/workflows.md +++ b/book/workflows.md @@ -46,6 +46,8 @@ In the earlier chapter on Data Management I discussed the FAIR (Findable, Access There are certainly some contexts where a more formal structure adhering in detail to the FAIR Workflows standard may be required, as in large collaborative projects with specific compliance objectives, but these rough guidelines should get a researcher most of the way there. +Many of these practices also correspond directly to STAMPED principles [@Macdonald:2026aa] (see the [data management chapter](data_management.md)): version control and requirements files support **Tracking** and **Portability**, standard organization and file formats support **Modularity** and **Actionability**, and a public repository with clear metadata supports **Distributability**. + ## Piping and chaining @@ -261,6 +263,8 @@ The use of DAGs to represent workflows provides a number of important benefits: There are a couple of additional benefits to using a workflow engine, which I will discuss in more detail in the context of a more complex workflow. The first is that they generally deal automatically with the storage of intermediate results (known as *caching* or *checkpointing*), which can help speed up execution when nothing has changed and allow continued execution if the process is interrupted. The second is that the workflow engine uses the execution graph to optimize the schedule of computations, only performing those operations that are actually needed. This is similar in spirit to the concept of *lazy execution* used by packages like *Polars*, in which the system optimizes computational efficiency by first analyzing the full computational graph. +Expressing an analysis as a workflow rather than a prose description is what the STAMPED framework calls the **A**ctionability principle: the workflow file is a machine-actionable specification that others (and future you) can execute, not just documentation of what was done. + ### General-purpose versus domain-specific workflow engines With the growth of data science within industry and research, there has been an explosion of new workflow management systems that aim to solve particular problems; a list of these can be found at [awesome-workflow-engines](https://github.com/meirwah/awesome-workflow-engines). It's also worth noting that there are a number of domain-specific workflow engines that are specialized for particular kinds of data and workflows. Examples include [*Galaxy*](https://galaxyproject.org/) which is specialized for bioinformatics and genomics, and [*Nipype*](https://nipype.readthedocs.io/en/latest/index.html) which is specialized for neuroimaging analysis workflows. If your research community uses one of these then it's worth exploring that engine as your first option, since it will probably be well supported within the community. However, a benefit of using a general-purpose engine is that they will often be better maintained and supported, and AI tools will likely have more examples to work from when generating new workflows. @@ -619,7 +623,9 @@ Pulling singularity image docker://jupyter/scipy-notebook:x86_64-ubuntu-22.04. ... ``` -As with conda, it's worth noting that *Snakemake* will store the *Apptainer* image within the `.snakemake` directory, which can sometimes be quite large; for the *Jupyter* image linked above, it was about 1.2 GB, but I have seen containers up to 10 GB or more on occasion. +As with conda, it's worth noting that *Snakemake* will store the *Apptainer* image within the `.snakemake` directory, which can sometimes be quite large; for the *Jupyter* image linked above, it was about 1.2 GB, but I have seen containers up to 10 GB or more on occasion. + +Running a workflow inside a fresh container image, rather than in-place against whatever happens to be installed on the host, is a practical form of the STAMPED **E**phemerality principle: the environment is staged from an explicit specification, used, and can be discarded, so that a successful run confirms the specification was complete. The same pattern applies when a continuous-integration system executes the workflow on a fresh worker for every commit, and on high-performance computing systems where each job runs in its own scratch directory. ### Best practices for *Snakemake* workflows diff --git a/docs/stamped-integration-plan.md b/docs/stamped-integration-plan.md new file mode 100644 index 0000000..db4df11 --- /dev/null +++ b/docs/stamped-integration-plan.md @@ -0,0 +1,312 @@ +# Plan: Weave STAMPED principles into the book + +**Status:** draft for review — nothing has been edited in the book yet. +**Branch:** `enh-stamped` +**Owner:** Yaroslav (with Russ as author-of-record on the book). +**Preprint to cite:** *STAMPED principles for reproducible research objects*, +Macdonald et al., 2026, [doi:10.31222/osf.io/f3h82_v1](https://doi.org/10.31222/osf.io/f3h82_v1) +(canonical BibTeX in §10). +**Companion artifact to reference:** interactive checklist at +. +**Paper sources / verification base:** `/home/yoh/proj/stamped-principles/` +(paper under `stamped-paper/`, schema under `stamped-principles-schema/`, +checklist schema under `stamped-checklist-schema/`). + +> How to review: leave comments inline (any lines) or add a `> yoh:` / +> `> russ:` block under a section. Sections labelled **[decision needed]** +> block the next step. +> +> **Revision status (2026-07-08):** Decisions received on all five §8 +> questions. §4/§7/§8/§9 updated to reflect those decisions; the STAMPED +> preprint's canonical BibTeX (via `doiref`) is captured in §10. Ready +> for Russ's pass or for execution to begin. + +--- + +## 1. Motivation + +STAMPED formalises seven principles (**S**elf-containment, **T**racking, +**A**ctionability, **M**odularity, **P**ortability, **E**phemerality, +**D**istributability) for how research objects — code + data + environment + +provenance — ought to be structured and managed so that others can re-execute +and extend them. + +The paper explicitly positions STAMPED as the **operational-layer companion to +FAIR**: FAIR is discovery/governance-oriented; STAMPED covers day-to-day +practices. The book already teaches most of the underlying practices, cites +FAIR / FAIR4RS / WCI-FW, but never names or attributes them as STAMPED. + +**Goal of the integration:** give readers the vocabulary and the checklist +they need to self-assess and communicate operational maturity — without +duplicating existing content and without rewriting chapters. + +--- + +## 2. Baseline: what the book already covers + +### Cited frameworks + +| Framework | Where | Citation | +|------------------------------------------|-----------------------------------------|-----------------------------------------| +| FAIR (data) | `book/data_management.md:11–47` | `[@Wilkinson:2016aa]` | +| FAIR4RS (software) | `book/sharing.md` (cross-refs) | `[@Barker:2022aa]` | +| FAIR for workflows (WCI-FW) | `book/workflows.md:34–47` (8-item list) | `[@Wilkinson:2025aa; @Visser:2023aa]` | +| Reproducibility terminology (Turing Way) | `book/introduction.md:132–143` | `[@book.the-turing-way.org]` | +| Software citation principles | `book/sharing.md:68–75` | `[@Smith:2016aa]` | +| DataLad | `book/data_management.md:956` | `[@Halchenko:2021aa]` (added on `main`) | + +### Taught implicitly, not attributed to any principle framework + +| Practice | Where | STAMPED principle it exemplifies | +|----------------------------------------------------------|-------------------------------------------------------------------|----------------------------------| +| BIDS layout / subdatasets | `book/project_organization.md:129–141` | Modularity, Self-containment | +| Traceability & logging in workflows | `book/workflows.md:25` | Tracking, Actionability | +| Containers (Docker/Apptainer) as reproducibility posture | `book/project_organization.md:447–458`; `book/sharing.md:312–315` | Portability, Ephemerality | +| Environment pinning (`uv.lock`, Docker) | `book/sharing.md:312+`; `book/project_organization.md` | Portability, Distributability | +| DataLad-based provenance | `book/data_management.md:954–1000+` | Tracking | +| Zenodo / Software Heritage archiving | `book/sharing.md` (PIDs section) | Distributability | + +**No mentions of STAMPED anywhere in the book.** + +--- + +## 3. Guiding constraints for the edits + +1. **Complement, don't duplicate.** STAMPED enters at two anchor points + (per §8.4): a short framing paragraph in `book/introduction.md` and a + deeper subsection right after FAIR in `book/data_management.md`. Not + a new chapter. +2. **Attach at existing seams.** Every principle maps 1:1 to a chapter that + already teaches the practice. No new headings unless necessary. +3. **Push readers to the checklist** as the actionable artifact rather than + reproducing normative text in prose. Link + and reference specific + `must/NNN` / `should/NNN` IDs where useful. +4. **One PR, multiple reviewable commits** (per §7 / §8 responses) so + Russ sees the whole solution but can drop or request changes to + individual pieces. +5. **Follow the book's existing BibTeX convention** (`Author:YEARaa`), not + the STAMPED paper's own natbib style (`author_word_year`). +6. **Preserve Russ's voice.** Insertions are short, prose-style, and match + surrounding tone. No sidebars/callouts unless approved in Phase 3. + +--- + +## 4. Concrete per-chapter edit plan + +### Phase 1 — Anchor + primary attribution + +Goal: one PR, small, review-friendly, no per-chapter surgery yet. + +| File | Location | Edit | +|---------------------------|--------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `book/references.bib` | append | Add `@misc{Macdonald:2026aa, ...}` for the STAMPED preprint (DOI `10.31222/osf.io/f3h82_v1`). Full author list: Macdonald, Baker, To, Halchenko. Canonical BibTeX (from `doiref`) captured verbatim in §10; adjust key + fields to match BibDesk convention before commit. | +| `book/introduction.md` | near reproducibility crisis section (~line 148) | **Primary framing spot (per §8.4).** Short paragraph: FAIR is discovery-oriented; STAMPED is the operational companion — list the 7 principles by name, forward to `book/data_management.md` for deeper coverage. Cite `[@Macdonald:2026aa]`. | +| `book/data_management.md` | new subsection right after FAIR (~line 47) | **Deep coverage (per §8.4)**, referenced from the introduction. ~2 short paragraphs: 7 principles in one sentence each, graduated MUST/SHOULD/MAY structure, link to `checklist.stamped-principles.org`, brief COI disclosure per §8.5. Cite `[@Macdonald:2026aa]`. | +| `book/workflows.md` | inside "FAIR-inspired practices" list (~line 34) | Single sentence noting each listed practice is also a STAMPED requirement (Actionability, Portability, Distributability), one line pointer. | + + +### Phase 2 — Per-principle attachment points + +Small (1–3 sentence) insertions at natural seams. Could be one PR or split +per-chapter. + +| Principle | Chapter / section it attaches to | Proposed edit | +|----------------------|--------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| **S**elf-containment | `book/project_organization.md` (project-layout sections; BIDS) | Name the "everything under one root" practice as STAMPED-S. | +| **T**racking | `book/data_management.md:954+` (DataLad section) | One sentence: content-addressed VCS + code-driven provenance = STAMPED-T; DataLad exemplifies it. | +| **A**ctionability | `book/workflows.md` (executable pipelines); `book/sharing.md` §CITATION.cff / codemeta.json | Frame as "executable specifications, not documentation (although with AI might become such in some cases)" = STAMPED-A. | +| **M**odularity | `book/project_organization.md:129–141` (BIDS, subdatasets, code separately as might be different life cycle) | Name the practice as STAMPED-M; note per-module license check. | +| **P**ortability | `book/project_organization.md:447–458` (containers); `book/HPC.md` (env modules); `book/sharing.md:312` (Docker/uv.lock) | STAMPED-P: env explicit + version-controlled + not depending on undocumented host state. | +| **E**phemerality | `book/HPC.md`; `book/workflows.md` (CI re-runs; runs on HPC, greatly assists reproducibility!) | Reframe per §8.3 (git-independent): "stage inputs + code in a temporary location and run there — don't run in-place." Call out HPC per-job scratch dirs as an existing exemplar of STAMPED-E; CI runs similarly. | +| **D**istributability | `book/sharing.md` (Zenodo, Software Heritage, container hubs, wide range of portals supported by git-annex/DataLad) | STAMPED-D: all referenced modules persistently retrievable; each has resolvable license. | + +### Phase 3 — Pedagogical extension (in scope per §8.2; **separate commit**) + +- **Concise mapping table** in `book/extras.md`: book chapters ↔ STAMPED + principles ↔ representative checklist items. Kept small; not a full + duplication of `checklist.stamped-principles.org`. Lands in its own + commit within the same PR (per §8.2). +- **"STAMPED checkpoint" callouts** at chapter ends *(optional add-on, + not required)*: 1–2 concrete checklist items (`must/004`, + `should/001`, …) the reader can now tick. Draft only if Russ signals + interest during PR review; otherwise skip. + +--- + +## 5. Explicit non-goals + +- Not adding CARE, TRUST, PROV-DM, DUO, Ten Simple Rules. Those are separate + frameworks; a survey subagent listed them as "gaps" but they are out of + scope for this integration. +- Not touching the book's existing FAIR section, other than to append the + STAMPED companion right after it. +- Not renaming, restructuring, or reordering chapters. +- Not rewriting the DataLad / containers / BIDS coverage — just attaching + attribution. + +--- + +## 6. Verification steps before writing edits + +1. Read `stamped-paper/main.tex` §Introduction + §Principles fully so + one-line-per-principle summaries match canonical wording (avoid + paraphrase drift). +2. ~~Grab exact author list, year, DOI…~~ **Done** — canonical BibTeX + from `doiref` captured in §10 Appendix; preprint DOI verified to + resolve. +3. Confirm checklist site URL and stability, plus the `must/NNN` / + `should/NNN` anchor scheme (verify at least one anchor resolves). +4. Local build check (repo uses `myst.yml` at book root) that citations + render before pushing. + +--- + +## 7. Delivery / sequencing + +**Decided:** one PR against `main` from branch `enh-stamped`, with +multiple commits so Russ can review the whole integration end-to-end +while still being able to drop or request changes to individual pieces. + +Suggested commit sequence (each independently revertable): + +1. `docs/stamped-integration-plan.md` — this plan file *(already + staged).* +2. `book/references.bib` — add STAMPED entry (`Macdonald:2026aa`). +3. **Phase 1** anchor + attribution: + - `book/introduction.md` — primary STAMPED framing paragraph. + - `book/data_management.md` — deep STAMPED subsection after FAIR. + - `book/workflows.md` — cross-reference inside the FAIR-inspired list. + *(May split into 2–3 sub-commits if diffs get large.)* +4. **Phase 2** per-principle attachments — one commit per attachment + chapter to keep review chunks small: + - `book/project_organization.md` (S, M, P) + - `book/data_management.md` (T — one line under DataLad section) + - `book/workflows.md` (A, E) + - `book/sharing.md` (A, D) + - `book/HPC.md` (P, E) +5. **Phase 3** — `book/extras.md` mapping table, its own commit (per + §8.2). +6. *(Optional)* "STAMPED checkpoint" callouts — separate commit; drop + if Russ prefers. + +--- + +## 8. Decisions + +Recorded from Yaroslav's review of this plan. Original questions + +verbatim responses retained under each decision as an audit trail. + +### 8.1 Bib entry style — **Resolved** + +**Decision:** Full author list. Bibkey `Macdonald:2026aa` (Austin +Macdonald is first author, not Halchenko). Canonical BibTeX from +`doiref` captured verbatim in §10 for the reference. + +> **Original question:** should the entry use full author list, or +> shortened `{Halchenko, Yaroslav O. and others}`? +> +> **Response:** `@article{Halchenko:2026aa, ...}` does not sound right +> — first author is Austin Macdonald. `doiref` output captured; use full +> list and the more proper identifier. + +### 8.2 Phase 3 scope — **Resolved** + +**Decision:** In scope. Add a concise mapping table to `book/extras.md` +in a **separate commit** within the same PR. "STAMPED checkpoint" +callouts remain optional (draft only if Russ signals interest). + +> **Original question:** are the callouts + mapping table in, sample-first, or skipped? +> +> **Response:** (a) — add concise table to `book/extras.md`. Do it in +> a separate commit though. + +### 8.3 Ephemerality treatment — **Resolved** + +**Decision:** Full but concise treatment. Reframe independently of +`git`: "copy everything needed for execution into some temporary +location instead of running in-place." HPC's per-job scratch model +already exercises this naturally — call that out explicitly. + +> **Original question:** give Ephemerality full treatment or soft-pedal +> it? +> +> **Response:** `git` is just a helper tool here. Ephemerality could be +> presented as "copy everything you think needed for execution into +> some temporary location instead of running 'inplace'". Give it good +> treatment, albeit concise. + +### 8.4 STAMPED overview placement — **Resolved** + +**Decision:** Two-tier presentation. Brief framing paragraph in +`book/introduction.md` (per (a) — more prominent). Deep coverage +subsection after FAIR in `book/data_management.md` (per (c) — original +proposal), referenced from the introduction. + +> **Original question:** intro (a), standalone chapter (b), or after +> FAIR in `data_management.md` (c)? +> +> **Response:** (a) — mention in introduction, then extend after FAIR +> in `book/data_management.md`. + +### 8.5 Contributor / conflict-of-interest disclosure — **Resolved** + +**Decision:** Include the proposed disclosure line in the +`book/data_management.md` STAMPED subsection. Russ may adjust wording +during PR review. + +Line to insert (as drafted): + +> *Full disclosure: STAMPED was co-developed by Yaroslav Halchenko, who +> also contributed to this book.* + +> **Response:** Above is alright with me; Russ might adjust when in PR. + + +--- + +## 9. What lands where — quick summary table + +| Book file | Phase 1 | Phase 2 | Phase 3 | +|--------------------------------|---------------------------------------------------------------|-----------------------------------------------------|------------------------------------------| +| `book/references.bib` | + STAMPED entry (`Macdonald:2026aa`) | — | — | +| `book/introduction.md` | STAMPED framing paragraph (**primary** framing spot) | — | — | +| `book/data_management.md` | STAMPED subsection after FAIR (deep coverage, COI disclosure) | +1 line under DataLad (Tracking) | *(callout — optional)* | +| `book/workflows.md` | +1 line under FAIR-inspired list | +1 line (Actionability, Ephemerality) | *(callout — optional)* | +| `book/sharing.md` | — | +1 line (Distributability, Actionability) | *(callout — optional)* | +| `book/project_organization.md` | — | +1 line (Self-containment, Modularity, Portability) | *(callout — optional)* | +| `book/HPC.md` | — | +1 line (Portability, Ephemerality) | *(callout — optional)* | +| `book/extras.md` | — | — | **Mapping table** (own commit, per §8.2) | + +--- + +## 10. Appendix: canonical BibTeX for the STAMPED preprint + +Captured verbatim from `doiref https://doi.org/10.31222/osf.io/f3h82_v1` +(shell output, per §8.1 response): + +```bibtex +@article{Macdonald_2026, + title={STAMPED principles for reproducible research objects}, + url={http://dx.doi.org/10.31222/osf.io/f3h82_v1}, + DOI={10.31222/osf.io/f3h82_v1}, + publisher={Center for Open Science}, + author={Macdonald, Austin and Baker, Cody and To, Isaac and Halchenko, Yaroslav O}, + year={2026}, + month=May +} +``` + +Adjustments before appending to `book/references.bib`: + +- Rename key from `Macdonald_2026` → `Macdonald:2026aa` to match the + book's BibDesk convention (see `Halchenko:2021aa`, `Bannier:2021aa`, + etc.). +- Consider changing `@article` → `@misc` since this is a preprint (or + keep `@article` with a `note = {Preprint}` — check other preprint + entries in the book's .bib for the local convention). +- Add `bdsk-url-1 = {https://doi.org/10.31222/osf.io/f3h82_v1}` to + match the BibDesk-style entries in the rest of the file. +- Upgrade DOI URL from `http://` → `https://` in the `url` field. +- If the paper is accepted at a peer-reviewed venue (e.g. Sci Data) + before this PR merges, replace with the venue's citation instead.