Skip to content
repo.extract - step 3 of 3

Then the packages.

Two halves of the same job. One rebuilds and tests the approved package list inside the workspace analysts actually use; the other builds that workspace from a written recipe first. Both end in binaries plus per-package evidence, so a claim that these packages work here is a documented fact rather than a hope.

The extractions
Files in the repo. Pull it and run - nothing to unpack.

Package build

package-build 7 files 57 KB extracted from eks.regeneron.ndexr.io
In plain terms

A quality-control kit for the R and Python packages analysts rely on. It takes the approved package list, rebuilds every package from its original source code inside the exact workspace people use, tests each one there, and writes down what passed and what failed - so "these packages work here" is a documented fact, not a hope.

A manifest-driven R package pipeline - fetch, check, build - as a plugin: estate-agnostic naming throughout, so it drops into anyone's CI or runs by hand inside the container it validates, in the working directory. Same stages, cache layout, check flags and evidence CSV as the source pipeline; gone is the machinery around them - no podman, no mounts, no registry, no arch matrix. One run validates this container, this architecture, one R version.

What it is

build-packages.sh manifest pipeline README.md run.R walkthrough.qmd

Needs
  • The container to validate, with R at the requested version (/opt/R/<ver> or PATH)
  • A compiler toolchain (gcc, g++, gfortran, make) - preflight names anything missing
  • Network for fetch and check (the dependency closure installs live, as in CI)
  • ~20 GB free in the working directory
Get it
# from R, via the rxedn catalogue
rxedn::rxedn_install_module("repo.extract.ndexr.io")

# the extraction lands at <domains_dir>/repo.extract.ndexr.io/package-build/

No archive to unpack and no checksum to compare: rxedn publishes each repo as an immutable <host>-<sha7>.tar.gz and records its sha256 in the catalogue index, so the commit is the identity. Pull it, and the files are simply there.

Run it
cd package-build

# from R (Workbench session) - session-core-aware, R version = the session's R:
#   source('run.R'); kickoff('preflight'); kickoff('all', background = TRUE)

./build-packages.sh --r-version 4.6.1 --stage preflight   # shell lane - seconds
./build-packages.sh --r-version 4.6.1 --stage all         # hours; PKG_REPO=<dated snapshot> to pin

Output stays put, in ./pkg-cache/bin/<arch>/<rver>/ (evidence in ./pkg-cache/evidence/) - nothing is copied out.

Every file - 7
  • build-packages.sh
  • manifest/manifest.dcf
  • manifest/sysreqs-rhel9.txt
  • pipeline/gitlab-ci.package-build.yml
  • README.md
  • run.R
  • walkthrough.qmd

RHEL Posit container + package pipeline

rhel-posit-pipeline 17 files 75 KB extracted from eks.regeneron.ndexr.io
In plain terms

A factory for the analyst workspace itself. From a written recipe it builds the standard RStudio/Posit environment data scientists log into, then runs every approved package through that environment as proof. The result is a workspace that can be recreated identically anywhere, and packages with evidence that they work in it.

The session container and the pipeline that qualifies R packages through it: a RHEL 9 Posit Workbench/Connect session image (four R, three Pythons, the full system-library floor) plus a builder container that compiles the declared set from source against exactly that image - fetch, install, check, test, build - ending in binaries and per-package evidence. A session and source builder only: no Kubernetes, no Helm, no orchestration, no registry assumptions, no credentials.

What it is

build-image.sh containers manifest pipeline README.md run-stage.sh run.R

Needs
  • A build host with podman or docker - image builds cannot happen inside a Workbench session
  • ~30 GB free (the session image carries four R + three Python runtimes, TeX, chromium)
  • Network for the image build (Posit CDN, dnf) and for fetch/install (PKG_REPO)
  • Hours for the first image build; the package stages resume from ./pkg-cache
Get it
# from R, via the rxedn catalogue
rxedn::rxedn_install_module("repo.extract.ndexr.io")

# the extraction lands at <domains_dir>/repo.extract.ndexr.io/rhel-posit-pipeline/

No archive to unpack and no checksum to compare: rxedn publishes each repo as an immutable <host>-<sha7>.tar.gz and records its sha256 in the catalogue index, so the commit is the identity. Pull it, and the files are simply there.

Run it
cd rhel-posit-pipeline

# from R (RStudio on the build host) - session-core-aware:
#   source('run.R'); build_image(); kickoff('all', background = TRUE)

./build-image.sh                                   # the two containers
./run-stage.sh --r-version 4.6.1 --stage fetch     # then install, check, test, build

Output stays put, in ./pkg-cache/bin/<arch>/<rver>/ (evidence in ./pkg-cache/evidence/) + the session image in the local engine - nothing is copied out.

Every file - 17
  • build-image.sh
  • containers/package-builder/build-packages.sh
  • containers/package-builder/Dockerfile
  • containers/session/Dockerfile
  • manifest/environments/r-full/bioc_packages.txt
  • manifest/environments/r-full/cran_packages.txt
  • manifest/environments/r-full/git_repos.dcf
  • manifest/environments/r-full/manifest.yaml
  • manifest/environments/r-minimal/bioc_packages.txt
  • manifest/environments/r-minimal/cran_packages.txt
  • manifest/environments/r-minimal/git_repos.dcf
  • manifest/environments/r-minimal/manifest.yaml
  • manifest/sysreqs-rhel9.txt
  • pipeline/gitlab-ci.rhel-posit-pipeline.yml
  • README.md
  • run-stage.sh
  • run.R
Declared inputs
What each extraction will build.
Package build

Carried in manifest/ , verbatim - the declared set this extraction will build. It travels in the repo, so a copy taken to a machine with no route back here still knows the full list.

File Records Size
manifest.dcf 197 records 10 KB
sysreqs-rhel9.txt 31 entries 1 KB
RHEL Posit container + package pipeline

Carried in manifest/ , verbatim - the declared set this extraction will build. It travels in the repo, so a copy taken to a machine with no route back here still knows the full list.

File Records Size
environments/r-full/bioc_packages.txt empty 0 KB
environments/r-full/cran_packages.txt 159 entries 2 KB
environments/r-full/git_repos.dcf 3 records 1 KB
environments/r-full/manifest.yaml 12 entries 1 KB
environments/r-minimal/bioc_packages.txt empty 0 KB
environments/r-minimal/cran_packages.txt 87 entries 1 KB
environments/r-minimal/git_repos.dcf 3 records 1 KB
environments/r-minimal/manifest.yaml 12 entries 1 KB
sysreqs-rhel9.txt 31 entries 1 KB