Proposal: CRAN support

[JosiahParry]

I want to use this issue as a way to draft my thinking around how we should adjust the use_cran_defaults() and vendor_pkgs() functions for CRAN support. This is a product of my challenges submitting to CRAN https://stat.ethz.ch/pipermail/r-package-devel/2024q2/010725.html

I am proving the efficacy in arcgisutils. When this succeeds, I will use the same approach for arcgisgeocode and then arcgisplaces

The package install size is 1.2mb but the vendored dependencies are 18mb. Uwe L. rejected the package on the basis that the tarball cannot be larger than 5mb. As such, I have worked out a pattern that I think that can be generalized for other R packages.

This utilizes a configure file (generated generated by configure.ac) and an ancilliary R script in tools/.

• ./tools/vendor.R - vendors the packages, creates src/rust/vendor.tar.xz and creates a ./tools/vendor.md5 which stores the md5 checksum
• tools/get-deps.R is called from configure which checks to see if vendor.tar.xz is in the source, if not, it downloads it from GitHub
  • then, the checksum of the downloaded tar.xz is compared to ./tools/vendor.md5, if they don't match an error is elicited (and I think this stops the process from CRAN's side)
  • if they do, the install continues

This approach would allow us to exceed the 5mb limit and still be within CRANs policies

Changes needed

• use_cran_defaults() should create:
  • tools/get-deps.R
  • configure.ac and configure
  • .gitattributes (see R-ArcGIS/arcgisutils@17b2520)
• vendor_pkgs() should create tools/vendor.md5

We should have a way to inform users that if their vendor.tar.xz file is larger than 5mb and that it should not be included when submitting to CRAN. I think this is less important, though.

The vignette should be simplified to focus on the workflow and THEN describe the changes that exist.

Suggested changes

We should also have just one Makevars / Makevars.win template that supports both CRAN and non-CRAN installation. This would simplify the structure of a project. This is also being used in arcgisutils.

Additionally, afterwards, we could consider modifying the configure file to install Rust but that could be a bit of work! particularly for windows.

Draft Files

get-deps.R

# TODO none of this is required if it isn't on CRAN
# what environment variable can we check?

# this is where the vendored dependencies go as a zip file
out_path <- "src/rust/vendor.tar.xz"

# if the file doesn't exist, download it
if (!file.exists(out_path)) {
  # inform that the file is being downloaded
  message("vendored dependencies are being downloaded...")

  # download the file
  dl_res <- download.file(
    # TODO interpolate this path for rextendr
    "https://github.com/R-ArcGIS/arcgisutils/raw/main/src/rust/vendor.tar.xz",
    out_path
  )

  # error if the download failed
  if (dl_res != 0) {
    stop("Vendored dependencies failed to download. Check that you have an active internet connection.")
  } else {
    message("vendored dependencies downloaded successfully!")
  }
} else {
  message("vendor.tar.xz found")
}

# read in the known md5 checksum
known_md5 <- readLines("tools/vendor.md5")

# get the md5 for the existent vendor.tar.xz
local_md5 <- tools::md5sum(out_path)

# verify that the md5 are identical
is_known <- local_md5 == known_md5

# if the md5 dont match
if (!is_known) {
  # remove the downloaded file
  file.remove(out_path)

  # exit with informative error
  msg <- paste0(
    "Cannot verify the md5 of vendor.tar.xz\n",
    "Expected: ", known_md5,
    "\nFound:    ", local_md5
  )

  # throw error
  stop(msg)
} else {
  # inform that checksums matched
  message("vendor.tar.xz md5 matched the known checksum\ncontinuing with installation")
}

vendor.R

#!/usr/bin/env Rscript
# we need to vendor the dependencies and then store the checksum
rextendr::vendor_pkgs()

# make the checksum
writeLines(
  tools::md5sum("src/rust/vendor.tar.xz"),
  "./tools/vendor.md5"
)

[JosiahParry]

Note that an md5 is not sufficient.

This means in some way have to be on top of providing cryptographically secure hashes?

[eitsupi]

I recommend to check https://github.com/PRQL/prqlc-r/blob/83216a97fd234c6f4cd8e975df8ed605bba5e9b8/cran-comments.md