Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

switching this lesson to the new Carpentries Workbench Template #23

Merged
merged 70 commits into from
Oct 8, 2024
Merged
Show file tree
Hide file tree
Changes from 69 commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
a19df5b
switching this lesson to the new Carpentries Workbench Template
Aug 30, 2024
998b54c
update CITATION and CITATION.cff
Sep 13, 2024
546ea9c
rename FIXME.Rproj to hpc-chapel.Rproj
Sep 16, 2024
88ac528
edit FIXME tags in CONTRIBUTING.md and config.yaml
Sep 16, 2024
a345b2e
change all Carpentries references and links to High Performance Compu…
Sep 16, 2024
63c3f85
remove references to Cray (that no longer exists); add open-source in…
Sep 16, 2024
93e6b62
fix the 'form' typo in the code and output
Sep 16, 2024
fc3b50f
rewrite text in 14-parallel-case-study.md to remove references to pre…
Sep 16, 2024
7f36b65
remove prompt characters from bash blocks in all files
Sep 16, 2024
ce5893d
make the keypoints in 14-parallel-case-study.md more specific
Sep 16, 2024
9545546
add Chapel installation instructions for Mac, Linux, Windows; Windows…
Sep 16, 2024
9b606c0
add more generic 'module avail chapel' advice instead of the specific…
Sep 16, 2024
a5543b6
corrected a typo
Sep 16, 2024
5c72240
rewrite wording around 'basic maths'
Sep 19, 2024
6cd1e50
remove the paragraph distinguishing variable initialization and varia…
Sep 19, 2024
573402d
add a call-out where we try to use an un-initialized variable
Sep 19, 2024
5243788
update the error message to 'cannot assign to const variable'
Sep 19, 2024
c5f409d
floating-point division
Sep 20, 2024
6a43014
edit wording on memory
Sep 20, 2024
c8f1e34
change variable names: tt -> tmp, curdif -> delta, mindif -> toleranc…
Sep 20, 2024
0c328d3
at compile-time
Sep 20, 2024
899c5d6
mixed initialization and assignment: new wording, corrected syntax, a…
Sep 20, 2024
a6a92cf
rewrite 02-variables.md: first const vs. var, then initializing varia…
Sep 21, 2024
5b6426c
provide an explanation of the problem before introducing temperature-…
Sep 21, 2024
3dd4deb
change the keypoint on const vs. var variables
Sep 21, 2024
048fe12
correct a typo
Sep 21, 2024
dac45dc
added three key points
Sep 21, 2024
2639070
correct wording around arrays (values need not be sequential)
Sep 21, 2024
894963e
describe the ghost layer; set the initial temperature there to 0.0
Sep 23, 2024
8876d3d
rewrite the paragraph about points close to the boundary
Sep 23, 2024
b34c8cd
use consistent language with temp and temp_new throughout all chapters
Sep 23, 2024
e53093a
typo correction
Sep 23, 2024
a8d1aeb
increase niter to 10_000 so that we reach convergence after 7505 iter…
Sep 23, 2024
5b84e13
rewrite the key point discussing both for and while loops
Sep 23, 2024
2f08267
remove 'functional programming'
Sep 23, 2024
fe26452
add a description of the tuple example in 06-procedures.md; change *v…
Sep 23, 2024
4654984
moved recursion to the end of the key points
Sep 23, 2024
323ed66
in Chapel 2.0 no longer need -o to have a similar-named executable; r…
Sep 23, 2024
9601fe9
merge two bash blocks; remove all prompt characters
Sep 23, 2024
818d129
provide the solution to Challenge 4
Sep 23, 2024
611a716
change the key point about measuring code performance
Sep 23, 2024
f8cabc6
rename n outputFrequency in 14-parallel-case-study.md
Sep 23, 2024
6cfa825
finish sentences with a period
Sep 23, 2024
b69eb96
reformulate the analogy with multiple dispensers
Sep 24, 2024
fc5a9c0
improve text around distributed memory
Sep 24, 2024
3ec37d9
add a key point about Chapel communication under the hood
Sep 24, 2024
f020ce6
fix typo
Sep 24, 2024
29be289
add lesson objectives to several chapters
Sep 24, 2024
146bf4f
reduce the number of loops in this standalone example to 10 so that o…
Sep 24, 2024
abcd7f4
convert the discussion into two multiple-choice exercises
Sep 24, 2024
1b83e0e
add salloc command (as a refresher; covered earlier) to start an inte…
Sep 24, 2024
7401f55
include shells commands to check for CPU usage; remove the sentence a…
Sep 24, 2024
b5ae56b
update the code in the example; rewrite the sentence removing *polymo…
Sep 24, 2024
8a8ad54
fix the solution in Challenge 4; remoce the discussion on playing wit…
Sep 24, 2024
1c17a40
add objectives in 13-synchronization.md
Sep 24, 2024
1ab70df
provide more description before the example with the sync statement (…
Sep 24, 2024
0dd9f33
explain that *sync* can be either a statement or a type qualifier
Sep 24, 2024
cbd7f25
update the code for Chapel 2.x
Sep 24, 2024
cadea23
explain the example with writeEF/readFF in more detail; change the di…
Sep 24, 2024
7b29ed4
correct style
Sep 24, 2024
21db06a
reformulate the key point about all variables having a type
Sep 24, 2024
b5b6d20
break procedure definitions and usage into separate code blocks
Sep 24, 2024
bc91fe2
use dashes for itemized lists; restore original indentation in **No a…
Sep 24, 2024
6a41eb3
switch to using dashes for itemized lists through all chapters
Sep 24, 2024
23b7c07
use temp variable to write the solution to disk
Sep 24, 2024
bd3d7e6
move first mention of arrays down the text
Sep 27, 2024
c13a37e
introduce a very simple loop to show range usage
Sep 27, 2024
85c1a07
remove *One final thing*
Sep 27, 2024
1e1b70a
add a sentence about the dual use of square brackets with arrays: dec…
Sep 27, 2024
ab35382
Updated human-readable license text.
tkphd Oct 2, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
198 changes: 198 additions & 0 deletions .github/workflows/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,198 @@
# Carpentries Workflows

This directory contains workflows to be used for Lessons using the {sandpaper}
lesson infrastructure. Two of these workflows require R (`sandpaper-main.yaml`
and `pr-receive.yaml`) and the rest are bots to handle pull request management.

These workflows will likely change as {sandpaper} evolves, so it is important to
keep them up-to-date. To do this in your lesson you can do the following in your
R console:

```r
# Install/Update sandpaper
options(repos = c(carpentries = "https://carpentries.r-universe.dev/",
CRAN = "https://cloud.r-project.org"))
install.packages("sandpaper")

# update the workflows in your lesson
library("sandpaper")
update_github_workflows()
```

Inside this folder, you will find a file called `sandpaper-version.txt`, which
will contain a version number for sandpaper. This will be used in the future to
alert you if a workflow update is needed.

What follows are the descriptions of the workflow files:

## Deployment

### 01 Build and Deploy (sandpaper-main.yaml)

This is the main driver that will only act on the main branch of the repository.
This workflow does the following:

1. checks out the lesson
2. provisions the following resources
- R
- pandoc
- lesson infrastructure (stored in a cache)
- lesson dependencies if needed (stored in a cache)
3. builds the lesson via `sandpaper:::ci_deploy()`

#### Caching

This workflow has two caches; one cache is for the lesson infrastructure and
the other is for the the lesson dependencies if the lesson contains rendered
content. These caches are invalidated by new versions of the infrastructure and
the `renv.lock` file, respectively. If there is a problem with the cache,
manual invaliation is necessary. You will need maintain access to the repository
and you can either go to the actions tab and [click on the caches button to find
and invalidate the failing cache](https://github.blog/changelog/2022-10-20-manage-caches-in-your-actions-workflows-from-web-interface/)
or by setting the `CACHE_VERSION` secret to the current date (which will
invalidate all of the caches).

## Updates

### Setup Information

These workflows run on a schedule and at the maintainer's request. Because they
create pull requests that update workflows/require the downstream actions to run,
they need a special repository/organization secret token called
`SANDPAPER_WORKFLOW` and it must have the `public_repo` and `workflow` scope.

This can be an individual user token, OR it can be a trusted bot account. If you
have a repository in one of the official Carpentries accounts, then you do not
need to worry about this token being present because the Carpentries Core Team
will take care of supplying this token.

If you want to use your personal account: you can go to
<https://github.com/settings/tokens/new?scopes=public_repo,workflow&description=Sandpaper%20Token>
to create a token. Once you have created your token, you should copy it to your
clipboard and then go to your repository's settings > secrets > actions and
create or edit the `SANDPAPER_WORKFLOW` secret, pasting in the generated token.

If you do not specify your token correctly, the runs will not fail and they will
give you instructions to provide the token for your repository.

### 02 Maintain: Update Workflow Files (update-workflow.yaml)

The {sandpaper} repository was designed to do as much as possible to separate
the tools from the content. For local builds, this is absolutely true, but
there is a minor issue when it comes to workflow files: they must live inside
the repository.

This workflow ensures that the workflow files are up-to-date. The way it work is
to download the update-workflows.sh script from GitHub and run it. The script
will do the following:

1. check the recorded version of sandpaper against the current version on github
2. update the files if there is a difference in versions

After the files are updated, if there are any changes, they are pushed to a
branch called `update/workflows` and a pull request is created. Maintainers are
encouraged to review the changes and accept the pull request if the outputs
are okay.

This update is run weekly or on demand.

### 03 Maintain: Update Package Cache (update-cache.yaml)

For lessons that have generated content, we use {renv} to ensure that the output
is stable. This is controlled by a single lockfile which documents the packages
needed for the lesson and the version numbers. This workflow is skipped in
lessons that do not have generated content.

Because the lessons need to remain current with the package ecosystem, it's a
good idea to make sure these packages can be updated periodically. The
update cache workflow will do this by checking for updates, applying them in a
branch called `updates/packages` and creating a pull request with _only the
lockfile changed_.

From here, the markdown documents will be rebuilt and you can inspect what has
changed based on how the packages have updated.

## Pull Request and Review Management

Because our lessons execute code, pull requests are a secruity risk for any
lesson and thus have security measures associted with them. **Do not merge any
pull requests that do not pass checks and do not have bots commented on them.**

This series of workflows all go together and are described in the following
diagram and the below sections:

![Graph representation of a pull request](https://carpentries.github.io/sandpaper/articles/img/pr-flow.dot.svg)

### Pre Flight Pull Request Validation (pr-preflight.yaml)

This workflow runs every time a pull request is created and its purpose is to
validate that the pull request is okay to run. This means the following things:

1. The pull request does not contain modified workflow files
2. If the pull request contains modified workflow files, it does not contain
modified content files (such as a situation where @carpentries-bot will
make an automated pull request)
3. The pull request does not contain an invalid commit hash (e.g. from a fork
that was made before a lesson was transitioned from styles to use the
workbench).

Once the checks are finished, a comment is issued to the pull request, which
will allow maintainers to determine if it is safe to run the
"Receive Pull Request" workflow from new contributors.

### Receive Pull Request (pr-receive.yaml)

**Note of caution:** This workflow runs arbitrary code by anyone who creates a
pull request. GitHub has safeguarded the token used in this workflow to have no
priviledges in the repository, but we have taken precautions to protect against
spoofing.

This workflow is triggered with every push to a pull request. If this workflow
is already running and a new push is sent to the pull request, the workflow
running from the previous push will be cancelled and a new workflow run will be
started.

The first step of this workflow is to check if it is valid (e.g. that no
workflow files have been modified). If there are workflow files that have been
modified, a comment is made that indicates that the workflow is not run. If
both a workflow file and lesson content is modified, an error will occurr.

The second step (if valid) is to build the generated content from the pull
request. This builds the content and uploads three artifacts:

1. The pull request number (pr)
2. A summary of changes after the rendering process (diff)
3. The rendered files (build)

Because this workflow builds generated content, it follows the same general
process as the `sandpaper-main` workflow with the same caching mechanisms.

The artifacts produced are used by the next workflow.

### Comment on Pull Request (pr-comment.yaml)

This workflow is triggered if the `pr-receive.yaml` workflow is successful.
The steps in this workflow are:

1. Test if the workflow is valid and comment the validity of the workflow to the
pull request.
2. If it is valid: create an orphan branch with two commits: the current state
of the repository and the proposed changes.
3. If it is valid: update the pull request comment with the summary of changes

Importantly: if the pull request is invalid, the branch is not created so any
malicious code is not published.

From here, the maintainer can request changes from the author and eventually
either merge or reject the PR. When this happens, if the PR was valid, the
preview branch needs to be deleted.

### Send Close PR Signal (pr-close-signal.yaml)

Triggered any time a pull request is closed. This emits an artifact that is the
pull request number for the next action

### Remove Pull Request Branch (pr-post-remove-branch.yaml)

Tiggered by `pr-close-signal.yaml`. This removes the temporary branch associated with
the pull request (if it was created).
23 changes: 23 additions & 0 deletions .github/workflows/pr-close-signal.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
name: "Bot: Send Close Pull Request Signal"

on:
pull_request:
types:
[closed]

jobs:
send-close-signal:
name: "Send closing signal"
runs-on: ubuntu-latest
if: ${{ github.event.action == 'closed' }}
steps:
- name: "Create PRtifact"
run: |
mkdir -p ./pr
printf ${{ github.event.number }} > ./pr/NUM
- name: Upload Diff
uses: actions/upload-artifact@v4
with:
name: pr
path: ./pr

Loading