Skip to content
This repository has been archived by the owner on Sep 18, 2023. It is now read-only.

update installation instructions to current practice #120

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

update installation instructions to current practice #120

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
bc96542
MNT #119 remove installation shell script
prjemian Jul 12, 2022
6f553a8
DOC revise installation instructions
prjemian Jul 12, 2022
1ae2213
DOC #119 resolve fixme items
prjemian Jul 12, 2022
066afc4
DOC #119 describe & demo iconfig.yml
prjemian Jul 12, 2022
263c06c
MNT #119
prjemian Jul 12, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
198 changes: 151 additions & 47 deletions install/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ instrument.

**Contents**
- [Installation](#installation)
- [Typical Directory Layout](#typical-directory-layout)
- [Activate Conda "base" Environment](#activate-conda-base-environment)
- [Install MongoDB server](#install-mongodb-server)
- [Install Databroker Configuration File](#install-databroker-configuration-file)
Expand All @@ -14,6 +15,13 @@ instrument.
- [Test that Bluesky Works](#test-that-bluesky-works)
- [Create IPython Profile for Bluesky](#create-ipython-profile-for-bluesky)
- [Install Instrument Package](#install-instrument-package)
- [Download the Starting Point](#download-the-starting-point)
- [Edit Basic Details](#edit-basic-details)
- [conda environment name](#conda-environment-name)
- [databroker catalog name](#databroker-catalog-name)
- [instrument-based metadata](#instrument-based-metadata)
- [Remove training repository components](#remove-training-repository-components)
- [Install](#install)
- [Commit Instrument Package to Version Control](#commit-instrument-package-to-version-control)
- [Translate Previous SPEC Configuration](#translate-previous-spec-configuration)
- [Add Environment Configuration to .bash_aliases](#add-environment-configuration-to-bash_aliases)
Expand All @@ -31,6 +39,30 @@ commands, check that you are using the `bash` shell first.
Here's some [advice](https://stackoverflow.com/questions/3327013/how-to-determine-the-current-shell-im-working-on) to determine the
current shell.

## Typical Directory Layout

```text
~/
bin/
blueskyStarter.sh --> ~/bluesky/blueskyStarter.sh
bluesky/
blueskyStarter.sh
console/
__start_bluesky_instrument__.py
instrument/
.ipython-bluesky/
profile_bluesky/
startup/
__start_bluesky_instrument__.py --> ~/bluesky/console/__start_bluesky_instrument__.py
```

and [`__start_bluesky_instrument__.py`](https://github.com/BCDA-APS/bluesky_training/blob/main/bluesky/console/__start_bluesky_instrument__.py) is summarized by:

```py
import pathlib, sys
sys.path.append(str(pathlib.Path.home() / "bluesky"))
from instrument.collection import *
```

## Activate Conda "base" Environment

Expand Down Expand Up @@ -64,7 +96,6 @@ mintadmin@mint-vm:/tmp$ conda env list
# conda environments:
#
base * /home/mintadmin/Apps/anaconda

```


Expand All @@ -73,8 +104,8 @@ base * /home/mintadmin/Apps/anaconda
If you need to provide your own MongoDB server, follow this
[guide](https://github.com/BCDA-APS/use_bluesky/wiki/mongodb-server).

At the APS, the beam line controls group (BCDA) will assign the
REPOSITORY and its SERVER.
At the APS, the IT group manages the MongoDB servers and the beam line
controls group (BCDA) will make assignments (REPOSITORY and SERVER).

## Install Databroker Configuration File

Expand All @@ -83,7 +114,7 @@ databases) on the MongoDB database server.

Follow this template:

```
```yml
sources:
REPOSITORY:
args:
Expand All @@ -95,30 +126,35 @@ sources:
Replace the `REPOSITORY` and `SERVER` terms and write to file:
`~/.local/share/intake/catalogs.yml`.

NOTE: The actual directory is `${XDG_CONFIG_HOME}/intake/catalogs.yml`. The example above assumes the default value `XDG_CONFIG_HOME=$HOME}/.local/share`.

At the APS, the beam line controls group (BCDA) will assign the
REPOSITORY and its SERVER.
REPOSITORY and its SERVER. The maintain a list of all
[APS configurations](https://git.aps.anl.gov/bcda/bluesky-catalogs/-/blob/master/APS_assignments.yml)
(restricted access).

## Install Bluesky Environment

Get the installation configuration ([YAML](https://yaml.org)) file. The
file is versioned based on the APS run cycle. This YAML file is for the
first run (January-April) in 2021.
file is versioned based on the APS run cycle. The YAML file shown here is for the
third run (October-December) in 2022. Use the most recent YAML file available
in the repository. The appropriate environment name is built into the file.

```
```bash
cd /tmp
wget https://github.com/BCDA-APS/use_bluesky/raw/main/install/environment_2022_2.yml
wget https://github.com/BCDA-APS/use_bluesky/raw/main/install/environment_2022_3.yml
```

Create the custom conda environment named in the file (`bluesky_2021_1`):
Create the custom conda environment named in the file (`bluesky_2022_3`):

```
conda env create -f environment_2021_1.yml
```bash
conda env create -f environment_2022_3.yml
```

## Activate Bluesky Environment

```
conda activate bluesky_2021_1
```bash
conda activate bluesky_2022_3
```

### Test that Bluesky Works
Expand Down Expand Up @@ -335,45 +371,113 @@ If there is an existing `~/.ipython` directory (perhaps created for other use
from this account), then choose a unique directory for bluesky. Typical
alternative is `~/.ipython-bluesky`.

```
export BLUESKY_DIR=~/.ipython
```bash
export BLUESKY_DIR=~/.ipython-bluesky
mkdir -p "${BLUESKY_DIR}"
ipython profile create bluesky --ipython-dir="${BLUESKY_DIR}"
```


## Install Instrument Package

Remove the existing `startup` directory (created from `ipython profile
create` step above):
### Download the Starting Point

These steps install the instrument package from the training
repository. It is the best starting point for a new installation.

```bash
cd /tmp
git clone https://github.com/BCDA-APS/bluesky_training
cd bluesky_training/bluesky
```
cd "${BLUESKY_DIR}/profile_bluesky"
rm startup/README
rmdir startup

### Edit Basic Details

Edit content in the bluesky directory (and subdirectories below).

#### conda environment name

```bash
sed -i s+"training_2022"+"bluesky_2022_3"+g ./blueskyStarter.sh
sed -i s+"training_2022"+"bluesky_2022_3"+g ./run_qstarter_py.sh
```

#### databroker catalog name

Use the same name as `REPOSITORY` in the
[Install Databroker Configuration File](#install-databroker-configuration-file)
section above.

```bash
CATALOG_NAME=USE_ASSIGNED_CATALOG_NAME
sed -i s+"training"+"${CATALOG_NAME}"+g console/__start_bluesky_instrument__.py
sed -i s+"training"+"${CATALOG_NAME}"+g export_data.sh
sed -i s+"training"+"${CATALOG_NAME}"+g instrument/framework/initialize.py
sed -i s+"databroker_catalog training"+"databroker_catalog ${CATALOG_NAME}"+g instrument/iconfig.yml
sed -i s+"training"+"${CATALOG_NAME}"+g qserver.sh
sed -i s+"training"+"${CATALOG_NAME}"+g qstarter.py
sed -i s+"training"+"${CATALOG_NAME}"+g run_qstarter_py.sh
```

Download the install script:
#### instrument-based metadata

Start with some basic metadata about the instrument.
Leave a placeholder to record a user's proposal ID.
Keep in mind that metadata keys, such as these, may be used
to find and retireve data from the databroker.

```bash
# no white space in NAME_THE_INSTRUMENT
sed -i s+"beamline_id: Bluesky_training"+"beamline_id: NAME_THE_INSTRUMENT"+g instrument/iconfig.yml
# name you'd use to refer to this instrument
sed -i s+"instrument_name: BCDA EPICS Bluesky training"+"instrument_name: APS 44ID XYZ"+g instrument/iconfig.yml
# set a default value for proposal ID (update from shell for each new proposal's beam time)
sed -i s+"proposal_id: training"+"proposal_id: commissioning"+g instrument/iconfig.yml
```
cd "${BLUESKY_DIR}/profile_bluesky"
wget https://github.com/BCDA-APS/use_bluesky/raw/main/install/install_startup.sh

#### Remove training repository components

Rather than deleting the content, it is commented out
so the configuration and files described may be used
as reference.

The training repo uses some specific IOCs. Comment that out.

```bash
sed -i s+"^AD"+"# AD"+g instrument/iconfig.yml
sed -i s+"^GP"+"# GP"+g instrument/iconfig.yml
sed -i s+"^BLUESKY_MOUNT_PATH"+"# BLUESKY_MOUNT_PATH"+g instrument/iconfig.yml
```

Run the installer
The training repo uses some specific devices from those IOCs. Comment that out.

```bash
sed -i s+"^from ."+"# from ."+g instrument/devices/__init__.py
```
bash ./install_startup.sh BEAMLINE INSTRUMENT REPOSITORY

Comment out any example plans.

```bash
sed -i s+"^from ."+"# from ."+g instrument/plans/__init__.py
```

where `BEAMLINE` `INSTRUMENT` `REPOSITORY` each contain no white space
characters. The terms BEAMLINE and INSTRUMENT will be added to the standard
metadata added to every Bluesky
[_run_](https://blueskyproject.io/bluesky/documents.html?highlight=run).
Example: `bash ./install_startup.sh 45-ID FemtoScanner 45id_femtoscanner`
### Install

At the APS, the beam line controls group (BCDA) will assign the
REPOSITORY and its SERVER.
```bash
cd /tmp/bluesky_training
tar cf - bluesky | (cd "${HOME}" && tar xf -)
cd /tmp
/bin/rm -rf /tmp/bluesky_training

cd "${HOME}"
mkdir -p bin

cd "${HOME}"/bin
ln -s "${HOME}"/bluesky/blueskyStarter.sh ./

cp \
"${HOME}"/bluesky/console/__start_bluesky_instrument__.py \
"${HOME}"/.ipython-bluesky/profile_bluesky/startup/
```

## Commit Instrument Package to Version Control

Expand All @@ -397,7 +501,7 @@ The [`apstools`]() package has an application that will translate most
of the SPEC config file into ophyd commands. The output is to the
console. Use a pipe to direct the output to a new file:

```
```bash
export INSTRUMENT_DIR=${BLUESKY_DIR}/profile_bluesky/startup/instrument
spec2ophyd CONFIG_FILE | tee ${INSTRUMENT_DIR}/devices/spec.py
```
Expand All @@ -409,8 +513,8 @@ imports there.
<details>
<summary><tt>spec2ophyd</tt> example:</summary>

```
(bluesky_2021_1) mintadmin@mint-vm:/tmp$ spec2ophyd /home/mintadmin/Documents/projects/BCDA-APS/apstools/apstools/migration/config
```bash
(bluesky_2022_3) mintadmin@mint-vm:/tmp$ spec2ophyd /home/mintadmin/Documents/projects/BCDA-APS/apstools/apstools/migration/config
"""
ophyd commands from SPEC config file

Expand Down Expand Up @@ -501,18 +605,19 @@ I000 = c0.channels.chan06.s

## Add Environment Configuration to .bash_aliases

The `~/.bash_aliases` file is the usual place to customize the bash shell. It
is executed from `~/.bashrc` when a new console is created.
The `~/.bash_aliases` file is the usual place to customize the bash shell. If
it exists, it is executed from `~/.bashrc` when a new console is created.

Add these lines to `~/.bash_aliases`:
Add these lines to `~/.bash_aliases` (or `~/.bashrc` if the other does not exist):

```
export CONDA_ENVIRONMENT=bluesky_2021_1
export CONDA_ENVIRONMENT=bluesky_2022_3
alias become_bluesky='conda activate "${CONDA_ENVIRONMENT}" '
```

## Install Starter Script


The commands and settings to configure the local environment to begin a
Bluesky session can be arranged by a simple shell script that starts the
session. A sample starter has been provided:
Expand All @@ -536,12 +641,7 @@ and create the directory:
mkdir ~/bin
```

Add the starter script to `~/bin`:

```
cd ~/bin
ln -s ${BLUESKY_DIR}/profile_bluesky/startup/blueskyStarter.sh ./
```
The starter script was added above, in the [Install](#install) section.

TIP: You might rename `~/bin/blueskyStarter.sh` to something appropriate for
your instrument name, such as the hypothetical example above:
Expand All @@ -551,3 +651,7 @@ your instrument name, such as the hypothetical example above:

* Change to desired working directory.
* Start Bluesky session using the starter script.

```bash
blueskyStarter.sh
```
Loading