Skip to content

Commit

Permalink
09 10 import (#5)
Browse files Browse the repository at this point in the history
* Added some debugging and formatting pages

* Source Update
  • Loading branch information
CallumWalley authored Aug 10, 2023
1 parent 6ce2d04 commit 142debd
Show file tree
Hide file tree
Showing 80 changed files with 285 additions and 175 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,5 @@
from_zendesk/cache/**
from_zendesk/import_worktree/**
public/*
production/*
**.pyc
24 changes: 18 additions & 6 deletions SPECIFICATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,22 +65,34 @@ Not used for anything currently. Info imported from Zendesk Page.
- `zendesk_article_id`:
- `zendesk_section_id`:


## Formatting

Most details about the extra markdown features can be found here.
Will copy the specifics that apply to this site when I get time.

https://squidfunk.github.io/mkdocs-material/reference/

## Code Formatting
### Admonation

### Blocks
!!! type
Something

import tensorflow as tf
import numpy as numpy
Where 'type' is one of these https://squidfunk.github.io/mkdocs-material/reference/admonitions/#type:note

Preferably
### Content Tabs

Look like this

```
=== "Tab One"
someting in the tab
=== "Tab two"
something else
```

## Code Formatting

### Blocks

```py
import tensorflow as tf
Expand Down
2 changes: 1 addition & 1 deletion docs/General/Announcements/Māui_upgrade_is_complete.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ possible and will do our best to assist you.

This upgrade brings Māui's operating environment up to the latest
supported release available for Cray's XC50 supercomputing platforms,
with performance, reliability, and security benefits. This includes more
with performance, reliability, and security benefits. This ../../../ more
up-to-date tooling and libraries with associated features and
performance benefits. This work also enables further upgrades to NeSI's
shared HPC storage system.
Expand Down
8 changes: 4 additions & 4 deletions docs/General/FAQs/How_to_replace_my_2FA_token.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,23 +23,23 @@ Log in to [my.nesi.org.nz](https://my.nesi.org.nz) and select the option
Once you've created a token, you should see the 'Manage Two-Factor
token' option.

![image2018-10-26\_15-36-10.png](../includes/image2018-10-26_15-36-10.png)
![image2018-10-26\_15-36-10.png](../../includes/images/image2018-10-26_15-36-10.png)

## Delete the Two-Factor token

Select the option 'Manage Two-Factor token' under 'Account'.

You should see the confirmation of the previous token.

![](../includes/mceclip0_13.png)
![](../../includes/images/mceclip0_13.png)

After 'Delete Token' has been selected:

![](../includes/mceclip1_2.png)
![](../../includes/images/mceclip1_2.png)

You will also receive an email confirmation:

![](../includes/mceclip3.png)
![](../../includes/images/mceclip3.png)

## Related content

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ provide feedback on. Each idea/feature on the Portal is represented as a
Select a card, choose the importance and add let us know why this is
important to you as well as your email address.

![card\_feedback.gif](../includes/card_feedback.gif)
![card\_feedback.gif](../../includes/images/card_feedback.gif)

When users provide feedback on features/ideas in this way, their
feedback is routed back to our Insights board.
Expand All @@ -39,7 +39,7 @@ You can also submit brand new ideas on the Portal. These will be routed
to our Insights board as well and reviewed by the product management
team.

![submit\_idea.gif](../includes/submit_idea.gif)
![submit\_idea.gif](../../includes/images/submit_idea.gif)

### What happens to the feedback?

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ As pictured in the screenshot below, you can type keywords into the
catalogue's search field to browse by a specific software name or using
more broad terms such as "machine learning". 

![MachineLearningSoftwareEnvironments-May2021.png](../includes/MachineLearningSoftwareEnvironments-May2021.png)
![MachineLearningSoftwareEnvironments-May2021.png](../../includes/images/MachineLearningSoftwareEnvironments-May2021.png)

For more information on NeSI's model and approach to application
support, refer to our [policy for the management of scientific
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,6 @@ efficiency rather than speed, and running on a single CPU is almost
always more efficient than any form of parallelisation. 

![*Four serial tasks run concurrently will finish faster than for
parallel 4cpu tasks CPUs one after another. *](../includes/4tasks.png)
parallel 4cpu tasks CPUs one after another. *](../../includes/images/4tasks.png)

</figure>
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,4 @@ please email
and wait for a member of our support team to confirm your account and
group membership.

![mceclip0.png](../includes/mceclip0_14.png)
![mceclip0.png](../../includes/images/mceclip0_14.png)
2 changes: 1 addition & 1 deletion docs/General/NeSI_Knowledge_Base/Formatting.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ This is a [Link](https://www.w3schools.com/html/html_links.asp)

Image

![redirects-and-pipes.png](../includes/6014468037775)
![redirects-and-pipes.png](../../includes/images/6014468037775)

------------------------------------------------------------------------

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ affiliated with an organisation supported by the
federation, you can request access via
[my.nesi.org.nz/register](https://my.nesi.org.nz/register).

![mceclip0.png](../includes/mceclip0_15.png)
![mceclip0.png](../../includes/images/mceclip0_15.png)

> ### Warning
>
Expand Down
6 changes: 3 additions & 3 deletions docs/General/NeSI_Policies/NeSI_Application_Support_Model.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ three-tier model.

## Tier 1

Includes applications (meaning tools, libraries and science
../../../ applications (meaning tools, libraries and science
applications) which:

1. Have a wide user base among users of the NeSI Compute and Analytics
Expand All @@ -48,7 +48,7 @@ applications) which:

## Tier 2

Includes applications (meaning tools, libraries and science
../../../ applications (meaning tools, libraries and science
applications) which:

1. Have a small but important user base, meaning they are used by
Expand All @@ -63,7 +63,7 @@ applications) which:

## Tier 3

Includes applications (meaning tools, libraries and science
../../../ applications (meaning tools, libraries and science
applications) that are required by one user, or have very limited use,
in which case:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ different options, listed in order of preference.
- ## MobaXterm

In addition to being a terminal emulator, MobaXterm also includes
In addition to being a terminal emulator, MobaXterm also ../../../
several useful features like multiplexing, X11 forwarding and a file
transfer GUI.

Expand Down
6 changes: 3 additions & 3 deletions docs/Getting_Started/Accessing_the_HPCs/Port_Forwarding.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,20 +128,20 @@ method described above. This is the recommended method.
You can tell if MobaXterm is using WSL as it will appear in the banner
when starting a new terminal session. 

![mceclip0.png](../includes/360004708596)
![mceclip0.png](../../includes/images/360004708596)

You can also set up port forwarding using the MobaXterm tunnelling
interface.

![mceclip1.png](../includes/360004708616)
![mceclip1.png](../../includes/images/360004708616)

You will need to create **two** tunnels. One from lander to mahuika. And
another from mahuika to itself. (This is what using an alias in the
first two examples allows us to avoid).

The two tunnels should look like this.

![mobakey.png](../includes/360004580035)
![mobakey.png](../../includes/images/360004580035)

<span class="wysiwyg-color-green110">■</span> local port
<span class="wysiwyg-color-orange90">■</span> remote port
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,10 +45,10 @@ If you some reason you can't do this, please contact NeSI support.
2. Click **My HPC Account** on left hand panel  and then **Setup
Two-Factor Authentication device**

<img src="../includes/4414700806543" alt="authentication_factor_setup.png" width="560" height="210" />
<img src="../../includes/images/4414700806543" alt="authentication_factor_setup.png" width="560" height="210" />

3. Click the "**Setup Two-Factor Authentication device**" link.
![](../includes/360001267755)
![](../../includes/images/360001267755)
4. After clicking on "Continue" you will retrieve the QR code.

5. Open your Authy or Google Authenticator app and click on the add
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,22 +36,22 @@ zendesk_section_id: 360000034315
Password** (If you are resetting your password this will read
**Reset Password**).
Note your** Username.
<img src="../includes/authentication_factor_setup.png" alt="authentication_factor_setup.png" width="442" height="166" />
<img src="../../includes/images/authentication_factor_setup.png" alt="authentication_factor_setup.png" width="442" height="166" />
**
3. Enter and verify your new password, making sure it follows the
[password
policy](https://support.nesi.org.nz/hc/en-gb/articles/360000336015).


#### <img src="../includes/SetNeSIaccountPassword.png" alt="SetNeSIaccountPassword.png" width="628" height="314" />
#### <img src="../../includes/images/SetNeSIaccountPassword.png" alt="SetNeSIaccountPassword.png" width="628" height="314" />

4. If the password set was successful, following confirmation label
will appear on the same page within few seconds

![change\_success.png](../includes/5662974048911)
![change\_success.png](../../includes/images/5662974048911)
5. Followed by an email confirmation similar to below

<img src="../includes/password_set_confirmation.png" alt="password_set_confirmation.png" width="440" height="285" />
<img src="../../includes/images/password_set_confirmation.png" alt="password_set_confirmation.png" width="440" height="285" />



Expand All @@ -77,13 +77,13 @@ zendesk_section_id: 360000034315

4. If the password **reset** was successful, following confirmation
label will appear on the same page within few seconds
1. - ![change\_success.png](../includes/5662974048911_0.name_me)
1. - ![change\_success.png](../../includes/images/5662974048911_0.name_me)

5. Followed by an email confirmation similar to below



<img src="../includes/password_set_confirmation_0.png" alt="password_set_confirmation.png" width="440" height="285" />
<img src="../../includes/images/password_set_confirmation_0.png" alt="password_set_confirmation.png" width="440" height="285" />



Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,15 +26,15 @@ Allocations will start on the first day of the next month and run for
one year. The diagram below illustrates how these quarterly call periods
are scheduled during the year:

![Quarterly\_Allocation\_Periods\_2021\_\_1\_.png](../includes/Quarterly_Allocation_Periods_2021__1_.png)
![Quarterly\_Allocation\_Periods\_2021\_\_1\_.png](../../includes/images/Quarterly_Allocation_Periods_2021__1_.png)

For example, if you apply for a new allocation on your existing project
in the month of October, we will review your application in October or
early November, you will be notified of your allocation by the end of
November, and your allocation will start on 1 December (as shown in the
graphic below).

![Blank\_Diagram\_\_1\_.png](../includes/Blank_Diagram__1_.png)
![Blank\_Diagram\_\_1\_.png](../../includes/images/Blank_Diagram__1_.png)

## Existing allocations

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ supercomputer is allocated by node-hours. Though some Māui nodes have
more RAM than others, we do not currently distinguish between low-memory
and high-memory nodes for allocation, billing or Fair Share purposes.

Each allocation on Māui includes an entitlement to use the Māui
Each allocation on Māui ../../../ an entitlement to use the Māui
ancillary nodes equally with other NeSI projects having Māui allocations
at that time.

Expand Down
8 changes: 4 additions & 4 deletions docs/Getting_Started/Cheat_Sheets/Git:_Reference_Sheet.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ developing, managing and distributing code.
Full Git documentation can be
found [here](https://git-scm.com/docs/git), or using `man git`.

![Git\_Diagram.svg](../includes/Git_Diagram.svg)
![Git\_Diagram.svg](../../includes/images/Git_Diagram.svg)

## Authentication

Expand Down Expand Up @@ -59,21 +59,21 @@ documentation](https://docs.github.com/en/authentication/connecting-to-github-wi
page, click your profile photo click **Settings**.

<span class="procedural-image-wrapper">![Settings icon in the user
bar](../includes/userbar-account-settings.png)</span>
bar](../../includes/images/userbar-account-settings.png)</span>

- In the "Access" section of the sidebar, click **SSH and GPG keys**.

- Click **New SSH key** or **Add SSH key**.

<span class="procedural-image-wrapper">![SSH Key
button](../includes/ssh-add-ssh-key-with-auth.png)</span>
button](../../includes/images/ssh-add-ssh-key-with-auth.png)</span>

- In the "Title" field, put "Mahuika" or "NeSI".

- Paste your key into the "Key" field.

<span class="procedural-image-wrapper">![The key
field](../includes/ssh-key-paste-with-type.png)</span>
field](../../includes/images/ssh-key-paste-with-type.png)</span>

- Click **Add SSH key**.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -374,12 +374,12 @@ Bring up the manual of a command (in this case ls).
> Pressing the 'tab' key once will automatically complete the line if it
> is the only option. e.g. 
>
> ![complete1.gif](../includes/complete1.gif)
> ![complete1.gif](../../includes/images/complete1.gif)
>
> If there are more than one possible completions, pressing tab again
> will show all those options.
>
> ![complete2.gif](../includes/complete2.gif)
> ![complete2.gif](../../includes/images/complete2.gif)
>
> Use of the tab key can help navigate the filesystem, spellcheck your
> commands and save you time typing.
4 changes: 2 additions & 2 deletions docs/Getting_Started/Getting_Help/System_status.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ use the link included at the bottom of the notification email message
See also our support article [Managing NeSI notification
preferences](https://support.nesi.org.nz/hc/en-gb/articles/4563294188687)

![mceclip0.png](../includes/mceclip0_0.png)
![mceclip0.png](../../includes/images/mceclip0_0.png)

## status.nesi.org.nz

Expand All @@ -42,4 +42,4 @@ NeSI does publish service incidents and scheduled maintenance via
Interested parties are invited to subscribe to updates (via SMS or
email).

<img src="../includes/mceclip0_1.png" width="880" height="772" />
<img src="../../includes/images/mceclip0_1.png" width="880" height="772" />
6 changes: 3 additions & 3 deletions docs/Getting_Started/Next_Steps/Finding_Job_Efficiency.md
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,7 @@ parent process).

Processes in green can be ignored

<img src="../includes/how_to_read_htop.png" alt="how_to_read_htop.png" width="929" height="252" />
<img src="../../includes/images/how_to_read_htop.png" alt="how_to_read_htop.png" width="929" height="252" />

**RES** - Current memory being used (same thing as 'RSS' from sacct)

Expand Down Expand Up @@ -191,12 +191,12 @@ more details.

## Example

![qdyn\_eff.png](../includes/qdyn_eff.png)
![qdyn\_eff.png](../../includes/images/qdyn_eff.png)

From the above plot of CPU efficiency, you might decide a 5% reduction
of CPU efficiency is acceptable and scale your job up to 18 CPU cores . 

![qdyn\_walltime.png](../includes/qdyn_walltime.png)
![qdyn\_walltime.png](../../includes/images/qdyn_walltime.png)

However, when looking at a plot of walltime it becomes apparent that
performance gains per CPU added drop significantly after 4 CPUs, and in
Expand Down
4 changes: 2 additions & 2 deletions docs/Getting_Started/Next_Steps/MPI_Scaling_Example.md
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,7 @@ our script with 2, 3, 4, 5 and 6 physical CPUs and plot the results:
6054939.ext+ extern 00:06:57 00:00.006 12 44K COMPLETED
6054939.0 python 00:06:51 01:18:37 6 174028K COMPLETED

![MPIscalingMem.png](../includes/MPIscalingMem_0.png)
![MPIscalingMem.png](../../includes/images/MPIscalingMem_0.png)

First, looking at the plot of memory usage per task vs CPUs it would at
appears that memory usage per task remains constant, regardless of how
Expand Down Expand Up @@ -284,7 +284,7 @@ increasing as we add more seeds, but the maximum memory per CPU doesn't
seem to change much. Let's try plotting this data to help us better
understand what is happening:

![MPIseedsvtime.png](../includes/MPIseedsvtime_0.png)
![MPIseedsvtime.png](../../includes/images/MPIseedsvtime_0.png)

This confirms our assumption of wall-time scaling linearly with number
of iterations. Since our 5,000 seed job to 7 minutes and 41 seconds we
Expand Down
Loading

0 comments on commit 142debd

Please sign in to comment.