Merge pull request #124 from NYU-RTS/cleanup

reorganize

Author: s-sajid-ali

docs/hpc/01_getting_started/03_walkthrough_request_hpc_account.mdx

@@ -3,7 +3,7 @@ import TabItem from '@theme/TabItem';
 
 # How to request an HPC account
 
-First, check if you already have an account. You can check this by attempting to log in to the cluster, according to the instructions at [Connecting to the HPC Cluster](../02_connecting_to_hpc/01_connecting_to_hpc.md)
+First, check if you already have an account. You can check this by attempting to log in to the cluster, according to the instructions at [Connecting to the HPC Cluster](../02_connecting_to_hpc/01_connecting_to_hpc.mdx)
 
 :::info VPN Needed
 You need to be on the NYU VPN to perform this task!

docs/hpc/01_getting_started/04_walkthrough_approve_hpc_account_request.md

@@ -1,10 +1,12 @@
 # How to approve an HPC Account Request
 
+When someone nominates you as their HPC sponsor, you should be notified by email.
+
 :::info VPN Needed
 You need to be on the NYU VPN to perform this task!
 :::
 
-When someone nominates you as their HPC sponsor, you should be notified by email. You can also [log into IIQ at any time](https://iiq.nyu.edu/identityiq), and if you have a request awaiting your approval, it will appear in your "Actions Items" box, as per the following screenshot:
+You can also [log into IIQ at any time](https://iiq.nyu.edu/identityiq), and if you have a request awaiting your approval, it will appear in your "Actions Items" box, as per the following screenshot:
 !["Actions Items" box](./static/work_item.png)
 
 Another way to get to pending approvals is to click on the line item in the “Latest Approvals” section which will lead directly to the approval page. For new HPC Account Requests, the page will look like this:

docs/hpc/01_getting_started/05_walkthrough_renew_hpc_account.md

@@ -1,11 +1,11 @@
 # Renewing your HPC Account
 
-Login to the URL given below, using your NetID/password, to create or manage HPC Account Requests:
-
 :::info VPN Needed
 You need to be on the NYU VPN to perform this task!
 :::
 
+Login to the URL given below, using your NetID/password, to create or manage HPC Account Requests:
+
 https://identity.it.nyu.edu/
 
 Upon logging in, an end user’s landing page will look like this

…onnecting_to_hpc/01_connecting_to_hpc.md …nnecting_to_hpc/01_connecting_to_hpc.mdxdocs/hpc/02_connecting_to_hpc/01_connecting_to_hpc.md renamed to docs/hpc/02_connecting_to_hpc/01_connecting_to_hpc.mdx docs/hpc/02_connecting_to_hpc/01_connecting_to_hpc.md renamed to docs/hpc/02_connecting_to_hpc/01_connecting_to_hpc.mdx

@@ -1,22 +1,32 @@
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
 # Connecting to the HPC Cluster
 
 There are several ways to interact with the Greene HPC cluster. Similar to other Linux clusters, the most common method of connection is via a Command Line Interface (CLI). A CLI is a program that allows you to create and delete files, run programs, and navigate through directories and files via a typed prompt. On Mac, the built-in CLI application is called Terminal. While Windows 11 machines support a Linux Subsystem, which allows for similar functionality, a popular tool used to connect to a Linux server is a free application called [PuTTY][putty link]. 
 
+:::tip
 If you'd prefer to access the HPC cluster via a web gateway instead, please refer to [the section describing OpenOnDemand](./03_ood.md).
+:::
 
 The following sections will outline basic ways to connect to the Greene cluster. Access to the clusters is primarily handled via the Secure Shell Protocol, or ssh. Below we outline ways to connect via ssh on Mac, Linux, and Windows machines.
 
-## Remote Connections with the NYU VPN & HPC Gateway Server
+:::info Connecting to Torch from outside NYU Network
 
-If you are connecting from a remote location that is not on the NYU network (your home for example), you have two options:
 
-1.  **VPN Option:** [Set up your computer to use the NYU VPN][nyu vpn link]. Once you've created a VPN connection, you can proceed as if you were connected to the NYU net
-
-2.  **Gateway Option:** Go through our gateway servers (example below). Gateways are designed to support only a very minimal set of commands and their only purpose is to let users connect HPC systems without needing to first connect to the VPN
+<Tabs>
+  <TabItem value="VPN" label="VPN (Preferred)">
+    [Set up your computer to use the NYU VPN][nyu vpn link]. Once you've created a VPN connection, you can proceed as if you were connected to the NYU net
+  </TabItem>
+  <TabItem value="Gateway" label="Gateway">
+    Go through our gateway servers (example below). Gateways are designed to support only a very minimal set of commands and their only purpose is to let users connect HPC systems without needing to first connect to the VPN
+  </TabItem>
+</Tabs>
+:::
 
 You do not need to use the NYU VPN or gateways if you are connected to the NYU network (wired connection in your office or WiFi) or if you have VPN connection initiated. In this case you can ssh directly to the clusters.
 
-## Command Line Interface (Use Terminal)
+## Command Line Interface (with a Terminal)
 
 ### Mac & Linux Access
 
@@ -48,7 +58,7 @@ Instructions on WSL installation can be found here: [https://docs.microsoft.com/
 -   If you are using WSL 2 (Windows subsystem for Linux), you may not be able to access internet when Cisco AnyConnect VPN, installed from exe file, is activated. A potential solution: uninstall Cisco AnyConnect and install AnyConnect using Microsoft Store, and then setup new VPN connection using settings described on [IT webpage][install vpn on windows link].
 :::
 
-### Setting up SSH Keys
+## Setting up SSH Keys
 
 Instead of typing your password every time you need to log in, you can also specify an ssh key.
 

docs/hpc/03_storage/06_best_practices.md

@@ -48,5 +48,5 @@ Your home directory has a relatively small number of inodes.
 If you create a conda or python environment in you home directory, this can eat up all the inodes. 
 :::
 
-Please review the [Package Management section](../06_tools_and_software/05_software_on_greene.md#package-management-for-r-python--julia-and-conda-in-general) of the [Greene Software Page](../06_tools_and_software/05_software_on_greene.md).
+Please review the [Package Management section](../06_tools_and_software/01_software_on_greene.md#package-management-for-r-python--julia-and-conda-in-general) of the [Greene Software Page](../06_tools_and_software/01_software_on_greene.md).
 

docs/hpc/03_storage/07_large_number_of_small_files.md

@@ -45,7 +45,7 @@ This option may require a lot of RAM - thus you may have to wait a long time for
 Special containers, which allow to either load data fast fully or access chosen elements without loading the whole dataset into RAM.
 
 ### SQLite
-If you have structured data, a good option would be to use SQLite.  Please see [SQLite: Handling Large Structured Data](../06_tools_and_software/07_sqlite_handling_large_structured_data.md) for more information.
+If you have structured data, a good option would be to use SQLite.  Please see [SQLite: Handling Large Structured Data](../06_tools_and_software/05_sqlite_handling_large_structured_data.md) for more information.
 
 ### HDF5
 One can think about HDF5 file as a "container file" (database of a sort), which holds a lot of objects inside.

docs/hpc/03_storage/08_transferring_cloud_storage_data_with_rclone.md

@@ -29,7 +29,7 @@ There are specific instruction on the [rclone web site](https://rclone.org/drive
 
 ### Step 1: Login to Greene:
 
-Follow [instructions](../02_connecting_to_hpc/01_connecting_to_hpc.md) to log into the Greene HPC cluster.
+Follow [instructions](../02_connecting_to_hpc/01_connecting_to_hpc.mdx) to log into the Greene HPC cluster.
 
 ### Step 2: Load the rclone module
 ```sh

docs/hpc/05_submitting_jobs/01_slurm_submitting_jobs.md

@@ -6,7 +6,7 @@
 -   HPC workloads are usually better suited to *batch processing* than *interactive* working.
 -   A batch job is sent to the system when submitted with an **sbatch** command. 
 -   The working pattern we are all familiar with is *interactive* - where we type ( or click ) something interactively, and the computer performs the associated action. Then we type ( or click ) the next thing.
--   Comments at the start of the script, which match a special pattern ( #SBATCH ) are read as Slurm options.
+-   Comments at the start of the script, which match a special pattern ( `#SBATCH` ) are read as Slurm options.
 
 ### The trouble with interactive environments
 
@@ -15,8 +15,7 @@ There is a reason why GUIs are less common in HPC environments: **point-and-clic
 :::tip
 The job might not start immediately, and might take hours or days, so we prefer a *batch* approach:
 
--   Plan the sequence of commands which will perform the actions we need
-    -   Write them into a script.
+-   Plan the sequence of commands which will perform the actions we need and write the commands into a script.
 
 You can now run the script interactively, which is a great way to save effort if i frequently use the same workflow, or ...
 -   Submit the script to a batch system, to run on dedicated resources when they become available.

docs/hpc/06_tools_and_software/01_intro.md

@@ -7,7 +7,7 @@ There are different types of software packages available
     -   This includes the licensed software listed below
 -   Singularity Containers
     -   You can find those already built and ready to use, at location `/scratch/work/public/singularity/`
-    -   For more information on running software with Singularity, [click here](../06_tools_and_software/06_singularity_run_custom_applications_with_containers.md).
+    -   For more information on running software with Singularity, [click here](../07_containers/01_intro.md).
 -   Python/R/Julia packages can be installed by a user 
 
 If you need another linux program installed, please contact us at [hpc@nyu.edu](mailto:hpc@nyu.edu)