Merge pull request #135 from NYU-RTS/more_ood

Added pages to OOD

Author: s-sajid-ali

docs/hpc/02_connecting_to_hpc/01_connecting_to_hpc.mdx

@@ -6,7 +6,7 @@ import TabItem from '@theme/TabItem';
 There are several ways to interact with the Greene HPC cluster. Similar to other Linux clusters, the most common method of connection is via a Command Line Interface (CLI). A CLI is a program that allows you to create and delete files, run programs, and navigate through directories and files via a typed prompt. On Mac, the built-in CLI application is called Terminal. While Windows 11 machines support a Linux Subsystem, which allows for similar functionality, a popular tool used to connect to a Linux server is a free application called [PuTTY][putty link]. 
 
 :::tip
-If you'd prefer to access the HPC cluster via a web gateway instead, please refer to [the section describing OpenOnDemand](./03_ood.md).
+If you'd prefer to access the HPC cluster via a web gateway instead, please refer to [the section describing Open OnDemand (OOD)](../09_ood/01_ood_intro.md).
 :::
 
 The following sections will outline basic ways to connect to the Greene cluster. Access to the clusters is primarily handled via the Secure Shell Protocol, or ssh. Below we outline ways to connect via ssh on Mac, Linux, and Windows machines.

docs/hpc/02_connecting_to_hpc/03_ood.md

@@ -57,7 +57,7 @@ The Globus endpoint for Greene is available at `nyu#greene`. The endpoint `nyu#p
 rclone - rsync for cloud storage, is a command line program to sync files and directories to and from cloud storage systems such as Google Drive, Amazon Drive, S3, B2 etc. rclone is available on DTNs. [Please see the documentation for how to use it.](https://rclone.org/)
 
 ### Open OnDemand (OOD)
-One can use [Open OnDemand (OOD)](../09_ood/01_open_on_demand.md) interface to upload data.
+One can use [Open OnDemand (OOD)](../09_ood/01_ood_intro.md) interface to upload data.
 :::warning
 Please only use OOD for small data transfers!  Please use Data-Transfer Nodes (DTNs) for moving large data.
 :::

docs/hpc/03_storage/03_data_transfers.md

@@ -0,0 +1,66 @@
+# Introduction to Open OnDemand (OOD)
+
+## A Web-based Graphical User Interface
+
+OOD is designed to create easier access for users to interface with HPC systems. It comes with a variety of convenient tools to manage files, access the command line, manage and monitor jobs, and launch interactive applications, such as Jupyter Notebooks, RStudio sessions, and even full Linux Desktops. Features Include:
+
+-   Easy file management - upload and download files, view HTML and pictures without downloading
+-   Command-line shell access without any SSH client locally installed
+-   Job management and monitoring
+-   Full Linux desktop experience without X11
+-   Interactive Apps such as JupyterHub and RStudio without the need for port forwarding
+
+It is accessible to all users with a valid NYU HPC account while on-campus network or through NYU VPN via https://ood.hpc.nyu.edu.
+
+## Access the Shell
+
+Under the **clusters** menu you can select the **Greene Shell Access** option to access the Linux shell. No local SSH client is required.
+
+![img](./static/open_ondemand_gif.gif)
+
+Please see our documentation on [Submitting Jobs](http://localhost:3000/rts-docs-dev/docs/hpc/submitting_jobs/slurm_submitting_jobs/) if you'd like detailed instructions.
+
+**Interactive Applications**
+
+GUI based applications are accessible without the need for port or X11 forwarding. Select the Interactive Apps menu, select the desired application, and submit the job based on required resources and options.
+
+![interactive-applications](./static/interactive-applications.png)
+
+## Troubleshooting
+
+### Connections to Open OnDemand
+
+A common issue that can occur is receiving an error that the Open OnDemand page cannot be reached. Sometimes this can indicate that the service is down, but often this is an issue with the the local browser cache. You can test this by opening a private browser window and seeing if https://ood.hpc.nyu.edu will load. If it does, try deleting the cache for https://ood.hpc.nyu.edu in your browser history to resolve this issue.
+
+In Chrome, this can be done by navigating to this page in your settings:
+
+
+```bash
+chrome://settings/content/all?searchSubpage=ood.hpc.nyu.edu&search
+```
+
+The link above will automatically search for the Open OnDemand site data and cookies. You can then simply click on the trashcan icon to delete the site cache.
+
+![Remove browser cache image](./static/ood_remove_cache.png)
+
+Once done, try navigating again to https://ood.hpc.nyu.edu and the site should load. For other issues please email hpc@nyu.edu.
+
+### Viewing Logs
+
+When problems arise one of the best ways to figure out what went wrong is to look in the logs.  You can do this in two ways for OOD: from within OOD itself or via the terminal.
+
+#### From within OOD
+
+You can view session logs for interactive apps from within OOD if they are still visible within `Home / My Interactive Sessions`.  
+
+![OOD Matlab in queue](./static/ood_matlab_in_queue.png)
+
+Just click on the `Session ID` link and a tab will open with the contents of the directory that contains your session logs.
+
+![OOD session ID directory](./static/ood_session_dir.png)
+
+`output.log` is probably the most informative for debugging.  Simply click on `output.log` and the contents will be displayed.
+
+#### From terminal
+
+If your session is no longer visible from within OOD you may still be able to find your logs via the terminal.  Simply [log into Greene](https://sites.google.com/nyu.edu/nyu-hpc/accessing-hpc) and `cd` to `/home/$USER/ondemand/data/sys/dashboard/batch_connect/sys/` and then `cd` into the directory for the app that you're interested in.  You should find the file `output.log` there.

docs/hpc/09_ood/01_ood_intro.md

@@ -1,21 +1,15 @@
-# Open OnDemand (OOD) with Conda/Singularity
-
-[Open OnDemand](https://ood.hpc.nyu.edu/) is a tool that allows users to launch Graphical User Interfaces (GUIs) based applications are accessible without modifying your HPC environment. You can log into the Open OnDemand interface at [https://ood.hpc.nyu.edu](https://ood.hpc.nyu.edu). Once logged in, select the **Interactive Apps** menu, select the desired application, and submit the job based on required resources and options. 
-
-:::note
-If you're not on the NYU network, you may need to connect via NYU's Virtual Private Network (VPN) before connecting to [https://ood.hpc.nyu.edu](https://ood.hpc.nyu.edu).  [Read the details about NYU's VPN and instructions for connecting](https://www.nyu.edu/life/information-technology/infrastructure/network-services/vpn.html).
-:::
+# Jupyter with Conda/Singularity in OOD
 
 ## OOD + Singularity + conda
-This page describes how to use your Singularity with conda environment in Open OnDemand (OOD) GUI at Greene. 
+This page describes how to use your Singularity with conda environment in OOD GUI at Greene. 
 
 ### Log Into Greene via the Terminal
-The following commands must be run from the terminal. Information on accessing via the terminal can be found at [Connecting to the HPC](../02_connecting_to_hpc/01_connecting_to_hpc.mdx). 
+The following commands must be run from the terminal. Information on accessing via the terminal can be found at [Connecting to the HPC](../02_connecting_to_hpc/01_connecting_to_hpc.mdx) or you can log into OOD and select the `Clusters` tab at the top of the page and select `Greene Shell Access`.
 
 ### Preinstallation Warning
 :::warning
 If you have initialized Conda in your base environment, your prompt on Greene may show something like:
-```sh
+```bash
 (base) [NetID@log-1 ~]$
 ```
 then you must first comment out or remove this portion of your `~/.bashrc` file:
@@ -59,7 +53,7 @@ Launching Singularity with the `--overlay` flag mounts the overlay file to a new
 
 :::warning
 Be sure that you have the Singularity prompt (`Singularity>`) and that `/ext3` is available before the next step.
-```sh
+```bash
 Singularity> ls -lah /ext3
 total 8.5K
 drwxrwxr-x.  2 root root 4.0K Oct 19 10:01 .
@@ -196,10 +190,11 @@ to
 ```
 Update the `"<Your NetID>"` to your own NetID without the `"<>"` symbols.
 
-### Launch an Open OnDemand Jupyter Notebook
-[https://ood.hpc.nyu.edu](https://ood.hpc.nyu.edu)
+### Launch an OOD Jupyter Notebook
+Go to [https://ood.hpc.nyu.edu](https://ood.hpc.nyu.edu) <br />
+and select Jupyter Notebook from the Interactive Apps pull-down menu:
 
-![OOD Launch](./static/OOD_launch.png)
+![OOD Launch](./static/start_jupyter.png)
 
 ### Configure and Launch your Notebook
 

docs/hpc/09_ood/01_open_on_demand.md …od/02_jupyter_with_conda_singularity.mdxdocs/hpc/09_ood/01_open_on_demand.md renamed to docs/hpc/09_ood/02_jupyter_with_conda_singularity.mdx docs/hpc/09_ood/01_open_on_demand.md renamed to docs/hpc/09_ood/02_jupyter_with_conda_singularity.mdx

@@ -0,0 +1,59 @@
+# Matlab in OOD
+
+## Getting Started
+You can run Matlab in OOD by going to the URL [ood.hpc.nyu.edu](http://ood.hpc.nyu.edu) in your browser and selecting `MATLAB` from the `Interactive Apps` pull-down menu at the top of the page.  As you can see below, once you've used it and other interactive apps they'll show up on your home screen under the `Recently Used Apps` header.
+
+:::note
+Be aware that when you start from `Recently Used Apps` it will start with the same configuration that you used previously.  If you'd like to configure your Matlab session differently, you'll need to select it from the menu.
+:::
+
+![OOD Interactive Apps menu](./static/ood_interactive_apps_matlab.png)
+
+## Configuration
+
+You can select the version of Matlab to use, the number or cores, amount of memory, GPU type (if any), amount of time, and optional Slurm options.
+
+![OOD Matlab Configuration](./static/ood_matlab_config.png)
+
+::::tip
+Please be aware that when you run each version of Matlab for the first time, the working directory will default to your home directory `/home/NetID`.  
+:::warning
+Be careful not to go over your quota for `/home`.  You can find your current usage with the `myquota` command.  Please see our [Storage documentation](../03_storage/01_intro_and_data_management.mdx) for details about your storage options.
+:::
+Each time you start it up after that it will set its working directory to the last directory you were in.  If you'd like to change this behavior, so that you always start at the same directory:
+1.   click on `ENVIRONMENT` in your `Home` panel
+1.   select `Preferences`
+1.   select `Current Folder`
+1.   select `initial working folder preferences`
+
+This will allow you to set the full path of the initial working folder you would prefer.  Be sure to hit the `Apply` button, so your changes are applied.
+::::
+
+:::tip
+Another step you may wish to take to keep Matlab from filling up your `/home` directory is to create symlinks from `/scratch` to `/home` for internal Matlab folders.  You can do this by running the following commands:
+```bash
+# make sure you're in your /home directory
+cd
+mkdir /scratch/$USER/.matlab
+mkdir /scratch/$USER/.MathWorks
+ln -s /scratch/$USER/.matlab
+ln -s /scratch/$USER/.MathWorks
+``` 
+:::
+
+## Matlab running in OOD
+
+After you hit the `Launch` button you'll have to wait for the scheduler to find you node(s) to run on:
+![OOD Matlab in queue](./static/ood_matlab_in_queue.png)
+
+Then you'll have a short wait for Matlab itself to start up.<br />
+Once that happens you'll get one last form that will allow you to:
+-   make changes to compression and image qualtiy
+-   open a terminal window on the compute node your Matlab session is running on
+-   get a link that you can share that will allow others to view your Matlab session
+
+![Pre-launch matlab OOD](./static/ood_matlab_prelaunch.png)
+
+Then after you hit the `Launch Matlab` button you'll have the familiar Matlab Desktop to use.
+
+![OOD Matlab Running](./static/ood_matlab_running.png)

docs/hpc/09_ood/03_matlab.mdx

@@ -0,0 +1,36 @@
+# RStudio in OOD
+
+
+## Getting Started
+You can run RStudio in OOD by going to the URL [ood.hpc.nyu.edu](http://ood.hpc.nyu.edu) in your browser and selecting `RStudio Server` from the `Interactive Apps` pull-down menu at the top of the page.  As you can see below, once you've used it and other interactive apps they'll show up on your home screen under the `Recently Used Apps` header.
+
+:::note
+Be aware that when you start from `Recently Used Apps` it will start with the same configuration that you used previously.  If you'd like to configure your RStudio session differently, you'll need to select it from the menu.
+:::
+
+![OOD Interactive Apps menu](./static/ood_interactive_apps_rstudio.png)
+
+## Configuration
+
+You can select the version of R to use, the R package location, the number or cores, amount of memory, GPU type (if any), amount of time, and optional Slurm options.
+
+![OOD RStudio Configuration](./static/ood_rstudio_config.png)
+
+::::tip
+You may wish to set the 'R Package Location' to `/scratch` since space in `/home` is limited.
+Also, be aware that RStudio always starts in your `/home/NetID` directory.
+:::warning
+Be careful not to go over your quota for `/home`.  You can find your current usage with the `myquota` command.  Please see our [Storage documentation](../03_storage/01_intro_and_data_management.mdx) for details about your storage options.
+:::
+::::
+
+## RStudio running in OOD
+
+After you hit the `Launch` button you'll have to wait for the scheduler to find you node(s) to run on:
+![OOD RStudio in queue](./static/ood_rstudio_in_queue.png)
+
+Then you'll have a short wait for RStudio itself to start up.<br />
+
+Then after you hit the `Connect to RStudio Server` button you'll have the familiar RStudio Desktop to use.
+
+![OOD RStudio Running](./static/ood_rstudio_running.png)

docs/hpc/09_ood/04_RStudio.mdx

@@ -0,0 +1,24 @@
+# FileZilla
+
+As mentioned in [File Transfer Clients](../03_storage/03_data_transfers.md#file-transfer-clients), FileZilla is a client for transferring files.  It has been installed on OOD as a convenience, but be aware that you can use many of the other methods described in the [Data Transfers](../03_storage/03_data_transfers.md) section of our documentation for transferring your files on OOD, since they all use the same underlying filesystems.
+
+## FileZilla running in OOD
+
+First, you'll be able to select the number of cores, amount of memory, job duration and optional slurm options, you'd like.
+![OOD FileZilla initial options](./static/ood_filezilla_init_options.png)
+
+After you hit the `Launch` button you'll have to wait for the scheduler to find you node(s) to run on, and then you'll have a short wait for FileZilla itself to start up.<br />
+
+After all that, you'll get the chance to set some options for your environment:
+-   Compression
+-   Image Quality
+
+:::tip
+You can also get a 'view only' link that you can share with others and a link to your session files for debugging
+:::
+
+![OOD FileZilla in queue](./static/ood_filezilla_startup.png)
+
+After you hit the `Launch FileZilla` button you'll have the familiar FileZilla Desktop to use.
+
+![OOD FileZilla Running](./static/ood_filezilla_running.png)