Merge pull request #47 from NYU-ITS/srde_rework

SRDE docs

Author: s-sajid-ali

docs/srde/01_getting_started/01_intro.md

@@ -0,0 +1,5 @@
+# Start here!
+
+Welcome to the Secure Research Data Environment documentation! If you do not have an active project, please proceed to the next section that explains the eligibility criteria and how you may request one.
+
+If you are an active user, you can proceed to one of the categories on the left.

docs/srde/01-intro.md …tting_started/02_eligibility_accounts.mddocs/srde/01-intro.md renamed to docs/srde/01_getting_started/02_eligibility_accounts.md docs/srde/01-intro.md renamed to docs/srde/01_getting_started/02_eligibility_accounts.md

@@ -1,10 +1,6 @@
----
-sidebar_position: 1
----
 
 # Secure Research Data Environment (SRDE)
 
-
 The Secure Research Data Environment (SRDE) is a centralized secure computing platform designed to support research projects that require storage and computational resources.  It provides a space for researchers to design and build secure, scalable, and resilient environments to store, share, and analyze moderate and high-risk data, as per the [NYU Electronic Data and System Risk Classification Policy](https://www.nyu.edu/about/policies-guidelines-compliance/policies-and-guidelines/electronic-data-and-system-risk-classification.html).
 
 The Research Technology Services team leverages cloud computing resources to provide flexible, scalable, remotely accessible secure virtual environments. The team provides researchers with consultations and resources to comply with security requirements of research grants and Data Use Agreements (DUAs). SRDE resources intend to meet the security controls outlined in the NIST SP 800-171 to safeguard Controlled Unclassified Information (CUI).
@@ -26,11 +22,3 @@ The SRDE form contains details about the project, such as if the project require
 [Link to the Secure Research Data Environment Intake Form](https://nyu.qualtrics.com/jfe/form/SV_3Vok9ax87Bxxdsy).
 
 After you submit the intake form, the SRDE team will review the submitted documents and will respond to schedule the consultation.
-
-## SRDE User Guide
-
-Coming soon!
-
-## Support
-
-Please email your questions to: [email protected]

docs/srde/01_getting_started/_category_.json

@@ -0,0 +1,3 @@
+{
+  "label": "Getting Started",
+}

docs/srde/02_user_guide/01_connecting.mdx

@@ -0,0 +1,335 @@
+# Connecting to SRDE
+
+Remote access to the secure environment workspace via the Command Line Interface. The SRDE consists of two separate servers: 
+-   The Workspace Host is where you access and analyze data
+-   The Bastion Host acts as a proxy that allows your laptop/workstation to connect securely via the internet to the Workspace Host.
+
+Accessing the Secure environment Workspace Host remotely via the Command Line Interface (CLI) is a two-step process: First you must connect to the Bastion Host and then from the Bastion Host access your Workspace Host.  The two-step process is demonstrated below:
+![Virtual Private Network](./static/vpc_basics.png)
+
+This two-step process is enabled with the use of SSH keys and SSH Agent Forwarding and is described in detail for some of the common Operating Systems (MacOS/Linux and Windows)  in the following sections of the User Guide. For more general information about using SSH keys and the use of Bastion Host [see here](https://medium.com/devops-dudes/setting-up-an-ssh-agent-to-access-the-bastion-in-vpc-532918577949).
+
+User access to the secure environment is controlled by [Identity-Aware Proxy (IAP)](https://cloud.google.com/security/products/iap?hl=en) Google Cloud service. IAP provides a central way of managing user access and enforcing access control policies, without requiring external/public IP addresses on the Bastion Host and the Workspace Host.
+
+::::tip[Prerequisites]
+In order to be able to access your Secure Environment Workspace Host, you will need the following information, provided by the Secure Research Data Admins:
+-   **Project Id** for the Bastion Host (ex. test-dev1-bastion-1234)    
+-   **Zone Name** (ex. us-east4-a)
+
+:::note
+At this time you are not required to be on the NYU Network (or VPN into the NYU Network) in order to access the Secure Environment workspace.
+
+:::
+
+::::
+
+
+## Connecting through Google Cloud Console
+Navigate to Google Cloud Console https://console.cloud.google.com/welcome and login with your NetID. Click the Select a project drop-down list at the top left corner of the page. In the Select a project window that appears, search and select the bastion project using the provided project ID (ex. test-dev1-bastion-1234).
+
+![Select a project](./static/select_project.png)
+
+Once selected, navigate to the VM Instances page via the Navigation menu (Menu in the top left corner of the page ) > Compute Engine > VM Instances. A running Bastion instance will be visible in the page as shown below:
+
+![Bastion Instance](./static/bastion_instance.png)
+
+ssh to the Bastion instance by clicking on the SSH button, a new SSH-in-browser tab will appear with a restricted CLI ( Command line interface ) connected to the instance. We are now inside the Bastion Host. 
+
+![SSH in browser](./static/ssh_in_browser.png)
+
+Now we can ssh to our workspace host by using the workspace internal IP address `10.0.0.2`: 
+```sh
+ssh 10.0.0.2
+```
+This will open the workspace CLI, with access to the workspace host having the computing needs to work on our data.
+
+## Connecting through Google Cloud Shell
+
+Navigate to https://shell.cloud.google.com/ while logged in using your NetID.
+
+### Setting project and zone
+
+Note - Ask your SRDE administrator for the appropriate GCP PROJECT_ID and ZONE_NAME. Replace the values in the two commands below and run them
+
+
+```sh
+gcloud config set project PROJECT_ID
+gcloud config set compute/zone ZONE_NAME
+```
+
+### Confirm settings
+
+Before proceeding, confirm that the project and zone match your GCP project ID and zone:
+
+
+```sh
+gcloud config list
+
+
+[compute]
+region = us-east4
+zone = us-east4-a
+[core]
+account = [email protected]
+disable_usage_reporting = False
+project = test-dev1-bastion-1234
+
+
+Your active configuration is: [default]
+```
+
+
+### Generate SSH keys
+
+:::tip[Unused keys expire!]
+Google Cloud Shell will delete your files, including generated SSH keys, if they are not accessed for 120 days. If this happens you will need to generate them again.
+
+:::
+
+The simplest way to generate SSH keys is to delegate the key generation to gcloud. In order to trigger key creation, run the following command. 
+
+:::note
+Ignore the result of this command. It will most likely print errors to the output console.
+
+:::
+
+```sh
+gcloud compute ssh bastion-vm
+```
+
+You will be prompted to enter an SSH passphrase. This is optional, however it is recommended for additional user security.
+![Getting into Bastion](./static/getting_into_bastion.png)
+
+The above command should log you into the bastion VM. You will see a prompt like:
+```sh
+-bash-4.4$”
+```
+Before proceeding, exit back to your local machine
+
+```sh
+exit
+```
+
+Then make sure the above step created two keys in your ssh home directory (`~/.ssh`) as shown below:
+
+```sh
+ls ~/.ssh
+```
+
+![List ssh keys](./static/ls_dot_ssh.png)
+
+Start the ssh-agent on your local machine
+
+```sh
+eval `ssh-agent -s`
+```
+Add the google_compute_engine key to your ssh session
+
+```sh
+ssh-add ~/.ssh/google_compute_engine
+```
+Connect to the instance with gcloud using the –ssh-flag-”-A” flag
+:::note
+This command uses the default project and zone set above.
+
+:::
+
+```sh
+gcloud compute ssh bastion-vm --ssh-flag="-A"  --tunnel-through-iap
+```
+
+### Add SSH key to session
+Run the following command to add the google_compute_engine key to the current session:ssh
+```sh
+ssh-add -L
+```
+Connect to the workstation-vm
+
+```sh
+ssh 10.0.0.2
+```
+
+### Future logins
+After the initial login, you will not need to regenerate the SSH keys, but you will need the rest of the command sequence from “Start the SSH agent”. On your local machine:
+```sh
+eval `ssh-agent -s`
+ssh-add ~/.ssh/google_compute_engine 
+gcloud compute ssh bastion-vm --ssh-flag="-A" --tunnel-through-iap --project=PROJECT_ID
+```
+
+And then on the bastion VM:
+
+```sh
+ssh 10.0.0.2
+```
+
+
+## Connecting on MacOS/Linux
+
+### Install gcloud CLI
+Follow the [official guidelines](https://cloud.google.com/sdk/docs/install-sdk#installing_the_latest_version) to install the latest version of gcloud CLI locally on your computer.
+
+:::note
+After completing the gcloud installation, verify that the `gcloud` binary is in your $PATH environment variable.
+
+:::
+
+### Configure local gcloud settings
+Run the following command. It generates a link as shown below
+
+```sh
+gcloud auth login --no-launch-browser
+```
+
+![GCP authenticate login](./static/auth_login.png)
+
+Copy the link and open your chrome browser in incognito mode to perform user sign in.Username is your NYU NetID email address. For e.g. [email protected]
+
+
+You will be redirected to the NYU SSO page and MFA verification through Duo Push. After successfully logging in, you will be asked to allow google SDK to access your account as shown below
+
+![OAuth consent screen](./static/oauth_consent.png)
+
+Pressing the “Allow” button on this page will present the authorization code. Copy the code and paste it in the terminal. If this step is successful, you should see this text printed to the console. **You are now logged in as [[email protected]].**
+
+### Connect to the workspace
+Follow the same instructions for connecting with Google Cloud Shell above, starting from section on setting project and zone above.
+
+## Connecting on Windows 10/11
+
+### Start and Configure SSH-Agent Service 
+Using an elevated PowerShell window (run as admin), execute the following command to install the SSH-Agent service and configure it to start automatically when you log into your machine:
+```ps
+Get-Service ssh-agent | Set-Service -StartupType Automatic -PassThru | Start-Service
+
+```
+![Run a script in PowerShell](./static/powershell_script.png)
+
+### Install gcloud CLI
+Download the [Google Cloud CLI installer] (https://dl.google.com/dl/cloudsdk/channels/rapid/GoogleCloudSDKInstaller.exe) and run the installer
+
+![GCloud Installer](./static/gcloud_windows_installer.png)
+
+Alternatively, run the following command to download and install:
+```ps
+(New-Object Net.WebClient).DownloadFile("https://dl.google.com/dl/cloudsdk/channels/rapid/GoogleCloudSDKInstaller.exe", "$env:Temp\GoogleCloudSDKInstaller.exe")
+
+& $env:Temp\GoogleCloudSDKInstaller.exe
+
+```
+
+### Install Git
+Download the Git Bash setup from the official website: https://git-scm.com/ and run the installer
+
+
+### Install Putty
+Download and install Putty from this link https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html
+
+Post installation verify that the Putty authentication agent is installed and available
+
+For 64-bit installer, you will find this executable at `C:/Program Files/PuTTY/pageant.exe`
+
+
+### Install Python (>version 3.0)
+Install Python from the official website:https://www.python.org/downloads/
+
+Remember to check “Add python to the environment path.” ***add screenshot
+
+Make sure it's installed and available on PATH. On many systems Python comes pre-installed, you can try running the python command to start the Python interpreter to check and see if it is already installed.
+
+![Python in CMD Prompt](./static/cmd_prompt_python.png)
+
+On windows you can also try the py command which is a launcher which is more likely to work. If it is installed you will see a response which will include the version number, for example:
+
+
+![Py in CMD Prompt](./static/cmd_prompt_py.png)
+
+### Logging in:
+Authenticate gcloud by starting a new session of command line or powershell. initialize and login to gcloud with your account (you will be redirected to the browser for authentication)
+```ps
+gcloud auth login
+```
+
+![OAuth on windows](./static/oauth_consent_windows.png)
+
+![GCP login CMD prompt](./static/cmd_prompt_gcp_login.png)
+
+Run Git Bash and start the ssh-agent on your local machine
+
+```ps
+eval `ssh-agent -s`
+```
+![SSH Agent command](./static/mingw_eval_ssh.png)
+
+Add the SSH key to agent by running
+
+```ps
+pageant.exe
+```
+![PAgent Windows](./static/pagent_windows.png)
+
+The app runs in the background. you can find it in the tray.
+![PAgent in tray](./static/pagent_in_tray.png)
+
+Right click the icon and select "Add Key". Add the google_compute_engine key with the PPK extension (~/.ssh/google_compute_engine) to your agent:
+![PPK Add Key](./static/ppk_add_key.png)
+
+
+:::Skip this step in the future
+Go to the Pageant shortcut icon from the Windows Start Menu or your desktop.
+
+Right click on the icon, and click on Properties. (If Properties is not an option on the menu, click on Open file location, then right click on the Pageant icon, and click on Properties)
+
+:::
+
+![PPK properties](./static/ppk_properties_trust.png)
+
+From the Shortcut tab, edit the Target field. Leave the path to pageant.exe intact. After that path, add the path to your Google .ppk key file.
+
+:::warning[Critical]
+The key path should be outside the quotation marks. i
+
+:::
+
+Here’s an example:
+```ps
+"C:\Program Files\PuTTY\pageant.exe" C:\Users\Sam\.ssh\google_compute_engine.ppk
+```
+
+![PAgent properties](./static/pagent_properties_popout.png)
+
+### SSH into the bastion VM from Git Bash
+:::tip
+Ask your SRDE administrator for the appropriate GCP project ID.
+
+:::
+
+Replace gcp-project-id with that information in the below command:
+```ps
+export PROJECT_ID=gcp-project-id;
+
+gcloud compute ssh bastion-vm --ssh-flag="-A" --zone=us-east4-a --tunnel-through-iap --project=${PROJECT_ID}
+
+```
+![Export Project ID and login](./static/mingw_export_project_gcloud_login.png)
+
+When SSHing to bastion in the git bash window, a new terminal in putty appears with the bastion connection
+![PUTTy bastion](./static/putty_bastion.png)
+
+A PuTTY security alert window may pop up to accept the host key, click on Accept
+![PUTTy security alert](./static/putty_security_alert.png)
+
+### Add SSH key to session
+
+Run ssh-add to add the google_compute_engine key to the current session
+```ps
+ssh-add  -L 
+```
+
+Connect to the workstation-vm
+
+```ps
+ssh 10.0.0.2
+```
+
+![PUTTy ssh to vm](./static/putty_ssh_to_vm.png)