diff --git a/docs/guides/connect-to-airflow.md b/docs/guides/connect-to-airflow.md new file mode 100644 index 0000000..b8a4ed7 --- /dev/null +++ b/docs/guides/connect-to-airflow.md @@ -0,0 +1,27 @@ +# How to connect to the hosted Airflow instance + +This guide documents how to connect to the AWS hosted Airflow instance + +## First Time Setup + +For first time setup, see the instructions [here](./connect-to-hosted-infrastructure.md). +If you already completed this setup to connect to some other infrastructure in +the INCLUDE AWS account, such as the warehouse, you do not need to complete +the first time setup again. + +## Connecting to Airflow + +1. run the command `saml2aws login`. When using this tool, you will want to create a +profile for the AWS account `373997854230`. +2. Navigate to the `ssm_forward_to_mwaa` directory within the `devops_utils` +repository you cloned on to your machine during first time setup +3. Run the script `./mwaa_connect.sh include-mwaa-prd-airflow prd 443 us-east-1` +This script should should that you can now connect to airflow +4. Open your web browser and navigate to google. Click on the `google apps` menu +to the left of your user profile picture at the right of the page and then +scroll down to and click on `Amazon Web Services`. You will then be presented +with a page to choose which aws account and role you would like to log into. +Choose the same one you chose in step 1 above. +5. In your web browser, navigate to [https://localhost](https://localhost). This +should display an airflow login page. Login and then the page should now show +airflow's main page. diff --git a/docs/guides/connect-to-hosted-infrastructure.md b/docs/guides/connect-to-hosted-infrastructure.md new file mode 100644 index 0000000..b7e3f1e --- /dev/null +++ b/docs/guides/connect-to-hosted-infrastructure.md @@ -0,0 +1,54 @@ +# How to connect to hosted infrastructure + +The tools used to run and deploy dbt models are hosted in AWS and require setup +to use. This guide describes the steps needed to connect to hosted +infrastructure the first time, as well as what is needed to authenticate after +setup. + +## First Time Setup + +### Pre-requisites: Security, Complianze, and Permissions + +Before connecting to the INCLUDE data warehouse, you must perform a couple of +actions to get an acount with permissions in the INCLUDE data warehouse. + +First, you must request that your aws account have access to the INCLUDE AWS +account. At this time, this access is controlled by the D3B center's Devops +team. To request access, please reach out to @uklineale (Neel Kuila) via slack. +He will walk you through the process of getting access to the INCLUDE AWS +account and any compliance activities you need to complete. Please note that +the process to gain access may take a few days to complete. + +Second, you must request that an account be created for you in the INCLUDE data +warehouse. To request an account, please reach out to your program's +administrator(s), requesting that an account be created for you. The +administrator(s) will create an account for you and add your account to the +appropriate role(s) within the warehouse, so that you will have access to the +appropriate tables immediately. + +After this step is completed you will be furnished with database credentials as +well as the hostname of the database to connect to. + +### Setup SAML CLI access + +You will connect to the warehouse using a command line interface (CLI) that will +authenticate that you can have access to the warehouse. To configure your access +you will use a CLI program from AWS, called `saml2aws`. + +Instructions for installing saml2aws and using this CLI are [here](https://www.notion.so/d3b/Setup-SAML-Login-1056131f1200806ba182f7b7c1793a40?source=copy_link#18a6131f12008082af91f427d2bd71fa) +in the section `CLI Access`. Complete all steps in this section. When using +this tool, you will want to connect create a profile for the AWS account +`373997854230`. + +### Clone `devops_utils` repository + +The [`devops_utils` repository](https://github.com/d3b-center/devops-utils) has +many scripts and tools for interacting with AWS resources. Clone this repository +to your machine. + +## Authentication + +To authenticate your connection to aws resources, you will use the `saml2aws` +CLI described [above](#setup-saml-cli-access), using the command +`saml2aws login`. When using this tool, you will want to connect to a +profile for the AWS account `373997854230`. diff --git a/docs/guides/connect-to-the-warehouse.md b/docs/guides/connect-to-the-warehouse.md index 2014ea8..99714e8 100644 --- a/docs/guides/connect-to-the-warehouse.md +++ b/docs/guides/connect-to-the-warehouse.md @@ -4,47 +4,12 @@ This guide documents how to connect to the include postgres data warehouse. ## First Time Setup -### Pre-requisites: Security, Complianze, and Permissions +For first time setup, see the instructions [here](./connect-to-hosted-infrastructure.md) +If you already completed this setup to connect to some other infrastructure in +the INCLUDE AWS account, such as Airflow, you do not need to complete +the first time setup again. -Before connecting to the INCLUDE data warehouse, you must perform a couple of -actions to get an acount with permissions in the INCLUDE data warehouse. - -First, you must request that your aws account have access to the INCLUDE AWS -account. At this time, this access is controlled by the D3B center's Devops -team. To request access, please reach out to @uklineale (Neel Kuila) via slack. -He will walk you through the process of getting access to the INCLUDE AWS -account and any compliance activities you need to complete. Please note that -the process to gain access may take a few days to complete. - -Second, you must request that an account be created for you in the INCLUDE data -warehouse. To request an account, please reach out to your program's -administrator(s), requesting that an account be created for you. The -administrator(s) will create an account for you and add your account to the -appropriate role(s) within the warehouse, so that you will have access to the -appropriate tables immediately. - -After this step is completed you will be furnished with database credentials as -well as the hostname of the database to connect to. - -### Setup SAML CLI access - -You will connect to the warehouse using a command line interface (CLI) that will -authenticate that you can have access to the warehouse. To configure your access -you will use a CLI program from AWS, called `saml2aws`. - -Instructions for installing saml2aws and using this CLI are [here](https://www.notion.so/d3b/Setup-SAML-Login-1056131f1200806ba182f7b7c1793a40?source=copy_link#18a6131f12008082af91f427d2bd71fa) -in the section `CLI Access`. Complete all steps in this section. When using -this tool, you will want to connect create a profile for the AWS account -`373997854230`. - -### Clone `devops_utils` repository - -The [`devops_utils` repository](https://github.com/d3b-center/devops-utils) has -many scripts and tools for interacting with AWS resources. Clone this repository -to your machine and then navigate to the directory within the repository -`ssm_forward_to_db`. - -## Connecting to the warehouse after the first time +## Connecting to the warehouse 1. run the command `saml2aws login`. When using this tool, you will want to connect create a profile for the AWS account `373997854230`.