Merge pull request #106 from Zohair-coder/auth-usability

Improve authorization usability for new users

Author: Zohair-coder

README.md

@@ -27,7 +27,8 @@ pip install -r requirements.txt
 
 ## Usage
 
-To run the scraper, simply run:
+First, [set up authentication](#Authentication).
+Then, to run the scraper, simply run:
 
 ###### Mac/Linux
 ```bash
@@ -43,6 +44,36 @@ The scraper will output a JSON file called `data.json` in the same directory as
 
 You can modify the scraper to scrape other terms by changing the `year`, `quarter`, and `college_code` variables in `src/config.py`.
 
+#### Authentication
+
+Since the term master schedule is only accessible to logged-in Drexel students, to run the scraper, you will need to provide your Drexel credentials as well as provide multi-factor authentication (MFA).
+
+To provide your Drexel credentials, set the environment variable `DREXEL_USERNAME` to your Drexel username (abc123) and `DREXEL_PASSWORD` to the password you use to login to Drexel One. You can follow [this](https://phoenixnap.com/kb/windows-set-environment-variable) guide for Windows, and [this](https://phoenixnap.com/kb/set-environment-variable-mac) guide for MacOS to set environment variables.
+
+There are two ways to provide MFA for the script to authenticate with. The first is easier if you're looking to run the script manually and quickly. The second is better if you are going to be running the script frequently, or if it needs to be automated.
+
+###### Authenticate manually
+
+You will authenticate the scraper manually as if you were logging into Drexel One, using a one-time code either from an authenticator app or that is texted to you. After setting the `DREXEL_USERNAME` and `DREXEL_PASSWORD` environment variables, run the scraper as explained [above](#Usage), and you will be prompted for your verification code.
+
+###### Authenticate using a secret key
+
+If you set this up, you will not need to manually enter an authentication code each time you run the scraper. 
+
+1. Go to [connect.drexel.edu](connect.drexel.edu). 
+2. Click 'Help & Settings', then 'Change MFA settings'.
+3. Log in to the Microsoft portal, then click 'Add sign-in method' on the 'Security info' tab.
+4. Select 'Authenticator app' for the method, and click 'Add'.
+5. Select 'I want to use a different authenticator app', and then 'Next'.
+6. Select 'Can't scan image?' when prompted with a QR code, and you should see an Account name and Secret key.
+7. Set your `DREXEL_MFA_SECRET_KEY` environment variable to the given Secret key.
+8. Select 'Next', you should be prompted to enter an authentication code.
+9. With the secret key environment variable set, run `python3 src/totp.py` which will generate a one-time code.
+10. Enter this code into the Microsoft website, and select 'Next'
+11. 'Authenticator app' with TOTP should have been added to the list of available methods.
+
+Now, when you run the scraper as explained [above](#Usage), it should authenticate itself automatically using this secret key.
+
 #### All Colleges
 
 To scrape all colleges instead of just the one specified in the `src/config.py`, run the following command:

src/config.py

@@ -1,4 +1,5 @@
 import os
+import sys
 
 # in format YYYY (e.g. 2022)
 # example value: 2022
@@ -15,10 +16,29 @@
 # example values = CI, A, AS
 college_code = "CI"
 
+# Warn users if they have missing required environment variables
+environ_help_url = (
+    "https://github.com/Zohair-coder/drexel-scraper?tab=readme-ov-file#authentication"
+)
+
+
+def get_environ(key: str, required: bool = True) -> str:
+    if key in os.environ:
+        return os.environ[key]
+    elif required:
+        print(
+            f"{key} is missing from your environment variables and is required to run this script. See {environ_help_url} for more information and help."
+        )
+        sys.exit(1)
+    else:
+        return ""
+
+
 # Drexel Connect Credentials
-drexel_username = os.environ["DREXEL_USERNAME"]
-drexel_password = os.environ["DREXEL_PASSWORD"]
-drexel_mfa_secret_key = os.environ["DREXEL_MFA_SECRET_KEY"]
+drexel_username = get_environ("DREXEL_USERNAME")
+drexel_password = get_environ("DREXEL_PASSWORD")
+# This is not required if the user is using a separate authenticator app and will manually approve the login attempt
+drexel_mfa_secret_key = get_environ("DREXEL_MFA_SECRET_KEY", False) or None
 
 # URL's
 tms_base_url = "https://termmasterschedule.drexel.edu"

src/login.py

@@ -78,12 +78,15 @@ def login_with_drexel_connect(session: Session) -> Session:
 
     parsed_data = parse_final_mfa_page(soup)
 
-    totp_code = totp.get_token(config.drexel_mfa_secret_key)
+    if config.drexel_mfa_secret_key is not None:
+        mfa_token = totp.get_token(config.drexel_mfa_secret_key)
+    else:
+        mfa_token = input("Please input your MFA verification code: ")
 
     data = {
         "csrf_token": parsed_data["csrf_token"],
         "_eventId": "proceed",
-        "j_mfaToken": totp_code,
+        "j_mfaToken": mfa_token,
     }
 
     # this request sends the MFA code to Drexel Connect

src/totp.py

@@ -15,3 +15,14 @@ def get_token(secret: str) -> str:
     o = h[19] & 15
     h = (struct.unpack(">I", h[o : o + 4])[0] & 0x7FFFFFFF) % 1000000
     return str(h).zfill(6)
+
+
+if __name__ == "__main__":
+    import config
+
+    if config.drexel_mfa_secret_key is not None:
+        print(get_token(config.drexel_mfa_secret_key))
+    else:
+        print(
+            f"Please set your MFA secret key to run this script. See {config.environ_help_url} for more information and help"
+        )