Skip to content

Commit d793d98

Browse files
Zohair-coderZohair-coder
authored
Merge pull request #110 from Zohair-coder/auth-fix
Fixed auth
2 parents 58e398c + 1df6672 commit d793d98

File tree

11 files changed

+516
-229
lines changed

11 files changed

+516
-229
lines changed

‎.github/workflows/functional_test.yml‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ jobs:
3434
uses: docker/setup-buildx-action@v3
3535

3636
- name: Start PostgreSQL
37-
run: docker-compose up -d postgres
37+
run: docker compose up -d postgres
3838

3939
- name: Wait for PostgreSQL to become ready
4040
run: |
@@ -48,7 +48,7 @@ jobs:
4848

4949
- name: Run the scraper
5050
env:
51-
DREXEL_USERNAME: ${{ secrets.DREXEL_USERNAME }}
51+
DREXEL_EMAIL: ${{ secrets.DREXEL_EMAIL }}
5252
DREXEL_PASSWORD: ${{ secrets.DREXEL_PASSWORD }}
5353
DREXEL_MFA_SECRET_KEY: ${{ secrets.DREXEL_MFA_SECRET_KEY }}
5454
run: docker compose run scraper python3 src/main.py --db --all-colleges --ratings
@@ -104,11 +104,11 @@ jobs:
104104
fi
105105
106106
- name: Reset database
107-
run: docker compose run scraper sh -c 'apk add postgresql-client && ./scripts/reset-db.sh;'
107+
run: docker compose run scraper sh -c 'apt-get install -y postgresql-client && ./scripts/reset-db.sh;'
108108

109109
- name: Run scraper again (to test cache)
110110
env:
111-
DREXEL_USERNAME: ${{ secrets.DREXEL_USERNAME }}
111+
DREXEL_EMAIL: ${{ secrets.DREXEL_EMAIL }}
112112
DREXEL_PASSWORD: ${{ secrets.DREXEL_PASSWORD }}
113113
DREXEL_MFA_SECRET_KEY: ${{ secrets.DREXEL_MFA_SECRET_KEY }}
114114
run: docker compose run scraper python3 src/main.py --db --all-colleges --ratings
@@ -164,4 +164,4 @@ jobs:
164164
fi
165165
166166
- name: Cleanup
167-
run: docker-compose down -v
167+
run: docker compose down -v

‎Dockerfile‎

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,15 @@
1-
FROM python:3.12-alpine
1+
FROM python:3.12
22

33
# set the working directory in the container
44
WORKDIR /app
55

66
# copy the current directory contents into the container at /app
77
COPY . /app
88

9-
# upgrade pip
10-
RUN pip install --upgrade pip
11-
129
# install dependencies
13-
RUN pip install -r requirements.txt
10+
RUN pip install --upgrade pip && \
11+
pip install -r requirements.txt && \
12+
playwright install --with-deps
1413

1514
# Run the Python script
1615
CMD ["python3", "src/main.py", "--db", "--all-colleges", "--ratings", "--email"]

‎README.md‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,11 +18,13 @@ Make sure you have [Python 3](https://www.python.org/downloads/) installed. Then
1818
###### Mac/Linux
1919
```bash
2020
pip3 install -r requirements.txt
21+
playwright install
2122
```
2223

2324
###### Windows
2425
```bash
2526
pip install -r requirements.txt
27+
playwright install
2628
```
2729

2830
## Usage
@@ -50,13 +52,15 @@ To view all the options that the scraper supports, run `python3 src/main.py --he
5052

5153
Since the term master schedule is only accessible to logged-in Drexel students, to run the scraper, you will need to provide your Drexel credentials as well as provide multi-factor authentication (MFA).
5254

53-
To provide your Drexel credentials, set the environment variable `DREXEL_USERNAME` to your Drexel username (abc123) and `DREXEL_PASSWORD` to the password you use to login to Drexel One. You can follow [this](https://phoenixnap.com/kb/windows-set-environment-variable) guide for Windows, and [this](https://phoenixnap.com/kb/set-environment-variable-mac) guide for MacOS to set environment variables.
55+
To provide your Drexel credentials, set the environment variable `DREXEL_EMAIL` to your Drexel email ([email protected]) and `DREXEL_PASSWORD` to the password you use to login to Drexel One. You can follow [this](https://phoenixnap.com/kb/windows-set-environment-variable) guide for Windows, and [this](https://phoenixnap.com/kb/set-environment-variable-mac) guide for MacOS to set environment variables.
56+
57+
You will also need to go to [this page](https://mysignins.microsoft.com/security-info) and make sure "Authenticator app or hardware token" is the preferred sign-in method. Unfortunately, if you use Microsoft Authenticator as your MFA app you will not be able to run the scraper. You will have to delete the Microsoft Authenticator sign in method and install a different MFA app.
5458

5559
There are two ways to provide MFA for the script to authenticate with. The first is easier if you're looking to run the script manually and quickly. The second is better if you are going to be running the script frequently, or if it needs to be automated.
5660

5761
###### Authenticate manually
5862

59-
You will authenticate the scraper manually as if you were logging into Drexel One, using a one-time code either from an authenticator app or that is texted to you. After setting the `DREXEL_USERNAME` and `DREXEL_PASSWORD` environment variables, run the scraper as explained [above](#Usage), and you will be prompted for your verification code.
63+
You will authenticate the scraper manually as if you were logging into Drexel One, using a one-time code either from an authenticator app or that is texted to you. After setting the `DREXEL_EMAIL` and `DREXEL_PASSWORD` environment variables, run the scraper as explained [above](#Usage), and you will be prompted for your verification code.
6064

6165
###### Authenticate using a secret key
6266

‎cache/extra_course_data_cache.json‎

Lines changed: 212 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -854,5 +854,217 @@
854854
"13241": {
855855
"credits": "1.00",
856856
"prereqs": ""
857+
},
858+
"12555": {
859+
"credits": "0.00",
860+
"prereqs": ""
861+
},
862+
"14785": {
863+
"credits": "3.00",
864+
"prereqs": ""
865+
},
866+
"15543": {
867+
"credits": "0.00",
868+
"prereqs": "CS 265 Minimum Grade: C"
869+
},
870+
"15545": {
871+
"credits": "3.00",
872+
"prereqs": "CS 265 Minimum Grade: C"
873+
},
874+
"15546": {
875+
"credits": "3.00",
876+
"prereqs": "CS 265 Minimum Grade: C"
877+
},
878+
"14795": {
879+
"credits": "3.00",
880+
"prereqs": "CS 504 Minimum Grade: C"
881+
},
882+
"14796": {
883+
"credits": "3.00",
884+
"prereqs": "CS 504 Minimum Grade: C"
885+
},
886+
"15367": {
887+
"credits": "1.00 TO 12.00",
888+
"prereqs": ""
889+
},
890+
"15368": {
891+
"credits": "1.00 TO 12.00",
892+
"prereqs": ""
893+
},
894+
"15369": {
895+
"credits": "1.00 TO 12.00",
896+
"prereqs": ""
897+
},
898+
"15370": {
899+
"credits": "1.00 TO 12.00",
900+
"prereqs": ""
901+
},
902+
"15371": {
903+
"credits": "1.00 TO 12.00",
904+
"prereqs": ""
905+
},
906+
"15372": {
907+
"credits": "1.00 TO 12.00",
908+
"prereqs": ""
909+
},
910+
"15373": {
911+
"credits": "1.00 TO 12.00",
912+
"prereqs": ""
913+
},
914+
"15374": {
915+
"credits": "1.00 TO 12.00",
916+
"prereqs": ""
917+
},
918+
"15375": {
919+
"credits": "1.00 TO 12.00",
920+
"prereqs": ""
921+
},
922+
"15376": {
923+
"credits": "1.00 TO 12.00",
924+
"prereqs": ""
925+
},
926+
"15377": {
927+
"credits": "1.00 TO 12.00",
928+
"prereqs": ""
929+
},
930+
"15378": {
931+
"credits": "1.00 TO 12.00",
932+
"prereqs": ""
933+
},
934+
"15379": {
935+
"credits": "1.00 TO 12.00",
936+
"prereqs": ""
937+
},
938+
"15380": {
939+
"credits": "1.00 TO 12.00",
940+
"prereqs": ""
941+
},
942+
"15381": {
943+
"credits": "1.00 TO 12.00",
944+
"prereqs": ""
945+
},
946+
"15382": {
947+
"credits": "1.00 TO 12.00",
948+
"prereqs": ""
949+
},
950+
"15383": {
951+
"credits": "1.00 TO 12.00",
952+
"prereqs": ""
953+
},
954+
"15384": {
955+
"credits": "1.00 TO 12.00",
956+
"prereqs": ""
957+
},
958+
"15385": {
959+
"credits": "1.00 TO 12.00",
960+
"prereqs": ""
961+
},
962+
"15386": {
963+
"credits": "1.00 TO 12.00",
964+
"prereqs": ""
965+
},
966+
"15387": {
967+
"credits": "1.00 TO 12.00",
968+
"prereqs": ""
969+
},
970+
"15388": {
971+
"credits": "1.00 TO 12.00",
972+
"prereqs": ""
973+
},
974+
"15389": {
975+
"credits": "1.00 TO 12.00",
976+
"prereqs": ""
977+
},
978+
"15529": {
979+
"credits": "3.00",
980+
"prereqs": ""
981+
},
982+
"14819": {
983+
"credits": "3.00",
984+
"prereqs": ""
985+
},
986+
"15333": {
987+
"credits": "3.00",
988+
"prereqs": "INFO 881 Minimum Grade: C"
989+
},
990+
"15334": {
991+
"credits": "3.00",
992+
"prereqs": "INFO 881 Minimum Grade: C"
993+
},
994+
"15305": {
995+
"credits": "3.00 TO 6.00",
996+
"prereqs": ""
997+
},
998+
"15390": {
999+
"credits": "1.00 TO 12.00",
1000+
"prereqs": ""
1001+
},
1002+
"15391": {
1003+
"credits": "1.00 TO 12.00",
1004+
"prereqs": ""
1005+
},
1006+
"15392": {
1007+
"credits": "1.00 TO 12.00",
1008+
"prereqs": ""
1009+
},
1010+
"15393": {
1011+
"credits": "1.00 TO 12.00",
1012+
"prereqs": ""
1013+
},
1014+
"15394": {
1015+
"credits": "1.00 TO 12.00",
1016+
"prereqs": ""
1017+
},
1018+
"15395": {
1019+
"credits": "1.00 TO 12.00",
1020+
"prereqs": ""
1021+
},
1022+
"15396": {
1023+
"credits": "1.00 TO 12.00",
1024+
"prereqs": ""
1025+
},
1026+
"15397": {
1027+
"credits": "1.00 TO 12.00",
1028+
"prereqs": ""
1029+
},
1030+
"15398": {
1031+
"credits": "1.00 TO 12.00",
1032+
"prereqs": ""
1033+
},
1034+
"15399": {
1035+
"credits": "1.00 TO 12.00",
1036+
"prereqs": ""
1037+
},
1038+
"15400": {
1039+
"credits": "1.00 TO 12.00",
1040+
"prereqs": ""
1041+
},
1042+
"15401": {
1043+
"credits": "1.00 TO 12.00",
1044+
"prereqs": ""
1045+
},
1046+
"15402": {
1047+
"credits": "1.00 TO 12.00",
1048+
"prereqs": ""
1049+
},
1050+
"15403": {
1051+
"credits": "1.00 TO 12.00",
1052+
"prereqs": ""
1053+
},
1054+
"15404": {
1055+
"credits": "1.00 TO 12.00",
1056+
"prereqs": ""
1057+
},
1058+
"15405": {
1059+
"credits": "1.00 TO 12.00",
1060+
"prereqs": ""
1061+
},
1062+
"15517": {
1063+
"credits": "1.00 TO 12.00",
1064+
"prereqs": ""
1065+
},
1066+
"15544": {
1067+
"credits": "3.00",
1068+
"prereqs": "CS 265 Minimum Grade: C and CS 260 Minimum Grade: C and (SE 181 Minimum Grade: C or SE 201 Minimum Grade: C )"
8571069
}
8581070
}

0 commit comments

Comments
 (0)