Skip to content

Commit f8e1b18

Browse files
committed
added --no-file and -f options to cli
1 parent d7611f2 commit f8e1b18

File tree

2 files changed

+23
-7
lines changed

2 files changed

+23
-7
lines changed

README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,8 @@ The scraper will output a JSON file called `data.json` in the same directory as
4444

4545
You can modify the scraper to scrape other terms by changing the `year`, `quarter`, and `college_code` variables in `src/config.py`.
4646

47+
To view all the options that the scraper supports, run `python3 src/main.py --help` on Mac/Linux, or `python src/main.py --help` on Windows.
48+
4749
#### Authentication
4850

4951
Since the term master schedule is only accessible to logged-in Drexel students, to run the scraper, you will need to provide your Drexel credentials as well as provide multi-factor authentication (MFA).

src/main.py

Lines changed: 21 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -17,27 +17,36 @@ def main(args: argparse.Namespace) -> None:
1717
data = scrape(include_ratings=args.ratings, all_colleges=args.all_colleges)
1818

1919
assert len(data) > 0, "No data found"
20+
print("Found {} items".format(len(data)))
2021

21-
with open("data.json", "w") as f:
22-
json.dump(data, f, indent=4)
22+
if not args.no_file:
23+
with open(args.output_file, "w") as f:
24+
json.dump(data, f, indent=4)
2325

24-
print("Found {} items".format(len(data)))
25-
print("Data written to data.json")
26+
print(f"Data written to {args.output_file}")
2627

2728
if args.db:
2829
print("Time taken to scrape data: {} seconds".format(time.time() - start_time))
2930
print()
3031
print("Updating database...")
3132
db.populate_db(data)
32-
33-
print("Done!")
33+
print("Done!")
3434

3535
print("--- {} seconds ---".format(time.time() - start_time))
3636

3737

3838
if __name__ == "__main__":
3939
parser = argparse.ArgumentParser(
40-
description="Scrape data from Term Master Schedule and save it to a data.json file."
40+
prog="python3 src/main.py",
41+
description="Scrape data from Drexel Term Master Schedule and save it to a JSON file."
42+
)
43+
parser.add_argument(
44+
"-o",
45+
"--output-file",
46+
metavar="FILE",
47+
action="store",
48+
default="data.json",
49+
help="File to write the data to (include the .json extension in the file name)",
4150
)
4251
parser.add_argument(
4352
"--ratings",
@@ -49,6 +58,11 @@ def main(args: argparse.Namespace) -> None:
4958
action="store_true",
5059
help="Include all colleges in the data, not just the one in the config.py file",
5160
)
61+
parser.add_argument(
62+
"--no-file",
63+
action="store_true",
64+
help="Do not write the data to a file",
65+
)
5266
parser.add_argument(
5367
"--db",
5468
action="store_true",

0 commit comments

Comments
 (0)