config class updates

Author: chazlarson

.gitignore

@@ -10,6 +10,8 @@
 **/*.pickle
 **/*.sqlite
 
+config.yaml
+
 Kometa/metadata-items
 
 Plex/movies

Kometa/.env.example

@@ -6,14 +6,151 @@ Misc scripts and tools. Undocumented scripts probably do what I need them to but
 
 See the top-level [README](../README.md) for setup instructions.
 
-All these scripts use the same `.env` and requirements.
+All these scripts use the same `config.yaml` and requirements.
 
-Any that communicate with Plex require:
-```
-PLEXAPI_AUTH_SERVER_BASEURL=https://plex.domain.tld
-                            # Just the base URL, no /web or anything at the end.
-                            # i.e. http://192.168.1.11:32400 or the like
-PLEXAPI_AUTH_SERVER_TOKEN=PLEX-TOKEN
+### `config.template.yaml` contents
+
+```yaml
+plex_api:
+  header_identifier: "media-scripts"
+  timeout: 360
+  auth_server:
+    base_url: 'YOUR_PLEX_URL'
+    token: 'YOUR_PLEX_TOKEN'
+  log:
+    backup_count: 3
+    format: "%(asctime)s %(module)12s:%(lineno)-4s %(levelname)-9s %(message)s"
+    level: "INFO"
+    path: "plexapi.log"
+    rotate_bytes: 512000
+    show_secrets: 0
+  skip_verify_ssl: 0
+
+general:
+  tmdb_key: "TMDB_API_KEY" # https://developers.themoviedb.org/3/getting-started/introduction
+  tvdb_key: "TVDB_V4_API_KEY" # currently not used; https://thetvdb.com/api-information
+  delay: 1 # optional delay between items
+  library_names: Movies,TV Shows,Movies 4K # comma-separated list of libraries to act on
+  superchat: 0
+
+kometa:
+  config_dir: /kometa/is/here # location of Kometa config dir as seen by this script
+
+image_download:
+  what_to_grab:
+    ### collection-related
+    include_collection_artwork: 1 # should get-all-posters retrieve collection posters?
+    only_collection_artwork: 0 # should get-all-posters retrieve ONLY collection posters?
+    only_these_collections: "Bing|Bang|Boing" # only grab artwork for these collections and items in them
+
+    ### tv-related
+    seasons: 1 # should get-all-posters retrieve season posters?
+    episodes: 1 # should get-all-posters retrieve episode posters? [requires GRAB_SEASONS]
+
+    ### background-related
+    backgrounds: 1 # should get-all-posters retrieve backgrounds?
+    artwork: 1 # current background is downloaded with current poster
+
+    ### quantity-related
+    only_current: 0 # should get-all-posters retrieve ONLY current artwork?
+    poster_depth: 20 # grab this many posters [0 grabs all] [ONLY_CURRENT overrides this]
+
+    ### what-to-keep
+    keep_junk: 0 # keep files that script would normally delete [incorrect filetypes, mainly]
+    find_overlaid_images: 0 # check all downloaded images for overlays
+    retain_overlaid_images: 0 # keep images that have an overlay EXIF tag [this will override the following two]
+    retain_kometa_overlaid_images: 0 # keep images that have the Kometa overlay EXIF tag
+    retain_tcm_overlaid_images: 0 # keep images that have the TCM overlay EXIF tag
+
+  ## where-to-put-it
+  where_to_put_it:
+    use_asset_naming: 1 # should grab-all-posters name images to match Kometa's Asset Directory requirements?
+    use_asset_folders: 1 # should those Kometa-Asset-Directory names use asset folders?
+    use_asset_subfolders: 0 # create asset folders in subfolders ["Collections", "Other", or [0-9, A-Z]] ]
+    assets_by_libraries: 1 # should those Kometa-Asset-Directory images be sorted into library folders?
+    asset_dir: "assets" # top-level directory for those Kometa-Asset-Directory images
+    # if asset-directory naming is on, the next three are ignored
+    poster_dir: "extracted_posters" # put downloaded posters here
+    current_poster_dir: "current_posters" # put downloaded current posters and artwork here
+    poster_consolidate: 0 # if false, posters are separated into folders by library
+
+  ## tracking
+  tracking:
+    track_urls: 1 # If set to 1, URLS are tracked and won't be downloaded twice
+    track_completion: 1 # If set to 1, movies/shows are tracked as complete by rating id
+    track_image_sources: 1 # keep a file containing file names and source URLs
+
+  ## general
+  general:
+    poster_download: 1 # if false, generate a script rather than downloading
+    folders_only: 0 # Just build out the folder hierarchy; no image downloading
+    default_years_back: 2 # in absence of a "last run date", grab things added this many years back.
+    # 0 sets the fallback date to the beginning of time
+    threaded_downloads: 0 # should downloads be done in the background in threads?
+    reset_libraries: "Bing,Bang,Boing" # reset "last time" count to the fallback date for these libraries
+    reset_collections: "Bing,Bang,Boing" # CURRENTLY UNUSED
+    add_source_exif_comment: 1 # CURRENTLY UNUSED
+
+status:
+  plex_owner: "yournamehere" # account name of the server owner
+  target_plex_url: "https://plex.domain2.tld" # As above, the target of apply_all_status
+  target_plex_token: "PLEX-TOKEN-TWO" # As above, the target of apply_all_status
+  target_plex_owner: "yournamehere" # As above, the target of apply_all_status
+  library_map: '{"LIBRARY_ON_PLEX":"LIBRARY_ON_TARGET_PLEX", ...}'
+  # In apply_all_status, map libraries according to this JSON.
+
+reset_posters:
+  track_reset_status: 1 # should reset-posters-* keep track of status and pick up where it left off?
+  clear_reset_status: 0
+  local_reset_archive: 1 # should reset-posters-tmdb keep a local archive of posters?
+  override_overlay_status: 0
+  target_labels: this label, that label # comma-separated list of labels to reset posters on
+  remove_labels: 0 # attempt to remove the TARGET_LABELs from items after resetting the poster
+  reset_seasons: 1 # reset-posters-* resets season artwork as well in TV libraries
+  reset_episodes: 1 # reset-posters-* resets episode artwork as well in TV libraries [requires RESET_SEASONS=True]
+  retain_reset_status_file: 0 # Don't delete the reset progress file at the end
+  flush_status_at_start: 0 # Delete the reset progress file at the start instead of reading it
+  reset_seasons_with_series: 0 # If there isn't a season poster, use the series poster
+  dry_run: 0 # [currently only works with reset-posters-*]; don't actually do anything, just log
+
+list_item_ids:
+  include_collection_members: 0
+  only_collection_members: 0
+
+delete_collection:
+  keep_collections: "bing,bang" # List of collections to keep
+
+refresh_metadata:
+  refresh_1970_only: 1 # If 1, only refresh things that have an originally-available date of 1970-01-01
+
+rematch_items:
+  unmatched_only: 1 # If 1, only rematch things that are currently unmatched
+
+reset_added_at:
+  adjust_date_futures_only: 0 # Only look at items that show up as added in the future
+  adjust_date_epoch_only: 1 # Only adjust items that have "originally available" dates of `1970-01-01`
+
+actor:
+  cast_depth: 20 # how deep to go into the cast for actor collections
+  top_count: 10 # how many actors to export
+  job_type: "Actor"
+  known_for_only: 0 # ignore cast members who are not primarily known as actors
+  build_collections: 0 # build yaml for Kometa config.yml
+  num_collections: 20 # this many actors in Kometa yaml
+  track_gender: 1 # Pay attention to actor gender [as recorded on TMDB]
+  min_gender_none: 5 # include minimum this many "none" gendered actors in the YAML, if possible
+  min_gender_female: 5 # include minimum this many "female" gendered actors in the YAML, if possible
+  min_gender_male: 5 # include minimum this many "male" gendered actors in the YAML, if possible
+  min_gender_nb: 5 # include minimum this many "non-binary" gendered actors in the YAML, if possible
+
+low_poster_count:
+  poster_threshold: 10 # how many posters counts as a "low" count?
+
+crew:
+  depth: 20
+  count: 100
+  target_job: Director
+  show_jobs: 0
 ```
 
 ## Scripts:
@@ -34,18 +171,21 @@ You've deleted stuff from Plex and want to clean up the leftover backup art that
 
 ### Settings
 
-The script uses these settings from the `.env`:
-```
-LIBRARY_NAMES=Movies,TV Shows,Movies 4K      # comma-separated list of libraries to act on
-DELAY=1                                      # optional delay between items
-KOMETA_CONFIG_DIR=/opt/kometa/config/              # path to Kometa config directory
+The script uses these settings from the `config.yaml`:
+```yaml
+general:
+  delay: 1 # optional delay between items
+  library_names: Movies,TV Shows,Movies 4K # comma-separated list of libraries to act on
+
+kometa:
+  config_dir: /kometa/is/here # location of Kometa config dir as seen by this script
 ```
 
 ### Usage
 1. setup as above
 2. Run with `python clean-overlay-backup.py`
 
-The script will catalog the backup files and current Plex contents for each library listed in the `.env`.
+The script will catalog the backup files and current Plex contents for each library listed in the `config.yaml`.
 
 It then compares the two lists, and any files in the backup dir that do not correspond to current items in Plkex are deleted.
 
@@ -56,7 +196,7 @@ connecting to https://plex.bing.bang...
 Loading Movies ...
 Loading movies from Movies  ...
 Completed loading 6965 of 6965 movie(s) from Movies
-Clean Overlay Backup Movies |████████████████████████████████████████| 6965/6965 [100%] in 11.0s (633.23/s) 
+Clean Overlay Backup Movies |████████████████████████████████████████| 6965/6965 [100%] in 11.0s (633.23/s)
 Processed 6965 of 6965
 0 items to delete
 279 items in Plex with no backup art
@@ -72,6 +212,7 @@ You're getting started with Kometa and you want to export your existing collecti
 Here is a quick and dirty [emphasis on "quick" and "dirty"] way to do that.
 
 ### Usage
+
 1. setup as above
 2. Run with `python extract-collections.py`
 
@@ -110,7 +251,7 @@ Here is a basic script to do that.
 
 The script will clone or update the `Kometa-Images` repo, then iterate through it applying overlays to each image and storing them in a parallel file system rooted at `Kometa-Images-Overlaid`, ready for you to use with the Kometa Asset Directory [after moving them to that directory] or via template variables.
 
-It chooses the overlay by name based on the "group" that each collection is part of: 
+It chooses the overlay by name based on the "group" that each collection is part of:
 ```
 Kometa-Images
 ├── aspect
@@ -143,7 +284,7 @@ If there isn't a specific image for a "group", then `Kometa/default_collection_o
 Fetch/Pull on Kometa-Images
 Using default_collection_overlays/overlay-template.png as global overlay
 building list of targets
-Applying overlays |████████████████████████████▎           | ▇▅▃ 5027/7119 [71%] in 3:53 (21.6/s, eta: 1:37) 
+Applying overlays |████████████████████████████▎           | ▇▅▃ 5027/7119 [71%] in 3:53 (21.6/s, eta: 1:37)
 Kometa-Images/genre/Sword & Sandal.jpg
 ```
 
@@ -266,18 +407,29 @@ Note, this will only copy images that have received overlays, and for which the
 
 If you don't have overlays on any episodes, this script will not put any episode images in the asset directory, and so on.
 
+```yaml
+image_download:
+  where_to_put_it:
+    use_asset_folders: 1 # should those Kometa-Asset-Directory names use asset folders?
+    asset_dir: "assets" # top-level directory for those Kometa-Asset-Directory images
 ```
-# ORIGINAL TO ASSETS
-USE_ASSET_FOLDERS=1                          # should the asset directory use asset folders?
-ASSET_DIR=assets                             # top-level directory for those assets
+
+The asset file system will be rooted at the directory in the `asset_dir` setting, and `use_asset_folders` controls whether the images are stored as:
+
+```yaml
+    use_asset_folders: 1
 ```
-The asset file system will be rooted at the directory in the `ASSET_DIR` setting, and `USE_ASSET_FOLDERS` controls whether the images are stored as:
 
-`USE_ASSET_FOLDERS=1`
 ```
 Media-Scripts/Plex/assets/All That Jazz (1979) {imdb-tt0078754} {tmdb-16858}.jpg
 ```
-or `USE_ASSET_FOLDERS=0`
+
+or
+
+```yaml
+    use_asset_folders: 0
+```
+
 ```
 Media-Scripts/Plex/assets/All That Jazz (1979) {imdb-tt0078754} {tmdb-16858}/poster.jpg
 ```
@@ -289,7 +441,7 @@ connecting to https://test-plex.DOMAIN.TLD...
 Loading Test-Movies ...
 Loading movies  ...
 Completed loading 35 of 35 movie(s) from Test-Movies
-Grab all posters Test-Movies |████████████████████████████████████████| 35/35 [100%] in 0.2s (190.63/s) 
+Grab all posters Test-Movies |████████████████████████████████████████| 35/35 [100%] in 0.2s (190.63/s)
 Processed 35 of 35
 Complete!
 ```
@@ -370,10 +522,12 @@ For each movie, gets the cast from TMDB; keeps track across all movies how many
 
 At the end, builds a basic Kometa metadata file for the top N actors.
 
-Script-specific variables in .env:
-```
-CAST_DEPTH=20                   ### HOW DEEP TO GO INTO EACH MOVIE CAST
-TOP_COUNT=10                    ### PUT THIS MANY INTO THE FILE AT THE END
+Script-specific variables in `config.yaml`:
+
+```yaml
+actor:
+  cast_depth: 20 # how deep to go into the cast for actor collections
+  top_count: 10 # how many actors to export
 ```
 
 `CAST_DEPTH` is meant to prevent some journeyman character actor from showing up in the top ten; I'm thinking of someone like Clint Howard who's been in the cast of many movies, but I'm guessing when you think of the top ten actors in your library you're not thinking about Clint.  Maybe you are, though, in which case set that higher.