v2.00.4

Author: unexpectedpanda

changelog.md

@@ -1,3 +1,57 @@
+# 2.00.4
+
+* Added another level of language selection fallback if you have a language order
+  specified. If Retool finds none of the languages it's looking for in your language
+  order when it compares titles, its next step is to check the language order derived from
+  your region order.
+
+  This is most beneficial for European titles when you're filtering by a language, and
+  want to _preference_ but not _filter_ by other languages. You might want to do this when
+  you want a title that was released in a specific country, but due to No-Intro and Redump
+  naming rules gets assigned to the Europe region.
+
+  For example, if a DAT file has the following titles:
+
+  * _Example Title (Europe) (En,Fr,De,It)_
+  * _Example Title (Europe) (En,Es,Pt)_
+
+  And you have the following region order:
+
+  1. Europe
+  1. Spain
+  1. Portugal
+  1. France
+
+  And you filter by English and no other language, Retool used to choose
+  _Example Title (Europe) (En,Fr,De,It)_, as it contains English (the only language you
+  specified in your language priority) and has the most languages.
+
+  But given your region order, you've expressed a clear preference for Spanish and
+  Portuguese above French.
+
+  Now because of the new region order language fallback, the language order effectively
+  becomes:
+
+  1. English (explicity in the language filter, and also the implied language for Europe)
+  1. Spanish
+  1. Portuguese
+  1. French
+
+  You still only get titles that support English because of the filter, however this means
+  that Retool now selects _Example Title (Europe) (En,Es,Pt)_ instead of
+  _Example Title (Europe) (En,Fr,De,It)_.
+
+* Added granularity back to the kept/removed list. Instead of filing every user exclusion
+  under "category removes", Retool now categorizes under individual exclusions like
+  "application", "audio", "bad dump" and so on.
+
+* Made heading underlines in the kept/removed list variable depending on heading length.
+
+* Fixed an index heading in the kept/removed list that incorrectly had "system excludes"
+  instead of "global excludes".
+
+* Made some minor GUI tweaks for consistency.
+
 # 2.00.3
 
 * Due to the influx of pirate ROMs in the No-Intro NES DAT, pirate ROMs are now treated

docs/changelog.md

@@ -5,22 +5,84 @@ hide:
 
 # Changelog
 
-# 2.00.2
+## 2.00.4
+
+* Added another level of language selection fallback if you have a language order
+  specified. If Retool finds none of the languages it's looking for in your language
+  order when it compares titles, its next step is to check the language order derived from
+  your region order.
+
+  This is most beneficial for European titles when you're filtering by a language, and
+  want to _preference_ but not _filter_ by other languages. You might want to do this when
+  you want a title that was released in a specific country, but due to No-Intro and Redump
+  naming rules gets assigned to the Europe region.
+
+  For example, if a DAT file has the following titles:
+
+  * _Example Title (Europe) (En,Fr,De,It)_
+  * _Example Title (Europe) (En,Es,Pt)_
+
+  And you have the following region order:
+
+  1. Europe
+  1. Spain
+  1. Portugal
+  1. France
+
+  And you filter by English and no other language, Retool used to choose
+  _Example Title (Europe) (En,Fr,De,It)_, as it contains English (the only language you
+  specified in your language priority) and has the most languages.
+
+  But given your region order, you've expressed a clear preference for Spanish and
+  Portuguese above French.
+
+  Now because of the new region order language fallback, the language order effectively
+  becomes:
+
+  1. English (explicity in the language filter, and also the implied language for Europe)
+  1. Spanish
+  1. Portuguese
+  1. French
+
+  You still only get titles that support English because of the filter, however this means
+  that Retool now selects _Example Title (Europe) (En,Es,Pt)_ instead of
+  _Example Title (Europe) (En,Fr,De,It)_.
+
+* Added granularity back to the kept/removed list. Instead of filing every user exclusion
+  under "category removes", Retool now categorizes under individual exclusions like
+  "application", "audio", "bad dump" and so on.
+
+* Made heading underlines in the kept/removed list variable depending on heading length.
+
+* Fixed an index heading in the kept/removed list that incorrectly had "system excludes"
+  instead of "global excludes".
+
+* Made some minor GUI tweaks for consistency.
+
+
+## 2.00.3
+
+* Due to the influx of pirate ROMs in the No-Intro NES DAT, pirate ROMs are now treated
+  like bad dumps. That is, they are demoted below licensed variations of games, even if
+  the pirate ROM is in a preferred language and the licensed version isn't.
+
+
+## 2.00.2
 
 * Expanded support for some newer No-Intro DATs.
 
 
-# 2.00.1
+## 2.00.1
 
 * Expanded the demo regex to incorporate a set of Genesis demos.
 
 
-# 2.00.0
+## 2.00.0
 
 * Fixed grouping of titles with oddly-named video standards in the filename.
 
 
-# 2.00.0 Beta 9
+## 2.00.0 Beta 9
 
 * Fixed conditional override priorities not working.
 

docs/includes/file.md

@@ -1 +1 @@
-[retool-2.00.3-win-x86-64.zip](https://unexpectedpanda.github.io/files/retool-2.00.3-win-x86-64.zip)
+[retool-2.00.4-win-x86-64.zip](https://unexpectedpanda.github.io/files/retool-2.00.4-win-x86-64.zip)

docs/includes/sha256.md

@@ -1 +1 @@
-9b5d4d4de577e52c90c8409355c7ae2be068e1df00b7ded46e1add705570abef
+ced97dc724ceb0112395c6d713abd886c9f7dab8d1f8d4971641794cd166193f

modules/chooseparent.py

@@ -613,18 +613,14 @@ def choose_language(title_set: set[DatNode], config: Config, report_on_match: bo
         # Check if a system config is in play
         language_order: list[str] = []
 
-        for region in config.region_order_user:
-            language_order.extend(config.languages_filter[region])
-
-        # Make sure language entries are unique
-        language_order = reduce(lambda x,y: x + [y] if not y in x else x, language_order, [])
-
         if config.languages_filter:
             language_order = config.language_order_user
 
             if config.system_language_order_user:
                 if {'override': 'true'} in config.system_language_order_user:
                     language_order = [str(x) for x in config.system_language_order_user if 'override' not in x]
+        else:
+            language_order = config.region_order_languages_user
 
         # Select titles based on language
         remove_titles: set[DatNode] = set()
@@ -675,8 +671,38 @@ def choose_language(title_set: set[DatNode], config: Config, report_on_match: bo
                                 break
 
                     if not language_found:
-                        # Cycle through implied language order
+                        if config.languages_filter:
+                            # Cycle through implied languages from region order as the first fallback
+                            for language in config.region_order_languages_user:
+                                if (
+                                    re.search(language, ','.join(title_1.languages))
+                                    and not re.search(language, ','.join(title_2.languages))):
+                                    if title_2 in title_set:
+                                            if report_on_match:
+                                                TraceTools.trace_title('REF0097', [', '.join(config.region_order_languages_user)])
+                                                TraceTools.trace_title('', [f'{Font.italic}({",".join(title_1.languages) + ")":<30}{Font.end} [{title_1.short_name}] {title_1.full_name}', f'{Font.italic}({",".join(title_2.languages) + ")":<30}{Font.end}{Font.disabled} [{title_2.short_name}] {title_2.full_name}{Font.end}'], keep_remove=True)
+
+                                            remove_titles.add(title_2)
+                                            language_found = True
+                                            break
+                                elif (
+                                    re.search(language, ','.join(title_2.languages))
+                                    and not re.search(language, ','.join(title_1.languages))):
+                                        if title_1 in title_set:
+                                            if report_on_match:
+                                                TraceTools.trace_title('REF0098', [', '.join(list(config.region_order_languages_user))])
+                                                TraceTools.trace_title('', [f'{Font.italic}({",".join(title_2.languages) + ")":<30}{Font.end} [{title_2.short_name}] {title_2.full_name}', f'{Font.italic}({",".join(title_1.languages) + ")":<30}{Font.end}{Font.disabled} [{title_1.short_name}] {title_1.full_name}{Font.end}'], keep_remove=True)
+
+                                            remove_titles.add(title_1)
+                                            language_found = True
+                                            break
+
+                    if not language_found:
+                        # Cycle through implied languages from the default region order as the second fallback
                         implied_languages: list[str] = [x[0] for x in config.languages_implied.values()]
+
+                        # Make sure language entries are unique
+                        implied_languages = reduce(lambda x,y: x + [y] if not y in x else x, implied_languages, [])
                         for language in implied_languages:
                             if (
                                 re.search(language, ','.join(title_1.languages))
@@ -702,7 +728,7 @@ def choose_language(title_set: set[DatNode], config: Config, report_on_match: bo
                                         break
 
                     if not language_found:
-                        # Choose the title with more languages
+                        # Choose the title with more languages as the third fallback
                         if len(title_1.languages) > len (title_2.languages):
                             if report_on_match: TraceTools.trace_title('REF0081', [title_1.full_name, title_2.full_name], set(), keep_remove=True)
                             if title_2 in title_set:
@@ -1459,7 +1485,7 @@ def detect_parent_clone_clash(processed_titles: dict[str, set[DatNode]], config:
             for key, values in parent_clash.items():
                 if values['assigned_clone']:
                     eprint(f'\n{Font.warning}* {Font.warning_bold}{key}{Font.warning} should be a parent, but is set as a clone of\n  {Font.warning_bold}{values["assigned_clone"]}{Font.warning}{Font.end}')
-                    eprint(f'\n  {Font.warning}This likely isn\'t an issue, and just a side effect of region and language settings.{Font.end}')
+                    eprint(f'\n  {Font.warning}Sometimes this can happen because there\'s a duplicate entry in the clone list.\nOther times it\'s just a side effect of region and language settings.{Font.end}')
 
                 if values['clones']:
                     eprint(f'\n  {Font.warning}Titles that have {Font.warning_bold}{key}{Font.warning} as a parent:{Font.end}\n')
@@ -1696,18 +1722,14 @@ def choose_parent_process(config: Config, potential_parents: dict[str, set[DatNo
         # Check if a system config is in play
         language_order: list[str] = []
 
-        for region in config.region_order_user:
-            language_order.extend(config.languages_filter[region])
-
-        # Make sure language entries are unique
-        language_order = reduce(lambda x,y: x + [y] if not y in x else x, language_order, [])
-
         if config.languages_filter:
             language_order = config.language_order_user
 
             if config.system_language_order_user:
                 if {'override': 'true'} in config.system_language_order_user:
                     language_order = [str(x) for x in config.system_language_order_user if 'override' not in x]
+        else:
+            language_order = config.region_order_languages_user
 
         region_order: list[str] = config.region_order_user
 

modules/config.py

@@ -326,6 +326,7 @@ def key_missing(section_key: str) -> None:
                         first_run_gui=first_run_gui)
 
         # Import the user config file
+        self.region_order_languages_user: list[str] = []
         self.language_order_user: list[str] = []
         self.languages_user_found: bool = False
         self.region_order_user: list[str] = []
@@ -365,16 +366,18 @@ def key_missing(section_key: str) -> None:
         if user_config.data[user_gui_settings_key]:
             self.user_gui_settings = user_config.data[user_gui_settings_key]
 
-        # Change the user languages list to be regex strings instead of language
-        # names
+        # Get the language order as determined by user regions and user languages, change
+        # the user languages list to be regex strings instead of language names
+        for region in self.region_order_user:
+            self.region_order_languages_user.extend(self.languages_filter[region])
+
+        # Make sure language entries are unique
+        self.region_order_languages_user = reduce(lambda x,y: x + [y] if not y in x else x, self.region_order_languages_user, [])
+
         language_list: list[str] = []
 
         if not self.language_order_user:
-            for region in self.region_order_user:
-                language_list.extend(self.languages_filter[region])
-
-            # Make sure language entries are unique
-            language_list = reduce(lambda x,y: x + [y] if not y in x else x, language_list, [])
+            language_list = self.region_order_languages_user
         else:
             self.languages_user_found = True
 

modules/constants.py

@@ -1,6 +1,6 @@
 # Set the user files and options
 VERSION_MAJOR: str = '2.00'
-VERSION_MINOR: str = '3'
+VERSION_MINOR: str = '4'
 CLONE_LIST_METADATA_DOWNLOAD_LOCATION: str = 'https://raw.githubusercontent.com/unexpectedpanda/retool-clonelists-metadata/main'
 CLONE_LIST_METADATA_DOWNLOAD_LOCATION_KEY: str = 'cloneListMetadataUrl'
 PROGRAM_DOWNLOAD_LOCATION: str = 'https://raw.githubusercontent.com/unexpectedpanda/retool/main'

modules/dats.py

@@ -354,18 +354,14 @@ def category_assign(regexes: tuple[Any, ...], category: str) -> None:
         # Check if a system config is in play
         language_order: list[str] = []
 
-        for region in config.region_order_user:
-            language_order.extend(config.languages_filter[region])
-
-        # Make sure language entries are unique
-        language_order = reduce(lambda x,y: x + [y] if not y in x else x, language_order, [])
-
         if config.languages_filter:
             language_order = config.language_order_user
 
             if config.system_language_order_user:
                 if {'override': 'true'} in config.system_language_order_user:
                     language_order = [str(x) for x in config.system_language_order_user if 'override' not in x]
+        else:
+            language_order = config.region_order_languages_user
 
         region_order: list[str] = config.region_order_user
         if config.system_region_order_user:

modules/gui/retool_ui.py

@@ -2438,7 +2438,7 @@ def setupUi(self, MainWindow):
         self.scrollAreaSystemOptions.setWidgetResizable(True)
         self.layoutScrollAreaSystemOptions = QWidget()
         self.layoutScrollAreaSystemOptions.setObjectName(u"layoutScrollAreaSystemOptions")
-        self.layoutScrollAreaSystemOptions.setGeometry(QRect(0, 0, 422, 725))
+        self.layoutScrollAreaSystemOptions.setGeometry(QRect(0, 0, 573, 725))
         sizePolicy5.setHeightForWidth(self.layoutScrollAreaSystemOptions.sizePolicy().hasHeightForWidth())
         self.layoutScrollAreaSystemOptions.setSizePolicy(sizePolicy5)
         self.verticalLayout_22 = QVBoxLayout(self.layoutScrollAreaSystemOptions)
@@ -2702,6 +2702,7 @@ def setupUi(self, MainWindow):
         self.tabSystemUserFilters.setObjectName(u"tabSystemUserFilters")
         self.verticalLayout_4 = QVBoxLayout(self.tabSystemUserFilters)
         self.verticalLayout_4.setObjectName(u"verticalLayout_4")
+        self.verticalLayout_4.setContentsMargins(-1, -1, 9, 9)
         self.scrollAreaSystemUserFilters = QScrollArea(self.tabSystemUserFilters)
         self.scrollAreaSystemUserFilters.setObjectName(u"scrollAreaSystemUserFilters")
         self.scrollAreaSystemUserFilters.setFocusPolicy(Qt.NoFocus)
@@ -2712,12 +2713,12 @@ def setupUi(self, MainWindow):
         self.scrollAreaSystemUserFilters.setAlignment(Qt.AlignLeading|Qt.AlignLeft|Qt.AlignVCenter)
         self.scrollAreaWidgetContentsSystemUserFilters = QWidget()
         self.scrollAreaWidgetContentsSystemUserFilters.setObjectName(u"scrollAreaWidgetContentsSystemUserFilters")
-        self.scrollAreaWidgetContentsSystemUserFilters.setGeometry(QRect(0, 0, 182, 553))
+        self.scrollAreaWidgetContentsSystemUserFilters.setGeometry(QRect(0, 0, 590, 393))
         self.scrollAreaWidgetContentsSystemUserFilters.setMinimumSize(QSize(0, 0))
         self.gridLayoutSystemUserFilters = QGridLayout(self.scrollAreaWidgetContentsSystemUserFilters)
         self.gridLayoutSystemUserFilters.setObjectName(u"gridLayoutSystemUserFilters")
         self.gridLayoutSystemUserFilters.setSizeConstraint(QLayout.SetMinimumSize)
-        self.gridLayoutSystemUserFilters.setContentsMargins(0, 0, 18, 10)
+        self.gridLayoutSystemUserFilters.setContentsMargins(0, 0, 0, 10)
         self.textEditSystemInclude = CustomTextEdit(self.scrollAreaWidgetContentsSystemUserFilters)
         self.textEditSystemInclude.setObjectName(u"textEditSystemInclude")
         sizePolicy.setHeightForWidth(self.textEditSystemInclude.sizePolicy().hasHeightForWidth())
@@ -3419,7 +3420,7 @@ def retranslateUi(self, MainWindow):
 "that lists what titles have been kept, and what\n"
 "titles have been removed", None))
 #endif // QT_CONFIG(tooltip)
-        self.checkBoxGlobalOptionsKeepRemove.setText(QCoreApplication.translate("MainWindow", u"Also output lists of what titles have been kept and removed", None))
+        self.checkBoxGlobalOptionsKeepRemove.setText(QCoreApplication.translate("MainWindow", u"Also output a list of what titles have been kept and removed", None))
 #if QT_CONFIG(tooltip)
         self.checkBoxGlobalOptions1G1RNames.setToolTip(QCoreApplication.translate("MainWindow", u"In addition to the output DAT file, produce a TXT file that lists only\n"
 "the name of each title in the output DAT file, and optionally add a prefix\n"
@@ -3780,7 +3781,7 @@ def retranslateUi(self, MainWindow):
 "that lists what titles have been kept, and what\n"
 "titles have been removed", None))
 #endif // QT_CONFIG(tooltip)
-        self.checkBoxSystemOptionsKeepRemove.setText(QCoreApplication.translate("MainWindow", u"Also output lists of what titles have been kept and removed", None))
+        self.checkBoxSystemOptionsKeepRemove.setText(QCoreApplication.translate("MainWindow", u"Also output a list of what titles have been kept and removed", None))
 #if QT_CONFIG(tooltip)
         self.checkBoxSystemOptions1G1RNames.setToolTip(QCoreApplication.translate("MainWindow", u"In addition to the output DAT file, produce a TXT file that lists only\n"
 "the name of each title in the output DAT file, and optionally add a prefix\n"