Merge pull request #83 from homebysix/1.17.0

1.17.0 merge to main

Author: homebysix

.pre-commit-config.yaml

@@ -1,6 +1,6 @@
 repos:
   - repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: v4.6.0
+    rev: v5.0.0
     hooks:
       - id: check-ast
       - id: check-byte-order-marker
@@ -16,11 +16,19 @@ repos:
       - id: trailing-whitespace
         args: [--markdown-linebreak-ext=md]
   - repo: https://github.com/asottile/pyupgrade
-    rev: v3.16.0
+    rev: v3.19.1
     hooks:
       - id: pyupgrade
         args: ['--py36-plus']
+  - repo: https://github.com/pycqa/flake8
+    rev: 7.1.1
+    hooks:
+      - id: flake8
+  - repo: https://github.com/pycqa/isort
+    rev: 5.13.2
+    hooks:
+      - id: isort
   - repo: https://github.com/python/black
-    rev: 24.4.2
+    rev: 24.10.0
     hooks:
       - id: black

CHANGELOG.md

@@ -14,6 +14,22 @@ All notable changes to this project will be documented in this file. This projec
 
 Nothing yet.
 
+## [1.17.0] - 2024-12-22
+
+### Added
+
+- New `--warn-on-duplicate-imports` flag for use with Munki pkginfo checks, for Munki administrators who don't care about multiple potential versions of the same pkginfo/pkg in the repository (perhaps because of differing `supported_architectures` or other keys).
+
+    When this is specified, the pre-commit hook will warn when files with `__1` (and similar) suffixes are seen in the pkgsinfo/pkgs folders. This will enbale pre-commit hooks to pass, as long as there are no other errors. Omitting the `--warn-on-duplicate-imports` flag will continue generating an error and failing the hooks, as was the previous behavior.
+
+- Include `SignToolVerifier` and `URLDownloaderPython` AutoPkg processors when suggesting minimum versions.
+- Updated AutoPkg recipe type convention checking to include new `JamfUpload` processors as well as `URLDownloaderPython` and `MunkiInfoCreator`.
+- Added a suggestion to use Rich Trouton's [VariablePlaceholder](https://derflounder.wordpress.com/2024/08/16/setting-custom-variables-in-autopkg-using-the-variableplaceholder-processor/) processor for setting arbitrary environment variables instead of supplying unexpected arguments to existing processors.
+
+### Fixed
+
+- Fixed FileWaveImporter processor detection.
+
 ## [1.16.2] - 2024-06-10
 
 ### Fixed
@@ -367,7 +383,8 @@ Nothing yet.
 
 - Initial release
 
-[Unreleased]: https://github.com/homebysix/pre-commit-macadmin/compare/v1.16.2...HEAD
+[Unreleased]: https://github.com/homebysix/pre-commit-macadmin/compare/v1.17.0...HEAD
+[1.17.0]: https://github.com/homebysix/pre-commit-macadmin/compare/v1.16.2...v1.17.0
 [1.16.2]: https://github.com/homebysix/pre-commit-macadmin/compare/v1.16.1...v1.16.2
 [1.16.1]: https://github.com/homebysix/pre-commit-macadmin/compare/v1.15.0...v1.16.1
 [1.15.0]: https://github.com/homebysix/pre-commit-macadmin/compare/v1.14.1...v1.15.0

README.md

@@ -15,7 +15,7 @@ For any hook in this repo you wish to use, add the following to your pre-commit
 
 ```yaml
 -   repo: https://github.com/homebysix/pre-commit-macadmin
-    rev: v1.16.2
+    rev: v1.17.0
     hooks:
     -   id: check-plists
     # -   id: ...
@@ -61,18 +61,18 @@ After adding a hook to your pre-commit config, it's not a bad idea to run `pre-c
     This hook checks AutoPkg recipes to ensure they meet various requirements and conventions.
 
     - Optionally specify your preferred AutoPkg recipe and/or override prefix, if you wish to enforce them:
-        `args: ['--override-prefix=com.yourcompany.autopkg.']`  
-        (default: `local.`)  
-        `args: ['--recipe-prefix=com.github.yourusername.']`  
+        `args: ['--override-prefix=com.yourcompany.autopkg.']`
+        (default: `local.`)
+        `args: ['--recipe-prefix=com.github.yourusername.']`
         (default: `com.github.`)
 
     - Optionally specify the version of AutoPkg for which you want to ignore MinimumVersion mismatches with processors.
-        `args: ['--ignore-min-vers-before=0.5.0']`  
-        (default: `1.0.0`)  
+        `args: ['--ignore-min-vers-before=0.5.0']`
+        (default: `1.0.0`)
         Specifying `0.1.0` will not ignore any MinimumVersion mismatches.
 
-    - If you're a purist, you can also enable strict mode. This enforces recipe type conventions, all processor/MinimumVersion mismatches, forbids `<!-- -->` style comments, and ensures all processor input variables (arguments) are valid.  
-        `args: ['--strict']`  
+    - If you're a purist, you can also enable strict mode. This enforces recipe type conventions, all processor/MinimumVersion mismatches, forbids `<!-- -->` style comments, and ensures all processor input variables (arguments) are valid.
+        `args: ['--strict']`
         (default: False)
 
 - __forbid-autopkg-overrides__
@@ -117,9 +117,12 @@ After adding a hook to your pre-commit config, it's not a bad idea to run `pre-c
         `args: ['--munki-repo', './my_repo_location']`
         (default: ".")
 
-    - Choose to just warn on missing icons with a flag, note if no other issues exist this will allow pre-commit to pass without seeing the warnings:
+    - Choose to just warn if icons referenced in pkginfo files are missing (this will allow pre-commit checks to pass if no other issues exist):
         `args: ['--warn-on-missing-icons]`
 
+    - Choose to just warn if pkg/pkginfo files with __1 (or similar) suffixes are detected (this will allow pre-commit checks to pass if no other issues exist):
+        `args: ['--warn-on-duplicate-imports]`
+
     - Add additional shebangs that are valid for your environment:
         `args: ['--valid-shebangs', '#!/bin/macadmin/python37', '#!/bin/macadmin/python42', '--']`
 
@@ -141,7 +144,7 @@ When combining arguments that take lists (for example: `--required-keys`, `--cat
 
 ```yaml
 -   repo: https://github.com/homebysix/pre-commit-macadmin
-    rev: v1.16.2
+    rev: v1.17.0
     hooks:
     -   id: check-munki-pkgsinfo
         args: ['--catalogs', 'testing', 'stable', '--']
@@ -151,7 +154,7 @@ But if you also use the `--categories` argument, you would move the trailing `--
 
 ```yaml
 -   repo: https://github.com/homebysix/pre-commit-macadmin
-    rev: v1.16.2
+    rev: v1.17.0
     hooks:
     -   id: check-munki-pkgsinfo
         args: ['--catalogs', 'testing', 'stable', '--categories', 'Design', 'Engineering', 'Web Browsers', '--']
@@ -163,7 +166,7 @@ If it looks better to your eye, feel free to use a multi-line list for long argu
 
 ```yaml
 -   repo: https://github.com/homebysix/pre-commit-macadmin
-    rev: v1.16.2
+    rev: v1.17.0
     hooks:
     -   id: check-munki-pkgsinfo
         args: [

pre_commit_hooks/check_autopkg_recipes.py

@@ -6,6 +6,7 @@
 import os
 import sys
 from contextlib import contextmanager
+
 from packaging.version import Version
 
 from pre_commit_hooks.util import (
@@ -86,7 +87,11 @@ def validate_recipe_prefix(recipe, filename, prefix):
         print(
             "{}: identifier does not start with {}".format(
                 filename,
-                'one of: "%s"' % '", "'.join(prefix) if len(prefix) > 1 else prefix[0],
+                (
+                    'one of: "%s"' % '", "'.join(prefix)
+                    if len(prefix) > 1
+                    else prefix[0]
+                ),
             )
         )
         passed = False
@@ -174,13 +179,19 @@ def validate_minimumversion(process, min_vers, ignore_min_vers_before, filename)
     # Processors for which a minimum version of AutoPkg is required.
     # Note: packaging.version.Version considers this True: "1.0" == "1.0.0"
     proc_min_versions = {
+        "AppDmgVersioner": "0.0",
         "AppPkgCreator": "1.0",
         "BrewCaskInfoProvider": "0.2.5",
+        # "ChocolateyPackager": "3.0",  # hasn't been merged yet
         "CodeSignatureVerifier": "0.3.1",
+        "Copier": "0.0",
         "CURLDownloader": "0.5.1",
         "CURLTextSearcher": "0.5.1",
         "DeprecationWarning": "1.1",
+        "DmgCreator": "0.0",
+        "DmgMounter": "0.0",
         "EndOfCheckPhase": "0.1.0",
+        "FileCreator": "0.0",
         "FileFinder": "0.2.3",
         "FileMover": "0.2.9",
         "FlatPkgPacker": "0.2.4",
@@ -190,21 +201,28 @@ def validate_minimumversion(process, min_vers, ignore_min_vers_before, filename)
         "InstallFromDMG": "0.4.0",
         "MunkiCatalogBuilder": "0.1.0",
         "MunkiImporter": "0.1.0",
+        "MunkiInfoCreator": "0.0",
         "MunkiInstallsItemsCreator": "0.1.0",
         "MunkiOptionalReceiptEditor": "2.7",
         "MunkiPkginfoMerger": "0.1.0",
         "MunkiSetDefaultCatalog": "0.4.2",
         "PackageRequired": "0.5.1",
         "PathDeleter": "0.1.0",
         "PkgCopier": "0.1.0",
+        "PkgCreator": "0.0",
         "PkgExtractor": "0.1.0",
+        "PkgInfoCreator": "0.0",
         "PkgPayloadUnpacker": "0.1.0",
+        "PkgRootCreator": "0.0",
         "PlistEditor": "0.1.0",
         "PlistReader": "0.2.5",
+        "SignToolVerifier": "2.3",
         "SparkleUpdateInfoProvider": "0.1.0",
         "StopProcessingIf": "0.1.0",
         "Symlinker": "0.1.0",
         "Unarchiver": "0.1.0",
+        "URLDownloader": "0.0",
+        "URLDownloaderPython": "2.4.1",
         "URLTextSearcher": "0.2.9",
         "Versioner": "0.1.0",
     }
@@ -367,10 +385,12 @@ def validate_proc_type_conventions(process, filename):
             "SparkleUpdateInfoProvider",
             "GitHubReleasesInfoProvider",
             "URLDownloader",
+            "URLDownloaderPython",
             "CURLDownloader",
             "EndOfCheckPhase",
         ],
         "munki": [
+            "MunkiInfoCreator",
             "MunkiInstallsItemsCreator",
             "MunkiPkginfoMerger",
             "MunkiCatalogBuilder",
@@ -384,18 +404,35 @@ def validate_proc_type_conventions(process, filename):
         "jss": ["JSSImporter"],
         # https://github.com/grahampugh/jamf-upload
         "jamf": [
+            "com.github.grahampugh.jamf-upload.processors/JamfAccountUploader",
             "com.github.grahampugh.jamf-upload.processors/JamfCategoryUploader",
+            "com.github.grahampugh.jamf-upload.processors/JamfClassicAPIObjectUploader",
             "com.github.grahampugh.jamf-upload.processors/JamfComputerGroupUploader",
             "com.github.grahampugh.jamf-upload.processors/JamfComputerProfileUploader",
+            "com.github.grahampugh.jamf-upload.processors/JamfDockItemUploader",
             "com.github.grahampugh.jamf-upload.processors/JamfExtensionAttributeUploader",
+            "com.github.grahampugh.jamf-upload.processors/JamfIconUploader",
+            "com.github.grahampugh.jamf-upload.processors/JamfMacAppUploader",
+            "com.github.grahampugh.jamf-upload.processors/JamfMobileDeviceGroupUploader",
+            "com.github.grahampugh.jamf-upload.processors/JamfMobileDeviceProfileUploader",
+            "com.github.grahampugh.jamf-upload.processors/JamfPackageCleaner",
             "com.github.grahampugh.jamf-upload.processors/JamfPackageUploader",
+            "com.github.grahampugh.jamf-upload.processors/JamfPatchChecker",
+            "com.github.grahampugh.jamf-upload.processors/JamfPatchUploader",
             "com.github.grahampugh.jamf-upload.processors/JamfPolicyDeleter",
+            "com.github.grahampugh.jamf-upload.processors/JamfPolicyLogFlusher",
             "com.github.grahampugh.jamf-upload.processors/JamfPolicyUploader",
             "com.github.grahampugh.jamf-upload.processors/JamfScriptUploader",
             "com.github.grahampugh.jamf-upload.processors/JamfSoftwareRestrictionUploader",
+            "com.github.grahampugh.jamf-upload.processors/JamfUploaderSlacker",
+            "com.github.grahampugh.jamf-upload.processors/JamfUploaderTeamsNotifier",
         ],
         # https://github.com/autopkg/filewave
-        "filewave": ["FileWaveImporter"],
+        "filewave": [
+            "com.github.autopkg.filewave.FWTool/FileWaveImporter",
+            "com.github.johncclayton.filewave.FWTool/FileWaveImporter",
+            "com.github.autopkg.filewave.FWTool/FWTool",
+        ],
     }
 
     passed = True
@@ -489,17 +526,23 @@ def validate_proc_args(process, filename):
                 # Skip args in ignored list above.
                 continue
 
+            suggestion = (
+                "Consider using the VariablePlaceholder processor for adding custom environment variables:\n"
+                "https://derflounder.wordpress.com/2024/08/16/setting-custom-variables-in-autopkg-using-the-variableplaceholder-processor/"
+            )
             if not core_procs[proc["Processor"]]:
                 print(
                     f"{filename}: Unknown argument {arg} for processor {proc['Processor']}, "
                     "which does not accept any arguments."
                 )
+                print(suggestion)
                 passed = False
             elif arg not in core_procs[proc["Processor"]]:
                 print(
                     f"{filename}: Unknown argument {arg} for processor {proc['Processor']}. Allowed arguments are: "
                     + ", ".join(core_procs[proc["Processor"]])
                 )
+                print(suggestion)
                 passed = False
 
     return passed

pre_commit_hooks/check_munki_pkgsinfo.py

@@ -7,7 +7,6 @@
 from pathlib import Path
 from xml.parsers.expat import ExpatError
 
-
 from pre_commit_hooks.util import (
     validate_pkginfo_key_types,
     validate_required_keys,
@@ -45,6 +44,12 @@ def build_argument_parser():
         action="store_true",
         default=False,
     )
+    parser.add_argument(
+        "--warn-on-duplicate-imports",
+        help="If added, this will only warn if pkginfo/pkg files end with a __1 suffix.",
+        action="store_true",
+        default=False,
+    )
     parser.add_argument(
         "--valid-shebangs",
         nargs="+",
@@ -67,7 +72,7 @@ def _check_case_sensitive_path(path):
         if p == p.parent:
             return True
         # If string representation of path is not in parent directory, return False
-        if str(p) not in map(str, p.parent.iterdir()):
+        if str(p) not in list(map(str, p.parent.iterdir())):
             return False
         p = p.parent
 
@@ -154,10 +159,15 @@ def main(argv=None):
 
         # Check for pkg filenames showing signs of duplicate imports.
         if pkginfo.get("installer_item_location", "").endswith(tuple(dupe_suffixes)):
-            print(
-                f'{filename}: installer item "{pkginfo.get("installer_item_location")}" may be a duplicate import'
+            installer_item_location = pkginfo["installer_item_location"]
+            msg = (
+                f"installer item '{installer_item_location}' may be a duplicate import"
             )
-            retval = 1
+            if args.warn_on_missing_icons:
+                print(f"{filename}: WARNING: {msg}")
+            else:
+                print(f"{filename}: {msg}")
+                retval = 1
 
         # Checking for the absence of blocking_applications for pkg installers.
         # If a pkg doesn't require blocking_applications, use empty "<array/>" in pkginfo.
@@ -184,10 +194,11 @@ def main(argv=None):
                 pkginfo.get("installer_type") == "apple_update_metadata",
             )
         ):
+            msg = "missing icon"
             if args.warn_on_missing_icons:
-                print(f"WARNING: {filename}: missing icon")
+                print(f"{filename}: WARNING: {msg}")
             else:
-                print(f"{filename}: missing icon")
+                print(f"{filename}: {msg}")
                 retval = 1
 
         # Ensure uninstall method is set correctly if uninstall_script exists.

pre_commit_hooks/check_munkipkg_buildinfo.py

@@ -8,6 +8,7 @@
 from xml.parsers.expat import ExpatError
 
 import ruamel.yaml
+
 from pre_commit_hooks.util import validate_required_keys
 
 yaml = ruamel.yaml.YAML(typ="safe")

pre_commit_hooks/forbid_autopkg_trust_info.py

@@ -3,6 +3,7 @@
 repo."""
 
 import argparse
+
 from pre_commit_hooks.util import load_autopkg_recipe
 
 

pre_commit_hooks/util.py

@@ -2,7 +2,6 @@
 
 import json
 import plistlib
-import sys
 from datetime import datetime
 
 import ruamel.yaml

setup.py

@@ -6,7 +6,7 @@
     name="pre-commit-macadmin",
     description="Pre-commit hooks for Mac admins, client engineers, and IT consultants.",
     url="https://github.com/homebysix/pre-commit-macadmin",
-    version="1.16.2",
+    version="1.17.0",
     author="Elliot Jordan",
     author_email="[email protected]",
     packages=["pre_commit_hooks"],