Updates:

- Disabled urllib3 warnings
- Added timeout=0 on `SplunkVersionControl ChangeDetector Directory` savedsearch
- Added some minor comments about `/services/properties/savedsearches/default` (no changes in this version)
- Updated various internal calls to use sslVerify setting. Hopefully nothing will break but this will result in more SSL verification in various parts of the code

Library updates:
- Updated Splunk python SDK to 1.7.3

Author: gjanders

README.md

@@ -314,7 +314,26 @@ To do this you will need to install Version Control For SplunkCloud on your Splu
 
 [SplunkVersionControlCloud github](https://github.com/gjanders/SplunkVersionControlCloud)
 
+## How does this compare with other version control apps for Splunk?
+As of October 2022, there are still no signs of version control within the Splunk Enterprise (or cloud) product, however you do have a few options in terms of a version control app, these include:
+- [Git Version Control for Splunk](https://splunkbase.splunk.com/app/4182) - this app provides a modular input to help with getting configuration into a git repository from the filesystem. Note: on-prem instances only, no Splunk Cloud support.
+- [FN1315 - Cover Your Assets: Protect Your Knowledge Objects from Yourself (and Others) - A Paychex story github](https://github.com/paychex/Splunk.Conf19) - this git location provides a list of searches that produce curl commands you can use to restore objects. This can work on-prem or in Splunk Cloud
+- [Splunk2Git](https://github.com/paychex/splunk-python/tree/main/Splunk2Git) - Paychex's script to move Splunk knowledge objects into git using REST API
+- [Version Control for Splunk (this app)](https://splunkbase.splunk.com/app/4355) - this app uses the REST API to download configuration and store inside a git repository in JSON format. Supports restoration of objects via dashboard (no admin support required). This can work on-prem or on Splunk Cloud remotely (this app runs on prem)
+- [VersionControl for SplunkCloud](https://splunkbase.splunk.com/app/5061) - these are the dashboards and savedsearches that are installed on the SplunkCloud instance to support the version control app running remotely
+- [Search Head Backup](https://splunkbase.splunk.com/app/6438) - backup to an index, works in Splunk Cloud
+
 ## Release Notes
+### 1.2.10
+Updates:
+- Disabled urllib3 warnings
+- Added timeout=0 on `SplunkVersionControl ChangeDetector Directory` savedsearch
+- Added some minor comments about `/services/properties/savedsearches/default` (no changes in this version)
+- Updated various internal calls to use sslVerify setting. Hopefully nothing will break but this will result in more SSL verification in various parts of the code
+
+Library updates:
+- Updated Splunk python SDK to 1.7.3
+
 ### 1.2.9
 New features:
 - Added wildcard support for restores, so restore a savedsearch of `Test*` will now restore any savedsearch starting with Test, wildcards can be used on any knowledge object

bin/postversioncontrolrestore.py

@@ -6,13 +6,16 @@
 import re
 import logging
 from logging.config import dictConfig
+import urllib3
 
 sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "lib"))
 
 from splunklib.searchcommands import dispatch, GeneratingCommand, Configuration, Option
 from splunklib.searchcommands.validators import Validator, Boolean
 from splunklib.binding import HTTPError
 
+urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
+
 class OrValidator(Validator):
     def __init__(self, a, b):
         self.a = a

bin/splunkversioncontrol_backup.py

@@ -9,6 +9,7 @@
 import platform
 from splunkversioncontrol_backup_class import SplunkVersionControlBackup
 from splunkversioncontrol_utility import runOSProcess, get_password
+import urllib3
 
 """
 
@@ -18,6 +19,8 @@
 
 """
 
+urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
+
 #Define the XML scheme for the inputs page
 SCHEME = """<scheme>
     <title>Splunk Version Control Backup</title>
@@ -313,7 +316,7 @@ def validate_arguments():
         srcUsername = val_data['srcUsername']
         srcPassword = val_data['srcPassword']
         if srcPassword.find("password:") == 0:
-            srcPassword = get_password(srcPassword[9:], session_key, logger)
+            srcPassword = get_password(srcPassword[9:], session_key, logger, sslVerify)
 
         proxies = {}
         if 'proxy' in val_data:
@@ -322,7 +325,7 @@ def validate_arguments():
                 start = proxies['https'].find("password:") + 9
                 end = proxies['https'].find("@")
                 logger.debug("Attempting to replace proxy=%s by subsituting=%s with a password" % (proxies['https'], proxies['https'][start:end]))
-                temp_password = get_password(proxies['https'][start:end], session_key, logger)
+                temp_password = get_password(proxies['https'][start:end], session_key, logger, sslVerify)
                 proxies['https'] = proxies['https'][0:start-9] + temp_password + proxies['https'][end:]
 
         try:
@@ -346,7 +349,7 @@ def validate_arguments():
             start = gitRepoURL.find("password:") + 9
             end = gitRepoURL.find("@")
             logger.debug("Attempting to replace gitRepoURL=%s by subsituting=%s with a password" % (gitRepoURL, gitRepoURL[start:end]))
-            git_password = get_password(gitRepoURL[start:end], session_key, logger)
+            git_password = get_password(gitRepoURL[start:end], session_key, logger, sslVerify)
             gitRepoURL = gitRepoURL[0:start-9] + git_password + gitRepoURL[end:]
     else:
         gitRepoHTTP = False
@@ -358,7 +361,7 @@ def validate_arguments():
             start = git_proxies['https'].find("password:") + 9
             end = git_proxies['https'].find("@")
             logger.debug("Attempting to replace git_proxy=%s by subsituting=%s with a password" % (git_proxies['https'], git_proxies['https'][start:end]))
-            temp_password = get_password(git_proxies['https'][start:end], session_key, logger)
+            temp_password = get_password(git_proxies['https'][start:end], session_key, logger, sslVerify)
             git_proxies['https'] = git_proxies['https'][0:start-9] + temp_password + git_proxies['https'][end:]
 
     if gitRepoHTTP and len(git_proxies) > 0:

bin/splunkversioncontrol_backup_class.py

@@ -16,6 +16,7 @@
 from io import open
 import platform
 import hashlib
+import urllib3
 from splunkversioncontrol_utility import runOSProcess, get_password
 sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "lib"))
 
@@ -30,6 +31,8 @@
 
 """
 
+urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
+
 splunkLogsDir = os.environ['SPLUNK_HOME'] + "/var/log/splunk"
 #Setup the logging
 logging_config = dict(
@@ -233,6 +236,7 @@ def runQueries(self, app, endpoint, type, fieldIgnoreList, aliasAttributes={}, v
                     elif innerChild.tag.endswith("content"):
 
                         # optimisation to deal with the giant display.visualizations... on savedsearches only
+                        # an improved version of this optimisation may be to use /services/properties/savedsearches/default and to filter out *all* default parameters for searches
                         skip_visualizations = True
 
                         for theAttribute in innerChild[0]:
@@ -374,7 +378,8 @@ def runQueries(self, app, endpoint, type, fieldIgnoreList, aliasAttributes={}, v
                     if sharing not in infoList:
                         infoList[sharing] = []
 
-                    #REST API does not support the creation of null queue entries as tested in 7.0.5 and 7.2.1, these are also unused on search heads anyway so ignoring these with a warning
+                    # REST API does not support the creation of null queue entries as tested in 7.0.5 and 7.2.1, these are also unused on search heads anyway so ignoring these with a warning
+                    # however this might instead work on /services/properties/... or /services/configs/conf-... endpoints, to be tested if the requirement comes up
                     if type == "fieldtransformations" and "FORMAT" in info and info["FORMAT"] == "nullQueue":
                         logger.info("i=\"%s\" Dropping the backup of name=\"%s\" of type=%s in app context app=%s with owner=%s because nullQueue entries cannot be created via REST API (and they are not required in search heads)" % (self.stanzaName, info["name"], type, app, info["owner"]))
                     else:
@@ -949,7 +954,7 @@ def dashboards(self, app):
     ###########################
     def savedsearches(self, app):
         ignoreList = [ "embed.enabled", "triggered_alert_count", "next_scheduled_time", "qualifiedSearch" ]
-
+        # TODO /services/properties/savedsearches/default could be used to further filter out "default" values going into the git backup
         return self.runQueries(app, "/saved/searches", "savedsearches", ignoreList, extra_args="&listDefaultActionArgs=false")
 
     ###########################
@@ -1437,6 +1442,17 @@ def run_script(self):
 
         self.session_key = config['session_key']
 
+        if 'sslVerify' in config:
+            if config['sslVerify'].lower() == 'true' or config['sslVerify'] == "1":
+                self.sslVerify = True
+                logger.debug('sslverify set to boolean True from: ' + config['sslVerify'])
+            elif config['sslVerify'].lower() == 'false' or config['sslVerify'] == "0":
+                self.sslVerify = False
+                logger.debug('sslverify set to boolean False from: ' + config['sslVerify'])
+            else:
+                self.sslVerify = config['sslVerify']
+                logger.debug('sslverify set to: ' + config['sslVerify'])
+
         self.git_password = False
         # a flag for a http/https vs SSH based git repo
         if self.gitRepoURL.find("http") == 0:
@@ -1446,7 +1462,7 @@ def run_script(self):
                 start = self.gitRepoURL.find("password:") + 9
                 end = self.gitRepoURL.find("@")
                 logger.debug("Attempting to replace self.gitRepoURL=%s by subsituting=%s with a password" % (self.gitRepoURL, self.gitRepoURL[start:end]))
-                self.git_password = get_password(self.gitRepoURL[start:end], self.session_key, logger)
+                self.git_password = get_password(self.gitRepoURL[start:end], self.session_key, logger, self.sslVerify)
                 self.gitRepoURL = self.gitRepoURL[0:start-9] + self.git_password + self.gitRepoURL[end:]
             else:
                 self.gitRepoURL_logsafe = self.gitRepoURL
@@ -1495,7 +1511,7 @@ def run_script(self):
                 start = proxies['https'].find("password:") + 9
                 end = proxies['https'].find("@")
                 logger.debug("Attempting to replace proxy=%s by subsituting=%s with a password" % (proxies['https'], proxies['https'][start:end]))
-                temp_password = get_password(proxies['https'][start:end], self.session_key, logger)
+                temp_password = get_password(proxies['https'][start:end], self.session_key, logger, self.sslVerify)
                 proxies['https'] = proxies['https'][0:start-9] + temp_password + proxies['https'][end:]
 
         self.proxies = proxies
@@ -1507,22 +1523,11 @@ def run_script(self):
                 start = git_proxies['https'].find("password:") + 9
                 end = git_proxies['https'].find("@")
                 logger.debug("Attempting to replace git_proxy=%s by subsituting=%s with a password" % (git_proxies['https'], git_proxies['https'][start:end]))
-                temp_password = get_password(git_proxies['https'][start:end], self.session_key, logger)
+                temp_password = get_password(git_proxies['https'][start:end], self.session_key, logger, self.sslVerify)
                 git_proxies['https'] = git_proxies['https'][0:start-9] + temp_password + git_proxies['https'][end:]
 
         self.git_proxies = git_proxies
 
-        if 'sslVerify' in config:
-            if config['sslVerify'].lower() == 'true' or config['sslVerify'] == "1":
-                self.sslVerify = True
-                logger.debug('sslverify set to boolean True from: ' + config['sslVerify'])
-            elif config['sslVerify'].lower() == 'false' or config['sslVerify'] == "0":
-                self.sslVerify = False
-                logger.debug('sslverify set to boolean False from: ' + config['sslVerify'])
-            else:
-                self.sslVerify = config['sslVerify']
-                logger.debug('sslverify set to: ' + config['sslVerify'])
-
         self.file_per_ko = False
         if 'file_per_ko' in config:
             if config['file_per_ko'].lower() == 'true' or config['file_per_ko'] == "1":
@@ -1552,7 +1557,7 @@ def run_script(self):
         headers={'Authorization': 'Splunk %s' % config['session_key']}
 
         url = 'https://localhost:8089/services/shcluster/captain/info?output_mode=json'
-        res = requests.get(url, headers=headers, verify=False)
+        res = requests.get(url, headers=headers, verify=self.sslVerify)
         if (res.status_code == 503):
             logger.debug("i=\"%s\" Non-shcluster / standalone instance, safe to run on this node" % (self.stanzaName))
         elif (res.status_code != requests.codes.ok):
@@ -1567,7 +1572,7 @@ def run_script(self):
                 logger.info("i=\"%s\" we are on the captain node, running" % (self.stanzaName))
 
         if not useLocalAuth and self.srcPassword.find("password:") == 0:
-            self.srcPassword = get_password(self.srcPassword[9:], self.session_key, logger)
+            self.srcPassword = get_password(self.srcPassword[9:], self.session_key, logger, self.sslVerify)
 
         if 'appsList' in config and config['appsList']!="":
             appList = [app.strip() for app in config['appsList'].split(',')]
@@ -1876,7 +1881,7 @@ def run_script(self):
                 if len(self.git_proxies) > 0 and self.gitRepoHTTP:
                     push_str = "export HTTPS_PROXY=" + self.git_proxies["https"] + " ; " + push_str
                 (output, stderrout, res) = runOSProcess(push_str, logger, timeout=300, shell=True)
-                res=True
+                #res=True
             if res == False:
                 if not self.show_passwords and self.git_password:
                     output = output.replace(self.git_password, "password_removed")