Merge pull request #158 from skiyooka/master

Last minute fixes, examples, and updated documentation.

Author: skiyooka

README.md

@@ -1,26 +1,54 @@
-## Overview ##
+# Overview
 
 The hub-rest-api-python provides Python bindings for Hub REST API.
 
-:warning:Recently CVE-2020- 27589, a medium severity security defect, was discovered in the [blackduck PyPi](https://pypi.org/project/blackduck/) library which affects versions 0.0.25 – 0.0.52 that could suppress certificate validation if the calling code used either the upload_scan or download_project_scans methods. These methods did not enforce certificate validation. Other methods in the library are not affected. The defect was fixed in version 0.0.53.
+:warning:Recently [CVE-2020-27589](https://nvd.nist.gov/vuln/detail/CVE-2020-27589), a medium severity security defect,
+was discovered in the [blackduck PyPi](https://pypi.org/project/blackduck) library which affects versions 0.0.25 – 0.0.52
+that could suppress certificate validation if the calling code used either the upload_scan or download_project_scans
+methods. These methods did not enforce certificate validation. Other methods in the library are not affected.
+The defect was fixed in version 0.0.53.
 
-Customers using the [blackduck library](https://pypi.org/project/blackduck/) should upgrade to version 0.0.53, or later, to implement the fix.
+Customers using the [blackduck library](https://pypi.org/project/blackduck) should upgrade to version 0.0.53, or later, to implement the fix.
 
-## To use ##
+# New in 1.0.0
+
+Introducing the new Client class.
+
+In order to provide a more robust long-term connection, faster performance, and an overall better experience a new
+Client class has been designed.
+
+It is backed by a [Requests session](https://docs.python-requests.org/en/master/user/advanced/#session-objects)
+object. The user specifies a base URL, timeout, retries, proxies, and TLS verification upon initialization and these
+attributes are persisted across all requests.
+
+At the REST API level, the Client class provides a consistent way to discover and traverse public resources, uses a
+[generator](https://wiki.python.org/moin/Generators) to fetch all items using pagination, and automatically renews
+the bearer token.
+
+To read more about the new Client class see the [Client User Guide](https://github.com/blackducksoftware/hub-rest-api-python/wiki/Client-User-Guide)
+on the [Hub REST API Python Wiki](https://github.com/blackducksoftware/hub-rest-api-python/wiki).
+
+### Important Note
+The old HubInstance (in HubRestApi.py) keeps its existing functionality for backwards compatibility and therefore does
+**not** currently leverage any of the new features in the Client class.
+
+We believe that the new features are compelling enough to strongly encourage users to consider moving from HubInstance
+to Client. Please give it a try and let us know what you think!
+
+# To use
 
 ```
-pip install blackduck
+pip3 install blackduck
 ```
 
 ```python
 from blackduck import Client
-import json
 import logging
 import os
 
 logging.basicConfig(
     level=logging.INFO,
-    format='[%(asctime)s] {%(module)s:%(lineno)d} %(levelname)s - %(message)s'
+    format="[%(asctime)s] {%(module)s:%(lineno)d} %(levelname)s - %(message)s"
 )
 
 bd = Client(
@@ -35,57 +63,24 @@ for project in bd.get_resource(name='projects'):
 
 ### Examples
 
-Example code showing how to do various things can be found in the *examples* folder. 
-
-## Build ##
-
-You should be using [virtualenv](https://pypi.org/project/virtualenv/), [virtrualenvwrapper](https://virtualenvwrapper.readthedocs.io/en/latest/) to make things easy on yourself.
+Example code showing how to work with the new Client can be found in the *examples/client* folder.
 
-Ref: [Packaging Python Projects Tutorial](https://packaging.python.org/tutorials/packaging-projects/)
+# Test #
+Using [pytest](https://pytest.readthedocs.io/en/latest/contents.html)
 
-### Build the blackduck packages
-To build both the source distribution package and the wheel package,
+```bash
+git clone https://github.com/blackducksoftware/hub-rest-api-python.git
+cd hub-rest-api-python
+# optional but advisable: create/use virtualenv
+# you should have 3.x+, e.g. Python 3.8.0+
 
-```
 pip3 install -r requirements.txt
-python3 setup.py sdist bdist_wheel
-```
-
-### Distribute the package
-
-Requires you have an account on either/both [PyPi](https://pypi.org) and [Test PyPi](https://test.pypi.org) *AND* you must be a package maintainer.
-
-Send a request to [email protected] or [email protected] if you want to be listed as a package maintainer.
-
-#### To PyPi
-
-Upload to PyPi,
-
-```
-twine upload dist/*
-```
-
-Then try installing it from PyPy
-
-```
-pip install blackduck
-```
-
-#### To Test PyPi
-
-Upload to Test PyPi,
-
-```
-twine upload --repository-url https://test.pypi.org/legacy/ dist/*
-```
-
-Then try installing it from Test PyPy
-
-```
-pip install --index-url https://test.pypi.org/simple/ blackduck
+pip3 install .
+cd test
+pytest
 ```
 
-### Install package locally
+## Install package locally
 
 Do this when testing a new version.
 
@@ -96,27 +91,17 @@ pip3 install -r requirements.txt
 pip3 install .
 ```
 
-## Test ##
-Using (pytest)[https://pytest.readthedocs.io/en/latest/contents.html]
+To uninstall:
 
-```bash
-git clone https://github.com/blackducksoftware/hub-rest-api-python.git
-cd hub-rest-api-python
-# optional but advisable: create/use virtualenv
-# you should have 3.x+, e.g. Python 3.7.0
-
-pip3 install -r requirements.txt
-pip3 install .
-cd test
-pytest
+```
+pip3 uninstall blackduck
 ```
 
 ## Where can I get the latest release? ##
-This package is available on PyPi,
+This package is available on PyPi:
 
 `pip3 install blackduck`
 
 ## Documentation ##
-Documentation for hub-rest-api-python can be found on the base project:  [Hub REST API Python Wiki](https://github.com/blackducksoftware/hub-rest-api-python/wiki)
-
-
+Documentation for hub-rest-api-python can be found on the base project:
+[Hub REST API Python Wiki](https://github.com/blackducksoftware/hub-rest-api-python/wiki)

blackduck/Authentication.py

@@ -114,6 +114,7 @@ def __init__(self, session, username, password):
         self.username = username
         self.password = password
         self.bearer_token = None
+        self.csrf_token = None
         self.valid_until = datetime.utcnow()
 
     def __call__(self, request):
@@ -123,6 +124,7 @@ def __call__(self, request):
 
         request.headers.update({ 
             "authorization": f"bearer {self.bearer_token}",
+            "X-CSRF-TOKEN": self.csrf_token
         })
 
         return request
@@ -146,6 +148,7 @@ def authenticate(self):
             try:
                 cookie = response.headers['Set-Cookie']
                 self.bearer_token = cookie[cookie.index('=') + 1:cookie.index(';')]
+                self.csrf_token = response.headers['X-CSRF-TOKEN']
                 # As of 2021.2 the bearer token is good for 2 hours but there
                 # is no explicit reference to expiry time in the response.
                 #

blackduck/Client.py

@@ -58,13 +58,15 @@ def __init__(self, base_url, timeout, retries, verify):
     def request(self, method, url, **kwargs):
         kwargs['timeout'] = self._timeout
 
-        # set default media type if not provided
-        headers = {
-            'accept': "application/json",
-            'content-type': "application/json"
-        }
-        headers.update(kwargs.pop('headers', dict()))
-        kwargs['headers'] = headers
+        if method.lower() == 'get':
+            headers = kwargs.pop('headers', dict())
+            lc_keys = {key.lower(): value for (key, value) in headers.items()}
+            if 'accept' not in lc_keys and 'content-type' not in lc_keys:
+                # set default media type only if neither 'accept' nor 'content-type'
+                # exist as some endpoints may only accept one or the other but not both
+                lc_keys['accept'] = "application/json"
+                lc_keys['content-type'] = "application/json"
+            kwargs['headers'] = lc_keys
 
         url = urljoin(self.base_url, url)
         return super().request(method, url, **kwargs)

examples/client/create_get_and_delete_project.py

@@ -0,0 +1,64 @@
+from blackduck import Client
+
+import argparse
+import logging
+import requests
+
+logging.basicConfig(
+    level=logging.DEBUG,
+    format="[%(asctime)s] {%(module)s:%(lineno)d} %(levelname)s - %(message)s"
+)
+
+parser = argparse.ArgumentParser()
+parser.add_argument("--base-url", required=True, help="Hub server URL e.g. https://your.blackduck.url")
+parser.add_argument("--token-file", dest='token_file', required=True, help="containing access token")
+parser.add_argument("--no-verify", dest='verify', action='store_false', help="disable TLS certificate verification")
+parser.add_argument("--project", required=True, help="Project name")
+args = parser.parse_args()
+
+with open(args.token_file, 'r') as tf:
+    access_token = tf.readline().strip()
+
+bd = Client(
+    base_url=args.base_url,
+    token=access_token,
+    verify=args.verify
+)
+
+project_name = args.project
+
+# POST
+project_data = {
+    'name': project_name,
+    'description': "some description",
+    'projectLevelAdjustments': True,
+}
+
+try:
+    r = bd.session.post("/api/projects", json=project_data)
+    r.raise_for_status()
+    print(f"created project {r.links['project']['url']}")
+except requests.HTTPError as err:
+    # more fine grained error handling here; otherwise:
+    bd.http_error_handler(err)
+
+# GET
+params = {
+    'q': [f"name:{project_name}"]
+}
+project_url = None
+for project in bd.get_items("/api/projects", params=params):
+    if project['name'] == project_name:
+        project_url = bd.list_resources(project)['href']
+        print(f"project url: {project_url}")
+
+# DELETE
+try:
+    r = bd.session.delete(project_url)
+    r.raise_for_status()
+    print("deleted project")
+except requests.HTTPError as err:
+    if err.response.status_code == 404:
+        print("not found")
+    else:
+        bd.http_error_handler(err)

examples/client/password.py

@@ -0,0 +1,33 @@
+from blackduck import Client
+from blackduck.Client import HubSession
+from blackduck.Authentication import CookieAuth
+
+import argparse
+import logging
+from pprint import pprint
+
+logging.basicConfig(
+    level=logging.DEBUG,
+    format="[%(asctime)s] {%(module)s:%(lineno)d} %(levelname)s - %(message)s"
+)
+
+parser = argparse.ArgumentParser()
+parser.add_argument("--base-url", required=True, help="Hub server URL e.g. https://your.blackduck.url")
+parser.add_argument("--username", required=True, help="Hub user")
+parser.add_argument("--password", required=True, help="Hub user")
+parser.add_argument("--no-verify", dest='verify', action='store_false', help="disable TLS certificate verification")
+args = parser.parse_args()
+
+verify = args.verify
+base_url = args.base_url
+session = HubSession(base_url, timeout=15.0, retries=3, verify=verify)
+auth = CookieAuth(session, username=args.username, password=args.password)
+
+bd = Client(
+    base_url=base_url,
+    session=session,
+    auth=auth,
+    verify=verify
+)
+
+pprint(bd.list_resources())

examples/client/quickstart.py

@@ -0,0 +1,34 @@
+from blackduck import Client
+
+import argparse
+import logging
+from pprint import pprint
+
+logging.basicConfig(
+    level=logging.DEBUG,
+    format="[%(asctime)s] {%(module)s:%(lineno)d} %(levelname)s - %(message)s"
+)
+
+parser = argparse.ArgumentParser()
+parser.add_argument("--base-url", required=True, help="Hub server URL e.g. https://your.blackduck.url")
+parser.add_argument("--token-file", dest='token_file', required=True, help="containing access token")
+parser.add_argument("--no-verify", dest='verify', action='store_false', help="disable TLS certificate verification")
+args = parser.parse_args()
+
+with open(args.token_file, 'r') as tf:
+    access_token = tf.readline().strip()
+
+bd = Client(
+    base_url=args.base_url,
+    token=access_token,
+    verify=args.verify
+)
+
+for project in bd.get_resource('projects'):
+    print(f"Project: {project['name']}")
+    print("Project list_resources():")
+    pprint(bd.list_resources(project))
+    for version in bd.get_resource('versions', project):
+        print(f"Version: {version['versionName']}")
+        print("Exiting after printing first project and version")
+        quit(0)