Merge branch 'master' into 50-clowder20-submit-file-to-extractor

Author: ddey2

.github/workflows/pypi.yaml

@@ -10,10 +10,10 @@ on:
 jobs:
   publish:
     name: Build and publish python packages
-    runs-on: ubuntu-18.04
+    runs-on: ubuntu-22.04
 
     steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v3
 
       - name: get release info
         id: release_info

CHANGELOG.md

@@ -1,92 +1,138 @@
-# Change Log
+# Changelog
+
 All notable changes to this project will be documented in this file.
 
-The format is based on [Keep a Changelog](http://keepachangelog.com/)
-and this project adheres to [Semantic Versioning](http://semver.org/).
+The format is based on [Keep a Changelog](https://keepachangelog.com/)
+and this project adheres to [Semantic Versioning](https://semver.org/).
+
+## 3.0.2 - 2023-07-20
+
+### Added
+
+- Support for Clowder V2 visualization and visualization config
+  endpoints. [#70](https://github.com/clowder-framework/pyclowder/issues/70)
+
+### Fixed
+
+- fix bug with delete. [#38](https://github.com/clowder-framework/pyclowder/issues/38)
+
+## 3.0.1 - 2023-05-25
+
+### Changed
+
+This version updates Clowder 2 functionality to use API Key headers instead of Bearer tokens.
 
 ## 3.0.0 - 2022-12-16
-This version adds Clowder 2 support and removes the old method of extractor registration in favor of reliance on heartbeats.
+
+This version adds Clowder 2 support and removes the old method of extractor registration in favor of reliance on
+heartbeats.
 
 ### Added
+
 - api/v1 and api/v2 code split for back compatibility as v2 is introduced.
 - new simplified ClowderClient is used in the new split endpoints for future refactoring.
 
 ### Removed
+
 - remove RABBITMQ_EXCHANGE parameter and REGISTRATION_URL parameter.
 - remove DatasetsAPI and object-oriented ClowderClient.
 
+## 2.7.0 - 2023-02-14
+
+When extractors download a file from clowder it will ask clowder to not track that download.
+This will result in only those donwloads to be counted by users, not extractors.
+
+### Changed
+
+- Ask not to track a download from an extractor.
+
 ## 2.6.0 - 2022-06-14
 
 This will change how clowder sees the extractors. If you have an extractor, and you specify
 the queue name (eiter as command line argument or environment variable) the name of the
 extractor shown in clowder, will be the name of the queue.
 
 ### Fixed
+
 - both heartbeat and nax_retry need to be converted to in, not string
 
 ### Changed
+
 - when you set the RABBITMQ_QUEUE it will change the name of the extractor as well in the
   extractor_info document. [#47](https://github.com/clowder-framework/pyclowder/issues/47)
 - environment variable CLOWDER_MAX_RETRY is now MAX_RETRY
 
 ## 2.5.1 - 2022-03-04
 
 ### Changed
+
 - updated pypi documentation
 
 ## 2.5.0 - 2022-03-04
 
 ### Fixed
+
 - extractor would fail on empty dataset download [#36](https://github.com/clowder-framework/pyclowder/issues/36)
 
 ### Added
+
 - ability to set the heartbeat for an extractractor [#42](https://github.com/clowder-framework/pyclowder/issues/42)
 
 ### Changed
+
 - update wordcount extractor to not use docker image
 - using piptools for requirements
 
 ## 2.4.1 - 2021-07-21
 
 ### Added
+
 - Add `--max_retry` CLI flag and `CLOWDER_MAX_RETRY` environment variable.
 
 ### Changed
+
 - updated all of the requirements to latest versions
 - updated github actions to automatically create releases
 - use thread.daemon = True to fix python 3.10
 
 ## 2.4.0 - 2021-02-22
 
 ### Changed
+
 - clowder is no longer the default exchange. Exchanges are no longer used and
   this is deprecated.
 - fix check for thread is_alive, fixes warning in python 3.9
 
 ### Removed
+
 - Removed the extractors.<queue_name> since it was not used.
 
 ## 2.3.4 - 2020-10-04
 
 ### Fixed
+
 - Extractor would not run in case clowder_url was ""
 
 ## 2.3.3 - 2020-10-02
 
 ### Fixed
+
 - Thread of heartbeat was set as daemon after start.
 
 ## 2.3.2 - 2020-09-24
 
 ### Fixed
+
 - When rabbitmq restarts the extractor would not stop and restart, resulting
   in the extractor no longer receiving any messages. #17
 
 ### Added
+
 - Can specify url to use for extractor downloads, this is helpful for instances
   that have access to the internal URL for clowder, for example in docker/kubernetes.
 
 ### Removed
+
 - Removed ability to run multiple connectors in the same python process. If
   parallelism is needed, use multiple processes (or containers).
 
@@ -95,6 +141,7 @@ extractor shown in clowder, will be the name of the queue.
 With this version we no longer gurantee support for versions of python below 3.
 
 ### Fixed
+
 - There was an issue where status messages could cause an exception. This would
   prevent most extractors from running correctly.
 
@@ -109,6 +156,7 @@ From this version no more docker images are build, please use pip install to
 install pyclowder.
 
 ### Added
+
 - Simple extractors now support datasets, can also create new datasets.
 - Ability to add tags from simple extractor to files and datasets.
 - Ability to add additional files (outputs) to dataset in simple extractor.
@@ -117,11 +165,13 @@ install pyclowder.
 - PyClowderExtractionAbort to indicate the message shoudl not be retried.
 
 ### Changed
+
 - Better handling of status messages
 
 ## 2.2.3 - 2019-10-14
 
 ### Fixed
+
 - Heartbeat of 5 minutes would cause timeouts for RabbitMQ
   [CATSPYC-30](https://opensource.ncsa.illinois.edu/jira/browse/CATSPYC-30)
 - support uploading large file to dataset.
@@ -130,19 +180,23 @@ install pyclowder.
 ## 2.2.2 - 2019-09-27
 
 ### Changed
+
 - Heartbeat is now every five minutes, was every 5 seconds
 
 ### Fixed
+
 - the python3 images were actually python2 images
 
 ## 2.2.1 - 2019-08-02
 
 ### Changed
+
 - sample extractors had bad extractor_info.json. registry needs to be an array.
 
 ## 2.2.0 - 2019-04-03
 
 ### Fixed
+
 - Code is now compatible with both Python 2 and Python 3.
 - RabbitMQ queue name can be different from extractor name.
 - Updated dependencies.
@@ -152,24 +206,29 @@ install pyclowder.
   [CATSPYC-18](https://opensource.ncsa.illinois.edu/jira/browse/CATSPYC-18)
 
 ### Changed
+
 - files.upload_preview and collections.upload_preview does not require section_id for metadata
   [CATS-935](https://opensource.ncsa.illinois.edu/jira/browse/CATS-935)
 - Extractors will not register by default to clowder
   [CATSPYC-1](https://opensource.ncsa.illinois.edu/jira/browse/CATSPYC-1)
 
 ### Added
-- Simple R extractor. Allows wrapping R code as a clowder extractor, see sample-extractors/wordcount-simple-r-extractor for an example.
+
+- Simple R extractor. Allows wrapping R code as a clowder extractor, see sample-extractors/wordcount-simple-r-extractor
+  for an example.
 - Email notification when extraction is done
   [CATSPYC-17](https://opensource.ncsa.illinois.edu/jira/browse/CATSPYC-17)
 - Docker compose file for starting up the Clowder stack
   [BD-2226](https://opensource.ncsa.illinois.edu/jira/browse/BD-2226)
-- PyClowder will now send heartbeats on extractors exchange, processes can listen for broadcast to get notified when new extractors come online.
+- PyClowder will now send heartbeats on extractors exchange, processes can listen for broadcast to get notified when new
+  extractors come online.
 - Monitor application to leverage new heartbeat send out by extractors
 - Extractors now send version and name to clowder as part of the agent information.
 
 ## 2.1.1 - 2018-07-12
 
 ### Fixed
+
 - Error decoding json body from Clowder when filename had special characters
   [CATSPYC-18] (https://opensource.ncsa.illinois.edu/jira/browse/CATSPYC-18)
 - RABBITMQ_QUEUE variable/flag was ignored when set and would connect
@@ -178,10 +237,12 @@ install pyclowder.
 ## 2.1.0 - 2018-07-06
 
 ### Added
+
 - [Simple extractor](https://opensource.ncsa.illinois.edu/confluence/display/CATS/Simple+Extractor+wrapper+for+basic+functions),
   now can create a extractor from a single function.
 
 ### Fixed
+
 - Acks were not always send due to racing condition
   [CATS-886](https://opensource.ncsa.illinois.edu/jira/browse/CATS-886)
 - Queue name was not set from extractor_info.json
@@ -190,12 +251,16 @@ install pyclowder.
 ## 2.0.3 - 2018-04-18
 
 ### Added
+
 - push to pypi (can now do pip install pyclowder)
 - added binary-preview-extractor which can be configured using environment
   variables (see sample-extractors/binary-preview/README.rst).
 
 ### Changed
+
 - Now has onbuild version of pyclowder
 - release.sh will now tag images (e.g. this will be tagged 2.0.3 2.0 and 2)
 - RABBITMQ_URI is now set to amqp://guest:guest@rabbitmq/%2F to allow easy deployment
   using docker-compose of clowder
+
+[unreleased]: https://github.com/clowder-framework/pyclowder/compare/3.0.1...HEAD