Skip to content

Commit af79abc

Browse files
authored
Merge pull request #71 from clowder-framework/70-add-support-for-using-new-v2-visualization-endpoints
70 add support for using new v2 visualization endpoints
2 parents eeecbdc + 19d5852 commit af79abc

File tree

12 files changed

+474
-189
lines changed

12 files changed

+474
-189
lines changed

CHANGELOG.md

Lines changed: 55 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,25 +1,39 @@
1-
# Change Log
1+
# Changelog
2+
23
All notable changes to this project will be documented in this file.
34

4-
The format is based on [Keep a Changelog](http://keepachangelog.com/)
5-
and this project adheres to [Semantic Versioning](http://semver.org/).
5+
The format is based on [Keep a Changelog](https://keepachangelog.com/)
6+
and this project adheres to [Semantic Versioning](https://semver.org/).
7+
8+
## 3.0.2 - 2023-07-20
9+
10+
### Added
11+
12+
- Support for Clowder V2 visualization and visualization config
13+
endpoints. [#70](https://github.com/clowder-framework/pyclowder/issues/70)
614

715
### Fixed
16+
817
- fix bug with delete. [#38](https://github.com/clowder-framework/pyclowder/issues/38)
918

1019
## 3.0.1 - 2023-05-25
1120

1221
### Changed
22+
1323
This version updates Clowder 2 functionality to use API Key headers instead of Bearer tokens.
1424

1525
## 3.0.0 - 2022-12-16
16-
This version adds Clowder 2 support and removes the old method of extractor registration in favor of reliance on heartbeats.
26+
27+
This version adds Clowder 2 support and removes the old method of extractor registration in favor of reliance on
28+
heartbeats.
1729

1830
### Added
31+
1932
- api/v1 and api/v2 code split for back compatibility as v2 is introduced.
2033
- new simplified ClowderClient is used in the new split endpoints for future refactoring.
2134

2235
### Removed
36+
2337
- remove RABBITMQ_EXCHANGE parameter and REGISTRATION_URL parameter.
2438
- remove DatasetsAPI and object-oriented ClowderClient.
2539

@@ -29,6 +43,7 @@ When extractors download a file from clowder it will ask clowder to not track th
2943
This will result in only those donwloads to be counted by users, not extractors.
3044

3145
### Changed
46+
3247
- Ask not to track a download from an extractor.
3348

3449
## 2.6.0 - 2022-06-14
@@ -38,71 +53,86 @@ the queue name (eiter as command line argument or environment variable) the name
3853
extractor shown in clowder, will be the name of the queue.
3954

4055
### Fixed
56+
4157
- both heartbeat and nax_retry need to be converted to in, not string
4258

4359
### Changed
60+
4461
- when you set the RABBITMQ_QUEUE it will change the name of the extractor as well in the
4562
extractor_info document. [#47](https://github.com/clowder-framework/pyclowder/issues/47)
4663
- environment variable CLOWDER_MAX_RETRY is now MAX_RETRY
4764

4865
## 2.5.1 - 2022-03-04
4966

5067
### Changed
68+
5169
- updated pypi documentation
5270

5371
## 2.5.0 - 2022-03-04
5472

5573
### Fixed
74+
5675
- extractor would fail on empty dataset download [#36](https://github.com/clowder-framework/pyclowder/issues/36)
5776

5877
### Added
78+
5979
- ability to set the heartbeat for an extractractor [#42](https://github.com/clowder-framework/pyclowder/issues/42)
6080

6181
### Changed
82+
6283
- update wordcount extractor to not use docker image
6384
- using piptools for requirements
6485

6586
## 2.4.1 - 2021-07-21
6687

6788
### Added
89+
6890
- Add `--max_retry` CLI flag and `CLOWDER_MAX_RETRY` environment variable.
6991

7092
### Changed
93+
7194
- updated all of the requirements to latest versions
7295
- updated github actions to automatically create releases
7396
- use thread.daemon = True to fix python 3.10
7497

7598
## 2.4.0 - 2021-02-22
7699

77100
### Changed
101+
78102
- clowder is no longer the default exchange. Exchanges are no longer used and
79103
this is deprecated.
80104
- fix check for thread is_alive, fixes warning in python 3.9
81105

82106
### Removed
107+
83108
- Removed the extractors.<queue_name> since it was not used.
84109

85110
## 2.3.4 - 2020-10-04
86111

87112
### Fixed
113+
88114
- Extractor would not run in case clowder_url was ""
89115

90116
## 2.3.3 - 2020-10-02
91117

92118
### Fixed
119+
93120
- Thread of heartbeat was set as daemon after start.
94121

95122
## 2.3.2 - 2020-09-24
96123

97124
### Fixed
125+
98126
- When rabbitmq restarts the extractor would not stop and restart, resulting
99127
in the extractor no longer receiving any messages. #17
100128

101129
### Added
130+
102131
- Can specify url to use for extractor downloads, this is helpful for instances
103132
that have access to the internal URL for clowder, for example in docker/kubernetes.
104133

105134
### Removed
135+
106136
- Removed ability to run multiple connectors in the same python process. If
107137
parallelism is needed, use multiple processes (or containers).
108138

@@ -111,6 +141,7 @@ extractor shown in clowder, will be the name of the queue.
111141
With this version we no longer gurantee support for versions of python below 3.
112142

113143
### Fixed
144+
114145
- There was an issue where status messages could cause an exception. This would
115146
prevent most extractors from running correctly.
116147

@@ -125,6 +156,7 @@ From this version no more docker images are build, please use pip install to
125156
install pyclowder.
126157

127158
### Added
159+
128160
- Simple extractors now support datasets, can also create new datasets.
129161
- Ability to add tags from simple extractor to files and datasets.
130162
- Ability to add additional files (outputs) to dataset in simple extractor.
@@ -133,11 +165,13 @@ install pyclowder.
133165
- PyClowderExtractionAbort to indicate the message shoudl not be retried.
134166

135167
### Changed
168+
136169
- Better handling of status messages
137170

138171
## 2.2.3 - 2019-10-14
139172

140173
### Fixed
174+
141175
- Heartbeat of 5 minutes would cause timeouts for RabbitMQ
142176
[CATSPYC-30](https://opensource.ncsa.illinois.edu/jira/browse/CATSPYC-30)
143177
- support uploading large file to dataset.
@@ -146,19 +180,23 @@ install pyclowder.
146180
## 2.2.2 - 2019-09-27
147181

148182
### Changed
183+
149184
- Heartbeat is now every five minutes, was every 5 seconds
150185

151186
### Fixed
187+
152188
- the python3 images were actually python2 images
153189

154190
## 2.2.1 - 2019-08-02
155191

156192
### Changed
193+
157194
- sample extractors had bad extractor_info.json. registry needs to be an array.
158195

159196
## 2.2.0 - 2019-04-03
160197

161198
### Fixed
199+
162200
- Code is now compatible with both Python 2 and Python 3.
163201
- RabbitMQ queue name can be different from extractor name.
164202
- Updated dependencies.
@@ -168,24 +206,29 @@ install pyclowder.
168206
[CATSPYC-18](https://opensource.ncsa.illinois.edu/jira/browse/CATSPYC-18)
169207

170208
### Changed
209+
171210
- files.upload_preview and collections.upload_preview does not require section_id for metadata
172211
[CATS-935](https://opensource.ncsa.illinois.edu/jira/browse/CATS-935)
173212
- Extractors will not register by default to clowder
174213
[CATSPYC-1](https://opensource.ncsa.illinois.edu/jira/browse/CATSPYC-1)
175214

176215
### Added
177-
- Simple R extractor. Allows wrapping R code as a clowder extractor, see sample-extractors/wordcount-simple-r-extractor for an example.
216+
217+
- Simple R extractor. Allows wrapping R code as a clowder extractor, see sample-extractors/wordcount-simple-r-extractor
218+
for an example.
178219
- Email notification when extraction is done
179220
[CATSPYC-17](https://opensource.ncsa.illinois.edu/jira/browse/CATSPYC-17)
180221
- Docker compose file for starting up the Clowder stack
181222
[BD-2226](https://opensource.ncsa.illinois.edu/jira/browse/BD-2226)
182-
- PyClowder will now send heartbeats on extractors exchange, processes can listen for broadcast to get notified when new extractors come online.
223+
- PyClowder will now send heartbeats on extractors exchange, processes can listen for broadcast to get notified when new
224+
extractors come online.
183225
- Monitor application to leverage new heartbeat send out by extractors
184226
- Extractors now send version and name to clowder as part of the agent information.
185227

186228
## 2.1.1 - 2018-07-12
187229

188230
### Fixed
231+
189232
- Error decoding json body from Clowder when filename had special characters
190233
[CATSPYC-18] (https://opensource.ncsa.illinois.edu/jira/browse/CATSPYC-18)
191234
- RABBITMQ_QUEUE variable/flag was ignored when set and would connect
@@ -194,10 +237,12 @@ install pyclowder.
194237
## 2.1.0 - 2018-07-06
195238

196239
### Added
240+
197241
- [Simple extractor](https://opensource.ncsa.illinois.edu/confluence/display/CATS/Simple+Extractor+wrapper+for+basic+functions),
198242
now can create a extractor from a single function.
199243

200244
### Fixed
245+
201246
- Acks were not always send due to racing condition
202247
[CATS-886](https://opensource.ncsa.illinois.edu/jira/browse/CATS-886)
203248
- Queue name was not set from extractor_info.json
@@ -206,12 +251,16 @@ install pyclowder.
206251
## 2.0.3 - 2018-04-18
207252

208253
### Added
254+
209255
- push to pypi (can now do pip install pyclowder)
210256
- added binary-preview-extractor which can be configured using environment
211257
variables (see sample-extractors/binary-preview/README.rst).
212258

213259
### Changed
260+
214261
- Now has onbuild version of pyclowder
215262
- release.sh will now tag images (e.g. this will be tagged 2.0.3 2.0 and 2)
216263
- RABBITMQ_URI is now set to amqp://guest:guest@rabbitmq/%2F to allow easy deployment
217264
using docker-compose of clowder
265+
266+
[unreleased]: https://github.com/clowder-framework/pyclowder/compare/3.0.1...HEAD

0 commit comments

Comments
 (0)