Merge branch 'develop' into 11438-fix-file-version-summary

Author: stevenwinship

.github/workflows/copy_labels.yml

@@ -6,6 +6,10 @@ on:
 
 jobs:
   copy-labels:
+    # Avoid being triggered by forks
+    if: "! github.event.pull_request.head.repo.fork && github.actor != 'dependabot[bot]'"
+    permissions:
+      pull-requests: write
     runs-on: ubuntu-latest
     name: Copy labels from linked issues
     steps:

doc/release-notes/10680-Bagit-Change.md

@@ -0,0 +1,9 @@
+### Archival Bag Configuration Change
+
+Archival Bags now use the JVM option `dataverse.bagit.sourceorg.name` in generating the bag-info.txt file's "Internal-Sender-Identifier" (in addition to its use for "Source-Organization") rather than pulling the value from a deprecated `bagit.SourceOrganization` entry in Bundle.properties ("Internal-Sender-Identifier" is generated by appending " Catalog" in both cases). Sites using archival bags would not see a change if these settings were already using the same value. See #10680 and #11416.
+
+## Upgrade Instructions
+
+If you are using archival bags, be sure that the `dataverse.bagit.sourceorg.name` JVM option is set.
+
+

doc/release-notes/11305-export-drafts.md

@@ -0,0 +1,7 @@
+### Dataset Metadata Can Be Exported From Draft Datasets (via API)
+
+In previous versions of Dataverse, it was only possible to export metadata from published datasets. It is now possible to export metadata from draft datasets via API as long as you supply an API token that has access to the draft. As before, when exporting metadata from published datasets, only the latest published version is supported. Internal exporters have been updated to work with drafts but external exporters might need to be updated (Croissant definitely does). See "upgrade instructions" below for details. See [the guides](https://dataverse-guide--11398.org.readthedocs.build/en/11398/api/native-api.html#export-metadata-of-a-dataset-in-various-formats), #11305, and #11398.
+
+## Upgrade Instructions
+
+If you are using the Croissant exporter, [update it](https://github.com/gdcc/exporter-croissant) to version 0.1.4 or newer for compatibility with exporting drafts. Other external exporters may need to be updated as well. See https://github.com/gdcc/dataverse-exporters for a list.

doc/release-notes/9620-better-file-bundle-name.md

@@ -0,0 +1,2 @@
+The Data Access APIs that generate multi-file zipped bundles will offer file name suggestions based on the persistent identifiers (for example, `doi-10.70122-fk2-xxyyzz.zip`), instead of the fixed `dataverse_files.zip` as in prior versions.
+See the Data Access API guide for more info. 

doc/sphinx-guides/source/api/dataaccess.rst

@@ -21,7 +21,7 @@ There are a number of reasons why not all of the files can be downloaded:
 - Some of the files are restricted and your API token doesn't have access (you will still get the unrestricted files).
 - The Dataverse installation has limited how large the zip bundle can be.
 
-In the curl example below, the flags ``-O`` and ``J`` are used. When there are no errors, this has the effect of saving the file as "dataverse_files.zip" (just like the web interface). The flags force errors to be downloaded as a file.
+In the curl example below, the flags ``-O`` and ``-J`` are used. When there are no errors, this has the effect of saving the file under the name suggested by Dataverse (which as of v6.7 will be based on the persistent identifier of the dataset and the latest version number, for example ``doi-10.70122-fk2-n2xgbj_1.1.zip``; in prior versions the file name was ``dataverse_files.zip`` in all cases). This mirrors the way the files are saved when downloaded in a browser. The flags also force error messages to be downloaded as a file.
 
 Please note that in addition to the files from dataset, an additional file call "MANIFEST.TXT" will be included in the zipped bundle. It has additional information about the files.
 
@@ -70,6 +70,8 @@ A curl example using a DOI (with version):
 
   curl -O -J -H "X-Dataverse-key:$API_TOKEN" $SERVER_URL/api/access/dataset/:persistentId/versions/$VERSION?persistentId=$PERSISTENT_ID
 
+Similarly to the API above, this will save the downloaded bundle under the name based on the persistent identifier and the version number, for example, ``doi-10.70122-fk2-n2xgbj_1.1.zip`` or ``doi-10.70122-fk2-n2xgbj_draft.zip``.
+
 The fully expanded example above (without environment variables) looks like this:
 
 .. code-block:: bash
@@ -173,7 +175,7 @@ Multiple File ("bundle") download
 
 Alternate Form: POST to ``/api/access/datafiles`` with a ``fileIds`` input field containing the same comma separated list of file ids. This is most useful when your list of files surpasses the allowed URL length (varies but can be ~2000 characters).  
 
-Returns the files listed, zipped. 
+Returns the files listed, zipped. As of v6.7 the name of the zipped bundle will be based on the persistent identifier of the parent dataset, for example, ``doi-10.70122-fk2-xxyyzz.zip``; in prior versions the file name was ``dataverse_files.zip`` in all cases).
 
 .. note:: If the request can only be completed partially - if only *some* of the requested files can be served (because of the permissions and/or size restrictions), the file MANIFEST.TXT included in the zipped bundle will have entries specifying the reasons the missing files could not be downloaded. IN THE FUTURE the API will return a 207 status code to indicate that the result was a partial success. (As of writing this - v.4.11 - this hasn't been implemented yet)
 

doc/sphinx-guides/source/api/native-api.rst

@@ -1605,25 +1605,29 @@ Usage example:
 Export Metadata of a Dataset in Various Formats
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-|CORS| Export the metadata of the current published version of a dataset in various formats.
+|CORS| Export the metadata of either the current published version or the draft version of a dataset in various formats.
 
 To get a list of available formats, see :ref:`available-exporters` and :ref:`get-export-formats`.
 
+If you don't specify a version (see :ref:`dataset-version-specifiers`), ``:latest-published`` is assumed and an API token is not necessary. ``:draft`` is supported if you pass an API token that has access. If you try to pass a version number (e.g. "1.0"), it will only work if it happens to be the latest published version. That is to say, for published versions, only the latest published version is supported.
+
 See also :ref:`batch-exports-through-the-api` and the note below:
 
 .. code-block:: bash
 
   export SERVER_URL=https://demo.dataverse.org
   export PERSISTENT_IDENTIFIER=doi:10.5072/FK2/J8SJZB
   export METADATA_FORMAT=ddi
+  export VERSION=:draft
+  export API_TOKEN=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
 
-  curl "$SERVER_URL/api/datasets/export?exporter=$METADATA_FORMAT&persistentId=$PERSISTENT_IDENTIFIER"
+  curl -H "X-Dataverse-key: $API_TOKEN" "$SERVER_URL/api/datasets/export?exporter=$METADATA_FORMAT&persistentId=$PERSISTENT_IDENTIFIER&version=$VERSION"
 
 The fully expanded example above (without environment variables) looks like this:
 
 .. code-block:: bash
 
-  curl "https://demo.dataverse.org/api/datasets/export?exporter=ddi&persistentId=doi:10.5072/FK2/J8SJZB"
+  curl -H "X-Dataverse-key: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" "https://demo.dataverse.org/api/datasets/export?exporter=ddi&persistentId=doi:10.5072/FK2/J8SJZB&version=:draft"
 
 .. _available-exporters:
 
@@ -1955,6 +1959,8 @@ An example of a ``wget`` command line for crawling ("recursive downloading") of
 
 .. note:: In addition to the files and folders in the dataset, the command line above will also save the directory index of each folder, in a separate folder "dirindex".
 
+.. note:: The recipe above does NOT work if the Dataverse instance serves the files via direct download (i.e., when instead of streaming the content, the download API issues redirect links to the files stored on S3). Unfortunately, this describes **every** file served by some popular Dataverse instances. With direct download, saving an entire dataset while preserving its folders structure is possible, but requires some extra scripting.  
+	  
 List All Metadata Blocks for a Dataset
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 

scripts/issues/11429/add-featured-items.sh

@@ -0,0 +1,17 @@
+#!/bin/sh
+export API_TOKEN=e62ac03b-3bc8-4194-bcd6-d0381fe84e39
+export SERVER_URL=http://localhost:8080
+export ID=:root
+export FILENAME='π.png'
+export CONTENT='test1'
+export DISPLAY_ORDER=0
+
+curl -H "X-Dataverse-key:$API_TOKEN" \
+     -X POST \
+     -F "id=0" \
+     -F "content=$CONTENT" \
+     -F "displayOrder=$DISPLAY_ORDER" \
+     -F "fileName=$FILENAME" \
+     -F "keepFile=false" \
+     -F "file=@$FILENAME" \
+     "$SERVER_URL/api/dataverses/$ID/featuredItems"

scripts/issues/11429/update-featured-item.sh

@@ -0,0 +1,17 @@
+#!/bin/bash -x
+export API_TOKEN=e62ac03b-3bc8-4194-bcd6-d0381fe84e39
+export SERVER_URL=http://localhost:8080
+export ID=78
+export FILENAME='π.png'
+export CONTENT='test1'
+export DISPLAY_ORDER=0
+
+curl -H "X-Dataverse-key:$API_TOKEN" \
+     -X PUT \
+     -F "id=0" \
+     -F "content=$CONTENT" \
+     -F "displayOrder=$DISPLAY_ORDER" \
+     -F "fileName=$FILENAME" \
+     -F "keepFile=false" \
+     -F "file=@$FILENAME" \
+     "$SERVER_URL/api/dataverseFeaturedItems/$ID"

scripts/issues/11429/update-featured-items.sh

@@ -0,0 +1,17 @@
+#!/bin/sh
+export API_TOKEN=e62ac03b-3bc8-4194-bcd6-d0381fe84e39
+export SERVER_URL=http://localhost:8080
+export ID=:root
+export FILENAME='π.png'
+export CONTENT='test1'
+export DISPLAY_ORDER=0
+
+curl -H "X-Dataverse-key:$API_TOKEN" \
+     -X PUT \
+     -F "id=0" \
+     -F "content=$CONTENT" \
+     -F "displayOrder=$DISPLAY_ORDER" \
+     -F "fileName=$FILENAME" \
+     -F "keepFile=false" \
+     -F "file=@$FILENAME" \
+     "$SERVER_URL/api/dataverses/$ID/featuredItems"