Merge branch 'develop' into 11639-db-opts-idempotency

Author: sekmiller

.github/workflows/deploy_beta_testing.yml

@@ -36,7 +36,7 @@ jobs:
         run: echo "war_file=$(ls *.war | head -1)">> $GITHUB_ENV
 
       - name: Upload war artifact
-        uses: actions/upload-artifact@v4
+        uses: actions/upload-artifact@v5
         with:
           name: built-app
           path: ./target/${{ env.war_file }}
@@ -50,7 +50,7 @@ jobs:
       - uses: actions/checkout@v5
 
       - name: Download war artifact
-        uses: actions/download-artifact@v5
+        uses: actions/download-artifact@v6
         with:
           name: built-app
           path: ./

.github/workflows/maven_unit_test.yml

@@ -62,7 +62,7 @@ jobs:
 
           # Upload the built war file. For download, it will be wrapped in a ZIP by GitHub.
           # See also https://github.com/actions/upload-artifact#zipped-artifact-downloads
-          - uses: actions/upload-artifact@v4
+          - uses: actions/upload-artifact@v5
             with:
                 name: dataverse-java${{ matrix.jdk }}.war
                 path: target/dataverse*.war
@@ -72,7 +72,7 @@ jobs:
           - run: |
                 tar -cvf java-builddir.tar target
                 tar -cvf java-m2-selection.tar ~/.m2/repository/io/gdcc/dataverse-*
-          - uses: actions/upload-artifact@v4
+          - uses: actions/upload-artifact@v5
             with:
                 name: java-artifacts
                 path: |
@@ -112,7 +112,7 @@ jobs:
                   cache: maven
 
             # Get the build output from the unit test job
-            - uses: actions/download-artifact@v5
+            - uses: actions/download-artifact@v6
               with:
                   name: java-artifacts
             - run: |
@@ -124,7 +124,7 @@ jobs:
 
             # Wrap up and send to coverage job
             - run: tar -cvf java-reportdir.tar target/site
-            - uses: actions/upload-artifact@v4
+            - uses: actions/upload-artifact@v5
               with:
                   name: java-reportdir
                   path: java-reportdir.tar
@@ -145,7 +145,7 @@ jobs:
                 cache: maven
 
           # Get the build output from the integration test job
-          - uses: actions/download-artifact@v5
+          - uses: actions/download-artifact@v6
             with:
                 name: java-reportdir
           - run: tar -xvf java-reportdir.tar

doc/release-notes/11771-update-dataset-license-api.md

@@ -0,0 +1,13 @@
+## New Endpoint: `/datasets/{id}/license`
+
+A new endpoint has been implemented to manage dataset licenses.
+
+### Functionality
+- Updates the license of a dataset by applying it to the draft version.
+- If no draft exists, a new one is automatically created.
+
+### Usage
+This endpoint supports two ways of defining a license:
+1. **Predefined License** – Provide the license name (e.g., `CC BY 4.0`).
+2. **Custom Terms of Use and Access** – Provide a JSON body with the `customTerms` object.
+    - All fields are optional **except** `termsOfUse`, which is required.

doc/release-notes/11865-suppress-host-dataverse-field.md

@@ -0,0 +1,3 @@
+### Suppression of the Host Dataverse field
+
+When creating a dataset, the _host dataverse_ field is not shown when the user can only add datasets to one collection.

doc/release-notes/11921-dataset-version-summaries-metadatacount-fix.md

@@ -0,0 +1,10 @@
+### Dataset version summaries API changedFileMetaData count fix
+
+The endpoint ``{id}/versions/compareSummary`` was previously returning an incorrect count for
+the ``changedFileMetaData`` field.
+The logic for calculating this count has been fixed to accurately reflect the total number of file metadata changes
+across all files in the dataset version.
+
+### Related issues
+
+- https://github.com/IQSS/dataverse/issues/11921

doc/sphinx-guides/source/admin/integrations.rst

@@ -240,11 +240,6 @@ Discoverability
 
 A number of builtin features related to data discovery are listed under :doc:`discoverability` but you can further increase the discoverability of your data by setting up integrations.
 
-SHARE
-+++++
-
-`SHARE <http://www.share-research.org>`_ is building a free, open, data set about research and scholarly activities across their life cycle. It's possible to add a Dataverse installation as one of the `sources <https://share.osf.io/sources>`_ they include if you contact the SHARE team.
-
 Geodisy
 +++++++
 

doc/sphinx-guides/source/api/native-api.rst

@@ -4208,24 +4208,24 @@ Delete files from a dataset. This API call allows you to delete multiple files f
 
   curl -H "X-Dataverse-key:$API_TOKEN" -X PUT "$SERVER_URL/api/datasets/:persistentId/deleteFiles?persistentId=$PERSISTENT_IDENTIFIER" \
   -H "Content-Type: application/json" \
-  -d '{"fileIds": [1, 2, 3]}'
+  -d '[1, 2, 3]'
 
 The fully expanded example above (without environment variables) looks like this:
 
 .. code-block:: bash
 
   curl -H "X-Dataverse-key:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" -X PUT "https://demo.dataverse.org/api/datasets/:persistentId/deleteFiles?persistentId=doi:10.5072/FK2ABCDEF" \
   -H "Content-Type: application/json" \
-  -d '{"fileIds": [1, 2, 3]}'
+  -d '[1, 2, 3]'
 
-The ``fileIds`` in the JSON payload should be an array of file IDs that you want to delete from the dataset.
+The JSON payload should be an array of file IDs that you want to delete from the dataset.
 
 You must have the appropriate permissions to delete files from the dataset.
 
 Upon success, the API will return a JSON response with a success message and the number of files deleted.
 
 The API call will report a 400 (BAD REQUEST) error if any of the files specified do not exist or are not in the latest version of the specified dataset.
-The ``fileIds`` in the JSON payload should be an array of file IDs that you want to delete from the dataset.
+The JSON payload should be an array of file IDs that you want to delete from the dataset.
 
 .. _api-dataset-role-assignment-history:
 
@@ -4357,6 +4357,53 @@ The CSV response for this call is the same as for the /api/datasets/{id}/assignm
 
 Note: This feature requires the "role-assignment-history" feature flag to be enabled (see :ref:`feature-flags`).
 
+Update Dataset License
+~~~~~~~~~~~~~~~~~~~~~~
+
+Updates the license of a dataset by applying it to the draft version, or by creating a draft if none exists.
+
+The JSON representation of a license can take two forms, depending on whether you want to specify a predefined license or define custom terms of use and access.
+
+To set a predefined license (e.g., CC BY 4.0), provide a JSON body with the license name:
+
+.. code-block:: json
+
+  {
+    "name": "CC BY 4.0"
+  }
+
+To define custom terms of use and access, provide a JSON body with the following properties. All fields within ``customTerms`` are optional, except for the ``termsOfUse`` field, which is required:
+
+.. code-block:: json
+
+  {
+    "customTerms": {
+        "termsOfUse": "Your terms of use",
+        "confidentialityDeclaration": "Your confidentiality declaration",
+        "specialPermissions": "Your special permissions",
+        "restrictions": "Your restrictions",
+        "citationRequirements": "Your citation requirements",
+        "depositorRequirements": "Your depositor requirements",
+        "conditions": "Your conditions",
+        "disclaimer": "Your disclaimer"
+    }
+  }
+
+.. code-block:: bash
+
+  export API_TOKEN=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
+  export SERVER_URL=https://demo.dataverse.org
+  export ID=3
+  export FILE_PATH=license.json
+
+  curl -H "X-Dataverse-key:$API_TOKEN" -X PUT "$SERVER_URL/api/datasets/$ID/license" -H "Content-type:application/json" --upload-file $FILE_PATH
+
+The fully expanded example above (without environment variables) looks like this:
+
+.. code-block:: bash
+
+  curl -H "X-Dataverse-key:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" -X PUT "https://demo.dataverse.org/api/datasets/3/license" -H "Content-type:application/json" --upload-file license.json
+
 Files
 -----
 

doc/sphinx-guides/source/container/dev-usage.rst

@@ -145,13 +145,13 @@ Accessing Harvesting Log Files
 
 \1. Open a terminal and access the Dataverse container.
 
-Run the following command to access the Dataverse container (assuming your container is named dataverse-1):
+Run the following command to access the Dataverse container:
 
 .. code-block::
 
-  docker exec -it dataverse-1 bash
+  docker exec -it dev_dataverse bash
 
-This command opens an interactive shell within the dataverse-1 container.
+This command opens an interactive shell within the dev_dataverse container.
 
 \2. Navigate to the log files directory.
 
@@ -233,6 +233,13 @@ Hotswapping methods requires using JDWP (Debug Mode), but does not allow switchi
        **Requires IntelliJ Ultimate!**
        (Note that `free educational licenses <https://www.jetbrains.com/community/education/>`_ are available)
 
+       Go to settings, then plugins. Install "Payara Ultimate Tools". For more information:
+
+       - `plugin homepage <https://plugins.jetbrains.com/plugin/15114-payara-ultimate-tools>`_
+       - `docs <https://docs.payara.fish/community/docs/Technical%20Documentation/Ecosystem/IDE%20Integration/IntelliJ%20Plugin/Overview.html>`_
+       - `source <https://github.com/payara/ecosystem-intellij-plugin>`_
+       - `issues <https://github.com/payara/ecosystem-support>`_
+
        .. image:: img/intellij-payara-plugin-install.png
 
 #. Configure a connection to Payara:
@@ -284,6 +291,7 @@ Hotswapping methods requires using JDWP (Debug Mode), but does not allow switchi
 
         You might want to tweak the hot deploy behavior in the "Server" tab now.
         "Update action" can be found in the run window (see below).
+        By default it is "Hot Swap classes", which works fine, but as the screenshot shows you can also change it to "Redeploy".
         "Frame deactivation" means switching from IntelliJ window to something else, e.g. your browser.
         *Note: static resources like properties, XHTML etc will only update when redeploying!*
 
@@ -305,7 +313,11 @@ Hotswapping methods requires using JDWP (Debug Mode), but does not allow switchi
         See cheat sheet above for more options.
         Note that this command either assumes you built the :doc:`app-image` first or will download it from Docker Hub.
      .. group-tab:: IntelliJ
-        You can create a service configuration to automatically start services for you.
+        Note that you can skip this step if you're ok running the command under the "Maven" tab, which is this:
+
+        ``mvn -Pct docker:run -Dapp.skipDeploy``
+
+        In IntelliJ you can create a service configuration to automatically start services for you.
 
         **IMPORTANT**: This requires installation of the `Docker plugin <https://plugins.jetbrains.com/plugin/7724-docker>`_.
 
@@ -362,7 +374,7 @@ Hotswapping methods requires using JDWP (Debug Mode), but does not allow switchi
 
         .. image:: img/intellij-payara-run-output.png
 
-        Manually hotswap classes in "Debug" mode via "Run" > "Debugging Actions" > "Reload Changed Classes".
+        Manually hotswap classes in "Debug" mode via "Run" > "Debugging Actions" > "Compile and Reload Modified Files".
 
         .. image:: img/intellij-payara-run-menu-reload.png
 

doc/sphinx-guides/source/developers/making-library-releases.rst

@@ -13,7 +13,9 @@ Note: See :doc:`making-releases` for Dataverse itself.
 We release Java libraries to Maven Central that are used by Dataverse (and perhaps `other <https://github.com/gdcc/xoai/issues/141>`_ `software <https://github.com/gdcc/xoai/issues/170>`_!):
 
 - https://central.sonatype.com/namespace/org.dataverse
+- https://central.sonatype.com/namespace/org.dataverse.test
 - https://central.sonatype.com/namespace/io.gdcc
+- https://central.sonatype.com/namespace/io.gdcc.export
 
 We release JavaScript/TypeScript libraries to npm:
 
@@ -109,60 +111,18 @@ Releasing a New Library to Maven Central
 At a high level:
 
 - Start with a snapshot release.
-- Use an existing pom.xml as a starting point.
-- Use existing GitHub Actions workflows as a starting point.
-- Create secrets in the new library's GitHub repo used by the workflow.
-- If you need an entire new namespace, look at previous issues such as https://issues.sonatype.org/browse/OSSRH-94575 and https://issues.sonatype.org/browse/OSSRH-94577
-
-Updating pom.xml for a Snapshot Release
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Before publishing a final version to Maven Central, you should publish a snapshot release or two. For each snapshot release you publish, the jar name will be unique each time (e.g. ``foobar-0.0.1-20240430.175110-3.jar``), so you can safely publish over and over with the same version number.
-
-We use the `Nexus Staging Maven Plugin <https://github.com/sonatype/nexus-maven-plugins/blob/main/staging/maven-plugin/README.md>`_ to push snapshot releases to https://s01.oss.sonatype.org/content/groups/staging/io/gdcc/ and https://s01.oss.sonatype.org/content/groups/staging/org/dataverse/
-
-Add the following to your pom.xml:
-
-.. code-block:: xml
+- Use an existing pom.xml as a starting point, such as from `Croissant <https://github.com/gdcc/exporter-croissant>`_, that inherits from the common Maven parent (https://github.com/gdcc/maven-parent). You can also play around with the "hello" project (https://github.com/gdcc/hello) and even make releases from it since it is designed to be a sandbox for publishing to Maven Central.
+- Use existing GitHub Actions workflows as a starting point, such as from `Croissant <https://github.com/gdcc/exporter-croissant>`_. As of this writing we have separate actions for ``maven-snapshot.yml`` and ``maven-release.yml``.
+- For repos under https://github.com/IQSS, create secrets in the new library's GitHub repo used by the workflow. This is necessary for the IQSS org because "organization secrets are not available for organizations on legacy per-repository billing plans." For repos under https://github.com/gdcc you can make use of shared secrets at the org level. These are the environment variables we use:
 
-    <version>0.0.1-SNAPSHOT</version>
+  - DATAVERSEBOT_GPG_KEY
 
-    <distributionManagement>
-        <snapshotRepository>
-            <id>ossrh</id>
-            <url>https://s01.oss.sonatype.org/content/repositories/snapshots</url>
-        </snapshotRepository>
-        <repository>
-            <id>ossrh</id>
-            <url>https://s01.oss.sonatype.org/service/local/staging/deploy/maven2/</url>
-        </repository>
-    </distributionManagement>
+  - DATAVERSEBOT_GPG_PASSWORD
 
-   <plugin>
-       <groupId>org.sonatype.plugins</groupId>
-       <artifactId>nexus-staging-maven-plugin</artifactId>
-       <version>${nexus-staging.version}</version>
-       <extensions>true</extensions>
-       <configuration>
-           <serverId>ossrh</serverId>
-           <nexusUrl>https://s01.oss.sonatype.org</nexusUrl>
-           <autoReleaseAfterClose>true</autoReleaseAfterClose>
-       </configuration>
-   </plugin>
+  - DATAVERSEBOT_SONATYPE_TOKEN
 
-Configuring Secrets
-~~~~~~~~~~~~~~~~~~~
-
-In GitHub, you will likely need to configure the following secrets:
-
-- DATAVERSEBOT_GPG_KEY
-- DATAVERSEBOT_GPG_PASSWORD
-- DATAVERSEBOT_SONATYPE_TOKEN
-- DATAVERSEBOT_SONATYPE_USERNAME
-
-Note that some of these secrets might be configured at the org level (e.g. gdcc or IQSS).
-
-Many of the automated tasks are performed by the dataversebot account on GitHub: https://github.com/dataversebot
+  - DATAVERSEBOT_SONATYPE_USERNAME
+- If you need an entire new namespace, look at previous issues such as https://issues.sonatype.org/browse/OSSRH-94575 and https://issues.sonatype.org/browse/OSSRH-94577
 
 npm (JavaScript/TypeScript)
 ---------------------------

doc/sphinx-guides/source/developers/tips.rst

@@ -124,7 +124,7 @@ Here's an example of using these credentials from within the PostgreSQL containe
 
 .. code-block:: bash
 
-    pdurbin@beamish dataverse % docker exec -it postgres-1 bash
+    pdurbin@beamish dataverse % docker exec -it dev_postgres bash
     root@postgres:/# export PGPASSWORD=secret
     root@postgres:/# psql -h localhost -U dataverse dataverse
     psql (16.3 (Debian 16.3-1.pgdg120+1))