Merge branch 'develop' into 11198-list-file-versions-api

Author: stevenwinship

doc/release-notes/11057-globus-improvements.md

@@ -0,0 +1,11 @@
+## Globus framework improvements
+
+The improvements and optimizations in this release build on top of the earlier work (such as PR #10781). They are based on the experience gained at IQSS as part of the production rollout of the Large Data Storage services that utilizes Globus.
+
+The changes in this PR (#11125) focus on improving Globus *downloads* (i.e., transfers from Dataverse-linked Globus volumes to users' Globus collections). Most importantly, th mechanism of "Asynchronous Task Monitoring", first introduced in #10781 for *uploads*, has been extended to handle downloads as well. This generally makes downloads more reliable (specifically, in how Dataverse manages temporary access rules granted to users, minimizing the risk of consequent downloads failing because of stale access rules left in place).
+
+See `globus-use-experimental-async-framework` under [Feature Flags](https://guides.dataverse.org/en/latest/installation/config.html#feature-flags) and [dataverse.files.globus-monitoring-server](https://guides.dataverse.org/en/latest/installation/config.html#dataverse-files-globus-monitoring-server) in the Installation Guide.
+
+Multiple other improvements have been made making the underlying Globus framework more reliable and robust.
+
+

doc/release-notes/6.2025.2_update.md

@@ -0,0 +1,35 @@
+- Payara application server has been upgraded to version 6.2025.2. See #11126 and #11128.
+
+## Installation
+
+The following assumes the standard instructions to `export PAYARA=/usr/local/payara6`, undeploy v6.5, stop payara have been done already.
+
+### Upgrade to Payara 6.2025.2
+
+The steps below reuse your existing domain directory with the new distribution. You may also want to review the Payara upgrade instructions as it could be helpful during any troubleshooting:
+[Payara Release Notes](https://docs.payara.fish/community/docs/6.2025.2/Release%20Notes/Release%20Notes%206.2025.2.html).
+We also recommend you ensure you followed all update instructions from the past releases regarding Payara.
+(The most recent Payara update was for [v6.3](https://github.com/IQSS/dataverse/releases/tag/v6.3).)
+
+Move the current Payara directory out of the way:
+
+```shell
+mv $PAYARA $PAYARA.6.2024.6
+```
+
+Download the new Payara version 6.2025.2 (from https://www.payara.fish/downloads/payara-platform-community-edition/ or https://nexus.payara.fish/repository/payara-community/fish/payara/distributions/payara/6.2025.2/payara-6.2025.2.zip), and unzip it in its place:
+
+```shell
+cd /usr/local
+unzip payara-6.2025.2.zip
+```
+
+Replace the brand new `payara/glassfish/domains/domain1` with your old, preserved domain1:
+
+```shell
+mv payara6/glassfish/domains/domain1 payara6/glassfish/domains/domain1_DIST
+mv payara6.6.2024.6/glassfish/domains/domain1 payara6/glassfish/domains/
+```
+
+Deploy/install as normal
+

doc/sphinx-guides/source/developers/classic-dev-env.rst

@@ -93,15 +93,15 @@ On Linux, install ``jq`` from your package manager or download a binary from htt
 Install Payara
 ~~~~~~~~~~~~~~
 
-Payara 6.2024.6 or higher is required.
+Payara 6.2025.2 or higher is required.
 
 To install Payara, run the following commands:
 
 ``cd /usr/local``
 
-``sudo curl -O -L https://nexus.payara.fish/repository/payara-community/fish/payara/distributions/payara/6.2024.6/payara-6.2024.6.zip``
+``sudo curl -O -L https://nexus.payara.fish/repository/payara-community/fish/payara/distributions/payara/6.2025.2/payara-6.2025.2.zip``
 
-``sudo unzip payara-6.2024.6.zip``
+``sudo unzip payara-6.2025.2.zip``
 
 ``sudo chown -R $USER /usr/local/payara6``
 

doc/sphinx-guides/source/developers/tips.rst

@@ -296,3 +296,14 @@ with the following code in ``SettingsWrapper.java``:
 A more serious example would be direct calls to PermissionServiceBean methods used in render logic expressions. This is something that has happened and caused some problems in real life. A simple permission service lookup (for example, whether a user is authorized to create a dataset in the current dataverse) can easily take 15 database queries. Repeated multiple times, this can quickly become a measurable delay in rendering the page. PermissionsWrapper must be used exclusively for any such lookups from JSF pages.
 
 See also :doc:`performance`.
+
+JSF1103 Errors
+~~~~~~~~~~~~~~
+
+Errors of the form ``JSF1103: The metadata facet must be a direct child of the view in viewId /dataverse.xhtml`` come from use of the f:metadata tag at the wrong depth in the .xhtml.
+
+Most/all known instances of the problem were corrected in https://github.com/IQSS/dataverse/pull/11128.
+
+Any page that used <ui:composition template="/dataverse_template.xhtml"> was including the f:metadata farther down in the tree rather than as a direct child of the view.
+As of Payara 6.2025.2, it is not clear that this error was resulting in changes to UI behavior, but the error messages were in the log.
+If you see these errors, this note and the examples in the PR will hopefully provide some insight as to how to fix them.

doc/sphinx-guides/source/installation/config.rst

@@ -3501,7 +3501,7 @@ please find all known feature flags below. Any of these flags can be activated u
       - Turns off automatic selection of a dataset thumbnail from image files in that dataset. When set to ``On``, a user can still manually pick a thumbnail image or upload a dedicated thumbnail image.
       - ``Off``
     * - globus-use-experimental-async-framework
-      - Activates a new experimental implementation of Globus polling of ongoing remote data transfers that does not rely on the instance staying up continuously for the duration of the transfers and saves the state information about Globus upload requests in the database. Added in v6.4. Affects :ref:`:GlobusPollingInterval`. Note that the JVM option :ref:`dataverse.files.globus-monitoring-server` described above must also be enabled on one (and only one, in a multi-node installation) Dataverse instance. 
+      - Activates a new experimental implementation of Globus polling of ongoing remote data transfers that does not rely on the instance staying up continuously for the duration of the transfers and saves the state information about Globus upload requests in the database. Added in v6.4; extended in v6.6 to cover download transfers, in addition to uploads. Affects :ref:`:GlobusPollingInterval`. Note that the JVM option :ref:`dataverse.files.globus-monitoring-server` described above must also be enabled on one (and only one, in a multi-node installation) Dataverse instance. 
       - ``Off``
     * - index-harvested-metadata-source
       - Index the nickname or the source name (See the optional ``sourceName`` field in :ref:`create-a-harvesting-client`) of the harvesting client as the "metadata source" of harvested datasets and files. If enabled, the Metadata Source facet will show separate groupings of the content harvested from different sources (by harvesting client nickname or source name) instead of the default behavior where there is one "Harvested" grouping for all harvested content.

doc/sphinx-guides/source/installation/prerequisites.rst

@@ -44,7 +44,7 @@ On RHEL/derivative you can make Java 17 the default with the ``alternatives`` co
 Payara
 ------
 
-Payara 6.2024.6 is recommended. Newer versions might work fine. Regular updates are recommended.
+Payara 6.2025.2 is recommended. Newer versions might work fine. Regular updates are recommended.
 
 Installing Payara
 =================
@@ -55,8 +55,8 @@ Installing Payara
 
 - Download and install Payara (installed in ``/usr/local/payara6`` in the example commands below)::
 
-	# wget https://nexus.payara.fish/repository/payara-community/fish/payara/distributions/payara/6.2024.6/payara-6.2024.6.zip
-	# unzip payara-6.2024.6.zip
+	# wget https://nexus.payara.fish/repository/payara-community/fish/payara/distributions/payara/6.2025.2/payara-6.2025.2.zip
+	# unzip payara-6.2025.2.zip
 	# mv payara6 /usr/local
 
 If nexus.payara.fish is ever down for maintenance, Payara distributions are also available from https://repo1.maven.org/maven2/fish/payara/distributions/payara/

doc/sphinx-guides/source/qa/test-automation.md

@@ -52,7 +52,7 @@ Go to the end of the log and then scroll up, looking for the failure. A failed A
 
 ```
 TASK [dataverse : download payara zip] *****************************************
-fatal: [localhost]: FAILED! => {"changed": false, "dest": "/tmp/payara.zip", "elapsed": 10, "msg": "Request failed: <urlopen error timed out>", "url": "https://nexus.payara.fish/repository/payara-community/fish/payara/distributions/payara/6.2024.6/payara-6.2024.6.zip"}
+fatal: [localhost]: FAILED! => {"changed": false, "dest": "/tmp/payara.zip", "elapsed": 10, "msg": "Request failed: <urlopen error timed out>", "url": "https://nexus.payara.fish/repository/payara-community/fish/payara/distributions/payara/6.2025.2/payara-6.2025.2.zip"}
 ```
 
 In the example above, if Payara can't be downloaded, we're obviously going to have problems deploying Dataverse to it!

modules/dataverse-parent/pom.xml

@@ -148,7 +148,7 @@
         <argLine>-Duser.timezone=${project.timezone} -Dfile.encoding=${project.build.sourceEncoding} -Duser.language=${project.language} -Duser.region=${project.region}</argLine>
 
         <!-- Major system components and dependencies -->
-        <payara.version>6.2024.6</payara.version>
+        <payara.version>6.2025.2</payara.version>
         <postgresql.version>42.7.4</postgresql.version>
         <solr.version>9.8.0</solr.version>
         <aws.version>1.12.748</aws.version>

src/main/java/edu/harvard/iq/dataverse/DatasetPage.java

@@ -5867,13 +5867,12 @@ public List<DatasetField> getDatasetSummaryFields() {
         return DatasetUtil.getDatasetSummaryFields(workingVersion, customFields);
     }
 
-    public boolean isShowPreviewButton(Long fileId) {
-        List<ExternalTool> previewTools = getPreviewToolsForDataFile(fileId);
+    public boolean isShowPreviewButton(DataFile dataFile) {
+        List<ExternalTool> previewTools = getPreviewToolsForDataFile(dataFile);
         return previewTools.size() > 0;
     }
 
-    public boolean isShowQueryButton(Long fileId) { 
-        DataFile dataFile = datafileService.find(fileId);
+    public boolean isShowQueryButton(DataFile dataFile) { 
 
         if(dataFile.isRestricted()
                 || !dataFile.isReleased()
@@ -5882,26 +5881,28 @@ public boolean isShowQueryButton(Long fileId) {
             return false;
         }
 
-        List<ExternalTool> fileQueryTools = getQueryToolsForDataFile(fileId);
+        List<ExternalTool> fileQueryTools = getQueryToolsForDataFile(dataFile);
         return fileQueryTools.size() > 0;
     }
 
-    public List<ExternalTool> getPreviewToolsForDataFile(Long fileId) {
-        return getCachedToolsForDataFile(fileId, ExternalTool.Type.PREVIEW);
+    public List<ExternalTool> getPreviewToolsForDataFile(DataFile dataFile) {
+        return getCachedToolsForDataFile(dataFile, ExternalTool.Type.PREVIEW);
     }
 
-    public List<ExternalTool> getQueryToolsForDataFile(Long fileId) {
-        return getCachedToolsForDataFile(fileId, ExternalTool.Type.QUERY);
+    public List<ExternalTool> getQueryToolsForDataFile(DataFile dataFile) {
+        return getCachedToolsForDataFile(dataFile, ExternalTool.Type.QUERY);
     }
-    public List<ExternalTool> getConfigureToolsForDataFile(Long fileId) {
-        return getCachedToolsForDataFile(fileId, ExternalTool.Type.CONFIGURE);
+    
+    public List<ExternalTool> getConfigureToolsForDataFile(DataFile dataFile) {
+        return getCachedToolsForDataFile(dataFile, ExternalTool.Type.CONFIGURE);
     }
 
-    public List<ExternalTool> getExploreToolsForDataFile(Long fileId) {
-        return getCachedToolsForDataFile(fileId, ExternalTool.Type.EXPLORE);
+    public List<ExternalTool> getExploreToolsForDataFile(DataFile dataFile) {
+        return getCachedToolsForDataFile(dataFile, ExternalTool.Type.EXPLORE);
     }
 
-    public List<ExternalTool> getCachedToolsForDataFile(Long fileId, ExternalTool.Type type) {
+    public List<ExternalTool> getCachedToolsForDataFile(DataFile dataFile, ExternalTool.Type type) {
+        Long fileId = dataFile.getId();
         Map<Long, List<ExternalTool>> cachedToolsByFileId = new HashMap<>();
         List<ExternalTool> externalTools = new ArrayList<>();
         switch (type) {
@@ -5928,7 +5929,6 @@ public List<ExternalTool> getCachedToolsForDataFile(Long fileId, ExternalTool.Ty
         if (cachedTools != null) { //if already queried before and added to list
             return cachedTools;
         }
-        DataFile dataFile = datafileService.find(fileId);
         cachedTools = externalToolService.findExternalToolsByFile(externalTools, dataFile);
         cachedToolsByFileId.put(fileId, cachedTools); //add to map so we don't have to do the lifting again
         return cachedTools;
@@ -6235,7 +6235,7 @@ public String getEffectiveMetadataLanguage(boolean ofParent) {
 
     public String getLocaleDisplayName(String code) {
         String displayName = settingsWrapper.getBaseMetadataLanguageMap(false).get(code);
-        if(displayName==null && !code.equals(DvObjectContainer.UNDEFINED_CODE)) {
+        if(displayName==null && code!=null && !code.equals(DvObjectContainer.UNDEFINED_CODE)) {
             //Default (for cases such as :when a Dataset has a metadatalanguage code but :MetadataLanguages is no longer defined).
             displayName = new Locale(code).getDisplayName();
         }
@@ -6251,6 +6251,11 @@ public List<String> getVocabScripts() {
     }
 
     public String getFieldLanguage(String languages) {
+        //Prevent NPE in Payara 6-2024-12 with CVoc
+        logger.info("Languages: " + languages);
+        if(languages==null) {
+            languages="";
+        }
         return fieldService.getFieldLanguage(languages,session.getLocaleCode());
     }
 
@@ -6723,6 +6728,7 @@ public boolean isGlobusTransferRequested() {
      *                    valid files to transfer.
      */
     public void startGlobusTransfer(boolean transferAll, boolean popupShown) {
+        logger.fine("inside startGlobusTransfer; "+(transferAll ? "transferAll" : "NOTtransferAll") + " " + (popupShown ? "popupShown" : "NOTpopupShown"));
         if (transferAll) {
             this.setSelectedFiles(workingVersion.getFileMetadatas());
         }

src/main/java/edu/harvard/iq/dataverse/DataverseSession.java

@@ -209,19 +209,17 @@ public String getLocaleTitle() {
 
     public void initLocale() {
 
-        if(FacesContext.getCurrentInstance() == null) {
-            localeCode = "en";
-        }
-        else if (FacesContext.getCurrentInstance().getViewRoot() == null ) {
-            localeCode = FacesContext.getCurrentInstance().getExternalContext().getRequestLocale().getLanguage();
-        }
-        else if (FacesContext.getCurrentInstance().getViewRoot().getLocale().getLanguage().equals("en_US")) {
-            localeCode = "en";
-        }
-        else {
-            localeCode = FacesContext.getCurrentInstance().getViewRoot().getLocale().getLanguage();
+
+        localeCode = "en";
+        if (FacesContext.getCurrentInstance() != null) {
+            if (FacesContext.getCurrentInstance().getViewRoot() == null) {
+                localeCode = FacesContext.getCurrentInstance().getExternalContext().getRequestLocale().getLanguage();
+            } else if (FacesContext.getCurrentInstance().getViewRoot().getLocale().getLanguage().equals("en_US")) {
+                localeCode = "en";
+            } else {
+                localeCode = FacesContext.getCurrentInstance().getViewRoot().getLocale().getLanguage();
+            }
         }
-        
         logger.fine("init: locale set to "+localeCode);
     }