Merge branch 'develop' into metadataLanguage-API-call

Author: akos326

doc/release-notes/11865-suppress-host-dataverse-field.md

@@ -0,0 +1,3 @@
+### Suppression of the Host Dataverse field
+
+When creating a dataset, the _host dataverse_ field is not shown when the user can only add datasets to one collection.

doc/sphinx-guides/source/container/dev-usage.rst

@@ -145,13 +145,13 @@ Accessing Harvesting Log Files
 
 \1. Open a terminal and access the Dataverse container.
 
-Run the following command to access the Dataverse container (assuming your container is named dataverse-1):
+Run the following command to access the Dataverse container:
 
 .. code-block::
 
-  docker exec -it dataverse-1 bash
+  docker exec -it dev_dataverse bash
 
-This command opens an interactive shell within the dataverse-1 container.
+This command opens an interactive shell within the dev_dataverse container.
 
 \2. Navigate to the log files directory.
 
@@ -233,6 +233,13 @@ Hotswapping methods requires using JDWP (Debug Mode), but does not allow switchi
        **Requires IntelliJ Ultimate!**
        (Note that `free educational licenses <https://www.jetbrains.com/community/education/>`_ are available)
 
+       Go to settings, then plugins. Install "Payara Ultimate Tools". For more information:
+
+       - `plugin homepage <https://plugins.jetbrains.com/plugin/15114-payara-ultimate-tools>`_
+       - `docs <https://docs.payara.fish/community/docs/Technical%20Documentation/Ecosystem/IDE%20Integration/IntelliJ%20Plugin/Overview.html>`_
+       - `source <https://github.com/payara/ecosystem-intellij-plugin>`_
+       - `issues <https://github.com/payara/ecosystem-support>`_
+
        .. image:: img/intellij-payara-plugin-install.png
 
 #. Configure a connection to Payara:
@@ -284,6 +291,7 @@ Hotswapping methods requires using JDWP (Debug Mode), but does not allow switchi
 
         You might want to tweak the hot deploy behavior in the "Server" tab now.
         "Update action" can be found in the run window (see below).
+        By default it is "Hot Swap classes", which works fine, but as the screenshot shows you can also change it to "Redeploy".
         "Frame deactivation" means switching from IntelliJ window to something else, e.g. your browser.
         *Note: static resources like properties, XHTML etc will only update when redeploying!*
 
@@ -305,7 +313,11 @@ Hotswapping methods requires using JDWP (Debug Mode), but does not allow switchi
         See cheat sheet above for more options.
         Note that this command either assumes you built the :doc:`app-image` first or will download it from Docker Hub.
      .. group-tab:: IntelliJ
-        You can create a service configuration to automatically start services for you.
+        Note that you can skip this step if you're ok running the command under the "Maven" tab, which is this:
+
+        ``mvn -Pct docker:run -Dapp.skipDeploy``
+
+        In IntelliJ you can create a service configuration to automatically start services for you.
 
         **IMPORTANT**: This requires installation of the `Docker plugin <https://plugins.jetbrains.com/plugin/7724-docker>`_.
 
@@ -362,7 +374,7 @@ Hotswapping methods requires using JDWP (Debug Mode), but does not allow switchi
 
         .. image:: img/intellij-payara-run-output.png
 
-        Manually hotswap classes in "Debug" mode via "Run" > "Debugging Actions" > "Reload Changed Classes".
+        Manually hotswap classes in "Debug" mode via "Run" > "Debugging Actions" > "Compile and Reload Modified Files".
 
         .. image:: img/intellij-payara-run-menu-reload.png
 

doc/sphinx-guides/source/developers/tips.rst

@@ -124,7 +124,7 @@ Here's an example of using these credentials from within the PostgreSQL containe
 
 .. code-block:: bash
 
-    pdurbin@beamish dataverse % docker exec -it postgres-1 bash
+    pdurbin@beamish dataverse % docker exec -it dev_postgres bash
     root@postgres:/# export PGPASSWORD=secret
     root@postgres:/# psql -h localhost -U dataverse dataverse
     psql (16.3 (Debian 16.3-1.pgdg120+1))

doc/sphinx-guides/source/installation/config.rst

@@ -115,6 +115,23 @@ See the :ref:`payara` section of :doc:`prerequisites` for details and init scrip
 
 Related to this is that you should remove ``/root/.payara/pass`` to ensure that Payara isn't ever accidentally started as root. Without the password, Payara won't be able to start as root, which is a good thing.
 
+.. _payara-ports-localhost-only:
+
+Restricting Payara's Ports to localhost
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+In the recommended setup of Dataverse, you do not expose Payara's ports directly to the Internet. Rather, you front Payara with a proxy such as Apache.
+
+If you are running Payara and your proxy on the same server, we recommend having Payara listen only to localhost, which is how your proxy talks to it, with the following command:
+
+``./asadmin set server-config.network-config.network-listeners.network-listener.http-listener-1.address=127.0.0.1``
+
+(You should **NOT** use the configuration option above if you are running in a load-balanced environment, or otherwise have your proxy on a different host than Payara.)
+
+To test that Payara is now only listening on localhost, try hitting port 8080 from the Internet. Payara should not respond.
+
+See also :ref:`network-ports`.
+
 .. _secure-password-storage:
 
 Secure Password Storage
@@ -246,6 +263,8 @@ If you are running an installation with Apache and Payara on the same server, an
 
 You should **NOT** use the configuration option above if you are running in a load-balanced environment, or otherwise have the web server on a different host than the application server.
 
+This security tip is also mentioned at :ref:`payara-ports-localhost-only`.
+
 .. _root-collection-permissions:
 
 Root Dataverse Collection Permissions

pom.xml

@@ -1266,6 +1266,7 @@
                                 </image>
                             </images>
                             <autoCreateCustomNetworks>true</autoCreateCustomNetworks>
+                            <containerNamePattern>%a</containerNamePattern>
                         </configuration>
                     </plugin>
                     <plugin>

src/main/java/edu/harvard/iq/dataverse/DatasetPage.java

@@ -1,5 +1,8 @@
 package edu.harvard.iq.dataverse;
 
+import edu.harvard.iq.dataverse.authorization.DataverseRole;
+import edu.harvard.iq.dataverse.engine.command.DataverseRequest;
+import edu.harvard.iq.dataverse.globus.Permissions;
 import edu.harvard.iq.dataverse.provenance.ProvPopupFragmentBean;
 import edu.harvard.iq.dataverse.api.AbstractApiBean;
 import edu.harvard.iq.dataverse.authorization.AuthenticationServiceBean;
@@ -332,6 +335,7 @@ public enum DisplayMode {
     private List<SelectItem> linkingDVSelectItems;
     private Dataverse linkingDataverse;
     private Dataverse selectedHostDataverse;
+    private Boolean hasDataversesToChoose;
 
     public Dataverse getSelectedHostDataverse() {
         return selectedHostDataverse;
@@ -1781,6 +1785,22 @@ public void setDataverseTemplates(List<Template> dataverseTemplates) {
         this.dataverseTemplates = dataverseTemplates;
     }
 
+    public boolean isHasDataversesToChoose() {
+
+        if (this.hasDataversesToChoose == null) {
+            var user = this.session.getUser();
+            if (!user.isAuthenticated()) {
+                this.hasDataversesToChoose = false;
+            } else {
+                var req = dvRequestService.getDataverseRequest();
+                var permissionBit = 1 << Permission.AddDataset.ordinal();
+                var authenticatedUser = (AuthenticatedUser) user;
+                this.hasDataversesToChoose = permissionService.hasMultiplePermittedCollections(req, authenticatedUser, permissionBit);
+            }
+        }
+        return this.hasDataversesToChoose;
+    }
+
     public Template getDefaultTemplate() {
         return defaultTemplate;
     }
@@ -2118,6 +2138,10 @@ private String init(boolean initFull) {
 
             if (retrieveDatasetVersionResponse != null && !retrieveDatasetVersionResponse.wasRequestedVersionRetrieved()) {
                 //msg("checkit " + retrieveDatasetVersionResponse.getDifferentVersionMessage());
+                if ("DRAFT".equals(version)) {
+                    // redirect to the latest published instead:
+                    return "/dataset.xhtml?persistentId=" + dataset.getGlobalId().asString() + "&faces-redirect=true";
+                }
                 JsfHelper.addWarningMessage(retrieveDatasetVersionResponse.getDifferentVersionMessage());//BundleUtil.getStringFromBundle("dataset.message.metadataSuccess"));
             }
 

src/main/java/edu/harvard/iq/dataverse/DataverseConverter.java

@@ -13,19 +13,34 @@
 import jakarta.faces.convert.Converter;
 import jakarta.faces.convert.FacesConverter;
 
+import java.util.logging.Logger;
+
 /**
  *
  * @author skraffmiller
  */
 @FacesConverter("dataverseConverter")
 public class DataverseConverter implements Converter {
+    private static final Logger logger = Logger.getLogger(DataverseConverter.class.getCanonicalName());
+
 
     //@EJB
     DataverseServiceBean dataverseService = CDI.current().select(DataverseServiceBean.class).get();
 
     @Override
     public Object getAsObject(FacesContext facesContext, UIComponent component, String submittedValue) {
-        return dataverseService.find(new Long(submittedValue));
+        if (submittedValue == null || !submittedValue.matches("[0-9]+")) {
+            logger.fine("Submitted value is not a host dataverse number but: " + submittedValue);
+            return CDI.current().select(DatasetPage.class).get().getSelectedHostDataverse();
+        }
+        else {
+            try {
+                return dataverseService.find(Long.parseLong(submittedValue));
+            } catch (NumberFormatException e) {
+                logger.warning("Submitted value is out of range for a Long: " + submittedValue);
+                return CDI.current().select(DatasetPage.class).get().getSelectedHostDataverse();
+            }
+        }
         //return dataverseService.findByAlias(submittedValue);
     }
 

src/main/java/edu/harvard/iq/dataverse/PermissionServiceBean.java

@@ -953,96 +953,90 @@ public List<Dataverse> findPermittedCollections(DataverseRequest request, Authen
 
     public List<Dataverse> findPermittedCollections(DataverseRequest request, AuthenticatedUser user, int permissionBit, String searchTerm) {
         if (user != null) {
-            // IP Group - Only check IP if a User is calling for themself
-            String ipRangeSQL = "FALSE";
-            if (request != null
-                    && request.getAuthenticatedUser() != null
-                    && !request.getAuthenticatedUser().isSuperuser()
-                    && request.getSourceAddress() != null
-                    && request.getAuthenticatedUser().getUserIdentifier().equalsIgnoreCase(user.getUserIdentifier())) {
-                IpAddress ip = request.getSourceAddress();
-                if (ip instanceof IPv4Address) {
-                    IPv4Address ipv4 = (IPv4Address) ip;
-                    ipRangeSQL = ipv4.toBigInteger() + " BETWEEN ipv4range.bottomaslong AND ipv4range.topaslong";
-                } else if (ip instanceof IPv6Address) {
-                    IPv6Address ipv6 = (IPv6Address) ip;
-                    long[] vals = ipv6.toLongArray();
-                    if (vals.length == 4) {
-                        ipRangeSQL = """
-                                (@0 BETWEEN ipv6range.bottoma AND ipv6range.topa
-                                AND @1 BETWEEN ipv6range.bottomb AND ipv6range.topb
-                                AND @2 BETWEEN ipv6range.bottomc AND ipv6range.topc
-                                AND @3 BETWEEN ipv6range.bottomd AND ipv6range.topd)
-                                """;
-                        for (int i = 0; i < vals.length; i++) {
-                            ipRangeSQL = ipRangeSQL.replace("@" + i, String.valueOf(vals[i]));
-                        }
-                    }
-                }
+            var sqlCode = getBaseQueryForAllPermittedDataverses(request, user, permissionBit);
+            if (searchTerm == null || searchTerm.isEmpty()) {
+                return em.createNativeQuery(sqlCode, Dataverse.class).getResultList();
+            } else if (user.isSuperuser()) {
+                Query query = em.createNativeQuery(sqlCode.concat(WHERE).concat(SEARCH_PARAMS), Dataverse.class);
+                setSearchParamValues(searchTerm, query);
+                return query.getResultList();
             }
-            String sqlCode;
-            if (user.isSuperuser()) {
-                sqlCode = LIST_ALL_DATAVERSES_SUPERUSER_HAS_PERMISSION;
-
-                if (searchTerm == null || searchTerm.isEmpty()) {
-                    return em.createNativeQuery(sqlCode, Dataverse.class).getResultList();
-                } else {
-                    sqlCode = LIST_ALL_DATAVERSES_SUPERUSER_HAS_PERMISSION.concat(WHERE).concat(SEARCH_PARAMS);
+            else {
+                Query query = em.createNativeQuery(sqlCode.concat(AND).concat(SEARCH_PARAMS), Dataverse.class);
+                setSearchParamValues(searchTerm, query);
+                return query.getResultList();
+            }
+        }
+        return null;
+    }
 
-                    String pattern = searchTerm.toLowerCase();
-                    String pattern1 = pattern + "%";
-                    String pattern2 = "% " + pattern + "%";
+    public boolean hasMultiplePermittedCollections(DataverseRequest request, AuthenticatedUser user, int permissionBit) {
+        if (user != null) {
+            var sqlCode = getBaseQueryForAllPermittedDataverses(request, user, permissionBit) + " LIMIT 2";
+            var resultList = em.createNativeQuery(sqlCode, Dataverse.class).getResultList();
+            return resultList.size() > 1;
+        }
+        return false;
+    }
 
-                    // Adjust the queries for very short, 1 
-                    if (pattern.length() == 1) {
-                        pattern1 = pattern;
-                        pattern2 = pattern + " %";
-                    }
-                    Query query = em.createNativeQuery(sqlCode, Dataverse.class);
-                    query.setParameter(1, "%dataverse");
-                    query.setParameter(2, pattern1);
-                    query.setParameter(3, pattern2);
-                    query.setParameter(4, "%dataverse");
-                    query.setParameter(5, pattern1);
-                    query.setParameter(6, pattern2);
-                    return query.getResultList();
+    private String getBaseQueryForAllPermittedDataverses(DataverseRequest request, AuthenticatedUser user, int permissionBit) {
+        if (user.isSuperuser()) {
+            return LIST_ALL_DATAVERSES_SUPERUSER_HAS_PERMISSION;
+        } else {
+            return LIST_ALL_DATAVERSES_USER_HAS_PERMISSION
+                .replace("@USERID", String.valueOf(user.getId()))
+                .replace("@PERMISSIONBIT", String.valueOf(permissionBit))
+                .replace("@IPRANGESQL", getIpRange(request, user));
+        }
+    }
 
-                }
-            } else {
-                if (searchTerm == null || searchTerm.isEmpty()) {
-                    sqlCode = LIST_ALL_DATAVERSES_USER_HAS_PERMISSION
-                            .replace("@USERID", String.valueOf(user.getId()))
-                            .replace("@PERMISSIONBIT", String.valueOf(permissionBit))
-                            .replace("@IPRANGESQL", ipRangeSQL);
-                    return em.createNativeQuery(sqlCode, Dataverse.class).getResultList();
-                } else {
-                    String pattern = searchTerm.toLowerCase();
-                    String pattern1 = pattern + "%";
-                    String pattern2 = "% " + pattern + "%";
-
-                    // Adjust the queries for very short, 1 
-                    if (pattern.length() == 1) {
-                        pattern1 = pattern;
-                        pattern2 = pattern + " %";
+    private String getIpRange(DataverseRequest request, AuthenticatedUser user) {
+        // IP Group - Only check IP if a User is calling for themself
+        String ipRangeSQL = "FALSE";
+        if (request != null
+            && request.getAuthenticatedUser() != null
+            && !request.getAuthenticatedUser().isSuperuser()
+            && request.getSourceAddress() != null
+            && request.getAuthenticatedUser().getUserIdentifier().equalsIgnoreCase(user.getUserIdentifier())) {
+            IpAddress ip = request.getSourceAddress();
+            if (ip instanceof IPv4Address) {
+                IPv4Address ipv4 = (IPv4Address) ip;
+                ipRangeSQL = ipv4.toBigInteger() + " BETWEEN ipv4range.bottomaslong AND ipv4range.topaslong";
+            } else if (ip instanceof IPv6Address) {
+                IPv6Address ipv6 = (IPv6Address) ip;
+                long[] vals = ipv6.toLongArray();
+                if (vals.length == 4) {
+                    ipRangeSQL = """
+                            (@0 BETWEEN ipv6range.bottoma AND ipv6range.topa
+                            AND @1 BETWEEN ipv6range.bottomb AND ipv6range.topb
+                            AND @2 BETWEEN ipv6range.bottomc AND ipv6range.topc
+                            AND @3 BETWEEN ipv6range.bottomd AND ipv6range.topd)
+                            """;
+                    for (int i = 0; i < vals.length; i++) {
+                        ipRangeSQL = ipRangeSQL.replace("@" + i, String.valueOf(vals[i]));
                     }
-
-                    sqlCode = LIST_ALL_DATAVERSES_USER_HAS_PERMISSION.concat(AND).concat(SEARCH_PARAMS)
-                            .replace("@USERID", String.valueOf(user.getId()))
-                            .replace("@PERMISSIONBIT", String.valueOf(permissionBit))
-                            .replace("@IPRANGESQL", ipRangeSQL);
-                    
-                    Query query = em.createNativeQuery(sqlCode, Dataverse.class);
-                    query.setParameter(1, "%dataverse");
-                    query.setParameter(2, pattern1);
-                    query.setParameter(3, pattern2);
-                    query.setParameter(4, "%dataverse");
-                    query.setParameter(5, pattern1);
-                    query.setParameter(6, pattern2);
-                    return query.getResultList();
                 }
             }
         }
-        return null;
+        return ipRangeSQL;
+    }
+
+    private void setSearchParamValues(String searchTerm, Query query) {
+        String pattern = searchTerm.toLowerCase();
+        String pattern1 = pattern + "%";
+        String pattern2 = "% " + pattern + "%";
+
+        // Adjust the queries for very short, 1
+        if (pattern.length() == 1) {
+            pattern1 = pattern;
+            pattern2 = pattern + " %";
+        }
+        query.setParameter(1, "%dataverse");
+        query.setParameter(2, pattern1);
+        query.setParameter(3, pattern2);
+        query.setParameter(4, "%dataverse");
+        query.setParameter(5, pattern1);
+        query.setParameter(6, pattern2);
     }
 
     /**