Eureka (#413)

* Upgrade to Hazelcast 4

* Screenshots

* Update text to match screenshot counts

Author: neilstevenson

hazelcast-integration/springboot-eureka-partition-groups/README.md

@@ -616,7 +616,7 @@ Here we do it from the Hazelcast server. If the maps "__eurekast_safe__"
 and "__eurekast_unsafe__" are empty, we put some data in them.
 
 The maps "__eurekast_safe__" and "__eurekast_unsafe__" are 
-[IMap](http://docs.hazelcast.org/docs/3.8/javadoc/com/hazelcast/core/IMap.html),
+[IMap](http://docs.hazelcast.org/docs/4.0/javadoc/com/hazelcast/map/IMap.html),
 meaning they are split into sections and those sections spread across
 the available servers. The default is for 271 such sections, named _partitions_,
 so we create 271 entries to try to put one data record entry into each.
@@ -861,14 +861,12 @@ What you should see is that the "__eurekast_safe__" map has survived
 this event unscathed. There are still 271 entries.
 
 What you should see is that the "__eurekast_unsafe__" map has lost
-some data. In the screenshot only 201 entries remain, so 70 have
+some data. In the screenshot only 196 entries remain, so 75 have
 gone.
 
-You should expect to lost about a quarter of the unsafe (no backup)
-data. Four servers, one is killed.
-
-The exact number lost will depend on how the map partitions have been
-allocated to the servers. This is not random, but difficult to predict.
+This seems intuititve to have lost about a quarter of the unsafe (no backup)
+data. Four servers, one is killed. However, it is not a correct belief
+as we will discover later on.
 
 ### 14. Kill another specific Hazelcast Server
 
@@ -895,8 +893,8 @@ The "__eurekast_safe__" map is still fine, all 271 entries are there.
 We have lost both the `odd` zone servers but the other copy of the data was
 in the `even` zone servers, so no data has been lost.
 
-With two servers gone, the "__eurekast_unsafe__" map will be about half the
-original size. Here it shows 149, you should have a roughly similar number.
+With two servers gone, the "__eurekast_unsafe__" map will be smaller still.
+Here it shows 74, you will probably have a different number.
 
 ### 16. Kill yet another Hazelcast Server
 
@@ -934,9 +932,20 @@ zone available it's better than nothing. Here it has saved us.
 
 Surprised 2 ?
 
-The "__eurekast_unsafe__" has dropped from 149 to 145 by the most recent loss of a server.
-We've lost 75% of the hosts but not 75% of the data, and that's because it's the partitions
-that are spread across the hosts.
+The "__eurekast_unsafe__" dropped from 271 to 196 when a server crashed, then down to 74
+when another crashed, then down to 1 !!
+
+What is going on ?
+
+What is going on is partitioning. Data is organised into groups based on their keys,
+and the groups are spread across the servers. With small data volumes the groups may
+be different sizes. So if you lose a server which is hosting larger than average
+groups, you lost more data.
+
+It's not really a problem. If you have much larger data volumes, millions or billions,
+the deviation in the group sizes becomes almost unnoticable.
+
+And if you're worried about losing data, turn on data safety.
 
 ## Hazelcasts Mancenter
 

hazelcast-integration/springboot-eureka-partition-groups/my-eureka-common/pom.xml

@@ -23,7 +23,13 @@
     <dependencies>
         <dependency>
             <groupId>org.springframework.cloud</groupId>
-            <artifactId>spring-cloud-starter-eureka</artifactId>
+            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
+            <exclusions>
+            	<exclusion>
+            		<groupId>org.springframework.cloud</groupId>
+            		<artifactId>spring-cloud-netflix-ribbon</artifactId>
+            	</exclusion>
+            </exclusions>
         </dependency>
     </dependencies>
 </project>

hazelcast-integration/springboot-eureka-partition-groups/my-eureka-common/src/main/java/com/hazelcast/samples/eureka/partition/groups/MyEurekaDiscoveryService.java

@@ -1,6 +1,6 @@
 package com.hazelcast.samples.eureka.partition.groups;
 
-import com.hazelcast.nio.Address;
+import com.hazelcast.cluster.Address;
 import com.hazelcast.spi.discovery.DiscoveryNode;
 import com.hazelcast.spi.discovery.SimpleDiscoveryNode;
 import com.hazelcast.spi.discovery.integration.DiscoveryService;
@@ -11,11 +11,14 @@
 import org.springframework.cloud.client.discovery.DiscoveryClient;
 import org.springframework.stereotype.Component;
 
+import java.io.IOException;
+import java.net.ServerSocket;
 import java.util.ArrayList;
 import java.util.HashMap;
 import java.util.List;
 import java.util.Map;
 
+import static com.hazelcast.internal.util.EmptyStatement.ignore;
 import static com.hazelcast.spi.partitiongroup.PartitionGroupMetaData.PARTITION_GROUP_ZONE;
 
 /**
@@ -41,6 +44,7 @@ public class MyEurekaDiscoveryService implements DiscoveryService {
 
     private static final String YML_SEPARATOR = ".";
 
+
     @Value("${eureka.client.registerWithEureka:true}")
     public boolean registerWithEureka;
 
@@ -62,8 +66,8 @@ public class MyEurekaDiscoveryService implements DiscoveryService {
      * @return A map with one entry, the partition group for this host.
      */
     @Override
-    public Map<String, Object> discoverLocalMetadata() {
-        HashMap<String, Object> result = new HashMap<>();
+    public Map<String, String> discoverLocalMetadata() {
+        HashMap<String, String> result = new HashMap<>();
 
         /* The metadata is only for partition groups, so we don't need this if we are
          * a Hazelcast client. Hazelcast servers register with Eureka, clients only read.
@@ -76,7 +80,7 @@ public Map<String, Object> discoverLocalMetadata() {
         log.info("discoverLocalMetadata(): Hazelcast lookup to Eureka : start");
 
         // Find the web port this process is using.
-        String port = String.valueOf(discoveryClient.getLocalServiceInstance().getPort());
+        String port = this.findPort(System.getProperty("my.web.port"));
         // Since this is a one machine example, we know which machine we are on.
         String hostPort = "localhost" + YML_SEPARATOR + port;
 
@@ -174,4 +178,36 @@ public void start() {
     @Override
     public void destroy() {
     }
+
+    /**
+     * We need to know the web port this process will use, but this module
+     * is triggered before Spring decides. Using the base port (as a String)
+     * determine which port is free, assuming Spring picks it as next in
+     * sequence.
+     *
+     * @param Starting port
+     * @return Next free port, starting port if free or greater
+     */
+    private String findPort(String startingPort) {
+        try {
+            int startPort = Integer.parseInt(startingPort);
+            int freePort = 0;
+
+            for (int port = startPort; port < startPort + 100; port++) {
+                try {
+                    new ServerSocket(port).close();
+                    freePort = port;
+                    break;
+                } catch (IOException portInUse) {
+                    ignore(portInUse);
+                }
+            }
+
+            return String.valueOf(freePort);
+
+        } catch (NumberFormatException nfe) {
+            log.error("Not a number, starting port '{}'", startingPort);
+        }
+        return "";
+    }
 }

hazelcast-integration/springboot-eureka-partition-groups/my-eureka-server/.springBeans

@@ -40,6 +40,12 @@
         <dependency>
             <groupId>org.springframework.cloud</groupId>
             <artifactId>spring-cloud-netflix-eureka-server</artifactId>
+            <exclusions>
+            	<exclusion>
+            		<groupId>org.springframework.cloud</groupId>
+            		<artifactId>spring-cloud-starter-netflix-ribbon</artifactId>
+            	</exclusion>
+            </exclusions>
         </dependency>
     </dependencies>
 

hazelcast-integration/springboot-eureka-partition-groups/my-eureka-server/pom.xml

@@ -25,20 +25,6 @@
             <groupId>${project.groupId}</groupId>
             <artifactId>my-eureka-common</artifactId>
             <version>${project.version}</version>
-            <exclusions>
-                <exclusion>
-                    <groupId>com.hazelcast</groupId>
-                    <artifactId>hazelcast</artifactId>
-                </exclusion>
-                <exclusion>
-                    <groupId>com.hazelcast</groupId>
-                    <artifactId>hazelcast-all</artifactId>
-                </exclusion>
-                <exclusion>
-                    <groupId>com.hazelcast</groupId>
-                    <artifactId>hazelcast-client</artifactId>
-                </exclusion>
-            </exclusions>
         </dependency>
 
         <dependency>

hazelcast-integration/springboot-eureka-partition-groups/my-hazelcast-client/pom.xml

@@ -44,6 +44,9 @@ public class MyConfiguration {
     public ClientConfig clientConfig(DiscoveryServiceProvider discoveryServiceProvider) {
         ClientConfig clientConfig = new ClientConfig();
 
+        // Naming
+        clientConfig.setClusterName(Constants.CLUSTER_NAME);
+
         // Discovery
         clientConfig.setProperty("hazelcast.discovery.enabled", Boolean.TRUE.toString());
         clientConfig.getNetworkConfig().getDiscoveryConfig().setDiscoveryServiceProvider(discoveryServiceProvider);

hazelcast-integration/springboot-eureka-partition-groups/my-hazelcast-client/src/main/java/com/hazelcast/samples/eureka/partition/groups/MyConfiguration.java

@@ -15,6 +15,8 @@
     <description>Hazelcast Server</description>
 
     <properties>
+        <hazelcast.version>4.0</hazelcast.version>
+
         <!-- needed for checkstyle/findbugs -->
         <main.basedir>${project.parent.parent.parent.basedir}</main.basedir>
         <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
@@ -25,20 +27,6 @@
             <groupId>${project.groupId}</groupId>
             <artifactId>my-eureka-common</artifactId>
             <version>${project.version}</version>
-            <exclusions>
-                <exclusion>
-                    <groupId>com.hazelcast</groupId>
-                    <artifactId>hazelcast</artifactId>
-                </exclusion>
-                <exclusion>
-                    <groupId>com.hazelcast</groupId>
-                    <artifactId>hazelcast-all</artifactId>
-                </exclusion>
-                <exclusion>
-                    <groupId>com.hazelcast</groupId>
-                    <artifactId>hazelcast-client</artifactId>
-                </exclusion>
-            </exclusions>
         </dependency>
 
         <dependency>
@@ -51,6 +39,10 @@
             <groupId>org.springframework.boot</groupId>
             <artifactId>spring-boot-starter-actuator</artifactId>
         </dependency>
+        <dependency>
+            <groupId>org.springframework.boot</groupId>
+            <artifactId>spring-boot-starter-web</artifactId>
+        </dependency>
     </dependencies>
 
     <build>

hazelcast-integration/springboot-eureka-partition-groups/my-hazelcast-server/pom.xml

@@ -16,13 +16,15 @@
     <description>Project POM</description>
 
     <properties>
+    	<hazelcast.version>4.0</hazelcast.version>
+    
         <!-- needed for checkstyle/findbugs -->
         <main.basedir>${project.parent.parent.basedir}</main.basedir>
         <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
 
         <java.version>1.8</java.version>
-        <spring-boot.version>1.5.2.RELEASE</spring-boot.version>
-        <spring-cloud.version>Camden.SR5</spring-cloud.version>
+        <spring-boot.version>2.2.5.RELEASE</spring-boot.version>
+        <spring-cloud.version>Hoxton.SR3</spring-cloud.version>
     </properties>
 
     <modules>
@@ -36,9 +38,16 @@
     </modules>
 
     <dependencies>
+        <!-- https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-with-Java-9-and-above -->
+        <dependency>
+            <groupId>org.glassfish.jaxb</groupId>
+            <artifactId>jaxb-runtime</artifactId>
+        </dependency>
+
         <dependency>
             <groupId>com.hazelcast</groupId>
-            <artifactId>hazelcast</artifactId>
+            <artifactId>hazelcast-all</artifactId>
+            <version>${hazelcast.version}</version>
         </dependency>
 
         <dependency>