Skip to content

增加针对大推送的自适应攒批的能力 #378

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
huanglongchao merged 10 commits into from
Jan 5, 2026
Merged

增加针对大推送的自适应攒批的能力 #378

Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
d05289f
Optimize Hessian serialization and implement adaptive delay for large…
hui-cha Oct 30, 2025
78a06ff
bugfix: Fixed inconsistency between equals and compareTo methods in C…
hui-cha Nov 5, 2025
2b47503
bugfix: Fix potential inconsistency in worker configuration during st…
hui-cha Nov 6, 2025
0630e0b
Refactor task merge logic for small/large task transitions
hui-cha Nov 27, 2025
2c730e4
make realPubNum in NotThreadSafePublisherMap atomic
hui-cha Dec 26, 2025
fd714a7
Fix some edge cases
hui-cha Dec 26, 2025
eaf76be
update hessian versionto 3.5.6
hui-cha Dec 30, 2025
ba4dc93
add concurrent push and pop ut for ChangeTaskQueue
hui-cha Dec 30, 2025
b384043
update version to 6.6.2
hui-cha Dec 30, 2025
6ee7aed
Add automatic dump functionality & fix memory leak issues in unit tests.
hui-cha Dec 31, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion client/all/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

<groupId>com.alipay.sofa</groupId>
<artifactId>registry-client-all</artifactId>
<version>6.6.1-auto-regulator</version>
<version>6.6.2</version>

<name>${project.groupId}:${project.artifactId}</name>
<url>http://github.com/alipay/sofa-registry</url>
Expand Down
2 changes: 1 addition & 1 deletion client/api/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<groupId>com.alipay.sofa</groupId>
<artifactId>registry-client-parent</artifactId>
<version>6.6.1-auto-regulator</version>
<version>6.6.2</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
Expand Down
2 changes: 1 addition & 1 deletion client/impl/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<groupId>com.alipay.sofa</groupId>
<artifactId>registry-client-parent</artifactId>
<version>6.6.1-auto-regulator</version>
<version>6.6.2</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
Expand Down
2 changes: 1 addition & 1 deletion client/log/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<groupId>com.alipay.sofa</groupId>
<artifactId>registry-client-parent</artifactId>
<version>6.6.1-auto-regulator</version>
<version>6.6.2</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
Expand Down
2 changes: 1 addition & 1 deletion client/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
<parent>
<groupId>com.alipay.sofa</groupId>
<artifactId>registry-parent</artifactId>
<version>6.6.1-auto-regulator</version>
<version>6.6.2</version>
<relativePath>../pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion core/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<groupId>com.alipay.sofa</groupId>
<artifactId>registry-parent</artifactId>
<version>6.6.1-auto-regulator</version>
<version>6.6.2</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
Expand Down
4 changes: 2 additions & 2 deletions pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<modelVersion>4.0.0</modelVersion>
<groupId>com.alipay.sofa</groupId>
<artifactId>registry-parent</artifactId>
<version>6.6.1-auto-regulator</version>
<version>6.6.2</version>

<packaging>pom</packaging>

Expand Down Expand Up @@ -73,7 +73,7 @@
<junit.jupiter.version>5.0.2</junit.jupiter.version>
<junit.vintage.version>${junit.version}.2</junit.vintage.version>
<junit.platform.version>1.0.2</junit.platform.version>
<sofa.hessian.version>3.3.12</sofa.hessian.version>
<sofa.hessian.version>3.5.6</sofa.hessian.version>
Copy link

@coderabbitai coderabbitai bot Dec 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Check if Hessian 3.5.6 is available in Maven Central
echo "=== Checking Maven Central for Hessian 3.5.6 ==="
curl -s "https://search.maven.org/solrsearch/select?q=g:com.alipay.sofa+AND+a:hessian+AND+v:3.5.6&rows=5&wt=json" | jq '.response.docs[] | {id: .id, version: .v, timestamp: .timestamp}' 2>/dev/null || echo "Query failed or jq not available"

echo -e "\n=== Direct artifact check via Maven Central API ==="
curl -s "https://search.maven.org/artifact/com.alipay.sofa/hessian/3.5.6/jar" -I 2>&1 | head -10

echo -e "\n=== Checking Alipay's public nexus (alternative) ==="
curl -s "https://oss.sonatype.org/service/local/lucene/search?q=hessian&g=com.alipay.sofa&v=3.5.6" 2>&1 | head -20 || echo "Alternative repo not accessible"

Repository: sofastack/sofa-registry

Length of output: 742

🏁 Script executed:

#!/bin/bash
# Check for repository declarations in pom.xml
echo "=== Checking pom.xml for repository configurations ==="
grep -n "<repositories>" pom.xml -A 50 | head -60

echo -e "\n=== Checking for Alipay/Sofastack repository URLs ==="
grep -i "alipay\|sofastack\|repo" pom.xml | head -20

Repository: sofastack/sofa-registry

Length of output: 4028

Hessian 3.5.6 is not available in public Maven repositories.

Verification confirms that version 3.5.6 does not exist in Maven Central or OSS Sonatype. The build depends on internal Alipay repositories (mvn.test.alipay.net and mvn.dev.alipay.net) that are not publicly accessible. This contradicts the earlier indication that the official version would be released—it has not been published to public repositories. This creates a build dependency on proprietary Alipay infrastructure, which will prevent external developers from building this project.

🤖 Prompt for AI Agents 
In pom.xml at line 76, the property <sofa.hessian.version> is set to 3.5.6 which
is not published to public Maven repositories; update the project to use a
Hessian version that exists on Maven Central (or another public repo) or remove
reliance on internal Alipay repos. Replace the property value with a publicly
available version (verify the exact latest available version on Maven Central
before changing), remove any internal-only repository entries that enable 3.5.6,
and then run a build to confirm the artifact resolves; if you must keep 3.5.6
temporarily, add a clear comment and a ticket to publish or vendor the artifact
so external contributors can build.

<netty.version>4.1.25.Final</netty.version>
<jersey.version>2.26</jersey.version>
<protobuf.version>3.17.3</protobuf.version>
Expand Down
7 changes: 6 additions & 1 deletion server/common/model/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<groupId>com.alipay.sofa</groupId>
<artifactId>registry-common</artifactId>
<version>6.6.1-auto-regulator</version>
<version>6.6.2</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
Expand Down Expand Up @@ -51,5 +51,10 @@
<groupId>org.openjdk.jmh</groupId>
<artifactId>jmh-generator-annprocess</artifactId>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
</project>
14 changes: 13 additions & 1 deletion .../src/main/java/com/alipay/sofa/registry/common/model/sessionserver/DataChangeRequest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,21 +35,33 @@ public class DataChangeRequest implements Serializable {

private final Map<String, DatumVersion> dataInfoIds;

private final Map<String, Integer> publisherCounts;

private final String dataCenter;

private final TraceTimes times;

public DataChangeRequest(
String dataCenter, Map<String, DatumVersion> dataInfoIds, TraceTimes times) {
String dataCenter,
Map<String, DatumVersion> dataInfoIds,
Map<String, Integer> publisherCounts,
TraceTimes times) {
this.dataCenter = dataCenter;
this.dataInfoIds = dataInfoIds;
this.publisherCounts = publisherCounts;
this.times = times;
}
Comment on lines 44 to 53
Copy link

@coderabbitai coderabbitai bot Dec 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Add null safety check for publisherCounts parameter.

The constructor accepts publisherCounts but doesn't validate it for null. If null is passed, getPublisherCounts() will throw NPE when calling Collections.unmodifiableMap(publisherCounts).

🔎 Apply this diff to add null safety: 
   public DataChangeRequest(
       String dataCenter,
       Map<String, DatumVersion> dataInfoIds,
       Map<String, Integer> publisherCounts,
       TraceTimes times) {
     this.dataCenter = dataCenter;
     this.dataInfoIds = dataInfoIds;
-    this.publisherCounts = publisherCounts;
+    this.publisherCounts = publisherCounts != null ? publisherCounts : Collections.emptyMap();
     this.times = times;
   }
🤖 Prompt for AI Agents 
In
server/common/model/src/main/java/com/alipay/sofa/registry/common/model/sessionserver/DataChangeRequest.java
around lines 44 to 53, the constructor accepts publisherCounts but does not
guard against null which will cause an NPE when callers wrap it with
Collections.unmodifiableMap; add a null-safety assignment in the constructor
such that if publisherCounts is null you assign Collections.emptyMap() (or a new
HashMap<>() if mutability is needed elsewhere) before storing it to
this.publisherCounts so subsequent calls to getPublisherCounts() are safe.


public Map<String, DatumVersion> getDataInfoIds() {
return Collections.unmodifiableMap(dataInfoIds);
}

public Map<String, Integer> getPublisherCounts() {
return null == publisherCounts
? Collections.emptyMap()
: Collections.unmodifiableMap(publisherCounts);
}

public String getDataCenter() {
return dataCenter;
}
Expand Down
2 changes: 1 addition & 1 deletion server/common/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<groupId>com.alipay.sofa</groupId>
<artifactId>registry-server-parent</artifactId>
<version>6.6.1-auto-regulator</version>
<version>6.6.2</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
Expand Down
2 changes: 1 addition & 1 deletion server/common/util/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<groupId>com.alipay.sofa</groupId>
<artifactId>registry-common</artifactId>
<version>6.6.1-auto-regulator</version>
<version>6.6.2</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
Expand Down
2 changes: 1 addition & 1 deletion server/distribution/all/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<groupId>com.alipay.sofa</groupId>
<artifactId>registry-distribution</artifactId>
<version>6.6.1-auto-regulator</version>
<version>6.6.2</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
Expand Down
2 changes: 1 addition & 1 deletion server/distribution/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<groupId>com.alipay.sofa</groupId>
<artifactId>registry-server-parent</artifactId>
<version>6.6.1-auto-regulator</version>
<version>6.6.2</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
Expand Down
2 changes: 1 addition & 1 deletion server/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
<parent>
<groupId>com.alipay.sofa</groupId>
<artifactId>registry-parent</artifactId>
<version>6.6.1-auto-regulator</version>
<version>6.6.2</version>
<relativePath>../pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion server/remoting/api/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<groupId>com.alipay.sofa</groupId>
<artifactId>registry-remoting</artifactId>
<version>6.6.1-auto-regulator</version>
<version>6.6.2</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
Expand Down
2 changes: 1 addition & 1 deletion server/remoting/bolt/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<groupId>com.alipay.sofa</groupId>
<artifactId>registry-remoting</artifactId>
<version>6.6.1-auto-regulator</version>
<version>6.6.2</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
Expand Down
2 changes: 1 addition & 1 deletion server/remoting/http/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<groupId>com.alipay.sofa</groupId>
<artifactId>registry-remoting</artifactId>
<version>6.6.1-auto-regulator</version>
<version>6.6.2</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
Expand Down
2 changes: 1 addition & 1 deletion server/remoting/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<groupId>com.alipay.sofa</groupId>
<artifactId>registry-server-parent</artifactId>
<version>6.6.1-auto-regulator</version>
<version>6.6.2</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
Expand Down
2 changes: 1 addition & 1 deletion server/server/data/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<groupId>com.alipay.sofa</groupId>
<artifactId>registry-server</artifactId>
<version>6.6.1-auto-regulator</version>
<version>6.6.2</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
Expand Down
11 changes: 6 additions & 5 deletions ...erver/data/src/main/java/com/alipay/sofa/registry/server/data/cache/BaseDatumStorage.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,7 @@
import com.alipay.sofa.registry.util.ParaCheckUtil;
import com.google.common.collect.Maps;
import com.google.common.collect.Sets;
import java.util.Collection;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.*;
import java.util.function.BiConsumer;
import org.springframework.util.CollectionUtils;

Expand Down Expand Up @@ -106,6 +102,11 @@ public Map<String, Integer> getPubCount() {
return map;
}

public Integer getPubCount(String dataInfoId) {
PublisherGroups groups = getPublisherGroups(dataInfoId);
return groups == null ? null : groups.getRealPubNum(dataInfoId);
}

public Map<String, Publisher> getByConnectId(ConnectId connectId) {
Map<String, Publisher> m = Maps.newHashMapWithExpectedSize(64);
publisherGroupsMap.values().forEach(g -> m.putAll(g.getByConnectId(connectId)));
Expand Down
2 changes: 2 additions & 0 deletions ...er/server/data/src/main/java/com/alipay/sofa/registry/server/data/cache/DatumStorage.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,8 @@ Map<String, DatumVersion> getVersions(

Map<String, Integer> getPubCount(String dataCenter);

Integer getPubCount(String dataCenter, String dataInfoId);

void putPublisherGroups(String dataCenter, int slotId);

DatumVersion putPublisher(String dataCenter, Publisher publisher);
Expand Down
5 changes: 5 additions & 0 deletions ...r/data/src/main/java/com/alipay/sofa/registry/server/data/cache/DatumStorageDelegate.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,11 @@ public Map<String, Integer> getPubCount(String dataCenter) {
return storageOf(dataCenter).getPubCount(dataCenter);
}

@Override
public Integer getPubCount(String dataCenter, String dataInfoId) {
return storageOf(dataCenter).getPubCount(dataCenter, dataInfoId);
}

@Override
public void putPublisherGroups(String dataCenter, int slotId) {
storageOf(dataCenter).putPublisherGroups(dataCenter, slotId);
Expand Down
5 changes: 5 additions & 0 deletions ...rver/data/src/main/java/com/alipay/sofa/registry/server/data/cache/LocalDatumStorage.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,11 @@ public Map<String, Integer> getPubCount(String dataCenter) {
return storage.getPubCount();
}

@Override
public Integer getPubCount(String dataCenter, String dataInfoId) {
return storage.getPubCount(dataInfoId);
}

@Override
public void putPublisherGroups(String dataCenter, int slotId) {
storage.putPublisherGroups(slotId);
Expand Down
Loading
Loading