Skip to content

Feature: 下载并发控制 #5026

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Glavo merged 6 commits into from
Jan 11, 2026
Merged

Feature: 下载并发控制 #5026

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
4efe5b5
Feature: Download Semaphore
burningtnt Dec 20, 2025
b89e376
Merge remote-tracking branch 'official/main' into feature/download-se…
burningtnt Dec 20, 2025
194b9ac
Merge remote-tracking branch 'refs/remotes/official/main' into featur…
burningtnt Dec 27, 2025
07c4a44
Code cleanup.
burningtnt Dec 27, 2025
c60822c
Code cleanup.
burningtnt Dec 28, 2025
cf1e8c6
update
Glavo Jan 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions HMCL/src/main/java/org/jackhuang/hmcl/setting/ProxyManager.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
package org.jackhuang.hmcl.setting;

import javafx.beans.InvalidationListener;
import org.jackhuang.hmcl.task.FetchTask;
import org.jackhuang.hmcl.util.StringUtils;
import org.jackhuang.hmcl.util.io.NetworkUtils;
import org.jetbrains.annotations.NotNull;
Expand Down Expand Up @@ -114,6 +115,8 @@ protected PasswordAuthentication getPasswordAuthentication() {
config().hasProxyAuthProperty().addListener(updateAuthenticator);
config().proxyUserProperty().addListener(updateAuthenticator);
config().proxyPassProperty().addListener(updateAuthenticator);

FetchTask.notifyInitialized();
}

private static abstract class AbstractProxySelector extends ProxySelector {
Expand Down
3 changes: 3 additions & 0 deletions HMCL/src/main/java/org/jackhuang/hmcl/ui/FXUtils.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1188,12 +1188,14 @@ public static Image newBuiltinImage(String url, double requestedWidth, double re

public static Task<Image> getRemoteImageTask(String url, int requestedWidth, int requestedHeight, boolean preserveRatio, boolean smooth) {
return new CacheFileTask(url)
.setSignificance(Task.TaskSignificance.MINOR)
.thenApplyAsync(file -> loadImage(file, requestedWidth, requestedHeight, preserveRatio, smooth))
.setSignificance(Task.TaskSignificance.MINOR);
}

public static Task<Image> getRemoteImageTask(URI uri, int requestedWidth, int requestedHeight, boolean preserveRatio, boolean smooth) {
return new CacheFileTask(uri)
.setSignificance(Task.TaskSignificance.MINOR)
Comment on lines +1191 to +1198
Copy link

Copilot AI Dec 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The setSignificance(Task.TaskSignificance.MINOR) is called twice in this chain - once after the CacheFileTask and once after thenApplyAsync. The second call (line 1193) will override the first (line 1191), making the first call redundant. Consider removing the first call or clarify if both are intentional for some reason.

Suggested change
.setSignificance(Task.TaskSignificance.MINOR)
.thenApplyAsync(file -> loadImage(file, requestedWidth, requestedHeight, preserveRatio, smooth))
.setSignificance(Task.TaskSignificance.MINOR);
}
public static Task<Image> getRemoteImageTask(URI uri, int requestedWidth, int requestedHeight, boolean preserveRatio, boolean smooth) {
return new CacheFileTask(uri)
.setSignificance(Task.TaskSignificance.MINOR)
.thenApplyAsync(file -> loadImage(file, requestedWidth, requestedHeight, preserveRatio, smooth))
.setSignificance(Task.TaskSignificance.MINOR);
}
public static Task<Image> getRemoteImageTask(URI uri, int requestedWidth, int requestedHeight, boolean preserveRatio, boolean smooth) {
return new CacheFileTask(uri)

Copilot uses AI. Check for mistakes.
Comment on lines +1191 to +1198
Copy link

Copilot AI Dec 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The setSignificance(Task.TaskSignificance.MINOR) is called twice in this chain - once after the CacheFileTask and once after thenApplyAsync. The second call (line 1200) will override the first (line 1198), making the first call redundant. Consider removing the first call or clarify if both are intentional for some reason.

Suggested change
.setSignificance(Task.TaskSignificance.MINOR)
.thenApplyAsync(file -> loadImage(file, requestedWidth, requestedHeight, preserveRatio, smooth))
.setSignificance(Task.TaskSignificance.MINOR);
}
public static Task<Image> getRemoteImageTask(URI uri, int requestedWidth, int requestedHeight, boolean preserveRatio, boolean smooth) {
return new CacheFileTask(uri)
.setSignificance(Task.TaskSignificance.MINOR)
.thenApplyAsync(file -> loadImage(file, requestedWidth, requestedHeight, preserveRatio, smooth))
.setSignificance(Task.TaskSignificance.MINOR);
}
public static Task<Image> getRemoteImageTask(URI uri, int requestedWidth, int requestedHeight, boolean preserveRatio, boolean smooth) {
return new CacheFileTask(uri)

Copilot uses AI. Check for mistakes.
.thenApplyAsync(file -> loadImage(file, requestedWidth, requestedHeight, preserveRatio, smooth))
.setSignificance(Task.TaskSignificance.MINOR);
}
Expand All @@ -1208,6 +1210,7 @@ public static ObservableValue<Image> newRemoteImage(String url, int requestedWid
LOG.warning("An exception encountered while loading remote image: " + url, exception);
}
})
.setSignificance(Task.TaskSignificance.MINOR)
.start();
return image;
}
Expand Down
153 changes: 94 additions & 59 deletions HMCLCore/src/main/java/org/jackhuang/hmcl/mod/curse/CurseForgeRemoteModRepository.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,15 @@
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.*;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.HashMap;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Optional;
import java.util.concurrent.Semaphore;
import java.util.stream.Stream;

import static org.jackhuang.hmcl.util.Lang.mapOf;
Expand All @@ -46,6 +54,7 @@ public final class CurseForgeRemoteModRepository implements RemoteModRepository

private static final String PREFIX = "https://api.curseforge.com";
private static final String apiKey = System.getProperty("hmcl.curseforge.apikey", JarUtils.getAttribute("hmcl.curseforge.apikey", ""));
private static final Semaphore SEMAPHORE = new Semaphore(16);

private static final int WORD_PERFECT_MATCH_WEIGHT = 5;

Expand Down Expand Up @@ -110,46 +119,51 @@ private int calculateTotalPages(Response<List<CurseAddon>> response, int pageSiz

@Override
public SearchResult search(DownloadProvider downloadProvider, String gameVersion, @Nullable RemoteModRepository.Category category, int pageOffset, int pageSize, String searchFilter, SortType sortType, SortOrder sortOrder) throws IOException {
int categoryId = 0;
if (category != null && category.getSelf() instanceof CurseAddon.Category) {
categoryId = ((CurseAddon.Category) category.getSelf()).getId();
}
Response<List<CurseAddon>> response = withApiKey(HttpRequest.GET(downloadProvider.injectURL(NetworkUtils.withQuery(PREFIX + "/v1/mods/search", mapOf(
pair("gameId", "432"),
pair("classId", Integer.toString(section)),
pair("categoryId", Integer.toString(categoryId)),
pair("gameVersion", gameVersion),
pair("searchFilter", searchFilter),
pair("sortField", Integer.toString(toModsSearchSortField(sortType))),
pair("sortOrder", toSortOrder(sortOrder)),
pair("index", Integer.toString(pageOffset * pageSize)),
pair("pageSize", Integer.toString(pageSize)))))))
.getJson(Response.typeOf(listTypeOf(CurseAddon.class)));
if (searchFilter.isEmpty()) {
return new SearchResult(response.getData().stream().map(CurseAddon::toMod), calculateTotalPages(response, pageSize));
}
SEMAPHORE.acquireUninterruptibly();
try {
int categoryId = 0;
if (category != null && category.getSelf() instanceof CurseAddon.Category) {
categoryId = ((CurseAddon.Category) category.getSelf()).getId();
}
Response<List<CurseAddon>> response = withApiKey(HttpRequest.GET(downloadProvider.injectURL(NetworkUtils.withQuery(PREFIX + "/v1/mods/search", mapOf(
pair("gameId", "432"),
pair("classId", Integer.toString(section)),
pair("categoryId", Integer.toString(categoryId)),
pair("gameVersion", gameVersion),
pair("searchFilter", searchFilter),
pair("sortField", Integer.toString(toModsSearchSortField(sortType))),
pair("sortOrder", toSortOrder(sortOrder)),
pair("index", Integer.toString(pageOffset * pageSize)),
pair("pageSize", Integer.toString(pageSize)))))))
.getJson(Response.typeOf(listTypeOf(CurseAddon.class)));
if (searchFilter.isEmpty()) {
return new SearchResult(response.getData().stream().map(CurseAddon::toMod), calculateTotalPages(response, pageSize));
}

// https://github.com/HMCL-dev/HMCL/issues/1549
String lowerCaseSearchFilter = searchFilter.toLowerCase(Locale.ROOT);
Map<String, Integer> searchFilterWords = new HashMap<>();
for (String s : StringUtils.tokenize(lowerCaseSearchFilter)) {
searchFilterWords.put(s, searchFilterWords.getOrDefault(s, 0) + 1);
}
// https://github.com/HMCL-dev/HMCL/issues/1549
String lowerCaseSearchFilter = searchFilter.toLowerCase(Locale.ROOT);
Map<String, Integer> searchFilterWords = new HashMap<>();
for (String s : StringUtils.tokenize(lowerCaseSearchFilter)) {
searchFilterWords.put(s, searchFilterWords.getOrDefault(s, 0) + 1);
}

StringUtils.LevCalculator levCalculator = new StringUtils.LevCalculator();
StringUtils.LevCalculator levCalculator = new StringUtils.LevCalculator();

return new SearchResult(response.getData().stream().map(CurseAddon::toMod).map(remoteMod -> {
String lowerCaseResult = remoteMod.getTitle().toLowerCase(Locale.ROOT);
int diff = levCalculator.calc(lowerCaseSearchFilter, lowerCaseResult);
return new SearchResult(response.getData().stream().map(CurseAddon::toMod).map(remoteMod -> {
String lowerCaseResult = remoteMod.getTitle().toLowerCase(Locale.ROOT);
int diff = levCalculator.calc(lowerCaseSearchFilter, lowerCaseResult);

for (String s : StringUtils.tokenize(lowerCaseResult)) {
if (searchFilterWords.containsKey(s)) {
diff -= WORD_PERFECT_MATCH_WEIGHT * searchFilterWords.get(s) * s.length();
for (String s : StringUtils.tokenize(lowerCaseResult)) {
if (searchFilterWords.containsKey(s)) {
diff -= WORD_PERFECT_MATCH_WEIGHT * searchFilterWords.get(s) * s.length();
}
}
}

return pair(remoteMod, diff);
}).sorted(Comparator.comparingInt(Pair::getValue)).map(Pair::getKey), response.getData().stream().map(CurseAddon::toMod), calculateTotalPages(response, pageSize));
return pair(remoteMod, diff);
}).sorted(Comparator.comparingInt(Pair::getValue)).map(Pair::getKey), response.getData().stream().map(CurseAddon::toMod), calculateTotalPages(response, pageSize));
} finally {
SEMAPHORE.release();
}
}

@Override
Expand All @@ -173,48 +187,69 @@ public Optional<RemoteMod.Version> getRemoteVersionByLocalFile(LocalModFile loca
return Optional.empty();
}

Response<FingerprintMatchesResult> response = withApiKey(HttpRequest.POST(PREFIX + "/v1/fingerprints/432"))
.json(mapOf(pair("fingerprints", Collections.singletonList(hash))))
.getJson(Response.typeOf(FingerprintMatchesResult.class));
SEMAPHORE.acquireUninterruptibly();
try {
Response<FingerprintMatchesResult> response = withApiKey(HttpRequest.POST(PREFIX + "/v1/fingerprints/432"))
.json(mapOf(pair("fingerprints", Collections.singletonList(hash))))
.getJson(Response.typeOf(FingerprintMatchesResult.class));

if (response.getData().getExactMatches() == null || response.getData().getExactMatches().isEmpty()) {
return Optional.empty();
}
if (response.getData().getExactMatches() == null || response.getData().getExactMatches().isEmpty()) {
return Optional.empty();
}

return Optional.of(response.getData().getExactMatches().get(0).getFile().toVersion());
return Optional.of(response.getData().getExactMatches().get(0).getFile().toVersion());
} finally {
SEMAPHORE.release();
}
}

@Override
public RemoteMod getModById(String id) throws IOException {
Response<CurseAddon> response = withApiKey(HttpRequest.GET(PREFIX + "/v1/mods/" + id))
.getJson(Response.typeOf(CurseAddon.class));
return response.data.toMod();
SEMAPHORE.acquireUninterruptibly();
try {
Response<CurseAddon> response = withApiKey(HttpRequest.GET(PREFIX + "/v1/mods/" + id))
.getJson(Response.typeOf(CurseAddon.class));
return response.data.toMod();
} finally {
SEMAPHORE.release();
}
}

@Override
public RemoteMod.File getModFile(String modId, String fileId) throws IOException {
Response<CurseAddon.LatestFile> response = withApiKey(HttpRequest.GET(String.format("%s/v1/mods/%s/files/%s", PREFIX, modId, fileId)))
.getJson(Response.typeOf(CurseAddon.LatestFile.class));
return response.getData().toVersion().getFile();
SEMAPHORE.acquireUninterruptibly();
try {
Response<CurseAddon.LatestFile> response = withApiKey(HttpRequest.GET(String.format("%s/v1/mods/%s/files/%s", PREFIX, modId, fileId)))
.getJson(Response.typeOf(CurseAddon.LatestFile.class));
return response.getData().toVersion().getFile();
} finally {
SEMAPHORE.release();
}
}

@Override
public Stream<RemoteMod.Version> getRemoteVersionsById(String id) throws IOException {
Response<List<CurseAddon.LatestFile>> response = withApiKey(HttpRequest.GET(PREFIX + "/v1/mods/" + id + "/files",
pair("pageSize", "10000")))
.getJson(Response.typeOf(listTypeOf(CurseAddon.LatestFile.class)));
return response.getData().stream().map(CurseAddon.LatestFile::toVersion);
}

public List<CurseAddon.Category> getCategoriesImpl() throws IOException {
Response<List<CurseAddon.Category>> categories = withApiKey(HttpRequest.GET(PREFIX + "/v1/categories", pair("gameId", "432")))
.getJson(Response.typeOf(listTypeOf(CurseAddon.Category.class)));
return reorganizeCategories(categories.getData(), section);
SEMAPHORE.acquireUninterruptibly();
try {
Response<List<CurseAddon.LatestFile>> response = withApiKey(HttpRequest.GET(PREFIX + "/v1/mods/" + id + "/files",
pair("pageSize", "10000")))
.getJson(Response.typeOf(listTypeOf(CurseAddon.LatestFile.class)));
return response.getData().stream().map(CurseAddon.LatestFile::toVersion);
} finally {
SEMAPHORE.release();
}
}

@Override
public Stream<RemoteModRepository.Category> getCategories() throws IOException {
return getCategoriesImpl().stream().map(CurseAddon.Category::toCategory);
SEMAPHORE.acquireUninterruptibly();
try {
Response<List<CurseAddon.Category>> categories = withApiKey(HttpRequest.GET(PREFIX + "/v1/categories", pair("gameId", "432")))
.getJson(Response.typeOf(listTypeOf(CurseAddon.Category.class)));
return reorganizeCategories(categories.getData(), section).stream().map(CurseAddon.Category::toCategory);
} finally {
SEMAPHORE.release();
}
}
Comment on lines 244 to 253
Copy link

Copilot AI Jan 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The semaphore is released before the returned stream is consumed. Since streams are lazily evaluated, the actual iteration happens after the semaphore has been released, making the concurrency control ineffective. The list should be collected before returning or the semaphore should be held until the stream is consumed. Consider returning a List instead of a Stream, or collect the stream to a list before releasing the semaphore.

Copilot uses AI. Check for mistakes.

private List<CurseAddon.Category> reorganizeCategories(List<CurseAddon.Category> categories, int rootId) {
Expand Down
Loading
Loading