Add async PAPI leaderboard cache and placeholders

Fixes #5171
Fixes #4800
Fixes #4291

Author: nossr50

Changelog.txt

@@ -4,12 +4,25 @@ Version 2.2.055
     Fixed Maces Cripple playing an extra anvil break sound that could not be disabled in sounds.yml (See notes)
     Fixed Super Breaker and Tree Feller mishandling durability on items with a custom max_damage component (See notes)
     Fixed Tree Feller ignoring durability damage changes made by other plugins via PlayerItemDamageEvent
+    Fixed a thread-safety issue that could cause incorrect FlatFile leaderboard results
+    Fixed renamed players' leftover database rows showing up in /mctop and being counted by /mcrank on SQL databases (See notes)
+    Fixed FlatFile leaderboards not retrying a failed rebuild until the next refresh interval
     Improved performance when players gain skill XP (See notes)
     Improved performance when taking smelted items out of furnaces (See notes)
+    Added PlaceholderAPI leaderboard rank-position placeholders for all skills except child skills (Salvage, Smelting) and for overall power level (See notes)
+    Improved SQL leaderboard query performance on large databases (See notes)
+    Added automatic per-skill leaderboard indexes for SQL databases (See notes)
+    Changed SQL leaderboard ordering for players with identical levels (See notes)
+    Changed the leaderboard placeholder cache to rebuild from a single user file scan on FlatFile and to pause refreshes while no leaderboard placeholders are in use (See notes)
+    Added 'General.PlaceholderAPI.Leaderboards.Max_Tracked_Rank' to config.yml
+    Added 'General.Leaderboards.Refresh_Interval_Seconds.SQL' to config.yml
+    Added 'General.Leaderboards.Refresh_Interval_Seconds.FlatFile' to config.yml
     (API) McMMOScoreboardObjectiveEvent is only fired on non-Folia servers (See notes)
+    (API) Added DatabaseManager#readLeaderboardSnapshot for reading every leaderboard scope in one call
     (Codebase) Added scoreboard-library 2.8.0 as a shaded dependency for the packet-based scoreboard implementation
     (Codebase) Added ViaVersion to plugin.yml softdepend
     (Codebase) Added unit tests covering the power level cap and skill level cap checks
+    (Codebase) Added async leaderboard snapshot caching for rank-position PlaceholderAPI lookups
 
     NOTES:
     Scoreboards on Folia (and Folia forks like Canvas) now use a packet-based implementation, since the Bukkit scoreboard API is not safe to use there. Paper and Spigot servers keep the existing Bukkit scoreboard implementation and are unaffected.
@@ -21,6 +34,72 @@ Version 2.2.055
     Cripple now plays a single sound which can be adjusted or disabled with the CRIPPLE entry in sounds.yml.
     Items with a custom maximum durability (set through the max_damage item component by data packs or item plugins) previously could make super abilities stop working or appear to restore durability; ability durability loss now always uses the item's own maximum.
 
+    -- PlaceholderAPI leaderboard rank-position placeholders --
+    This update adds placeholders that let you request specific leaderboard positions directly.
+    Format examples:
+        %mcmmo_mctop_<skill>:<position>%
+        %mcmmo_mctop_name_<skill>:<position>%
+
+    Mining examples:
+        %mcmmo_mctop_mining:1%         -> skill level of the #1 ranked Mining player
+        %mcmmo_mctop_name_mining:1%    -> name of the #1 ranked Mining player
+        %mcmmo_mctop_mining:34%        -> skill level of the #34 ranked Mining player
+        %mcmmo_mctop_name_mining:34%   -> name of the #34 ranked Mining player
+
+    Overall / power level examples:
+        %mcmmo_mctop_overall:1%
+        %mcmmo_mctop_name_overall:1%
+        %mcmmo_mctop_all:10%
+        %mcmmo_mctop_name_all:10%
+        %mcmmo_mctop_powerlevel:25%
+        %mcmmo_mctop_name_powerlevel:25%
+
+    Alias behavior:
+    'overall', 'all', and 'powerlevel' all map to the same overall leaderboard data.
+
+    Config notes:
+    'General.PlaceholderAPI.Leaderboards.Max_Tracked_Rank' controls the highest supported placeholder position.
+    Default is 100. Values are clamped between 10 and 1000, since higher positions cost memory and database reads on every cache refresh.
+    Requests above this value return an empty result.
+    Invalid positions (blank, non-numeric, 0, negative) also return an empty result.
+    Positions that no player currently holds also return an empty result.
+
+    'General.Leaderboards.Refresh_Interval_Seconds.FlatFile' controls how often FlatFile leaderboards are rebuilt.
+    This affects /mctop, /mcrank, and the leaderboard placeholders.
+    Default is 600 seconds (10 minutes), matching previous behavior.
+    'General.Leaderboards.Refresh_Interval_Seconds.SQL' controls how often the placeholder cache refreshes on SQL databases.
+    /mctop and /mcrank on SQL are unaffected and always query live data.
+    Default is 60 seconds.
+    Both intervals have a 60 second minimum; values below 60 are treated as 60.
+    Placeholder requests are served from cache between refreshes.
+
+    Each cache refresh reads every leaderboard in one pass; on FlatFile this is a single scan
+    of the user file no matter how many skills are tracked.
+    If no leaderboard placeholder has been used since the previous refresh, the next refresh
+    is skipped and the cache sits idle. The first request afterwards is served from the cached
+    data and triggers an immediate background refresh. Servers that never use these
+    placeholders do no recurring leaderboard cache work.
+
+    -- SQL leaderboard performance --
+    Leaderboard queries on SQL databases are now much faster on servers with large player tables.
+    mcMMO will automatically add per-skill indexes to the skills table on startup if they are missing.
+    On very large databases the first startup after updating may take longer while these indexes are built.
+    This runs once, is safe to re-run, and mcMMO will continue to start normally even if an index cannot be added.
+    Fresh SQL installs include these indexes from the start, and skill columns added by future
+    mcMMO updates get their index automatically.
+    If an index named idx_<skill> already exists but covers a different column, mcMMO assumes
+    it was created deliberately, leaves it alone, and does not retry on later startups.
+
+    When a player renames, the database row for their old name is kept internally under a
+    placeholder name. On SQL databases these rows could previously appear in /mctop and the
+    leaderboard placeholders, and were counted when computing /mcrank positions; they are now
+    excluded from all of them.
+
+    As part of this change, players with identical levels on SQL databases are now ordered
+    newest-registered-first in /mctop, /mcrank, and the leaderboard placeholders (previously
+    ties were ordered alphabetically by name). /mctop and /mcrank agree on tie positions.
+    FlatFile databases still order ties alphabetically.
+
 Version 2.2.054
     Added compatibility for new blocks and items from Chaos Cubed (Minecraft 26.2) to mcMMO
     Fixed party/admin chat allowing players to use color codes without the 'mcmmo.chat.colors' permission

src/main/java/com/gmail/nossr50/config/GeneralConfig.java

@@ -23,6 +23,15 @@
 import org.jetbrains.annotations.Nullable;
 
 public class GeneralConfig extends BukkitConfig {
+    // Floor for how often leaderboards may rebuild; shared with FlatFileDatabaseManager's
+    // rebuild throttle so the config clamp and the runtime throttle can never disagree.
+    public static final int MIN_LEADERBOARD_REFRESH_INTERVAL_SECONDS = 60;
+    // Each tracked rank is cached for every non-child skill plus overall, and every cache
+    // refresh re-reads that many rows per leaderboard, so the ceiling bounds both memory
+    // use and periodic database load.
+    private static final int MIN_PAPI_LEADERBOARD_TRACKED_RANK = 10;
+    private static final int MAX_PAPI_LEADERBOARD_TRACKED_RANK = 1000;
+
     private @Nullable Material repairAnvilMaterial;
     private @Nullable Material salvageAnvilMaterial;
 
@@ -249,6 +258,35 @@ public boolean getRegionDataMigrationBackupsEnabled() {
         return config.getBoolean("General.RegionDataMigrationBackups", true);
     }
 
+    /**
+     * @return Highest leaderboard position kept in the PlaceholderAPI cache, clamped between
+     * {@value #MIN_PAPI_LEADERBOARD_TRACKED_RANK} and {@value #MAX_PAPI_LEADERBOARD_TRACKED_RANK}.
+     */
+    public int getPapiLeaderboardMaxTrackedRank() {
+        final int configured =
+                config.getInt("General.PlaceholderAPI.Leaderboards.Max_Tracked_Rank", 100);
+        return Math.min(MAX_PAPI_LEADERBOARD_TRACKED_RANK,
+                Math.max(MIN_PAPI_LEADERBOARD_TRACKED_RANK, configured));
+    }
+
+    /**
+     * @return SQL leaderboard cache refresh interval in seconds, never below
+     * {@value #MIN_LEADERBOARD_REFRESH_INTERVAL_SECONDS}.
+     */
+    public int getLeaderboardRefreshIntervalSecondsSQL() {
+        return Math.max(MIN_LEADERBOARD_REFRESH_INTERVAL_SECONDS,
+                config.getInt("General.Leaderboards.Refresh_Interval_Seconds.SQL", 60));
+    }
+
+    /**
+     * @return FlatFile leaderboard cache refresh interval in seconds, never below
+     * {@value #MIN_LEADERBOARD_REFRESH_INTERVAL_SECONDS}.
+     */
+    public int getLeaderboardRefreshIntervalSecondsFlatFile() {
+        return Math.max(MIN_LEADERBOARD_REFRESH_INTERVAL_SECONDS,
+                config.getInt("General.Leaderboards.Refresh_Interval_Seconds.FlatFile", 600));
+    }
+
     public boolean getMobHealthbarEnabled() {
         return config.getBoolean("Mob_Healthbar.Enabled", true);
     }

src/main/java/com/gmail/nossr50/database/DatabaseManager.java

@@ -2,9 +2,12 @@
 
 import com.gmail.nossr50.api.exceptions.InvalidSkillException;
 import com.gmail.nossr50.datatypes.database.DatabaseType;
+import com.gmail.nossr50.datatypes.database.LeaderboardSnapshot;
 import com.gmail.nossr50.datatypes.database.PlayerStat;
 import com.gmail.nossr50.datatypes.player.PlayerProfile;
 import com.gmail.nossr50.datatypes.skills.PrimarySkillType;
+import com.gmail.nossr50.util.skills.SkillTools;
+import java.util.EnumMap;
 import java.util.List;
 import java.util.Map;
 import java.util.UUID;
@@ -62,6 +65,40 @@ public interface DatabaseManager {
     @NotNull List<PlayerStat> readLeaderboard(@Nullable PrimarySkillType skill, int pageNumber,
             int statsPerPage) throws InvalidSkillException;
 
+    /**
+     * Retrieve the top rows of every leaderboard scope (each non-child skill plus the power level
+     * leaderboard) directly from the backend in one bulk call, for callers that build caches from
+     * the result.
+     * <p>
+     * Unlike {@link #readLeaderboard(PrimarySkillType, int, int)}, implementations must propagate
+     * backend read failures instead of returning a partial or empty result, so callers can tell a
+     * failed read apart from genuinely empty leaderboards. Implementations should also bypass any
+     * backend-level result caching so callers always observe current data. The default
+     * implementation reads each scope through
+     * {@link #readLeaderboard(PrimarySkillType, int, int)} and therefore inherits that method's
+     * failure handling; custom database managers should override it to honor this contract.
+     *
+     * @param perScopeLimit The maximum number of rows to include per leaderboard scope
+     * @return the top rows of every leaderboard scope
+     * @throws RuntimeException when the backend read fails
+     */
+    default @NotNull LeaderboardSnapshot readLeaderboardSnapshot(int perScopeLimit) {
+        final Map<PrimarySkillType, List<PlayerStat>> skillLeaderboards =
+                new EnumMap<>(PrimarySkillType.class);
+
+        try {
+            for (PrimarySkillType skill : SkillTools.NON_CHILD_SKILLS) {
+                skillLeaderboards.put(skill, readLeaderboard(skill, 1, perScopeLimit));
+            }
+
+            return new LeaderboardSnapshot(skillLeaderboards,
+                    readLeaderboard(null, 1, perScopeLimit));
+        } catch (InvalidSkillException e) {
+            // Scopes are fixed to non-child skills plus overall, so this cannot happen.
+            throw new IllegalStateException(e);
+        }
+    }
+
     /**
      * Retrieve rank info into a HashMap from PrimarySkillType to the rank.
      * <p>

src/main/java/com/gmail/nossr50/database/DatabaseManagerFactory.java

@@ -30,7 +30,12 @@ public static DatabaseManager getDatabaseManager(@NotNull String userFilePath,
 
         return mcMMO.p.getGeneralConfig().getUseMySQL()
                 ? new SQLDatabaseManager(logger, MYSQL_DRIVER)
-                : new FlatFileDatabaseManager(userFilePath, logger, purgeTime, startingLevel);
+                : new FlatFileDatabaseManager(userFilePath, logger, purgeTime, startingLevel,
+                        flatFileLeaderboardRefreshIntervalMillis());
+    }
+
+    private static long flatFileLeaderboardRefreshIntervalMillis() {
+        return 1000L * mcMMO.p.getGeneralConfig().getLeaderboardRefreshIntervalSecondsFlatFile();
     }
 
     /**
@@ -65,7 +70,8 @@ public static Class<? extends DatabaseManager> getCustomDatabaseManagerClass() {
         switch (type) {
             case FLATFILE:
                 LogUtils.debug(mcMMO.p.getLogger(), "Using FlatFile Database");
-                return new FlatFileDatabaseManager(userFilePath, logger, purgeTime, startingLevel);
+                return new FlatFileDatabaseManager(userFilePath, logger, purgeTime, startingLevel,
+                        flatFileLeaderboardRefreshIntervalMillis());
 
             case SQL:
                 LogUtils.debug(mcMMO.p.getLogger(), "Using SQL Database");