Interface Revamp, Upgrade Mouse Integration & Performance Enhancements (#46)

# PR #46: Interface Revamp, Humanized Mouse & Performance Overhaul

This is a massive update that updates almost every part of the bot. I wanted to make the client look better, feel more human, and run a lot smoother.

### Complete Web Interface Rehaul

The frontend has been completely rebuilt with a modern **GlassMorphism** aesthetic. Theres also several components in the tech stack which have been created to utilise `WebSockets` for real time updates.

* **Live Debug Viewport:** You can now see exactly what the bot "sees" in real-time. The `ColourContours` and `TemplateMatching` logic push their state directly to the browser. If the stream gets overloaded, the system discards old frames to keep the bot logic running.
* **Reactive Status:** The header now displays the live sensor state (`SEARCHING`, `ACTING`, `WAITING`, `ERROR`) so you know exactly what the bot is thinking.
* **Live Statistics:** Runtime, cycles, and inputs are streamed live.
* **Smart Logs:** Console logs are now colour-coded by severity in the web UI.
* **Better Controls:** Fixed the sliders (you can no longer break them by pushing min past max), and added a **Copy Button** to the colour picker. You can now generate the Java code for a `ColourObj` and paste it directly into your script.
* **Slider performance** Slider performance has been greatly improved by no longer writing modified.png to disk.

### WindMouse Integration

I’ve completely replaced the old `MousePathing` with **WindMouse**.

- Instead of calculating a geometric path, WindMouse simulates a physical cursor with mass. It uses `gravity` to pull towards the target and `wind` to apply randomness making it pixel imperfect.
- Movement includes natural overshoots, micro-jitters, and variable speeds, As a result, overshoot and undershoot methods are now deprecated and removed.

###  Performance & Multithreading

We've moved to a **Producer -> Consumer threading model** for inputs. The physics engine calculates mouse movement on one thread, while a separate background thread handles the hardware remote inputs. If the hardware input lag spikes, the physics calculation remains accurate.

**OpenCV Improvements:**

* **Font Caching:** OCR no longer lazy loads on each OCR call, it's cached in memory at startup.
* **Viewport:** The state tracker uses a no-op interface by default and is only hooked up when Spring loads, preventing memory leaks during headless runs.

### Background Mode

This is probably the biggest Quality of Life improvement **You can now hide the RuneLite.**

I replaced the standard Java `Robot` capture with native Windows calls. This reads the window's device context directly and writes it into a buffer.

The bot can see and use the game window even if it is covered by another window. You can finally use your laptop for other things while the bot runs in the background!
> You can't minimise it, however you can maximise other things and cover it completely.

### New Actions Utility
You can now call `ItemDropper.dropAll(controller());` to drop all the items in your inventory.
There are also different dropping patterns to choose from, see the class.

## BREAKING CHANGES: 
- `VirtualMouseUtils.moveToPause()` & `VirtualMouseUtils.moveToAndOvershoot()` have been removed.
- There is no longer a "fastest" option in mouse.moveTo().

Author: StaticSweep

README.md

@@ -36,34 +36,40 @@ Due to the separation of domain, core and actions utilities it provides a greate
 ## Mouse input
 
 
-https://github.com/user-attachments/assets/a7a39096-6d65-4ff6-8843-f3661cc9a92d
+https://github.com/user-attachments/assets/1267df86-db5c-4189-8c8d-e2f5fe047cab
 
 
 ### - Remote input
 
 ChromaScape uses advanced remote input techniques to function as a virtual second mouse dedicated to the client window. Unlike traditional input methods, this approach never hijacks your physical mouse, so you can continue using your PC without interruption while the bot runs in the background.
 
-This is achieved through KInput, which provides utilities to send mouse and keyboard events directly into a client’s Java Canvas object. By design, it cannot click outside this canvas and requires the Process ID of the target client to operate.
+This is achieved through KInput.
 
-ChromaScape uses the [64-bit](https://github.com/ThatOneGuyScripts/KInput) supported version of KInput. The [original](https://github.com/Kasi-R/KInput) KInput source is also available for reference.
-
-Feel free to browse the code or build from source if you’d like to dive deeper into how it works.
+ChromaScape uses a slightly modified version of the [64-bit](https://github.com/ThatOneGuyScripts/KInput) supported version of KInput. The [original](https://github.com/Kasi-R/KInput) KInput source is also available for reference.
+There are instructions on how to build KInput from source within the third_party directory in `DEV_README.md`
 
 ### - Humanised mouse movement
-To further reduce bot detection risks, ChromaScape uses humanised mouse movement patterns that mimic real user behavior. Through a combination of multiple bezier paths, easing functions and the ability to overshoot/undershoot then recorrect - this produces surprisingly natural-looking behavior.
+To further reduce bot detection risks, ChromaScape uses an adapted version of [WindMouse](https://ben.land/post/2021/04/25/windmouse-human-mouse-movement/), a physics based calculation of gravity and wind to ensure pixel imperfections unlike bezier curves. WindMouse has been a successful staple within the community for over a decade, and provides exceedingly human mouse movements.
 
 ## Web-Based Control Panel
 
-The UI is built with Spring Boot and served locally. This gives you a powerful way to view logs, manage scripts, and extend functionality - all from a browser tab. It's fully customizable with HTML/CSS/JS, so power users can tweak or overhaul it without modifying core bot code or needing to worry about tight coupling.
+
+https://github.com/user-attachments/assets/b1c601e9-b58a-4a54-865b-739a25ddf898
+
+
+The UI is built with Spring Boot, a mature industry framework, and served locally. This gives you a powerful way to view logs, manage scripts, see the bot's sensor information, and extend functionality. It's fully customizable with basic HTML/CSS/JS, so power users can tweak or overhaul it without modifying core bot code or needing to worry about tight coupling.
+
+**Newly: you can now have runelite covered by other applications while the bot runs**
+> You can't minimise it or put it entirely offscreen yet, but you can open other applications and have it run in the background.
 
 ## Colour and Image Detection
 
 ### - Colour Picker
 This utility allows you to pick exact pixel colours directly from the screen. It’s useful for identifying precise colour values needed to detect specific game elements or interface components. The picker supports real-time sampling and stores these colours for use in detection routines. Inspired by ThatOneGuy's [BCD](https://github.com/ThatOneGuyScripts/BetterColorDetection)
 
-<img width="1298" height="751" alt="colourScreenshot" src="https://github.com/user-attachments/assets/b93eb66c-2a61-40ba-9abb-24fb0596d7b5" />
+<img width="1298" height="751" alt="Colour_Picker" src="https://github.com/user-attachments/assets/650deb75-97c1-46af-b2f7-414af9ce63e6" />
 
-> Note: You will need all other colours except your desired one to be black. (This is just an example to show colour filtering)
+> Note: You will need all other colours except your desired one to be black.
 
 ### - Colour Detection
 Using the colours obtained from the picker, the framework scans defined screen regions to find matching outlines or clusters of pixels. This process enables the bot to recognise in-game objects, UI elements, or indicators by their unique colour signatures. The detection logic is optimized to handle slight variations in colour due to lighting or graphical effects by allowing for a lower and upper range of colours.

build.gradle.kts

@@ -39,6 +39,7 @@ dependencies {
 
 	// Logging
 	implementation("org.springframework.boot:spring-boot-starter-log4j2")
+	annotationProcessor("org.apache.logging.log4j:log4j-core:2.24.3")
 
 	// Other libraries
 	implementation("com.github.kwhat:jnativehook:2.2.2")

config/checkstyle/checkstyle-suppressions.xml

@@ -9,4 +9,7 @@
     <suppress files=".*WindowHandler\.java" checks="MethodName"/>
     <suppress files=".*ScreenManager\.java" checks="MethodName"/>
     <suppress files=".*Similarity\.java" checks=".*"/>
+    <suppress files=".*WindMouse\.java" checks=".*"/>
+    <suppress files=".*TemplateMatching\.java" checks="AvoidStarImport"/>
+    <suppress files=".*ScreenManager\.java" checks="LocalVariableName"/>
 </suppressions>

gradlew

@@ -3,6 +3,9 @@
 import com.chromascape.controller.Controller;
 import com.chromascape.utils.core.runtime.HotkeyListener;
 import com.chromascape.utils.core.runtime.ScriptStoppedException;
+import com.chromascape.utils.core.state.BotState;
+import com.chromascape.utils.core.state.StateManager;
+import com.chromascape.utils.core.statistics.StatisticsManager;
 import java.util.concurrent.ThreadLocalRandom;
 import org.apache.logging.log4j.LogManager;
 import org.apache.logging.log4j.Logger;
@@ -42,9 +45,11 @@ public final void run() {
     scriptThread = Thread.currentThread();
     controller.init();
     hotkeyListener.start();
+    StatisticsManager.reset();
 
     try {
       while (running) {
+        StatisticsManager.incrementCycles();
         if (Thread.currentThread().isInterrupted()) {
           logger.info("Thread interrupted, exiting.");
           break;
@@ -55,6 +60,7 @@ public final void run() {
           logger.error("Cycle interrupted: {}", e.getMessage());
           break;
         } catch (Exception e) {
+          StateManager.setState(BotState.ERROR);
           logger.error("Exception in cycle: {}", e.getMessage());
           break;
         }
@@ -95,7 +101,8 @@ public void stop() {
    * @param ms the duration to sleep in milliseconds
    * @throws ScriptStoppedException if the thread is interrupted during sleep
    */
-  public static void waitMillis(long ms) throws ScriptStoppedException {
+  public static void waitMillis(long ms) {
+    StateManager.setState(BotState.WAITING);
     try {
       Thread.sleep(ms);
     } catch (InterruptedException e) {

src/main/java/com/chromascape/base/BaseScript.java

@@ -5,6 +5,7 @@
 import com.chromascape.utils.core.input.remoteinput.Kinput;
 import com.chromascape.utils.core.screen.window.ScreenManager;
 import com.chromascape.utils.core.screen.window.WindowHandler;
+import com.chromascape.utils.domain.ocr.Ocr;
 import com.chromascape.utils.domain.walker.Walker;
 import com.chromascape.utils.domain.zones.ZoneManager;
 import org.apache.logging.log4j.LogManager;
@@ -49,20 +50,36 @@ public Controller() {
    * internal state for running.
    */
   public void init() {
+    logger.info("Setting up Font masks...");
+    // Warmup: Pre-load common fonts
+    try {
+      Ocr.loadFont("Plain 11");
+      Ocr.loadFont("Plain 12");
+      Ocr.loadFont("Bold 12");
+    } catch (Exception e) {
+      logger.error("Failed to pre-load fonts during init: {}", e.getMessage());
+    }
+
+    logger.info("Setting up Remote Input Library...");
     // Obtain process ID of the target window to initialize input injection
     kinput = new Kinput(WindowHandler.getPid(WindowHandler.getTargetWindow()));
 
     // Initialize virtual input utilities with current window bounds and fullscreen status
+    logger.info("Initialising mouse and keyboard utils...");
     virtualMouseUtils = new VirtualMouseUtils(kinput, ScreenManager.getWindowBounds());
     virtualKeyboardUtils = new VirtualKeyboardUtils(kinput);
 
+    logger.info("Pre-loading and instantiating zones...");
     // Initialize zone management with fixed mode option
     zoneManager = new ZoneManager();
+    // Initialise gameView instead of LazyLoading, to improve startup overhead
+    zoneManager.getGameView();
 
     state = ControllerState.RUNNING;
 
     // Initialises a walker to provide the script with Walking functionality through the DAX API
     walker = new Walker(this);
+    logger.info("Controller State: {}", state);
   }
 
   /**
@@ -74,10 +91,46 @@ public void init() {
   public void shutdown() {
     mouse().getMouseOverlay().eraseOverlay();
     kinput.destroy();
+    if (!killKinput()) {
+      logger.warn("Kinput failed to destroy");
+    }
     state = ControllerState.STOPPED;
     logger.info("Shutting down");
   }
 
+  /**
+   * Uses the command prompt to forcibly delete KInput.dll and KInputCtrl.dll from the build
+   * directory. Effectively freeing up the program to rerun.
+   *
+   * @return {@code true} if successful, {@code false} if not.
+   */
+  public boolean killKinput() {
+    try {
+      String distPath = new java.io.File("build/dist").getAbsolutePath();
+      String kinputCtrl = "\"" + distPath + "\\KInputCtrl.dll\"";
+      String kinput = "\"" + distPath + "\\KInput.dll\"";
+
+      // Wait 2s, then force delete files.
+      // We use cmd /c start "" /B to ensure it runs detached/background if possible.
+      // Using ProcessBuilder to avoid Runtime.exec deprecation.
+      new ProcessBuilder(
+              "cmd.exe",
+              "/c",
+              "start",
+              "/MIN",
+              "cmd.exe",
+              "/c",
+              "timeout /t 2 /nobreak > NUL & del /f /q " + kinputCtrl + " " + kinput)
+          .start();
+
+      logger.info("Scheduled forced Kinput DLL cleanup in 2 seconds.");
+      return true;
+    } catch (Exception e) {
+      logger.error("Failed to schedule Kinput cleanup: {}", e.getMessage());
+      return false;
+    }
+  }
+
   /**
    * Provides access to the virtual mouse utility.
    *

src/main/java/com/chromascape/controller/Controller.java

@@ -2,8 +2,8 @@
 
 import com.chromascape.base.BaseScript;
 import com.chromascape.utils.actions.Idler;
+import com.chromascape.utils.actions.ItemDropper;
 import com.chromascape.utils.actions.PointSelector;
-import com.chromascape.utils.core.input.distribution.ClickDistribution;
 import com.chromascape.utils.core.screen.topology.TemplateMatching;
 import com.chromascape.utils.core.screen.window.ScreenManager;
 import java.awt.Point;
@@ -45,7 +45,7 @@ public DemoMiningScript() {
   @Override
   protected void cycle() {
     if (isInventoryFull()) {
-      dropIronOre();
+      ItemDropper.dropAll(controller());
     }
     clickOre();
     waitRandomMillis(800, 1000);
@@ -90,28 +90,4 @@ private boolean isInventoryFull() {
     }
     return false;
   }
-
-  /**
-   * Drops all iron ore in the inventory using shift-click.
-   *
-   * <p>Iterates through all 28 slots, clicking each one while the shift modifier is held down.
-   */
-  private void dropIronOre() {
-    try {
-      controller().keyboard().sendModifierKey(401, "shift");
-      waitRandomMillis(800, 1200);
-      for (int i = 0; i < 28; i++) {
-        Rectangle invSlot = controller().zones().getInventorySlots().get(i);
-        Point clickLoc = ClickDistribution.generateRandomPoint(invSlot);
-        controller().mouse().moveTo(clickLoc, "medium");
-        controller().mouse().leftClick();
-        waitRandomMillis(200, 650);
-      }
-      waitRandomMillis(600, 800);
-      controller().keyboard().sendModifierKey(402, "shift");
-    } catch (Exception e) {
-      logger.error(e);
-      logger.error(e.getStackTrace());
-    }
-  }
 }

src/main/java/com/chromascape/scripts/DemoMiningScript.java

@@ -36,11 +36,15 @@ public class Idler {
    * @param timeoutSeconds the maximum number of seconds to remain idle before continuing
    */
   public static void waitUntilIdle(BaseScript base, int timeoutSeconds) {
+    // Initial wait to prevent race condition to previous idle message.
+    BaseScript.waitMillis(600);
     BaseScript.checkInterrupted();
     try {
       Instant start = Instant.now();
       Instant deadline = start.plus(Duration.ofSeconds(timeoutSeconds));
       while (Instant.now().isBefore(deadline)) {
+        // Throttle wait to reduce lag, this is enough.
+        BaseScript.waitMillis(300);
         Rectangle latestMessage = base.controller().zones().getChatTabs().get("Latest Message");
         ColourObj red = ColourInstances.getByName("ChatRed");
         ColourObj black = ColourInstances.getByName("Black");