Hint use agent (#316)

* fixes

* add new deps

* use external embedding service in task hints retrieval

* gpt5 fixes

* first cut

* update

* add event listeners and launcher

* Add codegen step-wise recoder agent

* adding task hints to generic agent

* fix repeated llm configs

* load env vars in codegen agent

* same hints retrieval for both generic and tooluse agents

* filter out current task hints if needed

* fix llm config, add gpt-5

* fix

* pass new flag and fix db path passing issue

* fix goal text

* fix current task hints exclusion

* remove old reqs

* remove recorder from that brach

* log task errors

* expore agentlabxray

* remove commented old chunk

* share xray only when env flag present

* Add StepWiseQueriesPrompt for enhanced query handling in GenericAgent

* update hinting agent retrieval

* stepwise hint retrieval

* added shrink method

* (wip) refactor hinting index

* (wip) clean up prompt file

* add scripts to run generic and hinter agents, update tmlr config for hinter

* move HintsSource to separate hinting file

* update hinter agent and prompt

* fix prompt for task hint

* undo changes to tmlr config

* update hinter agent

* formatting

* bug fix hint retrieval

* improve launch script

* get queries only for step level hint

* Add webarenalite to agentlab loop.py

* update stepwise hint queries prompt

* fix exc logging

* non empty instruction

* allow less then max hint queries

* add generic agent gpt5-nano config

* make ray available on toolkit

* check that hints db exists

* Fix assignment of queries_for_hints variable

* Improve generic agent hinter (#309)

* Make LLM retreival topic index selection more robust

* add new flag to skip hints with the current goal in the hint source t… (#310)

* add new flag to skip hints with the current goal in the hint source traces

* Rename generic agent hinter to hint_use_agent (#311)

* rename generic_agent_hinter to hint_use_agent for clarity

* Add deprecation warning and module alias for generic_agent_hinter

* improve module aliasing for submodules

* Add todo rename agent name

* black

* bugfix: check for hint_db only when use_task_hint is true.

* fix: address missing initialization and correct args reference in choose_hints method

* black

* bugfix: skip HintSource init if use_task_hint is false

* Fix incorrect references for docs retrieval hinter agent (#313)

* address comments

* format

* Add Environment Variable for Ray port (#315)

* add env variable for ray port

* document env variables

* undo removed llm_config

* undo unnessary change

* add missing default values for hint prompt flags

* black

* update names in scripts

* use default prompt in hintSource for Tool Use agent

* remove experiment scripts

---------

Co-authored-by: Oleh Shliazhko <[email protected]>
Co-authored-by: Hadi Nekoei <[email protected]>
Co-authored-by: Oleh Shliazhko <[email protected]>
Co-authored-by: recursix <[email protected]>
Co-authored-by: Patrice Bechard <[email protected]>
Co-authored-by: Hadi Nekoei <[email protected]>
Co-authored-by: Patrice Bechard <[email protected]>

Author: 5 people

README.md

@@ -278,13 +278,15 @@ dynamic benchmarks.
 
 ## Variables
 Here's a list of relevant env. variables that are used by AgentLab:
-- `OPEAI_API_KEY` which is used by default for OpenAI LLMs.
+- `OPENAI_API_KEY` which is used by default for OpenAI LLMs.
 - `AZURE_OPENAI_API_KEY`, used by default for AzureOpenAI LLMs.
 - `AZURE_OPENAI_ENDPOINT` to specify your Azure endpoint.
 - `OPENAI_API_VERSION` for the Azure API.
 - `OPENROUTER_API_KEY` for the Openrouter API
 - `AGENTLAB_EXP_ROOT`, desired path for your experiments to be stored, defaults to `~/agentlab-results`.
 - `AGENTXRAY_SHARE_GRADIO`, which prompts AgentXRay to open a public tunnel on launch.
+- `RAY_PUBLIC_DASHBOARD` (true / false), used to specify whether the ray dashboard should be made publicly accessible (`0.0.0.0`) or not (`127.0.0.1`).
+- `RAY_DASHBOARD_PORT` (int), used to specify the port on which the ray dashboard should be accessible.
 
 ## Misc
 

src/agentlab/agents/generic_agent_hinter/__init__.py

@@ -0,0 +1,19 @@
+import importlib, sys, warnings
+
+OLD = __name__
+NEW = "agentlab.agents.hint_use_agent"
+SUBS = ("agent_configs", "generic_agent_prompt", "generic_agent", "tmlr_config")
+
+warnings.warn(
+    f"{OLD} is renamed to {NEW}. {OLD} will be removed in future",
+    DeprecationWarning,
+    stacklevel=2,
+)
+
+# Alias the top-level
+new_mod = importlib.import_module(NEW)
+sys.modules[OLD] = new_mod
+
+# Alias known submodules
+for sub in SUBS:
+    sys.modules[f"{OLD}.{sub}"] = importlib.import_module(f"{NEW}.{sub}")

src/agentlab/agents/hint_use_agent/__init__.py

@@ -0,0 +1,54 @@
+"""
+Baseline agent for all ServiceNow papers
+
+This module contains the GenericAgent class, which is the baseline agent for all ServiceNow papers. \
+It is a simple agent that can be ran OOB on all BrowserGym environments. It is also shipped with \
+a few configurations that can be used to run it on different environments.
+"""
+
+from .agent_configs import (
+    AGENT_3_5,
+    AGENT_8B,
+    AGENT_37_SONNET,
+    AGENT_CLAUDE_SONNET_35,
+    AGENT_CLAUDE_SONNET_35_VISION,
+    AGENT_CUSTOM,
+    AGENT_GPT5_MINI,
+    AGENT_GPT5_NANO,
+    AGENT_LLAMA3_70B,
+    AGENT_LLAMA4_17B_INSTRUCT,
+    AGENT_LLAMA31_70B,
+    CHAT_MODEL_ARGS_DICT,
+    RANDOM_SEARCH_AGENT,
+    AGENT_4o,
+    AGENT_4o_MINI,
+    AGENT_4o_MINI_VISION,
+    AGENT_4o_VISION,
+    AGENT_o1_MINI,
+    AGENT_o3_MINI,
+    FLAGS_GPT_4o,
+    GenericAgentArgs,
+)
+from .generic_agent import GenericAgent, GenericAgentArgs
+
+__all__ = [
+    "AGENT_3_5",
+    "AGENT_4o",
+    "AGENT_4o_MINI",
+    "AGENT_4o_VISION",
+    "AGENT_o3_MINI",
+    "AGENT_o1_MINI",
+    "AGENT_LLAMA4_17B_INSTRUCT",
+    "AGENT_LLAMA3_70B",
+    "AGENT_LLAMA31_70B",
+    "AGENT_8B",
+    "RANDOM_SEARCH_AGENT",
+    "AGENT_CUSTOM",
+    "AGENT_CLAUDE_SONNET_35",
+    "AGENT_37_SONNET",
+    "AGENT_4o_VISION",
+    "AGENT_4o_MINI_VISION",
+    "AGENT_CLAUDE_SONNET_35_VISION",
+    "AGENT_GPT5_MINI",
+    "AGENT_GPT5_NANO",
+]