Error: UnboundLocalError: local variable 'results' referenced before assignment

[tangyipeng100]

Thank you for your excellent work. I have encountered the following problem and would appreciate your guidance on how to resolve it. Thank you.

(tts3) root@hy-hhht-test-jtfwjqr-vlm-001:/apprun/typ/compute-optimal-tts/src# bash scripts/run.sh --method cot --LM /data/ModelScopeModels/qwen/Qwen2-72B-Instruct-GPTQ-Int4 --RM dummy --width 1 --num_seq 1
usage: conda [-h] [-v] [--no-plugins] [-V] COMMAND ...
conda: error: argument COMMAND: invalid choice: 'activate' (choose from 'clean', 'compare', 'config', 'create', 'info', 'init', 'install', 'list', 'notices', 'package', 'remove', 'uninstall', 'rename', 'run', 'search', 'update', 'upgrade', 'build', 'content-trust', 'convert', 'debug', 'develop', 'doctor', 'index', 'inspect', 'metapackage', 'render', 'repoquery', 'skeleton', 'pack', 'token', 'repo', 'env', 'verify', 'server')
LM: /data/ModelScopeModels/qwen/Qwen2-72B-Instruct-GPTQ-Int4, RM: dummy, task: MATH, tree_max_width: 1, num_sequence: 1, question_parallel_num: 0
batch_size: 500, max_time: 3, n_gpus: 1, double_line_break: 1
CUDA_VISIBLE_DEVICES: 0, n_gpus: 1
GPU_LIST:
0 0
controller_ip: 0.0.0.0, controller_addr: http://0.0.0.0:10014
usage: conda [-h] [-v] [--no-plugins] [-V] COMMAND ...
conda: error: argument COMMAND: invalid choice: 'activate' (choose from 'clean', 'compare', 'config', 'create', 'info', 'init', 'install', 'list', 'notices', 'package', 'remove', 'uninstall', 'rename', 'run', 'search', 'update', 'upgrade', 'build', 'content-trust', 'convert', 'debug', 'develop', 'doctor', 'index', 'inspect', 'metapackage', 'render', 'repoquery', 'skeleton', 'verify', 'token', 'env', 'repo', 'server', 'pack')
CUDA_VISIBLE_DEVICES: 0
n_gpus: 1
GPU_LIST:
0 0
Session tts already exists. Killing it.
Wait 5 seconds ...
Starting workers
Reward worker 0 started on GPU 0 with port 10081, model: dummy
Policy worker 1 started on GPU 0 with port 10082, model: /data/ModelScopeModels/qwen/Qwen2-72B-Instruct-GPTQ-Int4
Wait 15 more seconds ...
Wait 50 seconds ...
Running cot evaluation ...
2025-02-17 22:15:16,903 INFO worker.py:1807 -- Started a local Ray instance. View the dashboard at 127.0.0.1:8265 
Auto set dir as /apprun/typ/compute-optimal-tts/src/output/MATH_cot/Qwen2-72B-Instruct-GPTQ-Int4/dummy/1_1_1
Loading MATH dataset
Batch size: 500, Max exist time: 3
Assign start at: 2025-02-17 22:15:19
Len: 500, Chosen idxs: [[0, 0], [1, 0], [2, 0], [3, 0], [4, 0], [5, 0], [6, 0], [7, 0], [8, 0], [9, 0], [10, 0], [11, 0], [12, 0], [13, 0], [14, 0], [15, 0], [16, 0], [17, 0], [18, 0], [19, 0], [20, 0], [21, 0], [22, 0], [23, 0], [24, 0], [25, 0], [26, 0], [27, 0], [28, 0], [29, 0], [30, 0], [31, 0], [32, 0], [33, 0], [34, 0], [35, 0], [36, 0], [37, 0], [38, 0], [39, 0], [40, 0], [41, 0], [42, 0], [43, 0], [44, 0], [45, 0], [46, 0], [47, 0], [48, 0], [49, 0], [50, 0], [51, 0], [52, 0], [53, 0], [54, 0], [55, 0], [56, 0], [57, 0], [58, 0], [59, 0], [60, 0], [61, 0], [62, 0], [63, 0], [64, 0], [65, 0], [66, 0], [67, 0], [68, 0], [69, 0], [70, 0], [71, 0], [72, 0], [73, 0], [74, 0], [75, 0], [76, 0], [77, 0], [78, 0], [79, 0], [80, 0], [81, 0], [82, 0], [83, 0], [84, 0], [85, 0], [86, 0], [87, 0], [88, 0], [89, 0], [90, 0], [91, 0], [92, 0], [93, 0], [94, 0], [95, 0], [96, 0], [97, 0], [98, 0], [99, 0], [100, 0], [101, 0], [102, 0], [103, 0], [104, 0], [105, 0], [106, 0], [107, 0], [108, 0], [109, 0], [110, 0], [111, 0], [112, 0], [113, 0], [114, 0], [115, 0], [116, 0], [117, 0], [118, 0], [119, 0], [120, 0], [121, 0], [122, 0], [123, 0], [124, 0], [125, 0], [126, 0], [127, 0], [128, 0], [129, 0], [130, 0], [131, 0], [132, 0], [133, 0], [134, 0], [135, 0], [136, 0], [137, 0], [138, 0], [139, 0], [140, 0], [141, 0], [142, 0], [143, 0], [144, 0], [145, 0], [146, 0], [147, 0], [148, 0], [149, 0], [150, 0], [151, 0], [152, 0], [153, 0], [154, 0], [155, 0], [156, 0], [157, 0], [158, 0], [159, 0], [160, 0], [161, 0], [162, 0], [163, 0], [164, 0], [165, 0], [166, 0], [167, 0], [168, 0], [169, 0], [170, 0], [171, 0], [172, 0], [173, 0], [174, 0], [175, 0], [176, 0], [177, 0], [178, 0], [179, 0], [180, 0], [181, 0], [182, 0], [183, 0], [184, 0], [185, 0], [186, 0], [187, 0], [188, 0], [189, 0], [190, 0], [191, 0], [192, 0], [193, 0], [194, 0], [195, 0], [196, 0], [197, 0], [198, 0], [199, 0], [200, 0], [201, 0], [202, 0], [203, 0], [204, 0], [205, 0], [206, 0], [207, 0], [208, 0], [209, 0], [210, 0], [211, 0], [212, 0], [213, 0], [214, 0], [215, 0], [216, 0], [217, 0], [218, 0], [219, 0], [220, 0], [221, 0], [222, 0], [223, 0], [224, 0], [225, 0], [226, 0], [227, 0], [228, 0], [229, 0], [230, 0], [231, 0], [232, 0], [233, 0], [234, 0], [235, 0], [236, 0], [237, 0], [238, 0], [239, 0], [240, 0], [241, 0], [242, 0], [243, 0], [244, 0], [245, 0], [246, 0], [247, 0], [248, 0], [249, 0], [250, 0], [251, 0], [252, 0], [253, 0], [254, 0], [255, 0], [256, 0], [257, 0], [258, 0], [259, 0], [260, 0], [261, 0], [262, 0], [263, 0], [264, 0], [265, 0], [266, 0], [267, 0], [268, 0], [269, 0], [270, 0], [271, 0], [272, 0], [273, 0], [274, 0], [275, 0], [276, 0], [277, 0], [278, 0], [279, 0], [280, 0], [281, 0], [282, 0], [283, 0], [284, 0], [285, 0], [286, 0], [287, 0], [288, 0], [289, 0], [290, 0], [291, 0], [292, 0], [293, 0], [294, 0], [295, 0], [296, 0], [297, 0], [298, 0], [299, 0], [300, 0], [301, 0], [302, 0], [303, 0], [304, 0], [305, 0], [306, 0], [307, 0], [308, 0], [309, 0], [310, 0], [311, 0], [312, 0], [313, 0], [314, 0], [315, 0], [316, 0], [317, 0], [318, 0], [319, 0], [320, 0], [321, 0], [322, 0], [323, 0], [324, 0], [325, 0], [326, 0], [327, 0], [328, 0], [329, 0], [330, 0], [331, 0], [332, 0], [333, 0], [334, 0], [335, 0], [336, 0], [337, 0], [338, 0], [339, 0], [340, 0], [341, 0], [342, 0], [343, 0], [344, 0], [345, 0], [346, 0], [347, 0], [348, 0], [349, 0], [350, 0], [351, 0], [352, 0], [353, 0], [354, 0], [355, 0], [356, 0], [357, 0], [358, 0], [359, 0], [360, 0], [361, 0], [362, 0], [363, 0], [364, 0], [365, 0], [366, 0], [367, 0], [368, 0], [369, 0], [370, 0], [371, 0], [372, 0], [373, 0], [374, 0], [375, 0], [376, 0], [377, 0], [378, 0], [379, 0], [380, 0], [381, 0], [382, 0], [383, 0], [384, 0], [385, 0], [386, 0], [387, 0], [388, 0], [389, 0], [390, 0], [391, 0], [392, 0], [393, 0], [394, 0], [395, 0], [396, 0], [397, 0], [398, 0], [399, 0], [400, 0], [401, 0], [402, 0], [403, 0], [404, 0], [405, 0], [406, 0], [407, 0], [408, 0], [409, 0], [410, 0], [411, 0], [412, 0], [413, 0], [414, 0], [415, 0], [416, 0], [417, 0], [418, 0], [419, 0], [420, 0], [421, 0], [422, 0], [423, 0], [424, 0], [425, 0], [426, 0], [427, 0], [428, 0], [429, 0], [430, 0], [431, 0], [432, 0], [433, 0], [434, 0], [435, 0], [436, 0], [437, 0], [438, 0], [439, 0], [440, 0], [441, 0], [442, 0], [443, 0], [444, 0], [445, 0], [446, 0], [447, 0], [448, 0], [449, 0], [450, 0], [451, 0], [452, 0], [453, 0], [454, 0], [455, 0], [456, 0], [457, 0], [458, 0], [459, 0], [460, 0], [461, 0], [462, 0], [463, 0], [464, 0], [465, 0], [466, 0], [467, 0], [468, 0], [469, 0], [470, 0], [471, 0], [472, 0], [473, 0], [474, 0], [475, 0], [476, 0], [477, 0], [478, 0], [479, 0], [480, 0], [481, 0], [482, 0], [483, 0], [484, 0], [485, 0], [486, 0], [487, 0], [488, 0], [489, 0], [490, 0], [491, 0], [492, 0], [493, 0], [494, 0], [495, 0], [496, 0], [497, 0], [498, 0], [499, 0]]
Assign end at: 2025-02-17 22:15:19, Time cost: 0.0 seconds
Len: 500, Chosen dict: {0: [0], 1: [0], 2: [0], 3: [0], 4: [0], 5: [0], 6: [0], 7: [0], 8: [0], 9: [0], 10: [0], 11: [0], 12: [0], 13: [0], 14: [0], 15: [0], 16: [0], 17: [0], 18: [0], 19: [0], 20: [0], 21: [0], 22: [0], 23: [0], 24: [0], 25: [0], 26: [0], 27: [0], 28: [0], 29: [0], 30: [0], 31: [0], 32: [0], 33: [0], 34: [0], 35: [0], 36: [0], 37: [0], 38: [0], 39: [0], 40: [0], 41: [0], 42: [0], 43: [0], 44: [0], 45: [0], 46: [0], 47: [0], 48: [0], 49: [0], 50: [0], 51: [0], 52: [0], 53: [0], 54: [0], 55: [0], 56: [0], 57: [0], 58: [0], 59: [0], 60: [0], 61: [0], 62: [0], 63: [0], 64: [0], 65: [0], 66: [0], 67: [0], 68: [0], 69: [0], 70: [0], 71: [0], 72: [0], 73: [0], 74: [0], 75: [0], 76: [0], 77: [0], 78: [0], 79: [0], 80: [0], 81: [0], 82: [0], 83: [0], 84: [0], 85: [0], 86: [0], 87: [0], 88: [0], 89: [0], 90: [0], 91: [0], 92: [0], 93: [0], 94: [0], 95: [0], 96: [0], 97: [0], 98: [0], 99: [0], 100: [0], 101: [0], 102: [0], 103: [0], 104: [0], 105: [0], 106: [0], 107: [0], 108: [0], 109: [0], 110: [0], 111: [0], 112: [0], 113: [0], 114: [0], 115: [0], 116: [0], 117: [0], 118: [0], 119: [0], 120: [0], 121: [0], 122: [0], 123: [0], 124: [0], 125: [0], 126: [0], 127: [0], 128: [0], 129: [0], 130: [0], 131: [0], 132: [0], 133: [0], 134: [0], 135: [0], 136: [0], 137: [0], 138: [0], 139: [0], 140: [0], 141: [0], 142: [0], 143: [0], 144: [0], 145: [0], 146: [0], 147: [0], 148: [0], 149: [0], 150: [0], 151: [0], 152: [0], 153: [0], 154: [0], 155: [0], 156: [0], 157: [0], 158: [0], 159: [0], 160: [0], 161: [0], 162: [0], 163: [0], 164: [0], 165: [0], 166: [0], 167: [0], 168: [0], 169: [0], 170: [0], 171: [0], 172: [0], 173: [0], 174: [0], 175: [0], 176: [0], 177: [0], 178: [0], 179: [0], 180: [0], 181: [0], 182: [0], 183: [0], 184: [0], 185: [0], 186: [0], 187: [0], 188: [0], 189: [0], 190: [0], 191: [0], 192: [0], 193: [0], 194: [0], 195: [0], 196: [0], 197: [0], 198: [0], 199: [0], 200: [0], 201: [0], 202: [0], 203: [0], 204: [0], 205: [0], 206: [0], 207: [0], 208: [0], 209: [0], 210: [0], 211: [0], 212: [0], 213: [0], 214: [0], 215: [0], 216: [0], 217: [0], 218: [0], 219: [0], 220: [0], 221: [0], 222: [0], 223: [0], 224: [0], 225: [0], 226: [0], 227: [0], 228: [0], 229: [0], 230: [0], 231: [0], 232: [0], 233: [0], 234: [0], 235: [0], 236: [0], 237: [0], 238: [0], 239: [0], 240: [0], 241: [0], 242: [0], 243: [0], 244: [0], 245: [0], 246: [0], 247: [0], 248: [0], 249: [0], 250: [0], 251: [0], 252: [0], 253: [0], 254: [0], 255: [0], 256: [0], 257: [0], 258: [0], 259: [0], 260: [0], 261: [0], 262: [0], 263: [0], 264: [0], 265: [0], 266: [0], 267: [0], 268: [0], 269: [0], 270: [0], 271: [0], 272: [0], 273: [0], 274: [0], 275: [0], 276: [0], 277: [0], 278: [0], 279: [0], 280: [0], 281: [0], 282: [0], 283: [0], 284: [0], 285: [0], 286: [0], 287: [0], 288: [0], 289: [0], 290: [0], 291: [0], 292: [0], 293: [0], 294: [0], 295: [0], 296: [0], 297: [0], 298: [0], 299: [0], 300: [0], 301: [0], 302: [0], 303: [0], 304: [0], 305: [0], 306: [0], 307: [0], 308: [0], 309: [0], 310: [0], 311: [0], 312: [0], 313: [0], 314: [0], 315: [0], 316: [0], 317: [0], 318: [0], 319: [0], 320: [0], 321: [0], 322: [0], 323: [0], 324: [0], 325: [0], 326: [0], 327: [0], 328: [0], 329: [0], 330: [0], 331: [0], 332: [0], 333: [0], 334: [0], 335: [0], 336: [0], 337: [0], 338: [0], 339: [0], 340: [0], 341: [0], 342: [0], 343: [0], 344: [0], 345: [0], 346: [0], 347: [0], 348: [0], 349: [0], 350: [0], 351: [0], 352: [0], 353: [0], 354: [0], 355: [0], 356: [0], 357: [0], 358: [0], 359: [0], 360: [0], 361: [0], 362: [0], 363: [0], 364: [0], 365: [0], 366: [0], 367: [0], 368: [0], 369: [0], 370: [0], 371: [0], 372: [0], 373: [0], 374: [0], 375: [0], 376: [0], 377: [0], 378: [0], 379: [0], 380: [0], 381: [0], 382: [0], 383: [0], 384: [0], 385: [0], 386: [0], 387: [0], 388: [0], 389: [0], 390: [0], 391: [0], 392: [0], 393: [0], 394: [0], 395: [0], 396: [0], 397: [0], 398: [0], 399: [0], 400: [0], 401: [0], 402: [0], 403: [0], 404: [0], 405: [0], 406: [0], 407: [0], 408: [0], 409: [0], 410: [0], 411: [0], 412: [0], 413: [0], 414: [0], 415: [0], 416: [0], 417: [0], 418: [0], 419: [0], 420: [0], 421: [0], 422: [0], 423: [0], 424: [0], 425: [0], 426: [0], 427: [0], 428: [0], 429: [0], 430: [0], 431: [0], 432: [0], 433: [0], 434: [0], 435: [0], 436: [0], 437: [0], 438: [0], 439: [0], 440: [0], 441: [0], 442: [0], 443: [0], 444: [0], 445: [0], 446: [0], 447: [0], 448: [0], 449: [0], 450: [0], 451: [0], 452: [0], 453: [0], 454: [0], 455: [0], 456: [0], 457: [0], 458: [0], 459: [0], 460: [0], 461: [0], 462: [0], 463: [0], 464: [0], 465: [0], 466: [0], 467: [0], 468: [0], 469: [0], 470: [0], 471: [0], 472: [0], 473: [0], 474: [0], 475: [0], 476: [0], 477: [0], 478: [0], 479: [0], 480: [0], 481: [0], 482: [0], 483: [0], 484: [0], 485: [0], 486: [0], 487: [0], 488: [0], 489: [0], 490: [0], 491: [0], 492: [0], 493: [0], 494: [0], 495: [0], 496: [0], 497: [0], 498: [0], 499: [0]}
(RemoteMathEvaluator pid=4062088) Error in _generate_fastchat: Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
  File "/apprun/typ/compute-optimal-tts/src/reason/evaluation/evaluate.py", line 350, in <module>
    returned_temp = parallel_evaluate_test_dataset(
  File "/apprun/typ/compute-optimal-tts/src/reason/evaluation/evaluate.py", line 194, in parallel_evaluate_test_dataset
    for i, (problem_inst, result, output) in enumerate(res_q):
  File "/data/anaconda3/envs/tts3/lib/python3.10/site-packages/ray/util/actor_pool.py", line 170, in get_generator
    yield self.get_next_unordered()
  File "/data/anaconda3/envs/tts3/lib/python3.10/site-packages/ray/util/actor_pool.py", line 370, in get_next_unordered
    return ray.get(future)
  File "/data/anaconda3/envs/tts3/lib/python3.10/site-packages/ray/_private/auto_init_hook.py", line 21, in auto_init_wrapper
    return fn(*args, **kwargs)
  File "/data/anaconda3/envs/tts3/lib/python3.10/site-packages/ray/_private/client_mode_hook.py", line 103, in wrapper
    return func(*args, **kwargs)
  File "/data/anaconda3/envs/tts3/lib/python3.10/site-packages/ray/_private/worker.py", line 2745, in get
    values, debugger_breakpoint = worker.get_objects(object_refs, timeout=timeout)
  File "/data/anaconda3/envs/tts3/lib/python3.10/site-packages/ray/_private/worker.py", line 901, in get_objects
    raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(UnboundLocalError): ray::RemoteMathEvaluator.evaluate_problem() (pid=4062088, ip=10.112.0.75, actor_id=bb121006524944978c47ca6801000000, repr=<reason.evaluation.evaluator.RemoteMathEvaluator object at 0x7fdce8d64700>)
  File "/apprun/typ/compute-optimal-tts/src/envs/base_env.py", line 256, in update_legal_actions
    result: ConcatedLMGenResult = self.llm_gen_fns[0](
  File "/apprun/typ/compute-optimal-tts/src/reason/inference/lm_call.py", line 60, in __call__
    return _generate_fastchat(
  File "/apprun/typ/compute-optimal-tts/src/reason/inference/text_generation.py", line 101, in _generate_fastchat
    output_token_lens = results["output_token_len"]
UnboundLocalError: local variable 'results' referenced before assignment

During handling of the above exception, another exception occurred:

ray::RemoteMathEvaluator.evaluate_problem() (pid=4062088, ip=10.112.0.75, actor_id=bb121006524944978c47ca6801000000, repr=<reason.evaluation.evaluator.RemoteMathEvaluator object at 0x7fdce8d64700>)
  File "/apprun/typ/compute-optimal-tts/src/reason/evaluation/evaluator.py", line 166, in evaluate_problem
    solution: SolutionOutput = solver_fn(problem_inst, self.lm_calls, self.rm_call)
  File "/apprun/typ/compute-optimal-tts/src/reason/evaluation/methods.py", line 89, in beam_search
    traj_list = search_tree.beam_search(env, config.beam_size, config.tree_max_depth, rm_call)
  File "/apprun/typ/compute-optimal-tts/src/reason/guided_search/tree.py", line 273, in beam_search
    _, info = simulate_env.reset(update_legal_action=True)
  File "/apprun/typ/compute-optimal-tts/src/envs/base_env.py", line 176, in reset
    self._legal_actions, api_completion_token = self.update_legal_actions(initial=True, force_update=True)
  File "/apprun/typ/compute-optimal-tts/src/envs/base_env.py", line 256, in update_legal_actions
    result: ConcatedLMGenResult = self.llm_gen_fns[0](
  File "/apprun/typ/compute-optimal-tts/src/reason/inference/lm_call.py", line 60, in __call__
    return _generate_fastchat(
  File "/apprun/typ/compute-optimal-tts/src/reason/inference/text_generation.py", line 101, in _generate_fastchat
    output_token_lens = results["output_token_len"]
UnboundLocalError: local variable 'results' referenced before assignment
(RemoteMathEvaluator pid=4062088) Error in _generate_fastchat: Expecting value: line 1 column 1 (char 0)
(RemoteMathEvaluator pid=4062082) Error in _generate_fastchat: Expecting value: line 1 column 1 (char 0) [repeated 11x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.)
(tts3) root@hy-hhht-test-jtfwjqr-vlm-001:/apprun/typ/compute-optimal-tts/src# bash scripts/run.sh --method best_of_n --LM /data/ModelScopeModels/qwen/Qwen2-72B-Instruct-GPTQ-Int4 --RM dummy --width 1 --num_seq 1 --num_q 256
usage: conda [-h] [-v] [--no-plugins] [-V] COMMAND ...
conda: error: argument COMMAND: invalid choice: 'activate' (choose from 'clean', 'compare', 'config', 'create', 'info', 'init', 'install', 'list', 'notices', 'package', 'remove', 'uninstall', 'rename', 'run', 'search', 'update', 'upgrade', 'build', 'content-trust', 'convert', 'debug', 'develop', 'doctor', 'index', 'inspect', 'metapackage', 'render', 'repoquery', 'skeleton', 'pack', 'token', 'env', 'repo', 'verify', 'server')
LM: /data/ModelScopeModels/qwen/Qwen2-72B-Instruct-GPTQ-Int4, RM: dummy, task: MATH, tree_max_width: 1, num_sequence: 1, question_parallel_num: 256
batch_size: 500, max_time: 3, n_gpus: 1, double_line_break: 1
CUDA_VISIBLE_DEVICES: 0, n_gpus: 1
GPU_LIST:
0 0
controller_ip: 0.0.0.0, controller_addr: http://0.0.0.0:10014
usage: conda [-h] [-v] [--no-plugins] [-V] COMMAND ...
conda: error: argument COMMAND: invalid choice: 'activate' (choose from 'clean', 'compare', 'config', 'create', 'info', 'init', 'install', 'list', 'notices', 'package', 'remove', 'uninstall', 'rename', 'run', 'search', 'update', 'upgrade', 'build', 'content-trust', 'convert', 'debug', 'develop', 'doctor', 'index', 'inspect', 'metapackage', 'render', 'repoquery', 'skeleton', 'token', 'verify', 'server', 'repo', 'pack', 'env')
CUDA_VISIBLE_DEVICES: 0
n_gpus: 1
GPU_LIST:
0 0
Session tts already exists. Killing it.
Wait 5 seconds ...
Starting workers
Reward worker 0 started on GPU 0 with port 10081, model: dummy
Policy worker 1 started on GPU 0 with port 10082, model: /data/ModelScopeModels/qwen/Qwen2-72B-Instruct-GPTQ-Int4
Wait 15 more seconds ...
Wait 50 seconds ...
Running best_of_n evaluation ...
2025-02-17 22:23:48,927 INFO worker.py:1807 -- Started a local Ray instance. View the dashboard at 127.0.0.1:8265 
Auto set dir as /apprun/typ/compute-optimal-tts/src/output/MATH_best_of_n/Qwen2-72B-Instruct-GPTQ-Int4/dummy/1_1_1
Loading MATH dataset
Batch size: 500, Max exist time: 3
Assign start at: 2025-02-17 22:23:52
Len: 500, Chosen idxs: [[0, 0], [1, 0], [2, 0], [3, 0], [4, 0], [5, 0], [6, 0], [7, 0], [8, 0], [9, 0], [10, 0], [11, 0], [12, 0], [13, 0], [14, 0], [15, 0], [16, 0], [17, 0], [18, 0], [19, 0], [20, 0], [21, 0], [22, 0], [23, 0], [24, 0], [25, 0], [26, 0], [27, 0], [28, 0], [29, 0], [30, 0], [31, 0], [32, 0], [33, 0], [34, 0], [35, 0], [36, 0], [37, 0], [38, 0], [39, 0], [40, 0], [41, 0], [42, 0], [43, 0], [44, 0], [45, 0], [46, 0], [47, 0], [48, 0], [49, 0], [50, 0], [51, 0], [52, 0], [53, 0], [54, 0], [55, 0], [56, 0], [57, 0], [58, 0], [59, 0], [60, 0], [61, 0], [62, 0], [63, 0], [64, 0], [65, 0], [66, 0], [67, 0], [68, 0], [69, 0], [70, 0], [71, 0], [72, 0], [73, 0], [74, 0], [75, 0], [76, 0], [77, 0], [78, 0], [79, 0], [80, 0], [81, 0], [82, 0], [83, 0], [84, 0], [85, 0], [86, 0], [87, 0], [88, 0], [89, 0], [90, 0], [91, 0], [92, 0], [93, 0], [94, 0], [95, 0], [96, 0], [97, 0], [98, 0], [99, 0], [100, 0], [101, 0], [102, 0], [103, 0], [104, 0], [105, 0], [106, 0], [107, 0], [108, 0], [109, 0], [110, 0], [111, 0], [112, 0], [113, 0], [114, 0], [115, 0], [116, 0], [117, 0], [118, 0], [119, 0], [120, 0], [121, 0], [122, 0], [123, 0], [124, 0], [125, 0], [126, 0], [127, 0], [128, 0], [129, 0], [130, 0], [131, 0], [132, 0], [133, 0], [134, 0], [135, 0], [136, 0], [137, 0], [138, 0], [139, 0], [140, 0], [141, 0], [142, 0], [143, 0], [144, 0], [145, 0], [146, 0], [147, 0], [148, 0], [149, 0], [150, 0], [151, 0], [152, 0], [153, 0], [154, 0], [155, 0], [156, 0], [157, 0], [158, 0], [159, 0], [160, 0], [161, 0], [162, 0], [163, 0], [164, 0], [165, 0], [166, 0], [167, 0], [168, 0], [169, 0], [170, 0], [171, 0], [172, 0], [173, 0], [174, 0], [175, 0], [176, 0], [177, 0], [178, 0], [179, 0], [180, 0], [181, 0], [182, 0], [183, 0], [184, 0], [185, 0], [186, 0], [187, 0], [188, 0], [189, 0], [190, 0], [191, 0], [192, 0], [193, 0], [194, 0], [195, 0], [196, 0], [197, 0], [198, 0], [199, 0], [200, 0], [201, 0], [202, 0], [203, 0], [204, 0], [205, 0], [206, 0], [207, 0], [208, 0], [209, 0], [210, 0], [211, 0], [212, 0], [213, 0], [214, 0], [215, 0], [216, 0], [217, 0], [218, 0], [219, 0], [220, 0], [221, 0], [222, 0], [223, 0], [224, 0], [225, 0], [226, 0], [227, 0], [228, 0], [229, 0], [230, 0], [231, 0], [232, 0], [233, 0], [234, 0], [235, 0], [236, 0], [237, 0], [238, 0], [239, 0], [240, 0], [241, 0], [242, 0], [243, 0], [244, 0], [245, 0], [246, 0], [247, 0], [248, 0], [249, 0], [250, 0], [251, 0], [252, 0], [253, 0], [254, 0], [255, 0], [256, 0], [257, 0], [258, 0], [259, 0], [260, 0], [261, 0], [262, 0], [263, 0], [264, 0], [265, 0], [266, 0], [267, 0], [268, 0], [269, 0], [270, 0], [271, 0], [272, 0], [273, 0], [274, 0], [275, 0], [276, 0], [277, 0], [278, 0], [279, 0], [280, 0], [281, 0], [282, 0], [283, 0], [284, 0], [285, 0], [286, 0], [287, 0], [288, 0], [289, 0], [290, 0], [291, 0], [292, 0], [293, 0], [294, 0], [295, 0], [296, 0], [297, 0], [298, 0], [299, 0], [300, 0], [301, 0], [302, 0], [303, 0], [304, 0], [305, 0], [306, 0], [307, 0], [308, 0], [309, 0], [310, 0], [311, 0], [312, 0], [313, 0], [314, 0], [315, 0], [316, 0], [317, 0], [318, 0], [319, 0], [320, 0], [321, 0], [322, 0], [323, 0], [324, 0], [325, 0], [326, 0], [327, 0], [328, 0], [329, 0], [330, 0], [331, 0], [332, 0], [333, 0], [334, 0], [335, 0], [336, 0], [337, 0], [338, 0], [339, 0], [340, 0], [341, 0], [342, 0], [343, 0], [344, 0], [345, 0], [346, 0], [347, 0], [348, 0], [349, 0], [350, 0], [351, 0], [352, 0], [353, 0], [354, 0], [355, 0], [356, 0], [357, 0], [358, 0], [359, 0], [360, 0], [361, 0], [362, 0], [363, 0], [364, 0], [365, 0], [366, 0], [367, 0], [368, 0], [369, 0], [370, 0], [371, 0], [372, 0], [373, 0], [374, 0], [375, 0], [376, 0], [377, 0], [378, 0], [379, 0], [380, 0], [381, 0], [382, 0], [383, 0], [384, 0], [385, 0], [386, 0], [387, 0], [388, 0], [389, 0], [390, 0], [391, 0], [392, 0], [393, 0], [394, 0], [395, 0], [396, 0], [397, 0], [398, 0], [399, 0], [400, 0], [401, 0], [402, 0], [403, 0], [404, 0], [405, 0], [406, 0], [407, 0], [408, 0], [409, 0], [410, 0], [411, 0], [412, 0], [413, 0], [414, 0], [415, 0], [416, 0], [417, 0], [418, 0], [419, 0], [420, 0], [421, 0], [422, 0], [423, 0], [424, 0], [425, 0], [426, 0], [427, 0], [428, 0], [429, 0], [430, 0], [431, 0], [432, 0], [433, 0], [434, 0], [435, 0], [436, 0], [437, 0], [438, 0], [439, 0], [440, 0], [441, 0], [442, 0], [443, 0], [444, 0], [445, 0], [446, 0], [447, 0], [448, 0], [449, 0], [450, 0], [451, 0], [452, 0], [453, 0], [454, 0], [455, 0], [456, 0], [457, 0], [458, 0], [459, 0], [460, 0], [461, 0], [462, 0], [463, 0], [464, 0], [465, 0], [466, 0], [467, 0], [468, 0], [469, 0], [470, 0], [471, 0], [472, 0], [473, 0], [474, 0], [475, 0], [476, 0], [477, 0], [478, 0], [479, 0], [480, 0], [481, 0], [482, 0], [483, 0], [484, 0], [485, 0], [486, 0], [487, 0], [488, 0], [489, 0], [490, 0], [491, 0], [492, 0], [493, 0], [494, 0], [495, 0], [496, 0], [497, 0], [498, 0], [499, 0]]
Assign end at: 2025-02-17 22:23:52, Time cost: 0.0 seconds
Len: 500, Chosen dict: {0: [0], 1: [0], 2: [0], 3: [0], 4: [0], 5: [0], 6: [0], 7: [0], 8: [0], 9: [0], 10: [0], 11: [0], 12: [0], 13: [0], 14: [0], 15: [0], 16: [0], 17: [0], 18: [0], 19: [0], 20: [0], 21: [0], 22: [0], 23: [0], 24: [0], 25: [0], 26: [0], 27: [0], 28: [0], 29: [0], 30: [0], 31: [0], 32: [0], 33: [0], 34: [0], 35: [0], 36: [0], 37: [0], 38: [0], 39: [0], 40: [0], 41: [0], 42: [0], 43: [0], 44: [0], 45: [0], 46: [0], 47: [0], 48: [0], 49: [0], 50: [0], 51: [0], 52: [0], 53: [0], 54: [0], 55: [0], 56: [0], 57: [0], 58: [0], 59: [0], 60: [0], 61: [0], 62: [0], 63: [0], 64: [0], 65: [0], 66: [0], 67: [0], 68: [0], 69: [0], 70: [0], 71: [0], 72: [0], 73: [0], 74: [0], 75: [0], 76: [0], 77: [0], 78: [0], 79: [0], 80: [0], 81: [0], 82: [0], 83: [0], 84: [0], 85: [0], 86: [0], 87: [0], 88: [0], 89: [0], 90: [0], 91: [0], 92: [0], 93: [0], 94: [0], 95: [0], 96: [0], 97: [0], 98: [0], 99: [0], 100: [0], 101: [0], 102: [0], 103: [0], 104: [0], 105: [0], 106: [0], 107: [0], 108: [0], 109: [0], 110: [0], 111: [0], 112: [0], 113: [0], 114: [0], 115: [0], 116: [0], 117: [0], 118: [0], 119: [0], 120: [0], 121: [0], 122: [0], 123: [0], 124: [0], 125: [0], 126: [0], 127: [0], 128: [0], 129: [0], 130: [0], 131: [0], 132: [0], 133: [0], 134: [0], 135: [0], 136: [0], 137: [0], 138: [0], 139: [0], 140: [0], 141: [0], 142: [0], 143: [0], 144: [0], 145: [0], 146: [0], 147: [0], 148: [0], 149: [0], 150: [0], 151: [0], 152: [0], 153: [0], 154: [0], 155: [0], 156: [0], 157: [0], 158: [0], 159: [0], 160: [0], 161: [0], 162: [0], 163: [0], 164: [0], 165: [0], 166: [0], 167: [0], 168: [0], 169: [0], 170: [0], 171: [0], 172: [0], 173: [0], 174: [0], 175: [0], 176: [0], 177: [0], 178: [0], 179: [0], 180: [0], 181: [0], 182: [0], 183: [0], 184: [0], 185: [0], 186: [0], 187: [0], 188: [0], 189: [0], 190: [0], 191: [0], 192: [0], 193: [0], 194: [0], 195: [0], 196: [0], 197: [0], 198: [0], 199: [0], 200: [0], 201: [0], 202: [0], 203: [0], 204: [0], 205: [0], 206: [0], 207: [0], 208: [0], 209: [0], 210: [0], 211: [0], 212: [0], 213: [0], 214: [0], 215: [0], 216: [0], 217: [0], 218: [0], 219: [0], 220: [0], 221: [0], 222: [0], 223: [0], 224: [0], 225: [0], 226: [0], 227: [0], 228: [0], 229: [0], 230: [0], 231: [0], 232: [0], 233: [0], 234: [0], 235: [0], 236: [0], 237: [0], 238: [0], 239: [0], 240: [0], 241: [0], 242: [0], 243: [0], 244: [0], 245: [0], 246: [0], 247: [0], 248: [0], 249: [0], 250: [0], 251: [0], 252: [0], 253: [0], 254: [0], 255: [0], 256: [0], 257: [0], 258: [0], 259: [0], 260: [0], 261: [0], 262: [0], 263: [0], 264: [0], 265: [0], 266: [0], 267: [0], 268: [0], 269: [0], 270: [0], 271: [0], 272: [0], 273: [0], 274: [0], 275: [0], 276: [0], 277: [0], 278: [0], 279: [0], 280: [0], 281: [0], 282: [0], 283: [0], 284: [0], 285: [0], 286: [0], 287: [0], 288: [0], 289: [0], 290: [0], 291: [0], 292: [0], 293: [0], 294: [0], 295: [0], 296: [0], 297: [0], 298: [0], 299: [0], 300: [0], 301: [0], 302: [0], 303: [0], 304: [0], 305: [0], 306: [0], 307: [0], 308: [0], 309: [0], 310: [0], 311: [0], 312: [0], 313: [0], 314: [0], 315: [0], 316: [0], 317: [0], 318: [0], 319: [0], 320: [0], 321: [0], 322: [0], 323: [0], 324: [0], 325: [0], 326: [0], 327: [0], 328: [0], 329: [0], 330: [0], 331: [0], 332: [0], 333: [0], 334: [0], 335: [0], 336: [0], 337: [0], 338: [0], 339: [0], 340: [0], 341: [0], 342: [0], 343: [0], 344: [0], 345: [0], 346: [0], 347: [0], 348: [0], 349: [0], 350: [0], 351: [0], 352: [0], 353: [0], 354: [0], 355: [0], 356: [0], 357: [0], 358: [0], 359: [0], 360: [0], 361: [0], 362: [0], 363: [0], 364: [0], 365: [0], 366: [0], 367: [0], 368: [0], 369: [0], 370: [0], 371: [0], 372: [0], 373: [0], 374: [0], 375: [0], 376: [0], 377: [0], 378: [0], 379: [0], 380: [0], 381: [0], 382: [0], 383: [0], 384: [0], 385: [0], 386: [0], 387: [0], 388: [0], 389: [0], 390: [0], 391: [0], 392: [0], 393: [0], 394: [0], 395: [0], 396: [0], 397: [0], 398: [0], 399: [0], 400: [0], 401: [0], 402: [0], 403: [0], 404: [0], 405: [0], 406: [0], 407: [0], 408: [0], 409: [0], 410: [0], 411: [0], 412: [0], 413: [0], 414: [0], 415: [0], 416: [0], 417: [0], 418: [0], 419: [0], 420: [0], 421: [0], 422: [0], 423: [0], 424: [0], 425: [0], 426: [0], 427: [0], 428: [0], 429: [0], 430: [0], 431: [0], 432: [0], 433: [0], 434: [0], 435: [0], 436: [0], 437: [0], 438: [0], 439: [0], 440: [0], 441: [0], 442: [0], 443: [0], 444: [0], 445: [0], 446: [0], 447: [0], 448: [0], 449: [0], 450: [0], 451: [0], 452: [0], 453: [0], 454: [0], 455: [0], 456: [0], 457: [0], 458: [0], 459: [0], 460: [0], 461: [0], 462: [0], 463: [0], 464: [0], 465: [0], 466: [0], 467: [0], 468: [0], 469: [0], 470: [0], 471: [0], 472: [0], 473: [0], 474: [0], 475: [0], 476: [0], 477: [0], 478: [0], 479: [0], 480: [0], 481: [0], 482: [0], 483: [0], 484: [0], 485: [0], 486: [0], 487: [0], 488: [0], 489: [0], 490: [0], 491: [0], 492: [0], 493: [0], 494: [0], 495: [0], 496: [0], 497: [0], 498: [0], 499: [0]}
(RemoteMathEvaluator pid=4087874) Error in _generate_fastchat: Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
  File "/apprun/typ/compute-optimal-tts/src/reason/evaluation/evaluate.py", line 350, in <module>
    returned_temp = parallel_evaluate_test_dataset(
  File "/apprun/typ/compute-optimal-tts/src/reason/evaluation/evaluate.py", line 194, in parallel_evaluate_test_dataset
    for i, (problem_inst, result, output) in enumerate(res_q):
  File "/data/anaconda3/envs/tts3/lib/python3.10/site-packages/ray/util/actor_pool.py", line 170, in get_generator
    yield self.get_next_unordered()
  File "/data/anaconda3/envs/tts3/lib/python3.10/site-packages/ray/util/actor_pool.py", line 370, in get_next_unordered
    return ray.get(future)
  File "/data/anaconda3/envs/tts3/lib/python3.10/site-packages/ray/_private/auto_init_hook.py", line 21, in auto_init_wrapper
    return fn(*args, **kwargs)
  File "/data/anaconda3/envs/tts3/lib/python3.10/site-packages/ray/_private/client_mode_hook.py", line 103, in wrapper
    return func(*args, **kwargs)
  File "/data/anaconda3/envs/tts3/lib/python3.10/site-packages/ray/_private/worker.py", line 2745, in get
    values, debugger_breakpoint = worker.get_objects(object_refs, timeout=timeout)
  File "/data/anaconda3/envs/tts3/lib/python3.10/site-packages/ray/_private/worker.py", line 901, in get_objects
    raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(UnboundLocalError): ray::RemoteMathEvaluator.evaluate_problem() (pid=4087874, ip=10.112.0.75, actor_id=17d2aa70359fe1a0a20a965501000000, repr=<reason.evaluation.evaluator.RemoteMathEvaluator object at 0x7f4440bd4700>)
  File "/apprun/typ/compute-optimal-tts/src/envs/base_env.py", line 256, in update_legal_actions
    result: ConcatedLMGenResult = self.llm_gen_fns[0](
  File "/apprun/typ/compute-optimal-tts/src/reason/inference/lm_call.py", line 60, in __call__
    return _generate_fastchat(
  File "/apprun/typ/compute-optimal-tts/src/reason/inference/text_generation.py", line 101, in _generate_fastchat
    output_token_lens = results["output_token_len"]
UnboundLocalError: local variable 'results' referenced before assignment

During handling of the above exception, another exception occurred:

ray::RemoteMathEvaluator.evaluate_problem() (pid=4087874, ip=10.112.0.75, actor_id=17d2aa70359fe1a0a20a965501000000, repr=<reason.evaluation.evaluator.RemoteMathEvaluator object at 0x7f4440bd4700>)
  File "/apprun/typ/compute-optimal-tts/src/reason/evaluation/evaluator.py", line 166, in evaluate_problem
    solution: SolutionOutput = solver_fn(problem_inst, self.lm_calls, self.rm_call)
  File "/apprun/typ/compute-optimal-tts/src/reason/evaluation/methods.py", line 89, in beam_search
    traj_list = search_tree.beam_search(env, config.beam_size, config.tree_max_depth, rm_call)
  File "/apprun/typ/compute-optimal-tts/src/reason/guided_search/tree.py", line 273, in beam_search
    _, info = simulate_env.reset(update_legal_action=True)
  File "/apprun/typ/compute-optimal-tts/src/envs/base_env.py", line 176, in reset
    self._legal_actions, api_completion_token = self.update_legal_actions(initial=True, force_update=True)
  File "/apprun/typ/compute-optimal-tts/src/envs/base_env.py", line 256, in update_legal_actions
    result: ConcatedLMGenResult = self.llm_gen_fns[0](
  File "/apprun/typ/compute-optimal-tts/src/reason/inference/lm_call.py", line 60, in __call__
    return _generate_fastchat(
  File "/apprun/typ/compute-optimal-tts/src/reason/inference/text_generation.py", line 101, in _generate_fastchat
    output_token_lens = results["output_token_len"]
UnboundLocalError: local variable 'results' referenced before assignment
(RemoteMathEvaluator pid=4087874) Error in _generate_fastchat: Expecting value: line 1 column 1 (char 0)
(RemoteMathEvaluator pid=4087874) Error in _generate_fastchat: Expecting value: line 1 column 1 (char 0)
(RemoteMathEvaluator pid=4087883) Error in _generate_fastchat: Expecting value: line 1 column 1 (char 0) [repeated 5x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.)

[nianbai006]

I have encountered the same problem

[RyanLiu112]

Hi, it looks like that the policy model may not be served correctly. We have updated the tmux installation instructions here. Please check whether the policy model is properly served through tmux.

[RyanLiu112]

If the policy model is running correctly, you should be able to observe these logs in the tmux window.

[ska278]

Hi, it looks like that the policy model may not be served correctly. We have updated the tmux installation instructions here. Please check whether the policy model is properly served through tmux.

Hi: I'm encountering the same problem. The above "Unbounded error..." problem occurs when we execute run.sh script per your instructions: bash scripts/run.sh --method cot --LM path/to/LM --RM dummy --width 1 --num_seq 1. This problem occurs irrespective of the LM chosen. Are we supposed to start a policy model with tmux in a separate window before executing run.sh? Please advise

[RyanLiu112]

Hi, it looks like that the policy model may not be served correctly. We have updated the tmux installation instructions here. Please check whether the policy model is properly served through tmux.

Hi: I'm encountering the same problem. The above "Unbounded error..." problem occurs when we execute run.sh script per your instructions: bash scripts/run.sh --method cot --LM path/to/LM --RM dummy --width 1 --num_seq 1. This problem occurs irrespective of the LM chosen. Are we supposed to start a policy model with tmux in a separate window before executing run.sh? Please advise

Thanks for your advice! Currently, I have implemented the model serving code in run.sh. I will reorganize the scripts to better separate the serving and execution components.

[ska278]

Hi, it looks like that the policy model may not be served correctly. We have updated the tmux installation instructions here. Please check whether the policy model is properly served through tmux.

Hi: I'm encountering the same problem. The above "Unbounded error..." problem occurs when we execute run.sh script per your instructions: bash scripts/run.sh --method cot --LM path/to/LM --RM dummy --width 1 --num_seq 1. This problem occurs irrespective of the LM chosen. Are we supposed to start a policy model with tmux in a separate window before executing run.sh? Please advise

Thanks for your advice! Currently, I have implemented the model serving code in run.sh. I will reorganize the scripts to better separate the serving and execution components.

Thanks! Can you also enable these scripts for the CPU?

[RyanLiu112]

Hi, it looks like that the policy model may not be served correctly. We have updated the tmux installation instructions here. Please check whether the policy model is properly served through tmux.

Hi: I'm encountering the same problem. The above "Unbounded error..." problem occurs when we execute run.sh script per your instructions: bash scripts/run.sh --method cot --LM path/to/LM --RM dummy --width 1 --num_seq 1. This problem occurs irrespective of the LM chosen. Are we supposed to start a policy model with tmux in a separate window before executing run.sh? Please advise

Thanks for your advice! Currently, I have implemented the model serving code in run.sh. I will reorganize the scripts to better separate the serving and execution components.

Thanks! Can you also enable these scripts for the CPU?

I have updated the code and the instructions in the README. Are you suggesting running policy models and PRMs on a pure CPU setup? I can attempt to implement this, but the inference speed may be significantly slower compared to GPU-accelerated execution.

[ska278]

Hi, it looks like that the policy model may not be served correctly. We have updated the tmux installation instructions here. Please check whether the policy model is properly served through tmux.

Hi: I'm encountering the same problem. The above "Unbounded error..." problem occurs when we execute run.sh script per your instructions: bash scripts/run.sh --method cot --LM path/to/LM --RM dummy --width 1 --num_seq 1. This problem occurs irrespective of the LM chosen. Are we supposed to start a policy model with tmux in a separate window before executing run.sh? Please advise

Thanks for your advice! Currently, I have implemented the model serving code in run.sh. I will reorganize the scripts to better separate the serving and execution components.

Thanks! Can you also enable these scripts for the CPU?

I have updated the code and the instructions in the README. Are you suggesting running policy models and PRMs on a pure CPU setup? I can attempt to implement this, but the inference speed may be significantly slower compared to GPU-accelerated execution.

Yes. I'm suggesting pure CPU setup. At least a single CPU and perhaps a multi-CPU setup that could provide a baseline.

[tangyipeng100]

Thank you for your prompt response and the repository update. I have implemented your changes; however, I am still encountering the following errors:

(tts3) root@hy-hhht-test-jtfwjqr-vlm-001:/apprun/typ/compute-optimal-tts/src# bash scripts/serve_gpu4.sh $POLICY_MODEL_PATH $VALUE_MODEL_PATH $HOST_ADDR $CONTROLLER_PORT $WORKER_BASE_PORT
usage: conda [-h] [-v] [--no-plugins] [-V] COMMAND ...
conda: error: argument COMMAND: invalid choice: 'activate' (choose from 'clean', 'compare', 'config', 'create', 'info', 'init', 'install', 'list', 'notices', 'package', 'remove', 'uninstall', 'rename', 'run', 'search', 'update', 'upgrade', 'build', 'content-trust', 'convert', 'debug', 'develop', 'doctor', 'index', 'inspect', 'metapackage', 'render', 'repoquery', 'skeleton', 'env', 'verify', 'pack', 'server', 'token', 'repo')
CUDA_VISIBLE_DEVICES: 0,1,2,3
n_gpus: 4
GPU_LIST:
0 1 2 3
Session tts already exists. Killing it.
Wait 5 seconds ...
Starting workers
Reward worker 0 started on GPU 0,1 with port 10081, model: /data/ModelScopeModels/AI-ModelScope/Skywork-o1-Open-PRM-Qwen-2___5-7B
Policy worker 1 started on GPU 2,3 with port 10082, model: /data/ModelScopeModels/LLM-Research/Llama-3___2-1B-Instruct
(tts3) root@hy-hhht-test-jtfwjqr-vlm-001:/apprun/typ/compute-optimal-tts/src# bash scripts/run.sh --method cot --LM $POLICY_MODEL_PATH --RM dummy --width 1 --num_seq 1
usage: conda [-h] [-v] [--no-plugins] [-V] COMMAND ...
conda: error: argument COMMAND: invalid choice: 'activate' (choose from 'clean', 'compare', 'config', 'create', 'info', 'init', 'install', 'list', 'notices', 'package', 'remove', 'uninstall', 'rename', 'run', 'search', 'update', 'upgrade', 'build', 'content-trust', 'convert', 'debug', 'develop', 'doctor', 'index', 'inspect', 'metapackage', 'render', 'repoquery', 'skeleton', 'pack', 'env', 'repo', 'server', 'verify', 'token')
LM: /data/ModelScopeModels/LLM-Research/Llama-3___2-1B-Instruct, RM: dummy, task: MATH, tree_max_width: 1, num_sequence: 1, question_parallel_num: 0
batch_size: 500, max_time: 3, n_gpus: 1, double_line_break: 1
CUDA_VISIBLE_DEVICES: 0, n_gpus: 1
GPU_LIST:
0 0
Running cot evaluation ...
2025-02-19 22:53:40,234 INFO worker.py:1807 -- Started a local Ray instance. View the dashboard at 127.0.0.1:8265 
Auto set dir as /apprun/typ/compute-optimal-tts/src/output/MATH_cot/Llama-3___2-1B-Instruct/dummy/1_1_1
Loading MATH dataset
Batch size: 500, Max exist time: 3
Assign start at: 2025-02-19 22:53:45
Len: 500, Chosen dict: {0: [0], 1: [0], 2: [0], 3: [0], 4: [0], 5: [0], 6: [0], 7: [0], 8: [0], 9: [0], 10: [0], 11: [0], 12: [0], 13: [0], 14: [0], 15: [0], 16: [0], 17: [0], 18: [0], 19: [0], 20: [0], 21: [0], 22: [0], 23: [0], 24: [0], 25: [0], 26: [0], 27: [0], 28: [0], 29: [0], 30: [0], 31: [0], 32: [0], 33: [0], 34: [0], 35: [0], 36: [0], 37: [0], 38: [0], 39: [0], 40: [0], 41: [0], 42: [0], 43: [0], 44: [0], 45: [0], 46: [0], 47: [0], 48: [0], 49: [0], 50: [0], 51: [0], 52: [0], 53: [0], 54: [0], 55: [0], 56: [0], 57: [0], 58: [0], 59: [0], 60: [0], 61: [0], 62: [0], 63: [0], 64: [0], 65: [0], 66: [0], 67: [0], 68: [0], 69: [0], 70: [0], 71: [0], 72: [0], 73: [0], 74: [0], 75: [0], 76: [0], 77: [0], 78: [0], 79: [0], 80: [0], 81: [0], 82: [0], 83: [0], 84: [0], 85: [0], 86: [0], 87: [0], 88: [0], 89: [0], 90: [0], 91: [0], 92: [0], 93: [0], 94: [0], 95: [0], 96: [0], 97: [0], 98: [0], 99: [0], 100: [0], 101: [0], 102: [0], 103: [0], 104: [0], 105: [0], 106: [0], 107: [0], 108: [0], 109: [0], 110: [0], 111: [0], 112: [0], 113: [0], 114: [0], 115: [0], 116: [0], 117: [0], 118: [0], 119: [0], 120: [0], 121: [0], 122: [0], 123: [0], 124: [0], 125: [0], 126: [0], 127: [0], 128: [0], 129: [0], 130: [0], 131: [0], 132: [0], 133: [0], 134: [0], 135: [0], 136: [0], 137: [0], 138: [0], 139: [0], 140: [0], 141: [0], 142: [0], 143: [0], 144: [0], 145: [0], 146: [0], 147: [0], 148: [0], 149: [0], 150: [0], 151: [0], 152: [0], 153: [0], 154: [0], 155: [0], 156: [0], 157: [0], 158: [0], 159: [0], 160: [0], 161: [0], 162: [0], 163: [0], 164: [0], 165: [0], 166: [0], 167: [0], 168: [0], 169: [0], 170: [0], 171: [0], 172: [0], 173: [0], 174: [0], 175: [0], 176: [0], 177: [0], 178: [0], 179: [0], 180: [0], 181: [0], 182: [0], 183: [0], 184: [0], 185: [0], 186: [0], 187: [0], 188: [0], 189: [0], 190: [0], 191: [0], 192: [0], 193: [0], 194: [0], 195: [0], 196: [0], 197: [0], 198: [0], 199: [0], 200: [0], 201: [0], 202: [0], 203: [0], 204: [0], 205: [0], 206: [0], 207: [0], 208: [0], 209: [0], 210: [0], 211: [0], 212: [0], 213: [0], 214: [0], 215: [0], 216: [0], 217: [0], 218: [0], 219: [0], 220: [0], 221: [0], 222: [0], 223: [0], 224: [0], 225: [0], 226: [0], 227: [0], 228: [0], 229: [0], 230: [0], 231: [0], 232: [0], 233: [0], 234: [0], 235: [0], 236: [0], 237: [0], 238: [0], 239: [0], 240: [0], 241: [0], 242: [0], 243: [0], 244: [0], 245: [0], 246: [0], 247: [0], 248: [0], 249: [0], 250: [0], 251: [0], 252: [0], 253: [0], 254: [0], 255: [0], 256: [0], 257: [0], 258: [0], 259: [0], 260: [0], 261: [0], 262: [0], 263: [0], 264: [0], 265: [0], 266: [0], 267: [0], 268: [0], 269: [0], 270: [0], 271: [0], 272: [0], 273: [0], 274: [0], 275: [0], 276: [0], 277: [0], 278: [0], 279: [0], 280: [0], 281: [0], 282: [0], 283: [0], 284: [0], 285: [0], 286: [0], 287: [0], 288: [0], 289: [0], 290: [0], 291: [0], 292: [0], 293: [0], 294: [0], 295: [0], 296: [0], 297: [0], 298: [0], 299: [0], 300: [0], 301: [0], 302: [0], 303: [0], 304: [0], 305: [0], 306: [0], 307: [0], 308: [0], 309: [0], 310: [0], 311: [0], 312: [0], 313: [0], 314: [0], 315: [0], 316: [0], 317: [0], 318: [0], 319: [0], 320: [0], 321: [0], 322: [0], 323: [0], 324: [0], 325: [0], 326: [0], 327: [0], 328: [0], 329: [0], 330: [0], 331: [0], 332: [0], 333: [0], 334: [0], 335: [0], 336: [0], 337: [0], 338: [0], 339: [0], 340: [0], 341: [0], 342: [0], 343: [0], 344: [0], 345: [0], 346: [0], 347: [0], 348: [0], 349: [0], 350: [0], 351: [0], 352: [0], 353: [0], 354: [0], 355: [0], 356: [0], 357: [0], 358: [0], 359: [0], 360: [0], 361: [0], 362: [0], 363: [0], 364: [0], 365: [0], 366: [0], 367: [0], 368: [0], 369: [0], 370: [0], 371: [0], 372: [0], 373: [0], 374: [0], 375: [0], 376: [0], 377: [0], 378: [0], 379: [0], 380: [0], 381: [0], 382: [0], 383: [0], 384: [0], 385: [0], 386: [0], 387: [0], 388: [0], 389: [0], 390: [0], 391: [0], 392: [0], 393: [0], 394: [0], 395: [0], 396: [0], 397: [0], 398: [0], 399: [0], 400: [0], 401: [0], 402: [0], 403: [0], 404: [0], 405: [0], 406: [0], 407: [0], 408: [0], 409: [0], 410: [0], 411: [0], 412: [0], 413: [0], 414: [0], 415: [0], 416: [0], 417: [0], 418: [0], 419: [0], 420: [0], 421: [0], 422: [0], 423: [0], 424: [0], 425: [0], 426: [0], 427: [0], 428: [0], 429: [0], 430: [0], 431: [0], 432: [0], 433: [0], 434: [0], 435: [0], 436: [0], 437: [0], 438: [0], 439: [0], 440: [0], 441: [0], 442: [0], 443: [0], 444: [0], 445: [0], 446: [0], 447: [0], 448: [0], 449: [0], 450: [0], 451: [0], 452: [0], 453: [0], 454: [0], 455: [0], 456: [0], 457: [0], 458: [0], 459: [0], 460: [0], 461: [0], 462: [0], 463: [0], 464: [0], 465: [0], 466: [0], 467: [0], 468: [0], 469: [0], 470: [0], 471: [0], 472: [0], 473: [0], 474: [0], 475: [0], 476: [0], 477: [0], 478: [0], 479: [0], 480: [0], 481: [0], 482: [0], 483: [0], 484: [0], 485: [0], 486: [0], 487: [0], 488: [0], 489: [0], 490: [0], 491: [0], 492: [0], 493: [0], 494: [0], 495: [0], 496: [0], 497: [0], 498: [0], 499: [0]}
Assign end at: 2025-02-19 22:53:45, Time cost: 0.0 seconds
(RemoteMathEvaluator pid=3330822) Error in _generate_fastchat: Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
  File "/apprun/typ/compute-optimal-tts/src/reason/evaluation/evaluate.py", line 350, in <module>
    returned_temp = parallel_evaluate_test_dataset(
  File "/apprun/typ/compute-optimal-tts/src/reason/evaluation/evaluate.py", line 197, in parallel_evaluate_test_dataset
    for i, (problem_inst, result, output) in enumerate(res_q):
  File "/data/anaconda3/envs/tts3/lib/python3.10/site-packages/ray/util/actor_pool.py", line 170, in get_generator
    yield self.get_next_unordered()
  File "/data/anaconda3/envs/tts3/lib/python3.10/site-packages/ray/util/actor_pool.py", line 370, in get_next_unordered
    return ray.get(future)
  File "/data/anaconda3/envs/tts3/lib/python3.10/site-packages/ray/_private/auto_init_hook.py", line 21, in auto_init_wrapper
    return fn(*args, **kwargs)
  File "/data/anaconda3/envs/tts3/lib/python3.10/site-packages/ray/_private/client_mode_hook.py", line 103, in wrapper
    return func(*args, **kwargs)
  File "/data/anaconda3/envs/tts3/lib/python3.10/site-packages/ray/_private/worker.py", line 2745, in get
    values, debugger_breakpoint = worker.get_objects(object_refs, timeout=timeout)
  File "/data/anaconda3/envs/tts3/lib/python3.10/site-packages/ray/_private/worker.py", line 901, in get_objects
    raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(UnboundLocalError): ray::RemoteMathEvaluator.evaluate_problem() (pid=3330822, ip=10.112.0.75, actor_id=f04572c12af018351da7366401000000, repr=<reason.evaluation.evaluator.RemoteMathEvaluator object at 0x7f5636670760>)
  File "/apprun/typ/compute-optimal-tts/src/envs/base_env.py", line 256, in update_legal_actions
    result: ConcatedLMGenResult = self.llm_gen_fns[0](
  File "/apprun/typ/compute-optimal-tts/src/reason/inference/lm_call.py", line 60, in __call__
    return _generate_fastchat(
  File "/apprun/typ/compute-optimal-tts/src/reason/inference/text_generation.py", line 101, in _generate_fastchat
    output_token_lens = results["output_token_len"]
UnboundLocalError: local variable 'results' referenced before assignment

During handling of the above exception, another exception occurred:

ray::RemoteMathEvaluator.evaluate_problem() (pid=3330822, ip=10.112.0.75, actor_id=f04572c12af018351da7366401000000, repr=<reason.evaluation.evaluator.RemoteMathEvaluator object at 0x7f5636670760>)
  File "/apprun/typ/compute-optimal-tts/src/reason/evaluation/evaluator.py", line 127, in evaluate_problem
    solution: SolutionOutput = solver_fn(problem_inst, self.lm_calls, self.rm_call)
  File "/apprun/typ/compute-optimal-tts/src/reason/evaluation/methods.py", line 89, in beam_search
    traj_list = search_tree.beam_search(env, config.beam_size, config.tree_max_depth, rm_call)
  File "/apprun/typ/compute-optimal-tts/src/reason/guided_search/tree.py", line 273, in beam_search
    _, info = simulate_env.reset(update_legal_action=True)
  File "/apprun/typ/compute-optimal-tts/src/envs/base_env.py", line 176, in reset
    self._legal_actions, api_completion_token = self.update_legal_actions(initial=True, force_update=True)
  File "/apprun/typ/compute-optimal-tts/src/envs/base_env.py", line 256, in update_legal_actions
    result: ConcatedLMGenResult = self.llm_gen_fns[0](
  File "/apprun/typ/compute-optimal-tts/src/reason/inference/lm_call.py", line 60, in __call__
    return _generate_fastchat(
  File "/apprun/typ/compute-optimal-tts/src/reason/inference/text_generation.py", line 101, in _generate_fastchat
    output_token_lens = results["output_token_len"]
UnboundLocalError: local variable 'results' referenced before assignment
(RemoteMathEvaluator pid=3330822) Error in _generate_fastchat: Expecting value: line 1 column 1 (char 0)
(RemoteMathEvaluator pid=3330822) Error in _generate_fastchat: Expecting value: line 1 column 1 (char 0)

If I specify the batch size in run.sh as 50, and I receive:

(tts3) root@hy-hhht-test-jtfwjqr-vlm-001:/apprun/typ/compute-optimal-tts/src# bash scripts/run.sh --method cot --LM $POLICY_MODEL_PATH --RM dummy --width 1 --num_seq 1
usage: conda [-h] [-v] [--no-plugins] [-V] COMMAND ...
conda: error: argument COMMAND: invalid choice: 'activate' (choose from 'clean', 'compare', 'config', 'create', 'info', 'init', 'install', 'list', 'notices', 'package', 'remove', 'uninstall', 'rename', 'run', 'search', 'update', 'upgrade', 'build', 'content-trust', 'convert', 'debug', 'develop', 'doctor', 'index', 'inspect', 'metapackage', 'render', 'repoquery', 'skeleton', 'verify', 'pack', 'token', 'repo', 'server', 'env')
LM: /data/ModelScopeModels/LLM-Research/Llama-3___2-1B-Instruct, RM: dummy, task: MATH, tree_max_width: 1, num_sequence: 1, question_parallel_num: 0
batch_size: 50, max_time: 3, n_gpus: 1, double_line_break: 1
CUDA_VISIBLE_DEVICES: 0, n_gpus: 1
GPU_LIST:
0 0
Running cot evaluation ...
2025-02-19 22:56:39,505 INFO worker.py:1807 -- Started a local Ray instance. View the dashboard at 127.0.0.1:8265 
Auto set dir as /apprun/typ/compute-optimal-tts/src/output/MATH_cot/Llama-3___2-1B-Instruct/dummy/1_1_1
Loading MATH dataset
Batch size: 50, Max exist time: 3
Assign start at: 2025-02-19 22:56:43
Len: 0, Chosen dict: {}
Assign end at: 2025-02-19 22:56:43, Time cost: 0.0 seconds

Could you please inform me of the procedure to obtain the final results? Thank you.

[sachith-gunasekara]

I am running into this issue as well. The error arises since the policy worker is not starting up correctly. I checked the tmux logs of the policy worker (tmux attach -t tts:policy_1) which reports that torchvision::nms does not exist (full log given below).

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1793, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
  File "/usr/local/lib/python3.10/importlib/__init__.py", line 126, in import_module        
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/usr/local/lib/python3.10/site-packages/transformers/processing_utils.py", line 33, in <module>
    from .image_utils import ChannelDimension, is_valid_image, is_vision_available
  File "/usr/local/lib/python3.10/site-packages/transformers/image_utils.py", line 59, in <module>
    from torchvision.transforms import InterpolationMode
  File "/usr/local/lib/python3.10/site-packages/torchvision/__init__.py", line 10, in <module>
    from torchvision import _meta_registrations, datasets, io, models, ops, transforms, utils  # usort:skip
  File "/usr/local/lib/python3.10/site-packages/torchvision/_meta_registrations.py", line 164, in <module>
    def meta_nms(dets, scores, iou_threshold):
  File "/usr/local/lib/python3.10/site-packages/torch/library.py", line 654, in register    
    use_lib._register_fake(op_name, func, _stacklevel=stacklevel + 1)
  File "/usr/local/lib/python3.10/site-packages/torch/library.py", line 154, in _register_fake
    handle = entry.abstract_impl.register(func_to_register, source)
  File "/usr/local/lib/python3.10/site-packages/torch/_library/abstract_impl.py", line 31, in register
    if torch._C._dispatch_has_kernel_for_dispatch_key(self.qualname, "Meta"):
RuntimeError: operator torchvision::nms does not exist

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/root/src/reason/llm_service/workers/vllm_worker.py", line 15, in <module>
    from vllm import AsyncLLMEngine
  File "/usr/local/lib/python3.10/site-packages/vllm/__init__.py", line 3, in <module>      
    from vllm.engine.arg_utils import AsyncEngineArgs, EngineArgs
  File "/usr/local/lib/python3.10/site-packages/vllm/engine/arg_utils.py", line 11, in <module>
    from vllm.config import (CacheConfig, ConfigFormat, DecodingConfig,
  File "/usr/local/lib/python3.10/site-packages/vllm/config.py", line 14, in <module>       
    from vllm.model_executor.layers.quantization import QUANTIZATION_METHODS
  File "/usr/local/lib/python3.10/site-packages/vllm/model_executor/__init__.py", line 3, in <module>
    from vllm.model_executor.sampling_metadata import (SamplingMetadata,
  File "/usr/local/lib/python3.10/site-packages/vllm/model_executor/sampling_metadata.py", line 8, in <module>
    from vllm.sequence import (VLLM_TOKEN_ID_ARRAY_TYPE, SequenceData,
  File "/usr/local/lib/python3.10/site-packages/vllm/sequence.py", line 16, in <module>     
    from vllm.inputs import SingletonInputs, SingletonInputsAdapter
  File "/usr/local/lib/python3.10/site-packages/vllm/inputs/__init__.py", line 7, in <module>
    from .registry import (DummyData, InputContext, InputProcessingContext,
  File "/usr/local/lib/python3.10/site-packages/vllm/inputs/registry.py", line 8, in <module>
    from transformers import PretrainedConfig, ProcessorMixin
  File "<frozen importlib._bootstrap>", line 1075, in _handle_fromlist
  File "/usr/local/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1781, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/usr/local/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1795, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.processing_utils because of the following error (look up to see its traceback):
operator torchvision::nms does not exist

@tangyipeng100 the error you see as local variable 'results' referenced before assignment is actually raised from reason.inference.text_generation.py _generate_fastchat() function. ie. Since the policy worker is not starting correctly, the /worker_generate endpoint is unavailable and hence results variable is not being initialized, exception occurs.

try:
    response = requests.post(worker_addr + "/worker_generate", headers=headers, json=gen_params, stream=True)
    results = response.json()
except Exception as e:
    print(f'Error in _generate_fastchat: {e}')

output_token_lens = results["output_token_len"]
cum_logps = results["cumulative_logprob"]

[sachith-gunasekara]

@RyanLiu112 What is the exact cuda version you ran this on? I suspect this traces to the specific cuda version.

[RyanLiu112]

Thank you for your prompt response and the repository update. I have implemented your changes; however, I am still encountering the following errors:

(tts3) root@hy-hhht-test-jtfwjqr-vlm-001:/apprun/typ/compute-optimal-tts/src# bash scripts/serve_gpu4.sh $POLICY_MODEL_PATH $VALUE_MODEL_PATH $HOST_ADDR $CONTROLLER_PORT $WORKER_BASE_PORT
usage: conda [-h] [-v] [--no-plugins] [-V] COMMAND ...
conda: error: argument COMMAND: invalid choice: 'activate' (choose from 'clean', 'compare', 'config', 'create', 'info', 'init', 'install', 'list', 'notices', 'package', 'remove', 'uninstall', 'rename', 'run', 'search', 'update', 'upgrade', 'build', 'content-trust', 'convert', 'debug', 'develop', 'doctor', 'index', 'inspect', 'metapackage', 'render', 'repoquery', 'skeleton', 'env', 'verify', 'pack', 'server', 'token', 'repo')
CUDA_VISIBLE_DEVICES: 0,1,2,3
n_gpus: 4
GPU_LIST:
0 1 2 3
Session tts already exists. Killing it.
Wait 5 seconds ...
Starting workers
Reward worker 0 started on GPU 0,1 with port 10081, model: /data/ModelScopeModels/AI-ModelScope/Skywork-o1-Open-PRM-Qwen-2___5-7B
Policy worker 1 started on GPU 2,3 with port 10082, model: /data/ModelScopeModels/LLM-Research/Llama-3___2-1B-Instruct
(tts3) root@hy-hhht-test-jtfwjqr-vlm-001:/apprun/typ/compute-optimal-tts/src# bash scripts/run.sh --method cot --LM $POLICY_MODEL_PATH --RM dummy --width 1 --num_seq 1
usage: conda [-h] [-v] [--no-plugins] [-V] COMMAND ...
conda: error: argument COMMAND: invalid choice: 'activate' (choose from 'clean', 'compare', 'config', 'create', 'info', 'init', 'install', 'list', 'notices', 'package', 'remove', 'uninstall', 'rename', 'run', 'search', 'update', 'upgrade', 'build', 'content-trust', 'convert', 'debug', 'develop', 'doctor', 'index', 'inspect', 'metapackage', 'render', 'repoquery', 'skeleton', 'pack', 'env', 'repo', 'server', 'verify', 'token')
LM: /data/ModelScopeModels/LLM-Research/Llama-3___2-1B-Instruct, RM: dummy, task: MATH, tree_max_width: 1, num_sequence: 1, question_parallel_num: 0
batch_size: 500, max_time: 3, n_gpus: 1, double_line_break: 1
CUDA_VISIBLE_DEVICES: 0, n_gpus: 1
GPU_LIST:
0 0
Running cot evaluation ...
2025-02-19 22:53:40,234 INFO worker.py:1807 -- Started a local Ray instance. View the dashboard at 127.0.0.1:8265
Auto set dir as /apprun/typ/compute-optimal-tts/src/output/MATH_cot/Llama-3___2-1B-Instruct/dummy/1_1_1
Loading MATH dataset
Batch size: 500, Max exist time: 3
Assign start at: 2025-02-19 22:53:45
Len: 500, Chosen dict: {0: [0], 1: [0], 2: [0], 3: [0], 4: [0], 5: [0], 6: [0], 7: [0], 8: [0], 9: [0], 10: [0], 11: [0], 12: [0], 13: [0], 14: [0], 15: [0], 16: [0], 17: [0], 18: [0], 19: [0], 20: [0], 21: [0], 22: [0], 23: [0], 24: [0], 25: [0], 26: [0], 27: [0], 28: [0], 29: [0], 30: [0], 31: [0], 32: [0], 33: [0], 34: [0], 35: [0], 36: [0], 37: [0], 38: [0], 39: [0], 40: [0], 41: [0], 42: [0], 43: [0], 44: [0], 45: [0], 46: [0], 47: [0], 48: [0], 49: [0], 50: [0], 51: [0], 52: [0], 53: [0], 54: [0], 55: [0], 56: [0], 57: [0], 58: [0], 59: [0], 60: [0], 61: [0], 62: [0], 63: [0], 64: [0], 65: [0], 66: [0], 67: [0], 68: [0], 69: [0], 70: [0], 71: [0], 72: [0], 73: [0], 74: [0], 75: [0], 76: [0], 77: [0], 78: [0], 79: [0], 80: [0], 81: [0], 82: [0], 83: [0], 84: [0], 85: [0], 86: [0], 87: [0], 88: [0], 89: [0], 90: [0], 91: [0], 92: [0], 93: [0], 94: [0], 95: [0], 96: [0], 97: [0], 98: [0], 99: [0], 100: [0], 101: [0], 102: [0], 103: [0], 104: [0], 105: [0], 106: [0], 107: [0], 108: [0], 109: [0], 110: [0], 111: [0], 112: [0], 113: [0], 114: [0], 115: [0], 116: [0], 117: [0], 118: [0], 119: [0], 120: [0], 121: [0], 122: [0], 123: [0], 124: [0], 125: [0], 126: [0], 127: [0], 128: [0], 129: [0], 130: [0], 131: [0], 132: [0], 133: [0], 134: [0], 135: [0], 136: [0], 137: [0], 138: [0], 139: [0], 140: [0], 141: [0], 142: [0], 143: [0], 144: [0], 145: [0], 146: [0], 147: [0], 148: [0], 149: [0], 150: [0], 151: [0], 152: [0], 153: [0], 154: [0], 155: [0], 156: [0], 157: [0], 158: [0], 159: [0], 160: [0], 161: [0], 162: [0], 163: [0], 164: [0], 165: [0], 166: [0], 167: [0], 168: [0], 169: [0], 170: [0], 171: [0], 172: [0], 173: [0], 174: [0], 175: [0], 176: [0], 177: [0], 178: [0], 179: [0], 180: [0], 181: [0], 182: [0], 183: [0], 184: [0], 185: [0], 186: [0], 187: [0], 188: [0], 189: [0], 190: [0], 191: [0], 192: [0], 193: [0], 194: [0], 195: [0], 196: [0], 197: [0], 198: [0], 199: [0], 200: [0], 201: [0], 202: [0], 203: [0], 204: [0], 205: [0], 206: [0], 207: [0], 208: [0], 209: [0], 210: [0], 211: [0], 212: [0], 213: [0], 214: [0], 215: [0], 216: [0], 217: [0], 218: [0], 219: [0], 220: [0], 221: [0], 222: [0], 223: [0], 224: [0], 225: [0], 226: [0], 227: [0], 228: [0], 229: [0], 230: [0], 231: [0], 232: [0], 233: [0], 234: [0], 235: [0], 236: [0], 237: [0], 238: [0], 239: [0], 240: [0], 241: [0], 242: [0], 243: [0], 244: [0], 245: [0], 246: [0], 247: [0], 248: [0], 249: [0], 250: [0], 251: [0], 252: [0], 253: [0], 254: [0], 255: [0], 256: [0], 257: [0], 258: [0], 259: [0], 260: [0], 261: [0], 262: [0], 263: [0], 264: [0], 265: [0], 266: [0], 267: [0], 268: [0], 269: [0], 270: [0], 271: [0], 272: [0], 273: [0], 274: [0], 275: [0], 276: [0], 277: [0], 278: [0], 279: [0], 280: [0], 281: [0], 282: [0], 283: [0], 284: [0], 285: [0], 286: [0], 287: [0], 288: [0], 289: [0], 290: [0], 291: [0], 292: [0], 293: [0], 294: [0], 295: [0], 296: [0], 297: [0], 298: [0], 299: [0], 300: [0], 301: [0], 302: [0], 303: [0], 304: [0], 305: [0], 306: [0], 307: [0], 308: [0], 309: [0], 310: [0], 311: [0], 312: [0], 313: [0], 314: [0], 315: [0], 316: [0], 317: [0], 318: [0], 319: [0], 320: [0], 321: [0], 322: [0], 323: [0], 324: [0], 325: [0], 326: [0], 327: [0], 328: [0], 329: [0], 330: [0], 331: [0], 332: [0], 333: [0], 334: [0], 335: [0], 336: [0], 337: [0], 338: [0], 339: [0], 340: [0], 341: [0], 342: [0], 343: [0], 344: [0], 345: [0], 346: [0], 347: [0], 348: [0], 349: [0], 350: [0], 351: [0], 352: [0], 353: [0], 354: [0], 355: [0], 356: [0], 357: [0], 358: [0], 359: [0], 360: [0], 361: [0], 362: [0], 363: [0], 364: [0], 365: [0], 366: [0], 367: [0], 368: [0], 369: [0], 370: [0], 371: [0], 372: [0], 373: [0], 374: [0], 375: [0], 376: [0], 377: [0], 378: [0], 379: [0], 380: [0], 381: [0], 382: [0], 383: [0], 384: [0], 385: [0], 386: [0], 387: [0], 388: [0], 389: [0], 390: [0], 391: [0], 392: [0], 393: [0], 394: [0], 395: [0], 396: [0], 397: [0], 398: [0], 399: [0], 400: [0], 401: [0], 402: [0], 403: [0], 404: [0], 405: [0], 406: [0], 407: [0], 408: [0], 409: [0], 410: [0], 411: [0], 412: [0], 413: [0], 414: [0], 415: [0], 416: [0], 417: [0], 418: [0], 419: [0], 420: [0], 421: [0], 422: [0], 423: [0], 424: [0], 425: [0], 426: [0], 427: [0], 428: [0], 429: [0], 430: [0], 431: [0], 432: [0], 433: [0], 434: [0], 435: [0], 436: [0], 437: [0], 438: [0], 439: [0], 440: [0], 441: [0], 442: [0], 443: [0], 444: [0], 445: [0], 446: [0], 447: [0], 448: [0], 449: [0], 450: [0], 451: [0], 452: [0], 453: [0], 454: [0], 455: [0], 456: [0], 457: [0], 458: [0], 459: [0], 460: [0], 461: [0], 462: [0], 463: [0], 464: [0], 465: [0], 466: [0], 467: [0], 468: [0], 469: [0], 470: [0], 471: [0], 472: [0], 473: [0], 474: [0], 475: [0], 476: [0], 477: [0], 478: [0], 479: [0], 480: [0], 481: [0], 482: [0], 483: [0], 484: [0], 485: [0], 486: [0], 487: [0], 488: [0], 489: [0], 490: [0], 491: [0], 492: [0], 493: [0], 494: [0], 495: [0], 496: [0], 497: [0], 498: [0], 499: [0]}
Assign end at: 2025-02-19 22:53:45, Time cost: 0.0 seconds
(RemoteMathEvaluator pid=3330822) Error in _generate_fastchat: Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
File "/apprun/typ/compute-optimal-tts/src/reason/evaluation/evaluate.py", line 350, in
returned_temp = parallel_evaluate_test_dataset(
File "/apprun/typ/compute-optimal-tts/src/reason/evaluation/evaluate.py", line 197, in parallel_evaluate_test_dataset
for i, (problem_inst, result, output) in enumerate(res_q):
File "/data/anaconda3/envs/tts3/lib/python3.10/site-packages/ray/util/actor_pool.py", line 170, in get_generator
yield self.get_next_unordered()
File "/data/anaconda3/envs/tts3/lib/python3.10/site-packages/ray/util/actor_pool.py", line 370, in get_next_unordered
return ray.get(future)
File "/data/anaconda3/envs/tts3/lib/python3.10/site-packages/ray/_private/auto_init_hook.py", line 21, in auto_init_wrapper
return fn(*args, **kwargs)
File "/data/anaconda3/envs/tts3/lib/python3.10/site-packages/ray/_private/client_mode_hook.py", line 103, in wrapper
return func(*args, **kwargs)
File "/data/anaconda3/envs/tts3/lib/python3.10/site-packages/ray/_private/worker.py", line 2745, in get
values, debugger_breakpoint = worker.get_objects(object_refs, timeout=timeout)
File "/data/anaconda3/envs/tts3/lib/python3.10/site-packages/ray/_private/worker.py", line 901, in get_objects
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(UnboundLocalError): ray::RemoteMathEvaluator.evaluate_problem() (pid=3330822, ip=10.112.0.75, actor_id=f04572c12af018351da7366401000000, repr=<reason.evaluation.evaluator.RemoteMathEvaluator object at 0x7f5636670760>)
File "/apprun/typ/compute-optimal-tts/src/envs/base_env.py", line 256, in update_legal_actions
result: ConcatedLMGenResult = self.llm_gen_fns[0](
File "/apprun/typ/compute-optimal-tts/src/reason/inference/lm_call.py", line 60, in call
return _generate_fastchat(
File "/apprun/typ/compute-optimal-tts/src/reason/inference/text_generation.py", line 101, in _generate_fastchat
output_token_lens = results["output_token_len"]
UnboundLocalError: local variable 'results' referenced before assignment

During handling of the above exception, another exception occurred:

ray::RemoteMathEvaluator.evaluate_problem() (pid=3330822, ip=10.112.0.75, actor_id=f04572c12af018351da7366401000000, repr=<reason.evaluation.evaluator.RemoteMathEvaluator object at 0x7f5636670760>)
File "/apprun/typ/compute-optimal-tts/src/reason/evaluation/evaluator.py", line 127, in evaluate_problem
solution: SolutionOutput = solver_fn(problem_inst, self.lm_calls, self.rm_call)
File "/apprun/typ/compute-optimal-tts/src/reason/evaluation/methods.py", line 89, in beam_search
traj_list = search_tree.beam_search(env, config.beam_size, config.tree_max_depth, rm_call)
File "/apprun/typ/compute-optimal-tts/src/reason/guided_search/tree.py", line 273, in beam_search
_, info = simulate_env.reset(update_legal_action=True)
File "/apprun/typ/compute-optimal-tts/src/envs/base_env.py", line 176, in reset
self._legal_actions, api_completion_token = self.update_legal_actions(initial=True, force_update=True)
File "/apprun/typ/compute-optimal-tts/src/envs/base_env.py", line 256, in update_legal_actions
result: ConcatedLMGenResult = self.llm_gen_fns[0](
File "/apprun/typ/compute-optimal-tts/src/reason/inference/lm_call.py", line 60, in call
return _generate_fastchat(
File "/apprun/typ/compute-optimal-tts/src/reason/inference/text_generation.py", line 101, in _generate_fastchat
output_token_lens = results["output_token_len"]
UnboundLocalError: local variable 'results' referenced before assignment
(RemoteMathEvaluator pid=3330822) Error in _generate_fastchat: Expecting value: line 1 column 1 (char 0)
(RemoteMathEvaluator pid=3330822) Error in _generate_fastchat: Expecting value: line 1 column 1 (char 0)
If I specify the batch size in run.sh as 50, and I receive:

(tts3) root@hy-hhht-test-jtfwjqr-vlm-001:/apprun/typ/compute-optimal-tts/src# bash scripts/run.sh --method cot --LM $POLICY_MODEL_PATH --RM dummy --width 1 --num_seq 1
usage: conda [-h] [-v] [--no-plugins] [-V] COMMAND ...
conda: error: argument COMMAND: invalid choice: 'activate' (choose from 'clean', 'compare', 'config', 'create', 'info', 'init', 'install', 'list', 'notices', 'package', 'remove', 'uninstall', 'rename', 'run', 'search', 'update', 'upgrade', 'build', 'content-trust', 'convert', 'debug', 'develop', 'doctor', 'index', 'inspect', 'metapackage', 'render', 'repoquery', 'skeleton', 'verify', 'pack', 'token', 'repo', 'server', 'env')
LM: /data/ModelScopeModels/LLM-Research/Llama-3___2-1B-Instruct, RM: dummy, task: MATH, tree_max_width: 1, num_sequence: 1, question_parallel_num: 0
batch_size: 50, max_time: 3, n_gpus: 1, double_line_break: 1
CUDA_VISIBLE_DEVICES: 0, n_gpus: 1
GPU_LIST:
0 0
Running cot evaluation ...
2025-02-19 22:56:39,505 INFO worker.py:1807 -- Started a local Ray instance. View the dashboard at 127.0.0.1:8265
Auto set dir as /apprun/typ/compute-optimal-tts/src/output/MATH_cot/Llama-3___2-1B-Instruct/dummy/1_1_1
Loading MATH dataset
Batch size: 50, Max exist time: 3
Assign start at: 2025-02-19 22:56:43
Len: 0, Chosen dict: {}
Assign end at: 2025-02-19 22:56:43, Time cost: 0.0 seconds
Could you please inform me of the procedure to obtain the final results? Thank you.

It looks that max_time is too large, please try to set max_time=1.

[RyanLiu112]

@RyanLiu112 What is the exact cuda version you ran this on? I suspect this traces to the specific cuda version.

Hi @sachith-gunasekara , my cuda version is 12.1.

[sachith-gunasekara]

imeError: Failed to import transformers.processing_utils because of the following error (look up to see its traceback):
operator torchvision::nms does not exist

Thank you for your quick response @RyanLiu112. I'll try with that exact same version as well. I still weren't able to fix this torchvision::nms does not exist issue though.

[RyanLiu112]

I am running into this issue as well. The error arises since the policy worker is not starting up correctly. I checked the tmux logs of the policy worker (tmux attach -t tts:policy_1) which reports that torchvision::nms does not exist (full log given below).

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1793, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
  File "/usr/local/lib/python3.10/importlib/__init__.py", line 126, in import_module        
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/usr/local/lib/python3.10/site-packages/transformers/processing_utils.py", line 33, in <module>
    from .image_utils import ChannelDimension, is_valid_image, is_vision_available
  File "/usr/local/lib/python3.10/site-packages/transformers/image_utils.py", line 59, in <module>
    from torchvision.transforms import InterpolationMode
  File "/usr/local/lib/python3.10/site-packages/torchvision/__init__.py", line 10, in <module>
    from torchvision import _meta_registrations, datasets, io, models, ops, transforms, utils  # usort:skip
  File "/usr/local/lib/python3.10/site-packages/torchvision/_meta_registrations.py", line 164, in <module>
    def meta_nms(dets, scores, iou_threshold):
  File "/usr/local/lib/python3.10/site-packages/torch/library.py", line 654, in register    
    use_lib._register_fake(op_name, func, _stacklevel=stacklevel + 1)
  File "/usr/local/lib/python3.10/site-packages/torch/library.py", line 154, in _register_fake
    handle = entry.abstract_impl.register(func_to_register, source)
  File "/usr/local/lib/python3.10/site-packages/torch/_library/abstract_impl.py", line 31, in register
    if torch._C._dispatch_has_kernel_for_dispatch_key(self.qualname, "Meta"):
RuntimeError: operator torchvision::nms does not exist

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/root/src/reason/llm_service/workers/vllm_worker.py", line 15, in <module>
    from vllm import AsyncLLMEngine
  File "/usr/local/lib/python3.10/site-packages/vllm/__init__.py", line 3, in <module>      
    from vllm.engine.arg_utils import AsyncEngineArgs, EngineArgs
  File "/usr/local/lib/python3.10/site-packages/vllm/engine/arg_utils.py", line 11, in <module>
    from vllm.config import (CacheConfig, ConfigFormat, DecodingConfig,
  File "/usr/local/lib/python3.10/site-packages/vllm/config.py", line 14, in <module>       
    from vllm.model_executor.layers.quantization import QUANTIZATION_METHODS
  File "/usr/local/lib/python3.10/site-packages/vllm/model_executor/__init__.py", line 3, in <module>
    from vllm.model_executor.sampling_metadata import (SamplingMetadata,
  File "/usr/local/lib/python3.10/site-packages/vllm/model_executor/sampling_metadata.py", line 8, in <module>
    from vllm.sequence import (VLLM_TOKEN_ID_ARRAY_TYPE, SequenceData,
  File "/usr/local/lib/python3.10/site-packages/vllm/sequence.py", line 16, in <module>     
    from vllm.inputs import SingletonInputs, SingletonInputsAdapter
  File "/usr/local/lib/python3.10/site-packages/vllm/inputs/__init__.py", line 7, in <module>
    from .registry import (DummyData, InputContext, InputProcessingContext,
  File "/usr/local/lib/python3.10/site-packages/vllm/inputs/registry.py", line 8, in <module>
    from transformers import PretrainedConfig, ProcessorMixin
  File "<frozen importlib._bootstrap>", line 1075, in _handle_fromlist
  File "/usr/local/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1781, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/usr/local/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1795, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.processing_utils because of the following error (look up to see its traceback):
operator torchvision::nms does not exist

@tangyipeng100 the error you see as local variable 'results' referenced before assignment is actually raised from reason.inference.text_generation.py _generate_fastchat() function. ie. Since the policy worker is not starting correctly, the /worker_generate endpoint is unavailable and hence results variable is not being initialized, exception occurs.

try:
    response = requests.post(worker_addr + "/worker_generate", headers=headers, json=gen_params, stream=True)
    results = response.json()
except Exception as e:
    print(f'Error in _generate_fastchat: {e}')

output_token_lens = results["output_token_len"]
cum_logps = results["cumulative_logprob"]

I haven't encountered this issue. My original conda environment doesn't even include torchvision since TTS doesn't require it. Perhaps you could try uninstalling torchvision?