[BUG] Fugue execution engine is not recognized (ray, None, {}). You may need to register a parser for it.

[miguelfdag]

Minimal Code To Reproduce

import pandas as pd
from fugue import transform 

df = pd.DataFrame({"col1": [1,2,3,4], "col2": [1,2,3,4]})

# schema: *, col3:int
def add_cols(df:pd.DataFrame) -> pd.DataFrame:
    return df.assign(col3 = df['col1'] + df['col2'])

ray_df = transform(df, add_cols, engine="ray")
ray_df.show(5)

Describe the bug
Running the example in https://fugue-tutorials.readthedocs.io/tutorials/beginner/execution_engine.html?highlight=engine%20ray I get an error that ray is not a recognized engine.

Expected behavior
A clear and concise description of what you expected to happen.

Environment (please complete the following information):

• Backend: ray
• Backend version: ray==2.42.1
• Python version: Python 3.11.0
• OS: linux

[miguelfdag]

I have seen a previous ticket: #512 (comment) that mentioned the same problem. I have run pip install fugue[ray] but the same error appears:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/triad/utils/convert.py", line 151, in str_to_object
    return eval(expr, _globals, _locals)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 1, in <module>
NameError: name 'ray' is not defined

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/triad/utils/convert.py", line 187, in str_to_type
    obj = str_to_object(s, global_vars, local_vars)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/triad/utils/convert.py", line 161, in str_to_object
    raise ValueError(expr)
ValueError: ray

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/fugue/execution/factory.py", line 393, in parse_execution_engine
    return to_instance(engine, ExecutionEngine, kwargs=dict(conf=conf, **kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/triad/utils/convert.py", line 289, in to_instance
    t = to_type(s, expected_base_type, global_vars, local_vars)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/triad/utils/convert.py", line 249, in to_type
    return str_to_type(s, expected_base_type, global_vars, local_vars)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/triad/utils/convert.py", line 189, in str_to_type
    raise TypeError(f"{s} is not a type")
TypeError: ray is not a type

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/engine_user/ibp_core_engine/core_engines/engines/executor_copy.py", line 37, in <module>
    ray_df = transform(df, add_cols, engine='ray')
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/fugue/workflow/api.py", line 174, in transform
    dag.run(make_execution_engine(engine, conf=engine_conf, infer_by=[df]))
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/fugue/execution/factory.py", line 334, in make_execution_engine
    result = parse_execution_engine(engine, conf, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/triad/utils/dispatcher.py", line 256, in __call__
    return self.run_top(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/triad/utils/dispatcher.py", line 280, in run_top
    return list(itertools.islice(self.run(*args, **kwargs), 1))[0]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/triad/utils/dispatcher.py", line 273, in run
    yield self._func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/fugue/execution/factory.py", line 395, in parse_execution_engine
    raise FuguePluginsRegistrationError(
fugue.exceptions.FuguePluginsRegistrationError: Fugue execution engine is not recognized (ray, None, {}). You may need to register a parser for it.

[kvnkho]

Hi @miguelfdag , thanks for reporting. I'll try to reproduce this later today.

[goodwanghan]

So actually this issue is already fixed in 0.9.2.dev1
We will release 0.9.2 soon, for now you can install 0.9.2.dev1 to try.

Thanks!

[miguelfdag]

Thank you!