Can't use find-kedro

[dataengineerone]

I get a pretty crazy error when I try to do a fresh install.

virtualenv venv
source venv/bin/activate
pip install kedro
pip install kedro[pandas]
pip install find-kedro
kedro new find-project
cd find-project
find-kedro

Results in this guy:

Traceback (most recent call last):
  File "/Users/tamu/code/demo/find-kedro-example/venv/lib/python3.7/site-packages/find_kedro/core.py", line 250, in _import
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "venv/lib/python3.7/site-packages/kedro/templates/project/{{ cookiecutter.repo_name }}/src/{{ cookiecutter.python_package }}/pipelines/data_science/pipeline.py", line 37, in <module>
    from .nodes import predict, report_accuracy, train_model
ModuleNotFoundError: No module named 'pipeline'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/tamu/code/demo/find-kedro-example/venv/bin/find-kedro", line 8, in <module>
    sys.exit(cli())
  File "/Users/tamu/code/demo/find-kedro-example/venv/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/Users/tamu/code/demo/find-kedro-example/venv/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/Users/tamu/code/demo/find-kedro-example/venv/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/tamu/code/demo/find-kedro-example/venv/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/Users/tamu/code/demo/find-kedro-example/venv/lib/python3.7/site-packages/find_kedro/cli.py", line 89, in cli
    verbose=verbose,
  File "/Users/tamu/code/demo/find-kedro-example/venv/lib/python3.7/site-packages/find_kedro/core.py", line 55, in find_kedro
    modules[key] = _import(nodes_file, directory)
  File "/Users/tamu/code/demo/find-kedro-example/venv/lib/python3.7/site-packages/find_kedro/core.py", line 253, in _import
    str(path).replace(os.sep, ".").replace(".py", ""), verbose=verbose
  File "/Users/tamu/code/demo/find-kedro-example/venv/lib/python3.7/site-packages/find_kedro/core.py", line 272, in _use_importmodule
    mod = importlib.import_module(path)
  File "/Users/tamu/code/demo/find-kedro-example/venv/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'venv.libthon3'```

I think there must be some super weird python path issues, perhaps?

[WaylonWalker]

I replicated your process on windows with wsl with success, see below.. I have only used venv a few times, conda is my daily driver. If I remember right you need to use venv rather than virtualenv. I could be missing something. by kedro new find-project did you mean that you ran kedro new then filled in find-project?

I do notice that you have braces in your path. Did the cookie-cutter render fail? Does kedro run work?

python -m venv venv
source venv/bin/activate
pip install kedro
pip install kedro[pandas]

/mnt/c/temp via 🐍 v3.8.5 (venv)
❯ kedro --version
kedro, version 0.16.5

/mnt/c/temp via 🐍 v3.8.5 (venv)
❯ find-kedro --version
find-kedro, version 0.1.0

/mnt/c/temp via 🐍 v3.8.5 (venv)
❯ python --version
Python 3.8.5

/mnt/c/temp via 🐍 v3.8.5 (venv)
❯ kedro new
Project Name:
=============
Please enter a human readable name for your new project.
Spaces and punctuation are allowed.
 [New Kedro Project]: find-project

Repository Name:
================
Please enter a directory name for your new project repository.
Alphanumeric characters, hyphens and underscores are allowed.
Lowercase is recommended.
 [find-project]:

Python Package Name:
====================
Please enter a valid Python package name for your project package.
Alphanumeric characters and underscores are allowed.
Lowercase is recommended. Package name must start with a letter or underscore.
 [find_project]:

Generate Example Pipeline:
==========================
Do you want to generate an example pipeline in your project?
Good for first-time users. (default=N)
 [y/N]: y

/mnt/c/temp via 🐍 v3.8.5 (venv)
❯ cd find-project/

c/temp/find-project via 🐍 v3.8.5 (venv) took 2m26s
❯ find-kedro
{
  "__default__": []
}

[WaylonWalker]

I did make a repo to show how you might incorporate find-kedro into the default 0.16.5 template. https://github.com/WaylonWalker/find-project-0165/tags

Note that I messed up the commits and the change to hooks.py belongs in 2.0.0. I need to figure out if/how to move that.

Also note that you can

find-project on  main via 🐍 v3.8.5 via 🅒 kedro 
❯ kedro pipeline list
- __default__
- data_engineering.nodes
- data_science.nodes

If you try the find-kedro cli command by itself it currently throws a really confusing message if you don't tell it to start in src directory or lower.

find-project on  main via 🐍 v3.8.5 via 🅒 kedro 
❯ find-kedro -d src/find_project/pipelines
{
  "__default__": [
    "split_data([example_iris_data,params:example_test_data_ratio]) -> [example_test_x,example_test_y,example_train_x,example_train_y]",
    "train_model([example_train_x,example_train_y,parameters]) -> [example_model]",
    "predict([example_model,example_test_x]) -> [example_predictions]",
    "report_accuracy([example_predictions,example_test_y]) -> None"
  ],
  "data_engineering.nodes": [
    "split_data([example_iris_data,params:example_test_data_ratio]) -> [example_test_x,example_test_y,example_train_x,example_train_y]"
  ],
  "data_science.nodes": [
    "train_model([example_train_x,example_train_y,parameters]) -> [example_model]",
    "predict([example_model,example_test_x]) -> [example_predictions]",
    "report_accuracy([example_predictions,example_test_y]) -> None"
  ]
}