ModuleNotFoundError: No module named '_pyIdlak_txp' when installing REST server

[ronggong]

I am following the documentation https://idlak.readthedocs.io/en/latest/rest_server.html#install-server to install REST server. When I run flask run, I got an error:

ModuleNotFoundError: No module named '_pyIdlak_txp'

Before running this command, I succeeded in compiling pyIdlak by typing make in src/pyIdlak folder.

The entire error message:

Error: While importing "runserver", an ImportError was raised:

Traceback (most recent call last):
  File "../src/pyIdlak/txp/pyIdlak_txp.py", line 14, in swig_import_helper
    return importlib.import_module(mname)
  File "/Users/ronggong/Documents_using/github/idlak/idlak-server/venv/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'pyIdlak.txp._pyIdlak_txp'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/ronggong/Documents_using/github/idlak/idlak-server/venv/lib/python3.7/site-packages/flask/cli.py", line 235, in locate_app
    __import__(module_name)
  File "/Users/ronggong/Documents_using/github/idlak/idlak-server/runserver.py", line 3, in <module>
    app = create_app('config.ini')
  File "/Users/ronggong/Documents_using/github/idlak/idlak-server/app/__init__.py", line 45, in create_app
    from app import models, endpoints, reqlogging       # noqa
  File "/Users/ronggong/Documents_using/github/idlak/idlak-server/app/endpoints/__init__.py", line 1, in <module>
    from app.endpoints import user, auth, voice, speech, language
  File "/Users/ronggong/Documents_using/github/idlak/idlak-server/app/endpoints/speech.py", line 20, in <module>
    from pyIdlak import TangleVoice         # noqa
  File "../src/pyIdlak/__init__.py", line 18, in <module>
    from . import txp
  File "../src/pyIdlak/txp/__init__.py", line 19, in <module>
    from . import pyIdlak_txp as c_api
  File "../src/pyIdlak/txp/pyIdlak_txp.py", line 17, in <module>
    _pyIdlak_txp = swig_import_helper()
  File "../src/pyIdlak/txp/pyIdlak_txp.py", line 16, in swig_import_helper
    return importlib.import_module('_pyIdlak_txp')
  File "/Users/ronggong/Documents_using/github/idlak/idlak-server/venv/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named '_pyIdlak_txp'

[dabraude]

are you able to import pyIdalk from the base of the source directory?

[ronggong]

I don't think so. I tried running the tests in src/pyidlak, also have the same import problem.

[dabraude]

can you see if there is a problem if you run make in idlak/src/pyIdlak/txp maybe there was a problem there that the make didn't report

[ronggong]

I ran make again, and no error has been reported. But I still got the import error.

Here is the tree structure of idlak/src/pyIdlak/txp. It seems both _pyIdlak_txp.a and _pyIdlak_txp.dylib have been compiled.

├── Makefile
├── __init__.py
├── __pycache__
│   ├── __init__.cpython-37.pyc
│   └── pyIdlak_txp.cpython-37.pyc
├── _pyIdlak_txp.a
├── _pyIdlak_txp.dylib -> /Users/ronggong/Documents_using/github/idlak/src/lib/lib_pyIdlak_txp.dylib
├── idargparse.py
├── lib_pyIdlak_txp.dylib
├── modulefactory.py
├── normaliser
│   ├── __init__.py
│   └── normaliser.py
├── postlex
│   ├── __init__.py
│   └── postlex.py
├── pyIdlak_txp.i
├── pyIdlak_txp.py
├── pyIdlak_txp_wrap.cc
├── pyIdlak_txp_wrap.o
├── python-txp-api.cc
├── python-txp-api.h
├── python-txp-api.o
├── pytxplib.py
└── xmldoc.py

[dabraude]

how about importing in the pyIdlak_txp directory

i.e (on my machine IDLAK is where I cloned idlak).
cd $IDLAK/src/pyIdlak/txp/
python3 -c "import pyIdlak_txp"

It might be something in the virtual environment that is causing the problem, but I have 3.6 instead of 3.7 so I'm hoping it's nothing to do with that.

[dabraude]

I think I see what could be the problem:
on line 19 in idlak-server/app/endpoints/speech.py can you change the import path to be the Idlak source directory?

[ronggong]

I tried with

cd $IDLAK/src/pyIdlak/txp/
python3 -c "import pyIdlak_txp"

But it still gives the same error:

Traceback (most recent call last):
  File "/Users/ronggong/Documents_using/github/idlak/src/pyIdlak/txp/pyIdlak_txp.py", line 14, in swig_import_helper
    return importlib.import_module(mname)
  File "/Users/ronggong/anaconda3/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named '_pyIdlak_txp'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/ronggong/Documents_using/github/idlak/src/pyIdlak/txp/pyIdlak_txp.py", line 17, in <module>
    _pyIdlak_txp = swig_import_helper()
  File "/Users/ronggong/Documents_using/github/idlak/src/pyIdlak/txp/pyIdlak_txp.py", line 16, in swig_import_helper
    return importlib.import_module('_pyIdlak_txp')
  File "/Users/ronggong/anaconda3/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named '_pyIdlak_txp'

I feel like this is a problem of swig_import_helper ...

Btw, I have changed line 19 in idlak-server/app/endpoints/speech.py to direct to src directory. But it gave the same error.

[dabraude]

So far I have been unable to replicate the error, what OS are you using?

[ronggong]

I am using macOS. I guess the problem is on the swig on macOS...

[KonZoX]

Hello, I am currently facing the same problem. I am trying to run the server on ubuntu 18. All the requirements are met but the result is as described above. Has anybody come to the solution for this problem?

Thanks

[gklasen]

Had the same issue on Ubuntu 18.08, recent version (11.04.2020) and resolved it.

Reason was that you might not have "kaldi.mk" present in path "idlak/src". The Makefile in "idlak/idlak-server" is including it, but obviously misses it, causing the abortion.

What i did:

I ran make on "idlak/tools" again and followed the instructions in "idlak/src/INSTALL" again to create kaldi.mk. In the course, it gave the error that csh is missing. Just run sudo apt-get install csh here and try again.

Afterwards, run make in "/idlak/src/pyIdlak/" (needed first!) and afterwards again in "/idlak/src/pyIdlak/txp". Afterwards i could start.

Hopefully helps, maybe somebody facing this can confirm.

[jmatsushita]

I got the same error on OSX, and there were 2 root causes for me

1. pyIdlak didn't compile properly because it was expecting .so files instead of .dylib files in a couple of Makefiles. Specifically:
   ./idlak/src/pyIdlak/pylib/Makefile line 13 should be:

EXTRA_LDLIBS = -Wl,-rpath,"$(PCREROOT)/lib" $(PCREROOT)/lib/libpcre.dylib $(EXPATROOT)/lib/libexpat.dylib

and ./idlak/src/pyIdlak/txp/Makefile line 13 should be:

EXTRA_LDLIBS = -Wl,-rpath,"$(PCREROOT)/lib" $(PCREROOT)/lib/libpcre.dylib $(EXPATROOT)/lib/libexpat.dylib

2. swig apparently needs the _pyIdlak_*.dylib files to have the .so extension. So I change all the Makefiles :

• line 44 in pyIdlak/gen to ln -fs $(KALDILIBDIR)/lib_pyIdlak_gen.dylib _pyIdlak_gen.so
• line 40 in pyIdlak/nnet_forward to ln -fs $(KALDILIBDIR)/lib_pyIdlak_nnet_forward.dylib _pyIdlak_nnet_forward.so
• line 38 in pyIdlak/pylib to ln -fs $(KALDILIBDIR)/lib_pyIdlak_pylib.dylib _pyIdlak_pylib.so
• line 38 in pyIdlak/txp to ln -fs $(KALDILIBDIR)/lib_pyIdlak_txp.dylib _pyIdlak_txp.so
• line 40 in pyIdlak/vocoder to ln -fs $(KALDILIBDIR)/lib_pyIdlak_vocoder.dylib _pyIdlak_vocoder.so

And it works now.

[lpierron]

I got the same error on OSX Catalina and none of your solution works.
My problem is that the two variables: KALDI_FLAVOR and KALDILIBDIR are not set.