[RFC] Subpackages for pykeyvi

[hendrikmuhs]

For a new feature I like to discuss modularization. I like to put the new feature into a subpackage ('pykeyvi.index'). Playing a bit with cython it unfortunately turns out, cython does not support it easily:

https://github.com/cython/cython/wiki/FAQ#how-to-compile-cython-with-subpackages

There is a recommendation on the wiki:

https://github.com/cython/cython/wiki/PackageHierarchy

which would build a 'so' for every subpackage, which would mean a lot of wasted space and higher memory usage.

RFC: Move the cython extension into a hidden namespace ('pykeyvi._core') keeping the flat structure, create a python module structure and import accordingly, this would keep a single so file but still provide subpackages:

 - pykeyvi
  - __init__.py
  - index
   - __init__.py

pykeyvi/init.py:

from pykeyvi._core import JsonDictionaryCompiler, ...

pykeyvi/index/init.py:

from pykeyvi._core import IndexReader, IndexWriter

In the longer run this also solves 2 problems:

• cleanup existing structure
• create python code extensions (like the keyvicli library)

[hendrikmuhs]

@narekgharibyan

[narekgharibyan]

@hendrikmuhs sounds good!
Have you tried to implement it already, to see how it works with packaging ?