Best practice to debug the cython part of the code?

[wangzhe3224]

Hi,

Great code base!

I am trying to debug through the code just to get a real feeling of how it runs. Could you please let me know what is the best practice to debug the cython code?

I tried to build with DEBUG_MODE=True, but it seems this variable is not used any where in build.py.

Thanks for helping.

Regards,
Zhe

[cjdsellers]

Hi Zhe,

Thanks for your kind feedback and interest in the project!

Apologies for taking so long to respond. It's a great question which has been asked a few times now. I had another hunt around for any good IDE support for debugging Cython code. I'm using PyCharm myself, and it looks like the issue didn't receive any traction (asked 9 years ago so lets not hold our breath).
https://youtrack.jetbrains.com/issue/PY-9476

Someone asked the question again in 2019, and they were asked to upvote the above linked issue.
https://intellij-support.jetbrains.com/hc/en-us/community/posts/360003478920-Is-it-possible-to-debug-a-cython-code-through-pycharm-

This leaves us with the recommendations in the Cython docs:
https://cython.readthedocs.io/en/latest/src/userguide/debugging.html

Which looks like manually running gdb from the command line, probably not an acceptable workflow or anything I would call best practice.

I did however spend some time cleaning up the build.py, as you pointed out there was a redundant env var, so thats been tidied up now on the latest develop branch.

So building in debug mode should allow debugging as per the Cython docs if you still wanted to give that a try.

export DEBUG_MODE=1
make clean-build

Going forward we'll be slowly moving away from Cython as the 'core systems language', and towards Rust.
Rust has first class debugging tools, however there will probably still be a divide between peripheral pure Python code, and what will end up being a bit of a black box of the core Rust code. In other words when debugging in Python it will allow you to step up to the FFI boundary to Rust, and then no further unless JetBrains somehow swing this in CLion which they may well do. Obviously debugging the Rust code only will work as expected.

Hope this helps!

[wangzhe3224]

Hi Christopher,

Appreciate for the comprehensive replay and the fix :)

yeah, it seems ( and I agree) that for now maybe the best we can get for Cython debug is cygdb. It is kind of ok for development.

--
A separate topic on Rust. I think it definitely helps for the dev process as the tool chain of rust is much better than cython and we probably get better performance. As far as I can see from the code base (my knowledge on the code base is very limited now obviously : > ), I think the core part is not that eager for Python ecosystem, ie numpy,pandas stuff. So move to more established language makes sense.

My only concern is that what if nautilus trader's core needs some of the battery from Python's numerical ecosystem?

Regards,
Zhe

[cjdsellers]

It's a good point you make, and indeed the architecture ensures that the core is agnostic to any language or ecosystem dependencies - and so Rust becomes the perfect language to implement it in.

It's a large undertaking however, and will take time before the experimental Rust branch is merged. There are several things which remain a higher priority for the moment.