use-case question: performance comparison

[RedLhek]

Hi,
First of all - great thanks for the library, it looks really amazing and well-written (from the framework architecture point or view).

I want to use it as my core component in personal TA and investing, nevertheless, I am a bit confused about the main focus. I was experimenting with vectorbt and ta-lib speed comparison and found out, that ta-lib is about 10 times faster for signal preparation.
I am not sure it's a fair comparison, so... could you please tell a bit more about the top use-cases for vectorbt? Portfolio optimization? Strategy backtesting? Could we speed up all of them with pure ta-lib usage? Or when should I use vectorbt?

Here is the link on performance comparison - https://colab.research.google.com/drive/1szojVeKW_HWHr6iXEJogD0PsPJ6Caa7s?usp=sharing

Thanks in advance!

[RedLhek]

A follow-up question could be: why should we use numba rather than numpy for vector/matrix-based operations?
Or is numba faster for pandas columns operations?

[polakowo]

Answering your first question: vectorbt does diverse pre- and postprocessing on your inputs, parameters, and outputs. If it simply called the particular talib function, the vbt.talib method would yield the same performance. But what happens is this in order to broadcast inputs and parameters, convert outputs back to pandas, build multi-level columns, prepare mappers for indexing, etc. This all introduces an ms overhead. In fact, if you run the same on an input of length 1e6, talib function executes in 27.2 ms while vectorbt in 48 ms. The difference will narrow with the number of elements.

Remember that a major focus of vectorbt is in hyperparameter optimization through the construction of large hyperparameter grids, which involve wide 2-dimensional arrays. Talib doesn't handle 2D so you must explicitly loop through columns and call the talib function on each. This is automatically handled by vectorbt, so all you have to do is give input of any shape and a list of parameters, and the rest is done for you. The price you pay for this is a bit slower execution. If you don't want all these features, of course, you should use talib library and nothing else.

The reason why run_unique is slower is that it only makes sense when you have repeating hyperparameters. For example, to compare a single faster moving average with hundreds of slower-moving averages, you wouldn't want to recalculate the faster MA hundreds of times. You just tell vectorbt to run the calculation on the unique parameters only and stack them in the final output data frame. The stacking operation both in NumPy and pandas is pretty slow, but still mostly worth it (just look at SMA.run_combs execution time in https://github.com/polakowo/vectorbt/blob/master/tests/notebooks/indicators.ipynb).

Answering the question on what the use case of vectorbt is: everything you mentioned. Talib is just a library with technical indicators - many sophisticated strategies don't use those at all. Running indicators is not vectorbt's main function. The main competitive advantage of vectorbt is the backtesting speed, that is, the amount of time needed to convert an array of signals into a performance metric. Without exaggeration, there is nothing comparable in the open-source. The only backtesters that can beat vectorbt are written in pure Java and C(++), but they don't have the flexibility of Python that vectorbt enjoys.

On the last question: because in Numba you can write any operation as a loop and it will have a slightly lower but still similar speed to a vectorized operation in NumPy. Except that your code is now more readable and allows a lot of use cases, such as calling functions inside functions. Plus you cannot express most backtesting operations in NumPy (just try implementing rolling standard deviation).