index -1 is out of bounds for axis 0 with size 0

[Yonatan19981]

I am getting this error:

Exception has occurred: IndexError
index -1 is out of bounds for axis 0 with size 0

in this function:

def is_overbought_or_oversold(self):
	[symbol_a, symbol_b] = self.symbols
	last_price_a = self.df_hist[symbol_a].dropna().values[-1]

Does anybody know of a way to solve it?

Thanks

[twopirllc]

Have you tried changing 'values[-1]' to 'iat[-1]'?

def is_overbought_or_oversold(self):
	[symbol_a, symbol_b] = self.symbols
	last_price_a = self.df_hist[symbol_a].dropna().iat[-1]
	last_price_b = self.df_hist[symbol_b].dropna().iat[-1]

Hope this helps,
KJ

[Romiko]

I am having the same problem. where can I find the member for the return type of dropna()?

V:\dev\trading\High-Frequency-Trading-Model-with-IB>docker-compose up
Starting high-frequency-trading-model-with-ib_hft_model_1_1 ... done
Attaching to high-frequency-trading-model-with-ib_hft_model_1_1
hft_model_1_1 | Connecting on host: 192.168.1.9 port: 6666
hft_model_1_1 | [2020-05-16 10:05:59.612149]started
hft_model_1_1 | Value ({Ticker(contract=Forex('EURUSD', exchange='IDEALPRO'), time=datetime.datetime(2020, 5, 16, 10, 6, 2, 943709, tzinfo=datetime.timezone.utc), bid=-1.0, bidSize=0, ask=-1.0, askSize=0, ticks=[TickData(time=datetime.datetime(2020, 5, 16, 10, 6, 2, 943709, tzinfo=datetime.timezone.utc), tickType=1, price=-1.0, size=0), TickData(time=datetime.datetime(2020, 5, 16, 10, 6, 2, 943709, tzinfo=datetime.timezone.utc), tickType=2, price=-1.0, size=0)], tickByTicks=[], domBids=[], domAsks=[], domTicks=[])},) caused exception for event Event<pendingTickersEvent, [[None, <weakref at 0x7f6d2d91d1d0; to 'HftModel1' at 0x7f6d4a1ec4d0>, <function HftModel1.on_tick at 0x7f6d2d9d6830>]]>
hft_model_1_1 | Traceback (most recent call last):
hft_model_1_1 | File "/usr/local/lib/python3.7/site-packages/eventkit/event.py", line 184, in emit
hft_model_1_1 | func(obj, *args)
hft_model_1_1 | File "/app/models/hft_model_1.py", line 77, in on_tick
hft_model_1_1 | self.perform_trade_logic()
hft_model_1_1 | File "/app/models/hft_model_1.py", line 94, in perform_trade_logic
hft_model_1_1 | self.calculate_signals()
hft_model_1_1 | File "/app/models/hft_model_1.py", line 193, in calculate_signals
hft_model_1_1 | is_overbought, is_oversold = self.is_overbought_or_oversold()
hft_model_1_1 | File "/app/models/hft_model_1.py", line 207, in is_overbought_or_oversold
hft_model_1_1 | last_price_b = self.df_hist[symbol_b].dropna().values[-1]
hft_model_1_1 | IndexError: index -1 is out of bounds for axis 0 with size 0

[Romiko]

Changed it to iat (could not find a reference to this anywhere, caused the same issue.

Attaching to high-frequency-trading-model-with-ib_hft_model_1_1
hft_model_1_1 | Connecting on host: 192.168.1.9 port: 6666
hft_model_1_1 | [2020-05-16 10:13:26.369352]started
hft_model_1_1 | Value ({Ticker(contract=Forex('EURUSD', exchange='IDEALPRO'), time=datetime.datetime(2020, 5, 16, 10, 13, 29, 452468, tzinfo=datetime.timezone.utc), bid=-1.0, bidSize=0, ask=-1.0, askSize=0, high=1.0851, low=1.07895, close=1.0805, halted=0.0, ticks=[TickData(time=datetime.datetime(2020, 5, 16, 10, 13, 29, 452468, tzinfo=datetime.timezone.utc), tickType=1, price=-1.0, size=0), TickData(time=datetime.datetime(2020, 5, 16, 10, 13, 29, 452468, tzinfo=datetime.timezone.utc), tickType=2, price=-1.0, size=0), TickData(time=datetime.datetime(2020, 5, 16, 10, 13, 29, 452468, tzinfo=datetime.timezone.utc), tickType=6, price=1.0851, size=0), TickData(time=datetime.datetime(2020, 5, 16, 10, 13, 29, 452468, tzinfo=datetime.timezone.utc), tickType=7, price=1.07895, size=0), TickData(time=datetime.datetime(2020, 5, 16, 10, 13, 29, 452468, tzinfo=datetime.timezone.utc), tickType=9, price=1.0805, size=0), TickData(time=datetime.datetime(2020, 5, 16, 10, 13, 29, 452468, tzinfo=datetime.timezone.utc), tickType=49, price=0.0, size=0)], tickByTicks=[], domBids=[], domAsks=[], domTicks=[])},) caused exception for event Event<pendingTickersEvent, [[None, <weakref at 0x7fe9a9b2de90; to 'HftModel1' at 0x7fe9c63ac510>, <function HftModel1.on_tick at 0x7fe9a9b565f0>]]>
hft_model_1_1 | Traceback (most recent call last):
hft_model_1_1 | File "/usr/local/lib/python3.7/site-packages/eventkit/event.py", line 184, in emit
hft_model_1_1 | func(obj, *args)
hft_model_1_1 | File "/app/models/hft_model_1.py", line 77, in on_tick
hft_model_1_1 | self.perform_trade_logic()
hft_model_1_1 | File "/app/models/hft_model_1.py", line 94, in perform_trade_logic
hft_model_1_1 | self.calculate_signals()
hft_model_1_1 | File "/app/models/hft_model_1.py", line 193, in calculate_signals
hft_model_1_1 | is_overbought, is_oversold = self.is_overbought_or_oversold()
hft_model_1_1 | File "/app/models/hft_model_1.py", line 207, in is_overbought_or_oversold
hft_model_1_1 | last_price_b = self.df_hist[symbol_b].dropna().iat[-1]
hft_model_1_1 | File "/usr/local/lib/python3.7/site-packages/pandas/core/indexing.py", line 2270, in getitem
hft_model_1_1 | return self.obj._get_value(*key, takeable=self._takeable)
hft_model_1_1 | File "/usr/local/lib/python3.7/site-packages/pandas/core/series.py", line 1186, in _get_value
hft_model_1_1 | return com.maybe_box_datetimelike(self._values[label])
hft_model_1_1 | IndexError: index -1 is out of bounds for axis 0 with size 0
hft_model_1_1 | [2020-05-16 10:13:29.948810][strategy params]beta:0.00 volatility:1.00|rpnl=0.00
hft_model_1_1 | [2020-05-16 10:13:30.205474][strategy params]beta:nan volatility:nan|rpnl=0.00

[Romiko]

Found the bug. It is Saturday here in Australia (9pm). Seems that this method is too greedy.

def trim_historical_data(self):
	""" Ensure historical data don't grow beyond a certain size """
	cutoff_time = dt.datetime.now(tz=dt_util.LOCAL_TIMEZONE) - self.moving_window_period
	self.df_hist = self.df_hist[self.df_hist.index >= cutoff_time]

For GBPUSD pair, it cleaned it out.

[twopirllc]

@Romiko

Good work debugging and resolving it. Those sneaky timezones.

KJ

[SC4RECOIN]

Setting the window to 4 hours instead of 1 resolved the issue for me

self.moving_window_period = dt.timedelta(hours=4)