Is it possible to support more Third Party Authentication tool?

[xscheme]

Thanks for you great work!!!

Da*n IBKR. I have been fighting with them to support the third party authentication tool 2FAS . They refused.

They say, Google Authenticator is supported. I searched: we can extract out the key for Google Authenticator. Is it okay to support Google Authenticator.

FYI, you can apply for more IBKR usernames.

[heshiming]

All "mobile authenticators" are essentially the same tech, technically called TOTP or Time-based One Time Password. The reason I recommended 2FAS over other solutions is the easiness of exporting the private key. But if you can extract that private key from other solutions, which takes a bit of work, you can use it the same way as documented, via TOTP_KEY variable.

[xscheme]

I make it almost work. You are correct. I managed to export the key from Google Authenticator to 2FAS. Both devices show the same code at the same time after my exporting.

There is a new issue. Probably, this is related with account setup. I have a few usernames for my IBKR account. 1) one user name uses the old physical card tokens; 2) one user name does not require 2FA at all (a magic). this account actually works with your docker; but IB Gateway displays a window asking me to enable 2FA; 3) most recently, i applied for a new account, to be used for this IB gateway. This is the account i am working on now.

For the new account, I have to apply for IBKEY first, and then enable Mobile Authenticator (Google Authenticator). So login for this account is slightly different from yours. The login window will appear as follows:

Can you help to resolve this flow? If you want to test, you might need to apply for a new username.

[xscheme]

The above window is captured from the NOVNC. This extra pop asks for choice of IB KEY/mobile authenticator.

Both options are available in the client portal. and i cannot remove the IB KEY, which will make my account similar to yours.

[heshiming]

Ok, I don't have an account to replicate your setup. But I've seen that choice window before so let me try to handle that choice. Hopefully I can get an update out soon.

What's weird is that I previous had IB Key and a physical device. But ever since IB allows me to link a new mobile authenticator, those options disappeared from my login process. Both TWS and IBG prompts mobile authenticator without a choice now. Only the customer portal behaves differently.

[heshiming]

Hi, I've just pushed out a version (docker hub image) that should theoretically select "Mobile Authenticator app" upon seeing that "Second Factor Authentication" prompt. Can you give it a try and let me know if it works?

[xscheme]

Sorry. this still does not work. It seems the choice for second factor device window passed, but it hangs on the authentication code window

The terminal looks like this:

[xscheme]

shiming, really appreciate your efforts on this work. It is a great endeavor.

Previously, I have to consider buying a Windows VPS to just host this IBGateway. IBKR really suckx. no idea why they host such a GUI application as the communication hub (slow performance). With your approach, at least we can host everything in the Linux console.

Understand it is a bit annoying for you to try without a proper user account, but my account is loaded with money :( it is a bit sensitive to lend out my account.

Log into the IBKR Client portal ==> Settings ==> Users & Access Rights ==> you can apply for new log in IDs. For example, I have many id in my account:

For future accounts, I believe they will go through the same flow as my newly applied account. I mentioned above: 1) my first ID still uses the tokens table; 2) my second ID does not go through the 2FA at all (a magic). These 2 user IDs are really special accounts. All the other IDs have to go through the 2FA.

[heshiming]

So I have a spare account, that previously had two options for 2nd factor: IB Key, and a physical device. Today I used the QR code in its control panel to link the Mobile Authenticator app. After that, I noticed the physical device was removed from 2nd factor list.

Then I tried logging in with IBG. The first time, it pops up IB Key directly, but I saw a link to select another way. I clicked on it, and I see the selection box with IB Key and Mobile Authenticator app. I noticed the latest version (heshiming/ibga:latest) will proceed to select the proper one. And then TOTP will kick in and automatically fill. In other words, it's working for me.

But this is just the first time. I noticed from the second time onwards, it no longer lets me choose a way, and default to TOTP every time. So I guess once you selected this option, it'll be kept and you no longer have to worry about it.

Configuration should be like this, (under environment of docker-compose.yml):

• IB_PREFER_IBKEY must be removed
• TOTP_KEY must be set

I can't think of another reason it would fail. You might want to check whether your container image is really the latest one. Even if you pulled the latest image, you have to recreate the container so that it's not using the older one.

I just tried pushing heshiming/ibga image again, even though dockerhub says the image was just updated now, all layers are not changed. So I'm pretty sure the latest version works

Using default tag: latest
The push refers to repository [docker.io/heshiming/ibga]
104ccefb43e6: Layer already exists
f316089e05ce: Layer already exists
66cbd2f16249: Layer already exists
12afed37965e: Layer already exists
5f70bf18a086: Layer already exists
82fdfb0e32d3: Layer already exists
88e9c28a681c: Layer already exists
f3d499b40059: Layer already exists
ea680fbff095: Layer already exists
latest: digest: sha256:27570ea8e6af50217a6ea0f6bbf7448a94123e276ab8c2e3e9d3208df255ecef size: 2407

[xscheme]

My issue is: the starting up gets stuck on the 2FA window (OTP Window). The log on is supposed to fill in the 6 digit code generated from the secret, but it does not fill in any code. At this step, if I manually fill in the OTP codes, the log on can still continue.

My question is: your docker image should already contain the oathtool package. correct? I even tried to install oathtool on my host machine (Oracle 9 Linux). After installation, the log on still get stuck on the OTP window.

[root@oc-docker cking]# oathtool --totp -b "MMP4SFIZYBGZZZZZZZZZZZZZZZZZZZZ4"
767396
[root@oc-docker cking]#

My docker-compose.yml is as follows:
[root@oc-docker cking]# cat docker-compose.yml
version: '2'
services:
my-ibga:
image: heshiming/ibga
restart: unless-stopped
environment:
- TERM=xterm
- IB_USERNAME=XXXXXXXXX
- IB_PASSWORD=YYYYYYYYY
- IB_REGION=America
- IB_TIMEZONE=America/New York
- IB_LOGINTAB=IB API
- IB_LOGINTYPE=Live Trading
- IB_LOGOFF=11:55 PM
- IB_APILOG=data
- IB_LOGLEVEL=Error
- TOTP_KEY=MMP4SFIZYBGZZZZZZZZZZZZZZZZZZZZ4
volumes:
- ./run/program:/home/ibg
- ./run/settings:/home/ibg_settings
ports:
- "15800:5800"
- "4000:4000"
[root@oc-docker cking]#

[heshiming]

I think I found the cause. The window class of that TOTP dialog changed during development, and my handling of previous versions had a typo. Details are here: a1cb50a

So if you could try pull the image again, and recreate your container, you should be good to go.

I suspect that during this period, you have never tried to reinstall IBG (by deleting ./run/program). Somehow that new version is released just couple days ago.

If that didn't work, you could still help debugging. The procedure is as follows:

1. Make sure that the TOTP dialog is showing, like your screenshot. It is going to timeout so move on to the next steps fast.
2. At host of the container, type sudo docker exec -ti my-ibga bash or whatever your container name is.
3. Once you get inside the container, type echo "/home/ibg/testoutput.txt|list_ui_components" > /tmp/ibg-jauto.in
4. Type cat /home/ibg/testoutput.txt and paste the contents here.

This will let me know the window-class of the TOTP dialog. Perhaps TOTP dialog is region-specific.

[xscheme]

Yeah. Indeed i tried to delete the run folder.

BTW, my host VPS is located in Tokyo.

[root@oc-docker cking]# docker exec -ti cking-my-ibga-1 bash
ibg@80a6692ec81b:$ echo "/home/ibg/testoutput.txt|list_ui_components" > /tmp/ibg-jauto.in
ibg@80a6692ec81b:$ cat /home/ibg/testoutput.txt
javax.swing.JFrame,twslaunch.feature.welcome.D,window:0,x:211,y:198,w:602,h:372,act:0,title:Authenticating...
javax.swing.JPanel,,x:211,y:198,w:602,h:372,mx:512,my:384
javax.swing.JLayeredPane,twslaunch.feature.welcome.E,x:211,y:222,w:602,h:302,mx:512,my:373
javax.swing.JPanel,twslaunch.feature.welcome.ads.g,x:493,y:254,w:300,h:250,mx:643,my:379
javax.swing.JLabel,twslaunch.feature.welcome.ads.f,x:493,y:254,w:300,h:250,mx:643,my:379,
javax.swing.JPanel,,x:211,y:524,w:602,h:46,mx:512,my:547
javax.swing.JPanel,,x:211,y:524,w:602,h:20,mx:512,my:534
javax.swing.JProgressBar,,x:216,y:529,w:592,h:10,mx:512,my:534,progress:28
javax.swing.JPanel,,x:211,y:544,w:602,h:23,mx:512,my:555
javax.swing.JPanel,,x:216,y:544,w:208,h:25,mx:320,my:556
javax.swing.JLabel,,x:221,y:549,w:89,h:15,mx:265,my:556,text:Build 10.36.1c
javax.swing.JLabel,,x:315,y:549,w:104,h:15,mx:367,my:556,text:Authenticating...
javax.swing.JLabel,,x:671,y:544,w:137,h:25,mx:739,my:556,text:Logging in gatapi998
javax.swing.JPanel,,x:211,y:567,w:602,h:3,mx:512,my:568
javax.swing.JFrame,ibgateway.aA,window:1,x:162,y:109,w:700,h:550,act:0,title:Loading...
javax.swing.JPanel,,x:166,y:132,w:692,h:523,mx:512,my:393
javax.swing.JPanel,ibgateway.aF,x:166,y:132,w:692,h:65,mx:512,my:164
javax.swing.JPanel,,x:168,y:149,w:688,h:23,mx:512,my:160
javax.swing.JLabel,,x:168,y:149,w:344,h:23,mx:340,my:160,text:Purpose
javax.swing.JLabel,,x:512,y:149,w:344,h:23,mx:684,my:160,text:Status
javax.swing.JPanel,ibgateway.aE,x:168,y:172,w:688,h:23,mx:512,my:183
javax.swing.JLabel,,x:168,y:172,w:344,h:23,mx:340,my:183,text:API Server
javax.swing.JPanel,feature.farm.x,x:512,y:172,w:344,h:23,mx:684,my:183
javax.swing.JPanel,,x:513,y:173,w:1,h:21,mx:513,my:183
javax.swing.JLabel,,x:514,y:173,w:340,h:21,mx:684,my:183,text:disconnected
javax.swing.JPanel,,x:854,y:173,w:1,h:21,mx:854,my:183
javax.swing.JComponent,jclient.qV,x:854,y:173,w:1,h:21,mx:854,my:183
javax.swing.JPanel,ibgateway.aD,x:166,y:197,w:692,h:458,mx:512,my:426
javax.swing.JPanel,ibgateway.aC,x:166,y:197,w:692,h:16,mx:512,my:205
javax.swing.JCheckBox,trader.common.tag.q,x:166,y:197,w:76,h:16,mx:204,my:205,selected:n,text:Show log
javax.swing.JCheckBox,trader.common.tag.q,x:268,y:197,w:144,h:16,mx:340,my:205,selected:n,text:Show API messages
javax.swing.JPanel,,x:166,y:213,w:692,h:415,mx:512,my:420
javax.swing.JTabbedPane,,x:166,y:213,w:692,h:415,mx:512,my:420
javax.swing.JPanel,,x:166,y:628,w:692,h:27,mx:512,my:641
javax.swing.JButton,trader.common.tag.p,x:478,y:628,w:68,h:27,mx:512,my:641,selected:n,text:Clear
ibg@80a6692ec81b:$ cat /tmp/ibg-jauto.in
/tmp/tmp.0dCvGiccqY|list_ui_components?window_type=dialog&window_title=Login failed
ibg@80a6692ec81b:$

[heshiming]

I can see you are running the latest Build 10.36.1c. I tested it out and it worked fine for me.

However, from your UI component dump, I don't see a TOTP window, which is probably why no TOTP automation can be performed. I'm not sure how to go from here. The text file you pasted shows all the titles of the windows and buttons. But there's nothing about Mobile Authenticator or Second Factor. I've got two individual accounts of the China region, and two institutional account from the U.S. region. None of them behaves this way.

[xscheme]

What do you mean by the TOTP window? this widow?

[heshiming]

Yes, I'm referring to the one in your screenshot. Did you keep it open while typing those commands according to my instructions?

[xscheme]

I have one IBKR account with my Chinese passport. I can give a try with that account, too.

[heshiming]

Well, the program is not seeing this window.

[xscheme]

yeah. this is weird. I reviewed the window list last night as well. Also, I checked the container image has oathtool installed.

[heshiming]

Well the culprit cannot be oathtool. The instructions I provided must contain some kind of window, a JDialog or a JFrame with title: "Second Factor Authentication", a JInput to enter the TOTP code, and a JButton to press. That's how things are supposed to work. If the underlying plugin cannot see this window, no automation can be performed.

[xscheme]

I get your point. let me check what I can at this step. This behaves as expected, as I can observe there is no automation to key in the OTP in the TOTP window.

Thanks a lot for your assistance.

[xscheme]

Hi, Shiming, good news!

I just launched a VPS (Rocky Linux) instance in HK on Microsoft Azure. It works perfectly :)

[heshiming]

Ok ... thanks for reporting back. Remember to harden security. I'm running all my instances locally so that I don't have to worry about it. For VPS, there are much cheaper alternatives, such as cloudcone, that will run this at a fraction of the cost.

[xscheme]

Yes. security is #1 concern for me on IBKR. I use the firewall to restrict the port access to only limited specified IPs.

If you want to use VPS, here is much cheaper options:
https://www.yecaoyun.com/Page/offers.html

I have been using them for more than one year, quite stable. Also, I do a daily back up of the data and upload the data to a free google cloud instance.