Skip to content

How to use q from streamlit? #47

@Kamilcuk

Description

@Kamilcuk

Describe the bug

I am trying to connect to q from streamlit using the following function:

However:

  1. The connection works the first time.
  2. The connection just stops working the second time. It blocks on is_healthy() call.
  3. It is impossible to kill straemlit. I have to pkill -s 9 -f streamlit every time.

To Reproduce

This is pseudocode:

import os
import pykx as kx
import contextlib
import streamlit as st

@contextlib.contextmanager
def with_qconn():
    """
    Make connection to q.
    Check https://code.kx.com/pykx/3.1/user-guide/advanced/streamlit.html
    """
    port = os.environ.get("Q_PORT")
    assert port, "Q_PORT must be set to use q"
    print(f"Connecting to Q on localhost:{port}")
    conn = st.connection(
        "pykx",
        type=kx.streamlit.PyKXConnection,
        host="localhost",
        port=int(port),
    )
    eprint("checking health")
    if not conn.is_healthy():
        print(f"Resetting connection to Q on localhost:{port}")
        conn.reset()
    else:
        print(f"Connected to Q on localhost:{port}")
    assert conn.is_healthy()
    try:
        yield conn
    finally:
        kx.shutdown_thread()

def query_q():
  with with_qconn() as qconn:
      qconn.query(r"\l /some/place", format="q")
      # ...
      return qconn.query("data", columns=columns, where=where, format="qsql").pd()

if st.button("clickme"):
   st.dataframe(query_q())

I am exeucting it something along:

$ export QLIB=/etc/pykx8 Q_PORT=15050 PYKX_BETA_FEATURES=1 PYKX_IGNORE_HOME=1 PYKX_THREADING=1 QHOME=... PYKX_Q_EXECUTABLE=... 
$ ./bin/multirun -v '/opt/sp/qlib/4.0.0.x86_64/q --no-qce --no-kurl --no-objstor --no-qlog --no-sql -p 15050' 'streamlit run --browser.serverAddress=nyzls3291d ./main.py'
multirun: launched command /opt/sp/qlib/4.0.0.x86_64/q --no-qce --no-kurl --no-objstor --no-qlog --no-sql -p 15050 with pid 3884235
multirun: launched command streamlit run --browser.serverAddress=nyzls3291d ./main.py with pid 3884236

  You can now view your Streamlit app in your browser.

  URL: http://nyzls3291d:8503


Connecting to Q on localhost:15050
checking health
Connected to Q on localhost:15050
Connecting to Q on localhost:15050
checking health
# now it is blocked...

Expected behavior

The connection to q should work on the second time streamlit is opened.

I should be able to kill streamlit normally with just SIGINT. It never exits.

Screenshots

Desktop (please complete the following information):

  • OS: rockylinux 9
  • KDB+ banner information KDB+ 4.0 2021.04.26 Copyright (C) 1993-2021 Kx Systems l64/ 8(8)core 514509MB cukrowsk nyzls3291d 10.67.3.208 EXPIRE 2055.01.01 squarepoint ops 8 cores #97088
  • Repository version pykx[streamlit]==3.1.4

Additional context

How to use pykx in streamlit? How to only kill the thread on exit? Thank you

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions