Feature: first version of Snowflake SPCS backend and authentication p…

[damianslee]

Feature: first version of Snowflake SPCS backend and authentication provider.
Notes: admin group IS ROLE IN SESSION is not working.

[damianslee]

HI @LEDfan ,
i've have a first running version of shinyproxy, containerproxy working with Snowflake Snowpark Container Services
https://docs.snowflake.com/en/developer-guide/snowpark-container-services/overview

have a couple questions

1. i need to solve how to dynamically update http header token coming into shinyproxy onto outgoing http headers on users containers. this is being done at container create time, but not sure how to refresh it.

2. i noticed the AWS ECS backend stores labels as ECS task env variables. is it consider safe to store all labels as env variables? also i noticed AWS ECS backend doesn't implement recovery.

An example configuration is

proxy:
  title: Open Analytics Shiny Proxy
  logo-url: https://www.openanalytics.eu/shinyproxy/logo.png
  landing-page: /
  heartbeat-rate: 10000
  heartbeat-timeout: 300000
  port: 8080
  authentication: spcs
  container-backend: spcs
  stop-proxies-on-shutdown: false
  recover-running-proxies: true
  admin-groups: <snowflake role(s) admin users are granted to>
  spcs:
    compute-pool: "my_compute_pool"

  specs:
    - id: 01_hello_sf
      container-image: "/MYDB/images/shinyproxy/shinyproxy-integration-test-app:latest"
      container-cmd: [ "R", "-e", "shinyproxy::run_01_hello()" ]
      container-memory-request: 2048
      container-cpu-request: 1024
      port: 3838

[damianslee]

Another question, the Snowflake Java client was generated from the snowflake openapi specification github project using okhttp3.
https://github.com/snowflakedb/snowflake-rest-api-specs

is there a preferred java http library to use to keep the package sizes and dependencies as small as possible?

[LEDfan]

Hi

Thanks a lot for the effort! We are always happy to add new backends, the only problem I see is that we don't have any snowflake deployments ourself. In order to guarantee maintenance on this backend, we need to look into whether we can use some trial or development license.

i need to solve how to dynamically update http header token coming into shinyproxy onto outgoing http headers on users containers. this is being done at container create time, but not sure how to refresh it.

In the current implementation,the headers that are send to the application are not updated after the application has been started. This can be changed, although I'm not yet sure about the exact implementation. The headers are injected here: https://github.com/openanalytics/containerproxy/blob/master/src/main/java/eu/openanalytics/containerproxy/util/ProxyMappingManager.java#L260-L261

i noticed the AWS ECS backend stores labels as ECS task env variables. is it consider safe to store all labels as env variables? also i noticed AWS ECS backend doesn't implement recovery.

I believe these are added as labels in the task. But in general we consider it safe to add them to labels in the backend, but I wold not add them as environment variables (since the app should not see all these values).

is there a preferred java http library to use to keep the package sizes and dependencies as small as possible?

okhttp3 is already a dependency, so it's fine to use. However, we use jackson for json processing, so if possible it would be better to use jackson instead of gson.

---

I don't have lots of experience with Snowflake, could you elaborate your use-case of running ShinyProxy on Snowflake?
What are the advantages? I'm also wondering how the initial deployment of ShinyProxy is done. Is ShinyProxy running on the Snowflake cluster, or is it deploy externally?

[damianslee]

typical deployment is shinyproxy running as snowflake service, then creates additional private services for users apps.

the code to support shinyproxy external is to make development easier and faster. i dont think it will ever be used this way in production.

use cases and benefits

1. makes the user auth really easy. inherits the snowflake identity setup.
2. can use pass through of users identity or service role to access the data
3. analytics with python or R. snowflake has an existing Notebook feature. python or SQL. some users like it, some don't.
   eg codeserver. how does one manage creating a codeserver snowflake service per user...
4. data apps with python or R. snowflake owns and supports streamlit in snowflake. is not meant to solve everything. snowflake has marketplace for vendor apps. half way is supporting other python/R data app frameworks.

[damianslee]

the list of openapi java codegen which have Jackson json are here with different webclients. any of these won't introduce additional packages?

search for "library template (sub-template) to use"
https://openapi-generator.tech/docs/generators/java

will add an example SQL to setup the environment soon.