Add example `functions` section to api-server #7398

bisgaard-itis · 2025-03-19T16:22:40Z

What's the point of this PR?

As part of out metamodeling efforts we plan to develop the concept of a function in the api server. These functions could point to concepts we already have in the api-server (studies or solvers) or to new types of functions. This PR does nothing more than setup a few files and connect a functions section in the api-server with a new functions domain in the api-server so that @wvangeit can start implementing the thing. I.e. it is a skeleton to guide the development. I.e. the functions which are introduced in this PR will eventually go away as we start to merge the real functionality for the functions.

What do these changes do?

Add functions section in api-server. Expose a single ping endpoint as an example
Add functions domain with example files of how to setup the domain.
Connect ping endpoint with webserver via rpc to webserver.

Related issue/s

fixes Add example functionsapi with a single endpoint which is connected all the way to the webserver via rpc #7348

How to test

Dev-ops checklist

No ENV changes or I properly updated ENV (read the instruction)

bisgaard-itis · 2025-03-19T16:25:52Z

@wvangeit please let me know if anything in this example is unclear. I am very happy to extend it to make it more realistic or add some actual functionality if required.

codecov · 2025-03-19T16:29:35Z

Codecov Report

Attention: Patch coverage is 62.26415% with 20 lines in your changes missing coverage. Please review.

Project coverage is 87.44%. Comparing base (c9296d7) to head (df7b5d8).
Report is 1 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #7398      +/-   ##
==========================================
+ Coverage   87.42%   87.44%   +0.01%     
==========================================
  Files        1723     1721       -2     
  Lines       66749    66612     -137     
  Branches     1133     1133              
==========================================
- Hits        58356    58247     -109     
+ Misses       8072     8044      -28     
  Partials      321      321

Flag	Coverage Δ
integrationtests	`65.15% <58.62%> (+0.01%)`	⬆️
unittests	`86.61% <62.26%> (+0.01%)`	⬆️

Components	Coverage Δ
api	`∅ <ø> (∅)`
pkg_aws_library	`93.90% <ø> (ø)`
pkg_dask_task_models_library	`97.09% <ø> (ø)`
pkg_models_library	`92.05% <ø> (ø)`
pkg_notifications_library	`84.57% <ø> (ø)`
pkg_postgres_database	`88.11% <ø> (ø)`
pkg_service_integration	`70.03% <ø> (ø)`
pkg_service_library	`72.49% <0.00%> (-0.16%)`	⬇️
pkg_settings_library	`90.78% <ø> (ø)`
pkg_simcore_sdk	`85.46% <ø> (ø)`
agent	`96.46% <ø> (ø)`
api_server	`90.73% <83.33%> (-0.03%)`	⬇️
autoscaling	`96.08% <ø> (ø)`
catalog	`91.82% <ø> (ø)`
clusters_keeper	`99.24% <ø> (ø)`
dask_sidecar	`91.25% <ø> (ø)`
datcore_adapter	`98.11% <ø> (ø)`
director	`76.70% <ø> (ø)`
director_v2	`91.30% <ø> (ø)`
dynamic_scheduler	`97.33% <ø> (ø)`
dynamic_sidecar	`90.11% <ø> (ø)`
efs_guardian	`89.79% <ø> (ø)`
invitations	`93.28% <ø> (ø)`
payments	`92.66% <ø> (ø)`
resource_usage_tracker	`89.12% <ø> (+0.21%)`	⬆️
storage	`86.61% <ø> (ø)`
webclient	`∅ <ø> (∅)`
webserver	`85.90% <79.31%> (-0.01%)`	⬇️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c9296d7...df7b5d8. Read the comment docs.

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

GitHK

I'm a bit lost. What would be te final goal of this functions?
I see no reference to that

sanderegg

I am also not clear what this "function" api is. all this seems very hard-coded in the end... I thought this was something where you send a "function" to be executed in there... but this looks like you have to implement each function precisely instead...

services/api-server/src/simcore_service_api_server/api/routes/functions.py

...ages/service-library/src/servicelib/rabbitmq/rpc_interfaces/webserver/functions/functions.py

services/web/server/src/simcore_service_webserver/functions/_controller_rpc.py

services/web/server/src/simcore_service_webserver/functions/_service.py

services/web/server/src/simcore_service_webserver/functions/plugin.py

services/web/server/src/simcore_service_webserver/functions/_controller_rpc.py

services/api-server/src/simcore_service_api_server/api/routes/functions.py

services/api-server/src/simcore_service_api_server/api/root.py

bisgaard-itis · 2025-03-20T13:05:55Z

I'm a bit lost. What would be te final goal of this functions? I see no reference to that

Yes, sorry. That's on me. I did not explain well in the header what exactly the goal of this PR is. I will update it

bisgaard-itis · 2025-03-20T13:41:34Z

I am also not clear what this "function" api is. all this seems very hard-coded in the end... I thought this was something where you send a "function" to be executed in there... but this looks like you have to implement each function precisely instead...

Yes, actually this PR just introduce some "dummy" functions so Werner can start implementing the actual api and has a pattern to follow. I have updated the header of this PR with some more info

sanderegg

ok thanks for the explanation. I am still a bit unsure. but let's see.

GitHK

Sorry, but I have doubts and issues with this approach. Please let's have chat about it. See my reasons below.

I am not convinced at all by these changes/pattern.

I believe that the current approach will just cause harm in the long run.

I see the following issues/uncertainties:

I did not understand the concept of a function
no generic interface to register, launch and get the result of a function
a function could also be long running (long lived), the current pattern will eventually fail (example, if the webserver is restarted)
why was the webserver chosen as the owner of these functions?
Can you please provide a schema/diagarm of how the interaction with a functions will work?

pcrespov

thx!

GitHK

As said on Friday, the reason for this PR being blocked is that it lacks any interface for long running execution.

I think we agreed that most, if not all the payloads would be long running. Such support would be useful.
I think it would greatly help to have such a system in place. Event if it's not perfect, we can always improve it, instead of having to change all the interfaces in the future.

Could this be added?

bisgaard-itis · 2025-03-25T14:37:37Z

As said on Friday, the reason for this PR being blocked is that it lacks any interface for long running execution.

I think we agreed that most, if not all the payloads would be long running. Such support would be useful. I think it would greatly help to have such a system in place. Event if it's not perfect, we can always improve it, instead of having to change all the interfaces in the future.

Could this be added?

That's part of the prototype Werner will implement. As I mentioned, this PR is just to show how the different services are connected.

GitHK

I will not block it, but I do not agree one bit with anything in this PR.

bisgaard-itis · 2025-03-26T14:00:37Z

I will not block it, but I do not agree one bit with anything in this PR.

Thanks a lot for unblocking 🙏🏻

bisgaard-itis · 2025-03-26T14:01:14Z

@Mergifyio queue

mergify · 2025-03-26T14:06:57Z

bisgaard-itis · 2025-03-28T09:19:57Z

@Mergifyio queue

mergify · 2025-03-28T09:20:21Z

sonarqubecloud · 2025-03-28T09:21:30Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

bisgaard-itis added 3 commits March 19, 2025 16:40

add dummy functions section in webserver add rpc interface

e4ed2fa

introduce functions section in api server and expose ping endpoint

4b49f93

fix minor issues and manual testing

5b1b06b

bisgaard-itis changed the title ~~7348 add dummy functions api~~ Add example functions section to api-server Mar 19, 2025

bisgaard-itis changed the title ~~Add example functions section to api-server~~ 👽️ Add example functions section to api-server Mar 19, 2025

bisgaard-itis self-assigned this Mar 19, 2025

bisgaard-itis added a:apiserver api-server service Meta-modeling labels Mar 19, 2025

bisgaard-itis added this to the The Awakening milestone Mar 19, 2025

bisgaard-itis requested a review from wvangeit March 19, 2025 16:23

Merge branch 'master' into 7348-add-dummy-functions-api

f65dff1

bisgaard-itis marked this pull request as ready for review March 19, 2025 16:24

bisgaard-itis requested review from GitHK, matusdrobuliak66, pcrespov and sanderegg as code owners March 19, 2025 16:24

add __init__ to functions domain

1a18bf2

bisgaard-itis added 2 commits March 19, 2025 17:37

added functions section in api

5004019

make pylint happy

a24b9e7

GitHK reviewed Mar 20, 2025

View reviewed changes

sanderegg reviewed Mar 20, 2025

View reviewed changes

pcrespov requested changes Mar 20, 2025

View reviewed changes

bisgaard-itis added 4 commits March 20, 2025 14:11

Merge branch 'master' into 7348-add-dummy-functions-api

e52bac8

@pcrespov remove ping from oas

394587e

@sanderegg assert input @pcrespov annotate ping function

cb107a0

fix relative imports

e29dd1b

regenerate oas

5d277a9

bisgaard-itis requested review from GitHK and sanderegg March 20, 2025 13:54

sanderegg approved these changes Mar 20, 2025

View reviewed changes

GitHK requested changes Mar 21, 2025

View reviewed changes

bisgaard-itis added 2 commits March 21, 2025 13:48

add module decorator and app_setting

f1050a1

Merge branch 'master' into 7348-add-dummy-functions-api

fc5fe09

bisgaard-itis requested a review from pcrespov March 24, 2025 10:39

pcrespov approved these changes Mar 24, 2025

View reviewed changes

bisgaard-itis requested a review from GitHK March 24, 2025 14:50

GitHK reviewed Mar 24, 2025

View reviewed changes

Merge branch 'master' into 7348-add-dummy-functions-api

dcea149

bisgaard-itis requested a review from GitHK March 25, 2025 14:37

GitHK approved these changes Mar 26, 2025

View reviewed changes

Merge branch 'master' into 7348-add-dummy-functions-api

a27ae21

bisgaard-itis added the 🤖-automerge marks PR as ready to be merged for Mergify label Mar 26, 2025

bisgaard-itis and others added 3 commits March 27, 2025 07:04

Merge branch 'master' into 7348-add-dummy-functions-api

1aa663e

Merge branch 'master' into 7348-add-dummy-functions-api

869f747

fix import

d066273

update openapi specs

df7b5d8

bisgaard-itis merged commit 45a8b63 into ITISFoundation:master Mar 28, 2025
92 of 94 checks passed

bisgaard-itis deleted the 7348-add-dummy-functions-api branch March 28, 2025 09:43

matusdrobuliak66 mentioned this pull request Apr 15, 2025

🚀 Release v1.81.0 #7357

Closed

56 tasks

Add example functions section to api-server #7398

Add example functions section to api-server #7398

Uh oh!

Conversation

bisgaard-itis commented Mar 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What's the point of this PR?

What do these changes do?

Related issue/s

How to test

Dev-ops checklist

Uh oh!

bisgaard-itis commented Mar 19, 2025

Uh oh!

codecov bot commented Mar 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

GitHK left a comment

Choose a reason for hiding this comment

Uh oh!

sanderegg left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bisgaard-itis commented Mar 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bisgaard-itis commented Mar 20, 2025

Uh oh!

sanderegg left a comment

Choose a reason for hiding this comment

Uh oh!

GitHK left a comment

Choose a reason for hiding this comment

Uh oh!

pcrespov left a comment

Choose a reason for hiding this comment

Uh oh!

GitHK left a comment

Choose a reason for hiding this comment

Uh oh!

bisgaard-itis commented Mar 25, 2025

Uh oh!

GitHK left a comment

Choose a reason for hiding this comment

Uh oh!

bisgaard-itis commented Mar 26, 2025

Uh oh!

bisgaard-itis commented Mar 26, 2025

Uh oh!

mergify bot commented Mar 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🟠 Waiting for conditions to match

Uh oh!

bisgaard-itis commented Mar 28, 2025

Uh oh!

mergify bot commented Mar 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🟠 Waiting for conditions to match

Uh oh!

sonarqubecloud bot commented Mar 28, 2025

Quality Gate passed

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

Add example `functions` section to api-server #7398

Add example `functions` section to api-server #7398

bisgaard-itis commented Mar 19, 2025 •

edited

Loading

codecov bot commented Mar 19, 2025 •

edited

Loading

bisgaard-itis commented Mar 20, 2025 •

edited

Loading

mergify bot commented Mar 26, 2025 •

edited

Loading

mergify bot commented Mar 28, 2025 •

edited

Loading