Running Kernel Memory as a service in Asp.Net #554
-
Context / ScenarioI am looking at running Kernel Memory as a service in Asp.Net API. QuestionI took a look at the Service.AspNetCore code and saw that IKernelMemory is registered as a singleton. With an API i was wondering why we should register IKernelMemory as a singleton instead of as a scoped service? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
|
The recommended deployment uses asynchronous queues to execute tasks in the background, to avoid blocking user requests and to deal with long running jobs. For example, when uploading a file it might take seconds/minutes to process, depending on the pipeline in use, depending on external services bandwidth, and in case of transient errors requiring the service to retry. To avoid blocking TCP/HTTP requests while documents are processed, the ingestion is designed to run as a task outside the scope of a HTTP request. Using a singleton allows KM components to work. It's also perfectly safe to use KM as a singleton. On the other hand, if for some reason you cannot use singletons, then I would recommend configuring KM in Serverless mode. Everything will run synchronously, blocking HTTP requests while ingestion is running, and once a request is complete the KM instance can be disposed. |
Beta Was this translation helpful? Give feedback.
The recommended deployment uses asynchronous queues to execute tasks in the background, to avoid blocking user requests and to deal with long running jobs.
For example, when uploading a file it might take seconds/minutes to process, depending on the pipeline in use, depending on external services bandwidth, and in case of transient errors requiring the service to retry.
To avoid blocking TCP/HTTP requests while documents are processed, the ingestion is designed to run as a task outside the scope of a HTTP request. Using a singleton allows KM components to work. It's also perfectly safe to use KM as a singleton.
On the other hand, if for some reason you cannot use singletons, then I would …