Skip to content

Commit a58e718

Browse files
mattporrittmattporritt
committed
For: MDL-82627
AI subsystem and associated placement and provider plugins documentation.
1 parent e796fa2 commit a58e718

File tree

4 files changed

+663
-0
lines changed

4 files changed

+663
-0
lines changed

‎docs/apis/plugintypes/ai/index.md‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
---
2+
title: AI Plugins
3+
tags:
4+
- AI
5+
- LLM
6+
- Provider
7+
- Placement
8+
---
9+
10+
The AI subsystem in the LMS is designed to be extensible, allowing for the integration of external AI services.
11+
This is achieved through the use of AI plugins, which are divided into two types: Providers and Placements.
12+
13+
### Placements
14+
15+
The aim of Placements is to provide a consistent UX and UI for users when they use AI backed functionality.
16+
17+
Placement plugins leverage the functionality of the other components of the AI subsystem.
18+
This means plugin authors can focus on how users interact with the AI functionality, without needing to
19+
implement the AI functionality itself.
20+
21+
Because Placements are LMS plugins in their own right and are not "other" types of LMS plugins,
22+
it gives great flexibility in how AI functionality is presented to users.
23+
24+
See the [Placements](/apis/plugintypes/ai/placement.md) documentation for more information
25+
on developing Placement plugins.
26+
27+
### Providers
28+
29+
Provider plugins are the interface between the LMS AI subsystem and external AI systems.
30+
Their focus is on converting the data requested by an Action into the format needed by the
31+
external AI services API, and then correctly providing the response back from the AI
32+
in an Action Response object.
33+
34+
Because of this design the Providers that provide the AI Actions can be swapped out, mix and matched
35+
or upgraded; both without the need to update the Placement code and without the need to change the
36+
way users interact with the functionality.
37+
38+
See the [Providers](/apis/plugintypes/ai/provider.md) documentation for more information
39+
on developing Provider plugins.

‎docs/apis/plugintypes/ai/placement.md‎

Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,129 @@
1+
---
2+
title: Placements
3+
tags:
4+
- AI
5+
- LLM
6+
- Placement
7+
---
8+
9+
The aim of Placements is to provide a consistent UX and UI for users when they use AI backed functionality.
10+
11+
Placement plugins leverage the functionality of the other components of the AI subsystem.
12+
This means plugin authors can focus on how users interact with the AI functionality, without needing to
13+
implement the AI functionality itself.
14+
15+
Because Placements are LMS plugins in their own right and are not "other" types of LMS plugins,
16+
it gives great flexibility in how AI functionality is presented to users.
17+
18+
:::warning The Golden Rule:
19+
20+
Placements DO NOT know about Providers, and Providers DO NOT know about Placements.
21+
Everything should go via the Manager.
22+
23+
:::
24+
25+
Placements are defined as classes in their own namespace according to their plugin name.
26+
The naming convention for Action classes is `aiplacement_<plugin name>`,
27+
for example: `aiplacement_editor`. With corresponding namespaces.
28+
29+
Each Placement MUST inherit from the `\core_ai\placement` abstract class.
30+
They must also implement the following methods:
31+
32+
- `get_action_list(): array` This is the list of Actions that are supported by this Placement, for example the `aiplacement_editor` plugin defines this as:
33+
34+
```php
35+
public function get_action_list(): array {
36+
return [
37+
\core_ai\aiactions\generate_text::class,
38+
\core_ai\aiactions\generate_image::class,
39+
];
40+
}
41+
```
42+
43+
## Capabilities and Permissions
44+
45+
Placements are responsible for determining who and where a Placement (and by extension an Action can be used).
46+
It is not the job of Actions or Providers to determine access.
47+
48+
## Action Processing
49+
50+
The following is the basic workflow in order for a placement to have an action processed for a user request:
51+
52+
- The Placement instantiates a new action object of type they wish to use.
53+
- The action must be instantiated and passing it the required data. Each action will define what configuration it needs. As an example:
54+
55+
```php
56+
// Prepare the action.
57+
$action = new \core_ai\aiactions\generate_image(
58+
contextid: $contextid,
59+
userid: $USER->id,
60+
prompttext: $prompttext,
61+
quality: $quality,
62+
aspectratio: $aspectratio,
63+
numimages: $numimages,
64+
style: $style,
65+
);
66+
```
67+
68+
- The Placement then instantiates the Manager class and calls `process_action()`
69+
- passing in the configured action object:
70+
71+
```php
72+
// Send the action to the AI manager.
73+
$manager = \core\di::get(\core_ai\manager::class);
74+
$response = $manager->process_action($action);
75+
````
76+
77+
- The process_action() method will then return a response object (instance of `responses\response_base`).
78+
- It is up to the Placement to check for success (or not) of the response and pass the result back to the
79+
user or for further processing.
80+
81+
## Plugin Structure
82+
83+
Placement plugins reside in the `ai/placement` directory.
84+
85+
Each Placement is in a separate subdirectory and consists of a number of mandatory files and any other
86+
files the developer is going to use.
87+
88+
The following is the typical structure of a Placement plugin, using the Editor Placement as an example:
89+
90+
```bash
91+
.
92+
├── classes
93+
│   ├── external
94+
│   │   ├── generate_image.php
95+
│   │   └── generate_text.php
96+
│   ├── placement.php
97+
│   └── privacy
98+
│   └── provider.php
99+
├── db
100+
│   ├── access.php
101+
│   └── services.php
102+
├── lang
103+
│   └── en
104+
│   └── aiplacement_editor.php
105+
└── version.php
106+
107+
```
108+
109+
## Settings
110+
111+
Settings for the Placement should be defined in the `settings.php` file.
112+
Each Placement plugin should create a new admin settings page using `core_ai\admin\admin_settingspage_provider` class.
113+
114+
This is the same as for Provider plugins, for example:
115+
116+
```php
117+
use core_ai\admin\admin_settingspage_provider;
118+
119+
if ($hassiteconfig) {
120+
// Placement specific settings heading.
121+
$settings = new admin_settingspage_provider(
122+
'aiprovider_openai',
123+
new lang_string('pluginname', 'aiprovider_openai'),
124+
'moodle/site:config',
125+
true,
126+
);
127+
128+
...
129+
```

‎docs/apis/plugintypes/ai/provider.md‎

Lines changed: 228 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,228 @@
1+
---
2+
title: Providers
3+
tags:
4+
- AI
5+
- LLM
6+
- Provider
7+
---
8+
Providers are the interface between the LMS AI subsystem and external AI systems.
9+
Their focus should be on converting the data requested by an Action into the format needed
10+
by the external AI services API, and then correctly providing the response back from the AI
11+
in an Action Response object.
12+
13+
:::warning The Golden Rule:
14+
15+
Placements DO NOT know about Providers, and Providers DO NOT know about Placements.
16+
Everything should go via the Manager.
17+
18+
:::
19+
20+
Providers are defined as classes in their own namespace according to their plugin name.
21+
The naming convention for Action classes is `aiprovider_<plugin name>`,
22+
for example: `aiprovider_openai`, `aiprovider_azureai`. With corresponding namespaces.
23+
24+
Each Provider MUST inherit from the `\core_ai\provider` abstract class.
25+
They must also implement the following methods:
26+
27+
- `get_action_list(): array` This is the list of Actions that are supported by this Provider, for example the `aiprovider_openai` plugin defines this as:
28+
29+
```php
30+
public function get_action_list(): array {
31+
return [
32+
\core_ai\aiactions\generate_text::class,
33+
\core_ai\aiactions\generate_image::class,
34+
\core_ai\aiactions\summarise_text::class,
35+
];
36+
}
37+
```
38+
39+
## Process classes
40+
41+
For each action supported by the provider, the provider plugin **MUST** implement a `process_<action>` class,
42+
where `<action>` is the name of the action. For example: `process_generate_image`.
43+
44+
Every process action class **MUST** inherit from the `\core_ai\process_base` abstract class.
45+
46+
The process action class **MUST** implement a `process()` method. This method is responsible for
47+
converting the data requested by an Action into the format needed by the external AI services API,
48+
and then correctly providing the response back from the AI in an Action Response object.
49+
50+
The process action classes and process method are expected by the manager to exist and be callable.
51+
52+
As most provider plugins will support more than one action, it is recommended to create an
53+
`abstract_processor` class that inherits from the `\core_ai\process_base` class and then have each
54+
process action class inherit from this abstract class.
55+
56+
For example, the `aiprovider_openai` plugin defines an `abstract_processor` class that inherits from
57+
the `\core_ai\process_base` class and then the `process_generate_image`, `process_generate_text` and
58+
`process_summarise_text` classes inherit from this abstract class.
59+
60+
This can be visualised as follows:
61+
62+
```mermaid
63+
graph TD;
64+
A[process_base] --> B[abstract_processor]
65+
B[abstract_processor] --> C[process_generate_image]
66+
B --> D[process_generate_text]
67+
B --> E[process_summarise_text]
68+
```
69+
70+
Apart from this, Providers are free to define their own structure. It should be kept in mind that Providers
71+
are designed to be a "thin wrapper" around the external AI systems API. They shouldn't store data,
72+
or have their own UI elements (beyond what is required for configuration).
73+
74+
## Plugin Structure
75+
76+
Provider plugins reside in the `ai/provider` directory.
77+
78+
Each Provider is in a separate subdirectory and consists of a number of mandatory files and any other
79+
files the developer is going to use.
80+
81+
The following is the typical structure of a Provider plugin, using the OpenAI Provider as an example:
82+
83+
```bash
84+
.
85+
├── classes
86+
│   ├── abstract_processor.php
87+
│   ├── privacy
88+
│   │   └── provider.php
89+
│   ├── process_generate_image.php
90+
│   ├── process_generate_text.php
91+
│   ├── process_summarise_text.php
92+
│   └── provider.php
93+
├── lang
94+
│   └── en
95+
│   └── aiprovider_openai.php
96+
├── settings.php
97+
├── tests
98+
│   ├── fixtures
99+
│   │   ├── image_request_success.json
100+
│   │   ├── test.jpg
101+
│   │   └── text_request_success.json
102+
│   ├── process_generate_image_test.php
103+
│   ├── process_generate_text_test.php
104+
│   ├── process_summarise_text_test.php
105+
│   └── provider_test.php
106+
└── version.php
107+
108+
```
109+
110+
## Settings
111+
112+
Settings for the Provider should be defined in the `settings.php` file.
113+
Each Provider plugin should create a new admin settings page using `core_ai\admin\admin_settingspage_provider` class.
114+
115+
For example, the `aiprovider_openai` plugin defines this:
116+
117+
```php
118+
use core_ai\admin\admin_settingspage_provider;
119+
120+
if ($hassiteconfig) {
121+
// Provider specific settings heading.
122+
$settings = new admin_settingspage_provider(
123+
'aiprovider_openai',
124+
new lang_string('pluginname', 'aiprovider_openai'),
125+
'moodle/site:config',
126+
true,
127+
);
128+
129+
...
130+
```
131+
132+
## Rate Limiting
133+
134+
It is recommended that Providers implement rate limiting to prevent abuse of the external AI services.
135+
136+
To assist with this, the AI subsystem provides a `core_ai\rate_limiter` class that can be used to implement rate limiting.
137+
This class supports both user and system level rate limiting.
138+
139+
This should be implemented in a `is_request_allowed()` method in the Provider class. For example, from the
140+
`aiprovider_openai` plugin:
141+
142+
```php
143+
/**
144+
* Check if the request is allowed by the rate limiter.
145+
*
146+
* @param aiactions\base $action The action to check.
147+
* @return array|bool True on success, array of error details on failure.
148+
*/
149+
public function is_request_allowed(aiactions\base $action): array|bool {
150+
$ratelimiter = \core\di::get(rate_limiter::class);
151+
$component = \core\component::get_component_from_classname(get_class($this));
152+
153+
// Check the user rate limit.
154+
if ($this->enableuserratelimit) {
155+
if (!$ratelimiter->check_user_rate_limit(
156+
component: $component,
157+
ratelimit: $this->userratelimit,
158+
userid: $action->get_configuration('userid')
159+
)) {
160+
return [
161+
'success' => false,
162+
'errorcode' => 429,
163+
'errormessage' => 'User rate limit exceeded',
164+
];
165+
}
166+
}
167+
168+
// Check the global rate limit.
169+
if ($this->enableglobalratelimit) {
170+
if (!$ratelimiter->check_global_rate_limit(
171+
component: $component,
172+
ratelimit: $this->globalratelimit
173+
)) {
174+
return [
175+
'success' => false,
176+
'errorcode' => 429,
177+
'errormessage' => 'Global rate limit exceeded',
178+
];
179+
}
180+
}
181+
182+
return true;
183+
}
184+
```
185+
186+
If implementing rate limiting, settings for the rate limits should be provided in the plugin settings.
187+
188+
For example, the `aiprovider_openai` plugin provides settings for the user and global rate limits:
189+
190+
```php
191+
// Setting to enable/disable global rate limiting.
192+
$settings->add(new admin_setting_configcheckbox(
193+
'aiprovider_openai/enableglobalratelimit',
194+
new lang_string('enableglobalratelimit', 'aiprovider_openai'),
195+
new lang_string('enableglobalratelimit_desc', 'aiprovider_openai'),
196+
0,
197+
));
198+
199+
// Setting to set how many requests per hour are allowed for the global rate limit.
200+
// Should only be enabled when global rate limiting is enabled.
201+
$settings->add(new admin_setting_configtext(
202+
'aiprovider_openai/globalratelimit',
203+
new lang_string('globalratelimit', 'aiprovider_openai'),
204+
new lang_string('globalratelimit_desc', 'aiprovider_openai'),
205+
100,
206+
PARAM_INT,
207+
));
208+
$settings->hide_if('aiprovider_openai/globalratelimit', 'aiprovider_openai/enableglobalratelimit', 'eq', 0);
209+
210+
// Setting to enable/disable user rate limiting.
211+
$settings->add(new admin_setting_configcheckbox(
212+
'aiprovider_openai/enableuserratelimit',
213+
new lang_string('enableuserratelimit', 'aiprovider_openai'),
214+
new lang_string('enableuserratelimit_desc', 'aiprovider_openai'),
215+
0,
216+
));
217+
218+
// Setting to set how many requests per hour are allowed for the user rate limit.
219+
// Should only be enabled when user rate limiting is enabled.
220+
$settings->add(new admin_setting_configtext(
221+
'aiprovider_openai/userratelimit',
222+
new lang_string('userratelimit', 'aiprovider_openai'),
223+
new lang_string('userratelimit_desc', 'aiprovider_openai'),
224+
10,
225+
PARAM_INT,
226+
));
227+
$settings->hide_if('aiprovider_openai/userratelimit', 'aiprovider_openai/enableuserratelimit', 'eq', 0);
228+
```

0 commit comments

Comments
 (0)