Skip to content

Commit 32076b1

Browse files
quanruyuyutaotao
andauthored
fix: lint and ts error, docs link (#77)
* fix: lint and ts error, docs link * fix: update API key instructions and add model configuration in README files * fix: update README files to include API key instructions and model configuration * Feat/new env config (#78) * fix: command in readme * feat(core): update config for connectivity test * fix: resolve merge conflicts and clean up environment variable handling in tests --------- Co-authored-by: yuyutaotao <167746126+yuyutaotao@users.noreply.github.com>
1 parent b7c1063 commit 32076b1

File tree

46 files changed

+489
-387
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

46 files changed

+489
-387
lines changed

android/javascript-sdk-demo/README.md

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -11,11 +11,14 @@ If you want to use Android with Vitest, please refer to [android-with-vitest-dem
1111
create `.env` file
1212

1313
```shell
14-
# replace by your gpt-4o api key
15-
OPENAI_API_KEY="YOUR_TOKEN"
14+
# Replace with your own API key
15+
MIDSCENE_MODEL_BASE_URL="https://.../compatible-mode/v1"
16+
MIDSCENE_MODEL_API_KEY="sk-abcdefghijklmnopqrstuvwxyz"
17+
MIDSCENE_MODEL_NAME="qwen3-vl-plus"
18+
MIDSCENE_MODEL_FAMILY="qwen3-vl"
1619
```
1720

18-
Refer to this document if your want to use other models like Qwen: https://midscenejs.com/choose-a-model
21+
Refer to this document if your want to use other models like Qwen: https://midscenejs.com/model-strategy.html
1922

2023
### Run demo
2124

android/javascript-sdk-demo/demo-run-yaml.ts

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,18 @@
1-
import { AndroidAgent, AndroidDevice, getConnectedDevices } from '@midscene/android';
2-
import "dotenv/config";
1+
import {
2+
AndroidAgent,
3+
AndroidDevice,
4+
getConnectedDevices,
5+
} from '@midscene/android';
6+
import 'dotenv/config';
37

4-
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
8+
const sleep = (ms: number | undefined) => new Promise((r) => setTimeout(r, ms));
59
Promise.resolve(
610
(async () => {
711
const devices = await getConnectedDevices();
812
const page = new AndroidDevice(devices[0].udid);
913

1014
// 👀 init Midscene agent
11-
const agent = new AndroidAgent(page,{
15+
const agent = new AndroidAgent(page, {
1216
aiActContext:
1317
'If any location, permission, user agreement, etc. popup, click agree. If login page pops up, close it.',
1418
});
@@ -17,7 +21,6 @@ Promise.resolve(
1721

1822
await sleep(5000);
1923

20-
2124
// 👀 run YAML with agent
2225
const { result } = await agent.runYaml(`
2326
tasks:
Lines changed: 30 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,18 @@
1-
import { AndroidAgent, AndroidDevice, getConnectedDevices } from '@midscene/android';
2-
import "dotenv/config"; // read environment variables from .env file
3-
4-
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
1+
import {
2+
AndroidAgent,
3+
AndroidDevice,
4+
getConnectedDevices,
5+
} from '@midscene/android';
6+
import 'dotenv/config'; // read environment variables from .env file
7+
8+
const sleep = (ms: number | undefined) => new Promise((r) => setTimeout(r, ms));
59
Promise.resolve(
610
(async () => {
711
const devices = await getConnectedDevices();
812
const page = new AndroidDevice(devices[0].udid);
913

1014
// 👀 init Midscene agent
11-
const agent = new AndroidAgent(page,{
15+
const agent = new AndroidAgent(page, {
1216
aiActContext:
1317
'If any location, permission, user agreement, etc. popup, click agree. If login page pops up, close it.',
1418
});
@@ -18,34 +22,40 @@ Promise.resolve(
1822
await sleep(5000);
1923

2024
// 👀 type keywords, perform a search
21-
await agent.aiAct(
22-
'type "Headphones" in search box, click search button'
23-
);
25+
await agent.aiAct('type "Headphones" in search box, click search button');
2426

2527
// 👀 wait for the loading
26-
await agent.aiWaitFor("there is at least one headphone item on page");
28+
await agent.aiWaitFor('there is at least one headphone item on page');
2729
// or you may use a plain sleep:
2830
// await sleep(5000);
2931

3032
// 👀 understand the page content, find the items
3133
const items = await agent.aiQuery(
32-
"{itemTitle: string, price: Number}[], find item in list and corresponding price"
34+
'{itemTitle: string, price: Number}[], find item in list and corresponding price'
3335
);
34-
console.log("headphones in stock", items);
36+
console.log('headphones in stock', items);
3537

36-
const isMoreThan1000 = await agent.aiBoolean("Is the price of the headphones more than 1000?");
37-
console.log("isMoreThan1000", isMoreThan1000);
38+
const isMoreThan1000 = await agent.aiBoolean(
39+
'Is the price of the headphones more than 1000?'
40+
);
41+
console.log('isMoreThan1000', isMoreThan1000);
3842

39-
const price = await agent.aiNumber("What is the price of the first headphone?");
40-
console.log("price", price);
43+
const price = await agent.aiNumber(
44+
'What is the price of the first headphone?'
45+
);
46+
console.log('price', price);
4147

42-
const name = await agent.aiString("What is the name of the first headphone?");
43-
console.log("name", name);
48+
const name = await agent.aiString(
49+
'What is the name of the first headphone?'
50+
);
51+
console.log('name', name);
4452

45-
const location = await agent.aiLocate("What is the location of the first headphone?");
46-
console.log("location", location);
53+
const location = await agent.aiLocate(
54+
'What is the location of the first headphone?'
55+
);
56+
console.log('location', location);
4757

4858
// 👀 assert by AI
49-
await agent.aiAssert("There is a category filter on the left");
59+
await agent.aiAssert('There is a category filter on the left');
5060
})()
5161
);

android/vitest-demo/README.md

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -11,13 +11,16 @@ This is a demo to show how to use adb to control android to do some automation t
1111
create `.env` file
1212

1313
```shell
14-
# replace by your gpt-4o api key
15-
OPENAI_API_KEY="YOUR_TOKEN"
14+
# Replace with your own API key
15+
MIDSCENE_MODEL_BASE_URL="https://.../compatible-mode/v1"
16+
MIDSCENE_MODEL_API_KEY="sk-abcdefghijklmnopqrstuvwxyz"
17+
MIDSCENE_MODEL_NAME="qwen3-vl-plus"
18+
MIDSCENE_MODEL_FAMILY="qwen3-vl"
1619
```
1720

1821
connect an Android device with [adb](https://developer.android.com/tools/adb)
1922

20-
Refer to this document if your want to use other models like Qwen: https://midscenejs.com/choose-a-model
23+
Refer to this document if your want to use other models like Qwen: https://midscenejs.com/model-strategy.html
2124

2225
### Install
2326

android/vitest-demo/tests/setting.test.ts

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -11,14 +11,13 @@ describe(
1111
async () => {
1212
await it('Android settings page demo for scroll', async () => {
1313
const devices = await getConnectedDevices();
14-
const agent = await agentFromAdbDevice(devices[0].udid,{
14+
const agent = await agentFromAdbDevice(devices[0].udid, {
1515
aiActContext:
1616
'If any location, permission, user agreement, etc. popup, click agree. If login page pops up, close it.',
1717
});
1818

1919
await agent.launch('com.android.settings/.Settings');
2020

21-
2221
await agent.aiAct('scroll list to bottom');
2322
await agent.aiAct('open "More settings"');
2423
await agent.aiAct('scroll list to bottom');
@@ -27,5 +26,5 @@ describe(
2726
await agent.aiAct('swipe up one screen');
2827
});
2928
},
30-
360 * 1000,
29+
360 * 1000
3130
);
Lines changed: 21 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,12 @@
1-
import { AndroidAgent, AndroidDevice, getConnectedDevices } from '@midscene/android';
1+
import {
2+
AndroidAgent,
3+
AndroidDevice,
4+
getConnectedDevices,
5+
} from '@midscene/android';
26
import { beforeAll, describe, expect, it, vi } from 'vitest';
37
import 'dotenv/config'; // read environment variables from .env file
48

5-
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
9+
const sleep = (ms: number | undefined) => new Promise((r) => setTimeout(r, ms));
610

711
vi.setConfig({
812
testTimeout: 240 * 1000,
@@ -16,7 +20,7 @@ describe('Test todo list', () => {
1620
beforeAll(async () => {
1721
const devices = await getConnectedDevices();
1822
const page = new AndroidDevice(devices[0].udid);
19-
agent = new AndroidAgent(page,{
23+
agent = new AndroidAgent(page, {
2024
aiActContext:
2125
'If any location, permission, user agreement, etc. popup, click agree. If login page pops up, close it.',
2226
});
@@ -29,43 +33,47 @@ describe('Test todo list', () => {
2933
'ai todo',
3034
async () => {
3135
await agent.aiAct(
32-
"type 'Study JS today' in the task box input and press the Enter key",
36+
"type 'Study JS today' in the task box input and press the Enter key"
3337
);
3438
await agent.aiAct(
35-
"type 'Study Rust tomorrow' in the task box input and press the Enter key",
39+
"type 'Study Rust tomorrow' in the task box input and press the Enter key"
3640
);
3741
await agent.aiAct(
38-
"type 'Study AI the day after tomorrow' in the task box input and press the Enter key",
42+
"type 'Study AI the day after tomorrow' in the task box input and press the Enter key"
3943
);
4044
await agent.aiAct(
41-
'move the mouse to the second item in the task list and click the delete button on the right of the second task',
45+
'move the mouse to the second item in the task list and click the delete button on the right of the second task'
4246
);
4347
await agent.aiAct(
44-
'click the check button on the left of the second task',
48+
'click the check button on the left of the second task'
4549
);
4650
await agent.aiAct(
47-
"click the 'completed' status button below the task list",
51+
"click the 'completed' status button below the task list"
4852
);
4953

5054
const list = await agent.aiQuery('string[], the complete task list');
5155
expect(list.length).toEqual(1);
5256

5357
await agent.aiAssert(
54-
'Near the bottom of the list, there is a tip shows "1 item left".',
58+
'Near the bottom of the list, there is a tip shows "1 item left".'
5559
);
5660

5761
const name = await agent.aiString('What is the name of the first todo?');
5862
console.log('name', name);
5963

60-
const todoCount = await agent.aiNumber('How many todos are there in the list?');
64+
const todoCount = await agent.aiNumber(
65+
'How many todos are there in the list?'
66+
);
6167
console.log('todoCount', todoCount);
6268

6369
const isAllCompleted = await agent.aiBoolean('Is all todos completed?');
6470
console.log('isAllCompleted', isAllCompleted);
6571

66-
const location = await agent.aiLocate('What is the location of the first todo?');
72+
const location = await agent.aiLocate(
73+
'What is the location of the first todo?'
74+
);
6775
console.log('location', location);
6876
},
69-
720 * 1000,
77+
720 * 1000
7078
);
7179
});

android/yaml-scripts-demo/README.md

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -6,11 +6,13 @@
66
create `.env` file
77

88
```shell
9-
# replace by your gpt-4o own
10-
OPENAI_API_KEY="YOUR_TOKEN"
11-
```
9+
# Replace with your own API key
10+
MIDSCENE_MODEL_BASE_URL="https://.../compatible-mode/v1"
11+
MIDSCENE_MODEL_API_KEY="sk-abcdefghijklmnopqrstuvwxyz"
12+
MIDSCENE_MODEL_NAME="qwen3-vl-plus"
13+
MIDSCENE_MODEL_FAMILY="qwen3-vl"
1214

13-
Refer to this document if your want to use other models like Qwen: https://midscenejs.com/choose-a-model
15+
Refer to this document if your want to use other models like Qwen: https://midscenejs.com/model-strategy.html
1416

1517
## Install
1618

android/yaml-scripts-demo/midscene-scripts/maps-navigation.yaml

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,3 @@
1-
# search headphone on ebay, extract the items info into a json file, and assert the shopping cart icon
2-
31
android:
42
# launch: https://www.ebay.com
53
deviceId: s4ey59ytbitot4yp

android/yaml-scripts-demo/midscene-scripts/twitter-auto-like.yaml

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,3 @@
1-
# search headphone on ebay, extract the items info into a json file, and assert the shopping cart icon
2-
31
android:
42
deviceId: s4ey59ytbitot4yp
53

bridge-mode-demo/README.md

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,11 +9,14 @@ This is a demo to show how to use bridge mode to control the page on your deskto
99
create `.env` file
1010

1111
```shell
12-
# replace by your gpt-4o api key
13-
OPENAI_API_KEY="YOUR_TOKEN"
12+
# Replace with your own API key
13+
MIDSCENE_MODEL_BASE_URL="https://.../compatible-mode/v1"
14+
MIDSCENE_MODEL_API_KEY="sk-abcdefghijklmnopqrstuvwxyz"
15+
MIDSCENE_MODEL_NAME="qwen3-vl-plus"
16+
MIDSCENE_MODEL_FAMILY="qwen3-vl"
1417
```
1518

16-
Refer to this document if your want to use other models like Qwen: https://midscenejs.com/choose-a-model
19+
Refer to this document if your want to use other models like Qwen: https://midscenejs.com/model-strategy.html
1720

1821
### Install
1922

0 commit comments

Comments
 (0)