fix(computer-use): click search bar initially (#30)

Co-authored-by: Matt Marangoni <[email protected]>

Author: matthewjmarangoni

templates/python/computer-use/loop.py

@@ -55,7 +55,8 @@ class APIProvider(StrEnum):
 * You are utilising an Ubuntu virtual machine using {os.uname().machine} architecture with internet access.
 * When you connect to the display, CHROMIUM IS ALREADY OPEN. The url bar is not visible but it is there.
 * If you need to navigate to a new page, use ctrl+l to focus the url bar and then enter the url.
-* You won't be able  to see the url bar from the screenshot but ctrl-l still works.
+* You won't be able to see the url bar from the screenshot but ctrl-l still works.
+* As the initial step click on the search bar.
 * When viewing a page it can be helpful to zoom out so that you can see everything on the page.
 * Either that, or make sure you scroll down to see everything before deciding something isn't available.
 * When using your computer function calls, they take a while to run and send back to you.

templates/typescript/computer-use/loop.ts

@@ -14,7 +14,8 @@ const SYSTEM_PROMPT = `<SYSTEM_CAPABILITY>
 * You are utilising an Ubuntu virtual machine using ${process.arch} architecture with internet access.
 * When you connect to the display, CHROMIUM IS ALREADY OPEN. The url bar is not visible but it is there.
 * If you need to navigate to a new page, use ctrl+l to focus the url bar and then enter the url.
-* You won't be able  to see the url bar from the screenshot but ctrl-l still works.
+* You won't be able to see the url bar from the screenshot but ctrl-l still works.
+* As the initial step click on the search bar.
 * When viewing a page it can be helpful to zoom out so that you can see everything on the page.
 * Either that, or make sure you scroll down to see everything before deciding something isn't available.
 * When using your computer function calls, they take a while to run and send back to you.