Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 8 additions & 66 deletions docs/advanced/14_dependencies_in_typescript/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ Both of these runtimes allow you to include dependencies directly in the script,

There are however methods to have more control on your dependencies:

- Using [workspace dependencies](../../core_concepts/55_workspace_dependencies/index.mdx) for centralized dependency management at the workspace level.
- Leveraging [standard mode](#lockfile-per-script-inferred-from-imports-standard) on [web IDE](#web-ide) or [locally](#cli).
- Overriding dependencies [providing a package.json](#lockfile-per-script-inferred-from-a-packagejson).
- [Bundling](#bundle-per-script-built-by-cli) per script with CLI, more powerful and local only.

Moreover, there are other tricks, compatible with the methodologies mentioned above:
Expand All @@ -19,9 +19,13 @@ Moreover, there are other tricks, compatible with the methodologies mentioned ab

To learn more about how dependencies from other languages are handled, see [Dependency management & imports](../../advanced/6_imports/index.mdx).

![Dependency management & imports](../6_imports/dependency_management.png 'Dependency management & imports')

<div className="grid grid-cols-2 gap-6 mb-4">
<DocCard
title="Workspace dependencies"
description="Manage dependencies centrally at the workspace level for consistent environments."
href="/docs/core_concepts/workspace_dependencies"
/>
<DocCard
title="Dependency management & imports"
description="Windmill's strength lies in its ability to run scripts without having to manage a package.json directly."
Expand All @@ -36,7 +40,7 @@ To learn more about how dependencies from other languages are handled, see [Depe

## Lockfile per script inferred from imports (Standard)

In Windmill, you can run scripts without having to [manage a package.json](#lockfile-per-script-inferred-from-a-packagejson) directly. This is achieved by automatically parsing the imports and resolving the dependencies.
In Windmill, you can run scripts without having to manage dependency files directly. This is achieved by automatically parsing the imports and resolving the dependencies. For more control, you can use [workspace dependencies](../../core_concepts/55_workspace_dependencies/index.mdx).

When using Bun as the runtime for TypeScript in Windmill, dependencies are resolved directly from the script imports and their imports when using [sharing common logic](../5_sharing_common_logic/index.mdx). The TypeScript runtime Bun ensures 100% compatibility with Node.js without requiring any code modifications.

Expand Down Expand Up @@ -86,7 +90,7 @@ You can get those 3 files for each script by pulling your workspace with command

Editing a script is as simple as editing its content. The code can be edited freely in your IDE, and there are possibilities to even run it locally if you have the correct development environment setup for the script language.

Using [wmill CLI](../3_cli/index.mdx) command [`wmill script generate-metadata`](../3_cli/script.md#re-generating-a-script-metadata-file), lockfiles can be generated and updated as files. The CLI asks the Windmill servers to run dependency job, using either the [package.json (if present)](#lockfile-per-script-inferred-from-a-packagejson) or asking Windmill to automatically resolve it from the script's code as input, and from the output of those jobs, create the lockfiles.
Using [wmill CLI](../3_cli/index.mdx) command [`wmill script generate-metadata`](../3_cli/script.md#re-generating-a-script-metadata-file), lockfiles can be generated and updated as files. The CLI asks the Windmill servers to run dependency job, using either [workspace dependencies](../../core_concepts/55_workspace_dependencies/index.mdx) (if configured) or automatically resolving from the script's imports, and from the output of those jobs, create the lockfiles.
When a lockfile is present alongside a script at time of deployment by the CLI, no dependency job is run and the present lockfile is used instead.

<div className="grid grid-cols-2 gap-6 mb-4">
Expand All @@ -102,68 +106,6 @@ When a lockfile is present alongside a script at time of deployment by the CLI,
/>
</div>

## Lockfile per script inferred from a package.json

Although Windmill can [automatically resolve imports](#lockfile-per-script-inferred-from-imports-standard). It is possible to override the dependencies by providing a `package.json` file in the same directory as the script as you would do in a standard Node.js project, building and maintaining a package.json to declare dependencies.

<iframe
style={{ aspectRatio: '16/9' }}
src="https://www.youtube.com/embed/T8jMjpNvC2g"
title="Override Inferred Dependencies with Custom Dependency Files"
frameBorder="0"
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"
allowFullScreen
className="border-2 rounded-lg object-cover w-full dark:border-gray-800"
></iframe>

<br />

When doing [`wmill script generate-metadata`](../3_cli/script.md#re-generating-a-script-metadata-file), if a package.json is discovered, the closest one will be used as source-of-truth instead of being discovered from the imports in the script directly to generate the lockfile from the server.

You can write those package.json manually or through a standar `npm install <X>`.

Several package.json files can therefore coexist, each having authority over the scripts closest to it:

```
└── windmill_folder/
├── package.json
├── f/foo/
│ ├── package.json
│ ├── script1.ts
│ ├── # script1.ts will use the dependencies from windmill_folder/f/foo/package.json
│ └── /bar/
│ ├── package.json
│ ├── script2.ts
│ └── # script2.ts will use the dependencies from windmill_folder/f/foo/bar/package.json
└── f/baz/
├── script3.ts
└── # script3.ts will use the dependencies from windmill_folder/package.json
```

The Windmill [VS Code extension](../../cli_local_dev/1_vscode-extension/index.mdx) has a toggle "Infer lockfile" / "Use current lockfile".

With this toggle, you can choose to use the metadata lockfile (derived from package.json after `wmill script generate-metadata`) instead of inferring them directly from the script.

![Toggle Lockfile](../../cli_local_dev/1_vscode-extension/toggle_lockfile.png 'Toggle Lockfile')

<div className="grid grid-cols-2 gap-6 mb-4">
<DocCard
title="Local development"
description="Develop locally, push to git and deploy automatically to Windmill."
href="/docs/advanced/local_development"
/>
<DocCard
title="Command-line interface (CLI)"
description="The Windmill CLI, `wmill` allows you to interact with Windmill instances right from your terminal."
href="/docs/advanced/cli"
/>
<DocCard
title="VS Code extension"
description="Build scripts and flows in the comfort of your VS Code editor, while leveraging Windmill UIs for test & flows edition."
href="/docs/cli_local_dev/vscode-extension"
/>
</div>

## Bundle per script built by CLI

This method can only be deployed from the [CLI](../3_cli/index.mdx), on [local development](../4_local_development/index.mdx).
Expand Down
69 changes: 8 additions & 61 deletions docs/advanced/15_dependencies_in_python/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@ In Windmill [standard mode](#lockfile-per-script-inferred-from-imports-standard)
From the import lines, Windmill automatically handles the resolution and caching of the script dependencies to ensure fast and consistent execution (this is standard mode).

There are however methods to have more control on your dependencies:
- Using [workspace dependencies](../../core_concepts/55_workspace_dependencies/index.mdx) for centralized dependency management at the workspace level.
- Leveraging [standard mode](#lockfile-per-script-inferred-from-imports-standard) on [web IDE](#web-ide) or [locally](#cli).
- Using [PEP-723 inline script metadata](#pep-723-inline-script-metadata) for standardized dependency specification.
- Overriding dependencies [providing a requirements.txt](#lockfile-per-script-inferred-from-a-requirementstxt).

Moreover, there are other tricks, compatible with the methodologies mentioned above:
- [Sharing common logic with Relative Imports](#sharing-common-logic-with-relative-imports).
Expand All @@ -18,9 +18,13 @@ Moreover, there are other tricks, compatible with the methodologies mentioned ab

To learn more about how dependencies from other languages are handled, see [Dependency management & imports](../../advanced/6_imports/index.mdx).

![Dependency management & imports](../6_imports/dependency_management.png "Dependency management & imports")

<div className="grid grid-cols-2 gap-6 mb-4">
<DocCard
title="Workspace dependencies"
description="Manage dependencies centrally at the workspace level for consistent environments."
href="/docs/core_concepts/workspace_dependencies"
/>
<DocCard
title="Dependency management & imports"
description="Windmill's strength lies in its ability to run scripts without having to manage a requirements.txt directly."
Expand All @@ -35,7 +39,7 @@ To learn more about how dependencies from other languages are handled, see [Depe

## Lockfile per script inferred from imports (Standard)

In Windmill, you can run scripts without having to [manage a requirements.txt](#lockfile-per-script-inferred-from-a-requirementstxt) directly. This is achieved by automatically parsing the top-level imports and resolving the dependencies. For automatic dependency installation, Windmill will only consider these top-level imports.
In Windmill, you can run scripts without having to manage a requirements.txt directly. This is achieved by automatically parsing the top-level imports and resolving the dependencies. For automatic dependency installation, Windmill will only consider these top-level imports.

In Python, the top-level imports are automatically parsed on saving of the script and a list of imports is generated. For automatic dependency installation, Windmill will only consider these top-level imports.

Expand Down Expand Up @@ -74,7 +78,7 @@ You can get those 3 files for each script by pulling your workspace with command

Editing a script is as simple as editing its content. The code can be edited freely in your IDE, and there are possibilities to even run it locally if you have the correct development environment setup for the script language.

Using [wmill CLI](../3_cli/index.mdx) command [`wmill script generate-metadata`](../3_cli/script.md#re-generating-a-script-metadata-file), lockfiles can be generated and updated as files. The CLI asks the Windmill servers to run dependency job, using either the [requirements.txt (if present)](#lockfile-per-script-inferred-from-a-requirementstxt) or asking Windmill to automatically resolve it from the script's code as input, and from the output of those jobs, create the lockfiles.
Using [wmill CLI](../3_cli/index.mdx) command [`wmill script generate-metadata`](../3_cli/script.md#re-generating-a-script-metadata-file), lockfiles can be generated and updated as files. The CLI asks the Windmill servers to run dependency job, asking Windmill to automatically resolve it from the script's code as input, and from the output of those jobs, create the lockfiles.
When a lockfile is present alongside a script at time of deployment by the CLI, no dependency job is run and the present lockfile is used instead.

<div className="grid grid-cols-2 gap-6 mb-4">
Expand All @@ -88,63 +92,6 @@ When a lockfile is present alongside a script at time of deployment by the CLI,
description="Develop locally, push to git and deploy automatically to Windmill."
href="/docs/advanced/local_development"
/>
</div>

## Lockfile per script inferred from a requirements.txt

Although Windmill can [automatically resolve imports](#lockfile-per-script-inferred-from-imports-standard). It is possible to override the dependencies by providing a `requirements.txt` file in the same directory as the script as you would do in a standard Python project, building and maintaining a requirements.txt to declare dependencies.

<iframe
style={{ aspectRatio: '16/9' }}
src="https://www.youtube.com/embed/T8jMjpNvC2g"
title="Override Inferred Dependencies with Custom Dependency Files"
frameBorder="0"
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"
allowFullScreen
className="border-2 rounded-lg object-cover w-full dark:border-gray-800"
></iframe>

<br/>

When doing [`wmill script generate-metadata`](../3_cli/script.md#re-generating-a-script-metadata-file), if a requirements.txt is discovered, the closest one will be used as source-of-truth instead of being discovered from the imports in the script directly to generate the lockfile from the server.

You can write those requirements.txt manually or through a standard `pip install package_name`.

Several requirements.txt files can therefore coexist, each having authority over the scripts closest to it:

```
└── windmill_folder/
├── requirements.txt
├── f/foo/
│ ├── requirements.txt
│ ├── script1.py
│ ├── # script1.py will use the dependencies from windmill_folder/f/foo/requirements.txt
│ └── /bar/
│ ├── requirements.txt
│ ├── script2.py
│ └── # script2.py will use the dependencies from windmill_folder/f/foo/bar/requirements.txt
└── f/baz/
├── script3.py
└── # script3.py will use the dependencies from windmill_folder/requirements.txt
```

The Windmill [VS Code extension](../../cli_local_dev/1_vscode-extension/index.mdx) has a toggle "Infer lockfile" / "Use current lockfile".

With this toggle, you can choose to use the metadata lockfile (derived from requirements.txt after `wmill script generate-metadata`) instead of inferring them directly from the script.

![Toggle Lockfile](../../cli_local_dev/1_vscode-extension/toggle_lockfile.png 'Toggle Lockfile')

<div className="grid grid-cols-2 gap-6 mb-4">
<DocCard
title="Local development"
description="Develop locally, push to git and deploy automatically to Windmill."
href="/docs/advanced/local_development"
/>
<DocCard
title="Command-line interface (CLI)"
description="The Windmill CLI, `wmill` allows you to interact with Windmill instances right from your terminal."
href="/docs/advanced/cli"
/>
<DocCard
title="VS Code extension"
description="Build scripts and flows in the comfort of your VS Code editor, while leveraging Windmill UIs for test & flows edition."
Expand Down
16 changes: 1 addition & 15 deletions docs/advanced/3_cli/script.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,21 +85,7 @@ Note that you can explicitly exclude (or include) specific files or folders to b
wmill flow generate-locks
```

### package.json & requirements.txt

When doing `wmill script generate-metadata`, if a `package.json` or `requirements.txt` is discovered, the closest one will be used as source-of-truth instead of being discovered from the imports in the script directly to generate the lockfile from the server.

Below is a video on how to override Windmill inferred dependencies by [providing custom package.json](../14_dependencies_in_typescript/index.mdx#lockfile-per-script-inferred-from-a-packagejson) or requirements.txt.

<iframe
style={{ aspectRatio: '16/9' }}
src="https://www.youtube.com/embed/T8jMjpNvC2g"
title="Override Inferred Dependencies with Custom Dependency Files"
frameBorder="0"
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"
allowFullScreen
className="border-2 rounded-lg object-cover w-full dark:border-gray-800"
></iframe>
For centralized dependency management, see [workspace dependencies](../../core_concepts/55_workspace_dependencies/index.mdx).

### Arguments

Expand Down
Binary file not shown.
8 changes: 1 addition & 7 deletions docs/advanced/4_local_development/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -168,13 +168,7 @@ Locally, the lockfile is respected. It "wins" over the dependencies pinned via t

Locally, flows and apps are stored in a folder ending with `.flow` and `.app` respectively. They include a `flow.yaml` and `app.yaml` file respectively, and the 3 files mentioned [above](#editing-and-creating-scripts-locally) for each inline script they contain.

### Package.json & requirements.txt

If you have a `package.json` or `requirements.txt` file in the same or parent directory as your script, the `wmill script generate-metadata` command will automatically include the dependencies in the metadata file.

The lockfile for a file generated with generated-metadata will be overriden by the closest package.json from a parent or current folder if there is one.

![Generate Metadata Path](./generate_metadata_path.png 'Generate Metadata Path')
For centralized dependency management, see [workspace dependencies](../../core_concepts/55_workspace_dependencies/index.mdx).

But what if you want to create a new script from scratch? It's also easy, just create a file with the correct extension (or run `wmill script bootstrap <path> <language>`), and simply run `generate-metadata` command to generate the metadata file. The name of the file and its location in the folder will become the script path in Windmill.

Expand Down
Binary file removed docs/advanced/6_imports/dependency_management.png
Binary file not shown.
Loading