[QUERY] How to deploy with npm module dependencies?

[oeed]

Question

I am trying to use an npm module as there is no native Deno equivalent. I am trying to convert GitHub markdown to Slack's custom markdown format (if there's another way to do this please let me know!).

import slackifyMarkdown from "npm:slackify-markdown"

This works perfectly locally with slack run. However, when deploying I get a bunch of errors from deno_esbuild relating to resolving sub-dependencies. For example:

✘ [ERROR] NPM package not found. [plugin deno-loader]
    ../../../../Library/Caches/deno/deno_esbuild/remark-gfm@1.0.0/node_modules/remark-gfm/index.js:3:21:
      3 │ var syntax = require('micromark-extension-gfm')
        ╵                      ~~~~~~~~~~~~~~~~~~~~~~~~~
✘ [ERROR] NPM package not found. [plugin deno-loader]
    ../../../../Library/Caches/deno/deno_esbuild/remark-parse@9.0.0/node_modules/remark-parse/index.js:5:27:
      5 │ var fromMarkdown = require('mdast-util-from-markdown')
✘ [ERROR] Import 'https://./exists.ts' failed: error sending request for url (https://./exists.ts): error trying to connect: invalid dnsname [plugin deno-loader]
    https://deno.land/std@0.67.0/fs/move.ts:2:35:
      2 │ import { exists, existsSync } from "./exists.ts";
        ╵                                    ~~~~~~~~~~~~~
✘ [ERROR] Import 'https://../path/mod.ts' failed: error sending request for url (https://../path/mod.ts): error trying to connect: invalid dnsname [plugin deno-loader]
    https://deno.land/std@0.67.0/fs/ensure_link.ts:2:22:
      2 │ import * as path from "../path/mod.ts";

The strange thing is, it seems to resolve direct dependencies of slackify-markdown such as remark-gfm okay, but not their dependencies. Additionally, all the dependency files are indeed in the Deno cache folder, so I don't know why it's not resolving them. I haven't used Deno before so apologies if this is obvious!

Environment

The deno-slack versions
deno_slack_sdk: 2.12.0
deno_slack_api: 2.4.0

Deno runtime version
CLI: 2.26.0
deno: 1.44.4
typescript: 5.4.5
v8: 12.6.228.9

OS info

macOS 14.5

Requirements

Please read the Contributing guidelines and Code of Conduct before creating this issue or pull request. By submitting, you are agreeing to those rules.

[zimeg]

Hey @oeed 👋 Thank you for sharing this detailed report, and sorry you're bumping into this! To me it seems like your imports are alright, but this issue is appearing with bundling attempts. Going to move this issue to the slackapi/deno-slack-hooks repo for closer tracking 🔖

The strange thing is, it seems to resolve direct dependencies of slackify-markdown such as remark-gfm okay, but not their dependencies.

It's super interesting that the dependencies of slackify-markdown are appearing in deno_esbuild but aren't being "linked" or found in imports...

Similar issues around bundling strangeness with esbuild have appeared in slackapi/deno-slack-sdk#316 but I think it's worth keeping this open to track progress and continue investigations. At this time I'm not sure of an immediate workaround, but this is an issue we're actively tracking 🙏

[jdppettit]

Has there been any movement or work arounds for this issue? We are running into a similar problem but with @smithy/util-utf8:

⠦ Packaging app for deployment✘ [ERROR] NPM package not found. [plugin deno-loader]
    ../../Library/Caches/deno/deno_esbuild/@aws-crypto/util@5.2.0/node_modules/@aws-crypto/util/build/module/convertToBuffer.js:3:44:
      3 │ import { fromUtf8 as fromUtf8Browser } from "@smithy/util-utf8";
        ╵                                             ~~~~~~~~~~~~~~~~~~~
⠧ Packaging app for deploymentError bundling function file "lock" with esbuild
error: Uncaught (in promise) Error: Build failed with 1 error:
../../Library/Caches/deno/deno_esbuild/@aws-crypto/util@5.2.0/node_modules/@aws-crypto/util/build/module/convertToBuffer.js:3:44: ERROR: [plugin: deno-loader] NPM package not found.
  let error = new Error(text);
              ^
    at failureErrorWithLog (https://deno.land/x/esbuild@v0.19.4/mod.js:1626:15)
    at https://deno.land/x/esbuild@v0.19.4/mod.js:1035:25
    at runOnEndCallbacks (https://deno.land/x/esbuild@v0.19.4/mod.js:1461:45)
    at buildResponseToResult (https://deno.land/x/esbuild@v0.19.4/mod.js:1033:7)
    at https://deno.land/x/esbuild@v0.19.4/mod.js:1062:16
    at responseCallbacks.<computed> (https://deno.land/x/esbuild@v0.19.4/mod.js:679:9)
    at handleIncomingPacket (https://deno.land/x/esbuild@v0.19.4/mod.js:739:9)
    at readFromStdout (https://deno.land/x/esbuild@v0.19.4/mod.js:655:7)
    at https://deno.land/x/esbuild@v0.19.4/mod.js:1924:11
    at eventLoopTick (ext:core/01_core.js:175:7)
Packaging app for deployment

[filmaj]

Sorry, no update. We are not prioritizing this class of issues at this time. If you have feedback about this, or wish for Slack to prioritize it, please email feedback@slack.com and/or use the /feedback command from the Slack client.

[ThisGuyCodes]

I am running into this issue trying to use Octokit. I tried using esm.sh instead, but this ran into other issues.

Not being able to build Github integrations is seriously limiting my ability or desire to use this tool.

[ThisGuyCodes]

I was able to get slack deploy to work with native npm:* dependencies by adding the following to my deno.jsonc:

  "scopes": {
    "https://deno.land/x/deno_slack_hooks@1.3.1/": {
      "https://deno.land/x/esbuild_deno_loader@0.8.2/mod.ts": "jsr:@luca/esbuild-deno-loader@0.11.0"
    }
  },

Looking at recent commits, it appears a similar update (with more sweeping changes) has been made since the last release. Perhaps this is solved by cutting a new release?

[emilygarner]

@ThisGuyCodes Hello, I am running into a similar issue, and I have added your snippit to my "import_map.jsonc" because it would not work in my deno.jsonc. So I am able to deploy it now, but after deployment, one of my functions will not run due to a weird error :

Caught error from user supplied module: TypeError: Relative import path "querystring" not prefixed with / or ./ or ../
        If you want to use a built-in Node module, add a "node:" prefix (ex. "node:querystring").

But I am not even using this module at all in my function. Any help would be appreciated greatly. Is there something else you updated along with this snippit? Thank you.

[ThisGuyCodes]

@emilygarner in theory it should be the same in the import map, so that change seems fine.

I did not run into the problem you are having, but it's probably a sub-dependency of something you're using. Based on the full stack trace you can probably figure out what dependency that is, and add an import scope for that:

"scopes": {
  "DEPENDENCY_WITH_UNPREFIXED_NODE_PACKAGE": {
    "querystring": "node:querystring"
  },
  ...
},

The scopes are a way of overriding dependency resolution within other packages (where as the import map affects dependency resolution of your package).

I don't exactly remember why, but I did also end up overriding the esbuild version, this is my full scopes section now:

  "scopes": {
    "https://deno.land/x/deno_slack_hooks@1.3.1/": {
      "https://deno.land/x/esbuild_deno_loader@0.8.2/mod.ts": "jsr:@luca/esbuild-deno-loader@0.11.0",
      "https://deno.land/x/esbuild@v0.19.4/mod.js": "https://deno.land/x/esbuild@v0.24.0/mod.js"
    }
  },

[emilygarner]

@ThisGuyCodes Thank you for the response! I added all of that to my project but unfortunately I am still seeing the same error even after redeploying😞