Trying to find async import() files

[verydanny]

I built a middleware for express that compiles the server's code for dev environments, it uses memfs presently, but the issue is when I require-from-string the compiler.outputFileSystem server file and that primary entry file tries to find a dynamic import(), it can't find that path based on webpack's output.path. It always checks for real paths (in my case /Users/dveremchuk/source/webpack4-react-new/lib/server/). I use '@babel/plugin-syntax-dynamic-import' to parse the import. I realize I can just convert the import to a deferred require, but I need to test things.

I tried rewriting node's Module._findPath() but that's a bit over my head now, and I didn't have time to dig into it.

Is it somehow possible to implement this with memfs/unionfs/fs-monkey?

Gist of what middleware is doing:
node_modules/my-custom-middleware/lib/main-middleware.js

const memoryFs = require('memory-fs')
const { vol } = require('memfs')
const join = require('memory-fs/lib/join')

client.compiler.outputFileSystem = new memoryFs()

// this actually gives me the error:
// TypeError: this.outputFileSystem.join is not a function (server)
// but I can "patch" it using memory-fs join

vol.join = join
server.compiler.outputFileSystem = vol

// later
// get main server entry file as serverFile
return new Promise((resolve, reject) => {
        webpackCompiler.outputFileSystem.readFile(serverFile, (err, buffer) => {
            if (err) {
                reject(err);
            } else {
                resolve(buffer.toString());
            }
        });
    })
// ERROR MODULE_NOT_FOUND
.then((source) => requireFromString(source, serverFile))

Test component I use:

import React, { Component } from 'react'

import('components/home')

export class App extends Component {
  render() {
    return (
      <div>
        Hello
      </div>
    )
  }
}

Error:

MODULE_NOT_FOUND: Cannot find module './home.js'
- loader.js:603 Function.Module._resolveFilename
    internal/modules/cjs/loader.js:603:15
  - loader.js:529 Function.Module._load
    internal/modules/cjs/loader.js:529:25
  - loader.js:657 Module.require
    internal/modules/cjs/loader.js:657:17
  - helpers.js:22 require
    internal/modules/cjs/helpers.js:22:18
  - server.js:708 Function.requireEnsure [as e]
    /Users/dveremchuk/source/webpack4-react-new/lib/server/server.js:708:25
  - server.js:86 Function.fn.e
    /Users/dveremchuk/source/webpack4-react-new/lib/server/server.js:86:40
  - app.tsx:12 eval
    webpack-internal:///./src/containers/app.tsx:12:21
  - server.js:796 Module../src/containers/app.tsx
    /Users/dveremchuk/source/webpack4-react-new/lib/server/server.js:796:1
  - server.js:689 __webpack_require__
    /Users/dveremchuk/source/webpack4-react-new/lib/server/server.js:689:30
  - server.js:60 fn
    /Users/dveremchuk/source/webpack4-react-new/lib/server/server.js:60:20

Webpack server config

module.exports = merge(base, {
  target: 'node',
  entry: [path.resolve(_root, 'src/server/server')],
  externals: [nodeExternals()],
  output: {
    path: path.resolve('lib/server'),
    filename: 'server.js',
    chunkFilename: '[name].js',
    hotUpdateMainFilename: 'hot-update-server.json',
    hotUpdateChunkFilename: '[id].hot-update-server.js',
    libraryTarget: 'commonjs',
  },
  module: {
    rules: [
      {
        test: /\.tsx?$/,
        exclude: /node_modules/,
        loader: 'babel-loader',
        options: {
          babelrc: false,
          configFile: path.resolve(__dirname, '../babel/babel.webpack.node'),
          cacheDirectory: true,
          cacheCompression: false,
        },
      },
    ],
  },
})

[heygrady]

You need to use unionfs and fs-monkey to make that work.

Rough example: webpack/webpack-dev-middleware#366 (comment)

When you use require-from-string it doesn’t do anything magic and won’t be able to resolve any other files from memory file system.

Unionfs and fs-monkey will patch require so that node will work with the memory file system and you won’t need to bother with requireFromString.

[heygrady]

Alternatively, you can configure your webpack bundle to not do any code splitting when bundling for server in dev mode. If you have one giant bundle file then webpack will handle all of the dynamic imports instead of relying on node to be able to find external files.

Node externals won’t work the way you’re expecting either, if you’re using requireFromString. So disable that too ;)

[verydanny]

This is super helpful, thank you!