Add full monorepo support for npm, yarn, and pnpm (#23)

fwang · web-flow · commit eeefffe0a0e3 · 2023-01-04T11:14:17.000-05:00
diff --git a/README.md b/README.md
@@ -63,6 +63,12 @@ OpenNext aims to support all Next.js 13 features. Some features are work in prog
 
    If your Next.js app does not use [middleware](https://nextjs.org/docs/advanced-features/middleware), `middleware-function` will not be generated.
 
+3. Add `.open-next` to your `.gitignore` file
+   ```
+   # OpenNext
+   /.open-next/
+   ```
+
 ## How does OpenNext work?
 
 When calling `open-next build`, OpenNext **builds the Next.js app** using the `@vercel/next` package. And then it **transforms the build output** to a format that can be deployed to AWS.
@@ -111,17 +117,55 @@ public,max-age=0,s-maxage=31536000,must-revalidate
 
 #### Image optimization function
 
-Create a Lambda function with the code from `.open-next/image-optimization-function`.
+Create a Lambda function with the code from `.open-next/image-optimization-function`, and the handler is `index.mjs`.
 
 This function handles image optimization requests when the Next.js `<Image>` component is used. The [sharp](https://www.npmjs.com/package/sharp) library is bundled with the function. And it is used to convert the image.
 
 Note that image optimization function responds with the `Cache-Control` header, and the image will be cached both at the CDN level and at the browser level.
 
 #### Server Lambda function
 
-Create a Lambda function with the code from `.open-next/server-function`.
+Create a Lambda function with the code from `.open-next/server-function`, and the handler is `index.mjs`.
+
+This function handles all the other types of requests from the Next.js app, including Server-side Rendering (SSR) requests and API requests. OpenNext builds the Next.js app in the **standalone** mode. The standalone mode generates a `.next` folder containing the **NextServer** class that does the request handling. It also generates a `node_modules` folder with **all the dependencies** required to run the `NextServer`.
+
+```
+  .next/              -> NextServer
+  node_modules/       -> dependencies
+```
+
+The server function adapter wraps around `NextServer` and exports a handler function that supports the Lambda request and response. The `server-function` bundle looks like:
+
+```diff
+  .next/              -> NextServer
+  node_modules/       -> dependencies
++ index.mjs           -> server function adapter
+```
+
+**Monorepo**
+The build output looks slightly different when the Next.js app is part of a monorepo. Imagine the app sits inside `packages/web`, the build output looks like:
+
+```
+  packages/
+    web/
+      .next/          -> NextServer
+      node_modules/   -> dependencies from root node_modules (optional)
+  node_modules/       -> dependencies from package node_modules
+```
 
-This function handles all the other types of requests from the Next.js app, including Server-side Rendering (SSR) requests and API requests. OpenNext builds the Next.js app in the **standalone** mode. The standalone mode generates a **NextServer** class that does the request handling. And the server function wraps around the NextServer.
+In this case, the server function adapter needs to be created inside `packages/web` next to `.next/`. This is to ensure the adapter can import dependencies from both `node_modules` folders. We could set the Lambda handler to point to `packages/web/index.mjs`, but it is a bad practice to have the Lambda configuration coupled with the project structure. Instead, we will add a wrapper `index.mjs` at the `server-function` bundle root that re-exports the adapter.
+
+```diff
+  packages/
+    web/
+      .next/          -> NextServer
+      node_modules/   -> dependencies from root node_modules (optional)
++     index.mjs       -> server function adapter
+  node_modules/       -> dependencies from package node_modules
++ index.mjs           -> adapter wrapper
+```
+
+This ensure the Lambda handler remains at `index.mjs`.
 
 #### CloudFront distribution
 
@@ -137,7 +181,7 @@ Create a CloudFront distribution, and dispatch requests to their cooresponding h
 
 #### Middleware Lambda@Edge function (optional)
 
-Create a Lambda function with the code from `.open-next/middleware-function`, and attach it to the `/_next/data/*` and `/*` behaviors as `viewer request` edge function. This allows the function to run your [Middleware](https://nextjs.org/docs/advanced-features/middleware) code before the request hits your server function, and also before cached content.
+Create a Lambda function with the code from `.open-next/middleware-function`, and the handler is `index.mjs`. Attach it to the `/_next/data/*` and `/*` behaviors as `viewer request` edge function. This allows the function to run your [Middleware](https://nextjs.org/docs/advanced-features/middleware) code before the request hits your server function, and also before cached content.
 
 The middleware function uses the Node.js 18 [global fetch API](https://nodejs.org/de/blog/announcements/v18-release-announce/#new-globally-available-browser-compatible-apis). It requires to run on Node.js 18 runtime. [See why Node.js 18 runtime is required.](#workaround-add-headersgetall-extension-to-the-middleware-function)
 
@@ -169,6 +213,55 @@ To workaround the issue, the server function checks if the request is to an HTML
 public, max-age=0, s-maxage=31536000, must-revalidate
 ```
 
+#### WORKAROUND: Set `NextServer` working directory (AWS specific)
+
+Next.js recommends using `process.cwd()` instead of `__dirname` to get the app directory. Imagine you have a `posts` folder in your app with markdown files:
+
+```
+pages/
+posts/
+  my-post.md
+public/
+next.config.js
+package.json
+```
+
+And you can build the file path like this:
+
+```ts
+path.join(process.cwd(), "posts", "my-post.md");
+```
+
+Recall in the [Server function](#server-lambda-function) section. In a non-monorepo setup, the `server-function` bundle looks like:
+
+```
+.next/
+node_modules/
+posts/
+  my-post.md    <- path is "posts/my-post.md"
+index.mjs
+```
+
+And `path.join(process.cwd(), "posts", "my-post.md")` resolves to the correct path.
+
+However, when the user's app is inside a monorepo (ie. at `/packages/web`), the `server-function` bundle looks like:
+
+```
+packages/
+  web/
+    .next/
+    node_modules/
+    posts/
+      my-post.md    <- path is "packages/web/posts/my-post.md"
+    index.mjs
+node_modules/
+index.mjs
+```
+
+And `path.join(process.cwd(), "posts", "my-post.md")` cannot be resolved.
+
+To workaround the issue, we change the working directory for the server function to where `.next/` is located, ie. `packages/web`.
+
 #### WORKAROUND: Pass headers from middleware function to server function (AWS specific)
 
 [Middleware](https://nextjs.org/docs/advanced-features/middleware) allows you to modify the request and response headers. This requires the middleware function to be able to pass custom headers defined in your Next.js app's middleware code to the server function.
@@ -210,22 +303,22 @@ To run `OpenNext` locally:
 1. Build `open-next`
    ```bash
    cd open-next
-   yarn build
+   pnpm build
    ```
 1. Run `open-next` in watch mode
    ```bash
-   yarn dev
+   pnpm dev
    ```
 1. Make `open-next` linkable from your Next.js app
    ```bash
-   yarn link
+   pnpm link --global
    ```
 1. Link `open-next` in your Next.js app
    ```bash
    cd path/to/my/nextjs/app
-   yarn link open-next
+   pnpm link --global open-next
    ```
-   Now you can make changes in `open-next`, and run `yarn open-next build` in your Next.js app to test the changes.
+   Now you can make changes in `open-next`, and run `pnpm open-next build` in your Next.js app to test the changes.
 
 ## FAQ
 
diff --git a/src/adapters/server-adapter.ts b/src/adapters/server-adapter.ts
@@ -11,6 +11,7 @@ import type {
 import NextServer from "next/dist/server/next-server.js";
 import { loadConfig } from "./util.js"
 
+setNextjsServerWorkingDirectory();
 const nextDir = path.join(__dirname, ".next");
 const config = loadConfig(nextDir);
 const htmlPages = loadHtmlPages();
@@ -49,8 +50,6 @@ const server = slsHttp(
   {
     binary: true,
     provider: "aws",
-    // TODO: add support for basePath
-    //basePath: process.env.NEXTJS_LAMBDA_BASE_PATH,
   },
 );
 
@@ -90,76 +89,15 @@ export async function handler(event: APIGatewayProxyEventV2, context: Context):
 // Helper functions //
 //////////////////////
 
+function setNextjsServerWorkingDirectory() {
+  // WORKAROUND: Set `NextServer` working directory (AWS specific) — https://github.com/serverless-stack/open-next#workaround-set-nextserver-working-directory-aws-specific
+  process.chdir(__dirname);
+}
+
 function loadHtmlPages() {
   const filePath = path.join(nextDir, "server", "pages-manifest.json");
   const json = fs.readFileSync(filePath, "utf-8");
   return Object.entries(JSON.parse(json))
     .filter(([_, value]) => (value as string).endsWith(".html"))
     .map(([key]) => key);
-}
-
-//const createApigHandler = () => {
-//  const config = loadConfig();
-//  const requestHandler = new NextServer(config).getRequestHandler();
-//
-//  return async (event) => {
-//    const request = convertApigRequestToNext(event);
-//    const response = await requestHandler(request);
-//    return convertNextResponseToApig(response);
-//  };
-//};
-//
-//export const handler = createApigHandler();
-
-//function convertApigRequestToNext(event) {
-//  let host = event.headers["x-forwarded-host"] || event.headers.host;
-//  let search = event.rawQueryString.length ? `?${event.rawQueryString}` : "";
-//  let scheme = "https";
-//  let url = new URL(event.rawPath + search, `${scheme}://${host}`);
-//  let isFormData = event.headers["content-type"]?.includes(
-//    "multipart/form-data"
-//  );
-//
-//  // Build headers
-//  const headers = new Headers();
-//  for (let [header, value] of Object.entries(event.headers)) {
-//    if (value) {
-//      headers.append(header, value);
-//    }
-//  }
-//
-//  return new Request(url.href, {
-//    method: event.requestContext.http.method,
-//    headers,
-//    body:
-//      event.body && event.isBase64Encoded
-//        ? isFormData
-//          ? Buffer.from(event.body, "base64")
-//          : Buffer.from(event.body, "base64").toString()
-//        : event.body,
-//  });
-//}
-//
-//async function convertNextResponseToApig(response) {
-//  // Build cookies
-//  // note: AWS API Gateway will send back set-cookies outside of response headers.
-//  const cookies = [];
-//  for (let [key, values] of Object.entries(response.headers.raw())) {
-//    if (key.toLowerCase() === "set-cookie") {
-//      for (let value of values) {
-//        cookies.push(value);
-//      }
-//    }
-//  }
-//
-//  if (cookies.length) {
-//    response.headers.delete("Set-Cookie");
-//  }
-//
-//  return {
-//    statusCode: response.status,
-//    headers: Object.fromEntries(response.headers.entries()),
-//    cookies,
-//    body: await response.text(),
-//  };
-//}
+}
diff --git a/src/build.ts b/src/build.ts
@@ -11,10 +11,8 @@ const outputDir = ".open-next";
 const tempDir = path.join(outputDir, ".build");
 
 export async function build() {
-  printVersion();
-
   // Pre-build validation
-  printHeader("Validating Next.js app");
+  printVersion();
   checkRunningInsideNextjsApp();
   setStandaloneBuildMode();
   const monorepoRoot = findMonorepoRoot();
@@ -46,7 +44,7 @@ function findMonorepoRoot() {
       || fs.existsSync(path.join(currentPath, "yarn.lock"))
       || fs.existsSync(path.join(currentPath, "pnpm-lock.yaml"))) {
       if (currentPath !== appPath) {
-        console.info("Monorepo root detected at", currentPath);
+        console.info("Monorepo detected at", currentPath);
       }
       return currentPath;
     }
@@ -78,9 +76,11 @@ function buildNextjsApp(monorepoRoot: string) {
 function printHeader(header: string) {
   header = `OpenNext — ${header}`;
   console.info([
+    "",
     "┌" + "─".repeat(header.length + 2) + "┐",
     `│ ${header} │`,
     "└" + "─".repeat(header.length + 2) + "┘",
+    "",
   ].join("\n"));
 }
 
@@ -116,25 +116,19 @@ function createServerBundle(monorepoRoot: string) {
     path.join(outputPath),
     { recursive: true, verbatimSymlinks: true }
   );
+
+  // Resolve path to the Next.js app if inside the monorepo
   // note: if user's app is inside a monorepo, standalone mode places
   //       `node_modules` inside `.next/standalone`, and others inside
   //       `.next/standalone/package/path` (ie. `.next`, `server.js`).
-  //       We need to move them to the root of the output folder.
-  if (monorepoRoot) {
-    const packagePath = path.relative(monorepoRoot, appPath);
-    fs.readdirSync(path.join(outputPath, packagePath))
-      .forEach(file => {
-        fs.renameSync(
-          path.join(outputPath, packagePath, file),
-          path.join(outputPath, file)
-        );
-      });
-  }
+  //       We need to output the handler file inside the package path.
+  const isMonorepo = monorepoRoot !== appPath;
+  const packagePath = path.relative(monorepoRoot, appPath);
 
   // Standalone output already has a Node server "server.js", remove it.
   // It will be replaced with the Lambda handler.
   fs.rmSync(
-    path.join(outputPath, "server.js"),
+    path.join(outputPath, packagePath, "server.js"),
     { force: true }
   );
 
@@ -145,7 +139,7 @@ function createServerBundle(monorepoRoot: string) {
   esbuildSync({
     entryPoints: [path.join(__dirname, "adapters", "server-adapter.js")],
     external: ["next"],
-    outfile: path.join(outputPath, "index.mjs"),
+    outfile: path.join(outputPath, packagePath, "index.mjs"),
     banner: {
       js: [
         "import { createRequire as topLevelCreateRequire } from 'module';",
@@ -155,6 +149,16 @@ function createServerBundle(monorepoRoot: string) {
       ].join(""),
     },
   });
+  // note: in the monorepo case, the handler file is output to
+  //       `.next/standalone/package/path/index.mjs`, but we want
+  //       the Lambda function to be able to find the handler at
+  //       the root of the bundle. We will create a dummy `index.mjs`
+  //       that re-exports the real handler.
+  if (isMonorepo) {
+    fs.writeFileSync(path.join(outputPath, "index.mjs"), [
+      `export * from "./${packagePath}/index.mjs";`,
+    ].join(""))
+  };
 }
 
 function createImageOptimizationBundle() {
