Add ability to render server components inside client components (add support for react-router) (#1736)

* replace RSCPayloadContainer with RSC injection utils

* Add renderRequestId to rendering options and context for improved request tracking

- Introduced renderRequestId in render options to uniquely identify each render request.
- Updated server rendering code to include renderRequestId in the context.
- Enhanced client-side rendering to handle and log renderRequestId.
- Modified type definitions to accommodate the new renderRequestId field in RailsContext.
- Improved debugging output in RailsContext component for better visibility of context data.

* tmp

* trace used rsc payload streams on server and pass them to client

* replace registerRouter with wrapServerComponentRenderer utility function

* add artifical delay amd log statements

* Revert "add artifical delay amd log statements"

This reverts commit cd57337.

* Update import statements to include file extensions

* add missing docs and specs

* remove RSCRoots and replace its test with registerServerComponent tests

* update injectRSCPayload tests to expect the new behavior

* fix streamServerRenderedReactComponent and helper specs

* Update renderContextRows to exclude 'componentSpecificMetadata' from rendering

* fix knip errors

* Remove RSCServerRoot entry from package.json

* add test to test the behavior of hydrating Suspensable components

* initialize the rsc payload array in a sync manner when the generate rsc function is called to avoid hydration race conditions

* fix failing jest tests

* add TypeScript ignore comment in SuspenseHydration test for Node 18+ compatibility

* remove options parameter from registerServerComponent and update related tests

* refactor: rename WrapServerComponentRenderer to wrapServerComponentRenderer

* docs: add detailed JSDoc comments for RSC functions and utilities to improve code clarity and maintainability

* feat: implement post-SSR hooks for enhanced server-side rendering control and update package dependencies

* refactor: improve type safety in server component loading and streamline import statements

* fix: update module loading configuration to use environment-specific prefix and set crossOrigin to null

* refactor: update react-on-rails-rsc dependency to use SSR support and streamline imports for server components

* chore: update acorn and acorn-loose dependencies in yarn.lock to latest versions for improved compatibility

* refactor: clarify test description for child async component hydration in SuspenseHydration test

* fix problem of returning the wrong stream

* refactor: update RailsContext types to include server component capabilities and streamline related functions

* refactor: rename rscPayloadGenerationUrl to rscPayloadGenerationUrlPath and enhance type safety in RailsContext

* refactor: enhance error handling and improve code clarity in server component rendering and tests

* handle trailing commas while removing packages from at CI "oldest" tests

* don't run SuspenseHydration tests with CI oldest tests

* chore: update testPathIgnorePatterns to exclude additional test cases in CI

* chore: escape quotes in testPathIgnorePatterns for proper parsing in CI

* refactor: update RailsContext usage in tests to utilize RailsContextWithServerComponentCapabilities for improved type safety

* refactor: update test cases to use rscPayloadGenerationUrlPath for consistency in RailsContext

* refactor: unify stream types by introducing PipeableOrReadableStream

* refactor: replace direct checks for RSC support with a utility method for improved clarity and maintainability

* refactor: update stubbing in packs_generator_spec to improve clarity and maintainability

* refactor: make renderRequestId optional in server component rendering for improved flexibility and clarity

* refactor: remove renderRequestId from script tags in ReactOnRailsHelper specs for improved clarity

* refactor: optimize RSC payload handling and improve hook management for better performance and clarity

* refactor: ensure RSC support variable is reset in packs_generator_spec for improved test isolation

* refactor: enhance RSC component handling by introducing promise wrapper and improving stream processing for better error management

* refactor: make serverSideRSCPayloadParameters optional in RailsContextWithServerComponentCapabilities for improved flexibility

* handle error happen during rsc payload generation

* add rsc payload url to context only if rsc support enabled

* removed unneeded rubocop disable statement

Author: AbanoubGhadban

docs/guides/configuration.md

@@ -104,6 +104,31 @@ ReactOnRails.configure do |config|
   # you should include a name that matches your bundle name in your Webpack config.
   config.server_bundle_js_file = "server-bundle.js"
 
+  # When using React on Rails Pro with RSC support enabled, these configuration options work together:
+  #
+  # 1. In RORP, set `config.enable_rsc_support = true` in your react_on_rails_pro.rb initializer
+  #
+  # 2. The `rsc_bundle_js_file` (typically "rsc-bundle.js") contains only server components and
+  #    references to client components. It's generated using the RSC Webpack Loader which transforms
+  #    client components into references. This bundle is specifically used for generating RSC payloads
+  #    and is configured with the `react-server` condition.
+  config.rsc_bundle_js_file = "rsc-bundle.js"
+  #
+  # 3. The `react_client_manifest_file` contains mappings for client components that need hydration.
+  #    It's generated by the React Server Components Webpack plugin and is required for client-side
+  #    hydration of components.
+  # This manifest file is automatically generated by the React Server Components Webpack plugin. Only set this if you've configured the plugin to use a different filename.
+  config.react_client_manifest_file = "react-client-manifest.json"
+  #
+  # 4. The `react_server_client_manifest_file` is used during server-side rendering with RSC to
+  #    properly resolve references between server and client components.
+  #
+  # These files are crucial when implementing React Server Components with streaming, which offers
+  # benefits like reduced JavaScript bundle sizes, faster page loading, and selective hydration
+  # of client components.
+  # This manifest file is automatically generated by the React Server Components Webpack plugin. Only set this if you've configured the plugin to use a different filename.
+  config.react_server_client_manifest_file = "react-server-client-manifest.json"
+
   # `prerender` means server-side rendering
   # default is false. This is an option for view helpers `render_component` and `render_component_hash`.
   # Set to true to change the default value to true.

eslint.config.ts

@@ -161,7 +161,7 @@ const config = tsEslint.config([
     languageOptions: {
       parserOptions: {
         projectService: {
-          allowDefaultProject: ['eslint.config.ts', 'knip.ts', 'node_package/tests/*.test.ts'],
+          allowDefaultProject: ['eslint.config.ts', 'knip.ts', 'node_package/tests/*.test.{ts,tsx}'],
           // Needed because `import * as ... from` instead of `import ... from` doesn't work in this file
           // for some imports.
           defaultProject: 'tsconfig.eslint.json',

jest.config.js

@@ -14,8 +14,7 @@ export default {
   }),
   testEnvironment: 'jsdom',
   setupFiles: ['<rootDir>/node_package/tests/jest.setup.js'],
-  // React Server Components tests are compatible with React 19
-  // That only run with node version 18 and above
+  // React Server Components tests require React 19 and only run with Node version 18 (`newest` in our CI matrix)
   moduleNameMapper:
     nodeVersion < 18
       ? {

knip.ts

@@ -7,10 +7,12 @@ const config: KnipConfig = {
       entry: [
         'node_package/src/ReactOnRails.node.ts!',
         'node_package/src/ReactOnRailsRSC.ts!',
-        'node_package/src/registerServerComponent/client.ts!',
-        'node_package/src/registerServerComponent/server.ts!',
+        'node_package/src/registerServerComponent/client.tsx!',
+        'node_package/src/registerServerComponent/server.tsx!',
         'node_package/src/registerServerComponent/server.rsc.ts!',
-        'node_package/src/RSCClientRoot.ts!',
+        'node_package/src/wrapServerComponentRenderer/server.tsx!',
+        'node_package/src/wrapServerComponentRenderer/server.rsc.tsx!',
+        'node_package/src/RSCRoute.ts!',
         'eslint.config.ts',
       ],
       project: ['node_package/src/**/*.[jt]s{x,}!', 'node_package/tests/**/*.[jt]s{x,}'],

lib/react_on_rails/configuration.rb

@@ -312,8 +312,6 @@ def ensure_webpack_generated_files_exists
         react_client_manifest_file,
         react_server_client_manifest_file
       ].compact_blank
-
-      self.webpack_generated_files = files
     end
 
     def configure_skip_display_none_deprecation

lib/react_on_rails/helper.rb

@@ -358,14 +358,10 @@ def json_safe_and_pretty(hash_or_string)
     # second parameter passed to both component and store Render-Functions.
     # This method can be called from views and from the controller, as `helpers.rails_context`
     #
-    # rubocop:disable Metrics/AbcSize, Metrics/CyclomaticComplexity, Metrics/PerceivedComplexity
+    # rubocop:disable Metrics/AbcSize, Metrics/CyclomaticComplexity
     def rails_context(server_side: true)
       # ALERT: Keep in sync with node_package/src/types/index.ts for the properties of RailsContext
       @rails_context ||= begin
-        rsc_url = if ReactOnRails::Utils.react_on_rails_pro?
-                    ReactOnRailsPro.configuration.rsc_payload_generation_url_path
-                  end
-
         result = {
           componentRegistryTimeout: ReactOnRails.configuration.component_registry_timeout,
           railsEnv: Rails.env,
@@ -381,7 +377,10 @@ def rails_context(server_side: true)
         if ReactOnRails::Utils.react_on_rails_pro?
           result[:rorProVersion] = ReactOnRails::Utils.react_on_rails_pro_version
 
-          result[:rscPayloadGenerationUrl] = rsc_url if ReactOnRailsPro.configuration.enable_rsc_support
+          if ReactOnRails::Utils.rsc_support_enabled?
+            rsc_payload_url = ReactOnRailsPro.configuration.rsc_payload_generation_url_path
+            result[:rscPayloadGenerationUrlPath] = rsc_payload_url
+          end
         end
 
         if defined?(request) && request.present?
@@ -439,7 +438,7 @@ def load_pack_for_generated_component(react_component_name, render_options)
       append_stylesheet_pack_tag("generated/#{react_component_name}")
     end
 
-    # rubocop:enable Metrics/AbcSize, Metrics/CyclomaticComplexity, Metrics/PerceivedComplexity
+    # rubocop:enable Metrics/AbcSize, Metrics/CyclomaticComplexity
 
     private
 
@@ -651,7 +650,8 @@ def internal_react_component(react_component_name, options = {})
                                                 "data-trace" => (render_options.trace ? true : nil),
                                                 "data-dom-id" => render_options.dom_id,
                                                 "data-store-dependencies" => render_options.store_dependencies&.to_json,
-                                                "data-force-load" => (render_options.force_load ? true : nil))
+                                                "data-force-load" => (render_options.force_load ? true : nil),
+                                                "data-render-request-id" => render_options.render_request_id)
 
       if render_options.force_load
         component_specification_tag.concat(

lib/react_on_rails/packs_generator.rb

@@ -89,18 +89,13 @@ def client_entrypoint?(file_path)
 
     def pack_file_contents(file_path)
       registered_component_name = component_name(file_path)
-      load_server_components = ReactOnRails::Utils.react_on_rails_pro? &&
-                               ReactOnRailsPro.configuration.enable_rsc_support
+      load_server_components = ReactOnRails::Utils.rsc_support_enabled?
 
       if load_server_components && !client_entrypoint?(file_path)
-        rsc_payload_generation_url_path = ReactOnRailsPro.configuration.rsc_payload_generation_url_path
-
         return <<~FILE_CONTENT.strip
           import registerServerComponent from 'react-on-rails/registerServerComponent/client';
 
-          registerServerComponent({
-            rscPayloadGenerationUrlPath: "#{rsc_payload_generation_url_path}",
-          }, "#{registered_component_name}")
+          registerServerComponent("#{registered_component_name}");
         FILE_CONTENT
       end
 
@@ -146,8 +141,7 @@ def generated_server_pack_file_content
         "import #{name} from '#{relative_path(generated_server_bundle_file_path, component_path)}';"
       end
 
-      load_server_components = ReactOnRails::Utils.react_on_rails_pro? &&
-                               ReactOnRailsPro.configuration.enable_rsc_support
+      load_server_components = ReactOnRails::Utils.rsc_support_enabled?
       server_components = component_for_server_registration_to_path.keys.delete_if do |name|
         next true unless load_server_components
 

lib/react_on_rails/react_component/render_options.rb

@@ -15,9 +15,17 @@ class RenderOptions
       def initialize(react_component_name: required("react_component_name"), options: required("options"))
         @react_component_name = react_component_name.camelize
         @options = options
+        # The render_request_id serves as a unique identifier for each render request.
+        # We cannot rely solely on dom_id, as it should be unique for each component on the page,
+        # but the server can render the same page multiple times concurrently for different users.
+        # Therefore, we need an additional unique identifier that can be used both on the client and server.
+        # This ID can also be used to associate specific data with a particular rendered component
+        # on either the server or client.
+        # This ID is only present if RSC support is enabled because it's only used in that case.
+        @render_request_id = self.class.generate_request_id if ReactOnRails::Utils.rsc_support_enabled?
       end
 
-      attr_reader :react_component_name
+      attr_reader :react_component_name, :render_request_id
 
       def throw_js_errors
         options.fetch(:throw_js_errors, false)
@@ -139,6 +147,10 @@ def store_dependencies
         options[:store_dependencies]
       end
 
+      def self.generate_request_id
+        SecureRandom.uuid
+      end
+
       private
 
       attr_reader :options

lib/react_on_rails/utils.rb

@@ -128,6 +128,11 @@ def self.react_server_client_manifest_file_path
       return @react_server_manifest_path if @react_server_manifest_path && !Rails.env.development?
 
       asset_name = ReactOnRails.configuration.react_server_client_manifest_file
+      if asset_name.nil?
+        raise ReactOnRails::Error,
+              "react_server_client_manifest_file is nil, ensure it is set in your configuration"
+      end
+
       @react_server_manifest_path = File.join(generated_assets_full_path, asset_name)
     end
 
@@ -208,6 +213,15 @@ def self.react_on_rails_pro_version
                                     end
     end
 
+    def self.rsc_support_enabled?
+      return false unless react_on_rails_pro?
+
+      return @rsc_support_enabled if defined?(@rsc_support_enabled)
+
+      rorp_config = ReactOnRailsPro.configuration
+      @rsc_support_enabled = rorp_config.respond_to?(:enable_rsc_support) && rorp_config.enable_rsc_support
+    end
+
     def self.full_text_errors_enabled?
       ENV["FULL_TEXT_ERRORS"] == "true"
     end

node_package/src/ClientSideRenderer.ts

@@ -83,6 +83,18 @@ class ComponentRenderer {
     const { domNodeId } = this;
     const props = el.textContent !== null ? (JSON.parse(el.textContent) as Record<string, unknown>) : {};
     const trace = el.getAttribute('data-trace') === 'true';
+    const renderRequestId = el.getAttribute('data-render-request-id');
+
+    // The renderRequestId is optional and only present when React Server Components (RSC) support is enabled.
+    // When RSC is enabled, this ID helps track and associate server-rendered components with their client-side hydration.
+    const componentSpecificRailsContext = renderRequestId
+      ? {
+          ...railsContext,
+          componentSpecificMetadata: {
+            renderRequestId,
+          },
+        }
+      : railsContext;
 
     try {
       const domNode = document.getElementById(domNodeId);
@@ -93,7 +105,7 @@ class ComponentRenderer {
         }
 
         if (
-          (await delegateToRenderer(componentObj, props, railsContext, domNodeId, trace)) ||
+          (await delegateToRenderer(componentObj, props, componentSpecificRailsContext, domNodeId, trace)) ||
           // @ts-expect-error The state can change while awaiting delegateToRenderer
           this.state === 'unmounted'
         ) {
@@ -108,7 +120,7 @@ class ComponentRenderer {
           props,
           domNodeId,
           trace,
-          railsContext,
+          railsContext: componentSpecificRailsContext,
           shouldHydrate,
         });