Product Collection block - data flow #44938
Replies: 0 comments 22 replies
-
Thanks for bringing this discussion up @kmanijak! I appreciate the thought you've put into this, providing us with multiple possible solutions and giving us a detailed brief with the pros and cons that come with each option. From my point of view Idea 3 sounds like the best solution based on how you've presented them so I think we're aligned on this. However, I am also interested to hear from @nerrad on his thoughts, specifically on Idea 2. Mostly because it appears to be quite complex and I'm wondering if we're missing some additional context on why this could be a good option to go with. As for Idea 1, given that it still leaves us with some additional requests being made, it still reduces the overall requests (I think, right?) so even though there may still be some performance concerns it's still an improvement which can be further iterated on in future. I'm also interested to hear why "Relies on the Store API as a data source" is listed as a con here. Since @\opr did a talk at WCEU which included usage of, and ways to extend the Store API I'm wondering why we're seeing this as a "con" so some more information would be helpful here! In summary, I'm aligned with you on Idea 3. I think it addresses both a reduction in complexity and improved performance but let's wait to see some further thoughts from others. Perhaps I'm missing something! (It wouldn't be the first time haha) |
Beta Was this translation helpful? Give feedback.
-
Hi @kmanijak, Thank you for a very detailed post. This will be very helpful in making the right decision. 🙌🏻 I concur with your & @tjcafferkey's assessment and find "Idea 3 - rely on getEntityRecords and redirect the call with api-fetch middleware to Store API" the most appealing due to its relative simplicity and minimized data manipulation. It also facilitates compatibility with core blocks like Title and Summary, which aligns with the overall philosophy of Gutenberg blocks. Regarding "Idea 2", I share the same concerns that it might introduce additional complexity and maintenance cost. Although I acknowledge that it might provide a path to a more reusable Product Collection block in the long run, the trade-off between potential benefits and the implementation complexity appears unfavorable at this stage. Regarding "Idea 1", I don't think this is a good solution because this isn't compatible with variations. In conclusion, I believe "Idea 3" provides the most straightforward, maintainable approach for the present. I'm eager to hear other perspectives on this, particularly from @nerrad, as his input could provide additional insights and factors to consider. 🙂 |
Beta Was this translation helpful? Give feedback.
-
Thanks for this fantastic write-up @kmanijak!
I think this at first glance seems like a good short-term solution, but has the potential to set us up for issues long-term. The product editor also uses middleware to route requests from the This seemingly works well and would guard against calls to the wrong API, however, as we move towards more React client-side routes, there is potential to hit the same While #37621 implemented a quick solution, I think we should look at reverting back to using the server-side registered endpoint and making calls for product information consistent, at least wherever we're using
I tend to agree with this as long as the information being added is not specific to blocks, but useful to all consumers of the REST API. Note that we can also return different data by passing a |
Beta Was this translation helpful? Give feedback.
-
The biggest problem with Idea 3 is that the Store API is a unauthed endpoint with no built-in permissions. In the editor context we need to use an endpoint that does support permissions checks so we could also implement the I don't think we should necessarily worry about changing the shape of the incoming request to the same as the Store API, but instead adapt the blocks to consume from the To be clear, there's a couple of important caveats:
|
Beta Was this translation helpful? Give feedback.
-
Specific to images I think we should try to make sure that our endpoints return images in the same shape as WP core REST endpoints do. Having srcset information (at a minimum) is fairly important for consumers. |
Beta Was this translation helpful? Give feedback.
-
Thanks for starting this discussion, @kmanijak!
I agree. |
Beta Was this translation helpful? Give feedback.
-
Thanks for your work on this and for sharing this detailed exploration @kmanijak! My thoughts on this topic are: IMO, the best solution is extending the WooCommerce REST API to fit our needs and upstream only the changes that can benefit the whole community. WooCommerce already has at least three different APIs: the core API, the store API, and the analytics API: IMO, this is an excellent opportunity to improve the current version of the core's API, and/or help pave the path for a new version. I believe this also allows us to move towards consolidating it into a single source of truth and perhaps deprecate the Store API at some point in the future (def. not right away, as @nerrad mentioned that other blocks still consume it). Elaborating a bit more on my opinion:
It is true that we would still need to intercept the request, but if we extend the WooCommerce REST API, we can shape the data to 100% fit our needs, so manipulating the data on the front end is unnecessary. Also, from my perspective, this is the ideal scenario: with the API returning the data with the shape and form we need, our components are cleaner/streamlined and solely focused on display/styles rather than modifying the API's response as an initial step.
From my perspective, focusing 100% on extending the core's API to serve our needs in this first moment is the ideal path to expedite things, as not all of our use cases will be fully aligned with what the rest of the community needs. As a follow-up task, we can then evaluate each one of our customizations and, in collaboration with other teams (most likely Proton), decide which ones are beneficial enough on a large scale and incorporate those into the core. |
Beta Was this translation helpful? Give feedback.
-
Just chiming in to mention that in the Command Palette we are also using However, after playing around with different solutions and reading this combo (thanks @imanish003 for pointing it out to me!), I'm leaning towards leaving it as-is until we have a generic solution that we can use everywhere. In the Command Palette, we only need the Product name and Product ID, so going with an interim solution would be straightforward. But I'm wary of implementing it with the risk of making things more difficult in the future, as it would be yet another instance to update. That's why I'm leaning towards not doing anything with it right now. I don't have a strong position on this, so happy to listen to your opinions if you think otherwise. I also wanted to bring it here mostly for awareness, as the Command Palette is a usage of |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Background
A few weeks ago we kicked off the development of Product Collection block. The idea behind this block is that it will replace the Products (Beta) block. Products (Beta) block was built as a variation of Query Loop block which gave us an advantage at the beginning, but we also faced multiple limitations while developing more advanced features. Based on the GitHub discussion: https://github.com/woocommerce/woocommerce-blocks/discussions/9010 we decided to start development of a standalone block that will cover Products (Beta) block functionalities, and, thanks to greater flexibility, allow us to provide more features in a shorter time.
Problem statement - data fetching
One of the challenges we faced with the Products (Beta) block was data (over)fetching in Editor.
Note: that applies only to Editor side - frontend is server-side-rendered, while Editor side is client-side-rendered and that’s where data fetching could be optimised.
In short:
Query Loop block does its own fetching of product postType data via
getEntityRecords
method (wp-json/wp/v2/product
). Some of that data is then passed down through the context to the blocks likecore/post-title
. There’s one request for the whole set of products (posts) being rendered within the Query Loop.Products (Beta) block is using Product Elements as inner blocks (e.g.
woocommerce/product-price
orwoocommerce/product-rating
and others). All of the Product Elements share the same React context. The first element that gets to be rendered for a particular product, triggers data fetch from Store API for this product (wp-json/wc/store/v1/products?include={id}
). There are as many requests as many products are rendered within Query Loop.(ref: per0F9-tT-p2)
As we can see there are opportunities here to optimise data fetching.
Products (Beta) is a variation of Query Loop block, hence we’re constrained by its implementation, however, with Product Collection block being a standalone block, we can optimize it significantly which is a scope of this issue: https://github.com/woocommerce/woocommerce-blocks/issues/9268.
Product Collection optimisations
Idea 1 - get all products from Store API in one request
The first approach to optimize data fetching was to stop using getEntityRecords and rely fully on the Store API. Also, instead of making requests for each product separately, we could get all the products at once and pass them downstream to the inner blocks.
It was suggested to explore other paths of getting the data to make Product Collection block better suited for other views (like Product Editor).
This idea was implemented as a first one and is available at this commit (be aware it still had troubles with Product Title and Product Summary which were not displayed correctly at that commit).
* More details about an unnecessary call mentioned in Pros and Cons table
There's still one unnecessary request made for products (with incorrect query details). This is due to the way with-product-data-context HOC is organized. There's a hook to fetch the product data and even though the product is provided and data from the hook will be ignored (check [if statement here](https://github.com/woocommerce/woocommerce-blocks/blob/trunk/assets/js/shared/hocs/with-product-data-context.js#L35)), it's still making a request. By default, it would make a request per each product separately, however thanks to the different data structure, the id [here](https://github.com/woocommerce/woocommerce-blocks/blob/trunk/assets/js/shared/hocs/with-product-data-context.js#L24), is not defined, hence there's just a single call made. This requires rethinking/refactoring, however, we need to keep in mind All Products and Products block (and potentially others) in which Product Elements are in use.Idea 2 - rely on
getEntityRecords
and redirect the call withapi-fetch
middleware to WooCommerce REST APIThis idea was raised as a code review idea.
WooCommerce Blocks relies mostly on Store API developed way before the entity data store (
@wordpress/core-data
) was created.The idea is that we should rely on
getEntityRecords
method and intercept the request to redirect it to WooCommerce REST API in order to fetch the products data by@wordpress/api-fetch
middleware. In that case, we rely on the core mechanism to get data and API that's designed to be used in the admin site.(Ref: pdToLP-Aj-p2)
The data flow in such a scenario is as follows:

The biggest problem here is that we cannot reconstruct the expected data shape based only on the fetched data, but we need to reach out to other resources as well.
But we need to keep in mind it happens on the front-end and we don’t have access to all the data available in the back-end, so that may require additional API calls.
Let’s take a look at images property as an example:
images
from Store API:images
from WooCommerce Store API:Our Image block relies on
srcset
, andsizes
properties. These properties are easily accessible via:wp_get_attachment_image_srcset
wp_get_attachment_image_sizes
methods on Product object, but we don’t have direct access to it on the front-end.
Options to overcome this:
If there are any other options you see - I’m open to suggestions!
Let’s keep in mind there are more differences like that, but some of them are easier to resolve. Also, I'd like to highlight that the data flow from the diagram is equivalent to calling the Store API directly from Product Template because essentially we get back a Store API-like data shape.
This whole idea is partially implemented in WIP PR.
Idea 3 - rely on
getEntityRecords
and redirect the call withapi-fetch
middleware to Store APIWe could implement the previous solution, but redirect the request to the Store API instead. In that case, we make a step towards getting data in a standardized way, but for the time being, we avoid the overhead of adjusting the data from WooCommerce REST API to Product Elements.
Also, with little response modification (showcased in this commit), we can easily use Product Title and Product Summary blocks as data is already stored in a store from which those blocks take the data.
Suggested next steps
My suggestion is to go with "Idea 3 - rely on getEntityRecords and redirect the call with api-fetch middleware to Store API". It simplifies the data fetching that's happening in Products (Beta) block. This allows us to easily use Product Title and Product Summary as a variation of core blocks.
Also, even though I see potential benefits of "Idea 2 - rely on
getEntityRecords
and redirect the call withapi-fetch
middleware to WooCommerce REST API", I don't think the value of this solution outweighs the effort and complexity it requires.Feedback required
I’d like to get feedback on how you think we should proceed with this before investing too much time into Idea 2 which is currently being implemented.
CC: @nerrad (I'd really appreciate your input considering you suggested exploring the Idea 2 🙏 ), @imanish003, @tjcafferkey
Beta Was this translation helpful? Give feedback.
All reactions