Skip to content

Commit fca5ad7

Browse files
authored
feat: process import from pre-processed chunks (medusajs#12527)
Fixes: FRMW-2968 In this PR we have done two major things. - First, we remove storing CSV contents within the workflow storage and neither store the JSON payloads to be created/updated in workflows. Earlier, they all were workflow inputs, hence were stored in the workflow - Introduce a naive concept of chunks and process chunks one by one. The next PR making chunking a bit more robust while using streams, adding ability to resume from the failed chunk and so on. > [!IMPORTANT] > The new endpoint `/admin/product/imports` is not in use yet. But it will be after the next (final) PR. ## Old context in workflow storage ![CleanShot 2025-05-19 at 17 11 08@2x](https://github.com/user-attachments/assets/798bdcc9-a368-4c1f-afdd-2a77f5ce43e0) ## New context in workflow storage ![CleanShot 2025-05-19 at 17 15 08@2x](https://github.com/user-attachments/assets/0463d035-403f-4600-a9cd-5af24d5fee7c)
1 parent 3e5794d commit fca5ad7

File tree

13 files changed

+868
-12
lines changed

13 files changed

+868
-12
lines changed

.changeset/hot-flies-ring.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
---
2+
"@medusajs/medusa": patch
3+
"@medusajs/core-flows": patch
4+
"@medusajs/js-sdk": patch
5+
"integration-tests-http": patch
6+
---
7+
8+
feat: process import from pre-processed chunks

integration-tests/http/__tests__/product/admin/product-export.spec.ts

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
import { IEventBusModuleService } from "@medusajs/types"
22
import { CommonEvents, Modules } from "@medusajs/utils"
3+
import os from "os"
34
import fs from "fs/promises"
45
import {
56
TestEventUtils,
@@ -16,7 +17,7 @@ import { csv2json } from "json-2-csv"
1617
jest.setTimeout(50000)
1718

1819
const getCSVContents = async (filePath: string) => {
19-
const asLocalPath = filePath.replace("http://localhost:9000", process.cwd())
20+
const asLocalPath = filePath.replace("http://localhost:9000", os.tmpdir())
2021
const fileContent = await fs.readFile(asLocalPath, { encoding: "utf-8" })
2122
await fs.rm(path.dirname(asLocalPath), { recursive: true, force: true })
2223
const csvRows = csv2json(fileContent)

integration-tests/http/__tests__/product/admin/product-import.spec.ts

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,6 @@ function prepareCSVForImport(fileContents: string, delimiter: string = ",") {
5555
}
5656

5757
medusaIntegrationTestRunner({
58-
dbName: "bulk-uploads-local",
5958
testSuite: ({ dbConnection, getContainer, api }) => {
6059
let baseCollection
6160
let baseType

0 commit comments

Comments
 (0)