Extending Upload to have nested folders

[m90khan]

I have a media collection which I use to save all media files. I want to organize the files into subfolders. Currently, all files are saved in folder: media

How can I save files from collection-1 to media/collection-1 when using media relationship.

Strapi provides media gallery out of the box in which you can create folders to structure your media gallery such as media/products or media/posts. Maybe we can define the folder inside the relationship such as below.

{ name: 'products', type: 'upload', relationTo: 'media', DESTINATION_FOLDER: '../../media/products' // Example },

[chladog]

Yes this would indeed be very nice feature to have. I'm in need of oraganizing the files to folders scoped to collection item. So when a Page is deleted I could just iterate the filenames in folder :pageId delete their DB records and wipe the whole folder. Nowadays I walk the whole document tree to find all uploads. Still have a lot of dangling files if they werent assigned to Page anymore.

[bertPB]

Would also really appreciate is this became a feature, right now I have way too many media files which I'm not able to view in an organised manner.

[NammDev]

Looking for this feature

[ryoheiw]

Has anybody found a workaround for this feature? Would be nice to have for sure.

[jobsteven]

Yeah, Only way to do it to give up and try to save all your files into S3. You can freely define your path as you want. I also have so so many images, saving all of them into ONE media folder is too silly and not logically organised as well. I've spent a whole day to look into it, but failed.

[SalahAdDin]

Any progress on this feature?

This should be a priority 1 feature.

[jmikrut]

Yes! This is actually already marked as priority one and you can expect it within a few months. 👍

#4190

[vishnumohanan404]

Shouldn't this be marked as closed?

[SalahAdDin]

It is marked as priority, but there is no PR yet @kingkong26 .

[kimstewartca]

Yes! This is actually already marked as priority one and you can expect it within a few months. 👍

#4190

Just checking in, would there happen to be any updates?
Thanks!

[ylahav]

I create a work-around...
My application is a photos albums... each album is a category and each media has a new (mandatory) field - category
I created two hooks functions (and it might that they can be combined together):

1. hook on 'BeforeChangeHook'
   export const mediaBeforeUpload: BeforeChangeHook = async ({ data, req, req: { payload, body = {}, res }, operation, }) => { var catId = data.category var mediaDir: string = path.resolve(__dirname, '../../../../media') if (!fs.existsSync(mediaDir + '/' + catId)) { fs.mkdirSync(mediaDir + '/' + catId) } data.filename = catId + "/" + data.filename }
2. hook on 'AfterChangeHook'
   `export const mediaAfterUpload: AfterChangeHook = async ({
   doc,
   req,
   req: { payload, body = {}, res },
   operation,
   }) => {
   var catId = doc.category
   var mediaDir: string = path.resolve(__dirname, '../../../../media')

if (!fs.existsSync(mediaDir + '/' + catId)) {
fs.mkdirSync(mediaDir + '/' + catId)
}
var origFile = path.basename(doc.filename)
fs.renameSync(mediaDir + "/" + origFile, mediaDir + '/' + doc.filename)
}`

[fhorray]

so, I've been searching for a while how can I do it natively, nothing yet?

[ringge]

It'd be great to have this feature supported natively

[janus-reith]

I think this might overlap with: #5664

[SalahAdDin]

It seems they are not beholding this feature.

[VenomApps]

Update on this?

[lazidoca]

I switched back to Strapi because of this issue. In my case, I have million of images (thumbnail, medium, large...), it is really slow if putting a large number of images in a single directory.

[m90khan]

I developed 2 applications on Payload but this was the reason that I switched to Strapi.
I have not checked the latest update. I hope they fixed it.

[SalahAdDin]

I developed 2 applications on Payload but this was the reason that I switched to Strapi. I have not checked the latest update. I hope they fixed it.

Strapi uses Slate and for the 5 version, it is broken, I was trying to add some plugins that are easy to add either on Lexical or Tiptap but... frustration.

[TechSynthesis]

@jmikrut, would really appreciate if this was launched soon

[koriolis]

@jmikrut Just checking in to see if this is still a priority as it's been nearly 10 months.

Not complaining, but because this is beyond my abilities I can't implement it myself or else I would already send in a PR.

[eloquent-peasant]

any updates of this ?

[matiduda]

I'm sad this hasn't been addressed yet

[mysteren]

I also tried to find a solution to this problem because I was interested in using this product. However, despite my searches, I have not been able to find an easy way to solve it.

After studying the source code, I was disappointed and realized that a quick solution was not possible. However, I can offer at least some kind of solution that is not ideal, but is capable of completing the task. Perhaps in the future I will be able to improve it or even create a plugin.

Sources

Create collections with hooks

Important: the collection is called 'Images' !

export const Images: CollectionConfig = {
  slug: 'images',
  access: {
    read: () => true,
  },
  fields: [
    {
      name: 'alt',
      type: 'text',
      required: true,
    },
  ],
  hooks: {
    beforeOperation: [
      ({ req, operation }) => {
        if ((operation === 'create' || operation === 'update') && req.file) {
          if (req.file) {
            const file = req.file as unknown as { md5: string }
            const hash = file.md5
            req.file.name = `${hash.slice(0, 2)}_${hash.slice(2, 4)}_${req.file.name}`
            console.log(req.file)
          }
        }
      },
    ],
    afterChange: [
      ({ collection, doc }) => {
        function move(dir: string, filename: string) {
          const oldPath = path.resolve(dir, filename)
          const matches = [...filename.matchAll(/^([^_]*)_([^_]*)_(.*)$/gm)][0]
          const newFolder = path.resolve(dir, matches[1], matches[2])
          const newPath = path.resolve(newFolder, matches[3])
          mkdirSync(newFolder, {
            recursive: true,
          })
          renameSync(oldPath, newPath)
        }

        if (collection.upload.staticDir) {
          move(collection.upload.staticDir, doc.filename)
          const sizes = doc.sizes as Record<string, { filename: string }>
          Object.values(sizes).forEach((item) => {
            move(collection.upload.staticDir ?? '', item.filename)
          })
        }
      },
    ],
    afterDelete: [
      ({ doc, collection }) => {
        function del(dir: string, filename: string) {
          const matches = [...filename.matchAll(/^([^_]*)_([^_]*)_(.*)$/gm)][0]
          const newFolder = path.resolve(dir, matches[1], matches[2])

          rmSync(path.resolve(newFolder, matches[3]), { force: true })
        }

        if (collection.upload.staticDir) {
          del(collection.upload.staticDir, doc.filename)
          const sizes = doc.sizes as Record<string, { filename: string }>
          Object.values(sizes).forEach((item) => {
            del(collection.upload.staticDir ?? '', item.filename)
          })
        }
      },
    ],
  },
  upload: {
    staticDir: uploadImagesFolder,
    adminThumbnail: '384',
    mimeTypes: ['image/jpeg', 'image/png', 'image/webp'],
    formatOptions: {
      format: 'webp',
      options: { quality: 85 },
    },
    imageSizes: [
     ...
    ],
  },
}

Create custom route

create file in app/api/{images - collection name}/file/[slug]/route.ts

import { uploadImagesFolder } from '@/collections/Images'
import { stat, readFile } from 'fs/promises'
import path from 'path'

function changePath(dir: string, filename: string) {
  const matches = [...filename.matchAll(/^([^_]*)_([^_]*)_(.*)$/gm)][0]
  const newPath = path.resolve(dir, matches[1], matches[2], matches[3])
  return newPath
}

export async function GET(_: Request, { params }: { params: Promise<{ slug: string }> }) {
  const slug = (await params).slug
  const filePath = changePath(uploadImagesFolder, slug)
  const stats = await stat(filePath)

  const headers = new Headers()
  headers.set('Content-Type', 'image/webp')
  headers.set('Content-Length', stats.size + '')
  const data = await readFile(filePath)

  return new Response(data, {
    headers,
    status: 200,
  })
}

[BedroomDev]

Why is there a screenshot of the landing page showing a Folder functionality for the Media if this is no where available ?

[SalahAdDin]

Such a good f question!

[davideperozzi]

probably available for the cloud customers only

[Coloby]

It was referring to the Digital Asset Management (DAM), still "coming soon" https://payloadcms.com/use-cases/digital-asset-management.
It doesn't seem so complex to make it enterprise-only, and the paragraph under the heading "The answer to build vs buy" might suggest it will be free? I hope

[BedroomDev]

Also with Live Edit and a few other features, i requested demos

[SalahAdDin]

Also with Live Edit and a few other features, i requested demos

Live edit is for pay, right?

[fidhal-ht]

I found a decently simple work around that works for me using the S3 cloud storage plugin with cloudflare R2.
I want each Project to have any number of images, and each image must be related to a project, and should be saved in a sub dir that is named after the project.

I gave the Media collection a field with a relationship to the project. Then set a beforeOperation hook so before it creates a Media document, it appends the project name to the prefix of the file.

The Project collection has a join field called mediaItems which lets you create media and edit it from the Project collection itself.

import type { CollectionConfig } from 'payload';
import slugify from 'slugify';

export const Media: CollectionConfig = {
  slug: 'media',
  upload: {
    staticDir: 'media',
    mimeTypes: ['image/*'],

  },
  fields: [
    {
      name: 'project',
      type: 'relationship',
      relationTo: 'projects', // Each item is associated with a project
      required: true, // Means every media item must be associated with a project
      hasMany: false, // It can only belong to 1 project
    },
  ],
  hooks: {

    beforeOperation: [
      async ({ req, operation }) => {
        if (operation === 'create' && req.data) {

          // get the project ID from the request data
          const projectId = req.data.project;
          if (!projectId) return;

          // find the project by ID
          const project = await req.payload.findByID({
            collection: 'projects',
            id: projectId,
          });

          if (project && project.Title) {
            const projectSlug = slugify(project.Title, { lower: true, strict: true, replacement: '_' });

            // set the prefix to the project slug, so the file gets saved in a sub directory inside the media folder
            req.data.prefix = `media/${projectSlug}`;

            console.log(`Media will be stored in: ${req.data.prefix}`);
          }
        }
      },
    ],

  },
};

import type { CollectionConfig } from 'payload'

export const Projects: CollectionConfig = {
  slug: 'projects',
  fields: [
    {
      name: 'Title',
      type: 'text',
      required: true,
    },
    {
      name: 'mediaItems',
      type: 'join', // join is a type of field that allows you to connect two collections
      collection: 'media', // joining with the media collection
      on: 'project', // the field in the media collection that we are joining on
    },
  ],
  admin: {
    useAsTitle: 'Title',
  },
};

[alexrah]

Having all uploads stored flat in a single folder it's ok just for toy websites, not for any real production website.

I don't understand why they don't care about this feature, maybe they think all production site should host assets on cloud storage.

I've keeping an eye on feat/folders about Folder View in List which is also mentioned in this thread, but I don't think it addresses the problem we are facing with uploads.

Besides some custom logic like folders named after collection and so on, it should be default as year/month/filename similar to what WP does.

I hope someone from the dev team can tell us if and when this issue will be resolved, then we can decide to move on to something that works best for assets or try to fix this by community efforts, like @fidhal-ht suggested by building a plugin

[coxmi]

Looks like it's very nearly there:
https://x.com/jarrodmflesch/status/1894508882837315933

[alexrah]

I'm aware of that. Can you point me where they state that Folder View also means uploading assets in subfolders?

[GermanJablo]

Closing as duplicated of #4190
This feature is a WIP and we are very close, so stay tuned! Thank you for your patience!

[alexrah]

Do you confirm that Folder View also means being able to upload assets in subfolders? Thanks

[GermanJablo]

Yes, that is in our plans!

[alexrah]

Great to hear that! Thanks

[jxh0414com]

`import { PayloadRequest } from "payload";
  export const getFolderPath = async (
    req: PayloadRequest,
    folderId: string
  ): Promise<string | undefined> => {
    const pathParts: string[] = [];
    let currentFolderId: string | null = folderId;

  if (!currentFolderId) {
    return undefined;
  }

while (currentFolderId) {
  try {
    const folder = await req.payload.findByID({
      collection: "payload-folders",
      id: currentFolderId,
      depth: 0,
    });

  if (folder?.slug) {
    pathParts.unshift(folder.slug);
  }

  currentFolderId = extractParentId(folder.folder);
} catch (error) {
  req.payload.logger.error(
    `Error fetching folder ${currentFolderId} for path generation:`,
    error
  );
  break;
}
}

return pathParts.join("/");};

/**
 * Helper function to extract parent folder ID from various formats
 */
const extractParentId = (parent: unknown): string | null => {
  if (parent && typeof parent === "string") {
    return parent;
  }

  return null;
};`

[SalahAdDin]

What is this for?

[jxh0414com]

this was my utils for adding prefix field in media , so it will upload to s3 with prefixes but might not be good since if user change folder name etc, there need to be follow up too. (i mean payload folder work but it just upload to s3 root bucket, my intention was when i create product folder in payloadcms then summer-edition folder, when i upload media it will check where is my current location and add prefix like /products/summer-edition/ for s3

[takayumi]

Does anyone know how to sync the folder path with your S3 Provider of choice? I was quite disappointed when I found out everything was being written to the root path of the S3 bucket.

The folder feature would've been perfect if it wasn't for this caveat...

Is it possible to read the folder hierarchy via the Payload API? You can probably use an on change hook to read & adjust the S3 paths based on the folder structure within Payload.

[SalahAdDin]

yeah you probably need to write some stuff yourself, which i find it annoying, debugging. (when i did my folder structure, the file doesnt delete in s3, when i click delete. what happen if your folder name change all the media prefix need to change? etc. i give up lol

is not there a plugin for that?

[jxh0414com]

well i dont think so. current i just have multiple media collection and do prefix, like all videos goes into videos/ instead of having folder prefix (using payloadcms folder ui and in s3 just have one large prefix like video, avatar, product-image etc.) since it might mess thing up. and i dont want custom plugin for that..... hope payloadcms have that in the future

[rezaarkan]

Im not too familiar with S3, but changing a "prefix" for a document wouldn't update its location in the bucket right? My assumption is you would need to invoke a specific operation from the S3 endpoint for moving files right?

[takayumi]

My assumption is you would need to invoke a specific operation from the S3 endpoint for moving files right?

Yep. The S3 adapter plugin only exposes upload and delete functionality. You'll need to create your own workaround for moving files around with the AWS S3 SDK.

[SalahAdDin]

My assumption is you would need to invoke a specific operation from the S3 endpoint for moving files right?

Yep. The S3 adapter plugin only exposes upload and delete functionality. You'll need to create your own workaround for moving files around with the AWS S3 SDK.

Open a PR on the plugin to expose the required API.

[cstamati]

Hi guys,

I just want to organize my assets like this in S3

my-s3-bucket/
├── projects/
│   └── {projectID}/
│       ├── textures/
│       ├── meshes/
│       └── images/
└── textures/  (global textures, so i can share with other projects)

I tried to change the prefix inside beforeOperation hook:

beforeOperation: [
	(args) => {
		if (args.operation === "create") {
			if (args.req.data?.project) {
				args.req.data.prefix = `projects/${args.req.data?.project}/textures`;
			}
		}
	},
]

but this will upload the texture in the "texture" folder and in projects/${args.req.data?.project}/textures

How can I fix this?

[jxh0414com]

why args.req.data.prefix?

just args.data.prefix

[cstamati]

Either way it dose not work.

export const beforeOperation: CollectionBeforeOperationHook = ({
	args,
	operation,
}) => {
	if (operation === "create") {
		if (args.data) {
			args.data.prefix = `projects/${args.data.project}/textures`;
			console.log("beforeOperation :: ", args.data.prefix);
		}
	}
};

I think it's related to clientUploads. When I disable this It works as expected!

[jxh0414com]

so what i did was creating a prefix field in the media collection (make sure you have that), well i didnt touch clientUploads so im not sure if that will break anything

async ({ req, operation, args }) => {
    if ((operation === "create" || operation === "update") && req.file) {
      args.data.prefix = await getFolderPath(req, args.data.folder);
    }
    return args;
  },