image conversion fails 100% of the time after ~30,000 images

[xxx]

Once there are around 30k entries in /tmp/node-image-magick directory, all further conversions fail. I'm going to be updating this to use a more granular hash-based scheme that uses multiple directories that will hopefully prevent this.

The quick workaround is to remove some of the entries, which gets things running again.

[sdepold]

There is a max number of files according to the fs u are using. But i see
the point :)

Am 06.09.2012 um 23:10 schrieb Michael Dungan notifications@github.com:

Once there are around 30k entries in /tmp/node-image-magick directory, all
further conversions fail. I'm going to be updating this to use a more
granular hash-based scheme that uses multiple directories that will
hopefully prevent this.

The quick workaround is to remove some of the entries, which gets things
running again.

—
Reply to this email directly or view it on
GitHubhttps://github.com/dawanda/node-imageable/issues/12.

[xxx]

Yeah, the system in question is ext3 so 32k is the max for a single directory. I've switched it over to use a multi-layer structured based on a sha1 hash, which will limit to 4096 files in a single directory, so inodes will be the limit at that point.

I'm doing to do some more work to prevent a single image from being downloaded multiple times, and to use the same directory for both the tempfile and the converted file. I'll send a pull request once that work is finished.

[sdepold]

i'm wondering if i programmed it the way, that it creates subfolders in that tmp folder ?

[xxx]

I have it doing that right now - https://github.com/stipple/node-imageable . It's backward-compatible so any running installation should be able to upgrade with zero changes, though I've also added a config switch to set the root of the tmp directory that's used.