Animated GIF thumbnailing causes CPU/RAM exhaustion

[syldrathecat]

I am aware this project is not being actively worked on. This is information to help those who need it. I am not a Go developer.

The thumbnailing process for GIF is very CPU and memory intensive (which is a problem on its own), however this issue compounds with several others to create a near-catastrophic failure of the media server:

• The process leaks memory / grows in size unbounded
• The GIF thumbnailing operation has a time limit (by default 20 seconds, but up to 60 seconds)
• No constraints are placed on the thumbnailing process
• Some clients will repeatedly request these thumbnails each time their request fails

This resulted in MMR, at a steady state, consuming multiple CPU cores at 100%, and growing by several GB every few minutes.

I made two code patches to work around this issue, the first being to implement "maxAnimateSizeBytes".

This was found to not be adequate on its own, so I hard-coded a size limitation in to the GIF thumbnailer, based on the number of frames, the size of the source image, and the size of the destination image -- using some chosen values that I would expect to be possible to thumbnail in under 20 seconds.

diff --git a/pipelines/_steps/thumbnails/generate.go b/pipelines/_steps/thumbnails/generate.go
index 6b8f7c9..96a28fc 100644
--- a/pipelines/_steps/thumbnails/generate.go
+++ b/pipelines/_steps/thumbnails/generate.go
@@ -49,7 +49,14 @@ func Generate(ctx rcontext.RequestContext, mediaRecord *database.DbMedia, width
                        return
                }
 
-               i, err := thumbnailing.GenerateThumbnail(mediaStream, fixedContentType, width, height, method, animated, ctx)
+               var generateAnimated = animated
+
+               // Disable animation if the file is too big
+               if mediaRecord.SizeBytes > ctx.Config.Thumbnails.MaxAnimateSizeBytes {
+                       generateAnimated = false;
+               }
+
+               i, err := thumbnailing.GenerateThumbnail(mediaStream, fixedContentType, width, height, method, generateAnimated, ctx)
                if err != nil {
                        if i != nil && i.Reader != nil {
                                err2 := i.Reader.Close()
diff --git a/thumbnailing/i/gif.go b/thumbnailing/i/gif.go
index 676d0cb..06223dc 100644
--- a/thumbnailing/i/gif.go
+++ b/thumbnailing/i/gif.go
@@ -69,6 +69,20 @@ func (d gifGenerator) GenerateThumbnail(b io.Reader, contentType string, width i
                targetImg := image.NewPaletted(frameThumb.Bounds(), img.Palette)
                draw.FloydSteinberg.Draw(targetImg, frameThumb.Bounds(), frameThumb, image.Point{X: 0, Y: 0})
 
+               // Don't animate if the total size is too big to reasonably convert in under 20 seconds
+               if (width >= 640) {
+                       if (len(g.Image) * g.Config.Width * g.Config.Height > 40000000) {
+                               animated = false
+                       }
+               } else if (width >= 320) {
+                       if (len(g.Image) * g.Config.Width * g.Config.Height > 120000000) {
+                               animated = false
+                       }
+               }
+
                if !animated && i == targetStaticFrame {
                        t, err := pngGenerator{}.GenerateThumbnailOf(targetImg, width, height, method, ctx)
                        if err != nil || t != nil {

There is still a DoS avenue here by repeatedly requesting borderline GIF files with a shorter timeout_ms parameter.