demux: clear packet pool when upper memory limit is lowered

This clears packets from the pool, that would never be reused under new
settings. Note that depending on demuxer state this clears only unused
packets, in some cases demuxer may still hold more packets than the new
limit. Fox example demuxer doesn't forcefully clear forward cache when
reducing the size.

Also keep packet pool, even if current file is not cached, this is
useful for all sorts of things and may be useful for next file.

This act only on user options values, regardless of internal demuxer
caching state, which can vary depending on track type or source.

Author: kasper93

demux/demux.c

@@ -2519,8 +2519,14 @@ static void update_opts(struct demuxer *demuxer)
 // Make demuxing progress. Return whether progress was made.
 static bool thread_work(struct demux_internal *in)
 {
-    if (m_config_cache_update(in->d_user->opts_cache))
+    struct demux_opts *opts = in->d_user->opts;
+    size_t old_max_bytes = opts->max_bytes;
+    size_t old_max_bytes_bw = opts->max_bytes_bw;
+    if (m_config_cache_update(in->d_user->opts_cache)) {
         update_opts(in->d_user);
+        if (opts->max_bytes + opts->max_bytes_bw < old_max_bytes + old_max_bytes_bw)
+            demux_packet_pool_clear(in->packet_pool);
+    }
     if (in->tracks_switched) {
         execute_trackswitch(in);
         return true;