bytearray resize memory penalty

[kjm1102]

If I run

import gc
Q=bytearray(20000); mv=memoryview(Q)
gc.collect(); b=gc.mem_free(); print(b)
q=memoryview(Q)[:18000]
Q=bytearray(q)
gc.collect(); a=gc.mem_free(); print(a, '  diff', b-a)

from the prompt in the IDE I get

167440
149408   diff 18032

If I run it again I get

147392
149392   diff -2000

Since I'm not setting up a new variable, just redefining Q to be 2k smaller with Q=bytearray(q), I was expecting the second result the first time around. Why the 18k memory loss the first time?

[jimmo]

@kjm1102 you still have q, the memoryview, pointing to the old bytearray.

[kjm1102]

I tried deleting q Jim

import gc
Q=bytearray(20000); mv=memoryview(Q)
gc.collect(); b=gc.mem_free(); print(b)
q=mv[:18000]
Q=bytearray(q)
del q
gc.collect(); a=gc.mem_free(); print(a, 'diff', b-a)

167440
149424 diff 18016

but no joy. Is there something else I could try to claw back that ram?

[jimmo]

Now you have mv still live. Also add del mv and you'll get what you expect.

[kjm1102]

I think I like your slice assignment idea better Jim

import gc
Q=bytearray(20000)
gc.collect(); b=gc.mem_free(); print(b)
Q[-2000:] = b''
gc.collect(); a=gc.mem_free(); print(a, 'diff', b-a)

167472
167456 diff 16

0.016k penalty to shrink an array by 2k Vs

import gc
Q=bytearray(20000); mv=memoryview(Q)
gc.collect(); b=gc.mem_free(); print(b)
Q=bytearray(mv[:1800])
del mv
gc.collect(); a=gc.mem_free(); print(a, 'diff', b-a)

167456
165632 diff 1824

1.8k via mv

[glenn20]

In the former, you have just modified your bytearray in place. in the second, you have created a second bytearray 1800 bytes long.

Note that in the second example, you dont need to instantiate the second bytearray - just use the memoryview object (eg. Q = mv[:1800]). Nonetheless, yes - the first approach is right for your application.

BTW, I think you meant to use 18000 instead of 1800 in the examples above if you are trying to compare apples with apples.

[kjm1102]

The penny has finally dropped for the distinction between modifying & creating Glenn, took me long enough! Sometimes the subtleties of upython, eg Q=Q[2000:] Vs Q[:]=Q[2000:], catch me out with respect to the memory penalty.

Is there an array equivalent of .find and .replace? I'm nearly there with this code refactoring to use a bytearray instead of a variable for a big file but I've hit another roadblock with substitutions.