[BUG] Copying array contents is slow and unnecessarily creates new array

[sfe-SparkFro]

Describe the bug

When copying one array's contents to another (eg. buf2[:] = buf1), it takes a long time, and ends up creating a new array in the process, unnecessarily increasing memory consumption.

ulab version: 6.8.0-2D-c

To Reproduce

Run the following (DMA part only works on RP2, tested with Pico 2):

from ulab import numpy as np
import time
import rp2
import gc

# Set buffer size in bytes
buffer_size = 100_000

# Log free memory
print("mem_free:", gc.mem_free())

# Create first buffer
t0 = time.ticks_us()
buf1 = np.zeros((buffer_size), dtype=np.uint8)
t1 = time.ticks_us()
print("create buf1:", t1 - t0, "microseconds")

# Log free memory
print("mem_free:", gc.mem_free())

# Create second buffer
t0 = time.ticks_us()
buf2 = np.zeros((buffer_size), dtype=np.uint8)
t1 = time.ticks_us()
print("create buf2:", t1 - t0, "microseconds")

# Log free memory
print("mem_free:", gc.mem_free())

# Create a DMA controller and configure it
dma = rp2.DMA()
bytes_per_transfer = 4 # 1, 2, or 4 bytes per transfer
dma_ctrl = dma.pack_ctrl(
    # 0 = 1 byte, 1 = 2 bytes, 2 = 4 bytes
    size = {1:0, 2:1, 4:2}[bytes_per_transfer],
    inc_write = True,
    inc_read = True
)
dma.config(
    read = buf1,
    write = buf2,
    count = buffer_size // bytes_per_transfer,
    ctrl = dma_ctrl
)

# Fill buf1 with data
t0 = time.ticks_us()
buf1[:] = 1
t1 = time.ticks_us()
print("filling buf1:", t1 - t0, "microseconds")

# Log free memory
print("mem_free:", gc.mem_free())

# Copy buf1 to buf2 using standard assignment
t0 = time.ticks_us()
buf2[:] = buf1
t1 = time.ticks_us()
print("copy buf1 to buf2:", t1 - t0, "microseconds")

# Log free memory
print("mem_free:", gc.mem_free())

# Copy buf1 to buf2 using DMA
t0 = time.ticks_us()
_ = dma.active(True)
while dma.active():
    pass
t1 = time.ticks_us()
print("copy buf1 to buf2 with dma:", t1 - t0, "microseconds")

# Log free memory
print("mem_free:", gc.mem_free())

Expected behavior

buf2[:] = buf1 should take about as long as filling one of the buffers, and should create a new array in the process.

With a Pico 2, the following is printed:

mem_free: 490016
create buf1: 1395 microseconds
mem_free: 389920
create buf2: 1576 microseconds
mem_free: 289824
filling buf1: 23131 microseconds
mem_free: 289408
copy buf1 to buf2: 47145 microseconds
mem_free: 189216
copy buf1 to buf2 with dma: 193 microseconds
mem_free: 189216

With a Pico 2 and buffer_size = 100_000, filling one buffer takes ~23ms, whereas copying takes ~47ms and unnecessarily allocates a whole extra array (100kB) for some reason. By contrast, the DMA can do the memory copy in just 0.2ms (~250x faster!).

Additional context

I understand that the copy with CPU will take longer than the DMA, but why does it take so much longer than simply filling the array, and why is whole new array being allocated? The project I'm working on uses large arrays, and it's pretty wasteful memory usage for whole new arrays to be create for a simple copy operation. Is that intended behavior? Is there a solution that's more efficient? I see copyto() isn't implemented, perhaps that should be a feature request? Just trying to better understand the problem before asking for that.

[v923z]

This is a more involving issue, I'll try to fix it in the coming days.

[v923z]

The comparison with the DMA transfer is not entirely fair, I believe. When you say buf2[:] = buf1, you actually have to loop through the elements of buf2, you can't simply pass the pointer. As far as micropython or ulab are concerned, buf2[:] and buf2[::2] have the same structure, and in the second case, it's obvious that passing the pointer wouldn't work. So,the statement buf2[:] = buf1 requires an iteration.

[v923z]

copyto might actually be useful for other purposes.

[v923z]

Describe the bug

When copying one array's contents to another (eg. buf2[:] = buf1), it takes a long time, and ends up creating a new array in the process, unnecessarily increasing memory consumption.

ulab version: 6.8.0-2D-c

To Reproduce

Run the following (DMA part only works on RP2, tested with Pico 2):

from ulab import numpy as np
import time
import rp2
import gc

# Set buffer size in bytes
buffer_size = 100_000

# Fill buf1 with data
t0 = time.ticks_us()
buf1[:] = 1
t1 = time.ticks_us()
print("filling buf1:", t1 - t0, "microseconds")

# Log free memory
print("mem_free:", gc.mem_free())

# Copy buf1 to buf2 using standard assignment
t0 = time.ticks_us()
buf2[:] = buf1
t1 = time.ticks_us()
print("copy buf1 to buf2:", t1 - t0, "microseconds")

With a Pico 2 and buffer_size = 100_000, filling one buffer takes ~23ms, whereas copying takes ~47ms

There is a lot of unrelated code in your example, so I couldn't separate the important segments on first reading. I'm not sure the last statement is entirely unexpected: when you say

buf2[:] = buf1

you have to iterate over both arrays, while in

buf1[:] = 1

you have one array and one value. This latter one doesn't require iteration.

[sfe-SparkFro]

copyto might actually be useful for other purposes.

#730 created!

[sfe-SparkFro]

The comparison with the DMA transfer is not entirely fair, I believe.

There is a lot of unrelated code in your example, so I couldn't separate the important segments on first reading.

Sorry, I probably should have left out the DMA stuff, it's definitely not relevant. My bad!

When you say buf2[:] = buf1, you actually have to loop through the elements of buf2, you can't simply pass the pointer. As far as micropython or ulab are concerned, buf2[:] and buf2[::2] have the same structure, and in the second case, it's obvious that passing the pointer wouldn't work. So,the statement buf2[:] = buf1 requires an iteration.

I'm not sure the last statement is entirely unexpected: when you say

buf2[:] = buf1
you have to iterate over both arrays, while in

buf1[:] = 1
you have one array and one value. This latter one doesn't require iteration.

Fair, iterating through both arrays could explain why buf2[:] = buf1 takes about twice as long as buf1[:] = 1. And yes, iterating through both is necessary if either array is not contiguous. Or even if the dtypes are different. And probably some other conditions.

However, that doesn't explain why a whole new array is being allocated. When buf2[:] = buf1 is called, is an intermediate copy of buf1 being created or something? If so, is that necessary?

Also, there is a special case when both arrays are contiguous and the dtypes match (and possibly other conditions), in which a direct copy can be performed with a simple memcopy(), which I expect will be faster. Perhaps that could be implemented as a feature? Maybe at least in #730?

[v923z]

Also, there is a special case when both arrays are contiguous and the dtypes match (and possibly other conditions), in which a direct copy can be performed with a simple memcopy(), which I expect will be faster. Perhaps that could be implemented as a feature? Maybe at least in #730?

The fact that the arrays are contiguous is not going to be enough: either of the arrays could be a dense view, but going backwards. So, memcopy could be used in the very special case only, when the arrays are contiguous, have the same dtype, and are advancing in the same direction. And you would have the same problem with copyto.

[v923z]

What you might want to consider is the frombuffer - tobytes methods, which are implemented: if you know what you're doing, then you can get access to the raw bytes in an array.

[sfe-SparkFro]

frombuffer

I need a copy, but frombuffer doesn't copy unfortunately.

tobytes

This works! Here's some simpler replication code:

from ulab import numpy as np
import time
import gc

# Set up buffers
buffer_size = 100_000
buf1 = np.linspace(0, buffer_size-1, buffer_size, dtype=np.uint8)
buf2 = np.zeros((buffer_size), dtype=np.uint8)

# Copy buf1 to buf2 using standard assignment
m0 = gc.mem_free()
t0 = time.ticks_us()
buf2[:] = buf1 # This is the line being tested
t1 = time.ticks_us()
m1 = gc.mem_free()
print("standard copy:", t1 - t0, "microseconds", m0 - m1, "bytes consumed")

# Copy buf1 to buf2 using tobytes
m0 = gc.mem_free()
t0 = time.ticks_us()
buf2.tobytes()[:] = buf1.tobytes() # This is the line being tested
t1 = time.ticks_us()
m1 = gc.mem_free()
print("tobytes copy:", t1 - t0, "microseconds", m0 - m1, "bytes consumed")

Output:

standard copy: 74881 microseconds 100240 bytes consumed
tobytes copy: 677 microseconds 64 bytes consumed

The conclusion here is that buf2[:] = buf1 consumes an excessive amount of memory (an entire array) and is substantially slower (~100x) when compared to buf2.tobytes()[:] = buf1.tobytes(), despite both performing functionally the same copy operation. Also using tobytes() is only ~3.5x slower than copying with the DMA, which is much closer to what I would expect.

My request is to improve the speed and memory consumption of buf2[:] = buf1 to be comparable to using tobytes(), at least for the special case when the arrays are contiguous, have the same dtype, and are advancing in the same direction (and whatever other conditions). This special case can occur frequently, so I think it's worth adding because of the massive performance benefit.