Changing the object’s class on the fly fails on Raspberry Pico

[BillKujawa]

The ring buffer code works using Thonny's 3.10 python but fails using Micropython v1.22.2.
The output appears to allow the change of the object's class on the fly but still references the original class function definitions.
(see the last section of each output below)
The work around code is commented out in the original class code.
Note: the initialization is modified from the original Oreilly example to allow saving the state of the buffer (data & cur) to allow
resetting upon restart.

Question: Is this ability not implemented in Micropython, did I miss something, or is this a bug? The test was not tried on
another microprocessor.

`class RingBuffer:
"""
Source: https://www.oreilly.com/library/view/python-cookbook/0596001673/ch05s19.html
Author: Sébastien Keim
Class that implements a not-yet-full buffer
Class changes when buffer is full + 1
"""
def init(self,size_max, data=None):
self.max = size_max
self.cur = -1 #None
self.full = False
if data != None:
self.data = data
else:
self.data = []

class __Full:
    """ class that implements a full buffer """
    def append(self, x):
        """ Append an element overwriting the oldest one. """
        print(f"{self.cur=}")
        self.data[self.cur] = x
        self.cur = (self.cur+1) % self.max
        print(f"{self.cur=}")
    def get(self):
        """ return list of elements in correct order """
        return self.data[self.cur:]+self.data[:self.cur]
    def get_cur(self):
        return self.cur
    def print_class(self):
        print(f"{self.__class__=}")

def append(self,x):
    """append an element at the end of the buffer"""
    '''
    if self.__class__ == self.__Full:
        print(f"{self.cur=}")
        self.data[self.cur] = x
        self.cur = (self.cur+1) % self.max
        print(f"{self.cur=}")
    else:
    '''
    self.data.append(x)
    print("Old append")
    if len(self.data) == self.max:
        self.cur = 0
        print("Permanently change self's class from non-full to full")
        self.__class__ = self.__Full

def get(self):
    """ Return a list of elements from the oldest to the newest. """
    '''
    if self.__class__ == self.__Full:
        return self.data[self.cur:]+self.data[:self.cur]
    else:
    '''
    return self.data

def get_cur(self):
    return self.cur

def print_class(self):
    print(f"{self.__class__=}")

'''
def change_class_to_full(self,cur):
    """ called when restarting to reset the class State """
    self.cur = cur
    # Permanently change self's class from non-full to full
    self.__class__ = self.__Full 
'''

C_DATA_BUFFER_SIZE = 3

if name == "main":

rb=RingBuffer(C_DATA_BUFFER_SIZE)

for i in range(4):
    initial_buffer_data = rb.get()
    print(f"{initial_buffer_data=}")
    rb.print_class()
    rb.append({"L":i})
    rb.print_class()
    final_buffer_data = rb.get() # updated ring buffer data
    print(f"{final_buffer_data=}\n")

`

Example output: (note differences in last section of each output)

/Thonny's Python3.10

%Run -c $EDITOR_CONTENT

initial_buffer_data=[]
self.class=<class 'main.RingBuffer'>
Old append
self.class=<class 'main.RingBuffer'>
final_buffer_data=[{'L': 0}]

initial_buffer_data=[{'L': 0}]
self.class=<class 'main.RingBuffer'>
Old append
self.class=<class 'main.RingBuffer'>
final_buffer_data=[{'L': 0}, {'L': 1}]

initial_buffer_data=[{'L': 0}, {'L': 1}]
self.class=<class 'main.RingBuffer'>
Old append
Permanently change self's class from non-full to full
self.class=<class 'main.RingBuffer.__Full'>
final_buffer_data=[{'L': 0}, {'L': 1}, {'L': 2}]

initial_buffer_data=[{'L': 0}, {'L': 1}, {'L': 2}]
self.class=<class 'main.RingBuffer.__Full'>
self.cur=0
self.cur=1
self.class=<class 'main.RingBuffer.__Full'>
final_buffer_data=[{'L': 1}, {'L': 2}, {'L': 3}]

###########################################

MicroPython v1.22.2 on 2024-02-22; Raspberry Pi Pico W with RP2040
Type "help()" for more information.

%Run -c $EDITOR_CONTENT

MPY: soft reboot
initial_buffer_data=[]
self.class=<class 'RingBuffer'>
Old append
self.class=<class 'RingBuffer'>
final_buffer_data=[{'L': 0}]

initial_buffer_data=[{'L': 0}]
self.class=<class 'RingBuffer'>
Old append
self.class=<class 'RingBuffer'>
final_buffer_data=[{'L': 0}, {'L': 1}]

initial_buffer_data=[{'L': 0}, {'L': 1}]
self.class=<class 'RingBuffer'>
Old append
Permanently change self's class from non-full to full
self.class=<class '__Full'>
final_buffer_data=[{'L': 0}, {'L': 1}, {'L': 2}]

initial_buffer_data=[{'L': 0}, {'L': 1}, {'L': 2}]
self.class=<class '__Full'>
Old append
self.class=<class '__Full'>
final_buffer_data=[{'L': 0}, {'L': 1}, {'L': 2}, {'L': 3}]

[peterhinch]

This approach is fine on a PC but is not microcontroller-friendly. From a memory allocation point of view it's best to pre-allocate a fixed-size buffer and use modular arithmetic to control read and write buffer indices. Here is an example:

Pass an array or a list to the constructor.

class ringbuf:
    def __init__(self, buf):
        self.buf = buf
        self.size = len(buf)
        self.widx = 0
        self.ridx = 0
    
    def put(self, v):
        w = self.widx
        wn = (w + 1) % self.size
        if wn == self.ridx:
            raise RuntimeError("Buffer is full.")
        self.buf[w] = v
        self.widx = wn

    def __iter__(self):
        return self

    def __next__(self):
        r = self.ridx
        if r == self.widx:
            raise StopIteration
        v = self.buf[r]
        self.ridx = (r + 1) % self.size
        return v

    def __len__(self):
        return (self.widx - self.ridx) % self.size

In this example an exception is thrown if the buffer fills. To allow overwrites, if the write pointer catches up with the read pointer, you increment the read pointer (modulo .size) to discard the oldest entry.

[BillKujawa]

Peter, thank you for taking the time to reply. The data stored in the buffer (in actual code) is five 30 character dictionaries. Using print(gc.mem_free()) before and after entering another buffer item typically indicated free memory: 181824.
My question is why
print("Permanently change self's class from non-full to full") self.__class__ = self.__Full
did not cause Micropython to execute the "Full" class code?

[peterhinch]

I appreciate that I didn't actually answer the question. I don't know whether MicroPython supports dynamically renaming classes; it's quite likely that it does not.

I do know that in over a quarter of a century using Python I've never felt the need to do this, nor have I seen it done before. I would question whether it is good programming practice when there are simpler means of achieving the same aim.

The RAM issue is this. Most firmware runs continuously. Designing a ringbuf on the basis of appending to a list (and subsequently deleting) causes an allocation on every append. While total RAM use remains fairly constant (because deletions match additions) repeated allocations can cause fragmentation. If the code subsequently tries to allocate a buffer for some reason, there may be insufficient contiguous RAM to accommodate it. Using a pre-allocated buffer avoids this hazard.

[BillKujawa]

Peter thank you for the explanation. It seems I am getting a lot of practice in Refactoring my current project. Hopefully that means I am learning a few new things. Bill.

[BillKujawa]

Peter, Is there a good book or other learning method to help alleviate the memory usage issue when dealing with strings on a microprocessor? I believe they are unmutable and are always rewritten to another memory location upon change. Is there a way to reuse the same memory when replacing a string? How do I pre-allocate space for this? Bill On Wednesday 10 April 2024 04:04:17 AM (-06:00), Peter Hinch wrote: The RAM issue is this. Most firmware runs continuously. Designing a ringbuf on the basis of appending to a list (and subsequently deleting) causes an allocation on every append. While total RAM use remains fairly constant (because deletions match additions) repeated allocations can cause fragmentation. If the code subsequently tries to allocate a buffer for some reason, there may be insufficient contiguous RAM to accommodate it. Using a pre-allocated buffer avoids this hazard. — Reply to this email directly, view it on GitHub <#14269 (reply in thread)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHK46UF6VVFEJLWI2KG7M5LY4UFCDAVCNFSM6AAAAABF5PDNKOVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TANRZGI4DC> . You are receiving this because you authored the thread.

…

-- Life is a trip, enjoy the journey. Please, fasten your seat belt! Note: The closest thing to "Normal" is a dryer setting.

[peterhinch]

The best source for general advice on programming microcontrollers is in the official docs. This references ways to manipulate strings. On your specific point, strings and bytes objects are immutable, but bytearray objects are mutable. It is possible to convert between strings and bytes with encode and decode.