Print Function casuing heap memory to overflow

[synapticonashaikh]

Hi all,

Short background: I modified the minimal port to run on my controller (XMOS XCORE). Now, I'm running a basic script where I'm printing an up counter clock. After running the script for a while, the code crashes with an error "Memory access exception" (check below for a complete error message). So, I performed some extra tests and noticed that somehow the print statement is continuously allocating heap memory.

I don't remember, and not sure, but I didn't have this issue a few months ago, but now I'm having it after updating the Python stack to a recent version (a month old or something). Could someone help me resolve this issue? Or a hint on what I'm doing wrong?

I'm also attaching some snippets of the script.

Error:

xrun: Program received signal ET_LOAD_STORE, Memory access exception.
      0x00085a08 in gc_collect_root ( )

Script executor:

#if MICROPY_ENABLE_COMPILER
void do_str(const char *src, mp_parse_input_kind_t input_kind) 
{
    nlr_buf_t nlr;
    if ( nlr_push(&nlr) == 0) 
       {
         mp_lexer_t *lex  = mp_lexer_new_from_str_len(MP_QSTR__lt_stdin_gt_, src, strlen(src), 0);
         qstr source_name = lex->source_name;
         mp_parse_tree_t parse_tree = mp_parse(lex, input_kind);
         mp_obj_t module_fun = mp_compile(&parse_tree, source_name, true);
         mp_call_function_0(module_fun);
         nlr_pop( );
       } 
    else 
       {
         // uncaught exception
         mp_obj_print_exception(&mp_plat_print, (mp_obj_t)nlr.ret_val);
       }
}
#endif

Script and init part:

#define  PARSE_SINGLE_INPUT   0
#define  PARSE_FILE_INPUT     1
#define  PARSE_EVAL_INPUT     2
#define _STATIC_HEAP_SIZE (uint16_t)3000

#if MICROPY_ENABLE_GC
    static char heap[_STATIC_HEAP_SIZE];
#endif  


int main ( )
{
char * data =
"from delay import delaySec\n"
"from delay import delaymSec\n"
"Count = 1\n"
"while True:\n"
"   print('Time MS =' + str(Count))\n"
"   delaymSec(100)\n"
"   Count = Count + 1\n";

 #if MICROPY_ENABLE_GC
        gc_init(heap, heap + sizeof(heap));
#endif

/*init the mp stack*/
 mp_init( );
 
 #if MICROPY_ENABLE_COMPILER  
    do_str(data, PARSE_FILE_INPUT);
#endif  

/*deinit*/
mp_deinit( );
return 0;
}

Script:

from delay import delaySec
from delay import delaymSec
Count = 1
while True:
   print('Time MS =' + str(Count))
   delaymSec(100)
   Count = Count + 1

[cwalther]

This sounds suspiciously like #11781.

[cwalther]

Sorry, I don’t have time to solve your problems for you. I just tried to give you a hint based on something I had seen myself recently, but doing the debugging is up to you.

I suspect what you are seeing is not a memory leak, but the normal behavior of filling up the heap until a failed allocation triggers garbage collection, and then that garbage collection erroneously reclaims an object that is still in use. When that object is later overwritten by a new allocation, any pointers contained in it are corrupted, and then trying to use the pointers causes the memory access exception. That is what I observed in #11781, and the cause there was that the stack was not fully covered by the garbage collection.

What does your gc_collect() function look like? Are you properly collecting the stack and registers?

[synapticonashaikh]

Hey, I know that I have to debug it myself. What I meant was, did you find the solution to your problem? As you have a kind of similar problem. So, the resolution might be close to what I'm after (required patches).

I didn't modify any Python library, so my gc_collect( ) is unchanged (default).

#if MICROPY_ENABLE_GC
void gc_collect(void) {
    // WARNING: This gc_collect implementation doesn't try to get root
    // pointers from CPU registers, and thus may function incorrectly.
    void *dummy;
    gc_collect_start();
    gc_collect_root(&dummy, ((mp_uint_t)stack_top - (mp_uint_t)&dummy) / sizeof(mp_uint_t));
    gc_collect_end();
    gc_dump_info(&mp_plat_print);   //printing the memory remaining is not necessary
}
#endif

[cwalther]

did you find the solution to your problem?

Yes, I proposed some solutions in #11781. None has been definitely chosen yet because the maintainers haven’t gotten around to looking at the issue yet.

my gc_collect( ) is unchanged (default).

Okay, as the comment there says, you may need to implement some way to collect registers. There are some default implementations in gchelper_generic.c, but as far as I understand, you’re on a new architecture, so you may need to write your own.

If the issue is not with registers but with the stack (more likely), then the question is, how are you setting the stack_top that is used in this gc_collect() function? It looks like minimal/main.c sets that in the first two lines of main(), but I don’t see that in your main(). I have no experience with the minimal port – in the embed port mp_stack_ctrl_init() and gc_helper_collect_regs_and_stack() are used, which use MP_STATE_THREAD(stack_top) instead.

[cwalther]

Actually, now that I think of it, seeing your error message 0x00085a08 in gc_collect_root ( ): It could be that you are not setting stack_top at all, so it is initialized to zero, and that is causing the invalid pointer access right there in gc_collect(). Much simpler than my story of prematurely freed objects.

[synapticonashaikh]

Thanks a lot! I noticed my mistake.

    int stack_dummy;
    stack_top = (char *)&stack_dummy;

I added this before the gc_init( ) function, and now it works properly.