Flashing a new main.py script via serial does not invalidate the data logging storage

[microbit-carlos]

If I remember correctly, the way CODAL determines if a new programme has been flashed to the micro:bit is something like this:

• It checks a location in flash
• If data is 0xFF:
  • it determines a new programme was flashed
  • Invalidates the DAPLink storage (a "fast erase")
  • Writes a flag in this location

This works well for the MakeCode and Python online editors because the users either download a hex file that does a full flash erase (clearing the flag) and then flashes the hex content, or via WebUSB which also clears the flag if it does a full flash or partial flashing (as it compares the flash contents with the hex contents, and any flash page not included in the hex is erased).

However, for programmes like Mu, the user code is flashed via the REPL by sending a main.py file via serial and writing it via the filesystem API.

Could we figure out a way in which flashing a user programme via serial could invalidate the DAPLink storage as well?

@finneyj @JohnVidler This could end up needing an update to CODAL, but opening the issue here first as this is a use-case specific for MicroPython.

[finneyj]

I think the implementation is pretty much there already @microbit-carlos. We would just need to replicate this call I think (and ideally associated #ifdef guard):
https://github.com/lancaster-university/codal-microbit-v2/blob/master/model/MicroBit.cpp#L394

We can also factor that out into a function CODAL side to avoid duplication if you prefer though.

Triggering the operation might be a bit trickier though. Is there any code on the micropython side that is only executed when such new program is injected by an editor like mu? Maybe something in the filesystem?

[dpgeorge]

Triggering the operation might be a bit trickier though. Is there any code on the micropython side that is only executed when such new program is injected by an editor like mu?

There isn't anything currently, and I'm not sure how to make that work.

Maybe mu can run a command on the board after writing a new main.py, eg log.clear()?

[microbit-carlos]

Maybe mu can run a command on the board after writing a new main.py, eg log.clear()?

This would be the simplest solution, but it'd only be applied to any editor we remember to alert of this change, or that carefully reads the release notes.

Ideally I would prefer if this was automatically managed by MicroPython/CODAL, as it ensures consistent behaviour across any micro:bit editor. It doesn't feel quite right that an editor would have to become responsible for things like erasing the log.

Options that we have:

• Somehow hook into the Python file system functions or microbit-fs low level filesystem to run additional actions when writing to a main.py file.
  • Because a user programme could be writing files to file system we should only do this with main.py and not other files
  • Disadvantage: We'd need to add hooks to source code from upstream MicroPython
• Somehow hook into the REPL or raw-REPL to perform the log erase when a file is written to flash
  • In this case we could consider any python file write would constitute a "flash action" and erase the log not only on main.py
  • Disadvantage: We'd need to add hooks to source code from upstream MicroPython
• On startup, check the contents of main.py and store the hash somewhere no-volatile, if the checksum is different erase the log
  • Disadvantage: If a program was self modifying, it would erase the log as a side effect
• On startup check some kind of metadata comment to main.py, if it's there already no need to do anything, if it isn't add it and erase log
  • Disadvantage: Comment would be present when a user extracts the main.py file
• Request to the editor to run something like log.clear() after serial flashing.
  • Disadvantage: Inconsistent behaviour for any editor not implementing is requirement.
• Do we have any other options?

[dpgeorge]

Somehow hook into the Python file system functions or microbit-fs low level filesystem to run additional actions when writing to a main.py file.

I think this is the most robust approach that would give the desired behaviour.

No need to modify upstream MicroPython, can just make a local copy of the microbit-filesystem code and modify it.

[dpgeorge]

Implemented in 6ade36b