Allow flash programming of partial pages (<256 bytes)

[hattesen]

The current hardware_flash/flash.c implementation validates parameters to flash_range_program(uint32_t flash_offs, const uint8_t *data, size_t count) ensuring that 'flash_offs' is on a page boundary (% 256 == 0) as well as byte countis a multiple of a full page (256), which is also documented in the API documentation:

// pico-sdk/src/rp2_common/hardware_flash/include/hardware/flash.h
/*! \brief  Program flash
 *  \ingroup hardware_flash
 *
 * \param flash_offs Flash address of the first byte to be programmed. Must be aligned to a 256-byte flash page.
 * \param data Pointer to the data to program into flash
 * \param count Number of bytes to program. Must be a multiple of 256 bytes (one page).
 */
void flash_range_program(uint32_t flash_offs, const uint8_t *data, size_t count);

Implementation: /src/rp2_common/hardware_flash/flash.c:

// pico-sdk/src/rp2_common/hardware_flash/flash.c
void __no_inline_not_in_flash_func(flash_range_program)(uint32_t flash_offs, const uint8_t *data, size_t count) {
    ...
    invalid_params_if(FLASH, flash_offs & (FLASH_PAGE_SIZE - 1));
    invalid_params_if(FLASH, count & (FLASH_PAGE_SIZE - 1));
    ...

When implementing EEPROM emulation or a File System backed by the QSPI (NOR) Flash (above the XIP address range), it severely restricts the potential speed and increases SRAM requirements, as every writes to Flash will require a read-modify-write cycle:

1. Full page (256 bytes) read
2. Modify page
3. Full page (256 bytes) program using flash_range_program()

The SPI connected Serial Flash IC used with the RP2040, does not have ANY restrictions regarding starting address or number of bytes written, when using the Page Program (02h) instruction (see Winbond W25Q16JV datasheet, section 9.2.13 for details):

The Page Program instruction allows from one byte to 256 bytes (a page) of data to be programmed at
previously erased (FFh) memory locations

Would it be possible to add a facility (library function) allowing partial page programming at arbitrary addresses, using an API similar to the flash_range_program(uint32_t flash_offs, const uint8_t *data, size_t count) function?
This function may likely restrict all data to be written to be located in ONE 256 byte page, and not allow wrapping (to the beginning of the page, which is the standard behavior of the underlying Serial Flash.

[taraskornuta]

Also want to support this meaning. Such an approach will also reduce the lifetime of the flash itself.

[hattesen]

@taraskornuta

Also want to support this meaning. Such an approach will also reduce the lifetime of the flash itself.

Allowing partial page writes will not, in any way, reduce the lifetime of the flash. It is page erasures that wear out flash, not page writes (which should obviously not be overlapping). For an effective EEPROM simulation in Flash, allowing partial page writes will reduce the number of flash erase operations required, and thus reduce the wear of the flash, increasing the lifetime of it.

[lurch]

I guess adding such an API / function would introduce the possibility of trying to overwrite some already-written data (i.e. not freshly-erased space), and thus introduces the possibility that what you thought you'd written into flash, won't be the same as what you actually read back from flash? 🤔

I know it's probably not the easiest solution, but instead of trying to write partial-pages <256 bytes, how hard would it be to modify your application to buffer write-data in RAM until you have a full page-worth to be written to flash?

[hattesen]

@lurch it wouldn't be at all difficult to buffer 256 bytes of data in RAM before writing to Flash, but it isn't always appropriate to do so:

1. It would make the system vulnerable to data loss in case of reset/power-loss, with data held in RAM and not committed to Flash.
2. The RP2040 (or the Pi Pico) has no (non-volatile) EEPROM to store runtime data or frequently changing configuration values, llike many other MCU's have, including the very popular ATMEL ATMEGA/ATTINY series . Being forced by the SDK to only write whole pages means that you have to use an entire page every time you want to store a 16 bit count in non-volatile memory, resulting in frequent 4K Sector Erase calls, which is SLOW and causes excessive/unneccesary flash wear.
3. The datasheet for the Serial Nor flash used in the Raspberry Pi Pico (W25Q16JV) specifically allows partial pages to be programmed:

   "Page Program instruction allows from one byte to 256 bytes (a page) of data to be programmed"
   What reason would the Pico SDK have for restricting writes to full pages (256 bytes) only?

There are many EEPROM emulation (in Flash) library designs with wear leveling and garbage collection, using partial page writes (and rewrites zeroing flag bits), and I would appreciate if the Pico SDK would not prevent me from implementing such a library.

• EEPROM EMULATION WITH WEAR-LEVELING FOR 8-BIT FLASH MCUS (SILABS)
• EEPROM Emulation Using Internal Flash (ATMEL)
• EEPROM emulation techniques and software (ST)

I fail to see any good reason for this arbitrary restriction in the Pico SDK, but would be happy to gain some insight into the reason – and protecting programmers from their own stupidity isn't a good reason. A partial page program method would likely only allow writing to bytes within ONE page, with no wrapping.

[peterharperuk]

Sounds reasonable to me to add an API to do this. It could be made optional returning an error if the flash didn't support partial page writes (is that even a thing?).

[hattesen]

@peterharperuk i have looked at a LOT of serial Nor flash datasheets, including those used by Adafruit, Pimoroni, seed, sparkfun and waveshare (many of which uses slow generic SPI access at half speed, compared to the "original" Winbond W25QxxJV), and they all seem to support partial page writes. They may differ in "wrap-at-end-of-page" behavior, but it would be reasonable for a partial page programming function to restrict writes to a single page without exceeding the end, typically causing wrap to beginning of (256 byte) page anyway.

The existing flash_range_program function could easily be extended to also support flash_offs outside page boundaries if count is less than 256, raising an error if count bytes will not fit within the page pointed to by flash_offs.

pico-sdk/src/rp2_common/hardware_flash/flash.c

Lines 86 to 106 in 2062372

	void __no_inline_not_in_flash_func(flash_range_program)(uint32_t flash_offs, const uint8_t *data, size_t count) {
	#ifdef PICO_FLASH_SIZE_BYTES
	hard_assert(flash_offs + count <= PICO_FLASH_SIZE_BYTES);
	#endif
	invalid_params_if(FLASH, flash_offs & (FLASH_PAGE_SIZE - 1));
	invalid_params_if(FLASH, count & (FLASH_PAGE_SIZE - 1));
	rom_connect_internal_flash_fn connect_internal_flash = (rom_connect_internal_flash_fn)rom_func_lookup_inline(ROM_FUNC_CONNECT_INTERNAL_FLASH);
	rom_flash_exit_xip_fn flash_exit_xip = (rom_flash_exit_xip_fn)rom_func_lookup_inline(ROM_FUNC_FLASH_EXIT_XIP);
	rom_flash_range_program_fn flash_range_program = (rom_flash_range_program_fn)rom_func_lookup_inline(ROM_FUNC_FLASH_RANGE_PROGRAM);
	rom_flash_flush_cache_fn flash_flush_cache = (rom_flash_flush_cache_fn)rom_func_lookup_inline(ROM_FUNC_FLASH_FLUSH_CACHE);
	assert(connect_internal_flash && flash_exit_xip && flash_range_program && flash_flush_cache);
	flash_init_boot2_copyout();
	__compiler_memory_barrier();
	connect_internal_flash();
	flash_exit_xip();
	flash_range_program(flash_offs, data, count);
	flash_flush_cache(); // Note this is needed to remove CSn IO force as well as cache flushing
	flash_enable_xip_via_boot2();
	}

... but since the implementation in flash.c calls on similarly restricted flash_page_program() methods in /bootrom/program_flash_generic.c, the only way forward is to reimplement a refactored version of the flash_page_program methods from allowing count < 256 bytes at arbitrary (byte aligned) ´flash_offs' addresses will be to refactor flash_page_program methods from /bootrom/program_flash_generic.c and place them in RAM using __no_inline_not_in_flash_func' like their pico-sdk:/src/rp2_common/hardware_flash/flash.c` brethren.

I may give it a try, implementing an updated version supporting partial page programming with a few of code samples making byte-sized updates, or even bit-sized updates (flipping flash bits individually from 1 to zero), resulting in as many as 256*8 calls to flash_range_program(uint32_t flash_offs, const uint8_t *data, size_t count) for each page in a 4KB sector (a grand total of 32,768 calls to flash_range_program on that sector), before erasing the sector and starting over flipping individual 1 bits to zero bits. Doing that 100 times will reveal whether the Flash sector has wear equalling 100 erasures, or if the 3,276,800 flash_range_program has caused additional wear to the sector, rendering it unusable.