fix(tusb_msc): Cache sector metrics, enable storage buffering

[igi540]

🚀 Optimize USB MSC Performance on ESP32-S3, ESP32-P4, and ESP32-S2

---

✅ Requirements

Why is this needed?

• ESP32-S3 USB Mass Storage (MSC) implementation showed suboptimal performance (slow file copy speed).
• Observed issues:
  • ⚠️ SPI Flash write speed was severely limited due to architectural constraints.
  • 📉 SD card speeds varied significantly based on FIFO size, but were still below expectations.
• Goal: Optimize data transfer efficiency for both SPI Flash and SD cards.

---

📝 Description

This PR improves USB Mass Storage (MSC) performance by:

• ✅ Caching get_sector_size and get_sector_count to reduce redundant queries and improve efficiency.
• ✅ Optimizing buffering logic:
  • A single-buffer approach is used for all targets (ESP32-S3, ESP32-P4, ESP32-S2)
  • Buffering significantly improves SD card write speed, while SPI Flash shows minimal improvement due to hardware constraints.

---

📊 Observed Results

ESP32-S3

SD Card Performance

FIFO Size	Read Speed (Old)	Read Speed (New)	Write Speed (Old)	Write Speed (New)
512B	0.561 MB/s	0.566 MB/s	0.191 MB/s	0.236 MB/s
8192B	0.925 MB/s	0.925 MB/s	0.726 MB/s	0.928 MB/s

✅ Write speed improved significantly with 8192B FIFO (~27% increase).
✅ Read speed remains stable at ~0.9 MB/s.

SPI Flash Performance

FIFO Size	Write Speed (Old)	Write Speed (New)
512B	8.21 KB/s	8.46 KB/s
8192B	30.31 KB/s	30.2 KB/s

⚠ Minimal improvement for SPI Flash (limited by write speed).

---

ESP32-P4

SD Card Performance

FIFO Size	Read Speed (Old)	Read Speed (New)	Write Speed (Old)	Write Speed (New)
512B	1.175 MB/s	1.174 MB/s	0.234 MB/s	0.238 MB/s
8192B	4.786 MB/s	4.744 MB/s	2.021 MB/s	2.157 MB/s
32768B	5.833 MB/s	5.998 MB/s	3.673 MB/s	4.485 MB/s

✅ Larger FIFO sizes (32KB) significantly improved SD card write speed (~22% increase).

SPI Flash Performance

FIFO Size	Write Speed (Old)	Write Speed (New)
512B	7.44 KB/s	7.57 KB/s
8192B	28.44 KB/s	28.42 KB/s
32768B	28.12 KB/s	28.24 KB/s

🔸 No major changes in SPI Flash performance on ESP32-P4.

---

ESP32-S2

SPI Flash Performance (Corrected Values)

FIFO Size	Write Speed (Old)	Write Speed (New)
512B	5.50 KB/s	5.59 KB/s
8192B	21.52 KB/s	21.54 KB/s

🔸 Minimal improvements observed, similar to ESP32-S3 results.
⚠ SPI Flash performance remains limited due to architectural constraints.

🚫 Note: SD card is currently not supported in MSC device mode for ESP32-S2.

---

🧪 Measurement Method

• SD card performance was measured using SpeedOut on Windows (applies to all supported SoCs).
• SPI Flash performance was measured using a custom Windows Bash script (applies to all supported SoCs).

---

🚩 Why SPI Flash Performance is Limited?

The core limitation is that program execution and storage reside on the same flash chip:

• 🧠 During a flash write operation, program execution from flash is suspended.
• ⏱ Interrupts not loaded into IRAM cannot run, severely slowing down USB transfers.
• ⚠️ This explains why SPI Flash remains slow, even with buffering optimizations.

SD cards, on the other hand, are not affected by this limitation, which is why they show significant improvements with optimized buffering.

---

🔗 Related

• Closes GitHub Issue #11110

---

📌 Conclusion

✅ SD card performance now approaches USB Full-Speed theoretical limit (~1 MB/s on ESP32-S3).
✅ Write speed improvements are significant for SD cards, especially with larger FIFO buffers (up to ~4.5 MB/s on ESP32-P4).
⚠ SPI Flash remains speed-limited due to architectural constraints, with only minor improvements observed.

🚀 This PR significantly enhances SD card performance and optimizes MSC efficiency across ESP32 targets!

[roma-jam]

Thanks a lot for the PR! Description is perfect! Nice idea to use the resource of the same task, which runs tud_task().

I pointed some general things that affects the logic. Based on the that, may I ask you to try and remove one buffer? The logic will be much simpler: get write10 callback, memcpy data, usbd_defer_func write to storage.

As far as we don't use both buffers at the same time, this change could help us:

• simplify the logic
• decrease memory usage (twice)
• do not loose in speed comparing to your current results

[tore-espressif]

@igi540 Overall LGTM.

Only 1 major question about adding the storage_buffer into tinyusb_msc_storage_handle struct. This way, we would have 1 handle for 1 storage instance

[roma-jam]

@igi540 and please check the CI fail: APP: Write data after disconnect

[tore-espressif]

@igi540 and please check the CI fail: APP: Write data after disconnect

This is really weird because it fails on v5.0 v5.1 and v5.2.
It passes on v5.3, v5.4 and latest...

EDIT: Might be related to espressif/esp-idf#9691

[roma-jam]

@igi540 Not it is not related, as in this error the espressif dcd layer is used: dcd_esp32sx.c:356

We are not using it since v0.15.

[tore-espressif]

@roma-jam sorry, I didn't provide any details. Very similar code with the same race condition and by-zero-division is also in the new DCD layer. However, I can't confirm it is the same problem now. Will look into it tomorrow

[tore-espressif]

@igi540 The problem with race condition is fixed espressif/tinyusb#55

Please update this PR, so we can merge it after the fix is released

[roma-jam]

The one thing that unleash the race condition - is immediate return from the tud_msc_write10_cb().

Maybe we could think of adding the test_app with emulated ram disk to:

1. measure the raw speed of mscd driver speed
2. test sudden disconnection in this case (as in this case reset comes sooner than we prepare_cbw(), so the reset flush the xfer->max_size)

Otherwise, LGTM.

Please, don't forget to address all the rest notes before merge.