Windows fails to read 4GiB file from device

[denisnone]

Windows file explorer hangs when trying to read 4GiB file from armhf device with uMTP-Responder.
Basically all files whose least significant 32 bits of size are equal to zero, 4GiB, 8GiB, etc. If the file is a little bit bigger or smaller than that, everything works well.

Writing such 4GiB files from Windows to MTP device has no problems. Reading it back is impossible.

Tried on Windows 7 and Windows 11. Device with linux kernel 6.12.19 armv7l.
uMTP-Responder with DEBUG=1 or without, the same result. Tried master branch (93cca39) and tag umtprd-1.6.8.

To reproduce create a file of that size on the device and try to read it:
dd if=/dev/urandom of=file4.dat bs=1M count=4096 status=progress

Debug info from uMTP-Responder shows the read process, but nothing happens on Windows side, the progress bar in file explorer is not moving and the underlying folder window is hanging. Cancel button will not be able to stop the process and everything hangs till USB removal or uMTP-Responder process got killed.

Some debug info when starting the file transfer:

[uMTPrd - 13:22:07 - Debug] --------------------------------------------------
[uMTPrd - 13:22:07 - Debug] Incoming_packet : 0x151dcc0 - rawsize : 16
[uMTPrd - 13:22:07 - Debug] MTP Packet size : 16 bytes
[uMTPrd - 13:22:07 - Debug] MTP Operation   : 0x0001 (OPERATION)
[uMTPrd - 13:22:07 - Debug] MTP code        : 0x1009 (MTP_OPERATION_GET_OBJECT)
[uMTPrd - 13:22:07 - Debug] MTP Tx ID       : 0x00000049
[uMTPrd - 13:22:07 - Debug] Header : 
[uMTPrd - 13:22:07 - Debug] 10 00 00 00 01 00 09 10 49 00 00 00              ........I...     
[uMTPrd - 13:22:07 - Debug] Payload : 
[uMTPrd - 13:22:07 - Debug] 02 00 00 00                                      ....             
[uMTPrd - 13:22:07 - Debug] mtp_get_storage_root : FFFF0001 -> /mnt/mmc1p1/RData
[uMTPrd - 13:22:07 - Debug] mtp_get_storage_flags : FFFF0001 -> 0x00000000
[uMTPrd - 13:22:07 - Debug] send_file_data : Offset 0x0 - Maxsize 0x100000000 - Size 0x100000000 - ActualSize 0x100000000
[uMTPrd - 13:22:07 - Debug] mtp_get_storage_root : FFFF0001 -> /mnt/mmc1p1/RData
[uMTPrd - 13:22:07 - Debug] build_full_path : file4.dat -> /mnt/mmc1p1/RData/file4.dat
[uMTPrd - 13:22:07 - Debug] mtp_get_storage_index_by_id : FFFF0001 -> 0
[uMTPrd - 13:22:07 - Debug] ---> 0x1ff4 (0x2000)
[uMTPrd - 13:22:07 - Debug] ---> 0x3ff4 (0x2000)
[uMTPrd - 13:22:07 - Debug] ---> 0x5ff4 (0x2000)
[uMTPrd - 13:22:07 - Debug] ---> 0x7ff4 (0x2000)
[uMTPrd - 13:22:07 - Debug] ---> 0x9ff4 (0x2000)
[uMTPrd - 13:22:07 - Debug] ---> 0xbff4 (0x2000)
[uMTPrd - 13:22:07 - Debug] ---> 0xdff4 (0x2000)
[uMTPrd - 13:22:07 - Debug] ---> 0xfff4 (0x2000)
[uMTPrd - 13:22:07 - Debug] ---> 0x11ff4 (0x2000)
[uMTPrd - 13:22:07 - Debug] ---> 0x13ff4 (0x2000)
[uMTPrd - 13:22:07 - Debug] ---> 0x15ff4 (0x2000)
[uMTPrd - 13:22:07 - Debug] ---> 0x17ff4 (0x2000)
[uMTPrd - 13:22:07 - Debug] ---> 0x19ff4 (0x2000)
[uMTPrd - 13:22:07 - Debug] ---> 0x1bff4 (0x2000)
[uMTPrd - 13:22:07 - Debug] ---> 0x1dff4 (0x2000)
[uMTPrd - 13:22:07 - Debug] ---> 0x1fff4 (0x2000)
[uMTPrd - 13:22:07 - Debug] ---> 0x21ff4 (0x2000)
[uMTPrd - 13:22:07 - Debug] ---> 0x23ff4 (0x2000)

After the file ends the read process overlaps and starts from the beginning of the file:

[uMTPrd - 15:08:26 - Debug] ---> 0xfffebff4 (0x2000)
[uMTPrd - 15:08:26 - Debug] ---> 0xfffedff4 (0x2000)
[uMTPrd - 15:08:26 - Debug] ---> 0xfffefff4 (0x2000)
[uMTPrd - 15:08:26 - Debug] ---> 0xffff1ff4 (0x2000)
[uMTPrd - 15:08:26 - Debug] ---> 0xffff3ff4 (0x2000)
[uMTPrd - 15:08:26 - Debug] ---> 0xffff5ff4 (0x2000)
[uMTPrd - 15:08:26 - Debug] ---> 0xffff7ff4 (0x2000)
[uMTPrd - 15:08:26 - Debug] ---> 0xffff9ff4 (0x2000)
[uMTPrd - 15:08:26 - Debug] ---> 0xffffbff4 (0x2000)
[uMTPrd - 15:08:26 - Debug] ---> 0xffffdff4 (0x2000)
[uMTPrd - 15:08:26 - Debug] ---> 0xfffffff4 (0x2000)
[uMTPrd - 15:08:26 - Debug] ---> 0x100000000 (0xC)
[uMTPrd - 15:08:26 - Debug] send_file_data : Full transfer done !
[uMTPrd - 15:08:26 - Debug] Status response (12 Bytes):
[uMTPrd - 15:08:26 - Debug] 0C 00 00 00 03 00 01 20 49 00 00 00              ........I...     
[uMTPrd - 15:08:26 - Debug] --------------------------------------------------
[uMTPrd - 15:08:26 - Debug] Incoming_packet : 0x212dcc0 - rawsize : 16
[uMTPrd - 15:08:26 - Debug] MTP Packet size : 16 bytes
[uMTPrd - 15:08:26 - Debug] MTP Operation   : 0x0001 (OPERATION)
[uMTPrd - 15:08:26 - Debug] MTP code        : 0x1009 (MTP_OPERATION_GET_OBJECT)
[uMTPrd - 15:08:26 - Debug] MTP Tx ID       : 0x0000004A
[uMTPrd - 15:08:26 - Debug] Header : 
[uMTPrd - 15:08:26 - Debug] 10 00 00 00 01 00 09 10 4A 00 00 00              ........J...     
[uMTPrd - 15:08:26 - Debug] Payload : 
[uMTPrd - 15:08:26 - Debug] 02 00 00 00                                      ....             
[uMTPrd - 15:08:26 - Debug] mtp_get_storage_root : FFFF0001 -> /mnt/mmc1p1/RData
[uMTPrd - 15:08:26 - Debug] mtp_get_storage_flags : FFFF0001 -> 0x00000000
[uMTPrd - 15:08:26 - Debug] send_file_data : Offset 0x0 - Maxsize 0x100000000 - Size 0x100000000 - ActualSize 0x100000000
[uMTPrd - 15:08:26 - Debug] mtp_get_storage_root : FFFF0001 -> /mnt/mmc1p1/RData
[uMTPrd - 15:08:26 - Debug] build_full_path : file4.dat -> /mnt/mmc1p1/RData/file4.dat
[uMTPrd - 15:08:26 - Debug] mtp_get_storage_index_by_id : FFFF0001 -> 0
[uMTPrd - 15:08:26 - Debug] ---> 0x1ff4 (0x2000)
[uMTPrd - 15:08:26 - Debug] ---> 0x3ff4 (0x2000)
[uMTPrd - 15:08:26 - Debug] ---> 0x5ff4 (0x2000)
[uMTPrd - 15:08:26 - Debug] ---> 0x7ff4 (0x2000)
[uMTPrd - 15:08:26 - Debug] ---> 0x9ff4 (0x2000)
[uMTPrd - 15:08:26 - Debug] ---> 0xbff4 (0x2000)
[uMTPrd - 15:08:26 - Debug] ---> 0xdff4 (0x2000)
[uMTPrd - 15:08:26 - Debug] ---> 0xfff4 (0x2000)
[uMTPrd - 15:08:26 - Debug] ---> 0x11ff4 (0x2000)
[uMTPrd - 15:08:26 - Debug] ---> 0x13ff4 (0x2000)
[uMTPrd - 15:08:26 - Debug] ---> 0x15ff4 (0x2000)
[uMTPrd - 15:08:26 - Debug] ---> 0x17ff4 (0x2000)
[uMTPrd - 15:08:26 - Debug] ---> 0x19ff4 (0x2000)
[uMTPrd - 15:08:26 - Debug] ---> 0x1bff4 (0x2000)
[uMTPrd - 15:08:26 - Debug] ---> 0x1dff4 (0x2000)

[jfdelnero]

Thanks for the report.
I reproduced the issue and fixed it. The fix is on the repository.

I will check the other way too (Windows -> Device)

[denisnone]

Great, thanks, now the 4GiB file gets copied well.
Although it's not possible to cancel the transfer while copying such file, the Windows Explorer hangs in the same way.
Is it a separate bug or somehow related?
I think I saw it earlier from time to time on different file sizes, but rarely. Now it's constantly reproducible on 4GiB file.

[jfdelnero]

This is a separate issue : i have just fixed some cancellation logic issues.
But : This doesn't fix the current 4GB transfer cancellation issue. And i managed to reproduce the exact same issue with the Android MTP responder on an Samsung A32 5G smartphone (Android 13). So this appears to be an bug in the Window MTP initiator.
BTW all is working fine with the Linux Initiator.

[jfdelnero]

Complement : Windows just don't send the MTP_REQ_CANCEL request if the file is >= to 4GiB. Reproductible with uMTP Responder and Android responders (v13 and probably others versions)