To be fair, this happens to me also in Debian Gnome, but very very rarely. On Windows, it keeps me from ejecting the drive until after I try it 4-5 times. Every time. In any case, lsof can help identify the culprit.
lsof | grep DRIVENAME should give which program is keeping it.



Might be worth checking https://open-slum.org/