Thanks for the confirmation @eisbaer
I did notice when using LibreOffice that is it not attaching the file directly from the disk, but instead sends a different file (with the same name) to the email client. So it must be something to do with the way the name is passed to the client, as @Son-of-A-Gun also confirmed that it works fine when adding a file directly from the disk without MS Word involved.
I wonder if this is somehow linked to this issue: