It's pretty obvious, but I managed to get it wrong ;-)

So if anyone else gets an error like this when mirroring poages with attachments:
Failed to transmit "somefile". Can't map the provider named "SomeProvider" to a configured provider.

It's because when setting up mirroring, you have to give the VirtualPath providers the same name on the source and destination sites otherwise EPiServer does not know what filesystem to choose for a given mirrored file.

/Morten