Yeah, it might sound strange, maybe even rare, but it does happen. Go
here, and go to the "Notes" section of the page and read point number four and five. Yeah, you might recover the weapons they take by exchanging loot or reverse pickpocket, but I still don't like the fact that they do it on their own. As glitchy as this game is, I'd hate to lose a special weapon for whatever glitchy reason, since I like collecting them.