I'm sure my father knows, but he really couldn't care less what somebody does in privacy... I doubt my mother even realizes hentai exists (she's still of the mind that anything animated = loony tunes or spongebob) My sister could probably assume... she knows I watch a lot of anime and she's on the internet enough that I'm sure she knows what hentai is... though it's never been brought up...
As for my friends? We're all video game geeks... we're more likely to talk about Halo than anything related to anime or hentai.
Family: No and I rather keep it that way. Hard enough to explain anime.
Work friends: a couple.
Close friends: They are the only ones who know how perverted I am and trade links regularly.
That is the perfect picture for this poll, haha! For anyone who doesn't know, it's Oreimo - the girl in the middle starts the series as a closet otaku.
Yeah, my friends and family know. My friends don't care, (nobody does really). My family may just take it as a joke, but in that, even I find it humorous.