When we feel guilty about something it is because someone is trying to make us feel guilty. This usually comes from religious upbringing. Are you hurting anyone else by listening to black metal? I would assume not unless you are a total psychopath, which would have nothing to do with the music you listen to. Are religious people hurting anyone else by listening to Keith Green or Michael Card? Definitely. Religious music contains dogma. Dogma tells us what we can and can not do. You are being hurt by being told what you can and can not listen to.
It is a shame that the ones that claim not to judge are always an overflowing cup of judgment. I never had a King Diamond enthusiast or any metal head judge me and I doubt I ever will.
Listen to your music man. Don’t ever let others dictate your life.