How can one person feel it's ok for them to have an affair but when the mate has an emotional one it's wrong?

If I can have a phsical affair with someone and my mate forgives me, why is it impossible for me to forgive him when he only has an emotional affair? I feel very betrayed because he does not consider he actually had an affair, he says he was just confinding to an old friend.