Married people, which do you think is worse?

And just to clarify, this is purely to see different opinions. I'm not in this situation nor planning on being in this situation. Everytime I post a question like this I have at least two people assuming it's about me and giving relationship advice.

Which is worse? A spouse who is cold, unloving, dishonest, withholds sex, lets themself go, belittles their husband or wife...

-OR-

A spouse that has an affair?

Aren't both situations breaking of vows?