My partner is always angry with me, criticises everything I say & do including in front of other people.

I've been with my partner for 10 years, we split for a short time 3 years ago but reconciled. We split for same reasons as above & I ended up moving into my own place & seeing other people. When we reconciled, the same old thing started not soon after & I had a fling & was caught in the act by my partner. I gave him the option that I would leave but he begged me to stay, he told me he could move on from it & we could start afresh but I believe that the reason he continues to be rude, belittling, critical & agro all the time is he just can't let go of what I did. This time I want to leave & not come back, no flings, nothing, just go. I am wondering if I should keep a diary of the daily crap he says to me so I can show him as he says he doesn't speak to me like crap as I claim he does. I'm starting to feel really depressed yet when he is happy & charming, I love him to bits.