I just dont know what to do in this case anymore, please give me some advice.

Ok so lets start from the beginning, I want you all to understand my story. When I was 17 I met a guy that I will call "Joe", he was significantly older than me, but what can I say I fell in love. Dispite my parents obligation(he was older than me by 8 years), I continued my relationship with Joe, for two years. Everything seemed to be perfect only that his mother dispised me! And I really couldnt figure out why. When I would ask Joe he would just shrug and say that it wasnt that she didnt like me, "It was complicated". Ok so anyway, 3 months after our 2 year anniv. I was snooping through Joes phone and found out that he was cheating on me all along with another woman(Well he was cheating on her, I was the other woman.) I was distraught and contacted this woman and told he what was going on, I split with joe and refused contact with his for about 4 months. I now knew why his mother dispised me(she liked this other woman and thought i was the homewrecker...I had no clue). After four months, I continued to be depressed until Joe contacted me. He told me he loved me, and that I was resally the girl he wanted to be with. Of course I believed him, and We moved in together. I tried to put the past behind me and start fresh, Im always around Joe, we are constantly together except for work and I dont really see anything odd except the fact that his phone is off limits. (he believes his phone is private...ok fine) He changed his number to make me happy but recently i snooped again and found a text message from that woman saying that its been months and how he finally decided to give her his number!! I confronted him about it and he seemed more angry about me snooping through his "personal" things. Im trying to be strong about this. And decided that to give it time and see what happens. He changed his number again, and told me he didnt know why text her and that he changed his number because he wanted to be happy. Ive been to his aunts house a couple of times and she is very nice to me and accepts that Im his girlfriend although his mother is a little harder to win over. This will be the first holiday that comes where we live together and I feel that since we are a couple we should spend the holidays together. I have asked him and he shrugs and says I should spend the time with my family. But we are a couple!!! Any views on this?