Always getting accused

Well, it all started like this. I told my fiance the previous night that I was taking my daughter shopping for school clothes. It was raining a bit, and he had called my cell phone. I didnt have my cell phone in the store with me, i guess it might have been an hour later I called him back and never could reach him. We finally talked later that night, and he started cussing and accusing me of meeting up with my ex husband, only because I didnt answer the phone right when he called. He never calls me during the day, so I wasnt expecting a call from him, so I didnt think to take my phone in the store with me. He called me names and really hurt my feelings, he kept saying that he cant believe that I done him that way, and I honestly never done anything wrong. I dont like being accused. What do you guys think is the best approach to this situation?