I've huge trust issues with my partner of 12 months (due to being cheated on by several ex's).

I'm in a great relationship but my man always seems to have a silly reason to go places without me (usually only for short time)-he picks his kids up from his ex (has them each w/end & one night a week) so has to see her often, which is hard for me -she is attractive and seems very nice) When we met he never stopped talking about her (they'd broken up 2yrs before we met-after 13yrs married), does he still love her/will he leave me to go back to her or is he seeing some other woman/ or will he cheat til I find out and boot him as with several of my past loves? How do I learn to trust? We've discussed this and he says understands, but I still worry. I work shift (away all night 2-3nights a week) so there's plenty of opportunity for him if he wanted....How do I deal with this? Am I being unreasonable, or am I one day eventually going to find out he's played up on me like so many men I've seen/know or been involved with?