Hi. Hope someone can help me see clearly. Am I right not to contact a man who has let me down?

Briefly..I met a man 3 years ago at work. We are both married. Instant attraction, lots of chemistry etc. Shared some confidences etc. Kissed and shared our feelings once or twice but no more. He has now left my workplace and our friendship has dwindled. A few months ago, he asked me out to lunch. I accepted but he cancelled last minute (by email) with work excuses. I rearranged. He cancelled again with different excuse (again by email), wanting to reschedule. I told him I wasn't happy being treated like this. He went mad and told me he had other commitments and sarcastically apologised for having a life, family etc. No word from him since. I am heartbroken and miss him SO much. Was I right to express my annoyance - was he right to get cross back at me?? I want to see him more than anything in the world - should I contact him? And what on earth is going on in his head?? He seemed to have such strong feelings for me - yes I know we are both married...but....
I'd really appreciate some opinions.
Thanks heaps