how do I get over the guilt of ending my marriage and seeing my husband still devastated after 3 years?

I left my husband 3 years ago after 9yrs of marriage. I felt alone in my marriage and we were just two parents raising kids, there was no partnership in the relationship. He was a good provider and good dad, but everything else was missing. Before we met he had done some dodgy and stupid things in his life and had been married twice before..I came along and was the ONE you know? Things from his past arose from time to time in our marriage and made me lose respect for him. Even though he was nothing but good to me and a family man, the above issues were always there. So, I am now with someone else, who is brilliant, my total match and everything I've found admirable in a man. But I find myself not showing my full happiness because I feel like a bitch for how my ex is....he is still not over us, I can't stop feeling guilty about leaving and turning his life upside down. We sold our house and divided money and furniture equally plus I gave him both cars. Now all his money is gone, he has incurred debts through stupid decision making and is back to a life with nothing but living for our two boys. I hate seeing him miserable and can't stop feeling responsible. How do I stop this feeling?