Am I wrong for being upset that my son's father is now living with my cousin?

After being in a relationship that I was unhappy in for 8yrs my exboyfriend and I broke up. It was sort of one of those "we grew apart" things. We never really spent time together and we were having money issues. But we were more like roomates that just had sex occasionally. I woke up one day and decided there was no need for two grown people to just be together and be miserable. He has now moved on to have a relationship with not one but two of my cousins. Now we have a 9yrs old son together that he never spends time with, because he says he has money problems. But she brags about how he takes care of her. I know I have no right to tell him who to be with, especially since i ended the relationship. But my family should automatically be off limits. Am I wrong to being upset that he has started a new life with my cousin and left his son behind?