Please give me an opinion on this situation.

My ex-husband and I were thinking about getting back together. We have been apart for about a year now. During that year I have been living near my small grandkids. I told my ex-husband that my grandkids are very important to me. His reply was "they're your daughter's kids'. I was offended to this answer. Should I go back to someone who thinks like that. Shouldn't we be able to come to some common ground on this. By the way, he doesn't have kids of his own.