Isn't eating together supposed to be together?

My boyfriend and I live together. Tonight he cooked dinner, we sat down to eat, and he finished eating before I did. Without saying anything, he got up and left the table and sat in front of the television. I said he wasn't supposed to leave me eating alone, and that the purpose of eating dinner together was not just for biological reasons, but also to spend time together. He says he was done eating and doesn't see why he should have to sit and watch me eat. Thoughts?