Don't Be Ignorant, Speak English!

My husbands family insists on speaking Italian even though I don't understand all of it. My husband has asked them to speak English as they all know it when I'm around. They agree, then a few minutes later are right back to speaking Italian. Alot of times now I refuse to go to his family with him because I feel left out. Am I asking too much that they speak English when I'm around them so I'm part of the conversation also?