What is more efficient--turning the air-conditioning up to 80F while not home or keeping temp stable at 72F?

My husband and I are arguing on how to set the thermostat. I think it is cheaper to raise the thermostat to 80 during the day when we're not home, then cool it down to 72 when we get home. He thinks it is cheaper to keep a steadier temperature throughout the entire day. We have the same argument in the winter with the heat.