Ex Boyfriend Thinks I owe him for money he gave me when together...

My boyfriend at the time, gave me the deposit when I purchased my appartment. Since then we have broken up, on his behalf, now he says I should pay him for the money he gave to me. I dont agree that I should have to and would like to know your thoughts on this....