Why should I get close to my biological when I don't know him that well?

My biological father left me when I was five years old. 17 years later, I found his old address where he lived at, sent him a letter, and he responded back five months later. (He went back to where lived to pick up his mail and found the letter) We finally met after 2 months talking on the phone and the first time I saw him, the man was a shorty! But however, I was happy to meet him. (I think.... at that moment) Well, he came to see me to celebrate his birthday. He gave me money because his words were, "I owe you a lot." Anyways, to be short, ever since then, I felt so uncomfortable and ughhhh.... Why? Because I have a father that took care of me when this guy left. I never felt this much love from him. He blames my mom for him leaving. She may be to blame but half of the blame is his because he left me and didn't bother to stay for me. (My mom is a lunatic and jealous type. She was married three times, different dad) My brother's dad took me in when this guy left. He loved me as I was his own. I was raised Italian, blood of a hispanic. As if right now, I don't want to talk to my bio dad because I believe we maxed out our conversation. This man needs to take it slow and Im not used to someone calling me sweetheart, my love, etc. Sorry, Im not use to that and he talks a lot! It will take me a long time for him to win my heart. To me, he's a total stranger.