So fed up, should I fight to work it out or should I just leave?

So, my boyfriend and I have our three month old baby and we live together. Im 20 he is 22. We live in his house and we dont really get along . I dont trust him much cus he has given me reason not to in the past plus the fact that he never thinks any of it was wrong does not help with me not getting over those things. So in short words, I feel very depressed with him. We were doing great until the other day when he called me many names just cause I asked him a question. He called me a F* dumb b* (female dog) and much more. He smokes weed which I dont agree with. I told him from day one there will be no weed smoked in the house & he agreed. Now he is changed his thoughts and becus he left weed around I was asking him why & he came at me with all the names. to top it off I slammed the door pist & he did it again & then told me to get the f out of here. when i consulted him about it the next day he told me he did nothing wrong. He could do what the f he wants and if i don't like it then i can get the f out. so he has kicked me out. But he knows I have no where to go. He doesnt care that i tell him he disrespects me cus he says its not disrespect. Last but not least, I found out he searches for certain porn girls on the web. A guy that doesnt take out the time to spend with me has time for that. I dont think I can try anymore. Ive offered to take couples counceling and he doesnt want to at all. I cant handle it anymore. I have a new job & Im getitng on my feet. But i need to make a final decision. Should I keep trying? Or should I let it go. I do care for him, but he killed th elove I had, I feel so much hate and dislike. I wake up crying in the morning bcus i sleep in his bed in his home I feel like im not worth any thing cus i depend on the person who makes me so miserable. & he sees me cry and its nothing to him. He just gets more mad. I hate confronting him cus he makes me feel worst. He says i exagerate. I need to do something my daughter cant grow up like this.