Should I leave my angry partner?

My partner of 4 years is the most negative, angry person I've ever met. He has a history of violence towards other men, but has toned this down a lot since meeting me. He's occasionally intimidating and verbally hostile to people in the world he gets into confrontations with, but no violence.  I feel like I'm walking on egg-shells living with him. The slightest trivial thing will cause him to go into a rage. Our house is full of holes in walls where he's punched them, and he's smashed up our belongings on occasion. We have lots of pets (incidentally that I do most of the looking after) and he often takes his irritations out on them. I feel like saying 'why do you have all these animals if they annoy you so much' but it would just cause an argument! If a cat is in his way or a dog won't do as he's told he'll hit them. I challenge him about his behaviour - it really upsets me, but he either ignores me or justifies it by saying he's irritated. He's never been violent towards me, but one time he pushed me over and another time when I stood nose to nose with him because he was trying to intimidate me (he's a big guy!) he put his hand on my shoulder and pushed me aggressively.  Whenever I talk to him about his anger he'll say it's because he's in pain or tired or some other excuse. He is disabled and struggles to walk because of extreme pain in a leg. So I try to take on more so things aren't so hard for him. But it's getting to the point where I'm running myself ragged to make his life easier, while my life becomes very stressful.  Now we have a baby. He's good with her but I do most of the childcare; partly because I want to and partly because he's in pain and stuff. But it's hard work and now I'm looking after her, the animals, doing most of the housework and tomorrow I go back to work part-time. I'm exhausted! And yet I still manage to stay upbeat and give him and our baby all my love and attention. I have a sling and carry her with me while I get everything done. But it's getting to the point where me and the baby are happy playing in a morning and then he'll get up and all hell breaks loose! Things are being thrown, he's shouting and swearing. I've started getting out of the way. I used to just roll my eyes and ignore him until he'd calmed down, and hoped when the baby came along he'd make an effort to control his anger more, but things are no different. But I do have a much lower tolerance for his behaviour and I don't want my daughter to think this is the right way to express anger.  When I confront him about his anger he usually finds some way to blame me or change the subject so somehow I'm in the wrong. He leaves me feeling manipulated and takes the wind from my sails.  Yesterday is a perfect example of an argument: he smokes heavily, and smokes around the baby. He knows I don't like it but I gave up on that argument ages ago. He doesn't believe in passive smoking. But we have an air purifier and he at least sits next to that on the other side of the room. But yesterday I was sat playing with the baby next to him and he lit up. Smoke went into her face. I picked her up quickly and walked away and asked him that in future could he tell me before he lights up so I can move the baby, or just not do it while she's right next to him. He told me not to criticise  him. I said I didn't, simply made a request, but he insisted I was criticising him. Then we had an argument about whether or not I'd criticised him. Eventually I just gave up and shut up.  I don't know what to do anymore. I don't want to split up my daughter's family and I don't want to leave him struggling to manage without me. I really don't know how he'll cope! I do almost everything around the house, walk the dogs, bring most of the money in, etc. But I don't want my daughter being brought up around such negativity and anger. Things are so calm and quiet when it's just me and her. I've started fantasising about how nice and easy our lives would be if it were just the two of us. And I worry about his behaviour towards her as a toddler, child, etc. He's great with her now, while she's a baby, but what about when she's two and she's being challenging. Will he have patience or shout at her like he does me? I don't know. I love him and really care for him. Otherwise we get on great and have loads in common. We like each others' company when things are ok. But his angry outbursts never change, despite all his promises that when we move he'll be less stressed, or when we pay off that debt he'll be less stressed. There's always another thing to replace the stressful thing. Should I just leave or keep trying to help him or talk things through?