Non fighter/tanky build?
I read strats that said go up the stairs and fight him from the ledge. I found that to be a complete waste of time. You have to be on the top landing or ledge to roll off, otherwise it's a straight narrow path you can neither roll off of or roll past him. And he can still swing at you while you're on the ledge.
I used the stairs to get his demon dogs in range so they could be killed without him swinging on me. Then I took the fight to the ground.
What really made the difference in the fight for me was a Caduceus Shield +3. (From a Hollow Shield +1) I could then block all his attacks but the two-handed swing without losing any life. Stability of 63 or so I think. (Full Elite Knight Armor +1, Lvl 33.)
The other thing that really helped were roll dodges. I do one right at the start of the fight to not die the minute I walk in there. Run forward to his right, and as he starts swinging, roll to a spot behind him and pray he's not doing 2 or 3 swing combo. Also pray his dogs don't block the stairwell.
I found that rolling under his attack was often was more effective than trying to block them, because he's got a fast swing. When I started rolling and taking no damage, and getting in position for a back attack, the fight suddenly seemed a lot less crazy.
After I had that stuff sorted, the fight didn't feel much different the black knights or the stone knight (albeit way faster than the stone knights.)
I can imagine for a light armor fighter/light shield user or someone without a quick, heavy damage weapon, Capra Demon is a nightmare.