Hi smileypants,
I'm sorry your going through this. A pwBPD view the world as a scary place and have difficulties seeing the grey areas, in people and themselves.
They emotionally merge with someone and a defense mechanism that they use to dispel negative feelings about themselves ( feelings of self-worth, shame, guilt ) and
project those feelings on loved ones.You're not a coward and not responsible for someone else's feelings I think you did the right thing with walking away if he's displaying highly dysfunctional behavior.
Could you have reasoned logic in this emotional state?