I would like to know what I can do when they do not LET you just not respond.
I deal with this often.
First, not responding is the core of ":)on't JADE" (Justify Attack Defend Engage) and it is a valuable tool that can occasionally help prevent or defuse a BPD episode. It's not a magic hammer, it's just a good mindset. But it does have it's drawbacks, particularly when an aggressive pwBPD is focused on you as the problem and is on the attack with buckets of black paint. Then you are forced to Engage - but not forced to Justify, Attack, or Defend. Don't do those, and be very careful with the Engagement.
What I do is make factual & empathetic observations, usually concerning emotional state, and boundary statements:
"I understand that you are very upset and want to fight. I do not want to fight."
I also ask pointed questions about their emotional state: "Are you hurt or are you angry?" "What is this surge of emotion coming from?"
When they hurl the imaginative but awful accusations and demand answers, ignore their content and reply with boundaries, "I'm not participating in this conversation if you want to fight."
Acting lessons: a calm voice pitched with concern (but not patronizing!) is of tantamount importance. They pick up on the subtle signals of body language of how upset you really are and of course interpret them horribly. If you can show them enough empathy you'll be painted white again and patiently endure listening to them rail about other targets (usually your family and friends but sometimes their family and friends and complete strangers). Do not let your guard down, your turn as target will come again, be ready with empathy.
He also does not let me leave.
This is kinda serious. If you are in danger you need an escape plan. Take care of yourself.
I am starting to consider leaving permanently. The bad is way outweighing the good.
-insert frowny face here
Sunshine, this is a core life choice that we all have to make over and over again as the BPD rollercoaster goes up and down. Did you read Ziniztar's heart-breaking "I ended it" thread in this Staying & Improving section? She realized that her own issues were preventing her from being able to Stay & Improve. In your case, like mine, you're seeing escalating-rage episodes focused on you that are very hard to deal with. If you cannot develop some strategies to defuse and/or deal with the BPD outbursts then by default you'll be making them worse. I did that for YEARS before I stumbled across the true nature of this disorder. Now I see that knowing about it and having tools to deal with the behaviors makes it possible to Stay & Improve.
I recommend work on boundaries (especially your right to leave!), and validation to try and defuse these episodes, however it may be that permanently leaving could end up being your escape plan.