Right now your frustration with your dilemma is overwhelming. We understand. The chaos, unpredictability and inconsistency of a person with BPD or some other acting-out disorder is so discouraging to say the least. However ponder this point. The problem is not your son, it his mother. So expressing that you feel like walking away from your son, I'm sure, is not really what you meant. You'd like to walk away from his mother and the situation she's caused and enabled. Right?
See? It's all in your perspective.

Your son, I'm sure, is wonderful, he didn't cause or ask for this situation. As a little child, he's essentially powerless to change his circumstances. That's where you come in as His Dad. And that's where we come in as Peer Support. We've been there, done that. With our collective wisdom you can find understanding, resources, topics on improving your skills, ideas, suggestions and strategies that work.
Your son will do best with you as His Dad and a significant part of his life.

All of us here have faced difficult persons in our lives. We either overcame the problems or found ways to deal with and improve our circumstances.
I have a couple questions. Previously you said you have been caring for him while his mother was in drug abuse treatment. Why was he with relatives? And why did they sand him to her and not you?
His mother is not stable enough to care for him.
This is where we can help. As the more stable parent you need to have more responsibility in caring for your child. She probably won't let you have more than mere crumbs, you will probably need the court to order/assign you substantial parental responsibility. You probably need to become the Primary Parent with, if not sole legal custody, then at least
Decision Making or
Tie Breaker status.