The biggest issue for me is that he was emotionally abusive in our relationship and I'm afraid he will do the same to our kids.
How old are the kids?
Do they have a good relationship with their dad?
There are relationship skills that you can use to support them, and ones you can teach them. It's helpful that there is a diagnosis, so you can be a little more straightforward with them -- to make sure they understand the behaviors belong to their dad, and are not their fault.
Is your ex spending 50 percent of the time with the kids? My ex (also bipolar/BPD) fought for equal time and then gave it away at the drop of a hat.