Hi Mike7284;
Coparenting makes detaching difficult, for sure. It's commendable that your kids are your priority and that you recognize there is some level of need to communicate with their mom.
In terms of your 17YO, like Pook075 mentioned, it sounds like he's at an age where it's less about the parents coordinating his relationship with them, and more about him starting to take responsibility for his individual relationship with each parent -- if that makes sense. You've loved him and supported him for years, and I'm sure he feels that.
our oldest child is not mine biologically, but I’ve been his dad since he was 2 (he is 17 now). But I have no legal rights to him. And she will use that against me, which is one of the reasons I give in so much to her.
What do you think she would threaten to do?
In terms of this issue:
My ex states she wants to be friends and wants to be able to coparent (which I am not opposed to, we have two kids). But she keeps crossing boundaries. I told her that when we talk, it needs to be strictly about the kids. But she will still call and text about random stuff. It’s been making my current girlfriend upset.
pwBPD often don't "respect boundaries" in the way that we would expect a "generally normal" person to "respect boundaries." Sometimes, our picture of "respecting boundaries" means we verbally tell another person "Don't do XYZ", and then we expect that person to... not do XYZ. It doesn't work that way with a pwBPD (as you and I and all of us here are aware

)
A key nuance about boundaries that's really important when interacting with pwBPD is:
-true boundaries aren't ultimatums or requests that require the other person to cooperate or agree.
-true boundaries are 100% under our control and don't require anyone else to get on board.Maybe there's a way to get some real boundaries at play in this "she texts about random stuff" situation.
In the past, was your "boundary" to communicate to your ex "only talk to me about the kids, nothing else"?
See how that would depend all on her, not on you? And of course, she doesn't respect that.
What if your new boundary was something you decided for yourself (maybe in conjunction with your GF), and that you didn't even have to explain to your ex -- it was something you could "just do" and didn't depend on your ex's agreement:
"I don't respond to texts that aren't about the kids. I ignore all communication from my ex that is not kid focused. I have a special email folder where I direct all emails from my ex, and I only reply to ones about our children, the rest get trashed. If we're on the phone, the moment she starts talking about anything besides the kids, I hang up."
Something like that? I wonder if you and your GF could get on the same page that your kids' mom is a boundary buster, it's really frustrating, and you are now committed to not engaging with her about anything besides your children. Do you think your GF might see where you're coming from -- that you can't control your ex, that you and GF agree together that your ex is a frustrating person, and that GF can support you in your new (real) boundary of only engaging about the kids?
...
Also, how old are the rest of your kids now?