Hi Me2345,
Welcome to the site, glad you found it. Your niece and nephew, and your brother, are fortunate to have you in their lives, and to care about how they manage having a BPD parent. It's not easy.
It sounds like your brother has had majority time with the kids for two years -- is there a temporary custody order in place, or is this something that sort of happened (maybe SIL did not make efforts to see the kids)? Either way, two years is a solid time to set up the status quo. Judges don't tend to want to see things change for the kids. Or, if they do, there is likely a period of adjustment. For example, the SIL might have to take parenting lessons, or anger management, or substance abuse treatment. Many people with BPD have a hard time regulating their emotions, and find these types of directives difficult, unless they are willing to do the hard work. I'm not BPD, but I found it incredibly difficult to follow court-ordered coparent counseling. It's even harder for pwBPD.
In other words, the burden is on your SIL to make the case that she is capable of handling overnights and getting 50/50. But a lot will depend on whether or not there was a legal agreement in place. She may go to court and claim that your brother prevented her from seeing the kids.
A really helpful book you can get your brother is Splitting by William Eddy, a former social worker turned family law attorney. It's on Amazon, and you can download a digital copy. We also have a pretty good article here about developing an assertive approach to divorce with someone who is a high-conflict personality.
https://bpdfamily.com/message_board/index.php?topic=270440.msg12566140#msg12566140One of the important lessons I've learned over the years here is that not all BPD sufferers are high-conflict personalities (HCPs). If your SIL has a PD, recruits negative advocates, and turns your brother into a target of blame, she may be an HCP. These are the people who can turn divorce into a nightmare, and it's important to take the assertive approach outlined in that PDF in the article above.
Another thing that might help -- people with BPD have a serious mental illness and a difficult time regulating their emotions. They are often overwhelmed by their feelings, and we witness the chaotic and volatile behavior that goes with this condition. But the feelings are real to them -- with BPD, feelings = facts. Sometimes when kids are involved, we feel afraid of what the pwBPD can do, and the stress and anxiety goes through the roof. It really helps to read about the disorder and understand some of the skills that are necessary to de-escalate conflict. It's hard to get rid of the conflict completely, but it is possible to prevent it from getting worse. I found that the more I understood BPD and could learn skills to help me respond, the less of an emotional roller coaster I was on.
Hopefully the GAL assigned to your case is good. They can be excellent -- like being able to tell when a child has been coached, which is what your SIL was doing. Or they can be awful, easily persuaded. What happens next with the GAL? Are you waiting for a report?
Some people here will get full-blown custody evaluations with psych assessment of both parents in order to get everything on the table. There are different approaches and strategies depending on the level of deception involved and laws in different countries/states. For example, in my state parenting coordinators have extension of judicial duties, other states don't have them at all.
Hope this helps a little bit. Let us know what kind of custody arrangement is in place, and how old the kids are, and maybe why SIL is now filing for custody. These are all details that will determined how things might go for your brother and the kids.