I want to go and have a special time with the boys and enjoy some carefree time without the weight of my husbands problems. I am caught up in worrying about how my husband will manage and what I will face when I get back if he dysregulates even more than he is now. Having said that, I was thinking how I might feel if the shoe was on the other foot and I probably wouldn't be thrilled about it.
It sounds to me from reading about your thoughts around going on this trip as if going on it could have a lot to give you.
Time with your babies!
Creating nice memories together with them!
A bit of well deserved reward for you, for hard work!
Time to reflect and stretching your independence!
A possibility of doing something for you!
A source of fun and spontaneity!
A possibility to let go of a lot of worry that is otherwise burdening you and a bit of letting go and be care free!
A moment to detach from co dependency deriving from your husbands many ailments, sleeping problems, noise and such!
A lot of good reasons to go actually! Not to mention the fact that you actually say that you want to go!
All these things are important because they address your needs and your desires, that often get to be shoved away into the backseat or even the trunk of the car, usually when we are in relationships with BPD partners who dysregulate a lot... .
Then comes the worry... . How will it effect things between you and your h, once you come home? And how would it feel for him? And even what if I manage somehow to hurt ex husband?
Isn't that a lot of responsibility right there for stuff that you really can't either foresee nor take upon you to regulate?
What if going gives you strength to deal with and handle things that already are a source of problems with your current husband? What if going away might help you gain new perspective on you husbands gaming and it's effect on you? And what if some time apart even might make you husband shift into a different and perhaps more positive gear when he gets you home again?
Again... . what can be gained from not going?
I think it is very positive the way you reason around your relationship as a whole in the beginning of your post about keeping an open mind for possible progress, but also creating a personal boundary, where you set a bar for what you want your life to be! And also already taking steps towards working on you, which often feels awkward after being washed in a PD partners presence for a long time... . Maybe allowing yourself to going on this trip might help you move in the direction you seem to want to go?
Again, I am in no position to tell you what to do or not, you have to make a choice that feels good to you! I do however want to make sure you take You into consideration too in making up your mind... .

If you haven't gotten around to it yet, there are a lot of information sources around this site on how to improve your relationship with a loved one with BPD that might be helpful. Here are a couple of examples:
Take the First Step Toward Improving Your Relationship [NEW]Supporting a Loved-one with Borderline Personality DisorderAnd again, keep posting and venting! We are here for you! And you are making a lot of progress already in your sound reasoning!
The staff on the staying board can also help you in the future with how to improve communication and how to handle difficult situations when in a relationship with a BPD partner.
You find that board here:
Staying: Improving a Relationship with a Borderline PartnerBest Wishes
Scout99