Hi there,
Hmmm it sounds like your sil and especially your brother are not meeting you half way. It's tough when it's family to really recognize when they aren't investing at the rate you are and when they aren't ever likely to. Even when you do see it, it's hard to accept, stop investing, and stop expecting them to act like it seems like family should. It doesn't sound like this unreciprocal relationship between your families is good enough for you. I know it would certainly not be good enough for me and I've dealt with my fair share of these weird hot/cold (mostly cold) family dynamics.
Being raised by a self absorbed parent makes this even more challenging. You grew up with a relationship model that involved loads of investment on your part and little from your parent - especially if you were the scapegoated child! This can lower your standards so that you find yourself tolerating the intolerable and trying to make it work.
It sounds as if you may be coming to terms with the low level of involvement your brother and sil want to have and need some support about the awkward situation you're finding yourself in with this party.
I wouldn't go because I'm introverted and have no energy to spare

The buzz of the party wouldn't make up for the energy lost being around the sil and brother.
Do you like parties? Are you dying for a night out? Do you enjoy spending time with these mutual friends? Would the benefits of going counter the energy lost seeing your brother and sil there?
If so, then go! If it were me, I'd put space between myself and them and focus on having a good time. If it gets uncomfortable, head home early and have a nice bath with your sparkling beverage of choice. Make it your night whatever that takes.
The key here is to ask yourself now and in every moment what is the kindest thing you can do for YOU. That can guide you in the right direction no matter what comes up.
Take care