What do you do when the boundary creates more work?
I'm not talking about extinction bursts where someone is testing the boundary. I mean more like telling someone you have a sore leg. Now they know where to kick you.

The suggestions in the article are things I aspire to but if I'm to be honest, most of my boundaries are not stated directly because the fall-out creates a mess. It's like there is a game, and the game is for her to be center of attention. As long as she's talking and people are listening, then she's winning the game. If she feels like she's being ignored, she has to find ways to win again. Nothing is off limits. As long as it gets attention, it's fair game.
Boundary No. 5: No gossiping about our family members.
Parents will sometimes talk poorly about or reveal private information about a relative or another one of their adult kids — even when that information isn’t theirs to disclose.
This is common “especially if the sibling didn’t live up to the parents’ expectations or if they are treated as the family scapegoat,” Hart said.
“If you were raised in a family where respecting your parents meant your feelings and thoughts were dismissed, where silence was expected when the parent was making a mistake or causing pain to others, or you were punished by them removing love for and connection to you when you advocated for you or your family members, then setting this boundary can feel very uncomfortable,” Hart said.
The author suggests saying the following:
If you want to draw a boundary here, you could try one of Hart’s suggestions: “This conversation makes me uncomfortable and I won’t participate in it,” “I’m not going to talk about someone in the family when they aren’t here to share their point of view,” or “This is not our information to share so I’m going to excuse myself.”
I guess I wish articles would role-play more what happens after people assert a boundary like that.
Talking about family members is a problem with my mother. She is not a reliable narrator, for one. And she makes me aware that my family of origin dynamic is alive and well in my sibling's family. One niece is golden and the other is the garbage can. The scapegoat niece spent the last 4 years saving money from her dog-walking business so she could go to college. My mother, who never finished college, says things like "She's not the smart one."
If I were to say, "This conversation makes me feel uncomfortable and I won't participate in it," she would try to get a drama triangle going with my father (rescuer) to punish me (perpetrator).
It's been much more effective for me to change the topic or announce some kind of interruption or say nothing and let the silence redirect her. Even that can create drama, though. I sometimes think my mother's existence is entirely binary: either she's getting attention or she isn't. Anyone preventing her from getting attention gets punished.
If a boundary provides a way for her to get more attention, then she'll use it. If she thinks she learned a way to push my button, she'll just do it more.