To him, I am sleeping with every person whom I come in contact with...our kids obviously only have friends because I am sleeping with the dads, our kids only make sports teams because obviously I am sleeping with the coaches, somehow whoever I am "sleeping with all day" while I am at work pays all 3 of the hospitals that I work for so that the hospitals in turn can pay me a salary from their accounts, etc. His accusations are crazy making and so obviously not true, but how do I deal with this? In the distant past, I would fight back and need to prove that I was right and did everything including taking a polygraph test...which obviously instead of doing the polygraph while I was in the room, I was sleeping with the man who ran the test! AHHHH...More recently, I have just told him that I will not discuss things that are not true and will end the conversation, text or walk away.
This is such a tough conversation because it's not only about you, but the kids too.
You told him that you wouldn't discuss this anymore, which is a hard boundary. That's a good thing, but we also know that a new boundary is going to be challenged. Keep doing what you're doing, reassure him that you're remaining faithful, but refuse to go into more detail.
Have you tried spinning it back around on him. Like asking, "Why do you think I'm cheating on you? When would I even have the time for that?" Make him be accountable instead of just defending or retreating. Conversations like this can be productive, while arguments cannot. If he's in an even mood and wants to talk about it, then talk about it in a reassuring way. Let him know that you're committed to him and taking care of the kids.
But...now he has brought it on to our 2 younger kids, because I walk away and they can't, and traps them in the car and questions them about "who is sleeping in mom's bed with her" and "who is mom dating". Its at the point where my kids want nothing to do with their dad when he is like this.
This is so incredibly hard and it can't continue. I would consider contacting the DV office and reporting it if he won't back off the kids immediately. This is dangerous behavior and like you said, the kids can't fight back. So this definitely needs a stronger boundary for their protection. If he can't back off this type of talk, he doesn't need to be around the kids at all until he's more balanced and stable.