Hi Miss Topaz,
I worry if I tell him I am leaving the convo it will infuriate him as he can't see he is being unreasonable(or about to becomes so) and a split will still occur.
you are really not responsible for his emotions. In fact you have not much control over them. He does not have much control over them.
When I was a child I was told that when I won't eat my spinach the sun would not shine the next day.
Now it is ok to worry about the weather.
But it is really not ok to feel responsible for it.
Now your behavior and his mental state are a bit closer linked than in my example. But still there are so many factors affecting him - external and internal to him - that you should refuse to carry that load.
It is important to keep the communication going in some way - we want to be connected.
Whether he thinks at any moment you are black or white - it is impossible to control. Tying to do that makes you easily game-able and a pawn to his emotions. Who wants to be connected to a pawn? Ultimately this is corrosive to the relationship.
Be more yourself

. Bart does not always appreciate his sister but it is his sister after all and he stands by her.