Here's a few ideas for thought:
A) He has found it upset you. Since a pwBPD doesn't like to be the only one in the room who is upset, not matter what about, you need to be made just as upset as he is, however that's accomplished. So in a really strange way, this is just a tool to make you share his emotional state. BF will poke at me to make me mad to help validate his emotions... even if it's just to retroactively blame me for him being mad (BF: "Well, you got mad at me." Me: "Not until you yelled at me for an hour." BF: "Well, you're mad at me."
B) He has a fear of abandonment - thank you BPD. What's more abandoning and invalidating that being cheated on and then left for someone more attractive, nicer, funnier - all the things the person with BPD think you don't see in him/her?
C) He may have cheated himself. Since you cannot possibly have thoughts and emotions or self control different than his own, you, too, must have succumbed to temptation and cheated. And in order to avoid his shame at being unfaithful, it's projected onto you... if he can prove YOU cheated, then he no longer has to feel bad if he did.
D) BF told me a few months past that something always has a person to blame. He does not believe in accidents. He does not believe I can go to the store and not find what he wanted - I must have left it out on purpose, I must have not paid attention because I don't care, etc. So forgetting lunchmeat and being smart-alecky all point to him you don't care about him, because he's the center of the world and all roads lead to him. So if nothing happens on accident and all things are planned to annoy or hurt him, and a person is needed to accept blame for all things that annoy and hurt him, then you win the prize.
Do any of these fit? I have had one lover in my whole life, and I am still with him. And he still, time to time, loves to accuse me of cheating, of wanting to cheat and of wanting to leave him. He can't face the fact that these are HIS feelings coming out - so he has to blame them on me. AND at its worst, we were dealing with the fact HE DID cheat on me.