I rang him and we got into a talk and I asked why he asked for a pause and he said I assaulted him and I didn’t really know what he meant because I’d never do that and he said I repeatedly tried to get with him while he was asleep. That never happened one time I kissed his forehead while he was sleeping before I left. This happened months before. He never tells me any of this stuff and I always ask him to talk to me and I try to not invalidate him when I disagree and I never shout or blame him, I tell him I’m not cross and I want to make sure he’s ok...
That he used the word "assaulted" is concerning because in the legal world of police and courts "assault" is an extremely serious allegation. Of course, this was his perception and a highly subjective one but this should cause you to step back and reconsider the relationship.
Though you're a woman and the professionals might find it hard to picture you as being abusive, can you risk such allegations should they be made in the future? Less than a year into the friendship and he said that. What might he say in future years?
As already noted, we refrain from saying stay or go since that is for you to decide. Weigh all the facts and risks and what you've learned thus far.