Was any of the agreement in writing? Can it be documented in emails or other communications?
If it went to court, the judge may view the claim as "stale" - perhaps owed but not legally collectible since he should have sought payment much sooner. In other words, the threats may not have teeth, but an experienced lawyer could better advise you on the risks and your options.
For a year of that, I did not pay rent, as we agreed my role in the home and the sacrifice of my business was payment enough.
Any documentation for this? As an example, if we were discussing wills, the later modification could be called a codicil or
amendment to the original agreement. If neither is well documented, then it may just end up as "he-said, she-said".