It depends on whether the evidence is complete and sufficient.
But the problem now is that the IOUs are made by lawyers and there will be no flaws.
If he sues you for repayment, I'm afraid you'll have to pay back. You have laid a hidden danger for yourself.