Which department should I complain about being cheated by a private loan company outside?

Complaints should be reported to the Internet Finance Association and the local financial office.

However, complaints should provide corresponding evidence, not just because of what you said. All this remains to be examined. If the corresponding evidence is not provided, the complaint will not be successful, and it will be mistaken for a dispute caused by the debtor's own failure to pay back the money.