Can I apply for a software patent for the software written with the patent formula?

The formula does not exist, so it is not a patent. It should be a method or technical scheme in a patented technology.

Generally, a single software cannot be patented, even if it can be patented, it is not cost-effective. Instead, you should register the copyright and protect your copyright. And in this case, even if you use other people's things, your software copyright is protected. If you use his patent, it infringes your rights and does not affect your copyright.