If it is a delicate flower, then the impact is still quite great. Ants like to make holes in the soil, so there will be many holes in the pot soil, like a hornet's nest, which will cause the roots of plants to be exposed to the air and gradually dry up. Moreover, the roots of plants are damaged, and they are prone to water shortage and poor growth. Ants have both advantages and disadvantages in dealing with insects, that is, they will eat most of the insects on nearby plants. In fact, potted plants are only prone to aphids, scale insects and whiteflies in most cases. At the same time, they like to "breed" aphids to get honey dew secreted by aphids, and they will carry young aphids around to breed, leading to the spread of aphids.
So ants are quasi-pests.