The answer can be traced back to the early 20th century, when Harvey Hubbell Jr applied for a number of patents for electric plugs (starting with the first detachable electric plug in 1904). Some designs of Hubbell use recessed pins that align with small protrusions on the power socket. The concave-convex system helps to hold the plug in place when you plug it into the socket. Indentation finally gave way to holes, which, according to HowStuffWorks, have the same function.
But this is only part of the story. As various YouTube users have proved, modern sockets are usually no longer bumpy-they use friction and pressure to prevent the plug from falling off the wall. Today, these holes have other uses. Some manufacturers, such as those in the video below, insert a rod into all the holes of a row of forks to lock their positions and wrap them in plastic.
Warning information can also pass through the hole, effectively ensuring that consumers can see the warning before using the additional equipment. Manufacturers can even fix a small lock or zipper through one or two holes as a means of factory sealing. Another popular theory is that these holes can save metal, which helps to reduce costs over time.
According to the official regulations made by American National Standards Institute (ANSI) and National Electrical Manufacturers Association (NEMA), these holes are "optional" and "for manufacturing purposes only". However, if you want to put a small padlock through the jack to prevent your child from powering some devices, NEMA may not mind.