What is a python reptile?

Web crawler (also called web spider, web robot, and more often called web chaser in FOAF community) is a program or script that automatically crawls information on the World Wide Web according to certain rules. Other less common names are ant, automatic index, emulator or worm.

In fact, it is popular to get the data you want on the web page through the program, that is, automatically grab the data.

What can reptiles do?

You can use a crawler to grab the pictures, videos and other data you want. As long as the data can be accessed through the browser, it can be obtained through the crawler.

What is the nature of reptiles?

Simulate a browser to open a webpage and get the data we want in the webpage.

The process of opening a web page with a browser:

When you enter an address in the browser, you find the server host through the DNS server and send a request to the server. The server parses and sends the results to the user's browser, including html, js, css and other file contents. The browser parses and finally presents the results to the user on the browser.

Therefore, the browser results that users see are all composed of html codes, and our crawler just obtains these contents, and obtains the resources we want by analyzing and filtering the HTML codes.