Spider here refers
to a bot or a Crawler which is a computer automated software used in the internet.
The spider is used by the search engines to get information and a copy of the
different pages that exist online. The system helps the search engines from
indexing the sites that are related to the users search. The spider works just
like a Robot where it functions on a given instruction in case of web spider,
it basically crawls over the millions of pages on the internet and captures the
text data and stores it with itself.
As a user when you
go to a search engine and enter keywords, searching for information on it, then
the crawler or the spider only helps the search engine to get you the listing.
It is not possible for the spider to check all the available websites together
so it is best to make use of SEO and other tools that can help the spider to
notice your site and capture it. it can then only add your name to the listing
and indexing. Since the spider captures text, it is important to use the right
keywords and tag the images appropriately so that they may be captured by it
and stored for later use.