Crawling vs Indexing
Crawling is the engine reading your page. Indexing is the engine deciding to keep it. A page can be read and not kept.
Crawling
How a search engine finds pages. A program reads a page and follows its links.
Crawling is how a search engine finds pages. A program called a crawler fetches a page and reads it. It then follows the links on that page to find more pages. Pages with no links pointing to them are hard to find. A crawler only reads a page, it does not decide to keep it.
Indexing
Storing a crawled page in the engine's big list. Then it can show up in results.
Indexing is when a search engine stores a page it has crawled so it can show it in results. The engine works out what the page is about and files it in a giant list called the index. Not every crawled page gets in. Pages that are thin, copied or marked noindex are often left out. Only an indexed page can rank.
In one line
- Crawling
- How a search engine finds pages. A program reads a page and follows its links.
- Indexing
- Storing a crawled page in the engine's big list. Then it can show up in results.