Sunday 17 July 2011

How does Google index Wikipedia so fast?

Does anybody know how Google indexes Wikipedia so fast?

Based on this post on the Google Blog (googlecustomsearch.blogspot.co ...), I though that Wikipedia pings Google every time something changes on its pages, similar to how Blogger platform pings Google. Is this true or no?

Or does Google have any special setup of its own to index Wikipedia's pages continuously?



It's dynamically generatet with a PHP script. For big sites it's probably better to check for changes and only generate if something changed -- or geneare it only all XY minutes/hours/days. Depends on the infrastructure.
The informations needed are all in the database, so it's not such a hard task.
And here is the proof:  

http://svn.wikimedia.org/viewvc/mediawiki/trunk/phase3/maintenance/generateSitemap.php?view=log

http://www.mediawiki.org/wiki/Manual:GenerateSitemap.php

0 comments:

Post a Comment

Related Posts Plugin for WordPress, Blogger...