While in the past there were other types of search engines or directories that relied more on human input/curation, search engines as we know them today all rely, in one way or another, on Spiders or Crawlers to index the internet for searching. 

The most visible search engines available today, Google, Bing, Yahoo!, and many more, deploy their own spiders to create an index of the internet.

Metasearch Engines search multiple indexes simultaneously. They might take your search terms and use them in Google, Bing, and Yahoo! simultaneously, before returning their own ranked results. 

Some great metasearch engines exist. 

dogpile.com - is a powerful metasearch engine that also provides a number of suggestions to assist in your search.

yippy.com - groups similar search results into "clusters", allowing you to narrow your search to exactly what you want it to be.