The goal of all search engines is to find and organize distributed data found on the Internet. Before search engines were developed, the Internet was a collection of File Transfer Protocol (FTP) sites in which users would navigate to find specific shared files. As the central list of web servers joining the Internet grew, and the World Wide Web became the interface of choice for accessing the Internet, the need for finding and organizing the distributed data files on FTP web servers grew. Search engines began due to this need to more easily navigate the web servers and files on the Internet.
Excite was born in February 1993 as a university project called Architext involving six undergraduate students at Stanford seeking to use statistical analysis of word relationships to improve relevancy of searches on the Internet. This school project eventually led to Excite’s commercial release as a crawling search engine at the end of 1995. With solid growth in 1996, Excite purchased WebCrawler and Magellan. Toward the end of the 1990s, Excite partnered with MSN and Netscape providing search services..
Lycos was one of the earliest search engines developed in 1994 at Carnegie Mellon University by Dr. Michael Mauldin and a team of researchers. The Lycos name came from the Latin term “lycosidea” referencing wolf spiders that hunt and actively stalk prey. The company was founded on $2 million in venture capital funding from CMGI. The company was headed by Bob Davis, who concentrated on building Lycos into an advertising-supported web portal. The company went public in April of the next year with little money. With phenomenal growth in its catalog, Lycos had the largest index at the end of 1996 with 60 million documents. In 1997, Lycos Pro was launched with a new search algorithm and continued to grow. By 1999, Lycos would emerge from a crowded pack to become the most-visited web portal. Over the next few years, Lycos would become one of the most profitable Internet businesses and acquired nearly two dozen high profile Internet brands.
AltaVista, meaning “a view from above,” developed out of research by scientists at Digital Equipment Corporation’s (DEC) Western Research Laboratory in Palo Alto, California during the spring of 1995. They were trying to showcase their computer database system called the Alpha 8400 TurboLaser that was faster than its competitors. Scientists developed a search tool to crawl, store and quickly index every word of all HTML web pages on the Internet. This new search tool was powerful. For example, in August 1995, it conducted its first full-scale crawl of the web bringing back about 10 million pages.
Google was founded in 1998 as another school project at Stanford University in California. In January 1996, Stanford PhD students Larry Page and Sergey Brin began researching the concept of a search engine based on relevancy ranking. Page and Brin believed that search engines should analyze and rank websites based on the number of times search terms appeared on web pages. Likewise, Page and Brin developed a search engine nicknamed “BackRub.” BackRub checked the number and quality of links coming back to websites in order to estimate the value of a website. Brin and Page’s research eventually led them to develop the trademarked PageRank™ link analysis algorithm that Google’s search engine would use to assign a numerical weighting to hyperlinked document elements.
MSN Search was a service offered as part of Microsoft’s network of web services. The Microsoft Network debuted as an online service and Internet service provider in August 1995. During the 1990s, Microsoft launched Internet Explorer as a bundled part of their operating system and software products. MSN Search first launched in 1998 displaying search results from Inktomi and later blending results with Looksmart. For a short time in 1999, AltaVista search results were used instead of Inktomi. Since 2004, MSN Search began using its own built-in search results.