CONTENTS

INTRODUCTION HISTORY WORKING TECHNICAL CHALLENGES APPLICATIONS CONCLUSION REFERENCES

INTRODUCTION
The deep Web (also called Deepnet , the invisible Web, dark Web or the hidden Web) refers to World Wide Web content that is not indexed by standard search engines.

Searching on the Internet today can be compared to dragging a net across the surface of the ocean.

Surface Web
The

surface Web (also known as the visible Web or indexable Web) is that portion of the World Wide Web that is indexed by conventional search engines.
Search engines construct a database of the Web by using programs called spiders or Web crawlers that begin with a list of known Web pages.

 The surface Web contains an estimated 2.5 billion documents, growing at a rate of 7.5 million documents per day.

Figure : Search Engines: Dragging a Net Across the Web's Surface

5

HISTORY

In the earliest days of the Web, there were relatively few documents and sites. First, database technology was introduced to the Internet as Bluestone's Sapphire/Web bought by HP and later Oracle. Los Alamos National Laboratory(LANL) founded Innovative Web Applications in 1996. Finally ,Deployed the first "deep web" application in Federal government, in February 1999.
7

The Deep Web: Surfacing Hidden Value

There is still a wealth of information that is deep, and therefore, missed. Reason : Most of the Web's information is buried in dynamically generated sites, and standard search engines never find it. Traditional search engines create their indices by spidering or crawling surface static Web pages. Deep Web sources store their content in databases.

CHARACTERISTICS

Deep Web is massive -- approximately 500 times greater than that visible to conventional search engines -- with much higher quality throughout. Fast, economical, provide depth knowledge.

Deep Web Coverage is Broad, Relevant.

Deep Web searchable databases and search engines combined total of more than 250,000 sites.

10

How Search Engines Work?

Invisible Web search engines are built to construct queries, which connect with dynamic content in real-time in order to obtain current information. Focused on searching pre-selected data sources.

Vertical Searches or specialized proprietary searching is done.

11

FIGURE 1.2 : Harvesting the Deep and Surface Web with a Directed Query Engine
12

MISCONCEPT: Invisible Web

While study/research , we have avoided the term "invisible Web" because it is inaccurate.
The only thing "invisible" about searchable databases is that they are not indexable.

13

BRIGHT PLANET'S TECHNOLOGY

Using BrightPlanet technology, they are totally "visible" to those who need to access them.

BrighT Planet's technology is uniquely suited to tap the deep Web and bring its results to the surface.
14

Why Hidden Web is Better Than Google ????

By using the Hidden Web will you find even more results, comprising the bulk of everything stored on the Internet. Using Google 10% of information Using hidden web 90% of information

15

TECHNICAL CHALLENGES

Site like Google rely on HTML text based searching.

The content of different files cant be searched using a traditional search engine. Also known as the deep web or the invisible web, the hidden web contains thousands of search engines that focus only on their little corner of the world .

16

Some Applications
1.Deep Web as a Search Engine

When youre searching the Web for what you need, youre missing about 90 percent of all the information on the web if you arent searching using Deep Web search engines. The deep web search engines are offering us to access specific searches across the web for sites which have stored data that cant be easily spidered by Google or any other surface web sites .
17

18

2 .The Deep Web in Google

The concept of the deep Web is becoming more complex as search engines such as Google have found ways to integrate deep Web content into their central search function. However, even a search engine as far-reaching as Google provides access to only a very small part of the deep Web.

19

20

Search Engines to Explore the Deep Web

DeepWebTech DeepPeep

Complete Planet
Infomine

21

CONCLUSION

The deep Web thus appears to be a critical source when it is imperative to find a "needle in a haystack." Going to play a major role in the future search engine industry. A rich and huge source of information for a seeker.
22

REFERENCES

Wright, Alex (2009-02-22). "Exploring a 'Deep Web' That Google Cant Grasp". New York Times. http://www.nytimes.com/2009/02/23/technology/internet/23search.ht ml?th&emc=th. Retrieved 2009-02-23. Bergman, Michael K. (August 2001). "The Deep Web: Surfacing Hidden Value". The Journal of Electronic Publishing 7 (1). doi:10.3998/3336451.0007.104. http://quod.lib.umich.edu/cgi/t/text/textidx?c=jep;view=text;rgn=main;idno=3336451.0007.104 Sriram Raghavan; Hector Garcia-Molina (2000) (PDF). Crawling the Hidden Web. Stanford Digital Libraries Technical Report. http://ilpubs.stanford.edu:8090/456/1/2000-36.pdf Barbosa, Luciano; Juliana Freire (2007). An Adaptive Crawler for Locating Hidden-Web Entry Points. WWW Conference 2007.
23