How the web works
This video explains how search works now and what the future of search might look like.
The World Wide Web is vast, and it is constantly expanding. To search it we access search engines like Google. Each search engine is different and returns different results for the same search. These search engines use crawlers or spiders to find information in the web and return it to the search engine’s database. Our searches then search this database.
However, web crawlers or spiders can’t search the entire web...what can’t be searched is often referred to as the deep web.
It is possible for websites to be designed to block access to these crawlers. Also, password protected content is similarly unattainable to these spiders. Sometimes it is not deliberate but web-crawlers can’t get some tricky data, for example, maybe you’re searching for a scene in TV series - unless it has been described or identified in words, then that information is invisible.
This is the deep web. Search engines can only search the visible surface web which roughly 10% of the total content of the web. To improve your access to information for your research, use the databases your library subscribe to. The Library subscribe to a number of databases that are part of the deep web and are therefore not searched by web search engines.
Remember - there’s more out there than a simple Google search can discover - see Beyond Google.
When we search online our search results are not just determined by the keywords we use - they are affected by our location, our previous searches, and things we click on. It is a way of personalising our search experience - a web algorithm designed to anticipate the information you’re likely to want. Find out here about Google's personalised search.
This can cause you problems when you are researching. You’re likely to be provided with search results containing information you agree with, sources that reinforce your belief systems, and keep you in your own ‘filter bubble’. You may notice, for example, that your news feed in Facebook is personalised.
The act of research should be about exploring all sorts of information,challenging your perspective, and broadening your world view.
The term was coined by Eli Pariser and you can see him talk about in the video. There is also a 30 minute radio programme about Bursting the Social Media Bubble .
Personalisation can only come about if the web industry tracks our online behaviour and searches. . Google services are only free because you and your data effectively become the product (which can be shared, sold and utilised for marketing). (See Your digital footprint - online privacy)
However, it is not only the web industry that is interested in your online behaviour. This short video explains how and why governments collect information about what you do online.
This documentary The dark web provides more detail on surveillance and it's consequences.