Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Journal articles
  4. On near-uniform URL sampling
 
research article

On near-uniform URL sampling

Henzinger, Monika R.  
•
Heydon, Allan
•
Mitzenmacher, Michael
Show more
2000
Comput. Networks

We consider the problem of sampling URLs uniformly at random from the Web. A tool for sampling URLs uniformly can be used to estimate various properties of Web pages, such as the fraction of pages in various Internet domains or written in various languages. Moreover, uniform URL sampling can be used to determine the sizes of various search engines relative to the entire Web. In this paper, we consider sampling approaches based on random walks of the Web graph. In particular, we suggest ways of improving sampling based on random walks to make the samples closer to uniform. We suggest a natural test bed based on random graphs for testing the effectiveness of our procedures. We then use our sampling approach to estimate the distribution of pages over various Internet domains and to estimate the coverage of various search engine indexes.

  • Files
  • Details
  • Metrics
Type
research article
DOI
10.1016/S1389-1286(00)00055-4
Scopus ID

2-s2.0-0033705620

Author(s)
Henzinger, Monika R.  
Heydon, Allan
Mitzenmacher, Michael
Najork, Marc
Date Issued

2000

Published in
Comput. Networks
Volume

33

Issue

1

Start page

295

End page

308

Subjects

HTML

•

Markov processes

•

Probability

•

Random processes

•

Resource allocation

•

Sampling

•

Search engines

•

Internet domain distribution

•

Random walks

•

Uniform resource locator

•

Web graph

•

World Wide Web

Editorial or Peer reviewed

REVIEWED

Written at

OTHER

EPFL units
LTAA  
Available on Infoscience
January 18, 2007
Use this identifier to reference this record
https://infoscience.epfl.ch/handle/20.500.14299/239626
Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés