Department of Mathematics and Computing ScienceWorld Wide Web OrganizationPaul De Bra

Example MOMspider Instruction File

Below is a MOMspider instruction file:

# Instructions file for MOMspider

# The file with hosts to avoid is here:
AvoidFile	/home/debra/.momspider-avoid

# The file with sites, written by momspider
SitesFile	/home/debra/.momspider-sites

# How many days between checks of a site's /robots.txt file?
SitesCheck	1

# Who should Webmasters send a report to when something goes wrong ?
ReplyTo	debra@win.tue.nl

# MaxDepth is to prevent infinite holes
MaxDepth	10

<Tree
	Name	WWW
	TopURL	http://wwwis.win.tue.nl/~debra/
	IndexURL	http://pcpaul.win.tue.nl/win-index.html
	IndexFile	/usr/local/etc/httpd/htdocs/win-index.html
	IndexTitle	MOMspider Index for homepage on www.win
	EmailAddress	debra@win.tue.nl
	EmailBroken
	EmailRedirected
	EmailChanged 7
	EmailExpired 7
	Exclude	http://www.win.tue.nl:8080/
>


home purple tour