MOMspider
uses four (user) files:
- An instruction file contains configuration parameters
and sites and document trees to examine. Its default name is
$HOME/.momspider-instruct.
- The avoid file contains names of documents and sites
which should not be checked. This file is updated by MOMspider, by
examining the /robots.txt file on every server.
Its default name is $HOME/.momspider-avoid.
- The sites file contains the names of all visited sites,
and the time at which the site needs to be visited again.
- The HTML index file contains the report generated by
MOMspider. It must reside somewhere where it can also have an URL.
