The MOMspider
instruction file may contain three types of tasks:
- Site
- Site traversal specifies that any URL which points to another
site than that of the top URL is a leaf node. (That site will not be
searched beyond the given URL.)
- Tree
- Tree traversal specifies that any URL which does not belong to the
tree under the top URL is a leaf node. (MOMspider will not search beyond
the given URL.)
- Owner
- Owner traversal specifies that any document with a different "Owner"
metainformation header, or no "Owner" is a leaf node. (The traversal will
not go beyond that URL.)
Site traversal is useful for Webmasters to verify the internal and
external consistency of links.
Tree traversal is useful for a user to verify her own directory (with
possible subdirectories) on a Web server.
Owner traversal is useful for verifying a single URL, since most documents
do not have "Owner" information.