Apart from verifying an infostructure, users often wish to have a local
copy of all documents, so they can retrieve them very quickly.
There are two ways to obtain local copies of documents:
- mirroring:
- A specific list of documents (URLs) is retrieved and stored on the
local system. A "conditional get" can be used to avoid retrieving documents
a second time when they are not modified.
Mirroring Web documents can (for instance) be done by means of
WebCopy.
The copied documents cannot be retrieved using their original URL.
- caching:
- A (proxy) cache system can be used to keep copies of documents for
a (short) while after retrieving them.
In order to have local copies before retrieving the documents,
they can be downloaded at night.
When MOMspider
is run through the cache, local copies are kept in the cache.
