er when
	<i>Process Forms</i> is enabled and when encountering HTML forms that
	have to be POSTed.

	<h3>Parse HTML Comments</h3>
	This option defines whether the spider should also spider the HTML
	comments searching for links to resources. Only the resources found in
	commented valid HTML tags will be processed.

	<h3>Parse 'robots.txt' files</h3>
	This option defines whether the spider should also spider the
	robots.txt files found on websites, searching for links to resources.
	This option does not defi