web crawler spider demo script - Web scraping and data integration software - Djuggler Forum - Default Group - Script Repository

Unanswered Active Topics

Forums Search

Forums > Default Group > Script Repository

web crawler spider demo script

Last Post 14 Sep 2009 11:14 AM by Marc. 1 Replies.

Sort:

	Prev Next
Add Reply

Author

Messages

Tijn

29 Aug 2009 10:05 AM
Attached is the web crawler demo script that can be found as compiled executable script on the demonstration page. It is a simulation how a bot like the Google bot will follow hyper links in a web site. The script asks for a URL and a crawler depth. Next it will follow and scan all hyper links found and report them in a HTML report, including server results as a 200 or 404 code. The script has been made with Djuggler Builder version 3 and demonstrates the deployment dialogs, it also shows how to use the read next link action and some other new features in Djuggler 3.
WebCrawlerDemo.djs

Marc

14 Sep 2009 11:14 AM
Nice demo. Djuggler is the best web scraper tool even, that is for sure! Thanks. Marc

Add Reply

Forums > Default Group > Script Repository

Quick Reply

toggle

	Username:
	Subject:
	Body:
	Security Code:	Enter the code shown above:
Submit

Powered by Active Forums

	Forum participation and optional registration You don't need to be registered to partcipate in the Djuggler forums, however if you want to subscribe to email notifications you need to register. You can also subscribe to the forum RSS feed. Forum participation and optional registration You don't need to be registered to partcipate in the Djuggler forums, however if you want to subscribe to email notifications you need to register. You can also subscribe to the forum RSS feed.