when running a script it gave me an error message of Djuggler Out of Memory, I was 5 hours into the extraction script, each loop takes about 80 seconds, and opens 20 Get Web requests and 10 Text Extractions from Source, then runs this loop on about 450 new pages with a Next >

But it seemed like each new loop would run a little slower then the last. Is there a function I can insert into the script that cleans out the memory or runs it a little quicker?

Did you use viewers like the web viewer or grid viewer in your script?
Filling a grid viewer would typically consume memory on each run.

i have 1 web viewer open which updates every 15 minutes with fresh links for the details page extraction..(which does not have a viewer open, just a "get web page"). i also have a grid viewer open to see the results during the extraction. So 2 viewers opened.

also, the extraction seems to run slower as the script progresses, but the extraction loop is the same on page 1 as it is on page 450, i got to about pg 125 before the error occurred,


does the "get web page" work off the IE browser? would increasing the disk space in IE help, would clearing the cache help?

thanks for any help with this.

It's difficult to determine why you run out of memory.
First you need to conclude if it's program memory or video memory you run out of.
While running the script look at the task manager. It you get an out of memory while there is still memory available for programs in the task manager, it is your video memory.

Next, two things can happen.
- The grid viewer consumes memory on each run because it has to remember more data.
- The web viewer (which is IE) consumes memory.

Solutions for the grid viewer should be adding a clear grid action to your script, so it will not hold thousands of text strings and data in video memomory (assuming it is much data in the data grid).
Solutions for the web viewer would be a Browser reset action (available in the last release) in your script before each loop. This should clear any memory consumption, program or video memory.

If all fails, mail us your script so we can debug if it's a bug in the software.
DS

thanks for the reply...

i assume it is the grid that is causing the slowdown and out of memory..

i am collecting 7 columns and at about 2400 rows it gives me the error.
I read your solution to add a Clear grid function, but i am collecting this info to export to Excel and create a DB, so that basically delets everything I have been extracting, correct?

Is there a way I can, for example, after 500 rows of collected data I can send to Excel, so I continously append to an excel file after every 500 rows?


please confirm how to use the Clear Grid and also what do you think is the best way to export the grid data and continue the extraction loop. thanks

2400 rows does not sound as an extremely amount of data. Unless each cell contains much data. Perhaps it's best to try the browser reset first.

It it's still the amount of data in the datagrid you can use the Write to CSV line. This action writes directly to a csv text file and appends a row each time it's called. This is the fastest way to write data without memory consumption. Note you can open csv files directly in Excel or you can put two actions at the end of your script. One imports the csv file, the other exports it to excel.

Use the support form on www.djuggler.com to send us your script if all fails so we can debug.
DS