Thanks for the explanation HB. Since I need to fix this somehow, I need to see what the server is sending (probably raw html right?) when I look in queuedir, I don't see any file name matching to the skipped item. How can I find the file for a particular item so I can open it up and see what the server is sending me?
EDIT: Also, what would be the best way to make sure I am not running into throttling from the server? If I want to do a slowest possible rate of requests, maybe one at a time, and some seconds between each request, by looking at the options I would do this:
URLRetries: 1 (but this is irrelevant to the problem you're saying)
URLInterval: 15 (retries only, it sounds like)
UrlTimeout: 25
UrlConnections: 1
How about time between requests? I don't see a setting for that.
thanks!