RSS feed choking on import of file regardless of timeout/retries

Get help, report and discuss bugs.
bluenote
Posts: 48
Joined: 31 Oct 2014, 03:26

RSS feed choking on import of file regardless of timeout/retries

Post by bluenote » 22 Nov 2017, 00:31

Hi HB

I run a RSS feed manually, and I get a lot of items put into History as 'skipped'. I have tried a lot of different retry, timeout, etc values, and nothing seems to help. I notice that when this is going on, my Chrome addon that queries state of nzbget also shows 'err'. My connectivity to the box is uninterrupted (pings continue).

Can you suggest any settings I should try? it is a pain to go into history all the time and 'download again' for URL entries.

I'm running on a raspi 2.

thanks

hugbug
Developer & Admin
Posts: 7645
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: RSS feed choking on import of file regardless of timeout/retries

Post by hugbug » 22 Nov 2017, 09:13

"Skipped" means the server returned a non-nzb file. Good servers return an HTTP error code if they can't process request. In this case nzbget retries after wait interval. Bad servers return HTTP status "OK" and a page with an error description. Due to OK-status nzbget will not retry.

After the file is successfully fetched nzbget scans it. Nzb-files are added to queue. All files (nzb and non-nzb) are processed by extension scripts, that allows to process files with other extensions such as zipped nzb-files if you have an extension script for that.

All fetched files are saved into QueueDir where you can inspect them.
bluenote wrote:
22 Nov 2017, 00:31
I have tried a lot of different retry, timeout, etc values, and nothing seems to help.
Nothing on nzbget side will help because that's a server issue.

bluenote
Posts: 48
Joined: 31 Oct 2014, 03:26

Re: RSS feed choking on import of file regardless of timeout/retries

Post by bluenote » 22 Nov 2017, 18:02

Thanks for the explanation HB. Since I need to fix this somehow, I need to see what the server is sending (probably raw html right?) when I look in queuedir, I don't see any file name matching to the skipped item. How can I find the file for a particular item so I can open it up and see what the server is sending me?

EDIT: Also, what would be the best way to make sure I am not running into throttling from the server? If I want to do a slowest possible rate of requests, maybe one at a time, and some seconds between each request, by looking at the options I would do this:

URLRetries: 1 (but this is irrelevant to the problem you're saying)
URLInterval: 15 (retries only, it sounds like)
UrlTimeout: 25
UrlConnections: 1

How about time between requests? I don't see a setting for that.

thanks! :)

hugbug
Developer & Admin
Posts: 7645
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: RSS feed choking on import of file regardless of timeout/retries

Post by hugbug » 22 Nov 2017, 18:52

Hm, it should have name of nzb. Try removing everything from NzbDir (backup somewhere) to narrow down the search or by time.

bluenote
Posts: 48
Joined: 31 Oct 2014, 03:26

Re: RSS feed choking on import of file regardless of timeout/retries

Post by bluenote » 22 Nov 2017, 19:07

Thanks I will try this.

Sorry I ninja-edited previous post and I think we crossed times probably. Could you give your opinion:

"EDIT: Also, what would be the best way to make sure I am not running into throttling from the server? If I want to do a slowest possible rate of requests, maybe one at a time, and some seconds between each request, by looking at the options I would do this:

URLRetries: 1 (but this is irrelevant to the problem you're saying)
URLInterval: 15 (retries only, it sounds like)
UrlTimeout: 25
UrlConnections: 1

How about time between requests? I don't see a setting for that."

thanks

hugbug
Developer & Admin
Posts: 7645
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: RSS feed choking on import of file regardless of timeout/retries

Post by hugbug » 22 Nov 2017, 19:13

RSS feed interval is set per RSS feed via FeedX.Interval.

bluenote
Posts: 48
Joined: 31 Oct 2014, 03:26

Re: RSS feed choking on import of file regardless of timeout/retries

Post by bluenote » 22 Nov 2017, 21:16

I know about Feed Interval, but I'm talking about request interval. For example, if I manually poll a RSS feed,
maybe 5 items come back. So if the URLConnections is 5, then all are requested at once.

But if I have URLConnections set to 1, what governs the time between requests?
URLInterval says it is a "retry" interval. Does it also govern time between requests?
If URLInterval is 5 seconds and URLConnections is 1 does this happen:

(Download NZB #1)
(wait 5 seconds)
(Download NZB #2)
(wait 5 seconds)
etc

Or is it immediate?

thanks :)

hugbug
Developer & Admin
Posts: 7645
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: RSS feed choking on import of file regardless of timeout/retries

Post by hugbug » 22 Nov 2017, 21:27

bluenote wrote:
22 Nov 2017, 21:16
Or is it immediate?
Yep!

bluenote
Posts: 48
Joined: 31 Oct 2014, 03:26

Re: RSS feed choking on import of file regardless of timeout/retries

Post by bluenote » 22 Nov 2017, 22:06

Thank you, I'll try what you suggested, and also investigate that maybe this is caused by rate limiting on the indexer.

hugbug
Developer & Admin
Posts: 7645
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: RSS feed choking on import of file regardless of timeout/retries

Post by hugbug » 22 Nov 2017, 22:20

Still no files in nzbdir?

See related https://github.com/nzbget/nzbget/issues ... -317532787

If indexer throttles you the feed fetching works OK with that but there is no retry for skipped or failed items which are already in history. Maybe I can improve this but first please investigate what is really hapenning here.

Post Reply

Who is online

Users browsing this forum: No registered users and 52 guests