Speed up failed articles

Discuss newly added features or request new features.
Post Reply
kloaknet
Posts: 327
Joined: 23 Jul 2014, 08:52

Speed up failed articles

Post by kloaknet » 19 Nov 2017, 11:07

What about let NZBGet ask for a STAT msg just before asking for ARTICLE / BODY (dont know what NZBget uses to get the actual article)? If the STAT responds as a failure, skip the ARTICLE / BODY request.

the STAT request is very very fast, and if that one returns a 430 instead of the 223, the article does not need to be requested. This because when articles don't exist, NZBGet seems to be rather slow in noticing that each article is gone. I guess thats because of the BODY request?


Downside, not all servers do respond correct to STAT, so if its worth the effort, it should be made a server specific option.

(You could also use it to check if the PAR articles will actually be there (not if they are intact of course), before downloading the RARs, so the maximum failure % can be adjusted to reduce download volume.

hugbug
Developer & Admin
Posts: 7028
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: Speed up failed articles

Post by hugbug » 19 Nov 2017, 11:34

I don't see why STAT would be faster than other commands. It needs to access the same storage.

For existing articles the approach with STAT + ARTICLE/BODY doubles the number of requests increasing latency and therefore negatively affecting speed.
kloaknet wrote:
19 Nov 2017, 11:07
NZBGet seems to be rather slow in noticing that each article is gone. I guess thats because of the BODY request?
Probably because it checks all servers.

Have you made any benchmarks comparing performance of STAT vs BODY for failed posts?

kloaknet
Posts: 327
Joined: 23 Jul 2014, 08:52

Re: Speed up failed articles

Post by kloaknet » 19 Nov 2017, 14:17

hadnt made benchmark between the 2 before the first post, but just did a little test. Both are about the same speed, for a typical nzb with 100% failure, it took 1.5 sec for both STAT or BODY to complete, about 2000 articles. About same time in NZBget, only main provider enabled. So NZBGet just does just fine with only BODY.

Was expecting some issue with NZBGet as it took rather long to complete, but it is related to my backup provider somehow, so I have to dig into that. It seems it gives the out of order / wrong replies, resulting in article timeouts being triggered in both a script and NZBget itself. I was expecting the STAT could improve this, but thats not the case

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest