Page 1 of 1

Only download NZB if required - RSS feeds?

Posted: 30 Jan 2018, 18:14
by get39678
Is it possible to only download the actual NZB file if and when NZBGet needs it?

Example, when fetching a RSS feed it may have 20 matches but with only say 3 unique dupekeys but it currently downloads all 20 NZB files instead of just the 3 NZB files of the ones with the highest dupescore.

If NZBget gets to a queue item where the NZB file is not available locally could it then download and use it instead of downloading all of them regardless and not using a large proportion of them?

If you have an indexer with limited NZB downloads this would be great feature and stop it from using resources when not required? e.g. it would make RSS/Queue processing much quicker if it wasn't downloading NZB files it didn't need.


Re: Only download NZB if required - RSS feeds?

Posted: 22 May 2018, 20:46
by get39678
hugbug I can see you seem to be giving new features a break at the moment (well deserved!) but do you see any method of achieving this with the current setup? If not possible I hope it could be considered for the future.

Re: Only download NZB if required - RSS feeds?

Posted: 24 May 2018, 15:56
by hugbug
You can try dupestatus filter in RSS but you know that already.

I added issue to tracker - Delayed loading of NZBs from URLs #541 and am hoping to implement it one day ;).

Re: Only download NZB if required - RSS feeds?

Posted: 27 May 2018, 11:25
by get39678
Thanks hugbug. As you have noted dupestatus does help in other circumstances but it doesn't help in this case as the downloading of the nzbs first is just the way it currently works.

Currently if RSS feeds find the following it will download 5 nzb files, the queue will be processed, duplicates appropriately removed using dupekey and test1 with dupescore 100, test2 and test3 will download. In the past (not sure if it still does this) but if there is a big list to process from the RSS it may also start downloading a dupekey with a lower score if it hasn't come across the higher dupescore yet but it will eventually sort itself out.

dupekey=test1 dupescore=100
dupekey=test1 dupescore=50
dupekey=test1 dupescore=25
dupekey=test2 dupescore=50
dupekey=test3 dupescolre=25

This proposed change will result in only 3 nzb files, test1 dupescore 100, test2 and test3 being downloaded and processed. test1 50 and test1 25 will be immediately added to the history as duplicates but *without* the nzb being downloaded. You can think of it as a preprocessing stage before all the matches in the RSS are added to the Queue dupekeys are checked and they go to either the queue to have their nzb downloaded and processed or straight to the History without the nzb being downloaded. If an action is later specified on test1 50 and test1 25 in the History nzbget will check if the nzb exists if it doesn't it will download it. The name, category, age and size used on the History screen has already been found when the RSS was processed.

Re: Only download NZB if required - RSS feeds?

Posted: 07 Jul 2018, 16:56
by hugbug
Your request (from the first post of this topic) has been implemented. Currently it is in branch 541-url-dupe. The feature required many changes and needs thorough testing. I'm going to test it myself for a while before merging into develop branch.

It would be nice if you could test it too. If you compile nzbget yourself please compile from the new branch. If you use nzbget installer please tell me which platform/CPU you run in on and I'll provide you with an installer.