Page 4 of 27

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Posted: 27 Aug 2015, 08:00
by hugbug
fernandog wrote:@hugbug is it possible to include this feature in nzbget code?
With github, the dev could maintain it
There is no easier way to integrate this into NZBGet than the current extension script method, since NZBGet is written in C++ and the script in Python.

There is also nothing bad in extension scripts and quite the contrary - this is one of the main NZBGet feature. Extensions allow other developers add missing functionality, and doing this using more comfortable languages (python) for tasks where performance doesn't matter that much.

Why do you ask about integration, do you have issues with this script (have you tried it at all?) which you believe can be solved if integrated natively or what's the matter?

The script is complex and has sophisticated logic. Reimplementing it as a whole requires a lot of time. If you can be more specific about your problem I might be able to develop a simpler solution.

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Posted: 27 Aug 2015, 10:09
by fernandog
No issues. Asked because could be a core feature. Agree with your points.

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Posted: 28 Aug 2015, 14:49
by kloaknet
fernandog wrote:@hugbug is it possible to include this feature in nzbget code?
With github, the dev could maintain it
Thanks for showing appreciation towards the script :-)
hugbug wrote:... The script is complex and has sophisticated logic. Reimplementing it as a whole requires a lot of time. ...
I have to agree with hugbug, its a nice feature script, but a lot of people don't really need it (at least when your provider is running ok), don't really understand retention issues and just download another release via automation tools, or don't bother that much about eating blocks, even when at the end the download fails. It is most useful when using RSS, having a slow internet connection, or a limited capacity machine.

The script is rather complex, more messed up than sophisticated (my first steps in python actually ;) ). Even though some tricks I have implemented can easier be extracted within NZBget itself, there is a big part that indeed will cost a lot of time to rewrite it towards C++ (where I have even more limited knowledge of). I think hugbug can spend his time more wisely adding all kinds of other cool features in NZBGet. Besides this, it probably would confuse a lot of users if it was an embedded function in NZBget to what / how to do with it /why their downloads don't start immediately.

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Posted: 20 Oct 2015, 01:58
by binreader
Hey There, im on nzbget v16.0

I believe i set everything up for this script to work per the instructions but all i get is error message in the log.

Completion: [V] No such NZB file, resuming NZB

What could be the cause of this issue? i try add nzb via local file or url same error

thanks

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Posted: 20 Oct 2015, 14:26
by kloaknet
binreader wrote:Hey There, im on nzbget v16.0

I believe i set everything up for this script to work per the instructions but all i get is error message in the log.

Completion: [V] No such NZB file, resuming NZB

What could be the cause of this issue? i try add nzb via local file or url same error

thanks
Is that for all of your added NZB or just for a specific one? The No such NZB file comment means that the NZB file cannot be found, while it is supposed to be there. It could be an dir error, because I may only tested the last one on a Windows machine, and not also a Linux device.

as you have the Verbose option enabled, I guess your log should show something like this (I have some sort of partially updated test version running):

Code: Select all

INFO	Tue Oct 20 2015 16:17:32	Completion: [V] NZB contains 253 articles, 225 rar articles, 28 par2 articles
INFO	Tue Oct 20 2015 16:17:32	Completion: [V] get_nzb_data(fname=C:\ProgramData\NZBGet\nzb\XXX.HDTV.x264-CBFM.nzb.queued)
INFO	Tue Oct 20 2015 16:17:32	Completion: Checking: "XXX.HDTV.x264-CBFM.nzb.queued"
INFO	Tue Oct 20 2015 16:17:32	Completion: [V] get_nzb_status(nzb=[800, 'XXX.HDTV.x264-CBFM.nzb.queued', 1445350540, 896, u'', 0])
INFO	Tue Oct 20 2015 16:17:32	Completion: [V] Ignoring priority of items older than AgeLimit of 4 hours
INFO	Tue Oct 20 2015 16:17:32	Completion: [V] Downloading for NZBGet paused
INFO	Tue Oct 20 2015 16:17:32	Completion: [V] Waiting for NZBGet to end downloading
INFO	Tue Oct 20 2015 16:17:32	Completion: [V] nzbget_paused()
INFO	Tue Oct 20 2015 16:17:32	Completion: [V] Found paused item in queue: "XXX.HDTV.x264-CBFM.nzb.queued"
INFO	Tue Oct 20 2015 16:17:32	Completion: [V] completion.lock file created
INFO	Tue Oct 20 2015 16:17:32	Completion: [V] lock_file()
INFO	Tue Oct 20 2015 16:17:31	Executing queue-script Completion.py for XXX.HDTV.x264-CBFM
INFO	Tue Oct 20 2015 16:17:31	Collection XXX.HDTV.x264-CBFM added to queue
INFO	Tue Oct 20 2015 16:17:31	Adding collection XXX.HDTV.x264-CBFM.nzb to queue
INFO	Tue Oct 20 2015 16:17:31	Completion: [V] Pausing: "XXX.HDTV.x264-CBFM.nzb"
INFO	Tue Oct 20 2015 16:17:31	Completion: [V] Expected queued file name: "XXX.HDTV.x264-CBFM.nzb.queued"
INFO	Tue Oct 20 2015 16:17:31	Executing scan-script Completion.py for XXX.HDTV.x264-CBFM.nzb
INFO	Tue Oct 20 2015 16:17:30	Queue XXX.HDTV.x264-CBFM @ indexer
Then it is important for me to get the dir parts on your end to see what is wrong:

Code: Select all

INFO	Tue Oct 20 2015 16:17:32	Completion: [V] get_nzb_data(fname=C:\ProgramData\NZBGet\nzb\XXX.HDTV.x264-CBFM.nzb.queued)
INFO	Tue Oct 20 2015 16:17:32	Completion: Checking: "XXX.HDTV.x264-CBFM.nzb.queued"
INFO	Tue Oct 20 2015 16:17:32	Completion: [V] get_nzb_status(nzb=[800, 'XXX.HDTV.x264-CBFM.nzb.queued', 1445350540, 896, u'', 0])
I expect some issue with the ".queued" part at the end, or somewhere in the first part of the file, like "C:\ProgramData\NZBGet\nzb\" So could you try to give me similar output as above? I might not be able to resolve the issue within a week from now, because of some busy schedule, but if you post some logs, I will look into it.

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Posted: 20 Oct 2015, 21:12
by binreader
Thanks for responding man, I will try to get the log information similar to you to share later today.

I had to disable script cuz was giving me error on each nzb

We having some propagation issues at the moment and seems this script is the right dude for the job.

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Posted: 21 Oct 2015, 02:01
by binreader
ok kloaknet here is my log snippet, it just proceeds to download and file is too incomplete, doesnt get paused or checked for completion as far as i can tell. and goes straight to the failure pile when done :(

Code: Select all

Tue Oct 20 21:47:05 2015	INFO	nzbget 16.0 server-mode
Tue Oct 20 21:47:50 2015	INFO	Executing scan-script Completion.py for Stuff.720p.HDTV.X264-GROUP.nzb
Tue Oct 20 21:47:50 2015	INFO	Completion: [V] Expected queued file name: "C:\ProgramData\NZBGet\nzb\Stuff.720p.HDTV.X264-GROUP.nzb.queued"
Tue Oct 20 21:47:50 2015	INFO	Completion: [V] Pausing: "Stuff.720p.HDTV.X264-GROUP.nzb"
Tue Oct 20 21:47:50 2015	INFO	Executing scan-script Pushbullet.py for Stuff.720p.HDTV.X264-GROUP.nzb
Tue Oct 20 21:47:50 2015	DETAIL	Pushbullet: Script successfully started
Tue Oct 20 21:47:50 2015	DETAIL	Pushbullet: ScanScript Notification
Tue Oct 20 21:47:50 2015	DETAIL	Pushbullet: Sending to Pushbullet
Tue Oct 20 21:47:52 2015	DETAIL	Pushbullet: Sent to Pushbullet
Tue Oct 20 21:47:52 2015	INFO	Adding collection Stuff.720p.HDTV.X264-GROUP.nzb to queue
Tue Oct 20 21:47:52 2015	INFO	Collection Stuff.720p.HDTV.X264-GROUP added to queue
Tue Oct 20 21:47:52 2015	INFO	Executing queue-script Completion.py for Stuff.720p.HDTV.X264-GROUP
Tue Oct 20 21:47:52 2015	INFO	Completion: [V] lock_file()
Tue Oct 20 21:47:52 2015	INFO	Completion: [V] completion.lock file created
Tue Oct 20 21:47:52 2015	INFO	Completion: [V] Found paused item in queue: "C:\\ProgramData\\NZBGet\\nzb\\Stuff.720p.HDTV.X264-GROUP.nzb.queued"
Tue Oct 20 21:47:52 2015	INFO	Completion: [V] nzbget_paused()
Tue Oct 20 21:47:52 2015	INFO	Completion: [V] Waiting for NZBGet to end downloading
Tue Oct 20 21:47:52 2015	INFO	Completion: [V] Downloading for NZBGet paused
Tue Oct 20 21:47:52 2015	INFO	Completion: [V] Ignoring priority of items older than AgeLimit of 6 hours
Tue Oct 20 21:47:52 2015	INFO	Completion: [V] get_nzb_status(nzb=[11738, 'C:\\\\ProgramData\\\\NZBGet\\\\nzb\\\\Stuff.720p.HDTV.X264-GROUP.nzb.queued', 1445389184, 897, u'', 0])
Tue Oct 20 21:47:52 2015	INFO	Completion: Checking: "C:\\ProgramData\\NZBGet\\nzb\\Stuff.720p.HDTV.X264-GROUP.nzb.queued"
Tue Oct 20 21:47:52 2015	INFO	Completion: [V] get_nzb_data(fname=C:\ProgramData\NZBGet/nzb\C:\\ProgramData\\NZBGet\\nzb\\Stuff.720p.HDTV.X264-GROUP.nzb.queued)
Tue Oct 20 21:47:52 2015	ERROR	Completion: no such nzb file
Tue Oct 20 21:47:52 2015	WARNING	Completion: [V] No such NZB file, resuming NZB
Tue Oct 20 21:47:52 2015	INFO	Completion: [V] nzbget_resume()

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Posted: 21 Oct 2015, 16:11
by kloaknet
Ok, apparently there is something wrong in the code for *nix (OS) users. The issue is shown here:

Code: Select all

Tue Oct 20 21:47:52 2015   INFO   Completion: [V] get_nzb_data(fname=C:\ProgramData\NZBGet/nzb\C:\\ProgramData\\NZBGet\\nzb\\Stuff.720p.HDTV.X264-GROUP.nzb.queued
and this part shouldn't be in it: "C:\ProgramData\NZBGet/nzb\"

a bit odd that the last \ is not a /, as (I think) I specifically ask in the code for the OS related separator :D. So make sure you got the PP-script Version: 0.2.3.

Probably the fastest solution for you if you are using 0.2.3:

open the Completion.py file in something like notepad / notepadd++, and go to line 375:

rar_msg_ids = get_nzb_data(os.environ['NZBOP_NZBDIR'] + os.sep + nzb[1]) and remove the red part.

the same should probably be done hat line 474/475
rar_msg_ids = get_nzb_data(os.environ['NZBOP_NZBDIR'] + os.sep +
nzb_filename)

I expect it should be working after these changes, but got no option to test it atm, and limited time until next week. First try to only change the first part, and if problems happen further in the checking, and it halts after the info message: Checking DUPE: and some file location with strange slashes and double C:... etc.

hope it works, and otherwise, please report back, and I rebuild my *nix device for further testing

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Posted: 22 Oct 2015, 03:43
by binreader
Hi made the edits you suggested i no longer get error message but now it seem the script just pauses nzbget permanently. not sure why it wants to do that i am using latest version of script. and i dont know nothing about no nix i dont even follow the sport....

here is what happens, the scheduler task is setup for every 10 minutes for testing purpose but seems like stuff gets stuck.

Code: Select all

Wed Oct 21 23:25:00 2015	INFO	Executing scan-script Completion.py for SomeNZB.nzb
Wed Oct 21 23:25:01 2015	INFO	Completion: [V] Expected queued file name: "C:\ProgramData\NZBGet\nzb\SomeNZB.nzb.queued"
Wed Oct 21 23:25:01 2015	INFO	Completion: [V] Pausing: "SomeNZB.nzb"
Wed Oct 21 23:25:02 2015	INFO	Adding collection SomeNZB.nzb to queue
Wed Oct 21 23:25:02 2015	INFO	Collection SomeNZB added to queue
Wed Oct 21 23:25:02 2015	INFO	Executing queue-script Completion.py for SomeNZB
Wed Oct 21 23:25:02 2015	INFO	Completion: [V] lock_file()
Wed Oct 21 23:25:02 2015	INFO	Completion: [V] Old completion.lock file overwritten
Wed Oct 21 23:25:02 2015	INFO	Completion: [V] Found paused item in queue: "C:\\ProgramData\\NZBGet\\nzb\\SomeNZB.nzb.queued"
Wed Oct 21 23:25:02 2015	INFO	Completion: [V] nzbget_paused()
Wed Oct 21 23:25:02 2015	INFO	Completion: [V] Waiting for NZBGet to end downloading
Wed Oct 21 23:25:02 2015	INFO	Completion: [V] Downloading for NZBGet paused
Wed Oct 21 23:30:00 2015	INFO	Executing scheduler-script Completion.py for Task1
Wed Oct 21 23:30:00 2015	INFO	Completion: [V] lock_file()
Wed Oct 21 23:30:00 2015	INFO	Completion: [V] Script is already running, check canceled

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Posted: 22 Oct 2015, 14:03
by kloaknet
I think its just a minor issue now :) , please reload NZBget via Settings -> System -> Reload.

The lock file is created to avoid running the device twice, but after a failure the file is not deleted, after a restart it will be deleted.

Edit: I have second thoughts if my above answer is the solution..., because your script shows already it overwrites the old lock file, and it just totally stops after printing the pausing message. (and also no error messages) But this is rather odd, as nothing special happens between pausing NZBget and running starting the check, it should print message before the nzb is looked for, and a possible file location would may create an error.

maybe it will work when toggling PRIORITIZE option off (as I currently have), but thats all I can help you with until next week :oops: