[SQS-Script] Completion-Propagation/DMCA/Retention check

Share your scripts or request scripts with specific features.
Forum rules
Please keep the forum clean - one topic per script. Questions not related to a specific script should be posted in Support forum.
kloaknet
Posts: 309
Joined: 23 Jul 2014, 08:52

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by kloaknet » 11 Sep 2017, 05:44

thanks for reporting the issue, please provide the complete log for the script, with the Verbose option of the script turned on. This part providing the error has not changed in a long while, and I cant reproduce. The lock file should contain an interger (timestamp) and it seems it does not.

Before you reload NZBGet, please check and copy the contents of the completion.lock file located in your NZBGet temp folder (can be located via PATHS in NZBGet).

If you still have the issue with Verbose on, please insert the following at line 1330:

Code: Select all

        if VERBOSE:
            print('[V] text from completion.lock file= ' 
                + str(fd.readline()))
so between

Code: Select all

        fd = open(f_name)
and

Code: Select all

        time_stamp = int(fd.readline())
and run again.

Feel free to pass the complete logs via PM, and remove possible passwords etc.

starkebn
Posts: 4
Joined: 17 Aug 2017, 06:44

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by starkebn » 11 Sep 2017, 13:30

http://i.imgur.com/OS71efQ.png

completion is checking two servers, but even if the second server has 0 failures it is keeping the percentage fail from the first server and refusing to run

kloaknet
Posts: 309
Joined: 23 Jul 2014, 08:52

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by kloaknet » 11 Sep 2017, 15:59

starkebn wrote:
11 Sep 2017, 13:30
http://i.imgur.com/OS71efQ.png

completion is checking two servers, but even if the second server has 0 failures it is keeping the percentage fail from the first server and refusing to run
Thanks for reporting the issue, it looks like something is going wrong while making the connections, as the checking is really fast, but the connection creation really slow.

Please turn on Verbose logging.

Then re-run the same NZB, (set it back to the queue, then pause nzb, and hit the test button on the script page, or wait 15 min to let it be picked up), and post the complete logs, or send them to me via PM. May check for passwords

kloaknet
Posts: 309
Joined: 23 Jul 2014, 08:52

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by kloaknet » 14 Sep 2017, 08:30

starkebn wrote:
11 Sep 2017, 13:30
http://i.imgur.com/OS71efQ.png

completion is checking two servers, but even if the second server has 0 failures it is keeping the percentage fail from the first server and refusing to run
Did some digging in the code, and it looks like the following is the issue:

after all STAT messages are requested from the newsserver, the Requested [64/64] articles is printed. After this, the requested status info still has to return. The failure of the youngest articles (the last ones to be checked) all fail is therefore not printed, and the last message "all requested replies received, XX failed" is not shown, probably due to some limitations I got due to having the handle all send/receive actions for the sockets, what is a bit tricky, and not covering all NNTP reply messages.

Guess it would benefit to port the code to python 3, to simplfy the communication with NNTP servers, and the work arounds to get SSL etc working.

kloaknet
Posts: 309
Joined: 23 Jul 2014, 08:52

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by kloaknet » 14 Sep 2017, 15:26

md500_pilot wrote:
10 Sep 2017, 19:22
I am running into an issue running the latest version of the script. There is some kind of error when the script runs that leaves everything in PAUSED mode. Here is the pastebin of the error:

https://pastebin.com/mW1wmbn9
solution to this issue is to replace

Code: Select all

time_stamp = int(fd.readline())
with this:

Code: Select all

time_stamp = int(float(fd.readline()))

note that the above issue causes barely any issues, this part is in the file for years, and never ran across it till today, so no update yet for the script. But if it happens more than once, replace the data above.

starkebn
Posts: 4
Joined: 17 Aug 2017, 06:44

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by starkebn » 16 Sep 2017, 13:04

kloaknet wrote:
14 Sep 2017, 08:30
starkebn wrote:
11 Sep 2017, 13:30
http://i.imgur.com/OS71efQ.png

completion is checking two servers, but even if the second server has 0 failures it is keeping the percentage fail from the first server and refusing to run
Did some digging in the code, and it looks like the following is the issue:

after all STAT messages are requested from the newsserver, the Requested [64/64] articles is printed. After this, the requested status info still has to return. The failure of the youngest articles (the last ones to be checked) all fail is therefore not printed, and the last message "all requested replies received, XX failed" is not shown, probably due to some limitations I got due to having the handle all send/receive actions for the sockets, what is a bit tricky, and not covering all NNTP reply messages.

Guess it would benefit to port the code to python 3, to simplfy the communication with NNTP servers, and the work arounds to get SSL etc working.
thanks for looking into it, I guess a fix might be a long way off if you have to rewrite the entire script. If it happens again it shouldn't be too much of a problem, I fixed it the first time by disabling the script for that download.

starkebn
Posts: 4
Joined: 17 Aug 2017, 06:44

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by starkebn » 19 Sep 2017, 11:59

Is it possible that, if there's nothing in the download queue, then Completion doesn't run? I guess it's not really important but there can be 100s of Completion messages in the history when there is nothing in the queue.

kloaknet
Posts: 309
Joined: 23 Jul 2014, 08:52

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by kloaknet » 20 Sep 2017, 05:00

starkebn wrote:
16 Sep 2017, 13:04
thanks for looking into it, I guess a fix might be a long way off if you have to rewrite the entire script. If it happens again it shouldn't be too much of a problem, I fixed it the first time by disabling the script for that download.
Its most likely only a message printing issue, so the articles werent there. But to be 100% sure I need Verbose logs if you spot it again. To avoid the issue, you could up the AgeLimit option to say 6 hours.
starkebn wrote:
19 Sep 2017, 11:59
Is it possible that, if there's nothing in the download queue, then Completion doesn't run? I guess it's not really important but there can be 100s of Completion messages in the history when there is nothing in the queue.
It would be cool to be able to do that, but the script only prints messages with "Completion: ....". The messages you are referring too are printed by NZBGet, when it starts the script based on the scheduler settings. After this start I can only determine what is printed.
These scheduler settings are hardcoded in the script for each 15 min (version NZBGet 18+). A double-ling of the interval each time the queue is emtpy would be cool, but I guess then I have to make my script to change its code itself each time the script has run, and that ain't easy or possible I guess. Probably antivirus or so would kick in to when a script changes itself?

it seems it is possible code wise: https://stackoverflow.com/questions/396 ... nge-itself. This would also require that NZBget reads the script prior to each check, so it updates the check interval each time the script runs, but I don't know if thats the case. I will look into it, but cant guarantee anything. :roll:

kloaknet
Posts: 309
Joined: 23 Jul 2014, 08:52

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by kloaknet » 20 Sep 2017, 15:20

I have done some testing on the above suggestion, it is not possible. Changing the check times within the script while it is running in NZBGet does not have an effect on the check times. NZBGet reads in the script (the changes) on a reload or on when someone clicks on the script page in settings, and not prior to the actual call/check on the scheduled times.

starkebn
Posts: 4
Joined: 17 Aug 2017, 06:44

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by starkebn » 21 Sep 2017, 10:13

Thanks for looking, it's no big deal. Like I said it's really only a cosmetic thing.

Post Reply

Who is online

Users browsing this forum: No registered users and 3 guests