[SQS-Script] Completion-Propagation/DMCA/Retention check

Share your scripts or request scripts with specific features.
Forum rules
Please keep the forum clean - one topic per script. Questions not related to a specific script should be posted in Support forum.
enormoz
Posts: 1
Joined: 21 Mar 2017, 21:11

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by enormoz » 14 Jan 2020, 20:42

Could you please add your script to a github repo and/or if possible update it for python3 as python2 is now EOL ?

[EDIT]
After running 2to3 this helped to get it running for me. Not ideal but seems to have worked.
189,193c189,193
< if py_ver < (2,7,9) or py_ver > (3,):
< print(('Python version: ' + sys.version))
< print(('[ERROR] This script requires python 2.7.9+, and does not work for ' +
< 'python 3, stopping'))
< sys.exit(1)
---
> #if py_ver < (2,7,9) or py_ver > (3,):
> # print(('Python version: ' + sys.version))
> # print(('[ERROR] This script requires python 2.7.9+, and does not work for ' +
> # 'python 3, stopping'))
> # sys.exit(1)
351,352c351,352
< base_64_string = base64.encodestring('%s:%s' % (USERNAME,
< PASSWORD)).replace('\n','')
---
> base_64_string = base64.encodestring(('%s:%s' % (USERNAME,
> PASSWORD)).encode()).decode().strip()
587c587
< server_reply = t[:3] # only first 3 chars are relevant
---
> server_reply = t[:3].decode("utf-8") # only first 3 chars are relevant
614c614
< sock.send(text)
---
> sock.send(text.encode())
637c637
< sock.send(text)
---
> sock.send(text.encode())
645c645
< sock.send(text)
---
> sock.send(text.encode())
661,662c661,662
< sock.send(text)
< elif server_reply[:2] in ('48', '50'):
---
> sock.send(text.encode())
> elif str(server_reply[:2]) in ('48', '50'):
693c693
< sock.send(text)
---
> sock.send(text.encode())
696c696
< sock.send(text)
---
> sock.send(text.encode())
705c705,706
< str(traceback.tb_lineno(sys.exc_info()[2])) + ': ' +
---
> traceback.format_exc() + ': ' +
> #str(traceback.tb_lineno(sys.exc_info()[2])) + ': ' +
1032c1033,1034
< str(traceback.tb_lineno(sys.exc_info()[2])) + ': ' +
---
> traceback.format_exc() + ': ' +
> #str(traceback.tb_lineno(sys.exc_info()[2])) + ': ' +
[EDIT2]
Completion_p3.py.zip
Python3 version of the script
(19.33 KiB) Downloaded 946 times
NO SUPPORT WILL BE PROVIDED !!

kloaknet
Posts: 337
Joined: 23 Jul 2014, 08:52

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by kloaknet » 24 Jan 2020, 17:31

ObaDobanan wrote:
02 Nov 2019, 13:47
Edit: I found the problem: The servers are sorted by first by their group, then by their level. I had my main server in group 1, but the backup server had no group, so it was checked first. What is the reasoning behind sorting by group, first?
Guess this is related to how NZBGet works, maybe check out the wiki pages of NZBGet, and then the info in the script itself.

kloaknet
Posts: 337
Joined: 23 Jul 2014, 08:52

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by kloaknet » 24 Jan 2020, 17:33

teh_niemer wrote:
04 Dec 2019, 13:37
I've come across an error in the script that may be an easy fix.

This is the error:

Code: Select all

Completion: UnicodeEncodeError: 'ascii' codec can't encode character u'\xf1' in position 18: ordinal not in range(128)
I believe this is the file it's stuck on

Code: Select all

Jack Peñate - After You (2019) FLAC
And I believe that this may be the solution
https://stackoverflow.com/questions/352 ... 18-ordinal

any chance this could get fixed/updated?
I am not gonna fix it, and for music avoid using this script as it won't save you bandwidth when its posted file by file etc. It will do more harm than good. Furthermore, posters should know the limitations of file naming of the old usenet ;)

kloaknet
Posts: 337
Joined: 23 Jul 2014, 08:52

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by kloaknet » 24 Jan 2020, 17:54

enormoz wrote:
14 Jan 2020, 20:42
Could you please add your script to a github repo and/or if possible update it for python3 as python2 is now EOL ?

[EDIT2]
Completion_p3.py.zip NO SUPPORT WILL BE PROVIDED !!
Nice that you made some tweaks to it for py3, always written it with the intention to port, but no time. Maybe others should let it here know if it works without hickups.

I am not gonna put it on github, as I dont understand how to use it :oops: . Also got no time to work on the script to convert it proper etc etc for a while now so dont expect an update anytime soon in the far future,

py3 is way better than py2 for this script, as it has https://docs.python.org/3/library/nntplib.html :). To use all features, and get rid of the massive buggy ans slow workaround in the py2 script, its a lot of work to get it to work again the proper way.

There are a lot of things involved in the code, and although you see ppl complain about the script oddly checking completion status, its main purpose has been always to check for propagation issues, and try again after a bit if the propagation completed, without just grabbing a next just posted nzb that fails because its not propagated either, while working within the limits of NZBget. So its not the simple way like SABnzbd did or maybe still does.

In short its a structured complicated mess of code, but anybody is free to put it on gitbub, after making it way better. I got some py3 code sort of working but thats for 2 years now, and still without any nzbget integration :? so not suitable for someone without NZBget integration skills. Feel free to PM me, but I am not often around anymore.

teh_niemer
Posts: 2
Joined: 04 Dec 2019, 13:25

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by teh_niemer » 23 Feb 2020, 14:01

I'm getting some errors with the latest version of the script, see below.

Code: Select all

Sun Feb 23 07:15:00 2020	INFO	Executing scheduler-script Completion.py for Task1
Sun Feb 23 07:15:00 2020	INFO	Executing scheduler-script Completion.py
Sun Feb 23 07:15:00 2020	ERROR	Completion: Script seems to be running for more than 30 minutes and has most likely crashed. Check your logs and report the log and errors at http://forum.NZBGet.net/viewtopic.php?f=8&t=1736
Sun Feb 23 07:15:00 2020	ERROR	Completion: Script seems to be running for more than 30 minutes and has most likely crashed. Check your logs and report the log and errors at http://forum.NZBGet.net/viewtopic.php?f=8&t=1736
Sun Feb 23 07:15:01 2020	INFO	Completion: Traceback (most recent call last):
Sun Feb 23 07:15:01 2020	INFO	Completion:   File "/config/scripts/Completion.py", line 1702, in <module>
Sun Feb 23 07:15:01 2020	INFO	Completion:     main()
Sun Feb 23 07:15:01 2020	INFO	Completion:   File "/config/scripts/Completion.py", line 1676, in main
Sun Feb 23 07:15:01 2020	INFO	Completion: Traceback (most recent call last):
Sun Feb 23 07:15:01 2020	INFO	Completion:   File "/config/scripts/Completion.py", line 1702, in <module>
Sun Feb 23 07:15:01 2020	INFO	Completion:     if 'NZBSP_TASKID' in os.environ: scheduler_call()
Sun Feb 23 07:15:01 2020	INFO	Completion:   File "/config/scripts/Completion.py", line 1554, in scheduler_call
Sun Feb 23 07:15:01 2020	INFO	Completion:     get_prio_nzb(jobs['result'], paused_jobs)
Sun Feb 23 07:15:01 2020	INFO	Completion:   File "/config/scripts/Completion.py", line 1526, in get_prio_nzb
Sun Feb 23 07:15:01 2020	INFO	Completion:     main()
Sun Feb 23 07:15:01 2020	INFO	Completion:   File "/config/scripts/Completion.py", line 1676, in main
Sun Feb 23 07:15:01 2020	INFO	Completion:     if get_nzb_status(nzb):
Sun Feb 23 07:15:01 2020	INFO	Completion:   File "/config/scripts/Completion.py", line 378, in get_nzb_status
Sun Feb 23 07:15:01 2020	INFO	Completion:     print('Checking: "' + nzb[1] + '"')
Sun Feb 23 07:15:01 2020	INFO	Completion: UnicodeEncodeError: 'ascii' codec can't encode characters in position 41-44: ordinal not in range(128)
Sun Feb 23 07:15:01 2020	INFO	Completion:     if 'NZBSP_TASKID' in os.environ: scheduler_call()
Sun Feb 23 07:15:01 2020	INFO	Completion:   File "/config/scripts/Completion.py", line 1554, in scheduler_call
Sun Feb 23 07:15:01 2020	INFO	Completion:     get_prio_nzb(jobs['result'], paused_jobs)
Sun Feb 23 07:15:01 2020	INFO	Completion:   File "/config/scripts/Completion.py", line 1526, in get_prio_nzb
Sun Feb 23 07:15:01 2020	INFO	Completion:     if get_nzb_status(nzb):
Sun Feb 23 07:15:01 2020	INFO	Completion:   File "/config/scripts/Completion.py", line 378, in get_nzb_status
Sun Feb 23 07:15:01 2020	INFO	Completion:     print('Checking: "' + nzb[1] + '"')
Sun Feb 23 07:15:01 2020	INFO	Completion: UnicodeEncodeError: 'ascii' codec can't encode characters in position 41-44: ordinal not in range(128)

borishellen
Posts: 2
Joined: 06 Feb 2016, 18:53

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by borishellen » 07 Sep 2020, 14:24

I found a bug where the script would fail with these errors below.

Code: Select all

ValueError: invalid literal for int() with base 10: ''
Removing the lockfile didn't fix it, the lock just contained a UNIX timestamp and was an integer. The problem was one of my servers had no entry for the group number. When I changed that to a 0, the script worked again.

iaTa
Posts: 5
Joined: 09 Mar 2019, 18:21

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by iaTa » 19 Apr 2021, 14:58

Updated the script for Python 3.9.

EDIT: I'm still having issues with this on Python 3.9.
Attachments
Completion_p3.9.py.zip
(18.33 KiB) Downloaded 313 times
Last edited by iaTa on 26 May 2021, 01:43, edited 1 time in total.

a3gill
Posts: 2
Joined: 17 Oct 2020, 09:48

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by a3gill » 21 Apr 2021, 08:59

Been waiting on this. I'd toyed around with it a bit in the past, but always had one issue or another...

Much appreciated! So far, so good.

Crowley
Posts: 37
Joined: 22 Nov 2016, 09:43

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by Crowley » 31 Aug 2021, 08:42

I tried that Python 3.9 version in Ubuntu, but it doesn't work too well, often downloads get stuck in paused state with this in the log:
Completion: UnboundLocalError: local variable 'server_reply' referenced before assignment
Completion: return (False, False, server_reply, -1)
Completion: File "/opt/nzbget/scripts/Completion.py", line 708, in check_send_server_reply
Completion: check_send_server_reply(sockets, reply, group, id,
Completion: File "/opt/nzbget/scripts/Completion.py", line 1189, in check_failure_status
Completion: failed_ratio = check_failure_status(rar_msg_ids, failed_limit, nzb[2])
Completion: File "/opt/nzbget/scripts/Completion.py", line 403, in get_nzb_status
Completion: if get_nzb_status(nzb):
Completion: File "/opt/nzbget/scripts/Completion.py", line 1528, in get_prio_nzb
Completion: get_prio_nzb(jobs['result'], paused_jobs)
Completion: File "/opt/nzbget/scripts/Completion.py", line 1594, in queue_call
Completion: if 'NZBNA_NZBNAME' in os.environ: queue_call()
Completion: File "/opt/nzbget/scripts/Completion.py", line 1680, in main
Completion: main()
Completion: File "/opt/nzbget/scripts/Completion.py", line 1704, in <module>
Completion: Traceback (most recent call last):
Completion: During handling of the above exception, another exception occurred:
Completion: AttributeError: 'str' object has no attribute 'decode'
Completion: server_reply = t[:3].decode("utf-8") # only first 3 chars are relevant
Completion: File "/opt/nzbget/scripts/Completion.py", line 587, in check_send_server_reply
Completion: Traceback (most recent call last):
Completion: : 'str' object has no attribute 'decode'
Completion: AttributeError: 'str' object has no attribute 'decode'
Completion: server_reply = t[:3].decode("utf-8") # only first 3 chars are relevant
Completion: File "/opt/nzbget/scripts/Completion.py", line 587, in check_send_server_reply
Completion: Exception LINE: Traceback (most recent call last):


Any tips? Should I use Shell override? I don't have python 2.7 anymore...

NamedUser
Posts: 2
Joined: 28 Sep 2021, 14:46

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by NamedUser » 28 Sep 2021, 15:31

Crowley wrote:
31 Aug 2021, 08:42
I tried that Python 3.9 version in Ubuntu, but it doesn't work too well, often downloads get stuck in paused state with this in the log:
Completion: UnboundLocalError: local variable 'server_reply' referenced before assignment
Completion: return (False, False, server_reply, -1)
Completion: File "/opt/nzbget/scripts/Completion.py", line 708, in check_send_server_reply
Completion: check_send_server_reply(sockets, reply, group, id,
Completion: File "/opt/nzbget/scripts/Completion.py", line 1189, in check_failure_status
Completion: failed_ratio = check_failure_status(rar_msg_ids, failed_limit, nzb[2])
Completion: File "/opt/nzbget/scripts/Completion.py", line 403, in get_nzb_status
Completion: if get_nzb_status(nzb):
Completion: File "/opt/nzbget/scripts/Completion.py", line 1528, in get_prio_nzb
Completion: get_prio_nzb(jobs['result'], paused_jobs)
Completion: File "/opt/nzbget/scripts/Completion.py", line 1594, in queue_call
Completion: if 'NZBNA_NZBNAME' in os.environ: queue_call()
Completion: File "/opt/nzbget/scripts/Completion.py", line 1680, in main
Completion: main()
Completion: File "/opt/nzbget/scripts/Completion.py", line 1704, in <module>
Completion: Traceback (most recent call last):
Completion: During handling of the above exception, another exception occurred:
Completion: AttributeError: 'str' object has no attribute 'decode'
Completion: server_reply = t[:3].decode("utf-8") # only first 3 chars are relevant
Completion: File "/opt/nzbget/scripts/Completion.py", line 587, in check_send_server_reply
Completion: Traceback (most recent call last):
Completion: : 'str' object has no attribute 'decode'
Completion: AttributeError: 'str' object has no attribute 'decode'
Completion: server_reply = t[:3].decode("utf-8") # only first 3 chars are relevant
Completion: File "/opt/nzbget/scripts/Completion.py", line 587, in check_send_server_reply
Completion: Exception LINE: Traceback (most recent call last):


Any tips? Should I use Shell override? I don't have python 2.7 anymore...


I've updated the script to run on python 3.9. I've tested it for like 2 weeks now and I haven't seen any errors. I only updated some variable types (string -> byte / byte -> string) as python 3 now makes a clear distinction between them. I'm not a programmer in any way neither have I coded in python before, I just read the docs and the stacktraces...

I've also uploaded the script to GitLab so anyone who wants to fork it or make an merge request can do it. https://gitlab.com/n701/nzbget-completi ... pletion.py
Attachments
Completion.py
(77.27 KiB) Downloaded 220 times

Post Reply

Who is online

Users browsing this forum: No registered users and 30 guests