[SQS-Script] Completion-Propagation/DMCA/Retention check

Share your scripts or request scripts with specific features.
Forum rules
Please keep the forum clean - one topic per script. Questions not related to a specific script should be posted in Support forum.
Crowley
Posts: 37
Joined: 22 Nov 2016, 09:43

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by Crowley » 29 Dec 2016, 12:30

Yeah, this version seems to be working again. I will report if I encounter any more problems.

dagoob
Posts: 21
Joined: 15 Jan 2016, 23:27

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by dagoob » 29 Dec 2016, 21:39

034 is looking really good in my testing so far, both on "problematic" things and very, very fresh downloads. Thanks kloaknet! :D

kloaknet
Posts: 337
Joined: 23 Jul 2014, 08:52

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by kloaknet » 02 Jan 2017, 15:07

- IPv6 support thanks to code provided by dagoob
- minor change on messages
- delay duration to avoid looping now based on number of sockets, 200 ms/ 'num of connections', max 5 consecutive loops on each socket, 2 s wait after 5 loops without result, and this version contains some minor test spamming in verbose mode to show how many loops the script made.

Please report some data like:

~20 loop messages on 100 articles on 6 connections seems reasonable, overall check in less than 2 sec.

when u have had some downloads, please report some results too.
Last edited by kloaknet on 21 Jan 2017, 11:43, edited 1 time in total.

dagoob
Posts: 21
Joined: 15 Jan 2016, 23:27

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by dagoob » 11 Jan 2017, 17:06

It's been working really well for me. Most of my loop messages are showing one loop (less than 10% hit the 5th loop).

kloaknet
Posts: 337
Joined: 23 Jul 2014, 08:52

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by kloaknet » 12 Jan 2017, 15:02

After almost 2 years a full update of the script, that fixes a lot of bugs and issues. The latest version of the script can be downloaded on the first page of this topic: http://forum.nzbget.net/viewtopic.php?f=8&t=1736

The script has become significantly faster when using multiple servers, now has IPv6 support, has limits on min and max amount of articles to be checked, and supports the upcoming simplification in NZBGet 18+ for adding extension scripts. For the full log of changes, check the end of the first post in this topic.

Note that this version has some minor update on the IPv6 support compared to the last version in the 0.3.x test series.

Crowley
Posts: 37
Joined: 22 Nov 2016, 09:43

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by Crowley » 26 Jan 2017, 21:40

Just wanted to report that script has been working really smoothly. Thanks again!

Nukien
Posts: 2
Joined: 05 Feb 2017, 18:05

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by Nukien » 05 Feb 2017, 18:21

I'm seeing some similar problems - the script is failing with supernews.com and thecubenet.com, but seems to be working with xsusenet.com

Code: Select all

WARNING Completion: [V] Socket: 2 news.supernews.com, NNTP reply: ['999', 'request', 'marked', 'as', 'failed', 'by', 'script.']
 INFO    Completion: Exception LINE: 674: 'NoneType' object has no attribute 'send'
 INFO    Completion: [V] Socket: 3 <type 'exceptions.AttributeError'> 'NoneType' object has no attribute 'recv'
 INFO    Completion: [V] Socket: 3 Still no data received after waiting for 1 sec,  marking request as failed.
 WARNING Completion: [V] Socket: 3 news.supernews.com, NNTP reply: ['999', 'request', 'marked', 'as', 'failed', 'by', 'script.']
 INFO    Completion: Exception LINE: 674: 'NoneType' object has no attribute 'send'
 INFO    Completion: [V] Socket: 4 <type 'exceptions.AttributeError'> 'NoneType' object has no attribute 'recv'
 INFO    Completion: [V] Socket: 4 Still no data received after waiting for 1 sec,  marking request as failed.

 WARNING Completion: [V] Socket: 1 secure.news.thecubenet.com, NNTP reply: ['999', 'Article', 'marked', 'as', 'failed', 'by', 'script.']
 INFO    Completion: [V] Socket: 2 <type 'exceptions.AttributeError'> 'NoneType' object has no attribute 'recv'
 INFO    Completion: [V] Socket: 2 Still no data received after waiting for 1 sec,  marking requested article as failed.
 WARNING Completion: [V] Socket: 2 secure.news.thecubenet.com, NNTP reply: ['999', 'Article', 'marked', 'as', 'failed', 'by', 'script.']
 INFO    Completion: [V] Socket: 3 <type 'exceptions.AttributeError'> 'NoneType' object has no attribute 'recv'
 INFO    Completion: [V] Socket: 3 Still no data received after waiting for 1 sec,  marking requested article as failed.

 WARNING Completion: [V] Socket: 2 free.xsusenet.com, NNTP reply: ['430', 'No', 'Such', 'Article']
 WARNING Completion: [V] Socket: 1 free.xsusenet.com, NNTP reply: ['430', 'No', 'Such', 'Article']
 WARNING Completion: [V] Socket: 0 free.xsusenet.com, NNTP reply: ['430', 'No', 'Such', 'Article']
I've had to disable Completion for now :(

kloaknet
Posts: 337
Joined: 23 Jul 2014, 08:52

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by kloaknet » 05 Feb 2017, 21:10

Thanks for your reply Nukien, to be able to figure out the issue, please provide the log from the start call of the script up to the failures by sending me a PM. Note that the VERBOSE logs prints your newsserver pass and user account data several times, so please replace those with something else.

Nukien
Posts: 2
Joined: 05 Feb 2017, 18:05

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by Nukien » 05 Feb 2017, 23:02

OK, going to set logging to VERBOSE and trigger the issue with a known bad nzb - that is, one that was failed/paused by Completion.py

kloaknet
Posts: 337
Joined: 23 Jul 2014, 08:52

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by kloaknet » 06 Feb 2017, 13:33

The above seems to be related to slow replying newsservers. Maybe others can check the ping times to their news servers, and drop them below, especially when u don't have issues with the script?

my ping times to my servers are below 50 ms, so for me a max delay between each connection/socket check on if a reply is received (200 ms) is not causing issues, but maybe others have way higher ping times, and I should make the 200 ms adaptable?

Post Reply

Who is online

Users browsing this forum: No registered users and 34 guests