[SQS-Script] Completion-Propagation/DMCA/Retention check

Share your scripts or request scripts with specific features.
Forum rules
Please keep the forum clean - one topic per script. Questions not related to a specific script should be posted in Support forum.
barenaked
Posts: 10
Joined: 20 May 2017, 13:27

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by barenaked » 06 Jun 2017, 05:26

kloaknet wrote:
05 Jun 2017, 10:27
My best guess is that NZBget does not change the actual file name, but only the name within NZBget. That it works now, as u mentioned above, seems to support that. So no reordering of the scripts needed.
Seems like that because now I got another error from an unencrypted nzb set.

Code: Select all

WARNING	Tue Jun 06 2017 06:44:35	Completion: The NZB file American.Dad.S13E13.720p.HDTV.x264-AVS.nzb.queued'}, {u'Name': u'FAKEDETECTOR_SORTED', u'Value': u'yes does not seem to exist, resuming NZB.

kloaknet
Posts: 316
Joined: 23 Jul 2014, 08:52

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by kloaknet » 06 Jun 2017, 14:47

:cry: the } sign, that I used to split on, is a rather special character, and in general not used in file names etc. But since someone decided to use the } to identify passwords, it messes up the data I get from NZBget. This data is enclosed in the { } brackets, so in the general script it searches for the 1st } after the { to extract the data, resulting in errors for you.

As reported issues during the development of the script has shown, more and more strange signs are used, messing up data handling. The wait is now people starting to use < > in file names :oops:

I don't know how to extract the data differently from the data I get from NZBget, that looks like:

Code: Select all

...}{u'CnpNZBFileName', u'Value': u'American.Dad.S13E13.720p.HDTV.x264-AVS.nzb.queued'}, {u'Name': u'FAKEDETECTOR_SORTED', u'Value': u'yes}, {...
as I cannot get the data stored for CnpNZBFileName back the way I want, so I use

Code: Select all

    s = str(parameters)
    loc = s.rfind("u'CnpNZBFileName', u'Value': u")  ## "/'
    s = s[loc+31:]
    loc = s.find('}')
    return s[:loc-1]
Maybe someone, maybe Hugbug, knows to get this sorted in a smart way, without using the python find('}') option??

barenaked
Posts: 10
Joined: 20 May 2017, 13:27

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by barenaked » 06 Jun 2017, 16:13

OK I'll revert the code change and try setting completion as the first script, we'll see what happens ;) I'll report back after I've tried both ways.

hugbug
Developer & Admin
Posts: 6710
Joined: 09 Sep 2008, 11:58
Location: Germany (NRW)

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by hugbug » 06 Jun 2017, 16:24

kloaknet wrote:
06 Jun 2017, 14:47

Code: Select all

    s = str(parameters)
    loc = s.rfind("u'CnpNZBFileName', u'Value': u")  ## "/'
Can't you iterate through parameters properties instead of converting it to text? What type is parameters?

kloaknet
Posts: 316
Joined: 23 Jul 2014, 08:52

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by kloaknet » 07 Jun 2017, 15:57

Thanks for trying to help hugbug,

first I ask for the 'listgroup' data from NZBget, and if I must believe the part of the code I once copied from other scripts, the data returned is a JSON-raw string.

This data looks like:

Code: Select all

{
"version" : "1.1",
"result" : [
{
"FirstID" : 432,
"LastID" : 432,
...
"ActiveDownloads" : 0,
"Status" : "PAUSED",
"NZBID" : 432,
"NZBName" : "The.Challenge.S30E04.HDTV.x264-YesTV",
"NZBNicename" : "The.Challenge.S30E04.HDTV.x264-YesTV",
...
"ExtraParBlocks" : 0,
"Parameters" : [...,
{
"Name" : "VideoSort\\VideoSort.py:",
"Value" : "yes"
},
{
"Name" : "Completion.py:",
"Value" : "yes"
},
{
"Name" : "CnpNZBFileName",
"Value" : "The.Challenge.S30E04.HDTV.x264-YesTV.nzb.queued"
}],....
So then I use this code

Code: Select all

jobs = json.loads(data)
So I can use

Code: Select all

 for job in jobs['result']:
to get the specific job in the queue I want do something with, like using

Code: Select all

job['DupeKey'], job['MaxPostTime'], or job['Parameters'] 
But when getting the data from job['Parameters'], I can't narrow it down to get the data from the 'CnpNZBFileName' thingy (the actual file name)

Code: Select all

, u'ExParStatus': u'NONE', u'Parameters': [{...}, {u'Name': u'*Unpack:', u'Value': u'yes'}, {u'Name': u'Subliminal.py:', u'Value': u'yes'}, {u'Name': u'VideoSort\\VideoSort.py:', u'Value': u'yes'}, {u'Name': u'Completion.py:', u'Value': u'yes'}, {u'Name': u'CnpNZBFileName', u'Value': u'The.Challenge.S30E04.HDTV.x264-YesTV.nzb.2.queued'}], ...
So to still be able to extract it (as I couldnt figure / google out how to get the data), I used the find option to get the data. So basically the above is just a string, in where I search to match the location of the filename:

Code: Select all

    s = str(parameters)
    loc = s.rfind("u'CnpNZBFileName', u'Value': u")  ## "/'
    s = s[loc+31:]
    loc = s.rfind('}')
    return s[:loc-1]
where parameters contains the job['Parameters'] data

hugbug
Developer & Admin
Posts: 6710
Joined: 09 Sep 2008, 11:58
Location: Germany (NRW)

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by hugbug » 07 Jun 2017, 16:39

u'Parameters': [{...}, {u'Name': u'*Unpack:', u'Value': u'yes'},
That looks like array of dictionaries. Wouldn't something like this work (not tested)?

Code: Select all

for param in job['Parameters']:
    if param['Name'] == 'CnpNZBFileName':
        print(param['Value'])

kloaknet
Posts: 316
Joined: 23 Jul 2014, 08:52

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by kloaknet » 08 Jun 2017, 14:24

Thanks hugbug, it worked

@barenaked:

you could try to replace the code within 'def get_nzb_filename(parameters)' with this:

Code: Select all

def get_nzb_filename(parameters):
    '''
        get the real nzb_filename from the added parameter CnpNZBFileName
        '''
    for p in parameters:
        if p['Name'] == 'CnpNZBFileName':
            break
    return p['Value']
and it will work for all odd signs in the filename

hugbug
Developer & Admin
Posts: 6710
Joined: 09 Sep 2008, 11:58
Location: Germany (NRW)

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by hugbug » 15 Jun 2017, 12:54

Since v17.0 nzbget passes new env var to queue and pp-scripts:
  • NZBNA_QUEUEDFILE/NZBPP_QUEUEDFILE - full filename of the queued (renamed) nzb-file.
Maybe that can be useful for your script.

kloaknet
Posts: 316
Joined: 23 Jul 2014, 08:52

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by kloaknet » 15 Jun 2017, 13:55

hugbug wrote:
15 Jun 2017, 12:54
Since v17.0 nzbget passes new env var to queue and pp-scripts:
  • NZBNA_QUEUEDFILE/NZBPP_QUEUEDFILE - full filename of the queued (renamed) nzb-file.
Maybe that can be useful for your script.
Ill add it on the todo list, but for now I use the workaround, otherwise I would have to force everyone to update to v17, what might not be possible for some atm. Guess I will implement it and clear away the workaround code when NZBget 20 arrives!

barenaked
Posts: 10
Joined: 20 May 2017, 13:27

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Post by barenaked » 17 Jun 2017, 19:31

kloaknet wrote:
08 Jun 2017, 14:24
Thanks hugbug, it worked

@barenaked:

you could try to replace the code within 'def get_nzb_filename(parameters)' with this:

Code: Select all

def get_nzb_filename(parameters):
    '''
        get the real nzb_filename from the added parameter CnpNZBFileName
        '''
    for p in parameters:
        if p['Name'] == 'CnpNZBFileName':
            break
    return p['Value']
and it will work for all odd signs in the filename
Awesome, just tried your code replacing the existing code and no more warnings or errors!


Last issue is the 15 minute scan even though nzbget is paused and/or the queue is empty. It's keeping my system awake for no apparent reason because either nzbget should rest (= paused queue) or the queue is empty so there is nothing to check. Would it be possible to let the 15 minute scan depend on whether the queue is even having items in it or if nzbget is in download mode?

Post Reply

Who is online

Users browsing this forum: No registered users and 3 guests