Page 20 of 28

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Posted: 06 Jun 2017, 05:26
by barenaked
kloaknet wrote:
05 Jun 2017, 10:27
My best guess is that NZBget does not change the actual file name, but only the name within NZBget. That it works now, as u mentioned above, seems to support that. So no reordering of the scripts needed.
Seems like that because now I got another error from an unencrypted nzb set.

Code: Select all

WARNING	Tue Jun 06 2017 06:44:35	Completion: The NZB file American.Dad.S13E13.720p.HDTV.x264-AVS.nzb.queued'}, {u'Name': u'FAKEDETECTOR_SORTED', u'Value': u'yes does not seem to exist, resuming NZB.

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Posted: 06 Jun 2017, 14:47
by kloaknet
:cry: the } sign, that I used to split on, is a rather special character, and in general not used in file names etc. But since someone decided to use the } to identify passwords, it messes up the data I get from NZBget. This data is enclosed in the { } brackets, so in the general script it searches for the 1st } after the { to extract the data, resulting in errors for you.

As reported issues during the development of the script has shown, more and more strange signs are used, messing up data handling. The wait is now people starting to use < > in file names :oops:

I don't know how to extract the data differently from the data I get from NZBget, that looks like:

Code: Select all

...}{u'CnpNZBFileName', u'Value': u'American.Dad.S13E13.720p.HDTV.x264-AVS.nzb.queued'}, {u'Name': u'FAKEDETECTOR_SORTED', u'Value': u'yes}, {...
as I cannot get the data stored for CnpNZBFileName back the way I want, so I use

Code: Select all

    s = str(parameters)
    loc = s.rfind("u'CnpNZBFileName', u'Value': u")  ## "/'
    s = s[loc+31:]
    loc = s.find('}')
    return s[:loc-1]
Maybe someone, maybe Hugbug, knows to get this sorted in a smart way, without using the python find('}') option??

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Posted: 06 Jun 2017, 16:13
by barenaked
OK I'll revert the code change and try setting completion as the first script, we'll see what happens ;) I'll report back after I've tried both ways.

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Posted: 06 Jun 2017, 16:24
by hugbug
kloaknet wrote:
06 Jun 2017, 14:47

Code: Select all

    s = str(parameters)
    loc = s.rfind("u'CnpNZBFileName', u'Value': u")  ## "/'
Can't you iterate through parameters properties instead of converting it to text? What type is parameters?

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Posted: 07 Jun 2017, 15:57
by kloaknet
Thanks for trying to help hugbug,

first I ask for the 'listgroup' data from NZBget, and if I must believe the part of the code I once copied from other scripts, the data returned is a JSON-raw string.

This data looks like:

Code: Select all

{
"version" : "1.1",
"result" : [
{
"FirstID" : 432,
"LastID" : 432,
...
"ActiveDownloads" : 0,
"Status" : "PAUSED",
"NZBID" : 432,
"NZBName" : "The.Challenge.S30E04.HDTV.x264-YesTV",
"NZBNicename" : "The.Challenge.S30E04.HDTV.x264-YesTV",
...
"ExtraParBlocks" : 0,
"Parameters" : [...,
{
"Name" : "VideoSort\\VideoSort.py:",
"Value" : "yes"
},
{
"Name" : "Completion.py:",
"Value" : "yes"
},
{
"Name" : "CnpNZBFileName",
"Value" : "The.Challenge.S30E04.HDTV.x264-YesTV.nzb.queued"
}],....
So then I use this code

Code: Select all

jobs = json.loads(data)
So I can use

Code: Select all

 for job in jobs['result']:
to get the specific job in the queue I want do something with, like using

Code: Select all

job['DupeKey'], job['MaxPostTime'], or job['Parameters'] 
But when getting the data from job['Parameters'], I can't narrow it down to get the data from the 'CnpNZBFileName' thingy (the actual file name)

Code: Select all

, u'ExParStatus': u'NONE', u'Parameters': [{...}, {u'Name': u'*Unpack:', u'Value': u'yes'}, {u'Name': u'Subliminal.py:', u'Value': u'yes'}, {u'Name': u'VideoSort\\VideoSort.py:', u'Value': u'yes'}, {u'Name': u'Completion.py:', u'Value': u'yes'}, {u'Name': u'CnpNZBFileName', u'Value': u'The.Challenge.S30E04.HDTV.x264-YesTV.nzb.2.queued'}], ...
So to still be able to extract it (as I couldnt figure / google out how to get the data), I used the find option to get the data. So basically the above is just a string, in where I search to match the location of the filename:

Code: Select all

    s = str(parameters)
    loc = s.rfind("u'CnpNZBFileName', u'Value': u")  ## "/'
    s = s[loc+31:]
    loc = s.rfind('}')
    return s[:loc-1]
where parameters contains the job['Parameters'] data

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Posted: 07 Jun 2017, 16:39
by hugbug
u'Parameters': [{...}, {u'Name': u'*Unpack:', u'Value': u'yes'},
That looks like array of dictionaries. Wouldn't something like this work (not tested)?

Code: Select all

for param in job['Parameters']:
    if param['Name'] == 'CnpNZBFileName':
        print(param['Value'])

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Posted: 08 Jun 2017, 14:24
by kloaknet
Thanks hugbug, it worked

@barenaked:

you could try to replace the code within 'def get_nzb_filename(parameters)' with this:

Code: Select all

def get_nzb_filename(parameters):
    '''
        get the real nzb_filename from the added parameter CnpNZBFileName
        '''
    for p in parameters:
        if p['Name'] == 'CnpNZBFileName':
            break
    return p['Value']
and it will work for all odd signs in the filename

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Posted: 15 Jun 2017, 12:54
by hugbug
Since v17.0 nzbget passes new env var to queue and pp-scripts:
  • NZBNA_QUEUEDFILE/NZBPP_QUEUEDFILE - full filename of the queued (renamed) nzb-file.
Maybe that can be useful for your script.

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Posted: 15 Jun 2017, 13:55
by kloaknet
hugbug wrote:
15 Jun 2017, 12:54
Since v17.0 nzbget passes new env var to queue and pp-scripts:
  • NZBNA_QUEUEDFILE/NZBPP_QUEUEDFILE - full filename of the queued (renamed) nzb-file.
Maybe that can be useful for your script.
Ill add it on the todo list, but for now I use the workaround, otherwise I would have to force everyone to update to v17, what might not be possible for some atm. Guess I will implement it and clear away the workaround code when NZBget 20 arrives!

Re: [SQS-Script] Completion-Propagation/DMCA/Retention check

Posted: 17 Jun 2017, 19:31
by barenaked
kloaknet wrote:
08 Jun 2017, 14:24
Thanks hugbug, it worked

@barenaked:

you could try to replace the code within 'def get_nzb_filename(parameters)' with this:

Code: Select all

def get_nzb_filename(parameters):
    '''
        get the real nzb_filename from the added parameter CnpNZBFileName
        '''
    for p in parameters:
        if p['Name'] == 'CnpNZBFileName':
            break
    return p['Value']
and it will work for all odd signs in the filename
Awesome, just tried your code replacing the existing code and no more warnings or errors!


Last issue is the 15 minute scan even though nzbget is paused and/or the queue is empty. It's keeping my system awake for no apparent reason because either nzbget should rest (= paused queue) or the queue is empty so there is nothing to check. Would it be possible to let the 15 minute scan depend on whether the queue is even having items in it or if nzbget is in download mode?