[Queue-Script] PasswordDetector - stop dl of protected .rar

Share your scripts or request scripts with specific features.
Forum rules
Please keep the forum clean - one topic per script. Questions not related to a specific script should be posted in Support forum.
prinz2311
Posts: 466
Joined: 08 Dec 2012, 00:03

Re: [Queue-Script] PasswordDetector - stop dl of protected .

Post by prinz2311 » 22 Aug 2014, 11:40

The Problem with testing for "CRC" is that this doesn't need to be because of a password, this can simply be a corrupted (but repairable) rar File.

The message from unrar says for a reason:
Corrupt file or wrong password.
Only since the rar5 format you can definitely say it's password protected or not. That is a the problem with the old rar4 and older formats.

get39678
Posts: 222
Joined: 09 Jun 2014, 10:49

Re: [Queue-Script] PasswordDetector - stop dl of protected .

Post by get39678 » 22 Aug 2014, 13:47

prinz2311 wrote:The Problem with testing for "CRC" is that this doesn't need to be because of a password, this can simply be a corrupted (but repairable) rar File.

The message from unrar says for a reason:
Corrupt file or wrong password.
Only since the rar5 format you can definitely say it's password protected or not. That is a the problem with the old rar4 and older formats.
Agree this may be an issue. Could the impact be lessened with the following?

JVM what do you think about changing to the following -

remove check for CRC string
use err.find not out.find to check for the text "wrong password"
use "wrong password" not just "password" as more specific than just password on own

Code: Select all

                if verbose:
                   print(out)
                   print(err)
                if "*" in out:
                   return True
                if err.find("wrong password") != -1:
                   return True

JVM
Posts: 83
Joined: 21 Oct 2013, 05:11

Re: [Queue-Script] PasswordDetector - stop dl of protected .

Post by JVM » 22 Aug 2014, 15:33

Totally agree with you guys with "wrong password". Updated OP.

Very weird that it won't delete temp files, just now I made it spot a password, it marked bad and deleted temp file in post processing.

get39678
Posts: 222
Joined: 09 Jun 2014, 10:49

Re: [Queue-Script] PasswordDetector - stop dl of protected .

Post by get39678 » 22 Aug 2014, 19:29

I have been tinkering and added several new configurable options I have called v1.2 should I just upload for people to look at, criticise as I do not know what I am doing and probably fix the errors?!

JVM
Posts: 83
Joined: 21 Oct 2013, 05:11

Re: [Queue-Script] PasswordDetector - stop dl of protected .

Post by JVM » 23 Aug 2014, 02:14

I created a Github repository so you and anyone can contribute, which can be later merged. You can upload v1.2 there. Then it should be easy to compare one anothers test code before committing to master.

I'm new to Github but it looks like you can do it all from the browser if you want. So in short:
create an account if you need
head to the Github repository,
create a fork by clicking "Fork" (top right),
in your fork, click on the file to edit (PasswordDetector.py),
then there is a pencil icon above the file contents, click that and delete contents and paste code in there,
type in some comments under "Commit changes",
click "Commit Changes",
on the right click on a button "Pull requests" (has arrow pointing to a node under <> tab),
"New pull request",
"Create pull request",
type a comment,
"Create pull request",
and that's it.

get39678
Posts: 222
Joined: 09 Jun 2014, 10:49

Re: [Queue-Script] PasswordDetector - stop dl of protected .

Post by get39678 » 23 Aug 2014, 11:23

JVM are you happy to receive via the forum? I think everything I have added is working now apart from 0 byte temp files are still being created for some items but it has always done that for me.

JVM
Posts: 83
Joined: 21 Oct 2013, 05:11

Re: [Queue-Script] PasswordDetector - stop dl of protected .

Post by JVM » 23 Aug 2014, 13:47

Yeah sure post it here. Also the files should be deleted if it is being run as a post processing script, I made it so the first thing it does is check if it is called as a pp script so it can delete. Are you sure it's being run as a pp script? Might need to define it in the default category pp scripts as well.

get39678
Posts: 222
Joined: 09 Jun 2014, 10:49

Re: [Queue-Script] PasswordDetector - stop dl of protected .

Post by get39678 » 23 Aug 2014, 16:11

JVM wrote:Yeah sure post it here.
I have attached a proposed new version. As this is my first attempt please do not use until someone (e.g. jvm, hugbug) has looked it over. Many things that I thought would work didn't until I got use to tabs/spaces are important but hopefully it will work for others now. I had to click Save on the extensions options screen for things to work but again that may be just because I originally had bad whitespace whilst I was tinkering.
JVM wrote:Also the files should be deleted if it is being run as a post processing script, I made it so the first thing it does is check if it is called as a pp script so it can delete. Are you sure it's being run as a pp script? Might need to define it in the default category pp scripts as well.
Someone on the FakeDetector thread has reported similar issues. I have no categories defined and I have PasswordDectector.py configured as a PostProcess script only (NZBGet auto picks up its a Queue one also). I am going to try to work out how to reproduce if 1.2 is fine - still think it is something in the area of when NZBGet Pauses an item.
Attachments
PasswordDetector.py
Proposed new version 1.2
(14.36 KiB) Downloaded 257 times

JVM
Posts: 83
Joined: 21 Oct 2013, 05:11

Re: [Queue-Script] PasswordDetector - stop dl of protected .

Post by JVM » 24 Aug 2014, 03:26

get39678 wrote: I have attached a proposed new version.
I did some testing with your script with r1103 and seems to work as expected. Here's my log when FakeDetector correctly finds an exe (I manully triggered with a .exe inside a rar I compressed), marks nzb bad, PasswordDetector deletes temp file 203 as expected but FakeDetector does not:

Code: Select all

INFO	Sun Aug 24 2014 0:28:32	Post-process-script PasswordDetector.py for TEST_NZB successful
DETAIL	Sun Aug 24 2014 0:28:32	PasswordDetector: Completed removing temp file /mnt/usbhdd/NZBget/tmp/PasswordDetector/203
INFO	Sun Aug 24 2014 0:28:31	Executing post-process-script PasswordDetector.py for TEST_NZB
INFO	Sun Aug 24 2014 0:28:31	Post-process-script FakeDetector.py for TEST_NZB successful
WARNING	Sun Aug 24 2014 0:28:31	FakeDetector: Download has media files and executables
INFO	Sun Aug 24 2014 0:28:30	Executing post-process-script FakeDetector.py for TEST_NZB
DETAIL	Sun Aug 24 2014 0:28:12	FakeDetector: Detecting completed for TEST_NZB
WARNING	Sun Aug 24 2014 0:28:12	FakeDetector: Download has media files and executables
INFO	Sun Aug 24 2014 0:28:12	FakeDetector: Found executable password.exe
DETAIL	Sun Aug 24 2014 0:28:0	FakeDetector: Detecting fake for TEST_NZB
DETAIL	Sun Aug 24 2014 0:28:09	Executing queue-script FakeDetector.py for TEST_NZB
Then I marked the nzb good, removed the exe rar, put in a password proctected non exe rar, restarted the download again, Passworddetector correctly found the password rar and passed nzb,

Code: Select all

DETAIL	Sun Aug 24 2014 0:40:40	PasswordDetector: Detecting completed for TEST_NZB
DETAIL	Sun Aug 24 2014 0:40:40	PasswordDetector: Paused TEST_NZB
WARNING	Sun Aug 24 2014 0:40:40	PasswordDetector: Password found in TEST_NZB
INFO	Sun Aug 24 2014 0:40:40	PasswordDetector: out: UNRAR 5.0 beta 1 freeware Copyright (c) 1993-2014 Alexander RoshalArchive: /mnt/usbhdd/NZBget/inter/TEST_NZB.#203/PASSWORD_RAR.rarDetails: RAR 4, volume Attributes Size Date Time Name----------- --------- -------- ----- ----* ..A.... 729808896 09-08-14 15:15 lubuntu-14.04.1-desktop-i386.iso----------- --------- -------- ----- ---- 0 volume 5 0
INFO	Sun Aug 24 2014 0:40:40	PasswordDetector: command: ['unrar', 'l -p-', '/mnt/usbhdd/NZBget/inter/TEST_NZB.#203/PASSWORD_RAR.rar']
INFO	Sun Aug 24 2014 0:40:40	PasswordDetector: out: UNRAR 5.0 beta 1 freeware Copyright (c) 1993-2014 Alexander RoshalArchive: /mnt/usbhdd/NZBget/inter/TEST_NZB.#203/TEST_NZB.r03Details: RAR 4, volume Attributes Size Date Time Name----------- --------- -------- ----- ---- ..A.... 1000000000 23-08-14 19:53 TEST_NZB.mkv----------- --------- -------- ----- ---- 0 volume 5 0
INFO	Sun Aug 24 2014 0:40:40	PasswordDetector: command: ['unrar', 'l -p-', '/mnt/usbhdd/NZBget/inter/TEST_NZB.#203/TEST_NZB.r03']
DETAIL	Sun Aug 24 2014 0:40:40	PasswordDetector: Detecting password for TEST_NZB
DETAIL	Sun Aug 24 2014 0:40:37	Executing queue-script PasswordDetector.py for TEST_NZB
DETAIL	Sun Aug 24 2014 0:40:37	FakeDetector: Detecting completed for TEST_NZB
DETAIL	Sun Aug 24 2014 0:40:37	FakeDetector: Detecting fake for TEST_NZB
I then resumed, Password detector skipped detection from now on as expected,

Code: Select all

DETAIL	Sun Aug 24 2014 1:24:22	PasswordDetector: Password previously found, skipping detection
DETAIL	Sun Aug 24 2014 1:24:19	Executing queue-script PasswordDetector.py for TEST_NZB
DETAIL	Sun Aug 24 2014 1:24:19	FakeDetector: Detecting completed for TEST_NZB
DETAIL	Sun Aug 24 2014 1:24:19	FakeDetector: Detecting fake for TEST_NZB
DETAIL	Sun Aug 24 2014 1:24:10	Executing queue-script FakeDetector.py for TEST_NZB
then in post processing, Passworddetector deleted temp file 203 where FakeDetector did not:

Code: Select all

INFO	Sun Aug 24 2014 1:48:40	Post-process-script PasswordDetector.py for TEST_NZB successful
DETAIL	Sun Aug 24 2014 1:48:39	PasswordDetector: Completed removing temp file /mnt/usbhdd/NZBget/tmp/PasswordDetector/203
INFO	Sun Aug 24 2014 1:48:39	Executing post-process-script PasswordDetector.py for TEST_NZB
INFO	Sun Aug 24 2014 1:48:39	Post-process-script FakeDetector.py for TEST_NZB skipped
INFO	Sun Aug 24 2014 1:48:39	FakeDetector: Destination directory doesn't exist, exiting
INFO	Sun Aug 24 2014 1:48:38	Executing post-process-script FakeDetector.py for TEST_NZB
so I'm not sure why PasswordDetector doesn't delete temp files for you. FakeDetector doesn't delete files at the moment as it checks for mark bad and other stuff before it gets to cleaning up the temp file.
get39678 wrote:I had to click Save on the extensions options screen for things to work but again that may be just because I originally had bad whitespace whilst I was tinkering.
Yeah need to save new config options because the script checks for options you added to the start_check() function. Also python is very picky about whitespace and line endings, gets me all the time because I edit remotely on a Windows machine. You can get text editors (I use Notepad++) that display white space (python finds lines 387 and 388 bad) and uses UNIX line endings.

get39678
Posts: 222
Joined: 09 Jun 2014, 10:49

Re: [Queue-Script] PasswordDetector - stop dl of protected .

Post by get39678 » 24 Aug 2014, 08:47

JVM wrote:I did some testing with your script with r1103 and seems to work as expected.
Great. Is it good enough to be included in the official version?
JVM wrote:Yeah need to save new config options because the script checks for options you added to the start_check() function. Also python is very picky about whitespace and line endings, gets me all the time because I edit remotely on a Windows machine. You can get text editors (I use Notepad++) that display white space (python finds lines 387 and 388 bad) and uses UNIX line endings.
Once I realised the problem I ended up with Notepad++ glad to know I am using the correct thing.

Post Reply

Who is online

Users browsing this forum: No registered users and 25 guests