[Queue-Script] PasswordDetector - stop dl of protected .rar

Share your scripts or request scripts with specific features.
Forum rules
Please keep the forum clean - one topic per script. Questions not related to a specific script should be posted in Support forum.
get39678
Posts: 222
Joined: 09 Jun 2014, 10:49

Re: [Queue-Script] PasswordDetector - stop dl of protected .

Post by get39678 » 19 Aug 2014, 10:38

JVM wrote:Yes it's feasible but I might leave them separate for now and see if any issues arise. Something more efficient again would be implementing the scripts in bash or similar. Hugbug is looking into an interval between calls of queue scripts.
I'm curious what sort of processing times do you and others get? Easy way to find out is searching "detector" in the Messages tab. Any impact of download speed?
Not had chance to try on many items yet but the download speed seems to be about the same with or without both the detectors. I am running on a reasonable machine (not nas).

Code: Select all

INFO PasswordDetector: Unexpected end of archive[DETAIL] Detecting completed for snip
I get the above INFO message - is this normal operation?

Should the [DETAIL] be on another line?

Thanks

hugbug
Developer & Admin
Posts: 7645
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: [Queue-Script] PasswordDetector - stop dl of protected .

Post by hugbug » 19 Aug 2014, 11:57

INFO PasswordDetector: Unexpected end of archive[DETAIL] Detecting completed for snip
@JVM:
Change the line

Code: Select all

proc = subprocess.Popen(command, stdout=subprocess.PIPE)
to

Code: Select all

proc = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)

JVM
Posts: 83
Joined: 21 Oct 2013, 05:11

Re: [Queue-Script] PasswordDetector - stop dl of protected .

Post by JVM » 20 Aug 2014, 11:31

hugbug wrote: @JVM:
Change the line

Code: Select all

proc = subprocess.Popen(command, stdout=subprocess.PIPE)
to

Code: Select all

proc = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
Done. You shouldn't see any errors now @get39678.

Also updated it to version 1.1:
  • copied hugbug's updated sort_inner_files() and a new web ui setting "SortInner" to manually turn off sort_inner_files() if also using your FakeDetector.py.
  • and created a manual verbose unrar output setting on line 90 as like FakeDetector.py
  • changed command to "unrar l -p-" and looking for keywords "CRC" and "password" as per here
Last edited by JVM on 21 Aug 2014, 07:36, edited 1 time in total.

get39678
Posts: 222
Joined: 09 Jun 2014, 10:49

Re: [Queue-Script] PasswordDetector - stop dl of protected .

Post by get39678 » 20 Aug 2014, 12:08

Thanks I have given it another try but I am still getting very similar errors. Running r1098.

Snippet from log can be found below. I have only PasswordDetector active.

Get a Traceback, all files are downloaded and NZBGet tries to unpack and it fails as a password is required, fails to remove PasswordDetector temp file - temp\PasswordDetector\67 still exists on drive. I do not know if this will help but if FakeDetector is also active a temp\FakeDetector\67 file is also left behind.

Code: Select all

Wed Aug 20 12:52:22 2014	INFO	Executing queue-script PasswordDetector.py for test2
Wed Aug 20 12:52:23 2014	DETAIL	PasswordDetector: Detecting password for test2
Wed Aug 20 12:52:23 2014	DETAIL	PasswordDetector: Created temp file /somewhere/temp/PasswordDetector/67
Wed Aug 20 12:52:23 2014	INFO	PasswordDetector: Traceback (most recent call last):
Wed Aug 20 12:52:23 2014	INFO	PasswordDetector:   File "/somewhere/scripts/PasswordDetector.py", line 361, in <module>
Wed Aug 20 12:52:23 2014	INFO	PasswordDetector:     main()
Wed Aug 20 12:52:23 2014	INFO	PasswordDetector:   File "/somewhere/scripts/PasswordDetector.py", line 348, in main
Wed Aug 20 12:52:23 2014	INFO	PasswordDetector:     if contains_password(Directory) is True:
Wed Aug 20 12:52:23 2014	INFO	PasswordDetector:   File "/somewhere/scripts/PasswordDetector.py", line 166, in contains_password
Wed Aug 20 12:52:23 2014	INFO	PasswordDetector:     files = get_latest_file(dir)
Wed Aug 20 12:52:23 2014	INFO	PasswordDetector:   File "/somewhere/scripts/PasswordDetector.py", line 157, in get_latest_file
Wed Aug 20 12:52:23 2014	INFO	PasswordDetector:     return os.listdir(dir)
Wed Aug 20 12:52:23 2014	INFO	PasswordDetector: OSError: [Errno 2] No such file or directory: '/somewhere/incomplete/test2.#67'

Code: Select all

Wed Aug 20 12:52:47 2014	DETAIL	Quickly verified good file something.part10.rar
Wed Aug 20 12:52:47 2014	INFO	Repair not needed for test2/something
Wed Aug 20 12:52:47 2014	INFO	Unpacking test2
Wed Aug 20 12:52:47 2014	INFO	Unrar: UNRAR 4.00 beta 3 freeware      Copyright (c) 1993-2010 Alexander Roshal
Wed Aug 20 12:52:47 2014	INFO	Unrar: CRC failed in the encrypted file something.part03.rar. Corrupt file or wrong password.
Wed Aug 20 12:52:47 2014	INFO	Unrar: CRC failed in the encrypted file something.part06.rar. Corrupt file or wrong password.
Wed Aug 20 12:52:47 2014	INFO	Unrar: CRC failed in the encrypted file something.part08.rar. Corrupt file or wrong password.
Wed Aug 20 12:52:47 2014	INFO	Unrar: CRC failed in the encrypted file something.part01.rar. Corrupt file or wrong password.
Wed Aug 20 12:52:47 2014	INFO	Unrar: CRC failed in the encrypted file something.part05.rar. Corrupt file or wrong password.
Wed Aug 20 12:52:47 2014	INFO	Unrar: CRC failed in the encrypted file something.part09.rar. Corrupt file or wrong password.
Wed Aug 20 12:52:47 2014	INFO	Unrar: CRC failed in the encrypted file something.part10.rar. Corrupt file or wrong password.
Wed Aug 20 12:52:47 2014	INFO	Unrar: CRC failed in the encrypted file something.part07.rar. Corrupt file or wrong password.
Wed Aug 20 12:52:47 2014	INFO	Unrar: CRC failed in the encrypted file something.part04.rar. Corrupt file or wrong password.
Wed Aug 20 12:52:47 2014	INFO	Unrar: CRC failed in the encrypted file something.part02.rar. Corrupt file or wrong password.
Wed Aug 20 12:52:47 2014	INFO	Unrar: No files to extract
Wed Aug 20 12:52:47 2014	ERROR	Unrar error code: 10
Wed Aug 20 12:52:47 2014	ERROR	Unpack for test2 failed
Wed Aug 20 12:52:47 2014	INFO	Cleaning up test2

Code: Select all

Wed Aug 20 12:52:47 2014	INFO	Executing post-process-script PasswordDetector.py for test2
Wed Aug 20 12:52:47 2014	ERROR	PasswordDetector: Removing temp file was unsuccesful.
Wed Aug 20 12:52:47 2014	INFO	Post-process-script PasswordDetector.py for test2 skipped

JVM
Posts: 83
Joined: 21 Oct 2013, 05:11

Re: [Queue-Script] PasswordDetector - stop dl of protected .

Post by JVM » 20 Aug 2014, 13:07

get39678 wrote:fails to remove PasswordDetector temp file - temp\PasswordDetector\67 still exists on drive. I do not know if this will help but if FakeDetector is also active a temp\FakeDetector\67 file is also left behind
I've noticed this too, I've got a folder of undeleted "temp" files aswell since I've upgraded to r1094, I don't think that the queue/post processing scripts (FakeDetector and PasswordDetector) are being called in post processing when they should delete the "temp" files.

.... ok since I've typed the above out, I've upgraded to r1098, downloaded a nzb, I can confirm that it's removed the temp file for both scripts and I haven't touched the code. Maybe check in the log and look for if they have been used as a post process script like this:

Code: Select all

INFO	Wed Aug 20 2014 0:50:13	Post-process-script PasswordDetector.py for TEST_NZB successful
DETAIL	Wed Aug 20 2014 0:50:13	PasswordDetector: Removing temp file complete
INFO	Wed Aug 20 2014 0:50:12	Executing post-process-script PasswordDetector.py for TEST_NZB
INFO	Wed Aug 20 2014 0:50:12	Post-process-script FakeDetector.py for TEST_NZB successful
DETAIL	Wed Aug 20 2014 0:50:12	FakeDetector: Detecting completed for TEST_NZB
DETAIL	Wed Aug 20 2014 0:50:12	FakeDetector: Removing temp file complete
DETAIL	Wed Aug 20 2014 0:50:10	FakeDetector: Detecting fake for TEST_NZB
INFO	Wed Aug 20 2014 0:50:09	Executing post-process-script FakeDetector.py for TEST_NZB
get39678 wrote: CODE: SELECT ALL
Wed Aug 20 12:52:22 2014   INFO   Executing queue-script PasswordDetector.py for test2
Wed Aug 20 12:52:23 2014   DETAIL   PasswordDetector: Detecting password for test2
Wed Aug 20 12:52:23 2014   DETAIL   PasswordDetector: Created temp file /somewhere/temp/PasswordDetector/67
Wed Aug 20 12:52:23 2014   INFO   PasswordDetector: Traceback (most recent call last):
Wed Aug 20 12:52:23 2014   INFO   PasswordDetector:   File "/somewhere/scripts/PasswordDetector.py", line 361, in <module>
Wed Aug 20 12:52:23 2014   INFO   PasswordDetector:     main()
Wed Aug 20 12:52:23 2014   INFO   PasswordDetector:   File "/somewhere/scripts/PasswordDetector.py", line 348, in main
Wed Aug 20 12:52:23 2014   INFO   PasswordDetector:     if contains_password(Directory) is True:
Wed Aug 20 12:52:23 2014   INFO   PasswordDetector:   File "/somewhere/scripts/PasswordDetector.py", line 166, in contains_password
Wed Aug 20 12:52:23 2014   INFO   PasswordDetector:     files = get_latest_file(dir)
Wed Aug 20 12:52:23 2014   INFO   PasswordDetector:   File "/somewhere/scripts/PasswordDetector.py", line 157, in get_latest_file
Wed Aug 20 12:52:23 2014   INFO   PasswordDetector:     return os.listdir(dir)
Wed Aug 20 12:52:23 2014   INFO   PasswordDetector: OSError: [Errno 2] No such file or directory: '/somewhere/incomplete/test2.#67'
Ok I got the same error here, I'll check it out. When I start a download, I can see the download folder in incomplete being created after that error, confirming that the folder doesn't exist.

JVM
Posts: 83
Joined: 21 Oct 2013, 05:11

Re: [Queue-Script] PasswordDetector - stop dl of protected .

Post by JVM » 20 Aug 2014, 15:01

I've updated it, not quite happy with the structure of the code now but I can't get it to fault anymore.

One other thing, the script picked up that password protected rar yeah? You just allowed it to continue downloading?

get39678
Posts: 222
Joined: 09 Jun 2014, 10:49

Re: [Queue-Script] PasswordDetector - stop dl of protected .

Post by get39678 » 20 Aug 2014, 15:48

I had problems with the temp files not being removed in 1094 and 1098.

Now using the new version of script.

There are no errors regarding the temp file anymore and no files left in the temp folder.

However it is not detecting that the archive is password protected.

Code: Select all

PasswordDetector: Detecting completed for test4
PasswordDetector: Detecting password for test4
Executing queue-script PasswordDetector.py for test4
*All* the archive parts are downloaded (with repeat of above three lines in the log) and then the unpack fails due to there being a password on the archive.

If I turn verbose on and see what command it is running and try that at bash prompt unrar immediately prompts for a password it does not list any files - perhaps this is the problem?

Code: Select all

unrar l /somewhere/incomplete/test4.#69/something.part10.rar
UNRAR 4.00 beta 3 freeware      Copyright (c) 1993-2010 Alexander Roshal
Enter password (will not be echoed) for something.part10.rar: (I press enter here and then it outputs)
CRC failed in the encrypted file /somewhere/incomplete/test4.#69/something.part10.rar. Corrupt file or wrong password.
Thanks

get39678
Posts: 222
Joined: 09 Jun 2014, 10:49

Re: [Queue-Script] PasswordDetector - stop dl of protected .

Post by get39678 » 20 Aug 2014, 16:05

If the following is used it does not request a password it just outputs an error message (no prompt is output for a password) could you add a check for this to whatever is being done now?

Code: Select all

unrar l -p- /somewhere/incomplete/test4.#69/something.part10.rar
UNRAR 4.00 beta 3 freeware      Copyright (c) 1993-2010 Alexander Roshal
CRC failed in the encrypted file /somewhere/incomplete/test4.#69/something.part10.rar. Corrupt file or wrong password.
Thanks

JVM
Posts: 83
Joined: 21 Oct 2013, 05:11

Re: [Queue-Script] PasswordDetector - stop dl of protected .

Post by JVM » 21 Aug 2014, 07:13

get39678 wrote:If the following is used it does not request a password it just outputs an error message (no prompt is output for a password) could you add a check for this to whatever is being done now?

Code: Select all

unrar l -p- /somewhere/incomplete/test4.#69/something.part10.rar
UNRAR 4.00 beta 3 freeware      Copyright (c) 1993-2010 Alexander Roshal
CRC failed in the encrypted file /somewhere/incomplete/test4.#69/something.part10.rar. Corrupt file or wrong password.
Thanks for that, now using "unrar l -p-" and if unrar returns with some text that contains "password" or "CRC", it pauses or marks bad. Updated file in OP

get39678
Posts: 222
Joined: 09 Jun 2014, 10:49

Re: [Queue-Script] PasswordDetector - stop dl of protected .

Post by get39678 » 22 Aug 2014, 11:29

Took me time to work out why this was not working - you need to use err for this to work.

Code: Select all

PasswordDetector: Detecting completed for test12
PasswordDetector: Paused test12
PasswordDetector: Password found in test12
PasswordDetector: CRC failed in the encrypted file /somewhere/incomplete/something.part01.rar. Corrupt file or wrong password.
PasswordDetector: Enter password (will not be echoed) for something.part01.rar:
PasswordDetector: UNRAR 4.00 beta 3 freeware Copyright (c) 1993-2010 Alexander Roshal
PasswordDetector: command: ['unrar', 'l -p-', '/somewhere/incomplete/test12.#78/something.part01.rar']
PasswordDetector: Detecting password for test12

Code: Select all

            if verbose:
					print(out)
					print(err)
				if "*" in out:
					return True
				if err.find("password") != -1:
					return True
				if err.find("CRC") != -1:
					return True
I am still having problems with temp files being left in the PasswordDetector directory.

If a password is detected does it still do the clean up as the history shows a created temp file entry but there is no entry in history about removing the temp file?

Thanks I find this script useful.
Last edited by get39678 on 22 Aug 2014, 13:48, edited 1 time in total.

Post Reply

Who is online

Users browsing this forum: No registered users and 37 guests