[Request] Flat unpack

[Vash]

I had this feature with sabnzbd, called "flat_unpack". It upacks files, but ignores the subfolders in the archive. Really helpfull when you have a ebook rss feed (like me) and you want the files extracted the the main folder. Now ebooks are extracted to a cryptic folder which has a second folder with the correct name in it, where the ebook is stored. Is there a way to extract the pdf files only to the main?

[hugbug]

A relatively simple post-processing script could do that. I'll move the topic to extension scripts subforum with the hope someone can write the script.

[clintonhall]

Hmmm.. here comes my stock-standard response.... "nzbToMedia includes this feature".

Basically, I have some very simple python code that will "flatten" all files within the download directory.

So here comes hugbug's response... "can you create this as a stand-alone script?"

So...

flatten.py

Code: Select all

#!/usr/bin/env python
#
##############################################################################
### NZBGET POST-PROCESSING SCRIPT                                          ###

# Flatten all downloaded files into the root download directory.
#
# This removes all of the sub-folders created by the unpack process.
# This should run before other scripts.
#
# NOTE: This script requires Python to be installed on your system.
##############################################################################
### OPTIONS                                                                ###

# Destination Directory.
#
# Set the directory where you want all files to be moved to.
# Use this if you want all downloaded files in a single "root" directory.
# If left blank, files will all be "flattened" into the individual download's sub-directory.
#DestinationDirectory=

# Append Categories (yes, no).
# 
# If using the Destination Directory above, then this option will append the download category.
#AppendCategories=no


### NZBGET POST-PROCESSING SCRIPT                                          ###
##############################################################################
import os
import sys
import shutil

# NZBGet Exit Codes
NZBGET_POSTPROCESS_PARCHECK = 92
NZBGET_POSTPROCESS_SUCCESS = 93
NZBGET_POSTPROCESS_ERROR = 94
NZBGET_POSTPROCESS_NONE = 95

if not os.environ.has_key('NZBOP_SCRIPTDIR'):
    print "This script can only be called from NZBGet (11.0 or later)."
    sys.exit(0)

if os.environ['NZBOP_VERSION'][0:5] < '11.0':
    print "[ERROR] NZBGet Version %s is not supported. Please update NZBGet." % (str(os.environ['NZBOP_VERSION']))
    sys.exit(0)

print "Script triggered from NZBGet Version %s." % (str(os.environ['NZBOP_VERSION']))
status = 0
if os.environ.has_key('NZBPP_TOTALSTATUS'):
    if not os.environ['NZBPP_TOTALSTATUS'] == 'SUCCESS':
        print "[ERROR] Download failed with status %s." % (os.environ['NZBPP_STATUS'])
        status = 1

else:
    # Check par status
    if os.environ['NZBPP_PARSTATUS'] == '1' or os.environ['NZBPP_PARSTATUS'] == '4':
        print "[ERROR] Par-repair failed, setting status \"failed\"."
        status = 1

    # Check unpack status
    if os.environ['NZBPP_UNPACKSTATUS'] == '1':
        print "[ERROR] Unpack failed, setting status \"failed\"."
        status = 1

    if os.environ['NZBPP_UNPACKSTATUS'] == '0' and os.environ['NZBPP_PARSTATUS'] == '0':
        # Unpack was skipped due to nzb-file properties or due to errors during par-check

        if os.environ['NZBPP_HEALTH'] < 1000:
            print "[ERROR] Download health is compromised and Par-check/repair disabled or no .par2 files found. Setting status \"failed\"."
            print "[ERROR] Please check your Par-check/repair settings for future downloads."
            status = 1

        else:
            print "[ERROR] Par-check/repair disabled or no .par2 files found, and Unpack not required. Health is ok so handle as though download successful."
            print "[WARNING] Please check your Par-check/repair settings for future downloads."

# Check if destination directory exists (important for reprocessing of history items)
if not os.path.isdir(os.environ['NZBPP_DIRECTORY']):
    print "[ERROR] Nothing to post-process: destination directory", os.environ['NZBPP_DIRECTORY'], "doesn't exist. Setting status \"failed\"."
    status = 1

# All checks done, now launching the script.
if status == 1:
    sys.exit(NZBGET_POSTPROCESS_NONE)

def removeEmptyFolders(path, removeRoot=True):
    #Function to remove empty folders
    if not os.path.isdir(path):
        return

    # remove empty subfolders
    print "[INFO] Checking for empty folders in:%s" % path
    files = os.listdir(path)
    if len(files):
        for f in files:
            fullpath = os.path.join(path, f)
            if os.path.isdir(fullpath):
                removeEmptyFolders(fullpath)

    # if folder empty, delete it
    files = os.listdir(path)
    if len(files) == 0 and removeRoot:
        print "[INFO] Removing empty folder:%s" % path
        os.rmdir(path)

directory = os.path.normpath(os.environ['NZBPP_DIRECTORY'])
if os.environ['NZBPO_DESTINATIONDIRECTORY'] and os.path.isdir(os.environ['NZBPO_DESTINATIONDIRECTORY']):
    destination = os.environ['NZBPO_DESTINATIONDIRECTORY']
    if os.environ['NZBPO_APPENDCATEGORIES'] == 'yes':
        destination = os.path.join(destination, os.environ['NZBPP_CATEGORY'])
else:
    destination = directory
print "Flattening directory: %s" % (directory)
for dirpath, dirnames, filenames in os.walk(directory):
    for fileName in filenames:
        outputFile = os.path.join(dirpath, fileName)
        if dirpath == destination:
            continue
        target = os.path.join(destination, fileName)
        try:
            print "Moving: %s to: %s" % (outputFile, target)
            shutil.move(outputFile, target)
        except:
            print "[ERROR] Could not flatten %s" % outputFile
removeEmptyFolders(directory)  # Cleanup empty directories
sys.exit(NZBGET_POSTPROCESS_SUCCESS)

Please try this and let me know if you have any issues with it.

[hugbug]

Clinton, you are the best

[clintonhall]

Hey... I was waiting for my VMs to complete the cross-compiling of NZBGet-14.0 for 5 different platforms

I am thinking of dropping ResetDateTimes and DeleteSamples from the nzbToMedia repo and creating a separate repo/host location for all of these stand-alone scripts...
Or perhaps even get them added to your source-forge location along with others?

I think so far these are scripts for:
-ResetDateTimes
-DeleteSamples
-Flatten (flat unpack)
-Unrar Password List

[hugbug]

Or perhaps even get them added to your source-forge location along with others?

It's better if developers host their scripts because that way they can update the scripts anytime.

[clintonhall]

Fair call...
I'll look at creating a separate github repo for all my "other" scripts.

I'l just wait to get some feedback about the password list. If it is merged with password detector, then I'm happy to "donate" some code to that project

[Vash]

Please try this and let me know if you have any issues with it.

First of all...Thank you very much Clinton Hall. I appreciate all your efforts. This Script is exacly what i need, but the script doesn't work for me. Here the log file.

Code: Select all

INFO	Tue Nov 11 2014 21:58:14	Collection WUK.Nuding.Haller.ebook.2014 added to history
INFO	Tue Nov 11 2014 21:58:14	Cleaning up download queue for WUK.Nuding.Haller.ebook.2014
ERROR	Tue Nov 11 2014 21:58:14	Post-process-script flatten.py for WUK.Nuding.Haller.ebook.2014 failed (terminated with unknown status)
INFO	Tue Nov 11 2014 21:58:14	flatten: NameError: name 'dirPath' is not defined
INFO	Tue Nov 11 2014 21:58:14	flatten: if dirPath == directory:
INFO	Tue Nov 11 2014 21:58:14	flatten: File "/volume1/Downloads/scripts/flatten.py", line 97, in <module>
INFO	Tue Nov 11 2014 21:58:14	flatten: Traceback (most recent call last):
INFO	Tue Nov 11 2014 21:58:14	flatten: Flattening directory: /volume1/Downloads/Incomplete/WUK.Nuding.Haller.ebook.2014.#785
INFO	Tue Nov 11 2014 21:58:14	flatten: Script triggered from NZBGet Version 14.0-testing.
INFO	Tue Nov 11 2014 21:58:14	Executing post-process-script flatten.py for WUK.Nuding.Haller.ebook.2014
INFO	Tue Nov 11 2014 21:58:14	Unpausing download after cleanup

[clintonhall]

I updated the code above... stupid typo had "dirPath" instead of "dirpath"

Try again.

[get39678]

I am using the correct dirpath not dirPath but I am getting the following errors:

Code: Select all

INFO	Executing post-process-script Flatten.py for 
INFO	Flatten: /usr/bin/env: python
: No such file or directory
ERROR	Post-process-script Flatten.py for failed (terminated with unknown status)

I have checked the /usr/bin/env: python line and this is the same as other scripts so I do not believe that line is causing the problem.

Any ideas? Thanks