Page 2 of 3

Re: Anime - join multiple mkv files

Posted: 21 Feb 2014, 21:39
by rubylaser
Thank you, it's working well here too :)

Re: Anime - join multiple mkv files

Posted: 06 Mar 2014, 20:53
by vergessen
I found an issue with the original version of this script. I didn't consider nzb's with multiple sets in them. Updated code is below and also on https://github.com/vergessen/nzbgetpp

Thank you

Re: Anime - join multiple mkv files

Posted: 06 Mar 2014, 21:31
by prinz2311
Suggestion:

You should use "with open" to open the files, to make sure that the files are always (error,...) closed and the File Handles are freed.

http://sdqali.in/blog/2012/07/09/unders ... hons-with/

Re: Anime - join multiple mkv files

Posted: 23 Mar 2014, 05:57
by NeedAnAccountToPost
Currently this script will make nzbget stuck when you have both kinds of par2 sets on one nzb
what it should be doing instead is
1. run par2 to merge files that have a parset before splitting, par2 works fine for this if the poster had any sense to pick correct block size for his post, if not then time to get more blocks
2. join .001 files that did not get merged by par2 (if you like posting like this then just use rar okay)

OR

delete split files after the par2 check not before

The flow should always be: par2 (skip IF parset for splitted files and everything is 100% complete) > join (if needed) > delete, not: join > delete > par2

and to be honest these files are a lot more common than people think and should be handeled by nzbget instead of some external hack.

Re: Anime - join multiple mkv files

Posted: 24 Mar 2014, 14:41
by vergessen
needanaccountopost
I made some changes to the script, you can also fork it https://github.com/vergessen/nzbgetpp/ and make changes as you see fit. I am thinking of some other things to make it more robust. I have tested it on half a dozen split files I could find and did not run into an issue. If you have time please test or send me an example nzb vergessen at gmail

Code: Select all

#!/usr/bin/env python

##############################################################################
### NZBGET POST-PROCESSING SCRIPT ###

# Join numbered split files after download
#
#
# This script joins *.001 etc files
#
#
# NOTE: Requires python
#
#
# Please note on the second pass it will skip output as this is by design because we just
# want the pars to check out the record again.

##############################################################################
### OPTIONS ###
# Preserve orginal files (yes, no).
#
#Preservefiles=no
#
# Overwrite output file (yes, no).
#
#
# In the case that both *.001 and the output file exists but is not the
# expected size should the output file be overwritten.
#
#Overwritefiles=yes
#
### NZBGET POST-PROCESSING SCRIPT ###
##############################################################################

import re
import os
import sys

# Exit codes used by NZBGet
POSTPROCESS_SUCCESS=93
POSTPROCESS_ERROR=94
POSTPROCESS_PAR2=92
POSTPROCESS_SKIPPED=95

if not os.environ.has_key('NZBOP_SCRIPTDIR'):
print('*** NZBGet post-processing script ***')
print('This script is supposed to be called from nzbget (11.0 or later).')
sys.exit(POSTPROCESS_ERROR)


sys.stdout.flush()

matcher = re.compile('\.[0-9]+$')
chunks = []
chunks2 = []

for dirpath, dirnames, filenames in os.walk(os.environ['NZBPP_DIRECTORY']):
    for file in filenames:
        chunks.append(os.path.join(dirpath,file))

chunks2 = [f for f in chunks if matcher.search(f)]

if not chunks2:
    print('[DETAIL] No files found to join. Skipping join!')
    sys.exit(POSTPROCESS_SKIPPED)



sets = {}
set = None

for file in chunks2:
    start, finish = os.path.splitext(file)
    if start not in sets:
        sets[start] = []
    sets[start].append(file)

for set in sets:
    sets[set].sort()
    current = sets[set]
    outfile = set
    if os.path.isfile(outfile):
        if os.path.getsize(outfile) <= os.path.getsize(current[0]):
            if outfile + '.001' not in current:
                os.rename(outfile, outfile + '.001')
                current.append(outfile + '.001')
                current.sort()
            else:
                if os.environ['NZBPO_OVERWRITEFILES'] == 'yes':
                    os.unlink(outfile)
                else:
                    print('[ERROR] %s was found and does not match expected output size. Not overwriting, OVERWRITEFILES is sent to %s' % (outfile, os.environ['NZBPO_OVERWRITEFILES']))
                    sys.exit(POSTPROCESS_ERROR)
        else:
            print('[DETAIL] %s was found at %s bytes. Output matches at %s bytes. Skipping join!' % (outfile, os.path.getsize(outfile),size))
            sys.exit(POSTPROCESS_SKIPPED)

    wdata = ''
    newfile = open(outfile, "ab")
    i = 0
    for f in current:
        i = i + 1
        print('[INFO] joining file %s' % f)
        part = open(f, "rb")
        reading = True
        while reading:
            wdata = part.read(4096)
            if (len(wdata) > 0):
                newfile.write(wdata)
            else:
                reading = False
        part.close()
        if os.environ['NZBPO_PRESERVEFILES'] == 'yes':
            print ('[INFO] File %s was not deleted' % f)
        else:
            print('[INFO] deleted file %s' % f)
            os.unlink(f)
    newfile.close()
sys.exit(POSTPROCESS_SUCCESS)



Re: Anime - join multiple mkv files

Posted: 17 Apr 2014, 17:12
by hugbug
Fixed in r999:
splitted files are now joined automatically (again)

Re: Anime - join multiple mkv files

Posted: 20 Apr 2014, 00:38
by vergessen
hugbug,

is there anything special one would need to do to activate this? I just downloaded a split file to test and it was moved as the split files?

update: Well I downloaded one with a par2 and it worked with that. Does this require a par2 file to be present?

Thanks

Re: Anime - join multiple mkv files

Posted: 20 Apr 2014, 10:35
by hugbug
The join is made during par-repair.
Are splitted files without pars common?

Re: Anime - join multiple mkv files

Posted: 20 Apr 2014, 12:22
by vergessen
Of my first 10 attempts I found 3 that failed. 2 had a par2 but didn't seem to trigger

Re: Anime - join multiple mkv files

Posted: 20 Apr 2014, 12:43
by hugbug
Can you send me that two nzbs with pars?