Anime - join multiple mkv files

Share your scripts or request scripts with specific features.
Forum rules
Please keep the forum clean - one topic per script. Questions not related to a specific script should be posted in Support forum.
rubylaser
Posts: 33
Joined: 06 Feb 2014, 14:47

Re: Anime - join multiple mkv files

Post by rubylaser » 21 Feb 2014, 21:39

Thank you, it's working well here too :)

vergessen
Posts: 23
Joined: 19 Feb 2014, 15:36

Re: Anime - join multiple mkv files

Post by vergessen » 06 Mar 2014, 20:53

I found an issue with the original version of this script. I didn't consider nzb's with multiple sets in them. Updated code is below and also on https://github.com/vergessen/nzbgetpp

Thank you
Last edited by vergessen on 24 Mar 2014, 14:42, edited 1 time in total.

prinz2311
Posts: 462
Joined: 08 Dec 2012, 00:03

Re: Anime - join multiple mkv files

Post by prinz2311 » 06 Mar 2014, 21:31

Suggestion:

You should use "with open" to open the files, to make sure that the files are always (error,...) closed and the File Handles are freed.

http://sdqali.in/blog/2012/07/09/unders ... hons-with/

NeedAnAccountToPost
Posts: 3
Joined: 23 Mar 2014, 05:43

Re: Anime - join multiple mkv files

Post by NeedAnAccountToPost » 23 Mar 2014, 05:57

Currently this script will make nzbget stuck when you have both kinds of par2 sets on one nzb
what it should be doing instead is
1. run par2 to merge files that have a parset before splitting, par2 works fine for this if the poster had any sense to pick correct block size for his post, if not then time to get more blocks
2. join .001 files that did not get merged by par2 (if you like posting like this then just use rar okay)

OR

delete split files after the par2 check not before

The flow should always be: par2 (skip IF parset for splitted files and everything is 100% complete) > join (if needed) > delete, not: join > delete > par2

and to be honest these files are a lot more common than people think and should be handeled by nzbget instead of some external hack.

vergessen
Posts: 23
Joined: 19 Feb 2014, 15:36

Re: Anime - join multiple mkv files

Post by vergessen » 24 Mar 2014, 14:41

needanaccountopost
I made some changes to the script, you can also fork it https://github.com/vergessen/nzbgetpp/ and make changes as you see fit. I am thinking of some other things to make it more robust. I have tested it on half a dozen split files I could find and did not run into an issue. If you have time please test or send me an example nzb vergessen at gmail

Code: Select all

#!/usr/bin/env python

##############################################################################
### NZBGET POST-PROCESSING SCRIPT ###

# Join numbered split files after download
#
#
# This script joins *.001 etc files
#
#
# NOTE: Requires python
#
#
# Please note on the second pass it will skip output as this is by design because we just
# want the pars to check out the record again.

##############################################################################
### OPTIONS ###
# Preserve orginal files (yes, no).
#
#Preservefiles=no
#
# Overwrite output file (yes, no).
#
#
# In the case that both *.001 and the output file exists but is not the
# expected size should the output file be overwritten.
#
#Overwritefiles=yes
#
### NZBGET POST-PROCESSING SCRIPT ###
##############################################################################

import re
import os
import sys

# Exit codes used by NZBGet
POSTPROCESS_SUCCESS=93
POSTPROCESS_ERROR=94
POSTPROCESS_PAR2=92
POSTPROCESS_SKIPPED=95

if not os.environ.has_key('NZBOP_SCRIPTDIR'):
print('*** NZBGet post-processing script ***')
print('This script is supposed to be called from nzbget (11.0 or later).')
sys.exit(POSTPROCESS_ERROR)


sys.stdout.flush()

matcher = re.compile('\.[0-9]+$')
chunks = []
chunks2 = []

for dirpath, dirnames, filenames in os.walk(os.environ['NZBPP_DIRECTORY']):
    for file in filenames:
        chunks.append(os.path.join(dirpath,file))

chunks2 = [f for f in chunks if matcher.search(f)]

if not chunks2:
    print('[DETAIL] No files found to join. Skipping join!')
    sys.exit(POSTPROCESS_SKIPPED)



sets = {}
set = None

for file in chunks2:
    start, finish = os.path.splitext(file)
    if start not in sets:
        sets[start] = []
    sets[start].append(file)

for set in sets:
    sets[set].sort()
    current = sets[set]
    outfile = set
    if os.path.isfile(outfile):
        if os.path.getsize(outfile) <= os.path.getsize(current[0]):
            if outfile + '.001' not in current:
                os.rename(outfile, outfile + '.001')
                current.append(outfile + '.001')
                current.sort()
            else:
                if os.environ['NZBPO_OVERWRITEFILES'] == 'yes':
                    os.unlink(outfile)
                else:
                    print('[ERROR] %s was found and does not match expected output size. Not overwriting, OVERWRITEFILES is sent to %s' % (outfile, os.environ['NZBPO_OVERWRITEFILES']))
                    sys.exit(POSTPROCESS_ERROR)
        else:
            print('[DETAIL] %s was found at %s bytes. Output matches at %s bytes. Skipping join!' % (outfile, os.path.getsize(outfile),size))
            sys.exit(POSTPROCESS_SKIPPED)

    wdata = ''
    newfile = open(outfile, "ab")
    i = 0
    for f in current:
        i = i + 1
        print('[INFO] joining file %s' % f)
        part = open(f, "rb")
        reading = True
        while reading:
            wdata = part.read(4096)
            if (len(wdata) > 0):
                newfile.write(wdata)
            else:
                reading = False
        part.close()
        if os.environ['NZBPO_PRESERVEFILES'] == 'yes':
            print ('[INFO] File %s was not deleted' % f)
        else:
            print('[INFO] deleted file %s' % f)
            os.unlink(f)
    newfile.close()
sys.exit(POSTPROCESS_SUCCESS)



hugbug
Developer & Admin
Posts: 7402
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: Anime - join multiple mkv files

Post by hugbug » 17 Apr 2014, 17:12

Fixed in r999:
splitted files are now joined automatically (again)

vergessen
Posts: 23
Joined: 19 Feb 2014, 15:36

Re: Anime - join multiple mkv files

Post by vergessen » 20 Apr 2014, 00:38

hugbug,

is there anything special one would need to do to activate this? I just downloaded a split file to test and it was moved as the split files?

update: Well I downloaded one with a par2 and it worked with that. Does this require a par2 file to be present?

Thanks

hugbug
Developer & Admin
Posts: 7402
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: Anime - join multiple mkv files

Post by hugbug » 20 Apr 2014, 10:35

The join is made during par-repair.
Are splitted files without pars common?

vergessen
Posts: 23
Joined: 19 Feb 2014, 15:36

Re: Anime - join multiple mkv files

Post by vergessen » 20 Apr 2014, 12:22

Of my first 10 attempts I found 3 that failed. 2 had a par2 but didn't seem to trigger

hugbug
Developer & Admin
Posts: 7402
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: Anime - join multiple mkv files

Post by hugbug » 20 Apr 2014, 12:43

Can you send me that two nzbs with pars?

Post Reply

Who is online

Users browsing this forum: No registered users and 2 guests