[PP-Script] VideoSort - better video sorting

Share your scripts or request scripts with specific features.
Forum rules
Please keep the forum clean - one topic per script. Questions not related to a specific script should be posted in Support forum.
Post Reply
hugbug
Developer & Admin
Posts: 6710
Joined: 09 Sep 2008, 11:58
Location: Germany (NRW)

[PP-Script] VideoSort - better video sorting

Post by hugbug » 11 Jul 2013, 21:43

What is this?
This is a script for downloaded TV shows and movies. It uses scene-standard naming conventions to match TV shows and movies and rename/move/sort/organize them as you like.

The script is written in Python and works on all platforms supporting Python including Linux, Mac, Windows.

VideoSort relies on python library "guessit" (http://guessit.readthedocs.org) to extract information from file names and shamelessly includes portions of formatting code from "SABnzbd+" (http://sabnzbd.org). This provides some degree of compatibility with SAB formatting strings.

Info:
Author: Andrey Prygunkov (nzbget@gmail.com).
Home page and source code: https://github.com/nzbget/VideoSort.
License: GPLv3 (http://www.gnu.org/licenses/gpl.html).

Installation, usage and ChangeLog
Please see VideoSort home page.

Questions, suggestions?
Please post in this topic.

naan
Posts: 17
Joined: 11 Jun 2013, 14:21

Re: [PP-Script] TurboSort - organize movies and tv shows

Post by naan » 16 Jul 2013, 19:38

First results have failed for me, both scenario 6. I've tried 2 different movies both similiar end result.

Structure

Code: Select all

<dir>     Dead Man Down (2013) BRRiP XViD DD5_1 Custom NLSubs _=-_lt_ Q_o_Q _gt_-=_
<dir>     XD607ebb-BRc59935-5155473f-1c5f49
<file>    XD607ebb-BRc59935-5155473f-1c5f49.avi 
Renamed to: "Xd607Ebb-Brc59935-5155473F-1C5F49 (2013).avi"

Code: Select all

info	Tue Jul 16 2013 21:09:43	VideoSort: Move satellites for /share/Download/movie/Dead Man Down (2013) BRRiP XViD DD5_1 Custom NLSubs _=-_lt_ Q_o_Q _gt_-=_/XD607ebb-BRc59935-5155473f-1c5f49/XD607ebb-BRc59935-5155473f-1c5f49.avi
info	Tue Jul 16 2013 21:09:43	VideoSort: Moved: /share/Download/test/Xd607Ebb-Brc59935-5155473F-1C5F49 (2013).avi
info	Tue Jul 16 2013 21:09:43	VideoSort: destination path: /share/Download/test/Xd607Ebb-Brc59935-5155473F-1C5F49 (2013).avi
info	Tue Jul 16 2013 21:09:43	VideoSort: path after cleanup: Xd607Ebb-Brc59935-5155473F-1C5F49 (2013).avi
info	Tue Jul 16 2013 21:09:43	VideoSort: path after subst: Xd607Ebb-Brc59935-5155473F-1C5F49 (2013)..avi
info	Tue Jul 16 2013 21:09:43	VideoSort: t (%y).%ext
info	Tue Jul 16 2013 21:09:43	VideoSort: }
info	Tue Jul 16 2013 21:09:43	VideoSort: [1.00] "type": "movie"
info	Tue Jul 16 2013 21:09:43	VideoSort: [1.00] "year": 2013,
info	Tue Jul 16 2013 21:09:43	VideoSort: [1.00] "format": "BluRay",
info	Tue Jul 16 2013 21:09:43	VideoSort: [0.80] "title": "XD607ebb-BRc59935-5155473f-1c5f49",
info	Tue Jul 16 2013 21:09:43	VideoSort: ],
info	Tue Jul 16 2013 21:09:43	VideoSort: "Lithuanian"
info	Tue Jul 16 2013 21:09:43	VideoSort: [0.80] "language": [
info	Tue Jul 16 2013 21:09:43	VideoSort: [1.00] "container": "avi",
info	Tue Jul 16 2013 21:09:43	VideoSort: [1.00] "videoCodec": "XviD",
info	Tue Jul 16 2013 21:09:43	VideoSort: [1.00] "mimetype": "video/x-msvideo",
info	Tue Jul 16 2013 21:09:43	VideoSort: {
info	Tue Jul 16 2013 21:09:43	VideoSort: filename: /share/Download/movie/Dead Man Down (2013) BRRiP XViD DD5_1 Custom NLSubs _=-_lt_ Q_o_Q _gt_-=_/XD607ebb-BRc59935-5155473f-1c5f49/XD607ebb-BRc59935-5155473f-1c5f49.avi
The sub folder did contain a whole lot of other crap, I've attached a screenshot. Also take notice of the extra par files in the unpacked folder, if these are processed after the initial unpack they will rename the movie correctly i.e "Dead Man Down (2013).mkv"
Image

I've used VideoSort, but this thread containing the scenario's seemed more logical to respond to, feel free to move if needed.

naan
Posts: 17
Joined: 11 Jun 2013, 14:21

Re: [PP-Script] TurboSort - organize movies and tv shows

Post by naan » 16 Jul 2013, 20:01

Additional info,

After moving all files out of the sub folder the renaming seemed to go flawless for both test movies

hugbug
Developer & Admin
Posts: 6710
Joined: 09 Sep 2008, 11:58
Location: Germany (NRW)

Re: [PP-Script] VideoSort - better video sorting

Post by hugbug » 16 Jul 2013, 22:03

***From TurboSort topic to continue discussion here***

Cases which make the thing complex:

1. The nzb-file doesn't have a meaningful name, for example 4242345345345.nzb.

Code: Select all

<dir>         4242345345345
<file>                Game of Thrones S02E01 720p BluRay X264-REWARD.mkv
The script should use the mkv-file name then.

2. The nzb-file has a meaningful name, but the video file has a bad name:

Code: Select all

<dir>         Dexter S02E01 720p BluRay X264-REWARD
<file>                reward-dexter.s03e01.mkv
The problem here is that video file match the pattern for a "good filename". The script thinks that the show name is "reward-dexter".

3. The nzb-file includes multiple video files. For example on binsearch.info I could select multiple collections and save them into one nzb file, for example "Game of Thrones.nzb"

Code: Select all

<dir>         Game.of.Thrones
<file>                Game of Thrones S02E01 720p BluRay X264-REWARD.mkv
<file>                Game of Thrones S02E02 720p BluRay X264-REWARD.mkv
<file>                Game of Thrones S02E03 720p BluRay X264-REWARD.mkv
TurboSort must use video files names in this case.

4. Like previous case but video files have bad names:

Code: Select all

<dir>         Game.of.Thrones
<file>                GOT0301.mkv
<file>                GOT0302.mkv
<file>                GOT0303.mkv
Here the script can't sort good.

5. The nzb-file has a meaningful name, but the video file has a non-meaningful name:

Code: Select all

<dir>         Game of Thrones S02E01 720p BluRay X264-REWARD
<file>                4242345345345
The difference between 2 and 5 is that in scenario 2 the file name is meaningful but still bad - the nzb-name is better. In the scenario 5 the file name is a garbage and it should be pretty easy to fall back to use the nzb-name in this case.

6. The nzb-file has a meaningful name, but the video file has a non-meaningful name and the video file is put into an extra subfolder:

Code: Select all

<dir>         Game of Thrones S02E01 720p BluRay X264-REWARD
<dir>                 aaaaaaaa
<file>                       ewerrgdfghgh.mkv
Guessit (the underlying library used by VideoSort) should be able to handle most of these scenarios. Here is the results of my tests:
1. correct (no wonder);
2. correct - that was a hard case but guessit was able to use the directory name;
3. correct;
4. failed;
5. correct;
6. correct.

hugbug
Developer & Admin
Posts: 6710
Joined: 09 Sep 2008, 11:58
Location: Germany (NRW)

Re: [PP-Script] VideoSort - better video sorting

Post by hugbug » 16 Jul 2013, 22:51

I've done some testing.
It seems that the file "XD607ebb-BRc59935-5155473f-1c5f49.avi" in folder "XD607ebb-BRc59935-5155473f-1c5f49" makes guessit pretty much confident about filename/folder-name being the movie name.
After renaming of folder from "XD607ebb-BRc59935-5155473f-1c5f49" to "aaaaaa" the title was correctly recognized.

I see two ways to solve the problem:
  1. automatically rename files using pars. This was already requested in the topic Par Rename, which becomes more weight now;
  2. report the issue to guessit (https://github.com/wackou/guessit/issues) for improving.

naan
Posts: 17
Joined: 11 Jun 2013, 14:21

Re: [PP-Script] VideoSort - better video sorting

Post by naan » 18 Jul 2013, 20:13

I'll report the issue on github later this week. Par rename function or script would be execellent.

naan
Posts: 17
Joined: 11 Jun 2013, 14:21

Re: [PP-Script] VideoSort - better video sorting

Post by naan » 25 Aug 2013, 15:19

The issue should be fixed in the latest master release from guessit, haven't been able to test it though.

minimeh
Posts: 32
Joined: 16 Aug 2013, 09:54

Re: [PP-Script] VideoSort - better video sorting

Post by minimeh » 28 Aug 2013, 10:47

I have been migrating my sabnzbd functionality to nzbget. That's been going very well but I have run into a few hitches with video sorting for dated episodes of tv shows.

I have two separate requirements for dated tv episodes. In one case, I need the show name, broadcast date, and episode title (which is usually the guest list), e.g. The.Colbert.Report.2013.08.13-Rep. John Lewis.mkv. The reason is that I have an application for binge watching The Daily Show with Jon Stewart paired with The Colbert Report. It uses the the broadcast date and guest list to display a list of shows to watch. This format also works well for plain directory file listings.

In the second case, I need the show name, season number, episode number, and episode title, e.g Real.Time.With.Bill.Maher.S11E25-Jay-Z, Barney Frank, Larry Miller, Josh Barro, Alexis Goldstein.mkv. This is a more typical need. Most video jukeboxes need the season and episode of a show to retrieve information and artwork.

My plan was to finish off video sorting for dated tv shows by calling on Filebot. Filebot has the ability to use tv databases in thetvdb.com and tvrage.com to lookup tv shows to provide the season, episode, and episode title. While I can do this manually with simple drag and drop, my preference is to have it automatically performed with a post process script.

Here is where I ran into the first hitch: After VideoSort runs, there is no way for any following pp-scripts to know the new name for a file. Nzbget does its post processing by using environment variables to pass values to a script. However, a parent's environment variable can not be changed by a child (nzbget being the parent and VideoSort being a child). Every pp-script gets its own copy of the environment. There is no means for informing the parent or any further scripts of changes.

I created a method to convey changes back to the parent and any sibling pp-scripts. This works by the pp-script writing the changed environment variables to a text file. When the script exits, the parents and siblings can retrieve the values from the text file.

To make that as pain free as possible, I wrote a Python script called childEnVars.py. Because it works closely with VideoSort, I put it into the lib folder of VideoSort.

childEnVars provides three functions for pp-scripts to use: Put, Get, and Clear. Pp-scripts call Put to save a value; Get to retrieve a value; and Clear to start with a clean slate.

Code: Select all

childEnVars.Clear()
childEnVars.Put('NZBPP_DIRECTORY', os.path.split(path)[0])
childEnVars.Put('NZBPP_NZBFILENAME', os.path.split(path)[1] + '.nzb')
finaldirectory = childEnVars.Get('NZBPP_DIRECTORY')
originalname = childEnVars.Get('NZBPP_NZBFILENAME')
A file "childenvars" is created in the same directory as the log is written to ensure that reading and writing rights are permitted. The pp-scripts need only put/get values without consideration for how the values are cached.

I modified VideoSort to use childEnVars by adding the following lines:

Code: Select all

import childEnVars
...
            # We need a means to let the parent and interested
            # sibling processes know what the new names are.
            #
            # The following will not work because chained
            # scripts are not run within the same environment
            #   os.environ['NZBPP_DIRECTORY'] = os.path.split(path)[0])
            #   os.environ['NZBPP_NZBFILENAME'] = os.path.split(path)[1] + '.nzb')
            #   os.environ['NZBPP_NZBNAME'] = os.path.split(path)[1])
            # Instead we'll stow the renamed file names into a file
            # where the parent and sibling processes can retrieve them.
            childEnVars.Clear()
            childEnVars.Put('NZBPP_DIRECTORY', os.path.split(path)[0])
            childEnVars.Put('NZBPP_NZBFILENAME', os.path.split(path)[1] + '.nzb')
            childEnVars.Put('NZBPP_NZBNAME', os.path.split(path)[1])
I created a pp-script "FinishDatedEpisode.py" which calls childEnVars.Get() to get the final file name after VideoSort is done. It then finishes the renaming to my requirements.

Another hitch I ran into is that VideoSort ignores specified destination directories for categories. It provides its own MoviesDir, SeriesDir, and DatedDir. But when an nzb's category is known, my preference would be to use the destination directory for the category.

I modified VideoSort to offer a new option: UseCatDestDir. If turned on, VideoSort will put the video files into the destination directory for the category if the category is known. If not turned on or the category is not known, then MoviesDir, SeriesDir, and DatedDir are used as before.

To support using the category destination, I added the following:

Code: Select all

 def set_dest_dir(default_dest_dir):
    # If it is optioned and we can do it, use the category
    # destination directory
    if use_cat_dest_dir:
        # See if the category is available, and then if there is
        # a category destination specified for it
        category = os.environ.get('NZBPP_CATEGORY', None)
        if category is not None:
            regex = re.compile(r"NZBOP_CATEGORY(\d+)_NAME")
            for k, v in os.environ.items():
                r = regex.search(k)
                if r is not None:
                    if category == v:
                        category_dest_dir_key = "NZBOP_CATEGORY%s_DESTDIR" % r.groups()[0]
                        category_dest_dir = os.environ.get(category_dest_dir_key, None)
                        if category_dest_dir is not None and len(category_dest_dir) > 0:
                            # Check if there are substitution macros
                            if category_dest_dir.startswith("${MainDir)"):
                                ddir = os.environ.get("NZBOP_MAINDIR", "")
                                category_dest_dir = category_dest_dir.replace("${MainDir)", ddir)
                            elif category_dest_dir.startswith("${DestDir)"):
                                ddir = os.environ.get("NZBOP_DESTDIR", "")
                                category_dest_dir = category_dest_dir.replace("${DestDir)", ddir)
                            if verbose:
                                print "[INFO] category_dest_dir = \"%s\"" % category_dest_dir
                            return category_dest_dir
    return default_dest_dir
I changed these lines:

Code: Select all

            dest_dir = dated_dir
            dest_dir = movies_dir
        dest_dir = series_dir
to

Code: Select all

            dest_dir = set_dest_dir(dated_dir)
            dest_dir = set_dest_dir(movies_dir)
        dest_dir = set_dest_dir(series_dir)
In working on these issues, I created a pp-script "Tattle.py". This is just a utility that logs the environment variables that nzbget is using. I found this to be very helpful.

I think these are useful modifications that others may benefit from. I have attached the scripts.
Attachments
VideoSortMods.zip
VideoSort modifications to support childEnVars and category destination directory and supporting scripts
(14.29 KiB) Downloaded 149 times

hugbug
Developer & Admin
Posts: 6710
Joined: 09 Sep 2008, 11:58
Location: Germany (NRW)

Re: [PP-Script] VideoSort - better video sorting

Post by hugbug » 28 Aug 2013, 11:55

Thanks for your suggestions and changes.

1. I'll provide a more convenient way to pass variables between scripts and extend VideoSort to pass the data you need.

2. the set_dest_dir can be much simpler. The files are already in category destination directory. You just need to take the parent path of NZBPP_DIRECTORY. Well, your code works slightly different - it use current destination only if category has a dest dir set. This can be achieved by comparing parent of NZBPP_DIRECTORY with NZBOP_DESTDIR. But I don't think it's necessary. I'll do it simple:
If MoviesDir/SeriesDir,etc. is empty the current destination will be used - which is CategoryX.DestDir if category was used or gloabl DestDir if no category was used.

BTW, I have made other improvements to the script and will post a new version soon.

minimeh
Posts: 32
Joined: 16 Aug 2013, 09:54

Re: [PP-Script] VideoSort - better video sorting

Post by minimeh » 28 Aug 2013, 19:23

hugbug wrote:Thanks for your suggestions and changes.
2. the set_dest_dir can be much simpler. The files are already in category destination directory.
That is not what I am seeing. VideoSort is putting the files into "Z:\Media\TV" when the expectation is that they be placed into "Z:\Media\dailycolbert".

I have a category called "dailycolbert". It has a DestDir of "Z:\Media\dailycolbert". The "Paths" "DestDir" is "Z:\Media". The "VideoSort" "SeriesDir" is "${DestDir}/TV". The "VideoSort" "DatedDir" is also "${DestDir}/TV".

To test and verify, I have reverted to the original VideoSort.py, turned on verbose, and removed any of my scripts from post processing. In other words, a stock installation. Then I dropped an NZB for a Daily Show into the NzbDir "${MainDir}/nzb/dailycolbert". I have attached the log snippet for the event for reference.

Nzbget downloaded the video using "dailycolbert" as the category into "Z:\Media\dailycolbert" (log lines 1362-1368). All good.

Code: Select all

Wed Aug 28 11:08:47 2013	INFO	Moving completed files for The.Daily.Show.2013.08.15.Simon.Pegg.720p.HDTV.x264-LMAO
Wed Aug 28 11:08:47 2013	INFO	Moving file the.daily.show.2013.08.15.simon.pegg.720p.hdtv.x264-lmao.nfo to Z:\Media\dailycolbert\The.Daily.Show.2013.08.15.Simon.Pegg.720p.HDTV.x264-LMAO
Wed Aug 28 11:08:47 2013	INFO	Moving file The.Daily.Show.2013.08.15.Simon.Pegg.720p.HDTV.x264-LMAO.nzb to Z:\Media\dailycolbert\The.Daily.Show.2013.08.15.Simon.Pegg.720p.HDTV.x264-LMAO
Wed Aug 28 11:08:47 2013	INFO	Moving file the.daily.show.2013.08.15.simon.pegg.720p.hdtv.x264-lmao.sample.mkv to Z:\Media\dailycolbert\The.Daily.Show.2013.08.15.Simon.Pegg.720p.HDTV.x264-LMAO
Wed Aug 28 11:08:49 2013	INFO	Moving file the.daily.show.2013.08.15.simon.pegg.720p.hdtv.x264-lmao.sample.srs to Z:\Media\dailycolbert\The.Daily.Show.2013.08.15.Simon.Pegg.720p.HDTV.x264-LMAO
Wed Aug 28 11:08:49 2013	INFO	Moving file the.daily.show.2013.08.15.simon.pegg.720p.hdtv.x264-lmao.srr to Z:\Media\dailycolbert\The.Daily.Show.2013.08.15.Simon.Pegg.720p.HDTV.x264-LMAO
Wed Aug 28 11:08:49 2013	INFO	Move for The.Daily.Show.2013.08.15.Simon.Pegg.720p.HDTV.x264-LMAO successful
Then VideoSort post processed and moved the video into "Z:\Media\TV"' (log lines 1384-1385):

Code: Select all

Wed Aug 28 11:08:49 2013	INFO	VideoSort: destination path: Z:\Media/TV\The Daily Show\The.Daily.Show.2013.08.15.mkv
Wed Aug 28 11:08:49 2013	INFO	VideoSort: Moved: Z:\Media/TV\The Daily Show\The.Daily.Show.2013.08.15.mkv
I assumed that was by design and so created the mods to serve my need. set_dest_dir could not assume that the files are already in the category destination directory because they aren't. :)
Attachments
nzbget_log.zip
Log of download and post processing
(7.79 KiB) Downloaded 112 times

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest