[PP-Script / Scan-Script] Automatically Fetch Subtitles

Share your scripts or request scripts with specific features.
Forum rules
Please keep the forum clean - one topic per script. Questions not related to a specific script should be posted in Support forum.
wesyr
Posts: 3
Joined: 29 May 2015, 20:56

Re: [PP-Script / Scan-Script] Automatically Fetch Subtitles

Post by wesyr » 29 May 2015, 21:01

Hi :)
Thanks for this script, i've been searching for something like this for quite a while :)
but i've encountered a problem...
My config:
RPi2 + xbian
Installed NZBGet, Sonarr, Couchpotato.
python version 2.7.3

When i launch this script like this:

Code: Select all

python Subliminal.py -f -S /media/USB/Movies/ -l en -D
And when he founds a subtitle it throws this:

Code: Select all

2015-05-29 22:54:07,510 - 30506 - WARNING - Podnapisi page matches: 1
2015-05-29 22:54:07,643 - 30506 - ERROR - Unexpected error in provider 'podnapisi'
Traceback (most recent call last):
  File "Subliminal/subliminal/api.py", line 256, in download_best_subtitles
    provider_subtitles = provider.list_subtitles(video, provider_video_languages)
  File "Subliminal/subliminal/providers/podnapisi.py", line 322, in list_subtitles
    year=video.year)]
  File "Subliminal/subliminal/providers/podnapisi.py", line 266, in query
    soup = bs4.BeautifulSoup(multi_release['data-content'], ['permissive'])
  File "Subliminal/bs4/element.py", line 905, in __getitem__
    return self.attrs[key]
KeyError: u'data-content'
2015-05-29 22:54:07,726 - 30506 - WARNING - No subtitles were found for Jurassic Park III.mkv
Can't find any solution in google :(
I'm running this script right now as root.
Can some help ? :)

l2g
Posts: 228
Joined: 27 Jun 2014, 22:13
Contact:

Re: [PP-Script / Scan-Script] Automatically Fetch Subtitles

Post by l2g » 31 May 2015, 00:06

wesyr wrote:Hi :)
i've encountered a problem...
Can some help ? :)
@wesyr: As it turns out podnapisi changed their site once again breaking the plugin. I've updated it to work with the subtle changes they made and just incremented the version too since the script is stable again. The fix can be found in the link below.

Release v0.9.6 (Direct Download Link) in place.

The release details can be found at the head of this thread (quick link).

wesyr
Posts: 3
Joined: 29 May 2015, 20:56

Re: [PP-Script / Scan-Script] Automatically Fetch Subtitles

Post by wesyr » 31 May 2015, 18:38

l2g wrote:
wesyr wrote:Hi :)
i've encountered a problem...
Can some help ? :)
@wesyr: As it turns out podnapisi changed their site once again breaking the plugin. I've updated it to work with the subtle changes they made and just incremented the version too since the script is stable again. The fix can be found in the link below...
Cheers @l2g, works like a charm :)
Thank You ! :)

wesyr
Posts: 3
Joined: 29 May 2015, 20:56

Re: [PP-Script / Scan-Script] Automatically Fetch Subtitles

Post by wesyr » 03 Jun 2015, 04:27

Hi :)
I've found 2 more problems :)

1.

Code: Select all

python Subliminal.py -f -S /media/USBHDD/TV/iZombie/ -l en -b -p opensubtitles,podnapisi
Results:

Code: Select all

2015-06-03 06:25:17,339 - 5627 - INFO - Found 12 matched file(s).
2015-06-03 06:25:17,340 - 5627 - INFO - Using basic search mode
2015-06-03 06:25:21,416 - 5627 - INFO - Listing subtitles with provider 'opensubtitles' for video <Episode [u'iZombie', 1x2]> with languages set([<Language [en]>])
2015-06-03 06:25:21,530 - 5627 - INFO - Found 3 subtitle(s) on opensubtitles
2015-06-03 06:25:21,531 - 5627 - INFO - Listing subtitles with provider 'podnapisi' for video <Episode [u'iZombie', 1x2]> with languages set([<Language [en]>])
2015-06-03 06:25:26,761 - 5627 - INFO - Found 5 subtitle(s) on podnapisi
2015-06-03 06:25:26,858 - 5627 - ERROR - Fatal Exception:
  Traceback (most recent call last):
    File "Subliminal/nzbget/ScriptBase.py", line 2367, in run
    exit_code = main_function(*args, **kwargs)
    File "Subliminal.py", line 1471, in main
    use_nzbheaders=False,
    File "Subliminal.py", line 1104, in subliminal_fetch
    hi_score_adjust=hi_score_adjust,
    File "Subliminal/subliminal/api.py", line 273, in download_best_subtitles
    for s in subtitles], key=operator.itemgetter(1), reverse=True):
    File "Subliminal/subliminal/subtitle.py", line 105, in compute_score
    initial_matches = self.compute_matches(video)
    File "Subliminal/subliminal/providers/opensubtitles.py", line 55, in compute_matches
    sanitize_string(self.series_name) == \
    File "Subliminal/subliminal/providers/opensubtitles.py", line 43, in series_name
    return self.series_re.match(self.movie_name).group('series_name')
  AttributeError: 'NoneType' object has no attribute 'group'
2.

Code: Select all

2015-06-03 06:21:32,871 - 2807 - ERROR - Unexpected error in provider 'tvsubtitles'
Traceback (most recent call last):
  File "Subliminal/subliminal/api.py", line 256, in download_best_subtitles
    provider_subtitles = provider.list_subtitles(video, provider_video_languages)
  File "Subliminal/subliminal/providers/tvsubtitles.py", line 178, in list_subtitles
    return [s for s in self.query(video.series, video.season, video.episode) if s.language in languages]
  File "Subliminal/subliminal/providers/tvsubtitles.py", line 162, in query
    show_id = self.find_show_id(series)
  File "Subliminal/dogpile/cache/region.py", line 1013, in decorate
    should_cache_fn)
  File "Subliminal/dogpile/cache/region.py", line 640, in get_or_create
    async_creator) as value:
  File "Subliminal/dogpile/core/dogpile.py", line 158, in __enter__
    return self._enter()
  File "Subliminal/dogpile/core/dogpile.py", line 98, in _enter
    generated = self._enter_create(createdtime)
  File "Subliminal/dogpile/core/dogpile.py", line 149, in _enter_create
    created = self.creator()
  File "Subliminal/dogpile/cache/region.py", line 612, in gen_value
    created_value = creator()
  File "Subliminal/dogpile/cache/region.py", line 1009, in creator
    return fn(*arg, **kw)
  File "Subliminal/subliminal/providers/tvsubtitles.py", line 107, in find_show_id
    soup = self.request('/search.php', data=data, method='POST')
  File "Subliminal/subliminal/providers/tvsubtitles.py", line 89, in request
    r = self.session.request(method, self.server + url, params=params, data=data, timeout=10)
  File "Subliminal/requests/sessions.py", line 456, in request
    resp = self.send(prep, **send_kwargs)
  File "Subliminal/requests/sessions.py", line 559, in send
    r = adapter.send(request, **kwargs)
  File "Subliminal/requests/adapters.py", line 375, in send
    raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='www.tvsubtitles.net', port=80): Max retries exceeded with url: /search.php (Caused by ReadTimeoutError("HTTPConnectionPool(host='www.tvsubtitles.net', port=80): Read timed out. (read timeout=10)",))

l2g
Posts: 228
Joined: 27 Jun 2014, 22:13
Contact:

Re: [PP-Script / Scan-Script] Automatically Fetch Subtitles

Post by l2g » 13 Jun 2015, 14:19

wesyr wrote:Hi :)
I've found 2 more problems :)
Sorry for the delay in response time; the site for some reason didn't notify me that there was an update to this thread.. In any case:

1. Is this still happening? Unfortunately I don't know the filename it's crashing on, just the detected entries from it. Perhaps you could run the command again; but include the --debug (or -D) switch? That would greatly help me out and will let me dig further as to what might be the cause of this situation.

2. This is just a result of http://www.tvsubtitles.net being unavailable at the time of the query; there isn't much you can do about this. The sites go offline and back online now and then. Sometime they're busy and throw back an error like this too. Or, if you make to many requests from the site within a short timeframe, sometimes they just block you momentarily. Usually trying again later will resolve it.

kloaknet
Posts: 337
Joined: 23 Jul 2014, 08:52

Re: [PP-Script / Scan-Script] Automatically Fetch Subtitles

Post by kloaknet » 16 Aug 2015, 07:20

What about an option to check within the MaxAge window for items with a higher score, so the script might update to the correct .srt file after like a proper version with different length etc is released?

So you basically need to save the score of the last .srt downloaded, and when within MaxAge a better one pops up, that one is stored, and the otherone is overwritten :).

Note that it barely happens that a sub isn't right, and this is mostly related that the sub didn't come from addic7ed, but... :oops: Hopes it doesnt require a completely rewrite of your code, and if so, just ignore this idea :D

l2g
Posts: 228
Joined: 27 Jun 2014, 22:13
Contact:

Re: [PP-Script / Scan-Script] Automatically Fetch Subtitles

Post by l2g » 23 Aug 2015, 03:52

kloaknet wrote:What about an option to check within the MaxAge window for items with a higher score, so the script might update to the correct .srt file after like a proper version with different length etc is released?

So you basically need to save the score of the last .srt downloaded, and when within MaxAge a better one pops up, that one is stored, and the other one is overwritten :).

Note that it barely happens that a sub isn't right, and this is mostly related that the sub didn't come from addic7ed, but... :oops: Hopes it doesn't require a completely rewrite of your code, and if so, just ignore this idea :D
Everything you said makes perfect sense. It does involve some signifigant work on the subliminal core (backend) again though... But the pynzbget framework I wrote has an SQLite database controller built into it (by just using the set() and get() functions). So I could easily hash the video entries and their respected subtitle score to it and check them the next time to determine if we need to download it again.

But yeah... The subliminal core would have to be completely reworked to fetch content in 2 phases (instead of 1). First it would have to get the scores and return them (so I could work with them) and then I'd have to pass the scores back into a second phase of it to do the actual fetching (if required).

Honesty though, does the scoring of subtitles change significantly enough that it would warrant this overhaul? I've seldom got a crappy sub, and if i do get one, i delete it and run the tool from the CLI again (it usually fixes the problem). Absolute worst case (maybe once every 3 or 4 months), i have to go download the file manually. Anyway.... I'm definitely not saying this is a bad idea (it's a really good one actually); but I just can't seem to justify the effort level involved. This script is pretty solid in it's current state now...

You thoughts?

kloaknet
Posts: 337
Joined: 23 Jul 2014, 08:52

Re: [PP-Script / Scan-Script] Automatically Fetch Subtitles

Post by kloaknet » 24 Aug 2015, 14:34

pfff, whole message gone for the 2nd time, the forum guru is having meditation issues again :D

but in short, when you have nothing to do on a cold rainy winterday or something, you could add it :D. A lot of people don't even understand why 'some guy is adding a scene name to a file', so they just scrap it from the file to have nice looking folder, and don't get why they can't find matching subs (and the modification won't be of use anyway).

EnZioBe
Posts: 30
Joined: 18 Oct 2015, 14:37

Re: [PP-Script / Scan-Script] Automatically Fetch Subtitles

Post by EnZioBe » 18 Oct 2015, 15:16

I'm a totally new user of NZBGet, but one of the features that I would dearly like to use is the automated subtitles downloading.

I have installed NZBGet on Windows 10 and have also installed Subliminal. However, it doesn't work for me. I can get it to work using the CLI, but not with NZBGet.
If it's useful to know, I have set the execution order for scripts as: nzbgetpp-master\unzip.py, videosort\VideoSort.py, Subliminal.py.

l2g
Posts: 228
Joined: 27 Jun 2014, 22:13
Contact:

Re: [PP-Script / Scan-Script] Automatically Fetch Subtitles

Post by l2g » 18 Oct 2015, 16:32

EnZioBe,

Makes sure both the Subliminal directory and Subliminal.py are next to each other in your C:\Path\To\NZBGet\scripts directory. Hence it should look like this:
  • C:\Path\To\NZBGet\scripts\Subliminal.py
  • C:\Path\To\NZBGet\scripts\Subliminal
It sounds like you might already have this part done because you said it works from the Command line.

The next thing you need to do is configure NZBGet to call the script after it downloads content and/or have it run on a schedule once or twice a day (is what most people do who use this tool). If you properly placed the files, then you should be able to configure all of this through the NZBGet Settings (from within the application).

Configure a Subliminal
  • In the Settings of NZBGet; click on Subliminal - Options (on the left hand side).
  • I mostly use the default settings personally, except for 2 changes I make:
    • MovieProviders = opensubtitles, podnapisi, thesubdb
    • TVShowProviders = podnapisi, addic7ed
  • Click on Subliminal - Post-Processing Mode (on the left hand side). Make any changes that you want. I personally just use the default settings.
  • Click on Subliminal - Scheduler Mode (on the left hand side).
  • If you plan on using this mode, this is a very important tab to click on! You must specify the ScanDirectories. Put as many as you want (separate them with a space/comma) hence: C:\Path1, \\Network\Path, X:\Path2, etc.
Configure a Subliminal Scheduler
Assuming you've configured Subliminal with some paths to scan (in it's configuration):
  • In the Settings of NZBGet; click on Scheduler (on the left hand side).
  • Create an entry with the optional settings:
    • TaskX.Time = 18:35,11:00
    • TaskX.Weekdays = 1-7
    • TaskX.Command = Script
    • TaskX.Param = Subliminal.py
  • You're done; save your settings, and this task will twice a day (6:35pm and 11:00am), 7 days a week. The scheduled times are intentionally 'way' outside of times content is available in because that allows the people who create the subtitles to... well... create them (so they exist for you).
Configure a Subliminal Post-Processing
This will have Subliminal run after each download against the contents downloaded. This doesn't always work; but not for reasons of a bug. Some content is just so new that there simply aren't subtitles yet (they'll be available in a few hours...). So consider setting up a ScanScript.
However, it still doesn't hurt to use this anyway, it works great for older content!

The easiest way is this (but more advance users might want to use categories instead and only apply this to video files). However... for those who want a quick and dirty solution:
  • In the Settings of NZBGet; click on Extension Scripts (on the left hand side).
  • Make sure that the PostScript entry (first one i believe) has Subliminal.py defined in it. If not, click the Choose button and add it.
Categories is another way to go (NZBGet is so powerful this way), as it can process content differently depending on the category the download was placed in. Some people make a Category for Video specific content. In these cases, you only need to apply Subliminal.py to these categories. Subliminal.py isn't that stupid either, if it detects a directory or download containing non-video files, it just won't do anything. So it's really not that much extra overhead to just go with the basic approach above. If you're just starting out... stick with the info above to get you started.

Good luck

Post Reply

Who is online

Users browsing this forum: No registered users and 32 guests