[PP-Script / Scan-Script] Automatically Fetch Subtitles

Share your scripts or request scripts with specific features.
Forum rules
Please keep the forum clean - one topic per script. Questions not related to a specific script should be posted in Support forum.
l2g
Posts: 228
Joined: 27 Jun 2014, 22:13
Contact:

Re: [PP-Script] Automatically Fetch Subtitles

Post by l2g » 02 Sep 2014, 14:16

Ayla,
Thanks to the details you provided in an earlier post:
Ayla wrote:EDIT: OK, I got some info from the log file (searched for 'Rover'):
http://pastebin.com/Y7xfBSFd
I still can't explain why your missing the subtitles even though the logs in that same post show that you've gotten them. I did however just fix this (master branch only):
Ayla wrote:

Code: Select all

INFO Wed Aug 27 2014 11:30:04 Subliminal: Listing subtitles with provider 'opensubtitles' for video <Movie [u'The Rover', 2014]> with languages set([<Language [und]>])
It was a result of an earlier patch i attempted to prevent subtitles from downloading more then once (i still solved this, but the logs were unnessisarily noisy, and additional un-nessisary code was still being ran).

As per your most recent post:
Ayla wrote:That command doesn't produce any error messages.
This is a good thing for sure, it tells me you definitely have lxml installed. But the only way you can recieve the warning message "Skipped; lxml is not installed" is if that command actually did return an error message (from within the Subliminal.py; specifically it's pynzbget depdendancy). Perhaps your local environment has a different PYTHON_PATH defined then the one granted to NZBGet when it calls Subliminal.py? That's the only explanation i can give you at this time as to why it's working for you and not NZBGet.

Ayla
Posts: 21
Joined: 26 Aug 2014, 17:02

Re: [PP-Script] Automatically Fetch Subtitles

Post by Ayla » 03 Sep 2014, 12:00

Thanks for the help,

I'm still seeing this error a lot:

Code: Select all

Subliminal: Could not locate subtitle previously fetched!

Ayla
Posts: 21
Joined: 26 Aug 2014, 17:02

Re: [PP-Script] Automatically Fetch Subtitles

Post by Ayla » 04 Sep 2014, 10:25

Also, when I run Subliminal.py from commandline, it doesn't seem to work with multiple languages?

For example:

Code: Select all

/usr/local/bin/python /Users/martin/Library/Application\ Support/NZBGet/scripts/nzbget-subliminal/Subliminal.py -l en -l da -f -S /Volumes/TVshow/myVideo.mkv
- only downloads the last language specified (da - danish). If I only use "-l en" it downloads the english subtitle just fine.

I also tried "-m" for multiple, but it doesn't change anything.

Also, the subs are always downloaded as "<NAME>.srt", not "<NAME>.en.srt", "<NAME>.da.srt" etc.

l2g
Posts: 228
Joined: 27 Jun 2014, 22:13
Contact:

Re: [PP-Script] Automatically Fetch Subtitles

Post by l2g » 04 Sep 2014, 19:14

Ayla wrote:Also, when I run Subliminal.py from commandline, it doesn't seem to work with multiple languages?
Multiple subtitles (languages being part of the extension) is implied unless you add the -s (for single mode). Try -l en,da instead; ie:

Code: Select all

/usr/local/bin/python /Users/martin/Library/Application\ Support/NZBGet/scripts/nzbget-subliminal/Subliminal.py -l en,da -f -S /Volumes/TVshow/myVideo.mkv
Ayla wrote:Also, the subs are always downloaded as "<NAME>.srt", not "<NAME>.en.srt", "<NAME>.da.srt" etc.
I'll look into this to see deeper and get back to you if this is indeed the case. What your describing sounds like 'single mode'. Which will happen if you you specify the -s switch (which you didn't). Make sure you don't have the single mode option set in your NZBGet configuration incase it's coming from there. If a subtitle file already exists without the .lang.srt type, then future content is not downloaded (because it's assumed you already have it). Is it possible the .srt file was already downloaded (using single mode) from NZBGet preventing you from being able to fetch multple languages for the same file? In any case, try deleting the .srt file already associated with that video and run the above command again.

Ayla
Posts: 21
Joined: 26 Aug 2014, 17:02

Re: [PP-Script] Automatically Fetch Subtitles

Post by Ayla » 05 Sep 2014, 08:51

l2g wrote:
Ayla wrote:Also, when I run Subliminal.py from commandline, it doesn't seem to work with multiple languages?
Multiple subtitles (languages being part of the extension) is implied unless you add the -s (for single mode). Try -l en,da instead; ie:

Code: Select all

/usr/local/bin/python /Users/martin/Library/Application\ Support/NZBGet/scripts/nzbget-subliminal/Subliminal.py -l en,da -f -S /Volumes/TVshow/myVideo.mkv
'-l en,da' seems to work, thanks.
Ayla wrote:Also, the subs are always downloaded as "<NAME>.srt", not "<NAME>.en.srt", "<NAME>.da.srt" etc.
I'll look into this to see deeper and get back to you if this is indeed the case. What your describing sounds like 'single mode'. Which will happen if you you specify the -s switch (which you didn't). Make sure you don't have the single mode option set in your NZBGet configuration incase it's coming from there. If a subtitle file already exists without the .lang.srt type, then future content is not downloaded (because it's assumed you already have it). Is it possible the .srt file was already downloaded (using single mode) from NZBGet preventing you from being able to fetch multple languages for the same file? In any case, try deleting the .srt file already associated with that video and run the above command again.
I tried to remove the .srt and run Subliminal.py with the '-l en,da'.

It finds the subtitle and downloads it, but the file is not named '.en.srt' just '.srt':

Code: Select all

server:nzbget-subliminal martin$ /usr/local/bin/python /Users/martin/Library/Application\ Support/NZBGet/sc
ripts/nzbget-subliminal/Subliminal.py -l en,da -f -S /Volumes/Promise/TV\ Shows/Extant/Season\ 1/Extant\ -\
 s01e11\ -\ WEBDL-720p-ECI.mkv 
2014-09-05 10:48:26,071 - 68275 - INFO - Found 1 matched file(s).
2014-09-05 10:48:26,071 - 68275 - INFO - Using advanced search mode
2014-09-05 10:48:26,071 - 68275 - INFO - Using the following providers: podnapisi, opensubtitles, addic7ed, thesubdb, tvsubtitles
2014-09-05 10:48:26,071 - 68275 - WARNING - Addic7ed provider dropped due to missing credentials
2014-09-05 10:48:26,085 - 68275 - INFO - Scanning video 'Extant - s01e11 - WEBDL-720p-ECI.mkv' in '/Volumes/Promise/TV Shows/Extant/Season 1'
2014-09-05 10:48:26,429 - 68275 - ERROR - Embedded subtitle track name u'CC' is not a valid language
2014-09-05 10:48:26,672 - 68275 - INFO - Skipping provider 'thesubdb': no video to search for
2014-09-05 10:48:26,673 - 68275 - INFO - Listing subtitles with provider 'podnapisi' for video <Episode [u'Extant', 1x11]> with languages set([<Language [en]>, <Language [da]>])
2014-09-05 10:48:27,278 - 68275 - INFO - Found 3 subtitles
2014-09-05 10:48:27,278 - 68275 - INFO - Listing subtitles with provider 'opensubtitles' for video <Episode [u'Extant', 1x11]> with languages set([<Language [en]>, <Language [da]>])
2014-09-05 10:48:27,358 - 68275 - INFO - Found 3 subtitles
2014-09-05 10:48:27,358 - 68275 - INFO - Listing subtitles with provider 'tvsubtitles' for video <Episode [u'Extant', 1x11]> with languages set([<Language [en]>, <Language [da]>])
2014-09-05 10:48:27,376 - 68275 - INFO - Episode 11 not found
2014-09-05 10:48:27,377 - 68275 - INFO - Found 0 subtitles
2014-09-05 10:48:27,416 - 68275 - INFO - Computed score 37 with matches set([u'episode', u'season', u'series', u'resolution'])
2014-09-05 10:48:27,438 - 68275 - INFO - Computed score 37 with matches set([u'episode', u'season', u'series', u'resolution'])
2014-09-05 10:48:27,461 - 68275 - INFO - Computed score 37 with matches set([u'episode', u'season', u'series', u'resolution'])
2014-09-05 10:48:27,470 - 68275 - INFO - Computed score 35 with matches set([u'series', u'episode', u'season'])
2014-09-05 10:48:27,482 - 68275 - INFO - Computed score 35 with matches set([u'series', u'episode', u'season'])
2014-09-05 10:48:27,494 - 68275 - INFO - Computed score 35 with matches set([u'series', u'episode', u'season'])
2014-09-05 10:48:27,494 - 68275 - INFO - Downloading subtitle <PodnapisiSubtitle [en]> with score 37 into u'Extant - s01e11 - WEBDL-720p-ECI.srt'
2014-09-05 10:48:28,000 - 68275 - INFO - Matched 1 possible subtitle(s) for Extant - s01e11 - WEBDL-720p-ECI.mkv
2014-09-05 10:48:28,002 - 68275 - INFO - Successfully placed Extant - s01e11 - WEBDL-720p-ECI.srt

kloaknet
Posts: 337
Joined: 23 Jul 2014, 08:52

Re: [PP-Script] Automatically Fetch Subtitles

Post by kloaknet » 05 Sep 2014, 15:01

I2g, hope you had a nice weekend off :)

I did some days of testing, and with the 0.7.0 version I noticed that it also works for multiple folders, it still downloads subtitles, and the logging is way more clear :) (ow and when you will add the .en. part in the subs please only add that when multiple languages are listed in the settings :) )

l2g
Posts: 228
Joined: 27 Jun 2014, 22:13
Contact:

Re: [PP-Script] Automatically Fetch Subtitles

Post by l2g » 06 Sep 2014, 23:05

Ayla wrote:It finds the subtitle and downloads it, but the file is not named '.en.srt' just '.srt':
Your right on the ball with this bug... i fixed single mode in general. For some reason i just always set it to yes (when ran from the command line). It would never be 'off' per say which is why your tests were yeilding the results you saw. I also created an empty file called Extant-s01e11-WEBDL-720p-ECI.mkv and did not find any da subs for it though. But the good news is, the bug you found is fixed.
kloaknet wrote:ow and when you will add the .en. part in the subs please only add that when multiple languages are listed in the settings
Done! If single mode is set and you only have 1 language, then it will just download a .srt file. If you have more then 1 language set, i'll automatically disable single mode and download .lang1.srt .lang2.srt, etc.

Both of your detected bugs/requests were only pushed to the 'master' branch.. Thanks again for all your great efforts and help in locating Subliminal issues! :)

I notice recently the podnapisi provider seems a bit unstable. I always get that it had an unexpected error checking the download link. Not sure if you guys get it too.. Not that it's a problem, Sub fetching is still working great. But it's nice to know 'all' resources are being checked. Now i'm not so sure anymore. If you guys have the same prob let me know. This is way out of my jurisdiction and in the core of subliminal. But maybe it's something thats fixable... *shrugs*.

kloaknet
Posts: 337
Joined: 23 Jul 2014, 08:52

Re: [PP-Script] Automatically Fetch Subtitles

Post by kloaknet » 07 Sep 2014, 06:47

first, this is not the latests from the master branch, but from about 4 days ago:

Code: Select all

info	Sun Sep 07 2014 08:29:40	Subliminal: [DEBUG] NZBParse - Exception __init__() got an unexpected keyword argument 'tag'
error	Sun Sep 07 2014 08:29:40	Subliminal: NZBParse - NZB-File is corrupt: /opt/sybhttpd/localhost.drives/SATA_DISK/.nzb/tmp/blablablabla.nzb
it happens of course when in pp-mode only, didnt test much but in the releases I downloaded I got the same error. No problem downloading the file, no 'tag' argument in the NZB file. Subs are however just downloaded ok

the script version is dated 02/09/2014 06:51
subliminal folder 04/09/2014 15:34 <- but that will be the time I added the updated version, and as far as I could see, there was no update on the stuff in the folder.

updated to the newest version:
new version has the same 'issue'. I think it is some info that is added in NZBget using the RSS feed?

EDIT: After todays testing, indeed errors for podnapisi, but I had them in more times in the scheduler than in the pp option(didn't test those at the same time), other thing I noted is that often (maybe even always) no subs are found on addic7ed when the sub is still only available on addi7ed. When it becomes available on other sites too, the stuff is found also on addic7ed:

Code: Select all

info	Sun Sep 07 2014 15:34:35	Subliminal: ProviderError: Cannot find the download link
info	Sun Sep 07 2014 15:34:35	Subliminal: raise ProviderError('Cannot find the download link')
info	Sun Sep 07 2014 15:34:35	Subliminal: File "/share/Apps/NZBget/ppscripts/Subliminal/subliminal/providers/podnapisi.py", line 160, in download_subtitle
info	Sun Sep 07 2014 15:34:35	Subliminal: subtitle_text = provider.download_subtitle(subtitle)
info	Sun Sep 07 2014 15:34:35	Subliminal: File "/share/Apps/NZBget/ppscripts/Subliminal/subliminal/api.py", line 275, in download_best_subtitles
info	Sun Sep 07 2014 15:34:35	Subliminal: Traceback (most recent call last):
error	Sun Sep 07 2014 15:34:35	Subliminal: Unexpected error in provider u'podnapisi'
info	Sun Sep 07 2014 15:34:31	Subliminal: Downloading subtitle <PodnapisiSubtitle [en]> with score 43 into u'/share/Series/blabla.srt'
info	Sun Sep 07 2014 15:34:31	Subliminal: ProviderError: Cannot find the download link
info	Sun Sep 07 2014 15:34:31	Subliminal: raise ProviderError('Cannot find the download link')
info	Sun Sep 07 2014 15:34:31	Subliminal: File "/share/Apps/NZBget/ppscripts/Subliminal/subliminal/providers/podnapisi.py", line 160, in download_subtitle
info	Sun Sep 07 2014 15:34:31	Subliminal: subtitle_text = provider.download_subtitle(subtitle)
info	Sun Sep 07 2014 15:34:31	Subliminal: File "/share/Apps/NZBget/ppscripts/Subliminal/subliminal/api.py", line 275, in download_best_subtitles
info	Sun Sep 07 2014 15:34:31	Subliminal: Traceback (most recent call last):
error	Sun Sep 07 2014 15:34:31	Subliminal: Unexpected error in provider u'podnapisi'
info	Sun Sep 07 2014 15:34:26	Subliminal: Downloading subtitle <PodnapisiSubtitle [en]> with score 43 into u'/share/Series/blabla.srt'

Ayla
Posts: 21
Joined: 26 Aug 2014, 17:02

Re: [PP-Script] Automatically Fetch Subtitles

Post by Ayla » 08 Sep 2014, 13:06

l2g wrote:
Ayla wrote:It finds the subtitle and downloads it, but the file is not named '.en.srt' just '.srt':
Your right on the ball with this bug... i fixed single mode in general. For some reason i just always set it to yes (when ran from the command line). It would never be 'off' per say which is why your tests were yeilding the results you saw. I also created an empty file called Extant-s01e11-WEBDL-720p-ECI.mkv and did not find any da subs for it though. But the good news is, the bug you found is fixed.
kloaknet wrote:ow and when you will add the .en. part in the subs please only add that when multiple languages are listed in the settings
Done! If single mode is set and you only have 1 language, then it will just download a .srt file. If you have more then 1 language set, i'll automatically disable single mode and download .lang1.srt .lang2.srt, etc.

Both of your detected bugs/requests were only pushed to the 'master' branch.. Thanks again for all your great efforts and help in locating Subliminal issues! :)

I notice recently the podnapisi provider seems a bit unstable. I always get that it had an unexpected error checking the download link. Not sure if you guys get it too.. Not that it's a problem, Sub fetching is still working great. But it's nice to know 'all' resources are being checked. Now i'm not so sure anymore. If you guys have the same prob let me know. This is way out of my jurisdiction and in the core of subliminal. But maybe it's something thats fixable... *shrugs*.
Thanks for fixing the bugs :)

I think I have found a new one;

When I run the command below, it downloads a French sub (.fr) instead of 'en' or 'da', and places it next to the movie folder, not inside the movie folder (it doesn't happen with all movies, just this one).

Log:

Code: Select all

/usr/local/bin/python /Users/martin/Library/Application\ Support/NZBGet/scripts/nzbget-subliminal/Subliminal.py -l en,da -f -S /Volumes/Promise/Movies/Virtuosity\ \(1995\)/virtuosity.1995.multi.1080p.bluray.x264-rough.mkv 
2014-09-08 15:00:59,261 - 79940 - INFO - Found 1 matched file(s).
2014-09-08 15:00:59,261 - 79940 - INFO - Using advanced search mode
2014-09-08 15:00:59,261 - 79940 - INFO - Using the following providers: podnapisi, opensubtitles, addic7ed, thesubdb, tvsubtitles
2014-09-08 15:00:59,261 - 79940 - WARNING - Addic7ed provider dropped due to missing credentials
2014-09-08 15:00:59,272 - 79940 - INFO - Scanning video 'virtuosity.1995.multi.1080p.bluray.x264-rough.mkv' in '/Volumes/Promise/Movies/Virtuosity (1995)'
2014-09-08 15:00:59,695 - 79940 - INFO - Skipping provider 'tvsubtitles': no video to search for
2014-09-08 15:00:59,695 - 79940 - INFO - Listing subtitles with provider 'podnapisi' for video <Movie [u'Virtuosity', 1995]> with languages set([<Language [en]>, <Language [da]>])
2014-09-08 15:00:59,992 - 79940 - INFO - Found 0 subtitles
2014-09-08 15:00:59,992 - 79940 - INFO - Listing subtitles with provider 'thesubdb' for video <Movie [u'Virtuosity', 1995]> with languages set([<Language [en]>])
2014-09-08 15:01:00,206 - 79940 - INFO - Found 0 subtitles
2014-09-08 15:01:00,206 - 79940 - INFO - Listing subtitles with provider 'opensubtitles' for video <Movie [u'Virtuosity', 1995]> with languages set([<Language [en]>, <Language [da]>])
2014-09-08 15:01:00,273 - 79940 - INFO - Found 0 subtitles
2014-09-08 15:01:00,273 - 79940 - INFO - Listing subtitles with provider 'podnapisi' for video <Movie [u'Virtuosity', 1995]> with languages set([<Language [fr]>])
2014-09-08 15:01:00,342 - 79940 - INFO - Found 0 subtitles
2014-09-08 15:01:00,342 - 79940 - INFO - Listing subtitles with provider 'opensubtitles' for video <Movie [u'Virtuosity', 1995]> with languages set([<Language [fr]>])
2014-09-08 15:01:00,415 - 79940 - INFO - Found 1 subtitles
2014-09-08 15:01:00,440 - 79940 - INFO - Computed score 20 with matches set([u'title', u'year'])
2014-09-08 15:01:00,440 - 79940 - INFO - Downloading subtitle <OpenSubtitlesSubtitle [fr]> with score 20 into u'virtuosity.1995.multi.1080p.bluray.x264-rough.fr.srt'
2014-09-08 15:01:01,404 - 79940 - INFO - Matched 1 possible subtitle(s) for virtuosity.1995.multi.1080p.bluray.x264-rough.mkv
2014-09-08 15:01:01,404 - 79940 - ERROR - Could not locate a fetched (en) subtitle.
2014-09-08 15:01:01,404 - 79940 - ERROR - Could not locate a fetched (da) subtitle.

l2g
Posts: 228
Joined: 27 Jun 2014, 22:13
Contact:

Re: [PP-Script] Automatically Fetch Subtitles

Post by l2g » 08 Sep 2014, 14:40

Ayla wrote:When I run the command below, it downloads a French sub (.fr) instead of 'en' or 'da', and places it next to the movie folder, not inside the movie folder (it doesn't happen with all movies, just this one).
Wow, that is the absolute weirdest thing i've ever seen! I have absolutely no idea what's going on there. What happens if you turn off 'advanced' mode and use 'basic' instead? I wonder if enzyme's results are confusing subliminal into fetching other items. Enzyme is only used in advance mode; it actually opens up the movie and tries to detect existing subtitles already set.
kloaknet wrote:After todays testing, indeed errors for podnapisi
Yeah, as an update to this, there are several issues taking place here (i looked into it):
  1. podnapisi changed their web interface around a tiny bit causing the provider to be utterly useless now. I made a few changes on my machine to accomodate their changes. I haven't pushed these chagnes to Git yet because even after i make them, it doesn't work yet. The reason it's not working because of a second new problem i ran into identified below.
  2. they deny all web requests made to their server with an Agent set to Subliminal 7.4 when accessing the subtitles. They just return a 404 (Page Not Found) page right away now when this is done. Now don't get me wrong, this is very easily fixed. I could make it seem like we're sending requests from a specific version of Chrome, Firefox, IE, etc instead of Subliminal 7.4. But that seems like a shady/hacky and untrustwory solution to do. I'm not sure the best direction to take for this one.
    • Perhaps they just want people to create accounts on their server too (like Addi7ted did)? Maybe the provider needs to be changed to log in first *shrugs*.
    • Perhaps just contacting the author of the site directly might be the best option. I haven't looked at the aproach the original author of subminal has taken.
Whatever was done with podnapisi; it was done just recently as this service worked fine several weeks ago.
kloaknet wrote:I noted is that often (maybe even always) no subs are found on addic7ed when the sub is still only available on addi7ed. When it becomes available on other sites too, the stuff is found also on addic7ed
I'll look into this, i'm not sure i experience the same problem or not (but i could be wrong). Addic7ed seems to work beautifuly for me right now.

Post Reply

Who is online

Users browsing this forum: No registered users and 31 guests