[Queue-Script] Stream video while downloading

Share your scripts or request scripts with specific features.
Forum rules
Please keep the forum clean - one topic per script. Questions not related to a specific script should be posted in Support forum.
Post Reply
BlkChockr
Posts: 10
Joined: 24 Jul 2015, 21:24

[Queue-Script] Stream video while downloading

Post by BlkChockr » 24 Jul 2015, 21:43

EDIT: For those who come from google with great hope: http://forum.nzbget.net/viewtopic.php?f ... 353#p15353 :D

----------------------------------- Original Post ------------------------------------------------

Hey Folks,

I am currently working on a Queue-Script which shall extract videos from .rar files while there are still parts remaining in the queue.

I already tested this by creating a multipart-rar-archive with some .mkv in it. It seems that WinRAR is "able" to update/append to extracted files, but does so by completely replacing the file.

Now I came along with a workaround, which would call WinRAR/Unrar every time a .rar file is downloaded and extract all data into a temp file, read the byte length from the existing file, substract that from the temp file and append the remaining data to the existing one. This way, it is possible to open the video in VLC (for example), when only one file of a multipart-rar-archive has been downloaded while continously adding data to the file/stream, so that you can watch a video while downloading it. :)

My problem with putting this nice behaviour in a Queue-Script is that I need the file name of the .rar as well as the number in the queue, to be able to tell wether it is the first file (or the last) and pass the right file to WinRAR/Unrar. Can anyone help me to get that information (maybe add it to os.environ, @hugbug? :mrgreen: )?

I'm currently testing this on Windows, but could imagine a version for OSMC, etc. as well.

Here's what I have so far (work in progress, dont hate :cry: ):

Code: Select all

##############################################################################
### NZBGET QUEUE SCRIPT 												   ###

### QUEUE EVENTS: FILE_DOWNLOADED

# Stream video while downloading.
#
# This script partially extracts video from .rar files
# so that you can view them while downloading.
#
# NOTE: This script requires Python to be installed on your system.

### NZBGET QUEUE SCRIPT 												   ###
##############################################################################

import sys, os

UNRAR_PATH = "C:\\Program Files\\WinRAR\\UnRAR.exe"

def appendFile(source, destination):
	with open(source, "rb") as s, open(destination, "a+b") as d:
		s.seek(os.path.getsize(destination))
		chunk = "."
		while chunk != "":
			chunk = s.read(1024)
			d.write(chunk)

appendFile("PATH_TO_TEMP_FILE", "PATH_TO_EXISTING_FILE");
Last edited by BlkChockr on 29 Dec 2015, 14:38, edited 2 times in total.

JVM
Posts: 83
Joined: 21 Oct 2013, 05:11

Re: [Queue-Script] Stream video while downloading

Post by JVM » 25 Jul 2015, 06:51

Nice idea! Check out the code for FakeDetector, I think you may find a fair bit of it useful. It is a queue/pp script.

The function list_all_rars(dir) is used to call unrar on each new file downloaded. You might be able to change the line

Code: Select all

command = [unrar(), "vb", dir + '/' + file]
to get unrar to extract the latest file.

The function sort_inner_files() moves the last rar file to top of queue. From memory, the variables file_num, file_id, file_name refer to the last .rar in the set. You could use a similar method to get the first.

BlkChockr
Posts: 10
Joined: 24 Jul 2015, 21:24

Re: [Queue-Script] Stream video while downloading

Post by BlkChockr » 25 Jul 2015, 19:24

Thank you for the information! It was really useful to me and I made quite a bit of progress today, but now I came across the next problem: How do I get the password to unrar the file? It does not seem to be in os.environ? :shock:

hugbug
Developer & Admin
Posts: 7645
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: [Queue-Script] Stream video while downloading

Post by hugbug » 25 Jul 2015, 19:37

BlkChockr wrote:How do I get the password to unrar the file? It does not seem to be in os.environ?
NZBPR__UNPACK_PASSWORD

Script to print all variables:

Code: Select all

for k, v in os.environ.items():
	if k.startswith('NZB'):
		print "%s=%s" % (k, v)

BlkChockr
Posts: 10
Joined: 24 Jul 2015, 21:24

Re: [Queue-Script] Stream video while downloading

Post by BlkChockr » 27 Jul 2015, 18:15

God, I have to be blind to not see that. Thank you! :oops:

Next Problem:

I am trying to find the largest file in the archive, to eliminate samples. This works well when I call my script from the command-line, but when NZBGet tries to use it, it does only seem to get the first line:

Code: Select all

UNRAR 5.21 freeware      Copyright (c) 1993-2015 Alexander Roshal
Usually it should look like this:

Code: Select all

UNRAR 5.21 freeware      Copyright (c) 1993-2015 Alexander Roshal

Rarset created with uprep v 0.1.1 

Archive: D:\Desktop\download\intermediate\somenzb\somearchive.part01.rar
Details: RAR 4, volume, recovery record, lock, encrypted headers

        Name: somevideo.mkv
        Type: File
        Size: 10362845297
 Packed size: 149954694
       Ratio: -->
       mtime: 2014-05-17 11:44,000
  Attributes: -rw-rw-r--
  Pack-CRC32: 70E4ECA9
     Host OS: Unix
 Compression: RAR 3.0(v29) -m0 -md=128K
       Flags: encrypted 
Any ideas?

Besides that, I had some trouble with RAR5, or more specifically: An archive with packed files in it (.r00, .r01, etc.) Is there any easy way to extract these partically? 7zip for example can show the inner video-file directly. WinRAR only show the .r0X files. :?

BlkChockr
Posts: 10
Joined: 24 Jul 2015, 21:24

Re: [Queue-Script] Stream video while downloading

Post by BlkChockr » 28 Jul 2015, 18:56

Maybe the code can tell you whats wrong:

Code: Select all

def findVideoFile(rarFile):
	try:
		out = ''
		command = [getUnrarExecutable(), "vt"]
		if 'NZBPR__UNPACK_PASSWORD' in os.environ:
			command += ['-p'+str(os.environ['NZBPR__UNPACK_PASSWORD'])]
		command += [rarFile]
		command += VIDEO_EXTENSIONS
		proc = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
		out_tmp, err = proc.communicate()
		out += out_tmp
		lines = out.splitlines()
		files = {}
		currentName = ''
		for line in lines:
			print line
			arr = line.strip().split(': ')
			if arr[0] == 'Name':
				currentName = arr[1]
			if arr[0] == 'Size':
				files[currentName] = int(arr[1])
		# assuming biggest video file is the one we want
		files = sorted(files, key=files.get, reverse=True)
		if files:
			return files[0]
		else: 
			print('[ERROR] No video file found in rar-archive!')
	except Exception as e:
		print('[ERROR] Failed %s: %s' % (file, e))
getUnrarExecutable() is the unrar()-Method from FakeDetector. :)

VIDEO_EXTENSIONS (self-explaining):

Code: Select all

# https://en.wikipedia.org/wiki/Video_file_format
VIDEO_EXTENSIONS = ["*.webm", "*.mkv", "*.flv", "*.vob", "*.ogv", "*.ogg",
					"*.drc", "*.mng", "*.avi", "*.mov", "*.qt", "*.wmv",
					"*.yuv", "*.rm", "*.rmvb", "*.asf", "*.mp4", "*.m4p",
					"*.m4v", "*.mpg", "*.mp2", "*.mpeg", "*.mpe", "*.mpv",
					"*.m2v", "*.svi", "*.3pg", "*.3g2", "*.mxf", "*.roq", "*.nsv"]
I might also open a github-repository for this and upload the complete code, if you want me to... ;)

BlkChockr
Posts: 10
Joined: 24 Jul 2015, 21:24

Re: [Queue-Script] Stream video while downloading

Post by BlkChockr » 29 Dec 2015, 14:37

UPDATE: Its done! https://github.com/BlkChockr/nzbget-stream I'm looking forward to your feedback and pull-requests ;)

wtf911
Posts: 2
Joined: 08 Apr 2017, 19:26

Re: [Queue-Script] Stream video while downloading

Post by wtf911 » 08 Apr 2017, 19:28

I just wanted to say that this script rocks and needs more attention! :)

Post Reply

Who is online

Users browsing this forum: No registered users and 23 guests