[New Feature] Unpack during downloading

Discuss newly added features or request new features.
defcon_three
Posts: 6
Joined: 14 Jan 2018, 11:20
Location: Germany

Re: [New Feature] Unpack during downloading

Post by defcon_three » 14 Jan 2018, 12:02

This is a really awesome feature. Thank you for implementing this, hugbug!
I tested this feature and it is working fine.

In my use case I will use this feature for an "instant streaming" of media files.
E.g. I download a 30GB media file with 100 Mbit/s in 45min.
Now I can instantly watch the file without having a waiting time of 45min.

Concerning this use case two questions are arising:
1st: I am using post-process scripts to add downloaded media files to my synology media index.
If I understood correctly the post-process script will not handle the temporary unpacked files.
Which kind of script do I need to add temporary unpacked media files to my synology media index?

2nd: If I understood correctly the temporary unpacked media file will be deleted/moved after finishing download.
Does this means that a streaming of the temporary unpacked media file will be stopped after finishing download since the file has been deleted?
How is it possible to "prevent" the deletion/movement of temporary unpacked media files?

hugbug
Developer & Admin
Posts: 7645
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: [New Feature] Unpack during downloading

Post by hugbug » 14 Jan 2018, 17:14

For post-processing scripts there is no difference.

Whether the streaming continues to work after unpack is finished depends on video player software. On my Linux PVR box the player continues playing (moved) file without issues (both the player and nzbget run on the same machine). In my tests on Windows the default media player worked fine whereas VLC stopped playback (both the player and nzbget run on the same machine).

badfolder
Posts: 1
Joined: 06 Apr 2018, 22:46

Re: [New Feature] Unpack during downloading

Post by badfolder » 06 Apr 2018, 23:00

I made myself a queue script which plays the currently downloading video in Kodi on another box. If Kodi is currently playing another video, it adds the new video to the playing queue so Kodi will play it afterwards. The fact that NZBGet unpacks the files in a subdirectory and moves the files later when it's done unpacking turns out to be a huge problem for my intended use case since Kodi might try to play the video in the queue when NZBGet finished unpacking it and won't be able to find it in the path my script has passed before. So it would be great if you could implement an option to change that behaviour.

hugbug
Developer & Admin
Posts: 7645
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: [New Feature] Unpack during downloading

Post by hugbug » 07 Apr 2018, 00:27

Unpacking directly into destination directory is prone to errors:
- unpacking is a long process; if something goes wrong the incomplete destination file may remain. When unpack starts again it is difficult for nzbget to decide what to do with the existing file. Moreover since unpacking is made by a separate tool nzbget doesn't have full control over the process, letting the tool to unpack into a directory which may be nonempty may cause issues.
- after unpacking files are usually renamed. Even if nzbget would unpack directly into destination directory the renaming stage (VideoSort or Sonarr/Radarr) will change the file name.

That's why I chose to unpack into a temporary directory, to make the whole process more robust. Sadly that prevents certain use cases.

Maybe you could do the following, if Kodi allows play list control from scripts: after the download is completed your script removes the temporary item from kodi play list and adds the completed item.

habskilla
Posts: 13
Joined: 29 Oct 2014, 13:50

Re: [New Feature] Unpack during downloading

Post by habskilla » 18 May 2018, 13:28

Simply amazing!

Code: Select all

Downloaded size	42.7 GB
Download speed	22.2 MB/s
Total time	0:33:02
Download time	0:32:51
Verification time	0:00:00
Repair time	0:00:00
Unpack time	0:00:03

isarrider
Posts: 29
Joined: 11 May 2015, 12:20

Re: [New Feature] Unpack during downloading

Post by isarrider » 03 Jul 2018, 08:42

It might be good to state in the description it works best with filesystem that support sparse files (so not exfat ;) )

hugbug
Developer & Admin
Posts: 7645
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: [New Feature] Unpack during downloading

Post by hugbug » 03 Jul 2018, 09:02

isarrider wrote:
03 Jul 2018, 08:42
It might be good to state in the description it works best with filesystem that support sparse files (so not exfat ;) )
There is nothing special about sparse files in regard to direct unpack feature.

Uukrul
Posts: 2
Joined: 15 Oct 2018, 23:18

Re: [New Feature] Unpack during downloading

Post by Uukrul » 16 Oct 2018, 02:31

Would it be possible to add an option that checks the downloaded files before unpacking? I ask because, some times, I've found nzbs that have a file with an error and DirectUnpack fails, so all unpacked files are deleted, NZBGet has to wait until the download is finished, check and repair the files and unpack them again.

And that is a problem with slow hardware if the uploader had the "brilliant idea" of creating a rar with the compression method set to "best" :roll:

I did a test with the built-in NNTP server, a 1GB mkv compressed with Winrar and the best compression, created the nzb and changed some bytes of .part4 to force an unraring error (see attached log).

With such an option, NZBGet would pause the unpacking if one of the parts has an error (this has to be a full verification because quick won't work since the download is correct) and continue it after the download is repaired.

I also can provide an nzb for a download with that kind of error.

Thanks in advance.
Attachments
best.log
(9.76 KiB) Downloaded 360 times

hugbug
Developer & Admin
Posts: 7645
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: [New Feature] Unpack during downloading

Post by hugbug » 16 Oct 2018, 10:32

Uukrul wrote:
16 Oct 2018, 02:31
Would it be possible to add an option that checks the downloaded files before unpacking?
The unpack isn't started if there are errors and is immediately interrupted if download errors occur. Pausing unpacking on a download error and continuing unpacking after repair would make the already complex system even more complex and this isn't something I can handle.

Uukrul
Posts: 2
Joined: 15 Oct 2018, 23:18

Re: [New Feature] Unpack during downloading

Post by Uukrul » 16 Oct 2018, 10:47

Fair enough :)

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest