Fast downloads but slow unpack

Get help, report and discuss bugs.
Post Reply
seannymurrs
Posts: 7
Joined: 19 Dec 2018, 11:10

Fast downloads but slow unpack

Post by seannymurrs » 05 Aug 2020, 00:06

I’m running NZBGet on my seedbox through Whatbox.ca. I’m very happy with my download speeds; they are typically between 300-500MB/s. Even though I’m on one of their SSD slots, however, I’m finding that unpacking seems to take quite a long time. Most of what I’m downloading are quite large files (4K Remux files >50GB), so I know it’s not going to be instantaneous. That being said, I still feel like it’s slower than I should be. Up until now, I’ve had all the “direct” options enabled (directwrite, directrename, directunpack, etc...). I’ve noticed that, unless the download is going slower than normal, I don’t actually see any evidence of directunpack actually happening. I’m assuming this is because the file is being downloaded fast enough that it doesn’t have an opportunity to start unpacking (though I could be completely wrong on this). What I’m wondering, is whether there are any specific settings to help in situations where my download speed is plenty fast, but unpacking seems to be the definite bottleneck. While I will occasionally have a large queue of files (if I’m trying to get an entire season of a specific tv show), most of the time I’m downloading only one or two things at once. Any suggestions would be greatly appreciated.

hugbug
Developer & Admin
Posts: 7645
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: Fast downloads but slow unpack

Post by hugbug » 06 Aug 2020, 09:44

Unpack is performed by external tool "unrar". For a test you could disable unpack in nzbget completely, then unpack manually using unrar and compare the results (unpack time).

seannymurrs
Posts: 7
Joined: 19 Dec 2018, 11:10

Re: Fast downloads but slow unpack

Post by seannymurrs » 06 Aug 2020, 16:37

Is it possible to see the unpack time for an individual NZB? I’m sure I could check the time stamps for when it started/finished unpacking, but I wasn’t sure if that info was easily viewable somewhere.

hugbug
Developer & Admin
Posts: 7645
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: Fast downloads but slow unpack

Post by hugbug » 06 Aug 2020, 16:52

Click on an item in history. Then on label "Total ->" in statistics box.

seannymurrs
Posts: 7
Joined: 19 Dec 2018, 11:10

Re: Fast downloads but slow unpack

Post by seannymurrs » 06 Aug 2020, 21:16

hugbug wrote:
06 Aug 2020, 16:52
Click on an item in history. Then on label "Total ->" in statistics box.
Got it, thanks. I downloaded a few large files, and below are their sizes and unpack times. To be honest, I don't know what is considered "good" or "bad" speeds for an SSD when it comes to unpacking. My results are confusing, to say the least. There seems to be a really wide range of times, and it doesn't seem to be directly related to the size of the file. Is there anything else I can look at to try and figure out what's going on?

45.0 GB, 0:03:35 Unpack time
47.2 GB, 0:08:50 Unpack time
50.8 GB, 0:31:32 Unpack time
15.5 GB, 0:00:15 Unpack time
49.0 GB, 0:54:35 Unpack time

sanderj
Posts: 184
Joined: 10 Feb 2014, 21:46

Re: Fast downloads but slow unpack

Post by sanderj » 07 Aug 2020, 08:29

seannymurrs wrote:
06 Aug 2020, 21:16
45.0 GB, 0:03:35 Unpack time
47.2 GB, 0:08:50 Unpack time
50.8 GB, 0:31:32 Unpack time
15.5 GB, 0:00:15 Unpack time
49.0 GB, 0:54:35 Unpack time
15.5 GB, 0:00:15 Unpack time ... so 15 GB in 15 seconds, or 1GB/s = 1000 MB/s ... which is very fast.
- A M.2 SATA SSD can achieve 500 MB/s or so.
- A M.2 NVME SSD can indeed achieve Gigabytes per second: Samsung’s 970 Evo Plus seems to do 2500 MB/s
So ... what kind of SSD have you got? Is 1000 MB/s possible at all? If not, it must be a bad measurement or presentation in NZBget (unlikely, but still).

Then:
45.0 GB, 0:03:35 Unpack time
49.0 GB, 0:54:35 Unpack time

Same size, very different unpack times (almost factor 20 !). Nzbget's "Unpack time" is the pure unpacking, not repairing (which can take quite some time), so I can only guess ... disk problem?

So: If you retry the same 45 resp 49 GB downloads, do you get the same Unpack Time? In other words: is it reproducible?

hugbug
Developer & Admin
Posts: 7645
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: Fast downloads but slow unpack

Post by hugbug » 07 Aug 2020, 09:56

"Unpack time" is the pure length of unpack post-processing stage. When direct unpack is enabled the file can be almost completely unpacked when the unpack post-processing stage starts. In that case the reported "Unpack time" can be very short.

When unpack is performed simultaneously with other activities (options UnpackPauseQueue, DirectUnpack, PostStrategy) then we can't really compare the numbers. Other activities (download, repair) may take a lot of CPU and disk resources.

Some releases use rar only as container, the compression is disabled. Such archives can be unpacked very fast, as fast as copying a file. Other releases may have compression enabled, then the unpack requires a lot of resources to decode compressed data.

To determine if unpack times in nzbget are ok the same archives must be unpacked using unrar tool manually for comparison (on the same computer). That's what I suggested in my first reply in this topic.

Post Reply

Who is online

Users browsing this forum: Google [Bot] and 52 guests