[New Feature] Quick par verification

Discuss newly added features or request new features.
hugbug
Developer & Admin
Posts: 7645
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: [New Feature] Quick par verification

Post by hugbug » 22 Aug 2014, 17:25

r1103:
  • when quick par verification is active the repaired files are not verified to save time;
  • the only reason for incorrect files after repair can be hardware errors (memory, disk) but this is not something NZBGet should care about

hugbug
Developer & Admin
Posts: 7645
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: [New Feature] Quick par verification

Post by hugbug » 18 Sep 2014, 16:53

prinz2311 wrote:If possible could nzbget cancel/kill the unrar process and trigger a par2 check as soon as it gets a CRC error or similar from unrar as a message? This could also make things faster...
Implemented in r1127.

prinz2311
Posts: 466
Joined: 08 Dec 2012, 00:03

Re: [New Feature] Quick par verification

Post by prinz2311 » 22 Sep 2014, 17:10

Shouldn't this errors also trigger a abort:

Code: Select all

... : packed data checksum error in volume ...

hugbug
Developer & Admin
Posts: 7645
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: [New Feature] Quick par verification

Post by hugbug » 22 Sep 2014, 17:26

It checks for " : packed data CRC failed in volume".
Looks like different unrar versions have different messages. I'll add it too.

hugbug
Developer & Admin
Posts: 7645
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: [New Feature] Quick par verification

Post by hugbug » 24 Sep 2014, 20:53

Now works with both unrar4 and unrar5.

Also added in r1130:
  • unpack is now immediately aborted if unrar reports wrong password (works for rar5 as well as for older formats);
  • unpack error status "PASSWORD" is now set for older formats (not only rar5).

prinz2311
Posts: 466
Joined: 08 Dec 2012, 00:03

Re: [New Feature] Quick par verification

Post by prinz2311 » 25 Sep 2014, 11:16

Shouldn't there be a par check be done, if a unrar fails?

Because I had multiple times that a unrar failed and then it was just send to PP with unrar:failed, instead of a par2 checking (and maybe a repair if possible) first.

hugbug
Developer & Admin
Posts: 7645
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: [New Feature] Quick par verification

Post by hugbug » 25 Sep 2014, 11:26

Yes, it should. Please send me the logs.

prinz2311
Posts: 466
Joined: 08 Dec 2012, 00:03

Re: [New Feature] Quick par verification

Post by prinz2311 » 25 Sep 2014, 13:46

I don't think it does a par2 check:

Code: Select all

info	Thu Sep 25 2014 15:33:02	Executing post-process-script
info	Thu Sep 25 2014 15:33:01	Cleanup for show successful
info	Thu Sep 25 2014 15:33:01	Deleting file something.par2
info	Thu Sep 25 2014 15:33:01	Cleaning up show
error	Thu Sep 25 2014 15:33:00	Unpack for show failed
warning	Thu Sep 25 2014 15:32:58	Interrupted unpack for show
warning	Thu Sep 25 2014 15:32:58	Cancelling unpack for show due to errors
info	Thu Sep 25 2014 15:32:58	Unrar: show.mkv : packed data checksum error in volume something.r08
It stops the unpack because of the error and then starts the cleanup/postprocessing without a par2 check.

It's probably the shitty astraweb servers at the moment. From the stats page, the download was quickchecked before unraring (probably) and no errors were found.

hugbug
Developer & Admin
Posts: 7645
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: [New Feature] Quick par verification

Post by hugbug » 25 Sep 2014, 14:48

Most likely the following has happened: the error occurred during unpack, then the par-check was performed and it was successful, then the second unpack attempts was made, which failed again. No another par-check was made after that.

I've had such case once. The quick verification has reported OK but the full verification (which I made via post-process again after changing settings) has found a damaged rar-file. I've inspected the files and there were about 30 bytes difference somewhere in the middle. The quick par-check is based on checksums calculated during download. If the news server would send wrong data the checksum would be different and the quick verification would found an error. I assume that one of following is true:
  • quick verification doesn't work properly; this is unlikely; the possibility that a bug in verification would result in a correct checksum match is very low;
  • checksum verification during download doesn't work properly; because the quick verification uses the expected checksum, which were read from the article, an error in the checksum verification during download may lead to a situation when the quick verification reports OK although there were download errors. I've checked the code and it looks OK, it's also simple and bugs are unlikely there;
  • as explained previously, the par-verification uses expected checksums; if the option CrcCheck is deactivated, NZBGet cannot be 100% sure if the articles are downloaded correctly; that wasn't the case for me because I have the option CrcCheck active (do you?);
  • memory management bug somewhere in NZBGet which leads to a memory corruption namely in the buffer of downloaded data before it is written; if such a bug exist it should lead to random crashes as well;
  • hardware error (memory or disk)
What I could do is to perform a full par-verification if unrar fails and a quick verification was already made. However it's merely a workaround whereas I'd like to find the cause of the problem.

prinz2311
Posts: 466
Joined: 08 Dec 2012, 00:03

Re: [New Feature] Quick par verification

Post by prinz2311 » 25 Sep 2014, 14:57

CRC Check is set to yes.

But it's the astraweb server. It has big problems for few days (uploaders know it). I had all the time "Could not read from TLS-Socket: Connection closed by remote host" in my log today. And after 2-3 re-downloads the download work without a error or need to repair.

I seen that before here too but that was also a time during the server gave errors like "Could not read from TLS-Socket: Connection closed by remote host". So I assume that because of the server errors the file is defective written to disk, but for some reason nzbget thinks it's ok.

I has gotten better the last few hours. So I would blame the server who sends incorrect data sometimes at the moment.

Post Reply

Who is online

Users browsing this forum: No registered users and 13 guests