Page 1 of 1

nzbget history tab getting wiped

Posted: 30 May 2015, 16:10
by void.pointer
Using latest trunk build of nzbget on linux, my download history tab is not showing anything. I think after each restart or something the list is getting cleared. This never used to happen before.

I checked my nzbget `queue` directory and it's full of files named after numbers like "19225".

Any tips on diagnosing this? Not sure what info to provide.

Re: nzbget history tab getting wiped

Posted: 30 May 2015, 16:19
by hugbug
Check the log-file for errors.
If you download something now (so that the history isn't empty), then restart. Is empty?

Re: nzbget history tab getting wiped

Posted: 06 Jun 2015, 16:52
by void.pointer
Well I finally encountered this with useful logs. Looks like the OS crashed (Ubuntu) and I had to reboot forcefully. On startup, I got these errors in the log:

Code: Select all

Sat Jun  6 11:41:35 2015        ERROR   Error reading server info from disk
Sat Jun  6 11:41:35 2015        ERROR   Error reading diskstate for file /home/robert/nzbget/queue/stats
Sat Jun  6 11:41:35 2015        ERROR   Could not load diskstate due to file version mismatch
My history tab is empty and I had ongoing downloads in my queue as well. Is this recoverable?

Re: nzbget history tab getting wiped

Posted: 06 Jun 2015, 21:19
by hugbug
File "stats" holds downloaded volume statistics. The history is in file "queue", which also holds download queue.
Are there other errors shown for that file?
If the file is corrupted it's content is most likely trashed. You can check it in a text editor, it's a plain text file.

When saving queue state NZBGet tries to do that in a safe manner but OS crashes can be fatal. I recommend making backups of queue-directory. The easiest way is to put a line into nzbget start up script, something like this:

Code: Select all

#!/bin/sh
zip -r queue-$(date +%Y%m%d_%H%M%S).zip /path/to/queuedir
nzbget -D

Re: nzbget history tab getting wiped

Posted: 06 Jun 2015, 21:51
by void.pointer
I don't see any other errors. The stat and queue files are both empty when I cat them.

This seems really fragile. Is there no way to make operations more atomic to avoid corruption, or perhaps create copies prior to writing or something of that nature?

Thanks for the help.

Re: nzbget history tab getting wiped

Posted: 06 Jun 2015, 21:54
by void.pointer
EDIT:

Here is my upstart script that does the backup. So far this is working.

Code: Select all

description "NZBGet upstart script"

setuid robert
setgid robert

start on runlevel [2345]
stop on runlevel [016]

respawn

expect fork

pre-start script
    cd /home/robert/nzbget
    exec tar -cvzf queue-$(date +%Y%m%d_%H%M%S).tar.gz queue
end script

script
    exec nzbget -D
end script

pre-stop script
    exec nzbget -Q
end script

Re: nzbget history tab getting wiped

Posted: 06 Jun 2015, 22:06
by hugbug
void.pointer wrote:This seems really fragile. Is there no way to make operations more atomic to avoid corruption, or perhaps create copies prior to writing or something of that nature?
The operations are atomic and copies are written too. The problem is the OS reports success and nzbget deletes old files. If in that moment the system crashes (not nzbget crash, that'd be OK) the system caches or hard drive caches may not be written to disk yet. OS may hold written data in memory for performance reason and this is transparent for programs.

If you have often system crashes there must be a hardware problem to fix.

Re: nzbget history tab getting wiped

Posted: 06 Jun 2015, 22:08
by void.pointer
hugbug wrote:
void.pointer wrote:This seems really fragile. Is there no way to make operations more atomic to avoid corruption, or perhaps create copies prior to writing or something of that nature?
The operations are atomic and copies are written too. The problem is the OS reports success and nzbget deletes old files. If in that moment the system crashes (not nzbget crash, that'd be OK) the system caches or hard drive caches may not be written to disk yet. OS may hold written data in memory for performance reason and this is transparent for programs.

If you have often system crashes there must be a hardware problem to fix.
I see, I thought it was a higher level problem you were describing. Fair enough. This machine is a piece of sh** anyway, so no biggies. Thank you for your help!!!

Re: nzbget history tab getting wiped

Posted: 13 Jun 2015, 15:10
by void.pointer
Interesting, I am seeing an entirely different way of the queue being cleared. This time it isn't errors:

Image

Re: nzbget history tab getting wiped

Posted: 13 Jun 2015, 19:03
by hugbug
Not "queue cleared" but history. It's option "KeepHistory" working.