nzbget history tab getting wiped

Get help, report and discuss bugs.
Post Reply
void.pointer
Posts: 60
Joined: 28 Sep 2014, 20:58

nzbget history tab getting wiped

Post by void.pointer » 30 May 2015, 16:10

Using latest trunk build of nzbget on linux, my download history tab is not showing anything. I think after each restart or something the list is getting cleared. This never used to happen before.

I checked my nzbget `queue` directory and it's full of files named after numbers like "19225".

Any tips on diagnosing this? Not sure what info to provide.

hugbug
Developer & Admin
Posts: 7645
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: nzbget history tab getting wiped

Post by hugbug » 30 May 2015, 16:19

Check the log-file for errors.
If you download something now (so that the history isn't empty), then restart. Is empty?

void.pointer
Posts: 60
Joined: 28 Sep 2014, 20:58

Re: nzbget history tab getting wiped

Post by void.pointer » 06 Jun 2015, 16:52

Well I finally encountered this with useful logs. Looks like the OS crashed (Ubuntu) and I had to reboot forcefully. On startup, I got these errors in the log:

Code: Select all

Sat Jun  6 11:41:35 2015        ERROR   Error reading server info from disk
Sat Jun  6 11:41:35 2015        ERROR   Error reading diskstate for file /home/robert/nzbget/queue/stats
Sat Jun  6 11:41:35 2015        ERROR   Could not load diskstate due to file version mismatch
My history tab is empty and I had ongoing downloads in my queue as well. Is this recoverable?

hugbug
Developer & Admin
Posts: 7645
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: nzbget history tab getting wiped

Post by hugbug » 06 Jun 2015, 21:19

File "stats" holds downloaded volume statistics. The history is in file "queue", which also holds download queue.
Are there other errors shown for that file?
If the file is corrupted it's content is most likely trashed. You can check it in a text editor, it's a plain text file.

When saving queue state NZBGet tries to do that in a safe manner but OS crashes can be fatal. I recommend making backups of queue-directory. The easiest way is to put a line into nzbget start up script, something like this:

Code: Select all

#!/bin/sh
zip -r queue-$(date +%Y%m%d_%H%M%S).zip /path/to/queuedir
nzbget -D

void.pointer
Posts: 60
Joined: 28 Sep 2014, 20:58

Re: nzbget history tab getting wiped

Post by void.pointer » 06 Jun 2015, 21:51

I don't see any other errors. The stat and queue files are both empty when I cat them.

This seems really fragile. Is there no way to make operations more atomic to avoid corruption, or perhaps create copies prior to writing or something of that nature?

Thanks for the help.

void.pointer
Posts: 60
Joined: 28 Sep 2014, 20:58

Re: nzbget history tab getting wiped

Post by void.pointer » 06 Jun 2015, 21:54

EDIT:

Here is my upstart script that does the backup. So far this is working.

Code: Select all

description "NZBGet upstart script"

setuid robert
setgid robert

start on runlevel [2345]
stop on runlevel [016]

respawn

expect fork

pre-start script
    cd /home/robert/nzbget
    exec tar -cvzf queue-$(date +%Y%m%d_%H%M%S).tar.gz queue
end script

script
    exec nzbget -D
end script

pre-stop script
    exec nzbget -Q
end script
Last edited by void.pointer on 06 Jun 2015, 22:07, edited 2 times in total.

hugbug
Developer & Admin
Posts: 7645
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: nzbget history tab getting wiped

Post by hugbug » 06 Jun 2015, 22:06

void.pointer wrote:This seems really fragile. Is there no way to make operations more atomic to avoid corruption, or perhaps create copies prior to writing or something of that nature?
The operations are atomic and copies are written too. The problem is the OS reports success and nzbget deletes old files. If in that moment the system crashes (not nzbget crash, that'd be OK) the system caches or hard drive caches may not be written to disk yet. OS may hold written data in memory for performance reason and this is transparent for programs.

If you have often system crashes there must be a hardware problem to fix.

void.pointer
Posts: 60
Joined: 28 Sep 2014, 20:58

Re: nzbget history tab getting wiped

Post by void.pointer » 06 Jun 2015, 22:08

hugbug wrote:
void.pointer wrote:This seems really fragile. Is there no way to make operations more atomic to avoid corruption, or perhaps create copies prior to writing or something of that nature?
The operations are atomic and copies are written too. The problem is the OS reports success and nzbget deletes old files. If in that moment the system crashes (not nzbget crash, that'd be OK) the system caches or hard drive caches may not be written to disk yet. OS may hold written data in memory for performance reason and this is transparent for programs.

If you have often system crashes there must be a hardware problem to fix.
I see, I thought it was a higher level problem you were describing. Fair enough. This machine is a piece of sh** anyway, so no biggies. Thank you for your help!!!

void.pointer
Posts: 60
Joined: 28 Sep 2014, 20:58

Re: nzbget history tab getting wiped

Post by void.pointer » 13 Jun 2015, 15:10

Interesting, I am seeing an entirely different way of the queue being cleared. This time it isn't errors:

Image

hugbug
Developer & Admin
Posts: 7645
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: nzbget history tab getting wiped

Post by hugbug » 13 Jun 2015, 19:03

Not "queue cleared" but history. It's option "KeepHistory" working.

Post Reply

Who is online

Users browsing this forum: Google [Bot] and 46 guests