Added functionality for nzbget: a pet project

Get help, report and discuss bugs.
placebodk

Added functionality for nzbget: a pet project

Post by placebodk » 12 Dec 2004, 13:56

Hi there,

I've added some functionality to the code for nzbget, since I felt that the code didn't meet my requirements for using with www.newzbin.com.

More specificly I've exchanged the colored frontend with libncurses, I've added a server mode, and last but not least a download rate limit.

I dont know how these things normally work with sourceforge, but since the code had good structure and was easy to read this was more or less straightforward.

If anyone want's to try it out, it can be found on my server at: http://files.interiority.dk/nzbget-0.1.2.1.tar.gz

I'm unsure wether or not this should make it into source-forge project download, since I've had to re-arrange some functionality and hence probably would break other pet-projects (like the apple-gui-addon)

Any comments, problems, or likewise with this nzbget-addon can be directed to: nzbget_at_interiority_dot_dk

The code has been tested on my AMD64 with gentoo, and my VIA-Epia with Debian - I use it often when downloading from www.newzbin.com. Have no idea if this works on apple as well.

/Placebodk

pmorch

RE: Added functionality for nzbget: a pet project

Post by pmorch » 13 Dec 2004, 07:25

FANTASTIC! FEDT NOK!

I was just about to go home and add the download rate limit tonight, but you beat me to it! Great!

I'll try it out tonight...

Peter

pmorch

RE: Added functionality for nzbget: a pet project

Post by pmorch » 13 Dec 2004, 17:37

Hi there, placebodk...

I've tried it out and have a few things to note:

* First off, I am so happy with the bandwidth limit!

* The ncurses frontend is really hot when run in an xterm!

Excellent work!!!

I also have gripes, and I hope they are taken in the constructive manner they are intended. I realize you've built what *you* need (so would I! :-D), but maybe some of these things can be fixed.

* Does nzbget now *always* run with the ncurses frontend? It seems that --outputmode and the corresponding option is completely ignored... I can easily see that the ncurses interface looks really cool in an xterm, but it would be nice to choose myself (see point below about "screen"). And if the option is to be completely disregarded, it should be removed from --help, nzbget.cfg.example and from option parsing code.

* Is there no way to run in the original "batch" mode and is client/server now the *only* way to run? I suggest "nzbget --server" to start the server and "nzbget --queue <nzbfile>" add to the server queue. Especially since client/server mode requires root to install it into hard-coded /usr/bin/nzbget (for now). Even with that being a config-option, I'd still like to be able to run in batch mode. (Whether or not to make it the default? I don't care about that.)

* Glad to see, that if the downloadrate option is missing it downloads "as fast as possible"!

* Most times when creating an executable you can run it in the creation directory with ./executable. However, line 95 in nzbget.ccp now contains
iKey = ftok( "/usr/bin/nzbget", 0 );
And so it now *requires* to be installed exactly there. Not very flexible... Also, it now requires installation by root, since the path is fixed and /usr/bin/ is writable by root only. (I hope!!) Also, it took me a while to figure out that was what went wrong, so maybe a test for the existance of /usr/bin/nzbget and approrpiate error message would have been in order. Also what if the nzbget found by $PATH and /usr/bin/nzbget are not the same? (I realize I don't know myself). At the very least, make the final location used by ftok an option in the configuration file.

* nzbget-0.1.2.1 behaves as if <option name="appendpostnametodir">no</option> regardless of yes or no. When a standard vanilla nzbget-0.1.2 is given an .nzb file and appendpostnametodir is yes, it downloads stuff into a sub-directory under option destinationdir, not into destinationdir itself (at least with the newzbin nzb files I've done so far).

* the ncurses interface does not work well with screen, which I use. Do you know and/or use screen? I like it because it doesn't require me to have an xterm running with nzbget in it, but allows me to ssh into the box and connect to the running nzbget whereever I am. But with the ncurses interface under screen, the frequent screen refreshes are *very* visible and don't look too hot. Especially with ssh over the internet. Probably worse the slower the connection... (Of course the bandwith limitation option helps here! Hurray!). For this reason alone, I'd like a choice of frontends.

* It seems that when a client does nzbget <nzbfile> and the filename is relative, the path is considered relative to the server, not the client. So if server is in dir S and client is in dir C and there is a file.nzb both places, when client does nzbget file.nzb, that is really S/file.nzb which is counterintuitive. If S/file.nzb is missing then nzbget file.nzb of even nzbget ./file.nzb fails...

* A server or daemon that has a working directory that is not / is a bad idea. It prevents unmounting of any volume containing the daemon's current directory. Did you know that? A daemon or server process should cd to /. (Correct me if I'm wrong)

* How to exit the server cleanly? CTRL-C sometimes generates a segmentation fault.

* I noticed that I now needed to install libncurses-dev and libuu-dev on Debian to find ncurses.h and libuu.a. Now it says "skipping non-compatible uulib/libuu.a" (or something similar - the verbatim message is gone...). Maybe ./configure should check for that instead of compilation failing - no biggie for me and I don't know configure (automake?) that well...


placebodk

RE: Added functionality for nzbget

Post by placebodk » 13 Dec 2004, 19:22

First of all, I've renamed the file on my server to: http://files.interiority.dk/nzbget-0.1.2-alt1.tar.gz, with the "alt1" indicating that it's an alternative version and not at all officially accepted in any way.

pmorch, thanx for the critisism - I do appreciate it.

1) Should be possible to run with option -m logable

2) Last time I checke before I made the .tar it was possible to download a single .nzb file normally with "nzbget <nzb-file>" (ie. as opposed to "nzbget" without the trailing .nzb file)

However trying it out just now it didnt work so obviously something went wrong. Having the server mode, I never download a single .nzb file myself. Will look into it.

3) I use System V IPC to communicate between the server-nzbget and the client-nzbget's, hence the need for the ftok (gives a "unique" key for an application). IPC has some cool features, but I wonder if it's the wisest of choices for this application.

For 1. iteration I'll leave it out when calling nzbget in "normal" mode (ie. not running in server/client mode)

But youre right, currently it's very unflexible for anything but having it installed at /usr/bin/nzbget

4) Regarding "appendpostnametodir" - will look into it. I remember it worked for me at some time, but can't remember when it broke.

5) I know & use screen, though not for this application - which would explain why I haven't stumbled upon this problem. I'll try it out with the -m logable option and see how it works out.

6) Regarding filenames relative to server's & clients - again, I've never experienced this since I open the .nzb files directly with nzbget through Firefox (ie. I click on the "get nzb files" and uses "open with nzbget") - this works fine for me since FireFox apparently stores the file in /tmp/some-nzb-file.nzb (hence using a full path).

Will look into sending over an absolute path from the client to the server (instead of a relative one).

7) "A server or daemon that has a working directory..." - had no idea but thanx for the heads-up :o)

8) "CTRL-C sometimes generates a segmentation fault" - sounds nasty. If you see any pattern or figure out how to reproduce please mail me.

Will look into the above problems (some are easier done than others) and get it stable.

/Placebodk

placebodk

Code revisited & stabilized

Post by placebodk » 14 Dec 2004, 20:49

New version uploaded to my server - still an alternative, and not officially sanctioned, accepted or anything like that. Can be found at: http://files.interiority.dk/nzbget-0.1.2-alt2.tar.gz

What has been done:

* Now both of the previous 2 frontends (logable & colored) works, and in addition there's the new curses frontend. Use "-m logable", "-m colored" or "-m curses" to specify either one. Just as it is possible with nzbget-0.1.2 it's also possible here to specify which frontend to use in the .cfg file.

This should make it possible to run with screen, or over slow connections.

* The original single-file download of an .nzb file should be working. To download a single .nzb file without envoking the "server/client" mode, use nzbget -n <nzb-file>. If nzbget <nzb-file> is executed without an nzbget server running in the background, it will figure this out and likewise download the indicated .nzb file.

* To start a server run nzbget with the -s option (ie. nzbget -s). A server will also be started if nzbget is exectuted without giving an <nzb-file>.

* Calls to nzbget <nzb-file> while a server is running should automaticly put the <nzb-file> on the servers queu (unless the -n is used, in which case it will explicitly ignore any servers running)

* The program now calculates the absolute path for the <nzb-file> which means that there shouldnt be any problems with where the <nzb-file> is located relative to either the client or the server.

* The append-postname-to-dir seemed to have an explicit filter for only working with files looking like "msgid_123456_SomeName.nzb". Extended the code to also create dirs for other .nzb files - this might cause trouble, but I can't think of any at this moment.

* The bug causing the segmentation faults should now have been squashed.

* Minor bugfixes

I think that's it for this time. If anyone is silly enough to use this alternative version, and runs into problems, errors or similar I can be contacted at: nzbget_at_interiority_dot_dk - or use the source-forge account which will arrive in the same mail-box.

/placebodk

PS. Was thinking a little bit about exchanging the System V IPC with sockets. This would in a perfect world make it possible to have a server running on one computer and from another computer (at work or elsewhere) contact the server and queue up downloads. Would also make it easier to make a GUI for it. Oh well...

pmorch

RE: Added functionality for nzbget: a pet project

Post by pmorch » 25 Dec 2004, 02:08

Sorry, Pacebodk, I forgot subscribe to this forum, so I haven't seen your updates till now. I'll give them a try ASAP. (Not now, though... - I'm sure you can guess why!)

I think these changes are really great, and should be "sanctioned" - checked into CVS etc. I wonder whats happened to the old maintainer. If you're around - what do *you* think?

If "you" aren't around, maybe we should try contacting Sourceforge to have this project "opened"....

God Jul!

Peter


pmorch

RE: Added functionality for nzbget: a pet project

Post by pmorch » 26 Dec 2004, 23:13

Great! I've done a little testing of it, and I have some comments:

Great that the colored frontend is back! It is missing some of the information it used to have, though. There used to be a line for each download thread so one could see what each thread was doing. More importantly, though, it used to print out when it had actually assembled a <file> from all the <segments>. That would be nice to get back... Would that be difficult?

There seemed to be some problems with duplicated logic in Options::initSpecials() and Coordinator::GetDestinationDir() when running with -n.

In Options::initSpecials() there was logic to calculate the OPTION_REALDESTDIR based on a nzb file name. Of course, if server, at this point there isn't yet a nzb file name, so this logic can't apply. Instead it is applied in Coordinator::GetDestinationDir(). But when running as -n, the nzb file is present and so the logic applied both in Options::initSpecials() and later too, resulting in wrong directory names such as /path/download/postname/postname/file.ext. I removed that logic from Options::initSpecials() so the logic now only gets applied once.

In Coordinator::GetDestinationDir, if OPTION_APPENDDIR was no, '.' was used so -d and config file's destinationdir were ignored. Isn't this wrong? Instead, now OPTION_REALDESTDIR is used.

Finally, I suggest we start changing VERSION in Makefile.in

In the new version I've made, I changed VERSION to 0.1.2-alt3 in keeping with your naming convention. I've included a -v and --version option to print out the current version.

Perhaps this is getting to be a developer thread... :-D

I've placed another new version containing this at:
http://www.morch.com/misc/nzbget/nzbget ... lt3.tar.gz
Diff between this and alt2 is:
http://www.morch.com/misc/nzbget/nzbget ... alt3.patch

Peter

tincanfury

RE: Added functionality for nzbget: a pet pro

Post by tincanfury » 08 Jan 2005, 17:46

agreed, adding the version info would be nice.

second, I would like to be able to run the server on one machine and pass it the nzb file from another. right now I use nzbq this way, but having it all in nzbget would be nicer.

it would also be nice to have a command to kill the server so that I can set it up as a service.

also, i get the "cannot start server, nzbget already running" even though its not. i don't know how you keep track of it, but its definitly not running.

also, built in par2 would be nice (ie have it run a par2 check before downloading the volxx.PAR2 files to see how many, if any, are needed and just download whats needed, then run par2repair)

and since I can't get the server now, one thing I know I'd like to be able to do with the server running is get the current download status(file, transfer speed, etc) somehow.

Thanks, hopefully one of the old developers can open this back up and give you control, you've done some great and much needed work!

hbokh

RE: Added functionality for nzbget: a pet pro

Post by hbokh » 12 Jan 2005, 18:39

Anyone managed to get this to compile on FreeBSD?
I keep on getting configure: error: "no pthread-library found".

On FreeBSD libpthread is in /usr/local/lib/pth/

hbokh

RE: Added functionality for nzbget: a pet pro

Post by hbokh » 12 Jan 2005, 19:03

Nevermind...

./configure --with-libxml2-libraries=/usr/local/lib --with-libxml2-includes=/usr/local/include/libxml2 CPPFLAGS="-I/usr/local/include/pth -I/usr/local/include" LDFLAGS="-L/usr/local/lib/pth"

will make a Makefile.
Running make gives "Makefile", line 138: Need an operator make: fatal errors encountered -- cannot continue

While nzbget v0.1.1 runs on FreeBSD, I guess this is Linux-only :-(

Post Reply

Who is online

Users browsing this forum: No registered users and 46 guests