nzbget 13.0-testing-r1053

Announcements about new stable and testing releases.
Subscribe to the forum for e-mail notifications.
Forum rules
This forum is readonly and is intended to inform users about new releases.
You can subscribe to the forum to receive e-mail notifications.
Locked
hugbug
Developer & Admin
Posts: 6746
Joined: 09 Sep 2008, 11:58
Location: Germany (NRW)

nzbget 13.0-testing-r1053

Post by hugbug » 06 Jul 2014, 13:43

This is a candidate for stable branch. Please report all issues.

nzbget 13.0-testing-r1053

Changes since 13.0-testing-r1049
  • improved RPC-API:
    • history items now preserve "NZBID" from queue items;
    • that makes the tracking of items across queue and history easier for third-party apps;
    • field "NZBID" returned by RPC-method "history" is now available for history items of all kinds (NZB, URL, DUP);
    • field "ID" is deprecated and should not be used
  • pp-script "EMail.py" now takes the status of previous pp-scripts into account and report a failure if any of the scripts has failed;
  • removed libpar2-patches from NZBGet source tree; the documentation now suggests to use the libpar2 version maintained by Debian/Ubuntu team, which already includes all necessary patches; also removed patches to create libpar2 and libsigc++ project files for Visual Studio on Windows, no one needed them anyway;
  • fixed: current download could be damaged if the program was restarted during download and the option "ContinuePartial" was active (bug introduced in v13).
Other changes since nzbget 12.0
  • reworked download queue:
    • queue now holds nzb-jobs instead of individual files (contained within nzbs);
    • this drastically improves performance when managing queue containing big nzb-files on operations such as pause/unpause/move items;
    • tested with queue of 30 nzb-files each 40-100GB size (total queue size 1.5TB) - queue managing is fast even on slow device;
    • limitation: individual files (contained within nzbs) now cannot be moved beyond nzb borders (in older version it was possible to move individual files freely and mix files from different nzbs, although this feature was not supported in web-interface and therefore was not much known);
    • this change opens doors for further speed optimizations and integration of download queue with post-processing queue and possibly url-queue;
    • current download data such as remained size or size of paused files is now internally automatically updated on related events (download of article is completed, queue edited, etc.);
    • this eliminates the need of calculating this data upon each RPC-request (from web-interface) and greatly decrease CPU load of processing RPC-requests when having large download queue (and/or large nzb-files in queue);
    • field "Priority" was removed from individual files;
    • instead nzb-files (collections) now have field "Priority";
    • nzb-files now also have new fields "MinTime" and "MaxTime", which are set when nzb-file is parsed and then kept;
    • this eliminates the need of recalculation file statistics (min and max priority, min and max time);
    • removed action "FileSetPriority" from RPC-command "editqueue";
    • removed action "I" from remote command "--edit/-E" for individual files (now it is allowed for groups only);
    • removed few (not more necessary) checks from duplicate manager;
    • merged post-processing queue into main download queue;
    • changing the order of (pp-queued) items in the download queue now also means changing the order of post-processing jobs;
    • priorities of downloads are now respected when picking the next queued post-processing job;
    • the moving of download items in web-interface is now allowed for downloads queued for post-processing;
    • removed actions of remote command "--edit/-E" and of RPC-method "editqueue" used to move post-processing jobs in the post-processing queue (the moving of download items should be used instead);
    • remote command "-E/--edit" and RPC-method "editqueue" now use NZBIDs of groups to edit groups (instead of using ID of any file in the group as in older versions);
    • remote command "-L/--list" for groups (G) and group-view in curses-frontend now print NZBIDs instead of "FirstID-LastID";
    • RPC-method "listgroups" returns NZBIDs in fields "FirstID" and "LastID", which are usually used as arguments to "editqueue" (for compatibility with existing third-party software);
    • items queued for post-processing and not having any remaining files now can be edited (to cancel post-processing), which was not possibly before due to lack of "LastID" in empty groups;
    • edit commands for download queue and post-processing queue are now both use the same IDs (NZBIDs);
    • merged url queue into main download queue;
    • urls added to queue are now immediately shown in web-interface;
    • urls can be reordered and deleted;
    • when urls are fetched the downloaded nzb-files are put into queue at the positions of their urls;
    • this solves the problem with fetched nzb-files ordered differently than the urls if the fetching of upper (position wise) urls were completed after the lower urls;
    • removed options "ReloadUrlQueue" and "ReloadPostQueue" since there are no separate url- and post-queues anymore;
    • nzb-files added via urls have new field "URL" which can be accessed via RPC-methods "listgroups" and "history";
    • new env. var. "NZBNP_URL", "NZBNA_URL" and "NZBPP_URL" passed to scan-, queue- and pp-scripts;
    • removed remote command "--list U", urls are now shown as groups by command "--list G";
    • RPC-method "urlqueue" is still supported for compatibility but should not be used since the urls are now returned by method "listgroups", the entries have new field "Kind" which can be "NZB" or "URL";
  • added collecting of download volume statistics data per news server:
    • in web-interface the data is shown as chart in "Statistics and Status" dialog;
    • new RPC-method "servervolumes" returns the collected data;
    • new RPC-method "resetservervolume" to reset the custom counter;
    • for screenshots see topic [New Feature] Downloaded volume statistics
  • fast par-renamer now automatically detects and renames misnamed (obfuscated) par2-files;
  • for downloads not having any (obviously named) par2-files the critical health is assumed 85% instead of 100% as the absense of par2-files suggests;
    • this avoids the possibly false triggering of health-check action (detele or pause) for downloads having misnamed (obfuscated) par2-files;
    • combined with improved fast par-renamer this provides proper processing of downloads with misnamed (obfuscated) par2-files;
  • fast par-renamer now detects missing files (files listed in par2-files but not present on disk):
    • when checking for missing files the files whose extensions match with option "ExtCleanupDisk" are ignored now (to avoid time consuming restoring of files which will be deleted later anyway);
    • added option "ParIgnoreExt" which lists files which do not trigger par-repair if they are missing (similar to option "ExtCleanupDisk" but those files are not deleted during cleanup);
  • added new choice "Always" for option "ParCheck":
    • it forces the par-check for every (even undamaged) download but in contrast to choice "Force" only one par2-file is downloaded first;
    • additional files are downloaded if needed;
  • improved par-check for damaged collections with multiple par-sets and having missing files:
    • only orphaned files (not belonging to any par-set) are scanned when looking for missing files;
    • this greatly decrease the par-check time for big collections;
  • eliminated the distinction between manual pause and soft-pause;
    • there is only one pause register now;
    • options "ParPauseQueue", "UnpackPauseQueue" and "ScriptPauseQueue" do not change the state of the pause but instead are respected directly;
    • RPC-methods "pausedownload2" and "resumedownload2" are aliases to "pausedownload" and "resumedownload" (kept for compatibility);
    • field "Download2Paused" of RPC-method "status" is an alias to "DownloadPaused" (kept for compatibility);
    • action "D2" of remote commands "--pause/-P" and "--unpause/-U" is not supported anymore;
  • implemented general scripts concept:
    • the concept is a logical extension of the post-processing scripts concept initially introduced in v11;
    • the general scripts concept applies to all scripts used in the program: scan-script, queue-script and scheduler-script (in addition to post-processing scripts);
    • option "NzbProcess" renamed to "ScanScript";
    • option "NzbAddedProcess" renamed to "QueueScript";
    • option "DefScript" and "CategoryX.DefScript" renamed to "PostScript" and "CategoryX.PostScript" (options with old names are recognized and automatically converted on first settings saving);
    • new option "TaskX.Script";
    • old option "TaskX.Process" kept for scheduling of external programs not related to nzbget (to avoid writing of intermediate proxy scripts);
    • scan-script, queue-script and scheduler-script now work similar to post-processing scripts:
      • scripts must be put into scripts-directory;
      • scripts can be configured via web-interface and can have options;
      • multiple scripts can be chosen for each scripts-option, all chosen scripts are executed;
      • program and script options are passed to the script as env. variables;
    • renamed default directory with scripts from "ppscripts" to "scripts";
    • script signature indicates the type of script (post-processing, scan, queue or scheduler);
    • one script can have mixed signature allowing it to be used for multiple purposes (for example a notification script can send a notification on both events: after adding to queue and after post-processing);
    • result of RPC-method "configtemplates" has new fields "PostScript", "ScanScript", "QueueScript", "SchedulerScript" to indicate the purpose of the script;
    • queue-script (formerly NzbAddedProcess) has new parameter "NZBNA_EVENT" indicating the reason of calling the script; currently the script is called only after adding of files to download queue and therefore the parameter is always set to "NZB_ADDED" but the queue-script can be called on other events in the future too;
  • post-processing scripts now have two new parameters:
    • env. var "NZBPP_STATUS" indicates the status of download including the total status (SUCCESS, FAILURE, etc.) and the detail field (for example in case of failures: PAR, UNPACK, etc.);
    • env. var "NZBPP_TOTALSTATUS" is equal to the total status of parameter "NZBPP_STATUS" and is provided for convenience (to avoid parsing of "NZBPP_STATUS");
    • the new parameters provide a simple way for pp-scripts to determine download status without a guess work needed in previous versions;
    • parameters "NZBPP_PARSTATUS" and "NZBPP_UNPACKSTATUS" are now considered deprecated (still passed for compatibility);
    • updated script "EMail.py" to use new parameters "NZBPP_TOTALSTATUS" and "NZBPP_STATUS" instead of "NZBPP_PARSTATUS" and "NZBPP_UNPACKSTATUS";
  • when changing category in web-interface the post-processing parameters are now automatically updated according to new category settings:
    • only parameters which are different in old and new category are changed;
    • parameters which present in both or in neither categories are not changed;
    • that ensures that only the relevant parameters are updated and parameters which were manually changed by user remain they settings when it make sense;
    • in the "download details dialog" the new parameters are updated on the postprocess-tab directly after changing of category and can be controlled before saving;
    • in the "edit multiple downloads dialog" the parameters are updated individually for each download on saving;
    • new action "CP" of remote command "--edit/-E" for groups to set category and apply parameters;
    • new action "GroupApplyCategory of RPC-method "editqueue" for the same purpose;
  • changed the way option "ContinuePartial" works:
    • now the information about completed articles is stored in a special file in QueueDir;
    • when option "DirectWrite" is active no separate flag-files per article are created in TempDir;
    • the file contains additional information, which were not stored/available before;
  • improved per-server/per-nzb article completion statistics:
    • the statistics are now available for active downloads in details dialog (not only for history);
    • the info on that page is constantly updated as long as the page is active (unless refresh is disabled);
    • download age info removed from details dialog to save place (it is shown in the download list anyway);
    • if backup news-servers start to be used for nzb-file a badge appears in the download list showing the percentage of articles downloaded from backup servers;
    • click on the badge opens download details dialog directly on the completion page;
    • per-server/per-nzb article completion statistics are now available via RPC-method "listgroups" for active downloads (not only for "history");
  • improved RPC-API:
    • RPC-method "listgroups" now returns info about post-processing similar to info returned by method "postqueue";
    • RPC-method "postqueue" is obsolete now;
    • web-interface requires less requests to NZBGet on each page update and it is now easier for third-party developers to obtain the info about download and post-processing status (no need to merge download queue and post queue);
    • RPC-method "listgroups" now returns new field "Status" making it easier for third-party apps to determine the status of download entry;
    • new field "Status" in RPC-method "history" to allow third-party apps easier determine the status of an item without inspecting status-fields of every processing step;
    • changed web-interface to use new field "Status";
  • improved RPC-API:
    • method "append" now returns id of added nzb-file or "0" on an error;
    • this makes it easier for third-party apps to track added nzb-files;
    • for backward compatibility with older software expecting a boolean result the old version of method "append" is still supported;
    • the new version of method "append" has a different signature (order of parameters);
    • parameter "content" can now be either nzb-file content (encoded in base 64) or an URL;
    • this makes the method "appendurl" obsolete (still supported for compatibility);
    • if an URL was added to queue the queue entry created for fetched nzb-file has the same "NZBID" for easier tracking
  • added force-priorities:
    • downloads with priorities equal to or greater than 900 are downloaded and post-processed even if the program is in paused state (force mode);
    • in web-interface the combo for choosing priority has new entry "force" (priority value 900);
    • new fields "ForcedSizeLo", "ForcedSizeHi" and "ForcedSizeMB" returned by RPC-method "status";
  • post-processing scripts which move the whole download into a new location can inform the program about new location using command "[NZB] DIRECTORY=/new/path", allowing other scripts to process files further;
  • added support for power management on windows to avoid pc going into sleep mode during download or post-processing;
  • apostrophe is not considered an invalid file name character anymore;
  • adjusted modules initialization to avoid possible bugs due to delayed thread starts;
  • reorganized source code directory structure: created directory 'daemon' with several subdirectories and put all source code files there;
  • added new option "PropagationDelay", which sets the minimum post age to download; newer posts are kept on hold in download queue until they get older than the defined delay, after that they are downloaded;
  • download speeds above 1024 KB/s are now indicated in MB/s;
  • data sizes above 1000 GB are now shown as TB in web-interface (instead of GB);
  • splitted files are now joined automatically (again);
  • adjusted modules initialization to avoid possible bugs due to delayed thread starts;
  • extended info printed by remote command "nzbget -B dump" (for debug purposes);
  • eliminated loop waiting time in queue coordinator on certain conditions - may improve performance on very high speed connections;
  • increased few wait intervals which were unnecessary too small;
  • improved error reporting: added error check when closing article file for writing and when deleting files or directories;
  • when building nzbget if both OpenSSL and GnuTLS are available now using OpenSSL by default (the preferred library can still be selected with configure-parameter --with-tlslib=OpenSSL/GnuTLS);
  • windows version is now configured to use OpenSSL instead of GnuTLS; windows binaries provided on download page now use OpenSSL;
  • column "age" in web-interface now shows minutes for recent posts (instead of "0 h");
  • remote command "-B dump" now can be used also in release (non-debug) versions and prints useful debug data as "INFO" instead of "DEBUG";
  • to detect daylight saving activation/deactivation the time zone information is now checked every minute if a download is active or once in 3 hours if the program is in stand-by; these delays should work well with hibernation mode on synology);
  • updated all links to go to new domain (nzbget.net);
  • improved error reporting if unpacker or par-renamer fail to move files;
  • fixed: the program could crash during cleanup if files with invalid timestamps were found in the directory (windows only);
  • fixed: RSS feed preview dialog displayed slightly incorrect post ages because of the wrong time zone conversion;
  • fixed: sometimes URLs were removed too early from the feed history causing them to be detected as "new" and fetched again; if duplicate check was not active the same nzb-files could be downloaded again;
  • fixed: strange (damaged?) par2-files could cause a crash during par-renaming;
  • fixed: damaged nzb-files containing multiple par-sets and not having enough par-blocks could cause a crash during par-check;
  • fixed: if during par-repair the downloaded extra par-files were damaged and the repair was terminated with failure status the post-processing scripts were executed twice sometimes;
  • fixed: post-processing scripts were not executed in standalone mode ("nzbget /path/to/file.nzb");
  • fixed: renaming or deleting of temporary files could fail, especially when options "UnpackPauseQueue" and "ScriptPauseQueue" were not active (windows only);
  • fixed: per-server/per-nzb article completion statistics could be inaccurate for nzb-files whose download were interrupted by reload/restart;
  • fixed: after deleting servers from config file the program could crash on start when loading server volume statistics data from disk;
  • fixed: download speeds above approx. 70 MB/s were not indicated correctly in web-interface and by RPC-method "status";
  • fixed: cancelling of active par-job sometimes didn't work;
  • fixed: par-check could hang on renamed and splitted files;
  • fixed: the program could crash during parsing of malformed nzb-files;
  • fixed: errors during loading of queue from disk state may render the already loaded parts useless too; now at least these parts of queue are used;
  • fixed: queue was not locked during loading on program start and that could cause problems;
  • fixed: data sizes exactly equal to 10, 100, 1000 MB or GB were formatted using 4 digits instead of 3 (one digit after decimal point too much);
  • fixed: if post-processing step "move" failed, the command "post-process again" did not try to move again;
  • fixed: nzb-files were sometimes not deleted from NzbDir (option "NzbCleanupDisk");
  • fixed: scheduler command "FetchFeed" did not work properly with parameter "0" (fetch all feeds);
  • fixed: port number was not sent in headers when downloading from URLs which could cause issues with RSS for web-sites using non-standard http ports;
  • fixed: queued nzb-files was not deleted from disk when deleting download without history tracking.
Download link

Locked

Who is online

Users browsing this forum: No registered users and 1 guest