[New Feature] Smart duplicates

Discuss newly added features or request new features.
prinz2311
Posts: 464
Joined: 08 Dec 2012, 00:03

Re: [New Feature] Smart duplicates

Post by prinz2311 » 27 Sep 2013, 21:43

hugbug wrote:
prinz2311 wrote:And at least one of per rpc-api added episodes has a dupe (in history) and doesn't show the badge.
This is bad. Can you check if there were a message "Marking xxx as duplicate to yyy" in the log?
Can't... no log file was activated.

hugbug
Developer & Admin
Posts: 7514
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: [New Feature] Smart duplicates

Post by hugbug » 27 Sep 2013, 21:46

prinz2311 wrote:And when i remove the dupekey and click save, i still have the badge but just saying:

title
Yes, the duplicate mark remains.

You know, I think I better remove that internal "has duplicates" mark (which is not always in sync with reality) and show dupekey-badge if dupekey is set.
Inconsistence is bad for usability. And when using rss most items will have duplicates anyway, so trying to show badges only for duplicates is almost worthless.

prinz2311
Posts: 464
Joined: 08 Dec 2012, 00:03

Re: [New Feature] Smart duplicates

Post by prinz2311 » 27 Sep 2013, 21:53

hugbug wrote:You know, I think I better remove that internal "has duplicates" mark (which is not always in sync with reality) and show dupekey-badge if dupekey is set.
Inconsistence is bad for usability. And when using rss most items will have duplicates anyway, so trying to show badges only for duplicates is almost worthless.
Yes, now with the ability to add the dupekey manually and that the badge is shown without any real duplicate it's even more confusing.

hugbug
Developer & Admin
Posts: 7514
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: [New Feature] Smart duplicates

Post by hugbug » 28 Sep 2013, 20:02

Update:
  • removed internal field "DupeMark" showing that the item has duplicates; this flag was not always in sync with reality and it was used only to show (or not) badges with duplicate key in web-interface;
  • now badges are always shown for items having non-empty duplicate keys;
  • the badges become red if duplicate mode is set to "force".

hugbug
Developer & Admin
Posts: 7514
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: [New Feature] Smart duplicates

Post by hugbug » 28 Sep 2013, 21:29

Update:
  • when comparing two items if the both have dupekey only dupekeys are compared (names are not checked);
  • when a new item without dupekey is added and there is another item with the same name having dupekey its dupekey was copied to new item, this is disabled now (if necessary the dupekey can be manually set via new "duplicate properties" page);
  • fixed: command "Mark as Good" in history removed all duplicates but should remove only records with status "DUPE".
With these changes and changes from previous post the overall transparency of how things work has been improved.

hugbug
Developer & Admin
Posts: 7514
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: [New Feature] Smart duplicates

Post by hugbug » 29 Sep 2013, 20:22

I have an idea - new Options-options "series" and "rageid". By using of these options a dupekey will be set automatically using the option + season + episode:

Code: Select all

A(rageid:1234): dexter
This will automatically set dupekey to a value like this "rageid=1234-S08-E12"

And the option "series":

Code: Select all

A(series:GOT): game of thrones
This will automatically set dupekey to a value like this "series=GOT-S08-E12"

Important note: dupekey will be set only if season and episode are both not empty.

The new options are not really necessary because you can use "${season}-${episode}" with the same result:

Code: Select all

A(dupekey:GOT-${season}-${episode}): game of thrones season:>0 episode:>0
But the new syntax is shorter and easier for use.

What do you think?

prinz2311
Posts: 464
Joined: 08 Dec 2012, 00:03

Re: [New Feature] Smart duplicates

Post by prinz2311 » 29 Sep 2013, 20:39

It's simpler syntax.

Just a warning about :

Code: Select all

season:>0 episode:>0
0 is sometimes used for specials in the releasename. ;) Only sometimes... (thetvdb uses for all specials season 0 btw)

hugbug
Developer & Admin
Posts: 7514
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: [New Feature] Smart duplicates

Post by hugbug » 30 Sep 2013, 09:54

prinz2311 wrote:0 is sometimes used for specials in the releasename. ;) Only sometimes... (thetvdb uses for all specials season 0 btw)
Do you have an example which I could check how it comes from newznab?

prinz2311
Posts: 464
Joined: 08 Dec 2012, 00:03

Re: [New Feature] Smart duplicates

Post by prinz2311 » 30 Sep 2013, 10:12

A few releases:
Deadly.Women.S00E00.480p.hdtv.x264
The.Real.Housewives.of.Orange.County.S00E00.720p.hdtv.x264
Farscape S02E00 ReUnion DVDRip XviD-OSiRiS
NYPD Blue S02E00 Wedding Bell Blues DVDRip XviD-FPM
NYPD Blue S02E00 The Music Of Mike Post DVDRip XviD-FPM
NYPD Blue S02E00 Season 2 A Season Of Change DVDRip XviD-FPM
Underbelly S06E00 720p WEB-DL AAC2 0 H 264-NTb
Bad thing is that 0 is not a unique identifer (example: NYPD Blue), because everyone uses it for different things. So a 0 should probably be ignored, when parsing a name.

hugbug
Developer & Admin
Posts: 7514
Joined: 09 Sep 2008, 11:58
Location: Germany

Re: [New Feature] Smart duplicates

Post by hugbug » 30 Sep 2013, 20:03

Update:
  • added two new item-options for RSS filter rules "Accept" and "Options": option "rageid" generates duplicate key using custom rageid and season/episode numbers; option "series" generates duplicate key using custom series name (any unique string) and season/episode numbers:

    Code: Select all

    A(rageid:1234): dexter
    A(series:GOT): game of thrones
  • hiding badges for dupekeys in downloads/history lists if option "DupeCheck" is disabled;
  • fixed: if download failed an existing queued duplicate was not automatically unpaused.
Bad thing is that 0 is not a unique identifer (example: NYPD Blue), because everyone uses it for different things. So a 0 should probably be ignored, when parsing a name.
Dupekeys are generated only if both season and episode are not 0. Otherwise a risk to create non-unique keys is too high (special episodes, whole seasons, etc.).

I'm writing a documentation for the feature - http://nzbget.sourceforge.net/RSS#Duplicates. When it's completed I'll make a testing release. If you don't find new bugs, of course ;).

Many thanks for testing and for all suggestions.

Post Reply

Who is online

Users browsing this forum: No registered users and 4 guests