[Scan-Script] TidyIt - Media Library House Cleaning

Share your scripts or request scripts with specific features.
Forum rules
Please keep the forum clean - one topic per script. Questions not related to a specific script should be posted in Support forum.
l2g
Posts: 228
Joined: 27 Jun 2014, 22:13
Contact:

Re: [Scan-Script] TidyIt - Media Library House Cleaning

Post by l2g » 12 Aug 2015, 17:06

kloaknet wrote: the above example is related to a movie with a .rar file with a .sub file included. Its not used because subliminal ;) is way easier. But from the above I understand that such a folder won't be removed, even when the movie is deleted? Its a difficult issue to cope with I guess, because how do you know its not part of a bigger archive? Maybe check the file contents?
This might already be achievable (but i haven't tested it yet); the rar files rarely share the same name as the video itself. So you could just try adding ,.rar,.zip to the end of your VideoExtras entries. I haven't tested this; but it's just a thought that came to me.
But i could still add a 'AlwaysTrash' list that you can define to destroy things you know you won't want... such as *-sample.mkv or something.
kloaknet wrote: Ill just hit subscribe to this topic too, so Ill test the option for file deletion. Maybe you can add an option to specify the directory where to send the crap files (preferably with windows option to just dump them into the bin, because I am lazy :oops: )
Sure, it might not be done until this weekend (or mid next week). On a side note, I found this link which seems promising.

kloaknet
Posts: 337
Joined: 23 Jul 2014, 08:52

Re: [Scan-Script] TidyIt - Media Library House Cleaning

Post by kloaknet » 12 Aug 2015, 17:59

Don't worry, better something really good, than something made in a hurry. :P And we should never forget how it was before all the automation!

l2g
Posts: 228
Joined: 27 Jun 2014, 22:13
Contact:

Re: [Scan-Script] TidyIt - Media Library House Cleaning

Post by l2g » 23 Aug 2015, 03:39

I'm a bit slow, but have finally got around to a 'test' version. I don't want to package it up yet until it's been tested more.
  1. Added an AlwaysTrash Option which defaults to being empty (thus TidyIt behaves as it always did). This option will specifically allow you to identify content (files) to be removed in all circumstances if matched. It was suggested that some people (kloaknet :) ) don't like the extra rar content that sometimes ships with .mkv files (containing subtitles). Well in the present state of the script, if files detected in your library can't be matched as certain media types (you define) such as .srt, .mkv, .avi, .nfo, .jpg, etc.. then the script doesn't know what to do and skips the directory. This is intentionally done for your safety.

    Now that is where this option comes in... If you identify anything using this new field, you're explicitly identifying files (only) that should be removed if matched. So use this option with caution. Directories that contain 'safe files' (files that prevent the further processing of a directory) will always protect you from the actions of this option.

    For example: .zip,.r??,.z7,.0??,.1??,.2??,.3??; this would cause any archive type content found in your media directory to be handled.
  2. Added a Trash Mode. Don't get this confused with my point above. This focuses on the second point you (kloaknet) brought up; having the ability to send content flagged for removal to the Trash (Linux/Mac) or the Recycle Bin (Microsoft) instead of flat out removing the file. This is experimental as I'm using someone's third party software to try it out.

    By introducing this mode; it pretty much breaks the old configuration; so once you install this script (into your script dir), you'll need to go into your NZBGet configuration and re-set the mode (Preview, Delete or Trash) and save it.
Edit: updated point #1 to (attempt to) make it more understandable. I also implemented all these actions through the CLI now too so i erased the paragraph about that.
Last edited by l2g on 29 Aug 2015, 20:13, edited 3 times in total.

kloaknet
Posts: 337
Joined: 23 Jul 2014, 08:52

Re: [Scan-Script] TidyIt - Media Library House Cleaning

Post by kloaknet » 24 Aug 2015, 14:20

I don't understand point 1.

Ill test the new version for option 2, and if you don't hear a thing, nothing is wrong, or my whole pc is wiped ;)

kloaknet
Posts: 337
Joined: 23 Jul 2014, 08:52

Re: [Scan-Script] TidyIt - Media Library House Cleaning

Post by kloaknet » 28 Aug 2015, 18:21

Hi, i think somehow it stays in preview mode, although I have chosen 'move' (and a lot of saves/restarts etc;

Code: Select all

INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,951 - 5116 - DEBUG - Planned removal (dir): C:\Downloads\Series\XXX
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,951 - 5116 - INFO - PREVIEW ONLY: Handle FILE: C:\Downloads\Series\XXX\The.Last.Ship.S02E11.720p.HDTV.X264-DIMENSION.srt
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,951 - 5116 - DEBUG - Potentially removable (extra): C:\Downloads\Series\XXX\The.Last.Ship.S02E11.720p.HDTV.X264-DIMENSION.srt
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,951 - 5116 - DEBUG - get_files('C:\Downloads\Series\XXX') with 1 filter(s)
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,937 - 5116 - DEBUG - get_files('C:\Downloads\Series') with 1 filter(s)
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,937 - 5116 - INFO - Scanning C:\Downloads\Series
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,937 - 5116 - DEBUG - get(config) ALWAYSTRASH=""
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,937 - 5116 - DEBUG - Compiled regex "(\.divx|\.iso|\.mkv|\.mov|\.mpg|\.avi|\.mpeg|\.vob|\.xvid|\.wmv|\.mp4)$"
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,937 - 5116 - DEBUG - Compiled regex "(\.tbn|\.idx|\.nzb|\.jpg|\.nfo|\.txt|\...\.srt|\.xml|\.srt|\.sub)$"
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,937 - 5116 - DEBUG - get(config) VIDEOEXTRAS=".nfo,.??.srt,.srt,.sub,.txt,.sub,.idx,.jpg,.tbn,.nzb,.xml"
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,937 - 5116 - DEBUG - get(config) ALWAYSTRASH=""
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,937 - 5116 - DEBUG - get(config) VIDEOPATHS="C:\Downloads\Series, C:\Downloads\Movies"
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,937 - 5116 - DEBUG - get(config) SYSTEMENCODING="UTF-8"
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,937 - 5116 - DEBUG - get(config) PROCESSMINAGE="600"
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,937 - 5116 - DEBUG - get(config) VIDEOMINSIZE="150"
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,937 - 5116 - DEBUG - get(config) VIDEOEXTENSIONS=".mkv,.avi,.divx,.xvid,.mov,.wmv,.mp4,.mpg,.mpeg,.vob,.iso"
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,937 - 5116 - DEBUG - get(config) SAFEENTRIES=".notidy"
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,937 - 5116 - DEBUG - get(config) MODE="Move"
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,921 - 5116 - DEBUG - get(config) VIDEOPATHS="C:\Downloads\Series, C:\Downloads\Movies"
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,921 - 5116 - DEBUG - get(config) VIDEOPATHS="C:\Downloads\Series, C:\Downloads\Movies"
INFO	Fri Aug 28 2015 20:14:00	TidyIt: [DEBUG] Script mode forced off.
INFO	Fri Aug 28 2015 20:14:00	Executing scheduler-script TidyIt.py for Task3
INFO	Fri Aug 28 2015 20:13:43	nzbget 15.0 service-mode

l2g
Posts: 228
Joined: 27 Jun 2014, 22:13
Contact:

Re: [Scan-Script] TidyIt - Media Library House Cleaning

Post by l2g » 29 Aug 2015, 20:34

edit: removed content since it was misleading. The Trash mode has been scrapped and I created the Move mode instead (see next post) which works much better.
Last edited by l2g on 25 Sep 2015, 20:43, edited 1 time in total.

l2g
Posts: 228
Joined: 27 Jun 2014, 22:13
Contact:

Re: [Scan-Script] TidyIt - Media Library House Cleaning

Post by l2g » 01 Sep 2015, 01:09

Release v0.4.0 (Direct Download Link) in place.

The release details can be found at the head of this thread (quick link).

This release adds quite a ton of new features:
  • Move Mode: The biggest new feature is the ability to have content moved to another directory instead of deleted. This is a little less harsh than the Delete Mode, but now still requires you to manage the directory yourself (delete it when your content that everything there is no use).
  • Identify Extra Meta Content: Meta content like Thumbs.db (Microsoft Thumbnail Database), .DS_Store (OSX directory), etc are automatically looked after. However, now you can additionally specify more meta content you want to be treated the same way. Meta Content is treated in such a way that it's contents is ONLY handled if and/or when the parent folder it resides in has no more other viable media in it. These can also be applied from the CLI using --meta-content= (-M).
  • Always Trash items allows you to specify one or more items you just don't ever want in your media library. This could be things such as .rar files, or .diz files, svf files. But default this is not ever populated; but the option is there anyway. Content that fits other categories will always over-ride this one. These can also be applied from the CLI using --always-trash= (-t).
  • Improved logging; that's right... the logs make more sense now.
  • Added Synology meta directory to default (@eaDir) meta listings.
Please feel free to let me know if there is anything I should add or if you find any bugs!
Last edited by l2g on 19 Feb 2017, 21:57, edited 2 times in total.

mike_sendrove
Posts: 3
Joined: 26 Jan 2016, 21:46

Re: [Scan-Script] TidyIt - Media Library House Cleaning

Post by mike_sendrove » 26 Jan 2016, 21:52

Hi - is it possible that the script can just remove files and not directories? I'd like to maintain my directory structure.

Thanks,
Mike.

l2g
Posts: 228
Joined: 27 Jun 2014, 22:13
Contact:

Re: [Scan-Script] TidyIt - Media Library House Cleaning

Post by l2g » 02 Feb 2016, 02:18

mike_sendrove wrote:Is it possible that the script can just remove files and not directories? I'd like to maintain my directory structure
It shouldn't be too difficult to implement; but can can you give me a reason why you'd want to do that? It kind of defeats the purpose of the script in a way. I mean directories are 'only' removed if deemed to be:
  • an empty folder
  • an empty meta folder without any matching video content to go with it
If there is a specific folder you don't want it to ever remove; you can add it to the safe list, or simply put a safe file inside of the directory you want to preserve and this tool won't touch the contents of it (empty or not).

habskilla
Posts: 13
Joined: 29 Oct 2014, 13:50

Re: [Scan-Script] TidyIt - Media Library House Cleaning

Post by habskilla » 05 Feb 2016, 16:14

My WDTV media player creates these .wdtv directories. Can you script handle these?

Post Reply

Who is online

Users browsing this forum: No registered users and 29 guests