Page 2 of 3

Re: [Scan-Script] TidyIt - Media Library House Cleaning

Posted: 12 Aug 2015, 17:06
by l2g
kloaknet wrote: the above example is related to a movie with a .rar file with a .sub file included. Its not used because subliminal ;) is way easier. But from the above I understand that such a folder won't be removed, even when the movie is deleted? Its a difficult issue to cope with I guess, because how do you know its not part of a bigger archive? Maybe check the file contents?
This might already be achievable (but i haven't tested it yet); the rar files rarely share the same name as the video itself. So you could just try adding ,.rar,.zip to the end of your VideoExtras entries. I haven't tested this; but it's just a thought that came to me.
But i could still add a 'AlwaysTrash' list that you can define to destroy things you know you won't want... such as *-sample.mkv or something.
kloaknet wrote: Ill just hit subscribe to this topic too, so Ill test the option for file deletion. Maybe you can add an option to specify the directory where to send the crap files (preferably with windows option to just dump them into the bin, because I am lazy :oops: )
Sure, it might not be done until this weekend (or mid next week). On a side note, I found this link which seems promising.

Re: [Scan-Script] TidyIt - Media Library House Cleaning

Posted: 12 Aug 2015, 17:59
by kloaknet
Don't worry, better something really good, than something made in a hurry. :P And we should never forget how it was before all the automation!

Re: [Scan-Script] TidyIt - Media Library House Cleaning

Posted: 23 Aug 2015, 03:39
by l2g
I'm a bit slow, but have finally got around to a 'test' version. I don't want to package it up yet until it's been tested more.
  1. Added an AlwaysTrash Option which defaults to being empty (thus TidyIt behaves as it always did). This option will specifically allow you to identify content (files) to be removed in all circumstances if matched. It was suggested that some people (kloaknet :) ) don't like the extra rar content that sometimes ships with .mkv files (containing subtitles). Well in the present state of the script, if files detected in your library can't be matched as certain media types (you define) such as .srt, .mkv, .avi, .nfo, .jpg, etc.. then the script doesn't know what to do and skips the directory. This is intentionally done for your safety.

    Now that is where this option comes in... If you identify anything using this new field, you're explicitly identifying files (only) that should be removed if matched. So use this option with caution. Directories that contain 'safe files' (files that prevent the further processing of a directory) will always protect you from the actions of this option.

    For example: .zip,.r??,.z7,.0??,.1??,.2??,.3??; this would cause any archive type content found in your media directory to be handled.
  2. Added a Trash Mode. Don't get this confused with my point above. This focuses on the second point you (kloaknet) brought up; having the ability to send content flagged for removal to the Trash (Linux/Mac) or the Recycle Bin (Microsoft) instead of flat out removing the file. This is experimental as I'm using someone's third party software to try it out.

    By introducing this mode; it pretty much breaks the old configuration; so once you install this script (into your script dir), you'll need to go into your NZBGet configuration and re-set the mode (Preview, Delete or Trash) and save it.
Edit: updated point #1 to (attempt to) make it more understandable. I also implemented all these actions through the CLI now too so i erased the paragraph about that.

Re: [Scan-Script] TidyIt - Media Library House Cleaning

Posted: 24 Aug 2015, 14:20
by kloaknet
I don't understand point 1.

Ill test the new version for option 2, and if you don't hear a thing, nothing is wrong, or my whole pc is wiped ;)

Re: [Scan-Script] TidyIt - Media Library House Cleaning

Posted: 28 Aug 2015, 18:21
by kloaknet
Hi, i think somehow it stays in preview mode, although I have chosen 'move' (and a lot of saves/restarts etc;

Code: Select all

INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,951 - 5116 - DEBUG - Planned removal (dir): C:\Downloads\Series\XXX
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,951 - 5116 - INFO - PREVIEW ONLY: Handle FILE: C:\Downloads\Series\XXX\The.Last.Ship.S02E11.720p.HDTV.X264-DIMENSION.srt
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,951 - 5116 - DEBUG - Potentially removable (extra): C:\Downloads\Series\XXX\The.Last.Ship.S02E11.720p.HDTV.X264-DIMENSION.srt
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,951 - 5116 - DEBUG - get_files('C:\Downloads\Series\XXX') with 1 filter(s)
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,937 - 5116 - DEBUG - get_files('C:\Downloads\Series') with 1 filter(s)
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,937 - 5116 - INFO - Scanning C:\Downloads\Series
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,937 - 5116 - DEBUG - get(config) ALWAYSTRASH=""
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,937 - 5116 - DEBUG - Compiled regex "(\.divx|\.iso|\.mkv|\.mov|\.mpg|\.avi|\.mpeg|\.vob|\.xvid|\.wmv|\.mp4)$"
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,937 - 5116 - DEBUG - Compiled regex "(\.tbn|\.idx|\.nzb|\.jpg|\.nfo|\.txt|\...\.srt|\.xml|\.srt|\.sub)$"
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,937 - 5116 - DEBUG - get(config) VIDEOEXTRAS=".nfo,.??.srt,.srt,.sub,.txt,.sub,.idx,.jpg,.tbn,.nzb,.xml"
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,937 - 5116 - DEBUG - get(config) ALWAYSTRASH=""
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,937 - 5116 - DEBUG - get(config) VIDEOPATHS="C:\Downloads\Series, C:\Downloads\Movies"
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,937 - 5116 - DEBUG - get(config) SYSTEMENCODING="UTF-8"
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,937 - 5116 - DEBUG - get(config) PROCESSMINAGE="600"
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,937 - 5116 - DEBUG - get(config) VIDEOMINSIZE="150"
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,937 - 5116 - DEBUG - get(config) VIDEOEXTENSIONS=".mkv,.avi,.divx,.xvid,.mov,.wmv,.mp4,.mpg,.mpeg,.vob,.iso"
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,937 - 5116 - DEBUG - get(config) SAFEENTRIES=".notidy"
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,937 - 5116 - DEBUG - get(config) MODE="Move"
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,921 - 5116 - DEBUG - get(config) VIDEOPATHS="C:\Downloads\Series, C:\Downloads\Movies"
INFO	Fri Aug 28 2015 20:14:00	TidyIt: 2015-08-28 20:14:00,921 - 5116 - DEBUG - get(config) VIDEOPATHS="C:\Downloads\Series, C:\Downloads\Movies"
INFO	Fri Aug 28 2015 20:14:00	TidyIt: [DEBUG] Script mode forced off.
INFO	Fri Aug 28 2015 20:14:00	Executing scheduler-script TidyIt.py for Task3
INFO	Fri Aug 28 2015 20:13:43	nzbget 15.0 service-mode

Re: [Scan-Script] TidyIt - Media Library House Cleaning

Posted: 29 Aug 2015, 20:34
by l2g
edit: removed content since it was misleading. The Trash mode has been scrapped and I created the Move mode instead (see next post) which works much better.

Re: [Scan-Script] TidyIt - Media Library House Cleaning

Posted: 01 Sep 2015, 01:09
by l2g
Release v0.4.0 (Direct Download Link) in place.

The release details can be found at the head of this thread (quick link).

This release adds quite a ton of new features:
  • Move Mode: The biggest new feature is the ability to have content moved to another directory instead of deleted. This is a little less harsh than the Delete Mode, but now still requires you to manage the directory yourself (delete it when your content that everything there is no use).
  • Identify Extra Meta Content: Meta content like Thumbs.db (Microsoft Thumbnail Database), .DS_Store (OSX directory), etc are automatically looked after. However, now you can additionally specify more meta content you want to be treated the same way. Meta Content is treated in such a way that it's contents is ONLY handled if and/or when the parent folder it resides in has no more other viable media in it. These can also be applied from the CLI using --meta-content= (-M).
  • Always Trash items allows you to specify one or more items you just don't ever want in your media library. This could be things such as .rar files, or .diz files, svf files. But default this is not ever populated; but the option is there anyway. Content that fits other categories will always over-ride this one. These can also be applied from the CLI using --always-trash= (-t).
  • Improved logging; that's right... the logs make more sense now.
  • Added Synology meta directory to default (@eaDir) meta listings.
Please feel free to let me know if there is anything I should add or if you find any bugs!

Re: [Scan-Script] TidyIt - Media Library House Cleaning

Posted: 26 Jan 2016, 21:52
by mike_sendrove
Hi - is it possible that the script can just remove files and not directories? I'd like to maintain my directory structure.

Thanks,
Mike.

Re: [Scan-Script] TidyIt - Media Library House Cleaning

Posted: 02 Feb 2016, 02:18
by l2g
mike_sendrove wrote:Is it possible that the script can just remove files and not directories? I'd like to maintain my directory structure
It shouldn't be too difficult to implement; but can can you give me a reason why you'd want to do that? It kind of defeats the purpose of the script in a way. I mean directories are 'only' removed if deemed to be:
  • an empty folder
  • an empty meta folder without any matching video content to go with it
If there is a specific folder you don't want it to ever remove; you can add it to the safe list, or simply put a safe file inside of the directory you want to preserve and this tool won't touch the contents of it (empty or not).

Re: [Scan-Script] TidyIt - Media Library House Cleaning

Posted: 05 Feb 2016, 16:14
by habskilla
My WDTV media player creates these .wdtv directories. Can you script handle these?