anthirian / script.service.janitor Goto Github PK

Once you delete a video and that folder then becomes empty, delete that folder as well as any subfolders. This should not be default behavior and should be enabled manually in the settings.

Development status

• Recursively scan for empty folders
• Ignore certain file extensions when checking for emptiness
• Add setting to allow users to specify file extensions to ignore

Some people prefer not to be disturbed by notifications when watching a video. We should add support for that behavior.

zorvalth: It never deletes my subtitles. I have usually two subtitle files and they look like this:

name.mkv
name.en.srt
name.bg.srt

But it never deletes the subtitles...

Instead of having to dig through the videos database using sqlite, we should aim to use JSON-RPC queries for requesting the videos that have to be deleted.

Note that filtering of RPC results is only supported from XBMC Frodo and above, which means no more backwards compatibility with Eden.

Development status

• Finding watched music videos
• Finding watched movies
• Finding watched TV show episodes
• Selecting multiple filters
• Dynamic filtering based on conditions selected in settings
• Error handling
• Code optimizations
• Testing

Can you please update the number of excluded folders to more than 3.

Reason is, have 6 separate media source and would like to exclude 4 of the 6.

Cheers

Hi, thanks for a great add-on. This and the subtitle add-on is basically the only reason I installed XBMC. So as you can understand, I love your add-on.

The only problem I've got, is that I once watched a movie from my DVD-rom, and now Filecleaner is always trying to delete it when I start XBMC. Obviously it can't, as a DVD is read-only. And now I don't even have a disc in the drive.

But it still gives me an error upon start of XBMC, "E:\Moviename\ Path not found, can't delete media" or something. It's getting a little annoying.

So I tried to exclude the DVD-rom from the FileCleaner check-list, but it's not there. I can't exclude the DVD-rom.. It's not listed.

Any ideas what to do?

Thanks for any responce :)

• donkeyhigh/ Fredrik Ravlo

using the field rating isn't doing the job. instead use userrating which is the field from the INFO Dialog and can be modified by the user.

I currently have filecleaner running on a Rasberry Pi using RaspBMC and pulling videos from a Windows host over SMB. When the script tries to clean it creates the directories in the destination folder but doesn't actually move the files. Log output is here (file paths anonymized):

11:31:22 T:2732586048  NOTICE: Kodi File Cleaner: Unstacking 'smb://WindowsPC/Video/Show1/TVShow.mkv' is not needed.
11:31:23 T:2732586048  NOTICE: Previous line repeats 1 times.
11:31:23 T:2732586048  NOTICE: Kodi File Cleaner: Path exclusion is disabled.
11:31:23 T:2732586048  NOTICE: Kodi File Cleaner: Attempting to move 'smb://WindowsPC/Video/Show1/TVShow.mkv' to 'smb://WindowsPC/Watched/Show1/TVShow'.
11:31:23 T:2732586048  NOTICE: Kodi File Cleaner: Moving 'smb://WindowsPC/Video/Show1/TVShow.mkv' to 'smb://WindowsPC/Watched/Show1/TVShow/TVShow.mkv'.
11:31:23 T:2732586048   ERROR: Rename - Error( Invalid cross-device link )
11:31:23 T:2732586048   ERROR: Rename - Error renaming file smb://WindowsPC/Video/Show1/TVShow.mkv
11:31:23 T:2732586048  NOTICE: Kodi File Cleaner: No cleaning required

We can't use SMB or NFS URLs with the os.statvfs() method provided by non-Windows file systems. XBMC however is still able to use these protocols for mounting shares via its internal file system. Unfortunately the xbmcvfs Python addon module has no statvfs() method that we can use. To resolve this, we need to either use the internal mounting file system used by XBMC to access shares using these protocols, or build a custom library for accessing SMB and NFS shares that does contain a statvfs() method.

This great addon is not available for the new Kodi yet. Is this one coming to the latest repo for Kodi as well? I really miss having a good file cleaner!

At the moment it is not possible to delete a file on an SMB or NFS share from XMBC on Windows or Linux.

Frodo (and future versions of XBMC) use a new type of localization strings. If those cannot be found, XBMC automatically falls back to the old type strings.xml.

We should add support for this new type in a future release.

I cannot find this script on Kodi RC2 official addon-repo. Is there a reason why it is not listed?

I downloaded it from github as zip and tried to install via Zip but the error message was "Dependencies Not Met. Contact Author"

Can you please get this script to work with Kodi? Thanks!

Some users may prefer to have cleaning occur right after watching a video, instead of at fixed intervals. We should probably add support for that somewhere in a future release.

would be nice to get support for viewing the log in linux based systems as well.

I've been using Janitor for a long time without a problem. It hasn't been able to run since 4 days ago, this is the debug output from the log (with file paths changed):

12:22:13.839 T:1492 NOTICE: Janitor: Not cleaning u"file.mkv".
12:22:13.843 T:1492 NOTICE: Janitor: Not cleaning u'file2.mkv'.
12:22:13.844 T:1492 NOTICE: Janitor: Not cleaning u'file3.mkv'.
12:22:13.844 T:1492 NOTICE: Janitor: Not cleaning u'file4.mkv'.
12:22:13.844 T:1492 NOTICE: Janitor: Not cleaning u'file5.mkv'.
12:22:13.845 T:1492 NOTICE: Janitor: Not cleaning u'file6.mkv'.
12:22:13.846 T:1492 ERROR: EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--
- NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
Error Type: <type 'exceptions.UnicodeEncodeError'>
Error Contents: 'ascii' codec can't encode character u'\u2019' in position 114: ordinal not in range(128)
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Roaming\Kodi\addons\script.service.janitor\service.py", line 45, in
autostart()
File "C:\Users\Administrator\AppData\Roaming\Kodi\addons\script.service.janitor\service.py", line 31, in autostart
results, _ = cleaner.clean_all()
File "C:\Users\Administrator\AppData\Roaming\Kodi\addons\script.service.janitor\default.py", line 228, in clean_all
cleaned_files, count, status = self.clean(video_type)
File "C:\Users\Administrator\AppData\Roaming\Kodi\addons\script.service.janitor\default.py", line 148, in clean
unstacked_path = self.unstack(filename)
File "C:\Users\Administrator\AppData\Roaming\Kodi\addons\script.service.janitor\default.py", line 385, in unstack
path = path.encode()
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2019' in position 114: ordinal not in range(128)
-->End of Python script error report<--
12:22:13.852 T:1492 WARNING: CPythonInvoker(2, C:\Users\Administrator\AppData\Roaming\Kodi\addons\script.service.janitor\service.py): the python script "C:\Users\Administrator\AppData\Roaming\Kodi\addons\script.service.janitor\service.py" has left several classes in memory that we couldn't clean up. The classes include: class XBMCAddon::xbmc::Monitor,class XBMCAddon::xbmcgui::DialogProgress
12:22:18.809 T:6096 WARNING: Janitor: User canceled.
12:22:18.834 T:6096 WARNING: Previous line repeats 1 times.
12:22:18.834 T:6096 WARNING: CPythonInvoker(3, C:\Users\Administrator\AppData\Roaming\Kodi\addons\script.service.janitor\default.py): the python script "C:\Users\Administrator\AppData\Roaming\Kodi\addons\script.service.janitor\default.py" has left several classes in memory that we couldn't clean up. The classes include: class XBMCAddon::xbmc::Monitor,class XBMCAddon::xbmcgui::DialogProgress

zorvalth: Something like delete tv show when they are watched and older than X episodes behind. For example i want to keep only the last 2 episodes. I tried to put 15 days but sometimes tv shows got pauses for a month or two.

moryoav: Add option to check the number of hard links to a file and only delete if there is just 1. This is particularly useful for torrents that are still being seeded.

Due to all the new settings, the addon configuration page has become cluttered. We should redesign it into more categories:

• Actions
• Frequency
• Conditions
• Notifications

Progress:

• Design
• Implement

Value for critia (number of days) of overlayed by the Dutch translation of this field. String value too long.
See:

OS: Running on QNAP 4.3.3 (Ubuntu 14.04.1 LTS, kernel: Linux x86 64-bit version 4.2.8)
Python Version: Python 2.7.13
Janitor Version: 4.3.1
Kodi Version: 17.1 (Kodi compiled Apr 22 2017 by GCC 4.8.4 for Linux x86 64-bit version 3.13.11)

Currently have only 3 exclusions enabled out of the possible 5

I have attached a summary of the debugging log file.
JanitorDebugLogSummary.txt

The problem seems to be that when an exclusion is not used, the buildup of the sql string has % added, and so for me there are 2 AND statements that contain the following:
AND ((episode_view.strPath NOT LIKE '%%'))

Workaround for me was to fill the last 2 empty exclusion locations with duplicate exclusion paths.

I updated the script today and I get this error :

ERROR: EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--
- NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
Error Type: <type 'exceptions.ImportError'>
Error Contents: cannot import name windll
Traceback (most recent call last):
File "/home/xbmc/.xbmc/addons/xbmcfilecleaner-master/default.py", line 11, in
from ctypes import c_wchar_p, c_ulonglong, pointer, windll
ImportError: cannot import name windll
-->End of Python script error report<--

Kodi features a tagging option that could be used by Kodi File Cleaner to include or exclude certain files. We should look into this option as it might make scanning for watched videos considerably faster.

There is no ETA for this feature however, as I have no clue how tagging works at the moment. If you do, you may give it a try and submit a pull request.

will you please consider adding cleaning of unwatched items too thanks

Chubbs: Cleaning shows with Russian Cyrillic characters in their titles results in a UnicodeDecodeError.

- NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
Error Type: <type 'exceptions.UnicodeDecodeError'>
Error Contents: 'utf8' codec can't decode byte 0xc1 in position 0: invalid start byte
Traceback (most recent call last):
    File "C:\..\script.filecleaner-master\service.py", line 47, in <module>autostart()
    File "C:\..\script.filecleaner-master\service.py", line 33, in autostart results, _ = cleaner.clean_all()
    File "C:\..\script.filecleaner-master\default.py", line 233, in clean_all cleaned_files, count, status = self.clean(video_type)
    File "C:\..\script.filecleaner-master\default.py", line 141, in clean expired_videos = self.get_expired_videos(video_type)
    File "C:\..\script.filecleaner-master\default.py", line 335, in get_expired_videos result = json.loads(response)
    File "C:..\Kodi\system\python\Lib\json\__init__.py", line 338, in loads return _default_decoder.decode(s)
    File "C:\..\Kodi\system\python\Lib\json\decoder.py", line 366, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    File "C:\..\Kodi\system\python\Lib\json\decoder.py", line 382, in raw_decode obj, end = self.scan_once(s, idx)
    UnicodeDecodeError: 'utf8' codec can't decode byte 0xc1 in position 0: invalid start byte
    -->End of Python script error report<--

It's an old movie but I don't think I've watched it on my current setup the first time. So maybe trakt syncs my library and that way filecleaner shows it as watched?

"NOTICE: XBMC File Cleaner: Expired videos: [[u'smb://K-KZ664AAR-ABA-A/Dl/Movies/Reservoir Dogs (1992)/Reservoir Dogs (1992).mkv', u'Reservoir Dogs', 1992]]"

What do I need to do uf I want to rewatch an old movie without getting it deleted right away?

Thanks.

There should be an option to prevent cleaning of videos that have bookmarks set.

Currently the shared library functionality of XBMC is fully untested. It may work, but it also may very well not.

Deleting files permanently causes no problems, but moving files to a temporary directory does not work and reports that the files cannot be found.

I love you script because it saves so much HD space for TV shows I want to watch but not keep around.
Here is the thing. I use SMB to get all of my stuff from my FreeNAS server I have setup. But because sometimes for whatever reason my computers stop seeing the NetBIOS name for my NAS. But if I smb into it with the IP address I can get into it no mater what. So my SMBs in the DB look like this smb://192.168.0.65/tv/Video/**** you get the point. Now I hope you can understand why I'm asking for the help I am?
So what I was needing to know is in my user settings folder for your addon. What is a template line of code so I can tell the cleaner to not remove anything from xyz folder that can be added to the setting.xml file?

As for the 2nd part I was wanting to see what I would need to mod in the main addon folder so that I could have say upto 9 Exclusions. And be able to have it work on all 9 items.
I found in the main addons settings.xml file where I can add more lines to the Exclusions, but when I do all it does is show up there it does not use them outside of the main 3 items.
Any help with that mod would be great because I know XML coding but as far as PY coding no clue there never had then need to learn it till now. Got a book on it's way but it will be a bit before I can really say I understand how to read the code let alone make it work.

Thanks for the great work keep it up. If I had the money to get you drunk I would 50 times over.
Michael

It would be nice if the script could delete all the *.srt an *.sub files with the same name and in the same folder of the watched media.

benharvey1985: Would it be possible to add the feature to move the destination folder for watched shows / movies to separate locations rather than all going to one folder...

Since I store my movies & TV shows on different drives this creates quite an overhead of work for my drives as it currently stands.

Hello

My "work" flow is such I often download content from YT to watch it offline(-ish). Those are indexed by the Kodi library, but not in the standard movie/TV show/music video sections. So they're not covered by the VideoLibrary.Get* methods.
It could be done by using VideoLibrary.Export and subtracting the output of VideoLibrary.Get{Episodes,Movies,MusicVideos}, but it seems costly.

Sometimes cleaning takes up a lot of resources that are better spent decoding and displaying videos. We should add a setting to hold off on cleaning while watching a video.

I configured the add-on to wait 5 days before deleting movies/series, and this has been working great. But a few days ago it just started deleting automaticly the moment the videos are done playing. I've tried disabling the add-on and re-enableing it, tried setting it to 3 days, but still it just deletes instantly. Even tried downgrading it to 4.0.0, but still the problem resist.
I really like the delay function, as my girlfriend watches some series when I'm at school, and I want to watch them when she's at her job. Please help me fix this.

Kodi-17.1 (osmc edition)
latest plugin from official repo (4.3.0)

14:11:11.907 T:1378874352   ERROR: EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--
                                             - NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
                                            Error Type: <type 'exceptions.UnicodeEncodeError'>
                                            Error Contents: 'ascii' codec can't encode characters in position 0-8: ordinal not in range(128)
                                            Traceback (most recent call last):
                                              File "/home/osmc/.kodi/addons/script.service.janitor/default.py", line 646, in <module>
                                                results, return_status = cleaner.clean_all()
                                              File "/home/osmc/.kodi/addons/script.service.janitor/default.py", line 228, in clean_all
                                                cleaned_files, count, status = self.clean(video_type)
                                              File "/home/osmc/.kodi/addons/script.service.janitor/default.py", line 193, in clean
                                                title = title.encode()
                                            UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-8: ordinal not in range(128)
                                            -->End of Python script error report<--
14:11:12.186 T:1378874352 WARNING: CPythonInvoker(9, /home/osmc/.kodi/addons/script.service.janitor/default.py): the python script "/home/osmc/.kodi/addons/script.service.janitor/default.py" has left several classes in memory that we couldn't clean up. The classes include: N9XBMCAddon7xbmcgui14DialogProgressE,N9XBMCAddon4xbmc7MonitorE

PS everything works good before update. On Kodi 16.x and Janitor 4.2.1

If you have two different versions of the same video with the same file name, and you have one in your library and the other in the holding folder, then moving the one from the library to the holding folder will fail to be moved.

The solution is to check for an existing file at the destination, and replace it with the one from the library if and only if it has a greater file size.

When you update the library and the cleaner deletes videos during that update, XBMC will hang if you shutdown immediately after cleaning. This (5 minute) sleep should be timed, just like the regular looping does, to prevent hanging.

Lately this addon is getting auto matically "updated" from the official xbmc repository. And that completely breaks it. I find myself eleteing the new folder "script.filecleaner" to make it work as the right folder is scripr.filecleaner-master.
Please fix this so we don't have to do this on a daily basis.
Thank you.

As pointed out by RockerC, we should rename Kodi File Cleaner to meet the official trademark policy. Relevant excerpt:

Related Software

XBMC Foundation products are designed to be extended, and we recognize that community members writing add-ons and supplemental software need some way to identify the XBMC Foundation product to which their add-on/software pertain. Our main concern about add-ons and related software is that consumers not be confused as to whether they are official (meaning approved by XBMC Foundation) or not. To address that concern, we request that add-ons and supplemental software names not include, in whole or in part, the words "XBMC Foundation" or "Kodi" in a way that suggests a connection between XBMC Foundation and the add-on or software (e.g., "Frobnicator for Kodi," would be acceptable, but "Kodi Frobnicator" would not).

I'm taking suggestions. This doesn't mean the new name has to feature "Kodi" or "File Cleaner." I'd rather pick a new name that consists of one simple word that people associate with cleaning in general.

zer04c: On the raspberry pi, it really slows down the UI when it's searching through all the files, and because you want it to clean the library when it's done, you don't want it to do that while you are watching something (to avoid the cleaning library bar showing) so I've set it not to scan while watching stuff. What it does with these settings, is kick in when once you've finished viewing something , and browsing the menus - exactly the time when you don't want. I was thinking a once a day setting would be prefect - having a few watched movies and TV shows around for a day before they are cleaned is fine.

I had a little poke around in the settings and changed my frequency value to 1440 which is working great now, when I turn it on each day, it fires up the addon , cleans what it needs, and then the UI has full access - a bit like the 'update library on startup' feature in the settings.

People who don't run the cleaner at startup may wish to manually start it at a time of their choosing. This should be a single launch, meaning it only does one cleaning cycle before terminating.

When you change the scan interval or the delayed start in the settings, it will only take effect after a restart of XBMC.

lysin: please consider adding an option to monitor deleted files. Or at least a "dump" file of deleted items. It would be really helpful...

patria o muerte, joaomneto: Movies are being deleted but the library isn't automatically cleaned afterwards.

Log file: http://pastebin.com/F1TdLPUS

Can you include an option to select which folders/shows are scanned? E.g. I would like to keep certain shows forever while I do want to delete some just after I have watched them.

Stacked videos do not get deleted correctly. These are videos that consist of several parts, named video.cd1.ext, video.cd2.ext, etc.

I get an Error accessing video library database with the 3.0.0 rep on XBMC Frodo. Any ideas or is this a normal error? I dit a clean re-install of Frodo but problem doesn't solve. It operates on Win8.

It would be nice to have different settings for Movies vs. TV.

e.g. Movies I want to delete watched with ratings < 6, but TV I want to delete all watched.

Often when large libraries need to be cleaned there is no visual indication of any action being performed by the cleaner. This may cause users to start cleaning twice or more. When users choose to manually clean, we should show its progress, preferably using a dialog that can be cancelled.

• Show a dialog during cleaning
• Allow users to cancel cleaning via the dialog

ShedDweller: "Would it be possible to have an option that always leaves the last episode, or last few episodes of a show when running the cleanup. So, rather than deleting or moving the files based on age, it's based on the number of files.

Under the conditions settings page have the option "Leave a certain number of the most recent files" which when set to "on" would give you a choice of how many files to leave."

• Add setting in Conditions tab,
• Group to be cleaned episodes per TV series,
• Sort episodes per TV series,
• Remove the X episodes with the highest episode number,
• Clean the remaining episodes in the group,