samuelmeuli / tmignore Goto Github PK

View Code? Open in Web Editor NEW

317.0 9.0 3.0 169 KB

🕔 Exclude development files from Time Machine backups

License: MIT License

Makefile 0.65% Swift 99.35%

time-machine macos script homebrew-formula

tmignore's People

Contributors

Stargazers

Watchers

Forkers

frizlab d10xa kasukasu-collection

tmignore's Issues

Errors with high number of exclusions

terminal output:

2019-10-17 08:44:31.001 tmignore[75321:5970347] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'Failed to set posix_spawn_file_actions for fd -1 at index 0 with errno 9'
*** First throw call stack:
(
	0   CoreFoundation                      0x00007fff2cbf8a7d __exceptionPreprocess + 256
	1   libobjc.A.dylib                     0x00007fff572efa17 objc_exception_throw + 48
	2   CoreFoundation                      0x00007fff2cbf88af +[NSException raise:format:] + 201
	3   Foundation                          0x00007fff2edeca88 -[NSConcreteTask launchWithDictionary:error:] + 3172
	4   tmignore                            0x000000010216314c $s8tmignore10runCommand7commands5Int32V6status_SSSg6stdoutAG6stderrtSS_tFTf4x_n + 444
	5   tmignore                            0x0000000102161dca $s8tmignore11TimeMachineC12addExclusion4pathySS_tFZTf4xd_n + 202
	6   tmignore                            0x000000010215baaa $s8tmignore10RunCommandC7executeyyKFTf4d_n + 3626
	7   tmignore                            0x0000000102159639 $s8tmignore10RunCommandC8SwiftCLI0C0AadEP7executeyyKFTW + 9
	8   tmignore                            0x00000001021085f8 $s8SwiftCLI0B0C2go4withs5Int32VAA12ArgumentListC_tF + 1352
	9   tmignore                            0x0000000102107f97 $s8SwiftCLI0B0C9goAndExits5NeverOyF + 119
	10  tmignore                            0x000000010215956d main + 557
	11  libdyld.dylib                       0x00007fff58abd3d5 start + 1
)
libc++abi.dylib: terminating with uncaught exception of type NSException
Abort trap: 6

Add blacklist support

Ignore git submitted files, such as LFS files, which are usually large

Option to restrict ignore to folders instead of files

Wonderful tool!

Due to some package managers using src mode I have a multitude of gitignored folders which have subfolders with a gitignore. This results in a listing like:

- /Users/jpmschuler/Documents/workspace/myproject/.Build/nextlts/vendor/somepackage/ComposerPackagesCommands.php
- /Users/jpmschuler/Documents/workspace/myproject/.Build/staging/vendor/somepackage/ComposerPackagesCommands.php
- /Users/jpmschuler/Documents/workspace/myproject/.Build/nextlts2/vendor/somepackage/Console/ComposerPackagesCommands.php
- /Users/jpmschuler/Documents/workspace/myproject/.Build/production/vendor/somepackage/ComposerPackagesCommands.php
- /Users/jpmschuler/Documents/workspace/myproject/.Build/ddev/vendor/somepackage/ComposerPackagesCommands.php
- /Users/jpmschuler/Documents/workspace/myproject/.Build/dev/vendor/somepackage/ComposerPackagesCommands.php
- /Users/jpmschuler/Documents/workspace/myproject/.Build/

where the last entry would be sufficient

 - /Users/jpmschuler/Documents/workspace/myproject/.Build/

However it came to mind that this only happens with subfiles to an excluded dir, not subfolders. This led to my conclusion, that I don't get why to exclude single files anyways, as it would only be some single files and lead to a load of ignore configuration, which isn't that helpful.

So I'd suggest to have an option (or even default) to only exclude folders.

Proof that this might make sense: a screens long list if I tmignore list | grep .DS_Store

Why not publish to homebrew?

Simplify install method and steps.

node_modules being backed up despite being tmignored (?)

I've been using tmignore happily in the past, it always worked well but in the recent backup, I see several node_modules using the BackupLoupe utility:

The yarn-install-demo is actually this public repo: https://github.com/borekb/yarn-install-demo where you can see two .gitignore files:

I did a quick tmignore check which looks alright:

$ tmignore list | rg 'yarn-install-demo'
  - /Users/borekb/dev/borekb/yarn-install-demo/packages-v1/app/node_modules/
  - /Users/borekb/dev/borekb/yarn-install-demo/packages-v1/lib/node_modules/
  - /Users/borekb/dev/borekb/yarn-install-demo/packages-v1/node_modules/
  - /Users/borekb/dev/borekb/yarn-install-demo/packages-v2/.yarn/build-state.yml
  - /Users/borekb/dev/borekb/yarn-install-demo/packages-v2/.yarn/cache/
  - /Users/borekb/dev/borekb/yarn-install-demo/packages-v2/.yarn/install-state.gz
  - /Users/borekb/dev/borekb/yarn-install-demo/packages-v2/app/node_modules/
  - /Users/borekb/dev/borekb/yarn-install-demo/packages-v2/lib/node_modules/
  - /Users/borekb/dev/borekb/yarn-install-demo/packages-v2/node_modules/

How comes that .yarn cache files are excluded from the backup but node_modules aren't? I also ran this, which again contained only .yarn/* entries and no node_modules entries:

$ find /Users/borekb/dev/borekb/yarn-install-demo -exec tmutil isexcluded {} + | grep -F "[Excluded]" | sed -E 's/^\[Excluded\][[:space:]]*//'
/System/Volumes/Data/Users/borekb/dev/borekb/yarn-install-demo/packages-v2/.yarn/cache
/System/Volumes/Data/Users/borekb/dev/borekb/yarn-install-demo/packages-v2/.yarn/cache/fb-watchman-npm-2.0.0-98fd97d6b7-192298457e.zip
/System/Volumes/Data/Users/borekb/dev/borekb/yarn-install-demo/packages-v2/.yarn/cache/elegant-spinner-npm-1.0.1-8b799f39a6-69837a8a88.zip
...

(The yarn-install-demo folder on my disk is about 3 days old while the Time Machine backup was run 1 hour ago, so there shouldn't be any problem with the node_modules folder not being known to tmignore yet.)

Any ideas?

Installation via homebrew fails (missing xctest)

Hi,

I tried to install the tool as described in project readme.

$ brew install samuelmeuli/tap/tmignore
==> Tapping samuelmeuli/tap
Cloning into '/usr/local/Homebrew/Library/Taps/samuelmeuli/homebrew-tap'...
remote: Enumerating objects: 40, done.
remote: Counting objects: 100% (40/40), done.
remote: Compressing objects: 100% (20/20), done.
remote: Total 40 (delta 17), reused 33 (delta 10), pack-reused 0
Receiving objects: 100% (40/40), 10.13 KiB | 5.06 MiB/s, done.
Resolving deltas: 100% (17/17), done.
Tapped 1 formula (28 files, 37.4KB).
==> Installing tmignore from samuelmeuli/tap
==> Downloading https://github.com/samuelmeuli/tmignore/archive/v1.1.1.tar.gz
==> Downloading from https://codeload.github.com/samuelmeuli/tmignore/tar.gz/v1.1.1
######################################################################## 100.0%
==> make build
Last 15 lines from /Users/dennis/Library/Logs/Homebrew/tmignore/01.make:
2020-03-25 14:18:01 +0100

make
build

swift build --configuration release --disable-sandbox
error: terminated(72): xcrun --sdk macosx --find xctest output:
    xcrun: error: unable to find utility "xctest", not a developer tool or in PATH

make: *** [build] Error 1

I am running macOS 10.15.3 (with current Xcode 11.3.1).

Do you have an idea what's going wrong here? I don't have installed any special Xcode tools, yet, but only did xcode-select --install once some time ago.
Any help is appreciated.

Greetings,
Dennis

tmignore fails to install via Homebrew on macOS Big Sur

On macOS Big Sur (11.1) the tmignore installation step from the README.md file fails with the following error:

brew install samuelmeuli/tap/tmignore
==> Installing tmignore from samuelmeuli/tap
==> Downloading https://github.com/samuelmeuli/tmignore/archive/v1.2.2.tar.gz
Already downloaded: /Users/seb/Library/Caches/Homebrew/downloads/694e9e2acfa7fc1d36e4be32543d766098b853eaf69a98ec5ec66fbc803d0abd--tmignore-1.2.2.tar.gz
==> make build
Last 15 lines from /Users/seb/Library/Logs/Homebrew/tmignore/01.make:

Fetching https://github.com/jakeheis/SwiftCLI

Fetching https://github.com/samuelmeuli/swift-exec

Fetching https://github.com/apple/swift-log

Fetching https://github.com/SwiftyJSON/SwiftyJSON

Fetching https://github.com/IBM-Swift/HeliumLogger

xcodebuild: error: Could not resolve package dependencies:
  sandbox-exec: sandbox_apply: Operation not permitted

make: *** [build] Error 74

If reporting this issue please do so at (not Homebrew/brew or Homebrew/core):
  https://github.com/samuelmeuli/homebrew-tap/issues

These open issues may also help:
tmignore Build for macOS Big Sur  https://github.com/samuelmeuli/homebrew-tap/issues/2

Possibly related to:

Workaround

However, checking out this repo and running make manually does correctly build bin/tmignore successfully.

Setup Xcode

Install Xcode from the App Store
sudo xcode-select -s /Applications/Xcode.app/Contents/Developer
sudo xcodebuild -license accept

Manually build and Install tmignore

git clone https://github.com/samuelmeuli/tmignore.git && cd tmignore
make
cp bin/tmignore /usr/local/bin/
crontab -l | { cat; echo "0 10 * * * /usr/local/bin/tmignore run &>> /tmp/tmignore.log"; } | crontab -

Compatibility with Swift 5.0

I have Xcode 10.3 and Swift 5.0 and I dont wish to upgrade these for many complains about biggy Xcode, besides the giga size of download which I cant do in my current setup.

Is it possible to bring the compatibility to 5.0 Swift so it can compile in my Mac?

xyz/tmignore: error: package at 'xyz/tmignore' is using Swift tools version 5.1.0 
but the installed version is 5.0.0
make: *** [build] Error 1

Homebrew install fails

Hi, when doing brew install samuelmeuli/tap/tmignore I get this error:

xcodebuild: error: Could not resolve package dependencies:
  sandbox-exec: sandbox_apply_container: Operation not permitted

make: *** [build] Error 74

I'm running as standard user (no admin) if that matters. Make log is attached.

01.make.txt

Error handling path names with spaces

I get an error log for every path that contains spaces.

Example output:

2019-10-17T08:42:47+0200 error: Failed to add Time Machine exclusion: /Users/mario/Sites/CraftCMS/craftcms-starter/project_data/know: Error (-43) while attempting to change exclusion setting.
how/.DS_Store: Error (-43) while attempting to change exclusion setting.

Failed to add backup exclusions: /bin/bash: -c: line 0: unexpected EOF while looking for matching `''

Hi,
first of all thanks for your work! I just found out about your tool.

After I ran it for the first time - I got the following error:

stefan@Stefans-MBP ~ % tmignore run
Searching for Git repositories in /Users/stefan…
Found 88 Git repositories in /Users/stefan
Found 88 Git repositories in total
Building list of files to exclude from backups…
Identified 1110 paths to exclude from backups
Adding backup exclusions for 1110 paths…
Failed to add backup exclusions: /bin/bash: -c: line 0: unexpected EOF while looking for matching `''
/bin/bash: -c: line 1: syntax error: unexpected end of file
Failed to add backup exclusions: /bin/bash: -c: line 0: unexpected EOF while looking for matching `''
/bin/bash: -c: line 1: syntax error: unexpected end of file
Failed to add backup exclusions: /bin/bash: -c: line 0: unexpected EOF while looking for matching `''
/bin/bash: -c: line 1: syntax error: unexpected end of file
Failed to add backup exclusions: /Users/stefan/Development/Projects/01.Indegrade/Spokesman/Documents/Result PDFs/Summer/2019/Biking/.DS_Store /Users/stefan/Development/Projects/01.Indegrade/Spokesman/SourceCode/Spokesman.WebWidget/src/dist/ /Users/stefan/Development/Projects/01.Indegrade/Spokesman/SourceCode/Spokesman.Api.Next/src/Spokesman.WebApi/uploads/2020-02-14_11-13-12_KVRS-Giant-Slalom_2019-01-10.csv /Users/stefan/Development/Projects/01.Indegrade/Spokesman/SourceCode/Spokesman.Api.Next/src/Spokesman.WebApi/uploads/2020-02-13_22-31-26_KVRS-Giant-Slalom_2019-01-10.csv /Users/stefan/Development/VMs/.DS_Store /Users/stefan/Development/VMs/VMs.iml /Users/stefan/Development/Prototypes/mlnet-workshop-1/src/Shared/obj/ /Users/stefan/Development/foo/SourceCode/foo-website-v2-2/wp-blog-header.php /Users/stefan/Development/Projects/01.Indegrade/Spokesman/SourceCode/Spokesman.Api.Next/src/Spokesman.Application/Races/Queries/.DS_Store /Users/stefan/Development/Projects/01.Indegrade/Spokesman/SourceCode/Spokesman.Api.Next/src: Error (-50) while attempting to change exclusion setting.
PDFs/Summer/2019/.DS_Store: Error (-43) while attempting to change exclusion setting.

Please let me know if you need more infos.

`tmignore list` is difficult for other programs to parse

Summary: tmignore list is hard for other programs to parse. It would be awesome if tmignore list -0 would print just an ASCII NUL-separated list.

When I run tmignore list I get:

119 files/directories have been excluded from backups by tmignore:

  - /Users/nop/gh/project1/DerivedData/
  - /Users/nop/gh/project1/project1.xcodeproj/project.xcworkspace/xcuserdata/
[...]

I wanted to see how much space I was saving.

$ tmignore list | awk '/ - \// {print $3}' | xargs du -hsc
du: /Users/nop/gh/project1/DerivedData/\033[0;39m: No such file or directory
du: /Users/nop/gh/project1/project1.xcodeproj/project.xcworkspace/xcuserdata/[0;39m: No such file or directory
[...]

tmignore is emitting some kind of ANSI terminal code at the end of each line. That's rude if stdout isn't a terminal.

A more-working awk one-liner is:

$ tmignore list | awk -F$"( |\033)" '/ - \// {print $5}' | xargs du -hsc

Ideally, tmignore list -0 (or some other command) would produce a list of files separated by ASCII NUL characters, '\0'. The find, grep, and xargs commands support NUL-separated lists of files. (I really wish POSIX would prohibit filenames with newlines in them; as it is, only / and \0 are the only characters prohibited in filenames.)

Fail to install on high sierra with brew

I am trying to install the package with brew on OS X 10.13 high sierra but getting this error

==> Installing tmignore from samuelmeuli/tap
==> Downloading https://github.com/samuelmeuli/tmignore/archive/v1.2.2.tar.gz
Already downloaded: /Users/danieldoynov/Library/Caches/Homebrew/downloads/694e9e2acfa7fc1d36e4be32543d766098b853eaf69a98ec5ec66fbc803d0abd--tmignore-1.2.2.tar.gz
==> make build
Last 15 lines from /Users/danieldoynov/Library/Logs/Homebrew/tmignore/01.make:
Build system information
error: Unable to resolve build file: XCBCore.BuildFile (missingTargetProductReference("PACKAGE-PRODUCT:Logging")) (in target 'tmignore')

Build system information
error: Unable to resolve build file: XCBCore.BuildFile (missingTargetProductReference("PACKAGE-PRODUCT:SwiftCLI")) (in target 'tmignore')

Build system information
error: Unable to resolve build file: XCBCore.BuildFile (missingTargetProductReference("PACKAGE-PRODUCT:SwiftExec")) (in target 'tmignore')

Build system information
error: Unable to resolve build file: XCBCore.BuildFile (missingTargetProductReference("PACKAGE-PRODUCT:SwiftyJSON")) (in target 'tmignore')

** ARCHIVE FAILED **

make: *** [build] Error 65

Do not report this issue to Homebrew/brew or Homebrew/core!

These open issues may also help:
tmignore Build for macOS Big Sur  https://github.com/samuelmeuli/homebrew-tap/issues/2

Any ideas on what could cause this or is this package just not supported on 10.13?

Allow configuration of folder(s) to search in

My code base is in a different mounted partition and a specific projects folder.
The run only scanned the vim plugins in home folder and done.
Can tmignore run and scan using the current folder?

Unable to install: full installation of Xcode required

Hi! I found very useful your tool and I wanted to give it a try but when I try to install it:
brew install samuelmeuli/tap/tmignore
Homebrew throws this error:
A full installation of Xcode.app 10.0 is required to compile this software. Installing just the Command Line Tools is not sufficient.
Am I doing something wrong or truly I need to have full Xcode installed in order tu install and run tmignore? I'm using MacOSX Mojave.
Thank you very much for your help!

Anybody willing to maintain a fork?

It looks like @samuelmeuli hasn't been active on this repo for over a year. If somebody wants to maintain a fork that would be awesome, it seems like a great piece of code. @Frizlab maybe? :-)

Ability to list ignored files

Right now i don't see the ignored files in the macOS Timemachine GUI. Is it possible that tmignore lists the ignored files?

running tmignore added 341 files:
2019-10-21T10:42:57+0200 info: Identified 341 paths to exclude from backups
2019-10-21T10:42:57+0200 info: Excluding 341 files/directories from future backups…