Subdirectories are excluded in a way distinct from regular .gitignore rules about skypilot

[fozziethebeat]

I fixed it with updating my gitignore to have /data/* instead of data/*. That's goo enough for now but a less surprising fix would be nice.

[fozziethebeat]

Here's an example repository that showcases this more comprehensively: https://github.com/fozziethebeat/skypilot-bug/tree/broken-branch

[concretevitamin]

Thanks @fozziethebeat!

It appears the reason is: SkyPilot builds up an rsync command that looks like

rsync -Pavz --filter='dir-merge,- .gitignore' -e "ssh -i ~/.ssh/sky-key -o StrictHostKeyChecking=no ..."  <local src> ubuntu@<ip>:'~/sky_workdir'

The --filter='dir-merge,- .gitignore' part is supposed to mean "merge any subdir's .gitignore" but somehow recursively applies top level .gitignore to subdirs. Changing it to

--filter='- .gitignore'

made this problem go away.

Do you have a workaround for now? And perhaps the better way for us to handle this is to parse from git for a list of things to exclude, then use some other arg to pass them to rsync.