Issue with file/folder ignore rules

Support
1

I am using Kopia to backup my home folder on my MBP, which is /Users/budy. I ran across an issue with some files in my Library/Cache dir, so I decided to exclude the Cache dir completely. I created a new rule for this and it looked like this:

budy@budy ~ % kopia policy show  bduy@budy:/Users/budy
Policy for budy@budy:/Users/budy:

Retention:
  Annual snapshots:    3           inherited from (global)
  Monthly snapshots:  24           inherited from (global)
  Weekly snapshots:    4           inherited from (global)
  Daily snapshots:     7           inherited from (global)
  Hourly snapshots:   48           inherited from (global)
  Latest snapshots:   10           inherited from (global)

Files policy:
  Ignore rules:
    **/Library/Caches**            (defined for this target)
    **controlmaster-**             (defined for this target)
    /Users/budy/Library/Google/** (defined for this target)

  Read ignore rules from files:
    .kopiaignore                   inherited from (global)

However, this atually didn’t work and Kopia again stumled over a file having strange permissions on it. So either my rules are not right, or Kopia still enters these directories, but maybe just doesn’t upload the files/folders which are under ~/Library/Chaches.

2

try kopia snapshot estimate to figure out which files are excluded without uploading anything.

3

I will try that, but how are there ignore rules to be meant? Are they stating absolute paths or relative one to the root of the snapshot? And should a rule, which ignore everything under a certain target, say /tmp/** prevent Kopie from trying to read/enter the directory in the first place?

4

I see, the ignore rules have to be created relative to the snapshot root. kopia snapshot estimate showed that - great!

5

If I may add to that, the rules defined in a .kopiaignore file are relative to the folder where that file lives. Hence, we can use more than one.

For example, I have ~/.kopiaignore that ignores .Trash/ (who would want to back up garbage?).

I have ~/Library/.kopiaignore that ignores .Cache/ (among other things)

Then I also have ~/Downloads/.kopiaignore that contains

*.app
*.bz2
*.dmg
*.gz
*.tar
*.zip

as those are all downloadable content that I don’t need to back up.
But I don’t want to ignore those file types from everywhere in my $HOME.

Hope that helps.

6

Hi @budy, sorry to re-open this old thread, but I don’t fully understand by “the ignore rules have to be created relative to the snapshot root”.

I am trying to run a full linux system backup, and I included the following ignore rules to my host policy:

Files policy:
  Ignore cache directories:        true       (default)
  Ignore rules:
    /dev                           (defined for this target)
    /home/*/.cache/*               (defined for this target)
    /media                         (defined for this target)
    /mnt                           (defined for this target)
    /proc                          (defined for this target)
    /run                           (defined for this target)
    /sys                           (defined for this target)
    /tmp                           (defined for this target)
    /var/cache/*                   (defined for this target)
    /var/tmp/*                     (defined for this target)
    lost+found                     (defined for this target)
  Read ignore rules from files:
    .kopiaignore                   inherited from (global)
  Scan one filesystem only:       false       (default)

I found kopia snapshot create / to run indefinitely because of new files continuously getting created in /proc, which made me realize there was a problem with the ignore rules.

Do you have any idea about how one would go to ignore those directories without ignoring any subfolder matching e.g. tmp? I would rather avoid adding .kopiaignore all over the root filesystem, which is why I was trying to solved this using policies.

Thanks in advance!
Simone

7

You don’t need to use .kopiaignore files - instead you can define policies at all those directories that exclude specific files in them:

$ kopia policy set --add-ignore '*' ~/Downloads /tmp
Setting policy for jarek@jareks-mbp:/Users/jarek/Downloads

 - adding "*" to "ignore rules"

Setting policy for jarek@jareks-mbp:/tmp

 - adding "*" to "ignore rules"
8

I see, thank you @jkowalski, that’s very useful and I didn’t know it was possible! To make sure I understand correctly how policies work, this means that even kopia snapshot create / will recursively apply the policies of any given subfolder, and not just the policies of /, @host or global, right?

9

Another quick question, am I correct assuming that this pull request is now integrated in version 0.8.4? I have just quickly tested adding paths prefixed with / to one of my .kopiaignores and that seems to work now (it wasn’t the case the last time I tested it).

10

That’s exactly right - you specify which files you don’t want backed up via policies and/or .kopiaignore and it can be either global or at host, user or any directory level and the rules are inherited from the parent except when policy specifically disallows that.

11

Also yes, the pull request is now integrated into 0.8.4. Good to know that it improved things.

12

That’s great, thank you very much!

13

Quick question to better understand relationship of ignore rules on different policies.

Say I have a global policy with an ignore rule for FOO.

I also have a local policy with an ignore rule for BAR

When running a backup, should I expect both FOO & BAR to be ignored because global FOO ignore applies to all and BAR ignore applies locally? OR should I expect only BAR to be ignored because local overrides global?

Currently I’m observing global policy getting ignored due to an override of ignore rules on local. I am NOT disabling parent policy with the "noParent": true option on the local policy.

My guess is that I should keep my main ignore rules in global policy then just use .kopiaignore files to enforce the local rules I want to apply. I’m trying to keep as much as possible global to make backing up 5 sources easy. :grinning:

I just stumbled upon Kopia this week and I’m learning a lot about it and I’m feeling fairly confident that it is the backup utility I’ve been looking for.