Issue with file/folder ignore rules

Support
1

I am using Kopia to backup my home folder on my MBP, which is /Users/budy. I ran across an issue with some files in my Library/Cache dir, so I decided to exclude the Cache dir completely. I created a new rule for this and it looked like this:

budy@budy ~ % kopia policy show  bduy@budy:/Users/budy
Policy for budy@budy:/Users/budy:

Retention:
  Annual snapshots:    3           inherited from (global)
  Monthly snapshots:  24           inherited from (global)
  Weekly snapshots:    4           inherited from (global)
  Daily snapshots:     7           inherited from (global)
  Hourly snapshots:   48           inherited from (global)
  Latest snapshots:   10           inherited from (global)

Files policy:
  Ignore rules:
    **/Library/Caches**            (defined for this target)
    **controlmaster-**             (defined for this target)
    /Users/budy/Library/Google/** (defined for this target)

  Read ignore rules from files:
    .kopiaignore                   inherited from (global)

However, this atually didn’t work and Kopia again stumled over a file having strange permissions on it. So either my rules are not right, or Kopia still enters these directories, but maybe just doesn’t upload the files/folders which are under ~/Library/Chaches.

2

try kopia snapshot estimate to figure out which files are excluded without uploading anything.

1 Like
3

I will try that, but how are there ignore rules to be meant? Are they stating absolute paths or relative one to the root of the snapshot? And should a rule, which ignore everything under a certain target, say /tmp/** prevent Kopie from trying to read/enter the directory in the first place?

4

I see, the ignore rules have to be created relative to the snapshot root. kopia snapshot estimate showed that - great!

5

If I may add to that, the rules defined in a .kopiaignore file are relative to the folder where that file lives. Hence, we can use more than one.

For example, I have ~/.kopiaignore that ignores .Trash/ (who would want to back up garbage?).

I have ~/Library/.kopiaignore that ignores .Cache/ (among other things)

Then I also have ~/Downloads/.kopiaignore that contains

*.app
*.bz2
*.dmg
*.gz
*.tar
*.zip

as those are all downloadable content that I don’t need to back up.
But I don’t want to ignore those file types from everywhere in my $HOME.

Hope that helps.

6

Hi @budy, sorry to re-open this old thread, but I don’t fully understand by “the ignore rules have to be created relative to the snapshot root”.

I am trying to run a full linux system backup, and I included the following ignore rules to my host policy:

Files policy:
  Ignore cache directories:        true       (default)
  Ignore rules:
    /dev                           (defined for this target)
    /home/*/.cache/*               (defined for this target)
    /media                         (defined for this target)
    /mnt                           (defined for this target)
    /proc                          (defined for this target)
    /run                           (defined for this target)
    /sys                           (defined for this target)
    /tmp                           (defined for this target)
    /var/cache/*                   (defined for this target)
    /var/tmp/*                     (defined for this target)
    lost+found                     (defined for this target)
  Read ignore rules from files:
    .kopiaignore                   inherited from (global)
  Scan one filesystem only:       false       (default)

I found kopia snapshot create / to run indefinitely because of new files continuously getting created in /proc, which made me realize there was a problem with the ignore rules.

Do you have any idea about how one would go to ignore those directories without ignoring any subfolder matching e.g. tmp? I would rather avoid adding .kopiaignore all over the root filesystem, which is why I was trying to solved this using policies.

Thanks in advance!
Simone

7

You don’t need to use .kopiaignore files - instead you can define policies at all those directories that exclude specific files in them:

$ kopia policy set --add-ignore '*' ~/Downloads /tmp
Setting policy for jarek@jareks-mbp:/Users/jarek/Downloads

 - adding "*" to "ignore rules"

Setting policy for jarek@jareks-mbp:/tmp

 - adding "*" to "ignore rules"
8

I see, thank you @jkowalski, that’s very useful and I didn’t know it was possible! To make sure I understand correctly how policies work, this means that even kopia snapshot create / will recursively apply the policies of any given subfolder, and not just the policies of /, @host or global, right?

9

Another quick question, am I correct assuming that this pull request is now integrated in version 0.8.4? I have just quickly tested adding paths prefixed with / to one of my .kopiaignores and that seems to work now (it wasn’t the case the last time I tested it).

10

That’s exactly right - you specify which files you don’t want backed up via policies and/or .kopiaignore and it can be either global or at host, user or any directory level and the rules are inherited from the parent except when policy specifically disallows that.

1 Like
11

Also yes, the pull request is now integrated into 0.8.4. Good to know that it improved things.

1 Like
12

That’s great, thank you very much!

13

Quick question to better understand relationship of ignore rules on different policies.

Say I have a global policy with an ignore rule for FOO.

I also have a local policy with an ignore rule for BAR

When running a backup, should I expect both FOO & BAR to be ignored because global FOO ignore applies to all and BAR ignore applies locally? OR should I expect only BAR to be ignored because local overrides global?

Currently I’m observing global policy getting ignored due to an override of ignore rules on local. I am NOT disabling parent policy with the "noParent": true option on the local policy.

My guess is that I should keep my main ignore rules in global policy then just use .kopiaignore files to enforce the local rules I want to apply. I’m trying to keep as much as possible global to make backing up 5 sources easy. :grinning:

I just stumbled upon Kopia this week and I’m learning a lot about it and I’m feeling fairly confident that it is the backup utility I’ve been looking for.