.kopiaigonre file to partly backup Windows NTFS file systems from Linux

General Topics
1

The environment:

  • Dualboot notebook: Linux Mint 21.3 and Windows 10
  • Current system running: Linux (Mint 21.3)
  • Windows NTFS partition (on same notebook) mounted at /win-c, and /win-d

I’m trying to setup kopia to backup parts of the NTFS partition mounted at /win-d with a policy to backup /, excluding everything but the wanted directories under /win-c. I’ve been trying .kopiaignore (llocated in the root /) with no success so far.

Say the file system structure looks lke this:
/
+-- bin/
+-- etc/
...
+-- win-d
+-- Dont backup/
| +-- some files
+-- Backup this/
| +-- some file
| +-- anotherfile
| +-- someDirectory/
+-- @Backup2

I tried with in .kopiaignore (in /)

# exclude everything
*
# Include NTFS data
!/win-d/Backup This/
!/win-d/@Bachup2/

But no success. I also tried adding *, and */ to the “don’t exclude” lines.

How should the .kopiaignore look like?

2 
/*

!/win-d
/win-d/*
!/win-d/Backup This
!/win-d/@Bachup2
3

That doesn’t work. It still excludes everything.

Creating a new snapshot and clicking on “Estimate”, the summary line sais:
Total Bytes: 0 B (5.5 MB excluded) Files: 0 (**19** excluded) Directories: 2 (61 excluded) Errors: 0 (0 ignored)

Clicking at “ShoW Log” I can see all the Linux root level direcroties and the “Windows” second level directories in the list marked as “excluded”. Unfortunately, it doesn’t tell which are the two included directrories.

4

I have just tested and it works without any problem.

$ tree .
.
├── bin
├── etc
└── win-d
    ├── backup_this
    ├── backup_this_too
    └── do_not_backup

$ cat .kopiaignore
/*
!/win-d
/win-d/*
!/win-d/backup_this
!/win-d/backup_this_too

Now I backup this directory with kopia with and without .kopiaignore.

I mount repo (kopia mount all /path/to/mount --webdav) and here you are results:

snapshot without .kopiaignore:

$ tree 20241208-194044
20241208-194044
├── bin
├── etc
└── win-d
    ├── backup_this
    ├── backup_this_too
    └── do_not_backup

7 directories, 0 files

snapshot with .kopiaignore:

$ tree 20241208-194258
20241208-194258
└── win-d
    ├── backup_this
    └── backup_this_too

4 directories, 0 files

Make sure that your policies contain (kopia policy show /path/of/snapshot/data) .kopiaignore:
Make sure that you do not limit your backup to single filesystem.

  Read ignore rules from files:                inherited from (global)
    .kopiaignore
...
  Scan one filesystem only:            false   inherited from (global)

Otherwise I am not sure what you are doing wrong as you do not provide any details.

kopia does not have concept of include. Snapshot contains all data down from its root minus excluded items.

5

Thanks a lot for your time!
I guess your win-d directory is not the mount point of a NTFS file system mounted there, is it?

Yes, I understand this. I saw this in the log:

10:48:21.436 excluded dir var
10:48:21.436 Included files: None
10:48:21.437 Excluded files 10 KB ...100 KB: 1 files, total size 88.7 KB

The line Included file: None is a bit confusing.

Yes, I saw this. Here is what the show policysais:

...
Files policy:
  Ignore cache directories:             true   inherited from (global)
  No ignore rules:
  Read ignore rules from files:                inherited from (global)
    .kopiaignore
  Scan one filesystem only:            false   inherited from (global)
...

After all, my real structure is much too deep to be able to post here, so I just created a new directory win-t and some subdirectories, then copied some real files. Just to be able to have a something I can post lists here.

This is the structure of /win-t

$ tree /win-t
/win-t
├── 195_110.jpg
└── @Data
    ├── dont_backup
    │   ├── dira
    │   │   └── Mappe2.xlsx
    │   └── Mappe1.xlsx
    ├── dont_backup2
    │   └── Verbrauch_TG_ab_1995.xlsx
    ├── @eDokumente
    │   ├── 2up druck 2.pdf
    │   ├── 2up druck 3.pdf
    │   └── 2up druck.pdf
    ├── Gruppenfoto_Männerriege_2024_klein.jpg
    └── @Peter
        ├── Basisdaten_TG_ab_1995.pdf
        ├── Dir1
        │   ├── metanet-list-admin-error.png
        │   └── metanet-list-overview.png
        ├── dir2
        │   ├── metanet-list-overview.png
        │   └── _read_me.txt
        └── Information neues Zulassungsregime Fahrzeuge.pdf

and this is the .kopiaignore file:

$ cat /.kopiaignore
/*
!/win-t
/win-t/*
!/win-t/@Data/@eDokumente
!/win-t/@Peter

I really don’t see any difference to your test setup, except from some of the names starting with @ in my environment. To be sure the @is not the problem, I renamed and removed all @with the same result: Noting gets backed up.

At least I can say it is not an issue with my real data being in an NTFS file system permanently mounted. Hmmmm…

6

Ok. But now your dir structure is different than in your original example so obviously .kopiaignore structure has to be different too:)

You have additional sub level

/wit-t/@Data/backup-this
/wit-t/@Data/backup-this-not

vs. originally:

/wit-t/backup-this
/wit-t/backup-this-not

You have to allow data on every level:

/*
!/win-t
/win-t/*
!/win-t/@Data
/win-t/@Data/*
!/win-t/@Data/@eDokumente
!/win-t/@Data/@Peter

As when you exclude certain level nothing one level below can be “included/excluded” again (as it is invisible). So we have to bring it back to scope and then exclude everything else.

It is a bit confusing initially:) But run few experiments and hopefully you will see how it works.

And simple include option would make life so much easier in cases like this… But we have to work with what we have.

7

It is not really different, I just didn’t post every level. Didn’t expect this to be important (form exerience so far with the (windoes) backup tools I was using so far).

It indeed is.

The paths to my data to be backed has undefined depth, i.e. I often create new subdirectories, and want they’ll be backed up without the need to think about modifying .kopiaigonre each time. I’d surely forget often, if I had to.

I’ll try to come up with the required .kopiaignore content.

In the docs about Supported Patterns in .kopiaignore, I read:

*    wildcard that matches any character zero or multiple times|
**   Double `Wildcard` that matches zero or multiple directories|

What exactly is the difference? Since * matches any character I was assuming this includes / and thus including one or more subdirectory levels, with, or without a filename at the end.

It now seems, * only matches any character “between to slashes”, i.e. either a single directory name or a file name, stopping at the next /.

So, would ** help me out to keep my .kopiaignore simpler? If so, could you give me a starter?

Thanks again for your valuable help!

8

* does not match / I think

No idea:) depends on what you want to do.

This is not an issue at all.

/*
!/win-t
/win-t/*
!/win-t/@Data
/win-t/@Data/*
!/win-t/@Data/@eDokumente
!/win-t/@Data/@Peter

will backup everything (with subfolders) in /win-t/@Data/@eDokumente and /win-t/@Data/@Peter.

Maybe it is yet another confusion:) When you define dir to backup (excluding other dirs at the same level) it contains all this dir content (files and dirs).

9

I’m finally on my way. First two subdirs are selected for backup. I’m sure now, I’ll manage to include everything needed.

For the records, here is how my .kopiaignore looks like so far:


# Bring the /win-d directory back to scope, then exclude everything below it
# This makes all the directories at the /win-d/ level available for "do not exclude"
!/win-d
/win-d/*

# Bring the /win-d/@Data directory back to scope, then exclude everything below it
# This makes all the directories at the /win-d/@Data/ level available for "do not exclude"
!/win-d/@Data
/win-d/@Data/*

# Bring the /win-d/@Data/@Peter directory back to scope, then exclude everything below it
# This makes all the directories at the /win-d/@Data/@Peter level available for "do not exclude"
!/win-d/@Data/@Peter
/win-d/@Data/@Peter/*

# Now select the directories to "include" at the /win-d/@Data/@Peter/ level
!/win-d/@Data/@Peter/#-Documents
!/win-d/@Data/@Peter/#-Folders

Wow, is this fast! Incredibly fast.

Thanks again for bearing with me

1 Like