I have a non-standard use case. I am snapshotting a GCS bucket with “autoclass” enabled (basically as long as file contents are not read for 30 days, the object will auto transition to progressively colder storage). However, none of my files are transitioning to colder storage.
What I believe is happening is that when GCS transitions a file from one class to another, it updates the mtime modification time. Then the next time Kopia runs (usually nightly), Kopia sees that this file is modified and reads it to rehash. This read causes the file to move from cold storage (Nearline) to Standard storage - essentially never letting the file move to Nearline/cold storage.
How can I configure Kopia to ignore the mtime when choosing to re-hash a file? Or more specifically, how does Kopia decide to hash a file and check for changes?
Clone kopia github source code and make required changes creating your own custom kopia version. Beauty of open source.
Your use case is so specific that I doubt you will find anybody interested in adding it to the kopia where mtime is the key indicator used for detecting changed files.
I would also investigate GCS documentation - storage class transition should not change file mtime… Maybe problem is somewhere else? How you make kopia to access GCS bucket? Is it mounted in some way into local filesystem? If yes - maybe it is mounting layer messing up mtime? Is it the same mount all the time? Or you mount and unmount it all the time?
Fair enough - although I suspect I won’t be the only one asking (even if I’m the first ). Rsync for example has a --size-only flag which skips files that have the same size (ignoring time).
Kopia generally avoids rehashing files if the attributes (name, modification time and size) match the last snapshot - it will simply assume that the file has not changed (this behavior can be overridden).
I’m interested in exactly how to override this behavior. I couldn’t find it in the docs, but I will take a look at the source code.
I think that qoute works in the other direction, it is in a text describing that kopia doesn’t have to re-check contents again on a file if the mtime,name,size hasn’t changed but that this behaviour can be overriden so it actually does check contents even if the file metadata is identical, not that you can override the mtime,name,size-check and not even check those.
). Rsync for example has a