# Snapshot migrate v1 to v2

**URL:** <https://kopia.discourse.group/t/snapshot-migrate-v1-to-v2/669>\
**Category:** Support\
**Created:** [October 12, 2021, 4:51am UTC](https://kopia.discourse.group/t/snapshot-migrate-v1-to-v2/669 "2021-10-12T04:51:33Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![saviodsouza](https://yyz2.discourse-cdn.com/free1/user_avatar/kopia.discourse.group/saviodsouza/32/50_2.png) [@saviodsouza](https://kopia.discourse.group/u/saviodsouza)\
**Post date:** [October 12, 2021, 4:51am UTC](https://kopia.discourse.group/t/snapshot-migrate-v1-to-v2/669/1 "2021-10-12T04:51:33Z")

</div>

I just migrated a repository to v2 and would like to share some unexpected results

```auto
:~# kopia snapshot migrate --policies --all --source-config=.config/kopia/kv1.config
 * 0 hashing, 0 hashed (0 B), 0 cached (0 B), uploaded 0 B, estimating...
Migration finished.
default (64KiB) - allocated 102(5) chunks freed 101 alive 1 max 5 free list high water mark: 4
:~$
:~$ rclone size /mnt/backup/kopia
Total objects: 5596
Total size: 116.102 GBytes (124664020487 Bytes)
:~$ rclone size /mnt/backup/kopia.v2/
Total objects: 11135
Total size: 227.851 GBytes (244652958596 Bytes)

```

1. The size difference between repo1 and repo2, nearly double…
2. What does the the error message in red at the end of the migration mean?

I did a `kopia gc run` and `kopia gc run --full` but no changes  
Both the repositories have Compressor: “s2-parallel-8”.

---

<div class="post-metadata">

**Author:** ![jkowalski](https://yyz2.discourse-cdn.com/free1/user_avatar/kopia.discourse.group/jkowalski/32/3_2.png) [@jkowalski](https://kopia.discourse.group/u/jkowalski)\
**Post date:** [October 12, 2021, 4:53am UTC](https://kopia.discourse.group/t/snapshot-migrate-v1-to-v2/669/2 "2021-10-12T04:53:38Z")

</div>

Can you post `kopia content stats` from both repositories?

---

<div class="post-metadata">

**Author:** ![saviodsouza](https://yyz2.discourse-cdn.com/free1/user_avatar/kopia.discourse.group/saviodsouza/32/50_2.png) [@saviodsouza](https://kopia.discourse.group/u/saviodsouza)\
**Post date:** [October 12, 2021, 4:59am UTC](https://kopia.discourse.group/t/snapshot-migrate-v1-to-v2/669/3 "2021-10-12T04:59:06Z")

</div>

# Repo v1

```auto
kopia content stats --config-file=.config/kopia/k1.conf
Count: 100431
Total Bytes: 123.6 GB
Average: 1.2 MB
Histogram:

        0 between 0 B and 10 B (total 0 B)
      226 between 10 B and 100 B (total 14.4 KB)
     1719 between 100 B and 1 KB (total 812.8 KB)
     1623 between 1 KB and 10 KB (total 7.6 MB)
     3739 between 10 KB and 100 KB (total 204.2 MB)
    48927 between 100 KB and 1 MB (total 27.5 GB)
    44197 between 1 MB and 10 MB (total 96 GB)
        0 between 10 MB and 100 MB (total 0 B)

```

# Repo v2

```auto
 kopia content stats
Count: 100470
Total Bytes: 244.6 GB
Average: 2.4 MB
Histogram:

        0 between 0 B and 10 B (total 0 B)
      206 between 10 B and 100 B (total 12.9 KB)
     1226 between 100 B and 1 KB (total 585.6 KB)
     1037 between 1 KB and 10 KB (total 3.7 MB)
      782 between 10 KB and 100 KB (total 23.3 MB)
      240 between 100 KB and 1 MB (total 94.5 MB)
    96979 between 1 MB and 10 MB (total 244.5 GB)
        0 between 10 MB and 100 MB (total 0 B)

```

---

<div class="post-metadata">

**Author:** ![jkowalski](https://yyz2.discourse-cdn.com/free1/user_avatar/kopia.discourse.group/jkowalski/32/3_2.png) [@jkowalski](https://kopia.discourse.group/u/jkowalski)\
**Post date:** [October 12, 2021, 5:10am UTC](https://kopia.discourse.group/t/snapshot-migrate-v1-to-v2/669/4 "2021-10-12T05:10:19Z")

</div>

Did you have compression enabled on both repositories?

---

<div class="post-metadata">

**Author:** ![jkowalski](https://yyz2.discourse-cdn.com/free1/user_avatar/kopia.discourse.group/jkowalski/32/3_2.png) [@jkowalski](https://kopia.discourse.group/u/jkowalski)\
**Post date:** [October 12, 2021, 5:11am UTC](https://kopia.discourse.group/t/snapshot-migrate-v1-to-v2/669/5 "2021-10-12T05:11:51Z")

</div>

In v2 format you should see something similar to:

```auto
Count: 401633
Total Bytes: 697.4 GB
Total Packed: 568.9 GB (compression 18.4%)
By Method:
  (uncompressed) count: 160283 size: 147.9 GB
  pgzip-best-speed count: 241350 size: 549.5 GB packed: 421 GB compression: 23.4%
Average: 1.7 MB
Histogram:

        0 between 0 B and 10 B (total 0 B)
     6654 between 10 B and 100 B (total 472.7 KB)
   107324 between 100 B and 1 KB (total 47.4 MB)
    63772 between 1 KB and 10 KB (total 188.2 MB)
    14666 between 10 KB and 100 KB (total 559.2 MB)
    43786 between 100 KB and 1 MB (total 24 GB)
   165431 between 1 MB and 10 MB (total 544 GB)
        0 between 10 MB and 100 MB (total 0 B)

```

---

<div class="post-metadata">

**Author:** ![saviodsouza](https://yyz2.discourse-cdn.com/free1/user_avatar/kopia.discourse.group/saviodsouza/32/50_2.png) [@saviodsouza](https://kopia.discourse.group/u/saviodsouza)\
**Post date:** [October 12, 2021, 5:28am UTC](https://kopia.discourse.group/t/snapshot-migrate-v1-to-v2/669/6 "2021-10-12T05:28:10Z")

</div>

yes  
Both the repositories have Compressor: “s2-parallel-8”.

---

<div class="post-metadata">

**Author:** ![saviodsouza](https://yyz2.discourse-cdn.com/free1/user_avatar/kopia.discourse.group/saviodsouza/32/50_2.png) [@saviodsouza](https://kopia.discourse.group/u/saviodsouza)\
**Post date:** [October 12, 2021, 6:57am UTC](https://kopia.discourse.group/t/snapshot-migrate-v1-to-v2/669/7 "2021-10-12T06:57:54Z")

</div>

yes it should have

I have another repository in windows with s2-parallel-8 compression and is see the output like yours in content stats.  
But this repository v2 is on windows and it a fresh repository`kopia snap create ......` no migration.

The Repov2 outputs I have posted here was created this way

```auto
kopia policy set --global --compression=s2-parallel-8
for extensions in gns3project 7z rar zip bzip2 gz xz squashfs
do
   kopia policy set --global --add-never-compress=$extensions
done
kopia snapshot migrate --policies --all --source-config=.config/kopia/kv1.config

```

If i am not wrong it appears that there is no compression happening in snapshot migrate atleat for a destination repo which has no snapshots at all…

Wondering if i should recreate my v2 repository again check…

---

<div class="post-metadata">

**Author:** ![saviodsouza](https://yyz2.discourse-cdn.com/free1/user_avatar/kopia.discourse.group/saviodsouza/32/50_2.png) [@saviodsouza](https://kopia.discourse.group/u/saviodsouza)\
**Post date:** [October 12, 2021, 9:51am UTC](https://kopia.discourse.group/t/snapshot-migrate-v1-to-v2/669/8 "2021-10-12T09:51:35Z")

</div>

OK Done my research… This time on a Windows system with same results.

I just tested snapshot migrate with compression enabled on both repositories.  
My guess was right…

> [@saviodsouza](#):
>
> it appears that there is no compression happening in snapshot migrate atleat for a destination repo which has no snapshots at all…

Steps to Reproduce.

```auto
# disconnect
kopia repository disconnect
export KOPIA_PASSWORD="password for kopia repository"
# connect to old repository version 1
kopia repository connect filesystem --path /mnt/kv1 --config-file=k1.config
export KOPIA_PASSWORD="password for kopia repository2"
# create and conencto to repository version 2
kopia repository create filesystem --path /mnt/kv2
kopia repository connect filesystem --path /mnt/kv2
# setup policies for repository version 2
kopia policy set --global --compression=s2-parallel-8
for extensions in 7z rar zip bzip2 gz xz squashfs
do
    kopia policy set --global --add-never-compress=$extensions
done
# Start migration
kopia snapshot migrate --policies --all --source-config=$HOME/.config/kopia/k1.config
#check content status of repository2
kopia content stats

```

Can this be a bug or am I still missing something… 🤔

Thanks 🙂

---

<div class="post-metadata">

**Author:** ![saviodsouza](https://yyz2.discourse-cdn.com/free1/user_avatar/kopia.discourse.group/saviodsouza/32/50_2.png) [@saviodsouza](https://kopia.discourse.group/u/saviodsouza)\
**Post date:** [October 22, 2021, 4:01pm UTC](https://kopia.discourse.group/t/snapshot-migrate-v1-to-v2/669/9 "2021-10-22T16:01:11Z")

</div>

@jkowalski

Is this a bug?

---

<div class="post-metadata">

**Author:** ![jkowalski](https://yyz2.discourse-cdn.com/free1/user_avatar/kopia.discourse.group/jkowalski/32/3_2.png) [@jkowalski](https://kopia.discourse.group/u/jkowalski)\
**Post date:** [October 22, 2021, 4:16pm UTC](https://kopia.discourse.group/t/snapshot-migrate-v1-to-v2/669/10 "2021-10-22T16:16:40Z")

</div>

Yes, it was just fixed yesterday. Will be in v0.9.4 release, hopefully today.

> <https://github.com/kopia/kopia/issues/1429>
>
> Hi,
> 
> After migrating snapshots to a fresh repo, it turns out compression polic…ies were ignored. I used \`kopia snap migrate --all --policies --overwrite-policies\` in order to also copy the global policy enabling zstd-fastest, but results are identical when manually setting it beforehand and omitting \`--policies --overwrite-policies\`. Here you can see that no contents are compressed in the new repo:
> 
> \`\`\`
> $ kopia content stats
> Count: 393332
> Total Bytes: 12.2 GB
> Average: 31.1 KB
> Histogram:
> 
> 0 between 0 B and 10 B (total 0 B)
> 14461 between 10 B and 100 B (total 0.9 MB)
> 86901 between 100 B and 1 KB (total 46.1 MB)
> 205343 between 1 KB and 10 KB (total 651.7 MB)
> 66737 between 10 KB and 100 KB (total 2.1 GB)
> 16565 between 100 KB and 1 MB (total 4.4 GB)
> 3325 between 1 MB and 10 MB (total 5.1 GB)
> 0 between 10 MB and 100 MB (total 0 B)
> \`\`\`
> 
> Is this a bug or am I just overlooking something here?
> 
> Thanks & best regards
> Robert

---

<div class="post-metadata">

**Author:** ![jkowalski](https://yyz2.discourse-cdn.com/free1/user_avatar/kopia.discourse.group/jkowalski/32/3_2.png) [@jkowalski](https://kopia.discourse.group/u/jkowalski)\
**Post date:** [October 22, 2021, 4:17pm UTC](https://kopia.discourse.group/t/snapshot-migrate-v1-to-v2/669/11 "2021-10-22T16:17:21Z")

</div>

You can test it using a build from

> **[Releases · kopia/kopia-test-builds](https://github.com/kopia/kopia-test-builds/releases)**
>
> Pre-releases for Kopia UI. Contribute to kopia/kopia-test-builds development by creating an account on GitHub.
