zfs send/recv command

Moderators: jhartley, MSR734, nola

zfs send/recv command

Post by shuman » Mon Oct 29, 2012 9:26 pm

I'm performing my snapshots with the following command

Code: Select all
zfs snapshot -r zdata@`date +"%F-%H%M%S"`


The snapshots are recursive so each child filesystem has the same snapshot date/time. When performing a zfs send, to move send it to another pool, I'm not sure how to format the command to get it to send all the snapshots from the same date and time

Code: Select all
zbak                  758Ki  1.79Ti   306Ki  /Volumes/zbak
zbak/zdata            206Ki  1.79Ti   206Ki  /Volumes/zbak/zdata
zdata                1.35Ti  1.34Ti  1.19Mi  /Volumes/zdata
zdata/Backups        31.5Gi  1.34Ti   332Ki  /Volumes/zdata/Backups
zdata/Backups/Lion   31.5Gi  1.34Ti  31.5Gi  /Volumes/zdata/Backups/Lion
zdata/Storage        9.76Gi  1.34Ti  9.76Gi  /Volumes/zdata/Storage
zdata/Users           406Gi  1.34Ti   380Ki  /Volumes/zdata/Users
zdata/Users/anika    2.24Gi  1.34Ti   769Mi  /Volumes/zdata/Users/anika
zdata/Users/chris     401Gi  1.34Ti   342Gi  /Volumes/zdata/Users/chris
zdata/Users/jaren    1.27Gi  1.34Ti   611Mi  /Volumes/zdata/Users/jaren
zdata/Users/severin   851Mi  1.34Ti   469Mi  /Volumes/zdata/Users/severin
zdata/Videos          934Gi  1.34Ti   908Gi  /Volumes/zdata/Videos


From the above I'm wanting to send the last snapshot, zdata@2012-10-29-204904, and it's children to zbak/zdata

I've tried this
Code: Select all
zfs send zdata@2012-10-29-204904 | zfs recv -F zbak/zdata

but it isn't working. Not sure what I'm doing wrong. I essentially just want to duplicate the last snapshot filesystem structure to another pool for offsite backup.
- Mac Mini (Late 2012), 10.8.5, 16GB memory, pool - 2 Mirrored 3TB USB 3.0 External Drives
shuman Offline

User avatar
 
Posts: 96
Joined: Mon Sep 17, 2012 8:15 am

Re: zfs send/recv command

Post by grahamperrin » Tue Oct 30, 2012 12:48 am

For the send or receive: you probably need superuser privileges. Option -v on both sides (send and receive) should allow you to see progress.

… how to format the command to get it to send all the send all the snapshots from the same date and time …


If you'd like replication to include all properties, snapshots, descendent file systems, and clones:

  • on the zfs send side, option -R combined with either option -I (uppercase i) or option -i

… I'm a little out of my depth here, and in the past I might have confused replication with recursion (oops!) so it'll be good for someone else to offer an exact command for your use case.

Please, which OS?

(A possible precaution, if descendent file systems are to be replicated, looking ahead: the obscure behaviour outlined under problem child, eject g…grandparent, diskarbitrationd panic … at the moment I can't think of a way to simplify that topic.)
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Re: zfs send/recv command

Post by shuman » Tue Oct 30, 2012 11:06 am

My OS is 10.8. I guess I was looking for what I am familiar with which is rsync; replication. I want my backup to be an exact replication of the source data. Now that I think about, I think I would want the old snapshots as well. Can this be accomplished?

I guess it would look something like:
Code: Select all
tank
tank@1
tank@2
tank@3
tank/data
tank/data@1
tank/data@2
tank/data@3
tank/data/myfiles
tank/data/myfiles@1
tank/data/myfiles@2
tank/data/myfiles@3

send to

zbak/tank
zbak/tank@1
zbak/tank@2
zbak/tank@3
zbak/tank/data
zbak/tank/data@1
zbak/tank/data@2
zbak/tank/data@3
zbak/tank/data/myfiles
zbak/tank/data/myfiles@1
zbak/tank/data/myfiles@2
zbak/tank/data/myfiles@3

Preserving all properties of the source. Any ideas? Beuller Beuller Beuller?
- Mac Mini (Late 2012), 10.8.5, 16GB memory, pool - 2 Mirrored 3TB USB 3.0 External Drives
shuman Offline

User avatar
 
Posts: 96
Joined: Mon Sep 17, 2012 8:15 am

Re: zfs send/recv command

Post by shuman » Fri Nov 02, 2012 10:18 pm

I got the initial 'send' to successfully complete by running:
Code: Select all
zfs send -Rv zdata@2012-10-30-141631 | zfs receive zbak/zdata


Now I'm trying to run an incremental and can't figure out what I'm doing wrong. Examples show:
Code: Select all
zfs send -i tank/data@1 tank/data@2 | zfs receive zbak/tank


Is the problem that I'm trying to do an incremental from data@1 to data@5 ?
- Mac Mini (Late 2012), 10.8.5, 16GB memory, pool - 2 Mirrored 3TB USB 3.0 External Drives
shuman Offline

User avatar
 
Posts: 96
Joined: Mon Sep 17, 2012 8:15 am

Re: zfs send/recv command

Post by grahamperrin » Sat Nov 03, 2012 12:50 am

You might need force at reception.

Please post current output from the following:

Code: Select all
zpool list


Code: Select all
zfs list


In addition, with ZEVO Community Edition 1.1.1, a precaution may be necessary. Please post output from the following:

Code: Select all
zfs list -t all | grep zbak


Comparison

Before I began using the script provided by Patrick I kept a record of some commands and results. The examples below, in chronological order, may help you to decide what will be best for you.

Code: Select all
sh-3.2$ sudo zfs send -v -i zhandy@2012-08-30-221432 zhandy@2012-08-31-204330 | zfs receive -v -F tall/backups/zhandy
sending from @2012-08-30-221432 to zhandy@2012-08-31-204330
receiving incremental stream of zhandy@2012-08-31-204330 into tall/backups/zhandy@2012-08-31-204330
received 863MiB stream in 155 seconds (5.56MiB/sec)


Code: Select all
sudo zfs send -vR -i zhandy@2012-08-31-204330 zhandy@2012-09-05-195520 | zfs receive -v -F tall/backups/zhandy
attempting destroy tall/backups/zhandy@2012-08-30-171355
sending from @2012-08-31-204330 to zhandy@2012-09-05-195520
success
attempting destroy tall/backups/zhandy@2012-08-21-182308
success
receiving incremental stream of zhandy@2012-09-05-195520 into tall/backups/zhandy@2012-09-05-195520
WARNING: could not send zhandy/Pocket Time Machine@2012-09-05-195520: does not exist
received 21.3GiB stream in 1428 seconds (15.3MiB/sec)
sh-3.2$ zfs list -t snapshot | grep zhandy
tall/backups/zhandy@2012-08-18-102525  78.2Mi       -   426Gi  -
tall/backups/zhandy@2012-08-19-110836   645Mi       -   427Gi  -
tall/backups/zhandy@2012-08-22-185347  1.27Gi       -   429Gi  -
tall/backups/zhandy@2012-08-29-165742  3.33Gi       -   440Gi  -
tall/backups/zhandy@2012-08-30-221432  49.9Mi       -   436Gi  -
tall/backups/zhandy@2012-08-31-204330   745Mi       -   437Gi  -
tall/backups/zhandy@2012-09-05-195520       0       -   432Gi  -
zhandy@2012-08-18-102525               76.0Mi       -   439Gi  -
zhandy@2012-08-19-110836               28.9Mi       -   440Gi  -
zhandy@2012-08-20-114215               10.1Mi       -   442Gi  -
zhandy@2012-08-21-185514                180Mi       -   442Gi  -
zhandy@2012-08-22-185347                678Mi       -   442Gi  -
zhandy@2012-08-23-183849                645Mi       -   442Gi  -
zhandy@2012-08-25-121954                702Ki       -   454Gi  -
zhandy@2012-08-29-165742                701Ki       -   454Gi  -
zhandy@2012-08-30-221432               70.8Mi       -   450Gi  -
zhandy@2012-08-31-204330                802Mi       -   451Gi  -
zhandy@2012-09-04-190719               1.81Gi       -   446Gi  -
zhandy@2012-09-05-195520               4.51Mi       -   446Gi  -
sh-3.2$


Code: Select all
sh-3.2$ sudo zfs send -vR -i zhandy@2012-09-05-195520 zhandy@2012-09-14-182143 | zfs receive -v -F tall/backups/zhandy
attempting destroy tall/backups/zhandy@2012-08-31-204330
sending from @2012-09-05-195520 to zhandy@2012-09-14-182143
success
attempting destroy tall/backups/zhandy@2012-08-29-165742
success
attempting destroy tall/backups/zhandy@2012-08-30-221432
success
attempting destroy tall/backups/zhandy@2012-08-19-110836
success
attempting destroy tall/backups/zhandy@2012-08-18-102525
success
attempting destroy tall/backups/zhandy@2012-08-22-185347
success
receiving incremental stream of zhandy@2012-09-14-182143 into tall/backups/zhandy@2012-09-14-182143
WARNING: could not send zhandy/Pocket Time Machine@2012-09-14-182143: does not exist
received 31.2GiB stream in 2486 seconds (12.9MiB/sec)


Code: Select all
sh-3.2$ sudo zfs send -v -i zhandy@2012-09-14-182143 zhandy@2012-09-18-210251 | zfs receive -v -F tall/backups/zhandy
sending from @2012-09-14-182143 to zhandy@2012-09-18-210251
receiving incremental stream of zhandy@2012-09-18-210251 into tall/backups/zhandy@2012-09-18-210251
received 1.62GiB stream in 167 seconds (9.94MiB/sec)


Code: Select all
sh-3.2$ sudo zfs send -v "zhandy/Pocket Time Machine@2012-09-18-210251" | zfs receive -v -F "tall/backups/zhandy/Pocket Time Machine"
sending from @ to zhandy/Pocket Time Machine@2012-09-18-210251
receiving full stream of zhandy/Pocket Time Machine@2012-09-18-210251 into tall/backups/zhandy/Pocket Time Machine@2012-09-18-210251
received 56.4GiB stream in 2715 seconds (21.3MiB/sec)


Now:

Code: Select all
macbookpro08-centrim:~ gjp22$ date
Sat  3 Nov 2012 05:50:34 GMT
macbookpro08-centrim:~ gjp22$ zfs list | grep backup
tall/backups                              909Gi  5.79Gi  10.9Gi  /Volumes/tall/backups
tall/backups/LaCie d2 Extreme            2.63Mi  5.79Gi   612Ki  /Volumes/tall/backups/LaCie d2 Extreme
tall/backups/LaCie d2 Extreme/11G        1.03Mi  5.79Gi  1.03Mi  /Volumes/tall/backups/LaCie d2 Extreme/11G
tall/backups/LaCie d2 Extreme/12A        1.00Mi  5.79Gi  1.00Mi  /Volumes/tall/backups/LaCie d2 Extreme/12A
tall/backups/blocky                      3.31Gi  5.79Gi  3.31Gi  /Volumes/tall/backups/blocky
tall/backups/gjp22                        413Gi  5.79Gi   330Gi  /Volumes/tall/backups/gjp22
tall/backups/zhandy                       482Gi  5.79Gi   397Gi  /Volumes/tall/backups/zhandy
tall/backups/zhandy/Pocket Time Machine  52.8Gi  5.79Gi  52.8Gi  /Volumes/tall/backups/zhandy/Pocket Time Machine
macbookpro08-centrim:~ gjp22$


Code: Select all
macbookpro08-centrim:~ gjp22$ zfs list -t snapshot | grep backups/zhandy/Pocket
tall/backups/zhandy/Pocket Time Machine@2012-09-18-210251  6.30Mi       -  52.5Gi  -
tall/backups/zhandy/Pocket Time Machine@2012-09-23-0330         0       -  52.8Gi  -
macbookpro08-centrim:~ gjp22$


Code: Select all
macbookpro08-centrim:~ gjp22$ zfs list -t snapshot | grep backups/zhandy
tall/backups/zhandy@2012-09-21-065142                      82.2Mi       -   398Gi  -
tall/backups/zhandy@2012-09-22-195223                      33.8Mi       -   398Gi  -
tall/backups/zhandy@2012-09-27-065807                      37.6Mi       -   398Gi  -
tall/backups/zhandy@2012-09-28-092100                      4.50Mi       -   398Gi  -
tall/backups/zhandy@2012-09-30-190307                       288Ki       -   398Gi  -
tall/backups/zhandy@2012-10-01-210250                       328Ki       -   398Gi  -
tall/backups/zhandy@2012-10-05-033536                       109Mi       -   398Gi  -
tall/backups/zhandy@2012-10-08-192418                      6.20Mi       -   398Gi  -
tall/backups/zhandy@2012-10-10-090408                       648Ki       -   399Gi  -
tall/backups/zhandy@2012-10-12-082009                       436Ki       -   399Gi  -
tall/backups/zhandy@2012-10-13-210616                       268Ki       -   399Gi  -
tall/backups/zhandy@2012-10-15-075739                       272Ki       -   399Gi  -
tall/backups/zhandy@2012-10-16-220446                       272Ki       -   395Gi  -
tall/backups/zhandy@2012-10-18-200546                       272Ki       -   395Gi  -
tall/backups/zhandy@2012-10-19-191515                       392Ki       -   395Gi  -
tall/backups/zhandy@2012-10-20-220252                       780Ki       -   395Gi  -
tall/backups/zhandy@2012-10-21-203838                      11.9Mi       -   395Gi  -
tall/backups/zhandy@2012-10-22-215619                       555Mi       -   395Gi  -
tall/backups/zhandy@2012-10-23-200739                      1.27Gi       -   395Gi  -
tall/backups/zhandy@2012-10-25-053242                       492Ki       -   397Gi  -
tall/backups/zhandy@2012-10-25-201900                       348Ki       -   397Gi  -
tall/backups/zhandy@2012-10-26-072610                       280Ki       -   397Gi  -
tall/backups/zhandy@2012-10-27-140913                       280Ki       -   397Gi  -
tall/backups/zhandy@2012-10-28-211212                       296Ki       -   397Gi  -
tall/backups/zhandy@2012-10-30-212704                           0       -   397Gi  -
tall/backups/zhandy/Pocket Time Machine@2012-09-18-210251  6.30Mi       -  52.5Gi  -
tall/backups/zhandy/Pocket Time Machine@2012-09-23-0330         0       -  52.8Gi  -
macbookpro08-centrim:~ gjp22$
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom


Return to General Discussion

Who is online

Users browsing this forum: ilovezfs and 2 guests

cron