Linode Forum
Linode Community Forums 		 FAQFAQ    SearchSearch    MembersMembers      Register Register 
 LoginLogin [ Anonymous ] 
Post new topic  Reply to topic

Board index » Linux Community Forums » General Discussion
  Previous topic | Next topic 
Author Message
torstenb
PostPosted: Mon Jul 09, 2012 1:11 pm 
Offline
Newbie

Joined: Mon Jul 09, 2012 12:34 pm
Posts: 2
Hi,

the tutorial for copying a disk image over ssh can take a very long time depending on the size of the linode.

The command shown in the tutorial does not do any compressing:
Code:
ssh root@123.45.67.89 "dd if=/dev/xvda " | dd of=/home/archive/linode.img


Is it possible to compress the stream of data that is transferred from the linode to the receiving machine before it is being sent and then decompress it locally before it's passed to the dd?
What would the revised command need to be?

Thanks!
Torsten
Top
   
Reply with quote  
obs
PostPosted: Mon Jul 09, 2012 1:55 pm 
Offline
Senior Member

Joined: Sun Mar 07, 2010 7:47 pm
Posts: 1970
Website: http://www.rwky.net
Location: Earth
Code:
ssh -C root@123.45.67.89 "dd if=/dev/xvda " | dd of=/home/archive/linode.img


The -C adds ssh compression.

_________________
Paid support
How to ask for help
1. Give details of your problem
2. Post any errors
3. Post relevant logs.
4. Don't hide details i.e. your domain, it just makes things harder
5. Be polite or you'll be eaten by a grue
Top
   
Reply with quote  
Guspaz
PostPosted: Mon Jul 09, 2012 3:14 pm 
Offline
Senior Member
User avatar

Joined: Tue May 26, 2009 3:29 pm
Posts: 1691
Location: Montreal, QC
What's the block size used for SSH compression? It's possible that it might be terribly inefficient compared to a larger blocksize with better compression. For example, this article compares the effect of blocksize for dd via bzip2 over netcat, although unfortunately it doesn't compare SSH with compression on (only off):

http://www.ndchost.com/wiki/server-admi ... t-over-ssh
Top
   
Reply with quote  
obs
PostPosted: Mon Jul 09, 2012 7:11 pm 
Offline
Senior Member

Joined: Sun Mar 07, 2010 7:47 pm
Posts: 1970
Website: http://www.rwky.net
Location: Earth
man ssh says
Quote:
Requests compression of all data (including stdin, stdout,
stderr, and data for forwarded X11 and TCP connections). The
compression algorithm is the same used by gzip(1), and the
``level'' can be controlled by the CompressionLevel option for
protocol version 1. Compression is desirable on modem lines and
other slow connections, but will only slow down things on fast
networks. The default value can be set on a host-by-host basis
in the configuration files; see the Compression option.

So you can change the compression performance. Of course it depends what you're transferring if you're transferring a large MySQL database for example the -C flag on it's own vastly improves performance. I have my ssh_config file turn compression on my default.

_________________
Paid support
How to ask for help
1. Give details of your problem
2. Post any errors
3. Post relevant logs.
4. Don't hide details i.e. your domain, it just makes things harder
5. Be polite or you'll be eaten by a grue
Top
   
Reply with quote  
Guspaz
PostPosted: Tue Jul 10, 2012 10:07 am 
Offline
Senior Member
User avatar

Joined: Tue May 26, 2009 3:29 pm
Posts: 1691
Location: Montreal, QC
Sure, but gzip isn't as efficient as bzip, and if SSH is using 1500 byte blocks, while the alternative is using, say, 4MB...
Top
   
Reply with quote  
obs
PostPosted: Tue Jul 10, 2012 2:01 pm 
Offline
Senior Member

Joined: Sun Mar 07, 2010 7:47 pm
Posts: 1970
Website: http://www.rwky.net
Location: Earth
True bzip isn't as efficient as gzip but you could also say bzip isn't as efficient as 7zip in general adding the -C flag isn't a bad idea, a better command would be
Code:
ssh root@123.45.67.89 "dd if=/dev/xvda | bzip2 " | bunzip2 | dd of=/home/archive/linode.img

_________________
Paid support
How to ask for help
1. Give details of your problem
2. Post any errors
3. Post relevant logs.
4. Don't hide details i.e. your domain, it just makes things harder
5. Be polite or you'll be eaten by a grue
Top
   
Reply with quote  
Display posts from previous:  Sort by  
Post new topic  Reply to topic

Board index » Linux Community Forums » General Discussion


Who is online

Users browsing this forum: mattltm and 3 guests

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  
RSS

Powered by phpBB® Forum Software © phpBB Group