Five useful command one liners

24 December 2008

I had a browse through my shell history (history | less), and there are some interesting commands that I have used recently. The really experienced command line warriors among you will probably know them already, but it never hurts to have a reminder.

1. How to migrate a MySQL database

If you want to export data from an MySQL server on one server, and then import the data on a new server, this is pretty easy when you know how, just use the redirect symbols, greater than ('>') and less than ('<').

On the first server use the following command:

mysqldump --user root -p database-name > filename.dump

This will give you a new file called filename.dump, transfer that file to the second server using SSH or however else you want.

On the second Server, you can import the data with the following command:

mysql -u root -p -D database-name < filename.dump

2. How to grep a large number of files.

The following command searches for files that contain the word 'term':

grep term *

If you grep over a very large number of files, grep will fail with "the parameter list is too long".

The way around this is to use xargs and pipe the filenames to grep.

find . -print0 | xargs -0 grep -H term

This approach uses null characters instead of newlines to separate the results. This means things won't go screwy if there are files with white space or newlines in the file names, e.g. files that have come from Windows or are on a Windows partition.

3 How to force rpm install

I don't like the RPM tool, there are more modern, i.e. less brain-dead, package managers available. Anyway if you have a machine with an RPM based distribution that is completely messed up, you may want to reinstall packages that have gone wrong. RPM does not let you reinstall the same version over itself, however you can force it, here is how:

sudo rpm -iv --replacepkgs --replacefiles packagefile.rpm

4 Mounting Mac Disk images

Mounting regular disk images in the ISO format is easy:

sudo mount -t iso9660 -o loop filename.iso /media/iso

What about Macintosh disk images (commonly known as DMG format)?

Well we have a similar plan, we want to go:

sudo mount -t hfs -o loop filename.dmg /media/iso

Or for newer files:

sudo mount -t hfsplus -o loop filename.dmg /media/iso

There are a couple of problems that might arise:

  • You might not have the relevant kernel modules for reading hfs or hfsplus.
  • It might not be a true .dmg file. Sometimes the files are archived in a zip or tar, even though the files may end .dmg.

The second one can be sorted out pretty quickly using the trusty file command. For example:

file GnuPG1.4.8.dmg

Results in:

GnuPG1.4.8.dmg: Macintosh HFS Extended version 4 data (mounted) (unclean) last mounted by: 'H+Lx', created: Tue Dec 30 17:24:08 2003, last modified: Wed Dec 24 03:40:12 2008, last checked: Tue Dec 30 18:24:08 2003, block size: 4096, number of blocks: 4096, free blocks: 1377

This is a true dmg file in hfsplus format.

  1. Don't forget to watch

If you want to repeat a command many times, for example you are monitoring something, then don't forget about the watch command. It will print the results of the command to screen every 2 seconds (you can change the interval with -n).

I tend to use watch with disposable one-off Python scripts for which writing a proper interface is overkill. However you can also use watch with sys-admin commands. Lets do some generic and clichéd examples.

Watching the disk space:

watch --no-title "df -h"

Watching who is logged in:

watch --no-title who

Watching the current IP Address on eth0:

watch --no-title "ifconfig eth0 | grep 'inet addr:' | cut -d: -f2 | cut -d ' '  -f1"

You will know what commands you use and how watch can be helpful.

Conclusion

So that was some of the commands from my shell history. What commands have you been using recently?

1 Sam says...

I'd recommend checking out a program called 'ack' in lieu of using find + xargs + grep. Also, check out htop and dstat are fun.

Posted at 1:23 p.m. on December 24, 2008


2 eliasp says...

Trick '1' can be done easier, if one of both MySQL daemons is reachable via network...

mysqldump -uroot -p database-name -h hostname-A | mysql -uroot -p database-name -h hostname-B

Posted at 1:51 p.m. on December 24, 2008


3 tob says...

"man watch"

;-) thx didn't know this one

Posted at 2:05 p.m. on December 24, 2008


4 Troy Curtis says...

Much like the unneccesary use of 'cat' that you see in many posted scripts (i.e. cat /some/file | grep "mypattern" instead of just grep "mypattern" /some/file), tip 2 is an unnecessary use of find! :)

To duplicate that functionality with just grep you can use:

grep -R term

Not only is it less things to type and less syntax to get wrong, but it may be quicker because with the xargs version will cause multiple grep invocations (depending on just how many files there are).

Of course with the find command you could have a lot more flexibility in trimming out items you don't want (though grep does have '--include/--exclude' options), controlling the depth of recursion and those kind of things.

The great thing about CLI interfaces? There are so many ways to accomplish a given task!

Posted at 2:06 p.m. on December 24, 2008


5 tulcod says...

as for the grep problem, it's probably easier to do: grep needle -R .

Posted at 2:20 p.m. on December 24, 2008


6 Binny V A says...

I have actually setup a site to store just short commands... http://txt.binnyva.com/

Posted at 6:32 p.m. on December 25, 2008


7 CorkyAgain says...

Is the watch command you're describing a Linuxism?

On my FreeBSD box, "man watch" seems to be describing something completely different.

Posted at 6:46 p.m. on December 25, 2008


8 Zeth says...

CorkyAgain, good question, I don't have a FreeBSD box available at the moment so I can't comment. On Linux at least watch does as I have described.

Posted at 4:53 p.m. on December 26, 2008


9 Samuel Huckins says...

Great tips! I have had occasion to do a lot of MySQL instance migrations lately, so here is an improvement for Trick 1:

mysqldump <DATABASE_NAME> [mysqldump_options] | gzip -c | ssh user@remotehost "gunzip -c > mysql <DATABASE_NAME> [mysql_options]"

This creates the dump, compresses it, securely transfers to the remote host via SSH, decompresses the dump, and imports it into the new instance. Fastest possible method, no files to cleanup!

Posted at 5:04 a.m. on January 4, 2009


10 Stéphane Bortzmeyer says...

@CorkyAgain: On FreeBSD (I tested on a 7.0), you can have this watch by installing the port gnu-watch.

% sudo pkg_add -r gnu-watch

% gnu-watch --no-title "df -h"

Posted at 10:15 p.m. on January 10, 2009


11 Jadu Saikia says...

Nice one. A post on watch http://unstableme.blogspot.com/2008/03/execute-program-periodically-bash.html

Posted at 12:54 p.m. on August 13, 2009


What do you have to say?

Show Editing Help

About

Hello, my name is Zeth, I'll be your host here.

Command Line Warriors is about taking control of your own technology, it looks at our experiences of computing; especially using GNU/Linux, the Python programming language, the command-line and issues such as techno-ethics, best practices and whatever is cool now. If you take control of your technology then you are a Warrior too!

This site is your site too which means that you can contribute and get involved. You can leave comments using the facility provided. For me, the comments and discussions are by far the best part of the site. So please do have your say!

Latest Discussions

Zeth

November 29, 2009
Hi Jordan, yes that URL is gone now. I have a new contact form on this site.
Python CGI contact forms

Jordan

November 29, 2009
Zeth attention! Your form, http://zeth.me.uk/contact/, is not working The explorer says connecting ..but nothing happens Sorry for my poor English: I am Spanish Regards
Python CGI contact forms

Jordan

November 26, 2009
Sorry: tell me , not tellme (I'm spaniard) And http://zeth.me.uk/contact/ don't work
You got the touch, you got the power

David Jones

November 25, 2009
Your mad skillz are too l33t! for me. I specifically switched to Google Reader so that I could show people what blogs I read. But I couldn't work out how ...
How to find the fashionable blogs quickly

Brian R. Hickey

November 20, 2009
Symantec picked it up too.
How to bring down Internet Explorer with six words

Zeth

November 17, 2009
Thanks djm, I am the moose here. Christian, assuming one actually does Internationalise the countries, it should still work I guess, as the gettext stuff will happen before the list ...
Countries in Django

Phillip Temple

November 17, 2009
Good start, but: a) wouldn't I want None back rather than 'ZZ'? b) why not add a 'shortcut' boolean, then prepend flagged fields (plus usual '-----' separator) to the actual ...
Countries in Django

djm

November 17, 2009
Am I being a moose or did you mean: from whatever.countries import CountryField instead of from whatever.countries import CharField ? Good post though, cheers.
Countries in Django

Christian Joergensen

November 17, 2009
Wouldn't the ordering get messed up after i18n?
Countries in Django

Steve - Electronic Cigarettes Fan

November 17, 2009
Very well done. Is your blog just you writing? Nicely done, Steven.
Blogger vs Wordpress

vetetix

November 15, 2009
Sorry to bother you nearly two years after you wrote this blog article, but I can't manage to find how to modify an existing field. I am trying to change ...
Three Useful Python Bindings - ClamAV, Apt and Evolution

Manju

November 4, 2009
I am transferring some files using psftp to other device's FAT partition. But the filestamp of the file being transferred is modified to that of FAT device, after the transfer. ...
PuTTY Series: Using PSFTP

iki

November 2, 2009
or simpler: socket.gethostbyname_ex(socket.gethostname())[2]
How to find out your IP address in Python

iki

November 2, 2009
local_ip = set([ i[4][0] for i in socket.getaddrinfo(socket.gethostname(), None) if i[0] == 2 ])
How to find out your IP address in Python

Fred

November 2, 2009
testing rst ------------- - point 1
An Introduction to ReStructuredText

Ano

October 27, 2009
"You simply found the license of the StumbleUpon Toolbar for Internet Explorer." That's possible. I've got some more interesting information to add. Firstly, go to this page: https://addons.mozilla.org/en-US/firefox/addon/138 - this ...
Are your Firefox extensions proprietary software?

Ken

October 21, 2009
Stumbled in here at lunch. This is the best find of the week. Thanks.
Three classic command line tips

Jim

October 19, 2009
Thanks for the rtsp:// post - that's something that has been bugging me for a while!
Three classic command line tips

Zeth

October 18, 2009
Thanks for the comments guys. Great to see the all the gang are still here!
Three classic command line tips

Bubba

October 18, 2009
Is there any way psftp can return the true transfer rates oberved during the actual transfer?
PuTTY Series: Using PSFTP