Here’s a list of some command line tools in Mac OS X that range from the highly useful, through the quite interesting all the way to the slightly bizarre. A lot of them definitely aren’t specific to OS X, but I don’t have a Linux box to find out which ones those are. So, without further ado, and in no particular order:

• mdls (metadata list)

This tool uses the Spotlight search APIs to query the metadata server (mds) for all of the metadata associated with a particular file. It takes a file or directory name as an argument and returns output as a series of key-value pairs, as follows:

kMDItemAlbum = “By The Way”
kMDItemAttributeChangeDate = 2006-11-24 11:58:17 +0000
kMDItemAudioBitRate = 64
kMDItemAudioChannelCount = 2
kMDItemAudioEncodingApplication = “iTunes v3.0”
kMDItemAudioSampleRate = 44100
kMDItemAuthors = (“Red Hot Chili Peppers”)
kMDItemContentCreationDate = 2003-04-29 07:37:16 +0100
kMDItemContentModificationDate = 2003-04-29 07:37:16 +0100
kMDItemContentType = “public.mp3”
kMDItemContentTypeTree = (
“public.mp3”,
“public.audio”,
“public.audiovisual-content”,
“public.data”,
“public.item”,
“public.content”
)



The key for each value is a predefined metadata attribute. Spotlight makes provision for both generic attributes such as kMDItemFSCreationDate, but also for format-specific metadata as seen above with, for example, kMDItemAudioSampleRate. As a point of interest, the values that correspond to thekMDItemContentType and kMDItemContentTypeTree metadata attributes are Apple’s ingenious Uniform Type Identifiers, which go a long way toward potentially solving the manifold limitations of MIME types and file name extensions.

mdls only has one CLI flag, -name, which allows a specific metadata attribute to be retrieved. For example, $mdls -name kMDItemTotalBitRate *.mp3 would show the bit rate of every file with the .mp3 extension in the current directory.

• mdfind (metadata find)

The mdfind is the CLI equivalent of the Spotlight UI at the right of the menu bar. At its most basic level, a command such as $mdfind “Red Hot Chili Peppers” would return a list of all files on currently connected Spotlight-indexed volumes that contain the string “Red Hot Chili Peppers” anywhere in their metadata attributes or the Spotlight content index. Piping the output into wc -l neatly returns the number of hits.

At a slightly more complex level, the string provided can be a raw Spotlight query, for example $mdfind “(kMDItemDurationSeconds > 600) && (kMDItemContentTypeTree == ‘public.audio’)”, which would return all audio files longer than 10 minutes on Spotlight-indexed volumes. mdfind has two options: the -live and -onlyin flags. -live doesn’t require any arguments and causes mdfind to run an initial search, returning the file paths of all matching items, but then continue running giving a tally of the total number of matches as files are created or modified (this leverages the functionality provided by the kernel through /dev/fsevents). -onlyin does exactly what it says on the tin, allowing the search to be restricted to a certain directory.

Admittedly, the Spotlight attribute names are somewhat cryptic, but there’s a great guide to both the mdfind and mdls commands here, including the various logical operators that can be used to construct queries and even a Perl script that takes advantage of the mdfind command.

• file

The file command is fairly powerful, although it is perhaps of limited utility (on Mac OS X) considering the availability of the aforementioned tools. It has many flags, but basically takes one argument, which is a file path. Once given this file path, it outputs information about the file type, gleaned from either filesystem tests, “magic number” tests or language tests (all of which are briefly explained in the man page). Here’s an excerpt of the output from running $file /* on my machine:

/automount: directory
/bin: directory
/cores: sticky directory
/dev: directory
/etc: symbolic link to `private/etc’
/mach: symbolic link to `/mach.sym’
/mach.sym: Mach-O executable i386
/mach_kernel: Mach-O universal binary with 2 architectures
/mach_kernel (for architecture ppc): Mach-O executable ppc
/mach_kernel (for architecture i386): Mach-O executable i386
/private: directory
/sbin: directory
/tmp: symbolic link to `private/tmp’
/usr: directory
/var: symbolic link to `private/var’

Nice.

• lsof (list open files)

The manual page for lsof is vast, but basically it is, as the name implies, a tool that reports which files (including directories, device files and sockets) are currently open on the system. The extensive man page is due to the large number of options that can limit the scope of lsof to specific filesystem locations, process names, PIDs or UIDs. lsof, however, is probably most useful when piped into grep (or, if you’re a statistics junkie, wc -l).

For example, a quick lsof | grep “/Volumes/Backup” reveals the culprits when you try to eject your backup disk only to find this error message presented to you:


• fs_usage (file system usage)

Due to the kernel tracing facility that is used by fs_usage, it needs to be run as root under sudo. It’s primarily a debugging tool, reporting filesystem-related system calls and page faults in real time, but can also be useful in everyday situations. For example, it can be useful if a disk is “churning” and you want to know which process is causing all the activity; a simple $sudo fs_usage should reveal the offending process in that instance.

Alternatively, $sudo fs_usage -f filesys PID will show all filesystem activity for a particular process or $sudo fs_usage -f filesys -e PID will show all filesystem activity except that for the specified PID (or PIDs). Might be useful to someone!

• diskutil

Apple’s home grown CLI disk utility, named diskutil, utilises OS X’s Disk Management framework and therefore offers basically all of the functionality of the graphical Disk Utility found in /Applications/Utilities/. It’d be pointless to go through all the features of diskutil here when they’re so accurately and concisely described in the man page, but let it be said that they are very extensive, including RAID management, partitioning, secure erasing and much, much more.

• iostat (Input/Output statistics)

iostat is common to all BSD systems and (I think) to Linux and Solaris boxes as well. In the words of the man page, it “displays kernel I/O statistics on terminal, device and cpu operations”. That pretty much sums it up, but a fairly good way of invoking it is $iostat -w X which outputs a line to stdout every X seconds or, for logging purposes, $iostat -w X >> iostat.out &. The output, since you’re interested, looks a bit like this:

         disk1           disk0       cpu
 KB/t tps  MB/s   KB/t tps  MB/s  us sy id
169.90   1  0.10  20.88   3  0.06   7  3 90

• ioreg (Input/Output registry)

Amit Singh’s book (ISBN: 0321278542) has a very informative paragraph on Mac OS X’s I/O registry (pages 1251-52). Basically, ioreg provides a way to traverse the various planes of the I/O registry (including the Power, Service, USB and FireWire planes) from the command line. It’s the CLI equivalent of IORegistry-Explorer.app, which is part of the developer tools.

Much information can be gleaned from the IO registry, including visualising the downstream effects of shutting off power to a particular device (I haven’t done this, but Singh’s book asserts that it is possible). Personally, I think the most interesting statistics from ioreg are those regarding the batteries in laptops, some of which can be obtained by typing $ioreg -l | grep Capacity (obviously this will fail on a desktop machine)

• pbcopy and pbpaste (Pasteboard copy and paste)

These are two brilliant little tools, especially useful for making ever-so-slightly more user-friendly shell scripts. The functionality is really in the name; whatever you pipe into pbcopy ends up on the system “pasteboard” and pbpaste will happily spit it back out again to stdout. So, for example, if you wanted to remove all the white space from some text on the clipboard (particularly useful if, like me, you work with a lot of biological sequence data) the following command would do the job: $pbpaste | tr -d ‘[:space:]’ | pbcopy*. Both pbcopy and pbpaste may take one option, -pboard, which may take general, ruler, find or font as its argument, depending on which system pasteboard is to be used. The “general” pasteboard is the default. I think that Apple’s inclusion of these tools in OS X is really great and makes for a nice bridge between the GUI and the CLI worlds.

* Bonus tip: Because I use this feature so frequently, I set up an alias in Bash, as follows: $alias rmpbws=”pbpaste | tr -d ‘[:space:]’ | pbcopy; echo ‘Pasteboard white space removed.'”, so whenever I type rmpbws (remove pasteboard white space) at the Bash prompt, all white space is removed from the text currently on the pasteboard. Love it.

• open

In a similar vein to the pbcopy and paste tools above, the open command provides a very pleasant bridge between CLI and GUI. According the man page, the command originally appeared in NeXTStep and I’m really glad it was carried over to OS X. The open command does exactly what you would expect (i.e. it opens the specified file in the default application as determined by LaunchServices), but it also provides some slightly unexpected features, as follows:

• open http://en.wikipedia.org/wiki/Open_%28application%29 opens the Wikipedia page describing this command in the default web browser
• open . opens the current working directory in the Finder
• pbpaste | open -f opens the current contents of the system pasteboard in a new document in the current LaunchServices default text editor
• ps auxc | awk {‘print $1 “\t” $2 “\t” $11’} | open -f opens a list containing the UID, PID and process name for all currently running processes in the LaunchServices default text editor (just as a slightly more fancy example)

So there we have a round-up of some CLI tricks and tools that may or may not make your life easier in Mac OS X. I’ll try to follow this up with some more great CLI tools this weekend…