Using a shell to generate HTML files for viewing or selecting images

The commands below use these functions:

html(){ local f=/tmp/$(gdate +%s%3N).html;cat>$f;open $f; }
im(){ recode ..xml|sed s,^/,file:///,|sed 's,.*,<a href=&><img src=&></a>,'|tr -d \\n|cat <(echo "<style>body{text-align:center}img{height:${1-calc(50% - 20px)}}</style>") -|html; }
st(){ osascript -e'set text item delimiters to linefeed' -e'tell app"safari"to url of tabs of window 1 as text'|sed s,^file://,,; }
scm(){ osascript -e'on run{a}' -e'tell app"safari"to close tabs of window 1 whose url contains a' -eend "$1"; }
mt(){ gxargs -rd\\n gmv -t"$1"; }
p(){ printf %s\\n "$@"; }

The two osascript commands above also work with Chrome if you just change the application name.

View images so that two rows of images are displayed per screen:

p ~/dir/*.jpg|im

View images so that a single row of images is displayed per screen:

p ~/dir/*.jpg|im 'calc(100% - 40px)'

The page down key scrolls 100% - 40px down in Safari and Chrome and 100% - 32px down in Firefox. In the versions of Safari, Chrome, and Firefox I tested with, calc(50%-20px) (without spaces around the minus sign) does not work but calc(50% - 20px) does.

Move images opened in tabs to the desktop:

st|mt ~/Desktop

Move images opened in tabs to the desktop and close the tabs for images assuming that the URLs of the tabs for the images contain .jpg but the URLs of other tabs do not:

st|grep \\.jpg|mt ~/Desktop;scm .jpg

Create a file for the filenames of images that you have checked and view the first thousand images that you have not yet checked:

touch checked;printf %s\\n ~/dir/*.jpg|awk -F/ 'FILENAME==ARGV[1]{a[$0];next}!($NF in a)' checked -|head -1000|tee >(sed 's,.*/,,'>>checked)|im

To escape spaces or other characters in local paths, add jq -Rr @uri|sed s,%2F,/,g before the im function.

You can run defaults write -g NSScrollAnimationEnabled -bool false to disable the animation for scrolling a page down.

To sort the images in a directory in the order that they were added to the directory, you can use this function:

sbda(){ if (($#>0));then printf %s\\n "$@";else cat;fi|gxargs -rd\\n mdls -rnkMDItem{DateAdded,Path}|tr \\0 \\n|paste - -|sort -rk1,1|cut -f2; }

To select images using only the mouse, you can use a private.xml like this with Karabiner to map a mouse button to the page down key:

<?xml version="1.0"?>
    <autogen>__KeyToKey__ PointingButton::BUTTON4, KeyCode::PAGEDOWN</autogen>

Installing GNU command line utilities with Homebrew

This command installs coreutils and other GNU utilities:

brew install coreutils findutils gawk gnu-sed gnu-tar wget homebrew/dupes/grep

To use the utilities from coreutils without a g prefix, add /usr/local/opt/coreutils/libexec/gnubin to the path before the default directories. You can create symlinks for the utilities that are not in coreutils:

ln -s /usr/local/opt/findutils/bin/gfind /usr/local/bin/find
ln -s /usr/local/opt/findutils/bin/gxargs /usr/local/bin/xargs
ln -s /usr/local/opt/gawk/bin/gawk /usr/local/bin/awk
ln -s /usr/local/opt/gnu-sed/bin/gsed /usr/local/bin/sed
ln -s /usr/local/opt/grep/bin/ggrep /usr/local/bin/grep
ln -s /usr/local/opt/grep/bin/gegrep /usr/local/bin/egrep
ln -s /usr/local/opt/grep/bin/gfgrep /usr/local/bin/fgrep

Setting default applications for executable scripts and text files without an extension

One option is to use duti:

brew install duti
echo $'org.gnu.Emacs public.unix-executable all\norg.gnu.Emacs public.plain-text all'>~/.duti
duti ~/.duti

Another option is to modify ~/Library/Preferences/com.apple.LaunchServices.plist directly like below, but then you have to restart to apply the changes. Logging out and back in is not enough.

defaults write com.apple.LaunchServices LSHandlers -array-add '{LSHandlerContentType=public.unix-executable;LSHandlerRoleAll=org.gnu.Emacs;}'
defaults write com.apple.LaunchServices LSHandlers -array-add '{LSHandlerContentType=public.plain-text;LSHandlerRoleAll=org.gnu.Emacs;}'

Shell scripts for iTunes

Add a playlist:

osascript -e'on run {a}
    set l to {}
    repeat with f in (get paragraphs of a)
        set end of l to POSIX file f
    end repeat
    tell application "iTunes"
        if number of l is 1 then
            tell application "Finder" to set n to name of (item 1 of l as alias)
            delete playlists whose name is n
            set p to make new user playlist with properties {name:n}
            set p to make new user playlist
        end if
        try -- the add command sometimes times out
            add l to p
            set view of browser window 1 to p
        end try
    end tell
end run' "$(printf %s\\n "${@-$(cat)}"|gxargs -rd\\n greadlink -f)"

printf %s\\n "${@-$(cat)}" gets input as arguments if there are arguments but from STDIN otherwise.

Get the folders or files selected in Finder as input, convert FLAC files to MP3, split files with cue files, and delete playlist files:

fs(){ osascript -e{'set l to{}','tell app"finder"to repeat with f in(get selection)','set end of l to posix path of(f as alias)',end,'set text item delimiters to linefeed','l as text'}; }

alias p='printf %s\\n' ,=parallel e='gxargs -rd\\n'


p "$x"|, -X find {} -iname \*.flac|, ffmpeg -i {} -aq 2 {.}.mp3
p "$x"|, -X find {} -iname \*.cue|, mp3splt -c {} {.}.mp3
p "$x"|, -X find {}|egrep '\.(pls|m3u|m3u8)$'|e rm

The -X option in parallel fills a command line with multiple arguments like xargs. mp3splt -c file.{cue,mp3} splits an MP3 file with a cue file.

Enabling Apache and setting up virtual hosts

Run sudo -v to validate the sudo timestamp and then run the commands below.

echo '<Directory "/Users/'$USER'/Sites/">
  Options Indexes Multiviews
  AllowOverride AuthConfig Limit
  Order allow,deny
  Allow from all
</Directory>'|sudo tee -a /etc/apache2/users/$USER.conf
echo test.dev|sudo tee -a /etc/hosts
sudo sed -i '' 's,#\(Include /private/etc/apache2/extra/httpd-vhosts.conf\),\1,' /etc/apache2/httpd.conf
echo '<VirtualHost *:80>
  DocumentRoot "/Users/'$USER'/Sites/test"
  ServerName test.dev
<VirtualHost *:80>
  DocumentRoot "/Users/'$USER'/Sites"
  ServerName localhost
</VirtualHost>'|sudo tee -a /etc/apache2/extra/httpd-vhosts.conf
sudo apachectl restart
mkdir -p ~/Sites/test
echo a>~/Sites/test/index.html
sleep 1
open http://localhost/~$USER/test/index.html http://test.dev/index.html

/usr/sbin/apachectl is a shell script in OS X, and sudo apachectl start runs launchctl load -w /System/Library/LaunchDaemons/org.apache.httpd.plist as the superuser, which enables Apache permanently by modifying /var/db/launchd.db/com.apple.launchd/overrides.plist.

http://localhost/~$USER/ does not work if you do not add the Directory directive to /etc/apache2/users/$USER.conf.

Using a shell to manage text files for URLs to read

The commands below use these functions:

st(){ osascript -e'set text item delimiters to linefeed' -e'tell app"safari"to url of tabs of window 1 as text'; }
s1(){ osascript -e'tell app"safari"to url of document 1'; }
dl(){ local f;for f in "${@:2}";do ghead -n-$1 "$f"|sponge "$f";done; }
ua(){ touch "$1";awk '!a[$0]++' "$1" -|sponge "$1"; }
/(){ awk 'FILENAME==ARGV[1]{a[$0];next}!($0 in a)' - "$1"|sponge "$1"; }

The first two functions also work with Chrome if you replace safari with google chrome and document 1 with active tab of window 1.

sponge is part of moreutils.

Add the URLs of all tabs in the frontmost window in Safari to a text file and remove duplicates:

st|ua toread

Add the URL of the current tab in Safari to a text file:

s1 >>toread

Open and delete the last ten lines in a text file:

tail toread|xargs open;dl 10 toread

Delete the URLs of tabs in the frontmost Safari window from a text file:

st|/ toread

Scripts for previewing and selecting tracks in iTunes

Start playing the next track from the middle:

tell application "iTunes"
    next track
    set player position to 0.5 * (duration of current track)
    reveal current track
end tell

Set the rating of the current track to one star and start playing the next track from the middle:

tell application "iTunes"
    set rating of current track to 20
    next track
    set player position to 0.5 * (duration of current track)
    reveal current track
end tell

Move 30 seconds backward:

tell application "iTunes"
    if player position < 30 then
        previous track
        set player position to player position - 30
    end if
end tell

Move 30 seconds forward:

tell application "iTunes"
    set player position to player position + 30
end tell

Move the file of the current track to the trash and start playing the next track:

tell application "iTunes"
    set t to current track
    set l to location of t
    next track
    reveal current track
    delete t
    tell application "Finder" to move l to trash
end tell


sh can refer to:

As of 2015, /bin/sh is bash in OS X, Arch, Gentoo, Red Hat, and SUSE, dash in Debian, ash in FreeBSD, pdksh in OpenBSD, and ksh93 in Solaris. It was zsh in OS X in 10.0 and 10.1.

The bash manual includes a list of differences between bash and the Bourne shell and a list of changes that are caused by POSIX mode being enabled.

When bash is invoked as sh:

The last three are not implied by only enabling POSIX mode. Invoking bash as sh is meant to make bash fully conform to POSIX, but only invoking bash in POSIX mode intentionally does not conform to POSIX in the last two aspects.

The bash manual says that "If Bash is invoked with the name sh, it tries to mimic the startup behavior of historical versions of sh as closely as possible, while conforming to the POSIX standard as well." The mimicking of historical versions of sh is limited to the startup behavior though, and almost all of the changes when bash is invoked as sh are caused by POSIX mode being enabled. POSIX does not specify what startup files shells should read apart from requiring them to interpret the value of ENV as a startup file on XSI-conformant systems. (XSI-conformant systems include OS X, because OS X conforms to SUS which requires the XSI options to be supported.)

When bash is invoked as sh, the builtin echo command interprets escape sequences and does not support options unless xpg_echo is unset:

$ sh
sh-3.2$ echo -n '1\n2'
-n 1
sh-3.2$ shopt -u xpg_echo;echo -n '1\n2'

You can normally just use printf instead of unsetting xpg_echo though. shopt and xpg_echo are specific to bash. POSIX requires implementations of echo to not support any options, but it allows echo to either interpret escape sequences by default or not, and to either print or not print -n if the first argument is -n.

The /bin/sh file is not a symlink or a hard link in OS X, but it is almost identical with the /bin/bash file. If you compare the output of strings /bin/bash and strings /bin/sh, there are only three differences, one of which is the different default value for FCEDIT:

$ diff <(strings /bin/bash) <(strings /bin/sh)
> /bin/bash
< ${FCEDIT:-${EDITOR:-ed}}
> ${FCEDIT:-ed}
< @(#)PROGRAM:bash  PROJECT:bash-94.1.2
> @(#)PROGRAM:sh  PROJECT:bash-94.1.2

If you use #!/bin/sh as the interpreter line in a script (or #!/usr/bin/env sh, because POSIX does not require the sh executable to be in /bin), you should avoid features of shells that are not specified by POSIX. For example /bin/sh is dash in Debian, so many features that are supported by bash but not by dash work with the /bin/sh in OS X but not with the /bin/sh in Debian.

Collation order in UTF-8 locales

As far as I can tell, OS X uses code point collation order in UTF-8 locales, even though GNU/Linux and other platforms that use GNU libc do not.

This prints ABabÄä in OS X and aAäÄbB in Ubuntu:

printf %s\\n A B a b Ä ä|LC_ALL=en_US.UTF-8 sort|tr -d \\n

This prints 26 in OS X and 310 in Ubuntu:

export LC_ALL=en_US.UTF-8;printf %b $(printf \\\\U%08x\\\\n $(seq $((0x11)) $((0x10ffff))))|grep -a '[a-z]'|wc -l

The code below prints nothing in OS X, which indicates that the input is sorted. The six surrogate characters that are skipped would cause an illegal byte sequence error.

export LC_ALL=en_US.UTF-8
  x=$(printf %04x $i)
  [[ $x = @(000a|d800|db7f|db80|dbff|dc00|dfff) ]]&&continue
  printf %b \\U$x\\n
done|sort -c

The code below prints nothing in OS X, which indicates that if the few code points that are skipped are not considered, there are no two consecutive code points that have the same collation order.

export LC_ALL=en_US.UTF-8
  [[ $(printf %04x $i) = @(000a|d800|db7f|db80|dbff|dc00|dfff) ]]&&continue
  [[ $(printf %04x $((i+1))) = @(000a|d800|db7f|db80|dbff|dc00|dfff) ]]&&continue
  printf %b $(printf '\\U%08x\\n' $((i+1)) $i)|sort -c 2>/dev/null&&echo $i

Some characters and sequences of characters have the same collation order in GNU/Linux but not in OS X. This prints ① first in OS X (using either OS X's sort or GNU sort) but ② first in Ubuntu:

export LC_ALL=en_US.UTF-8;printf %s\\n ② ①|sort

This prints three lines in OS X (using either OS X's sort or GNU sort) but one line in Ubuntu:

export LC_ALL=en_US.UTF-8;printf %b\\n \\u0d4c \\u0d57 \\u0d46\\u0d57|sort -u

Setting a default PATH

Save a property list like this as ~/Library/LaunchAgents/my.startup.plist:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
    <string>launchctl setenv PATH /usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin</string>

The property list gets loaded the next time you log in, so you do not have to load it manually.

This method does not change the path in applications that are opened as login items or when applications are reopened at login after a forced shutdown. If you have a terminal application, text editors, or other applications where you want to change the path in login items, you'll have to remove them, and after a forced shutdown you will either have to choose the option to not restore open applications or to quit and reopen applications where you want to change the path after they get restored.

In 10.9 and earlier another option is to add setenv PATH /usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin to /etc/launchd.conf and to restart to apply the changes, but /etc/launchd.conf is ignored in 10.10 and later.

The lines in /etc/paths and /etc/paths.d/* are added to the path by /usr/libexec/path_helper, which is ran from /etc/profile, /etc/zsh.env, and /etc/csh.login. zsh reads /etc/zsh.env even in non-login shells but bash does not read /etc/profile in non-login shells. path_helper is also not ran by graphical applications like Xcode, Emacs.app, or Script Editor.

Finding programs with the highest CPU use over a long period of time

First run crontab -e and add a line like this:

*/5 * * * * ps -eco\%cpu=,command=>>~/.ps

-c changes -o command from the full command line to the name of the command in OS X's ps like in the procps implementation of ps (which comes with Linux distributions) by default. Percent signs have to be escaped in crontab because an unescaped percent sign is treated as a linefeed.

Then later run this command:

awk '{x=$1;$1="";a[$0]+=x}END{for(i in a)print a[i],i}' ~/.ps|sort -rn


Using a shell script to configure keyboard shortcuts

As an alternative to configuring keyboard shortcuts from System Preferences, you can modify the NSUserKeyEquivalents dictionaries in ~/Library/Preferences/.GlobalPreferences.plist and in the property lists of applications like the script below.

defaults write -g NSUserKeyEquivalents '{
"Show Previous Tab"="@j";
"Show Next Tab"="@k";
"Minimize All"="\u200b";

defaults write com.apple.finder NSUserKeyEquivalents '{
"Show Package Contents"="@\r";

defaults write -app safari NSUserKeyEquivalents '{
"Mail Contents of This Page"="\u200b";

defaults write -app mail NSUserKeyEquivalents '{

defaults write -app skim NSUserKeyEquivalents '{
"Single Page Continuous"="$@0";
"Single Page"="$@1";
"Two Pages"="$@2";

The shortcut strings use the same format as in DefaultKeyBinding.dict.

You can disable a shortcut by setting the shortcut string to for example \u200b (ZERO WIDTH SPACE). Some examples use nil, but it is triggered by the n key in some applications like Audacity, and property lists do not have anything like null values. I used \0 in place of \u200b before, but it started being triggered by any key in some version of Safari.

If you specify the name of a menu item as Format->Indentation->Increase in System Preferences, it is stored as \033Format\033Indentation\033Increase in a property list.

The -app option of defaults does not work with applications in /System/Library/CoreServices like Finder.

cp -r and cp -R

OS X's cp is based on an old version of the FreeBSD cp. The man page for cp in OS X and old versions of FreeBSD says that -r does not copy special files, symbolic links, or FIFOs "correctly":

Historic versions of the cp utility had a -r option. This implementation supports that option; however, its use is strongly discouraged, as it does not correctly copy special files, symbolic links, or fifo's.

In new versions of FreeBSD, -r is synonymous to -RL. See http://www.freebsd.org/cgi/man.cgi?query=cp:

Historic versions of the cp utility had a -r option. This implementation supports that option, however, its behavior is different from historical FreeBSD behavior. Use of this option is strongly discouraged as the behavior is implementation-dependent. In FreeBSD, -r is a synonym for -RL and works the same unless modified by other flags. Historical implementations of -r differ as they copy special files as normal files while recreating a hierarchy.

In GNU cp, the -r and -R options are identical, but adding the --copy-contents and -L options makes them behave like the -r option in OS X. The description of the --copy-contents option in the coreutils manual also describes the behavior of the -r option in OS X:


If copying recursively, copy the contents of any special files (e.g., FIFOs and device files) as if they were regular files. This means trying to read the data in each source file and writing it to the destination. It is usually a mistake to use this option, as it normally has undesirable effects on special files like FIFOs and the ones typically found in the /dev directory. In most cases, cp -R --copy-contents will hang indefinitely trying to read from FIFOs and special files like /dev/console, and it will fill up your destination disk if you use it to copy /dev/zero. This option has no effect unless copying recursively, and it does not affect the copying of symbolic links.

With OS X's cp, for example mkdir a;touch b;ln -s $PWD/b a;cp -r a c replaces the symlink with the target file, mkdir a;mkfifo a/b;cp -r a c gets blocked reading the contents of the FIFO, and mkdir a;ln -s /dev/zero a;cp -r a b starts filling b/zero with null bytes.

The -r option was removed from POSIX in the 2013 edition of the standard. (As of 2017, there have been three editions of the POSIX.1-2008 (aka Base Specifications, Issue 7) standard, which were published in 2008, 2013, and 2016.) This is what the page for cp in POSIX now says about the -r option:

Earlier versions of this standard included support for the -r option to copy file hierarchies. The -r option is historical practice on BSD and BSD-derived systems. This option is no longer specified by POSIX.1-2008 but may be present in some implementations. The -R option was added as a close synonym to the -r option, selected for consistency with all other options in this volume of POSIX.1-2008 that do recursive directory descent.

The difference between -R and the removed -r option is in the treatment by cp of file types other than regular and directory. It was implementation-defined how the -r option treated special files to allow both historical implementations and those that chose to support -r with the same abilities as -R defined by this volume of POSIX.1-2008. The original -r flag, for historic reasons, did not handle special files any differently from regular files, but always read the file and copied its contents. This had obvious problems in the presence of special file types; for example, character devices, FIFOs, and sockets.

Preserving non-ASCII characters with pbcopy and pbpaste

pbcopy and pbpaste do not preserve non-ASCII characters in the C locale. See the pbcopy manual:

For example, absent other locale settings, setting the environment variable LANG=en_US.UTF-8 will cause pbcopy and pbpaste to use UTF-8 for input and output. If an encoding cannot be determined from the locale, the standard C encoding will be used.

Terminal and iTerm 2 set LANG to a value like en_US.UTF-8 by default, where the language and territory parts depend on what region is selected from System Preferences, but the locale variables are unset by default in other places like in Automator and Script Editor. LC_CTYPE=UTF-8 pbcopy and LC_CTYPE=UTF-8 pbpaste always use UTF-8 (if LC_ALL is not set):

$ unset LANG LC_ALL
$ echo ä|pbcopy;osascript -e'the clipboard'
$ echo ä|LC_CTYPE=UTF-8 pbcopy;osascript -e'the clipboard'
$ echo ä|LC_CTYPE=UTF-8 pbcopy;pbpaste
$ echo ä|LC_CTYPE=UTF-8 pbcopy;LC_CTYPE=UTF-8 pbpaste

LC_CTYPE can be set to just UTF-8, but LANG and LC_ALL also have to include the language and territory parts (like en_US.UTF-8).

Locales in shells

Terminal and iTerm 2 set LANG based on the region selected from System Preferences by default. If you for example change the region to German from System Preferences, LANG is set to de_DE.UTF-8 by default:

$ defaults read -g AppleLocale
$ locale

The locale variables are unset by default in other contexts like in Automator and Script Editor.

You can make shells use the C locale by adding unset LANG or export LANG=C to a shell configuration file, by disabling the "Set locale environment variables on startup" setting in Terminal, or by disabling the "Set locale variables automatically" setting in iTerm 2.

If you use the C locale:

It is usually more convenient to use a UTF-8 locale. Either add a line like export LANG=en_US.UTF-8 to a shell configuration file or do not disable the option to set locale variables automatically in Terminal or iTerm 2.

Decomposed filenames in HFS+

HFS+ stores many non-ASCII characters in filenames in decomposed form (such as LATIN SMALL LETTER A followed by COMBINING DIAERESIS) instead of precomposed form (such as LATIN SMALL LETTER A WITH DIAERESIS).

iconv -f utf-8-mac -t utf-8 converts text from the decomposed form used by HFS+ to precomposed form:

$ printf \\ue4|xxd -p
$ touch $'\ue4'
$ ls|tr -d \\n|xxd -p
$ ls|tr -d \\n|iconv -f utf-8-mac -t utf-8|xxd -p

HFS+ does not use the standard NFD (normal form decomposed) but a variant of NFD where the characters from U+2000 to U+2FFF, from U+F900 to U+FAFF, and from U+2F800 to U+2FAFF are not decomposed. See http://developer.apple.com/library/mac/#qa/qa1173/_index.html.

In old versions of Bash, if you for example type a precomposed ä and press tab, a file whose name starts with a decomposed ä is not included in the completion results, but Bash 4.1 and later treat the precomposed and decomposed forms of characters as equivalent in contexts such as filename completion and globbing. See http://tiswww.case.edu/php/chet/bash/CHANGES:

kk. Bash and readline can now convert file names between precomposed and
    decomposed Unicode on Mac OS X ("keyboard" and file system forms,
    respectively).  This affects filename completion (using new
    rl_filename_rewrite_hook), globbing, and readline redisplay.

Disabling spell checking and options in the substitutions menu

This disables spell checking in most text views:

defaults write -g NSAllowContinuousSpellChecking -bool false

The command above also makes selecting the "Check Spelling While Typing" menu item have no effect, but you can still check spelling manually by pressing command-;.

This disables options in the substitutions menu:

for d in $(defaults domains|tr -d ,);do osascript -e"app id\"$d\""||continue;for x in CheckSpellingWhileTyping SmartDashes SmartLinks SmartQuotes SmartCopyPaste TextReplacement;do defaults write $d $x -bool false;done

Common extended attributes

DTrace scripts

$ apropos dtrace|sed $'s/(.*- /\t/;s/\. Uses DTrace//'|awk -F\\t '{printf"%20s %s\n",$1,$2}'
Tcl_CommandTraceInfo monitor renames and deletes of a command
          bitesize.d analyse disk I/O size by process
           cpuwalk.d Measure which CPUs a process runs on
       creatbyproc.d snoop creat()s by process name
            dappprof profile user and lib function usage
           dapptrace trace user and library function usage
            diskhits disk access by file offset
          dispqlen.d dispatcher queue length by CPU
              dtrace generic front-end to the DTrace facility
              dtruss process syscall details
             errinfo print errno for syscall fails
           execsnoop snoop new process execution
              fddist file descriptor usage distributions
        filebyproc.d snoop opens by process name
           hotspot.d print disk event by location
         httpdstat.d realtime httpd statistics
            iofile.d I/O wait time by file and process
           iofileb.d I/O bytes by file and process
           iopattern print disk I/O pattern
           iopending plot number of pending disk events
             iosnoop snoop I/O events as they occur
               iotop display top disk I/O events by process
              kill.d snoop process signals as they occur
           lastwords print syscalls before exit
             loads.d print load averages
           newproc.d snoop new processes
           opensnoop snoop file opens as they occur
         pathopens.d full pathnames opened ok count
          perldtrace Perl's support for DTrace
         pidpersec.d print new PIDs per sec
           plockstat front-end to DTrace to print statistics about POSIX mutexes and read/write locks
          priclass.d priority distribution by scheduling class
           pridist.d process priority distribution
         procsystime analyse system call times
            runocc.d run queue occupancy by CPU
           rwbypid.d read/write calls by PID
          rwbytype.d read/write bytes by vnode type
             rwsnoop snoop read/write events
          sampleproc sample processes on the CPUs
          seeksize.d print disk event seek report
           setuids.d snoop setuid calls as they occur
           sigdist.d signal distribution by process
      syscallbypid.d syscalls by process ID
     syscallbyproc.d syscalls by process name
     syscallbysysc.d syscalls by syscall
          topsyscall top syscalls by syscall name
          topsysproc top syscalls by process name
        weblatency.d website latency statistics

apropos descriptions of OS X executables

                 authopen open file with authorization
              axauthorize Accessibility Authorize
              compileHelp command-line utility to merge contextual help rtf snippets into one resource
        configureLocalKDC generate a LocalKDC
                    desdp scripting definition generator
                    fwkdp FireWire KDP Tool
                   fwkpfv FireWire kprintf viewer
      gen_bridge_metadata Objective-C Bridges Metadata Generator
               genstrings generate string table from source code
          graphicssession graphics session initialization utility
                  hdiutil manipulate disk images (attach, verify, burn, etc)
                 inituser Creates a user's home directory and sets quota
                  latency monitors scheduling and interrupt latency
                 mcxquery Managed Client (MCX) compositor query tool
          memory_pressure Tool to apply real or simulate memory pressure on the system
          migrateLocalKDC migrates a LocalKDC
                  mnthome mount an AFP (AppleShare) home directory with the correct privileges
               notifyutil notification command line utility
                  osalang information about installed OSA languages
                 pkgbuild Build an OS X Installer component package from on-disk files
                       pl ASCII property list utility
             productbuild Build a product archive for the OS X Installer or the Mac App Store
              productsign Sign an OS X Installer product archive
                 qc2movie Quartz Composer export tool
             qtmodernizer Tool to convert legacy movies to modern format with minimal changes
             sandbox-exec execute within a sandbox
         sandbox-simplify simplify a sandbox profile created by a trace directive
                 sc_usage show system call usage statistics
                     sdef scripting definition extractor
                      sdp scripting definition (sdef) processor
                 setquota Sets home directory quotas on the local computer
                  sw_vers print Mac OS X operating system version information
                tiff2icns converts TIFF to icns format
 update_dyld_shared_cache Updates dyld's shared cache
                  vm_stat show Mach virtual memory statistics
                wait4path wait for given path to show up in the namespace
                      xip Create a signed archive for secure distribution
     AOSNotificationAgent Find My Mac agent
AOSNotificationLoginAgent Find My Mac agent
              CVMCompiler Mac OS X Core Virtual Machine Compiler
                CVMServer Mac OS X Core Virtual Machine Server
         FDERecoveryAgent Full Disk Encryption Key Recovery Transmission Agent
           FileStatsAgent collect file stats
          PasswordService Mac OS X Server Password Server daemon
       StartupItemContext Execute a program in StartupItem context
            SystemStarter Start, stop, and restart system services
               aosnotifyd Find My Mac daemon
                appsleepd app sleep daemon
                     apsd Apple Push Notification service daemon
               aslmanager Apple System Log data life-cycle manager
                      asr Apple Software Restore; copy volumes (e.g. from disk images)
                  atsutil font registration system utility
                    audit audit management utility
                   auditd audit log management daemon
                    bless set volume bootability and startup disk options
                    blued The Mac OS X Bluetooth daemon
                    bnepd The Mac OS X Bluetooth network daemon
             btmmdiagnose gather information to aid in diagnosing Back to My Mac issues
                 cfprefsd defaults server
                  configd System Configuration Daemon
             corestoraged CoreStorage volume manager
                 csunique modify the UUIDs of CoreStorage metadata objects
                dirhelper helper for special directory creation
          diskmanagementd DiskManagement.framework server
                distnoted distributed notification server
                  dnsextd BIND Extension Daemon
              docsetbless Documentation set installation blesser
           docsetinstalld Documentation set installer daemon
               dsconfigad retrieves/changes configuration for Active Directory
                    dserr prints a description for an error code
            dynamic_pager external storage manager for dynamic pager
               exfat.util ExFAT file system utility
              fibreconfig Tool for configuring settings for Fibre Channel controllers and targets
        filecoordinationd system-wide file access coordination
            firmwaresyncd synchronize files used by the system firmware
                    fontd Mac OS X system font registration manager
                fontmover Mac OS X system font mover
              fontrestore Restore the available system fonts to a pristine state, if possible
               fontworker Mac OS X system font registration and validation daemon
                     gssd Generic Security Services Daemon
                     hdid historical mechanism for attaching disk images
                     hdik lightweight tool to attach and mount disk images in-kernel
                 ifcstart rebuilds international data caches
           installer-core helper for installer tool
                installer system software and package installer tool
                 ipconfig view and control IP configuration state
                  ippusbd ipp to usb bridging daemon
                  kcproxy Keychain Proxy
                   kdumpd Mac OS X remote kernel core dump server
                kextcache create kext cache files
                    kextd kernel extension server
                 kextfind find kernel extensions (kexts) based on a variety of criteria and print information
                 kextlibs find OSBundleLibraries needed by a kext
                 kextload load kernel extensions (kexts) into the kernel
               kextunload terminate driver I/O Kit driver instances and unload kernel extensions (kexts)
                 kextutil load, diagnose problems with, and generate symbols for kernel extensions (kexts)
                    kuncd The Kernel User Notification Center daemon
            mDNSResponder Multicast and Unicast DNS daemon
      mDNSResponderHelper mDNS privilege separation helper
               mddiagnose gather information to aid in diagnosing Spotlight issues
                      mds metadata server
                 mdworker metadata server worker
                    mkbom create a bill-of-materials file
                 mkpassdb Mac OS X Server Password Server database creation tool
                     mtmd Mobile Time Machine snapshot daemon
                    mtmfs Mobile Time Machine file system daemon
                  natutil Query and configure supported Network Address Translation (NAT) devices
                    nbdst NetBoot deferred shadow tool
                 networkd network daemon
      networkd_privileged networkd privileged helper
             networksetup configuration tool for network settings in System Preferences
                     nlcd Network Link Conditioner daemon
                  notifyd notification server
               nsnetworkd The background NSURLSession daemon
                ntfs.util NTFS file system utility
             ntpd-wrapper Wrapper for ntpdate/ntpd called by launchd
           opendirectoryd is a launchd(8) job for client access to local or remote directory systems
              path_helper helper for constructing PATH environment variable
                   pboard pasteboard server
                    pictd general helper tool
              productutil product archive utility
                 pwpolicy gets and sets password policies
          repair_packages verify or repair filesystem permissions and flags for packages
                revisiond storage manager for document revisions
                 scselect Select system configuration location
                   scutil Manage system configuration parameters
                    spctl SecAssessment system policy security
                  syslogd Apple System Log server
              systemsetup configuration tool for certain machine settings in System Preferences
                 talagent helper agent for the Transparent App Lifecycle feature
                      ubd ubiquity daemon, part of Documents in the Cloud
         universalaccessd universal access server
                     vpnd Mac OS X VPN service daemon
              warmd_agent general helper tool
                   wdutil Wireless Diagnostics command line utility
            weakpass_edit Mac OS X Server Password Server weak password dictionary insertion tool
                xpchelper XPC helper agent

I selected the lines above from the output of this command:

grep -l 'Mac OS X' /usr/share/man/man[18]/*|sed 's,.*/\(.*\)..,\1,'|while read l;do apropos $l|grep "^$l([18])";done

Shell commands for shutting down, restarting, logging out, putting a computer to sleep, and putting displays to sleep

Shut down without showing a confirmation dialog:

osascript -e'tell app"system events"to shut down'

Shut down after showing a confirmation dialog:

osascript -e'tell app"loginwindow"to«event aevtrsdn»'

Restart without showing a confirmation dialog:

osascript -e'tell app"system events"to restart'

Restart after showing a confirmation dialog:

osascript -e'tell app"loginwindow"to«event aevtrrst»'

Log out without showing a confirmation dialog:

osascript -e'tell app"system events"to«event aevtrlgo»'

Log out after showing a confirmation dialog:

osascript -e'tell app"system events"to log out'

Go to sleep:

pmset sleepnow

Put displays to sleep:

pmset displaysleepnow

The four letter codes for Apple events are listed in AERegistry.h. You can open it by running open -h AERegistry.h.

All of the osascript commands above send Apple events to the loginwindow process. loginwindow is sent the same Apple events when the user logs out, restarts, shuts down normally. See https://developer.apple.com/library/mac/qa/qa1134/_index.html.

According to man shutdown, shutdown -h now and shutdown -r now send processes a TERM signal followed by a KILL signal.

According to the Daemons and Services Programming Guide, when loginwindow is told to log out, processes that support sudden termination are sent a KILL signal, and processes that do not support sudden termination are terminated in different ways: Cocoa applications receive the applicationShouldTerminate: delegate method, foreground applications receive the kAEQuitApplication Apple event, background applications receive the kAEQuitApplication Apple event followed by a KILL signal, and daemons receive a TERM signal followed by a KILL signal after a few seconds.

Modifying tags with xattr

The tags for color labels have names like Red\n6 where \n is a linefeed. I added the tags for all color labels to a file before running the command below.

$ xattr -p com.apple.metadata:_kMDItemUserTags file|xxd -r -p|plutil -convert xml1 - -o -
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">

You can copy the tags of one file to another file by copying the com.apple.metadata:_kMDItemUserTags and com.apple.FinderInfo extended attributes:

xattr -wx com.apple.metadata:_kMDItemUserTags "$(xattr -px com.apple.metadata:_kMDItemUserTags file1)" file2
xattr -wx com.apple.FinderInfo "$(xattr -px com.apple.FinderInfo file1)" file2

If the kColor flag in com.apple.FinderInfo is not set, Finder does not show the circles for color labels below or next to a file. If the kColor flag is set to the value for orange and a file has the red tag, Finder shows both the red and orange circles next to the file. You can set the kColor flag with AppleScript:

xattr -w com.apple.metadata:_kMDItemUserTags '("Red\n6")' file
osascript -e'on run{a}' -e'tell app"finder"to set label index of(posix file a as alias)to item 1 of{2,1,3,6,4,5,7}' -eend file

You can also set the kColor flag with xattr:

xattr -wx com.apple.FinderInfo '00 00 00 00 00 00 00 00 00 0C 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00' file

The command above removes other file flags. It sets the color to red, but change C to E for orange, to A for yellow, to 4 for green, to 8 for blue, to 6 for magenta, or to 2 for gray. The flag that would add one to the values (kIsOnDesk) is not used in OS X.

Mounting an external drive in single user mode

Run these commands, where disk1s2 is the identifier of the disk shown by diskutil list:

mount -uw /
launchctl load /System/Library/LaunchDaemons/com.apple.notifyd.plist
launchctl load /System/Library/LaunchDaemons/com.apple.kextd.plist
mkdir /tmp/a
mount_hfs /dev/disk1s2 /tmp/a

Values of the com.apple.quarantine extended attributes

The com.apple.quarantine extended attributes have values like this:

$ xattr -p com.apple.quarantine Downloads/ubuntu-16.04-desktop-amd64.iso

The second field is a timestamp as seconds since epoch:

$ /bin/date -r 0x575eb8d6
Mon Jun 13 16:44:54 EEST 2016
$ gdate -d@$((16#575eb8d6))
Mon Jun 13 16:44:54 EEST 2016

The fourth field corresponds to a UUID in ~/Library/Preferences/com.apple.LaunchServices.QuarantineEventsv2:

$ sqlite3 ~/Library/Preferences/com.apple.LaunchServices.QuarantineEventsv2 .dump|grep B0F08FB0-5B78-42DE-AEAD-BD5785EF5B04
INSERT INTO "LSQuarantineEvent" VALUES('B0F08FB0-5B78-42DE-AEAD-BD5785EF5B04',484273804.205794,'com.apple.Safari','Safari','http://www.nic.funet.fi/pub/mirrors/releases.ubuntu.com/16.04/ubuntu-16.04-desktop-amd64.iso',NULL,NULL,0,NULL,'http://www.ubuntu.com/',NULL);
$ sqlite3 ~/Library/Preferences/com.apple.LaunchServices.QuarantineEventsv2 .schema
CREATE TABLE LSQuarantineEvent (  LSQuarantineEventIdentifier TEXT PRIMARY KEY NOT NULL,  LSQuarantineTimeStamp REAL,  LSQuarantineAgentBundleIdentifier TEXT,  LSQuarantineAgentName TEXT,  LSQuarantineDataURLString TEXT,  LSQuarantineSenderName TEXT,  LSQuarantineSenderAddress TEXT,  LSQuarantineTypeNumber INTEGER,  LSQuarantineOriginTitle TEXT,  LSQuarantineOriginURLString TEXT,  LSQuarantineOriginAlias BLOB );
CREATE INDEX LSQuarantineEventIndex  ON LSQuarantineEvent (  LSQuarantineEventIdentifier );
CREATE INDEX LSQuarantineTimeStampIndex  ON LSQuarantineEvent (  LSQuarantineTimeStamp );

The first field is a hexadecimal bitfield, where for example the seventh bit gets set when a file is opened, and the sixth bit gets set when a file is opened so that the warning dialog about opening a quarantined file is shown but the user presses a cancel button.

Finding the device name for the Wi-Fi network interface

Here are three ways to find the network device name (or what Apple sometimes calls the BSD name) for the Wi-Fi interface:

networksetup -listallhardwareports|awk '/^Hardware Port: Wi-Fi/{getline;print$2}'
sudo gem install plist;ruby -rplist -e'puts Plist::parse_xml("/Library/Preferences/SystemConfiguration/NetworkInterfaces.plist")["Interfaces"].select{|i|i["SCNetworkInterfaceType"]=="IEEE80211"}[0]["BSD Name"]'
networksetup -listnetworkserviceorder|sed -n 's/^(Hardware Port: Wi-Fi, Device: \(en.\))$/\2/p'

In MacBook Airs without an ethernet port the device name for Wi-Fi is usually (or might always be) en0, in Macs with a single ethernet port it is usually (or might always be) en1, and in at least some Mac Pros with two ethernet ports it is en2.