Vim has had syntax highlighting since version 5.0 in 1998. It quickly became indispensable. It's hard to go back to looking at monochromatic source code after you've become accustomed to syntax highlighting.

The syntax highlighting is tied into Vim's support for colorschemes, which define colors for the fundamental syntax groups like Number, Comment, and String. The syntax highlighting for a particular language defines custom syntax groups for specific language features, such as cppExceptions or htmlEndTag,

The custom syntax groups are linked to the underlying fundamental syntax groups. Hence, if you change your colorscheme, your syntax highlighting is updated automatically.

The reStructuredText syntax highlighting in Vim 7.2 has some shortcomings, in my opinion. For example, *italic* shows up as italic in gVim, but in the same color as regular text, while **bold** shows up in a different color, but not bolded.

When you declare a syntax group, you can either link it to another gropu and pick up that's one color and fontstyle, or you can give it a concrete fontstyle and color. If you do that, then the syntax group won't change color when you change the colorscheme.

After much poking around, I found a way to set a syntax group's fontstyle and link it to another group's color: see hi def rstItalic and hi def rstBold below.

I also make use of certain Unicode characters in my reStructuredText source, such as endash and emdash, which are very hard to tell apart in a fixed-width font—even though an emdash (—) is twice as wide as an endash (–) in a proportional font. Worse, a non-breaking space is invisible and can't easily be distinguished from a normal space.

I provided custom highlighting for these Unicode characters and the various ‘curly’ “quotes”.

All of this goes into ~/.vim/syntax/rst.vim, which treats $VIMRUNTIME/syntax/rst.vim as a subroutine. I tried putting it into ~/.vim/after/syntax/rst.vim, which gets executed after $VIMRUNTIME/syntax/rst.vim completes, but then I can't provide non-overrideable definitions for rstEmphasis and rstStrongEmphasis.

function! s:SynFgColor(hlgrp)
    return synIDattr(synIDtrans(hlID(a:hlgrp)), 'fg')
endfun

function! s:SynBgColor(hlgrp)
    return synIDattr(synIDtrans(hlID(a:hlgrp)), 'bg')
endfun

syn match rstEnumeratedList /^\s*[0-9#]\{1,3}\.\s/
syn match rstBulletedList /^\s*[+*-]\s/
syn match rstNbsp /[\xA0]/
syn match rstEmDash /[\u2014]/
syn match rstUnicode /[\u2013\u2018\u2019\u201C\u201D]/ " – ‘ ’ “ ”

exec 'hi def rstBold    term=bold cterm=bold gui=bold guifg=' . s:SynFgColor('PreProc')
exec 'hi def rstItalic  term=italic cterm=italic gui=italic guifg=' . s:SynFgColor('Statement')
exec 'hi def rstNbsp    gui=underline guibg=' . s:SynBgColor('ErrorMsg')
exec 'hi def rstEmDash  gui=bold guifg=' . s:SynFgColor('Title') . ' guibg='. s:SynBgColor('Folded')
exec 'hi def rstUnicode guifg=' . s:SynFgColor('Number')

hi link rstEmphasis       rstItalic
hi link rstStrongEmphasis rstBold
hi link rstEnumeratedList Operator
hi link rstBulletedList   Operator

source $VIMRUNTIME/syntax/rst.vim

syn cluster rstCruft                contains=rstEmphasis,rstStrongEmphasis,
      \ rstInterpretedText,rstInlineLiteral,rstSubstitutionReference,
      \ rstInlineInternalTargets,rstFootnoteReference,rstHyperlinkReference,
      \ rstNbsp,rstEmDash,rstUnicode

I have a long-standing fascination with typography. In the late '80s and early '90s, I became quite adept with TeX and LaTeX, the well-known scientific typesetting system. When I was at ICPC, I think I read the TeXbook cover to cover—twice. I became the TeX administrator for the CS department while I was at Brown.

And then I moved to Seattle to work for Microsoft and entered the world of Windows, and I left TeX behind for more than 15 years.

I wrote the other day that I prepared the Bloomsday scripts in XML for several years, using XSLT to generate HTML. I used to send the HTML to the readers, but everyone's browser paginated differently when printing, which led to confusion at rehearsals. So I started giving them PDFs: problem solved except for the person who needed a large-print version.

Last year, I prepared the script with reStructuredText. Normally, I use reST to generate HTML, but reST can also generate LaTeX. I decided to use rst2latex to take advantage of LaTeX's superior typesetting.

I wasn't happy with the results. The script looked like a crappy technical paper from the '90s, thanks to the tired Computer Modern layout. CM works well for math, less well for text, in my opinion.

The MacTeX extras included XeTeX, a modern variant of TeX that supports Unicode and OpenType fonts. I experimented with using Hoefler to set the script. You can see the results above: it looks gorgeous.

More to come.

I mentioned in my post on reStructuredText that I use a little command-line tool, pbcopy, to pipe the output into the clipboard. I finally found a similar tool for Linux, xsel.

• Mac: pbcopy (UTF-8 aware, unlike the built-in version of pbcopy) copies its input to the pasteboard (Mac name for the clipboard); pbpaste writes the pasteboard to stdout.
• Linux: xsel gets and sets the X selection.
• Windows: winclip reads and writes the clipboard in a variety of formats. Use -m for UTF-8 text. The winclip binary is available as part of the outwit package.

I hate composing anything longer than a couple of paragraphs in an online HTML editor. Specifically, I hate writing posts for this blog online. I'd much rather write in Vim and upload HTML. But I don't want to compose in raw HTML either.

I use reStructuredText (reST), an unobtrusive plaintext markup language popular in the Python world. reST can generate HTML, LaTeX, native PDF, ODF, and other formats. The picture at right shows a draft of this document in MacVim; reST is, as you can see, quite readable (though I work with a larger font). I use restview to preview the HTML locally and Pygments for syntax highlighting of code. Vim has its own syntax highlighting for reST and I've developed a set of keyboard macros for my own use.

The weak link in this scheme is posting to the blog. Right now, I have a little wrapper that generates HTML, extracts the body, and copies it to the pasteboard (clipboard). I then manually paste that into a raw HTML textarea in the blog's editor. Someday, I have to adapt mtsend or Firedrop2 to make this less painful. Or I could hack dasBlog to support reST in IronPython, or switch over to a blog that supports reST natively. Someday.

For a long time, I used VST (Vim reStructuredText) to generate HTML from reST. As I began using Python more and more, I realized that I was far better off with the real thing, which is well designed and quite fast. The VimL scripting language is not that good and VST pushes it to its limits.

As of the recent Python 2.6 release, all the official Python documentation is in reST format. Sphinx is a documentation build system that wraps a collection of reST documents into a larger navigable entity.

There are many other lightweight markup languages, such as Textile, Markdown, and AsciiDoc. No doubt they have their strengths, but I now have a significant investment in reST and it's well supported by the Python community.

Vim vs. Visual Studio

I've been an obsessive vi user for more than 20 years. Vi keystrokes are indelibly burned into my muscle memory. When I have to use Notepad or Word or Visual Studio, I feel crippled. I have to work harder to do simple things; I have to type too many chords with Alt and Ctrl; I have to take my hands off the home keys to use the cursor keys and the mouse.

In the mid-90s, I adopted Vim (Vi IMproved) to the point where I became a significant contributor, writing a big chunk of the Win32 code.

While I was at Microsoft, I hardly ever used Visual Studio. I edited my C/C++ code with Vim, I compiled and linked it with the NT Build Environment and I debugged it with WinDbg/ntsd/kd. I was hardly alone in this. In the Windows division, your code has to build with the NT build environment, and the Windows debuggers are much better supported than the Visual Studio debugger for developing the OS.

Now that I'm programming in C#, using the Visual Studio IDE makes a lot more sense. VS's IntelliSense for C# is much richer than Vim7's Omni completion, especially when coupled with ReSharper, and VS is the debugger of choice for managed code. I've been spending a fair amount of time in the VS IDE, especially when pair programming, but I've also been switching back to Vim a lot. When I'm struggling with unfamiliar code, VS's IntelliSense is a great comfort; when I'm moving a lot of text around, Vim suits me far better.

ViEmu

Earlier this week, by way of its graphical Vim cheat sheet, I found an interesting compromise. ViEmu is a vi/Vim emulator for VS 2003 and VS 2005.

ViEmu implements most of the vi keystrokes and many of the Vim extended keystrokes, right inside the Visual Studio IDE. It uses the native VS IntelliSense in place of Vim's completion functions. ViEmu even implements some of the more common Ex command line, including most of the :%s regular expression substitutions. The author, who seems to be known only as JNG, is responsive. Within 24 hours of my reporting some missing keystrokes, he had implemented them in a new minor release.

It does not, however, support VimL, the Vim extension language, so if you have an extensive suite of Vim plugins, as I do, they're not going to work in ViEmu.

All in all, I'm favorably impressed with ViEmu. It provides much of the muscle memory experience of Vim inside of Visual Studio. Technically, it can't have been easy to impose such a radically different input model on VS or to emulate Vim and Ex fairly faithfully.

Vim has always been free (actually charityware), but JNG charges for ViEmu. Right now, I'm in the 30-day trial period, but I fully expect that I'll pay for a license before the trial is up.

VisVim

Vim comes with a Visual Studio add-in called VisVim, which is based on another add-in called VisEmacs. It allows VS5 and VS6 to use Vim as the default editor, albeit externally to the IDE: Vim continues to run in its own window.

A few weeks ago, Bram asked me if I could get VisVim to compile with VS 2003. I tried, but I was unable. Necessary headers are no longer included with VS 2003 or VS 2005. No doubt this is because the Add-In architecture changed radically with the introduction of Visual Studio .NET.

Work is underway, albeit very slowly, to create VisEmacs.NET. At some point, it may be worth creating a merger of VisVim and VisEmacs.NET.

End Notes

viWord allows you to use vi keybindings in Microsoft Word. It's not nearly as full featured as ViEmu and I found that I didn't like it enough to keep it around.

This post was, of course, composed in Vim. I wrote it in lightly marked-up plain text and converted it to HTML with VST, Vim reStructured Text. Blogging with VST will be the topic of a future post.

To fully take advantage of Vim7's Omni completion, you need a patched version of Exuberant Ctags. I've made a Win32 binary available.