Create Your Own Syntax Highlighting in Vim

In text editors like Vim, syntax highlighting is the feature that shows defined components in different colors depending on their functions in the file. Vim has code syntax highlighting already available for a huge array of programming languages, including PHP, Python, HTML, Ruby, JavaScript, and more, and most of them are installed by default in the $VIM/syntax/ directory, so in most cases you can just open up your source code files and see different parts appropriately color-coded. But what if you’re using a little-known language that doesn’t have a preinstalled syntax file? Or if you’re working with a different type of file altogether and want to create syntax highlighting from scratch? This tutorial shows how easy and even fun it is to write your own syntax highlighting rules for Vim.

Before you start building your own syntax highlighting rules, it may help to look at an existing set of rules. There are two ways you can do this:

1. Open a file for which Vim has preloaded rules (e.g. an HTML file) and type :syntax.
2. Edit one of the *.vim files in $VIM/syntax/.

Most language syntax files have a lot of rules, so don’t get overwhelmed; just take a quick look at the style of the rules.

An easy way to get the right setup for your new syntax file is to copy one of the existing files from $VIM/syntax/ and then delete most of its content, keeping only what you need and modifying it as you like. Alternatively, just create a file with the entries here. The example I’m going to use is for a todo list, with categories such as work, home, and personal. It might look a bit like this:

h: sort out drawers in spare room
w: draft Vim highlighting article
p: email jb@example.com

The syntax highlighting will give the different categories different colors.

Create a todo.vim file and start it off like this:

" Vim syntax file
" Language:     Juliet's todo files
" Maintainer:   Juliet Kemp
" Last Change:  Sept 14, 2011
" Version:      1

if exists("b:current_syntax")
  finish
endif

setlocal iskeyword+=:
syn case ignore

This header is standard for preinstalled syntax files, so it’s good practice to use it (or something similar) for your files too. " is the comment character for Vim syntax files. The if clause checks for any already defined syntax highlighting, and abandons processing here if it finds one. As a rule, this shouldn’t happen, but it would be possible to have a global setting to read one file and a local setting to read another; it’s best practice to have a clause like this to avoid any possible confusion.

The next line tells Vim that the keywords we’ll use include the : character. Keywords match the iskeyword option (use the command :set isk ? to find out what it is currently set to), which should by default include digits, alphabetic characters, /, @, and _. If you have any other punctuation characters in your keywords, you’ll need to set them up, as we do here.

The following line tells Vim to ignore case when matching keywords. You can use case match instead if that’s more suitable for your language or file type.

Next, add a couple of actual rules:

syn match todoHome	"^h: "
syn match todoWork	"^w: "
syn match todoPersonal	"^p: "

syn match looks for a specific regular expression to match. Here, we’re looking for lines starting with a particular letter followed by a colon [and a space], and attaching a syntax label to each match.

Next, we have to set the highlights that apply to each label:

highlight link todoHome		Type
highlight link todoWork		String
highlight link todoPersonal	Statement

Type, String, Statement, and others are preset groups that are associated with defined formats. You can see the full list on the Vimdoc website. Unfortunately the site doesn’t list colors used for each, so you’ll have to experiment to get something you like. If you’re setting up a new syntax file for a language, you should probably stick with the group names that are closest to the language elements that you’re highlighting, for maximum compatibility with what you and others are already used to.

Finally, set the name of the current syntax file:

let b:current_syntax = "todo"

Now you can save the file.

Loading a Syntax File

The next task is to make sure your syntax definitions get loaded the next time you open a todo list. The usual way Vim identifies files is by extension. We’ll call any todo list files *.todo, and add this line to ~/.vimrc:

autocmd BufRead,BufNewFile *.todo set filetype=todo

The filetype references the name of the syntax file: here, todo.vim. You also need to make sure that ~/.vimrc includes the line:

syntax on

This turns syntax highlighting on in Vim. It will often be set by default, but it’s a good idea to set it explicitly. Save the changes, and open a sample test.todo file and enter some appropriate lines. You should see the syntax highlighting working. If you want to change anything (for example, add a rule, or change the setting for the highlighting colors), edit todo.vim and save it, then type :syntax on in your test.todo buffer, and the new syntax rules will automatically be loaded.

Matching Regions

Currently, only the initial labels are highlighted. To highlight a whole line, you need to use a region match instead. Try replacing your syn match lines with these:

syn region todoHome	start=/^h: / end=/\n/
syn region todoWork	start=/^w: / end=/\n/
syn region todoPersonal	start=/^p: / end=/\n/

The syntax is very similar to syn match. It defines a label, then a start and an end pattern of the region. As these rules stand, they look for a newline (\n; alternatively, you can use $ to match end-of-line), so they give you only one-line highlighting. You can play around to adapt this if you want multiple-line todo items.

You can set up syntax highlighting such that certain matches get “priority.” For example, these lines look for email addresses (as defined by the regular expression in quotes – and this pattern is imperfect, so don’t rely on it for anything vital) and set their color:

syn match todoEmail	"[a-zA-Z0-9._-]\+@[a-zA-Z0-9./-]\+"
highlight link todoEmail	Identifier

However, as things stand, the todoHome and other defined regions override this, so you won’t get the email address highlighting you expect until you alter the syn region lines:

syn region todoHome	start=/^h: / end=/\n/ contains=todoEmail

The contains element tells Vim to look for this type of element contained in the region area, and then to highlight it appropriately. You can have as many labels as you like in this section. If the definition of the contained element also includes an end-of-line character (here, it doesn’t), add keepend at the end of the syntax definition to avoid the end-of-line being “swallowed” and the containing element running over into the next line:

syn region todoHome	start=/^h: / end=/\n/ contains=todoEmail keepend

To take this a bit further, you can also use the contained keyword to limit matches to occur only within certain sections. For example, if you wanted to highlight email addresses only within a labeled line, and not if you’d just made a note at the bottom of the file, you could use this syntax:

syn match todoEmail	contained "[a-zA-Z0-9._-]\+@[a-zA-Z0-9./-]\+"

This means that the match is valid only if it is contained within another syntax element that is specified as contains todoEmail. So if you had these lines as well:

syn region todoHome	start=/^h: / end=/\n/
syn region todoWork	start=/^h: / end=/\n/ contains=todoEmail

then an email address would be highlighted only if it was contained in a todoWork region.

You might want to highlight todo items you’ve completed. You can add a “done” setting for when you add “x” at the start of lines:

syn region todoDone	start=/^x[hwp]: / end=/\n/
highlight link todoDone	Underlined

If you want “done” lines to be invisible (but not deleted), you could use Ignore instead of Underlined.

Going Further

All of the above is fine if what you’re defining is single, specific matches. But if you’re writing syntax for a whole language, you might want an easier way to do it. This is where syn keyword comes in. Suppose you were writing a syntax file for Perl, and you want to group various sorts of blocks together:

syn keyword groupBlockLabels if while for

You could then define the highlight color with highlight link as above.

And, of course, it’s possible to get a lot more complicated. For instance, you can use the nextgroup option to specify that a certain sort of match should always appear after a certain keyword. After the groupBlockLabels element above, you might want to define a code block, beginning and ending with curly braces, and then specify that this type of block should always occur after a groupBlockLabels element:

syn region codeBlock	start=/{/ end=/}/
syn keyword groupBlockLabels if while for nextgroup=codeBlock

In the todo example, you could use a character other than a newline to end the text part of the todo, put a completion date after that, and use nextgroup to specify that a date should come after a todo list item.

One of the best ways to get to grips with syntax highlighting in Vim is to experiment. For more information to support your experimentation, check out the Vim documentation chapter on writing syntax files. In particular, check out the matchgroup and transparent options for more complicated setups. Happy highlighting – and feel free to share your best syntax highlighting tips and tricks in the comments section below!

Share this post:

Related posts:

1. Tips for Using Vim as an IDE
2. Extending Vim with Scripting
3. Vim Undo Tips and Tricks
4. jEdit: The Force is Strong with This Programmer’s Editor
5. More Fun with Vimscript

Related Open-Source Packages