Using Clojure in Vim - Part 2 : accessing REPL via Nailgun

I explained how to install the Clojure plugin for Vim, Vimclojure, in a previous blog entry. Now that I'm starting to get familiar with Clojure, I thought it would be nice to be able to access REPL from Vim. To achieve this, I downloaded the Nailgun client and server. Download them from the following locations:

• Nailgun server
• Nailgun client

Starting the Nailgun server

As explained in the Vimclojure help documentation in Vim (:help vimclojure), the server has to be started with clojure and clojure-contrib jars in the classpath. For example : "java -cp clojure.jar;clojure-contrib.jar;server-2.2.0.jar vimclojure.nailgun.NGServer 127.0.0.1". Actually, in the Vimclojure plugin archive, there is a "bin" directory with two batch files: ng-server (for unix) and ng-server.bat (for Windows). These batch files allow to launch the server easily. Create a ".clojure" file in the directory where you launch the batch from, and put all the necessary jars and classpath directories in it. One per line.


After launching Nailgun, a message should appear, telling you that the server is running :
NGServer started on 127.0.0.1, port 2113

Installing the Nailgun client

Unpack the client archive in a directory of your choice. Edit your .vimrc (_vimrc) file, and add or modify the following lines :

let vimclojure#WantNailgun = 1 
let vimclojure#NailgunClient = "D:\\nailgun-0.7.1\\ng.exe"

Replace the directory with the directory where you unpacked the client.

Using REPL in Vim

Launch Vim and create a new Clojure file. For example, simple.clj. I assume that it is saved in one of the directories of the classpath used for the Nailgun server.

(ns simple)
(defn hello [] (println "Hello"))

To launch a new Vim REPL, type <LocalLeader>sr. <LocalLeader> is defined by the Vim maplocalleader variable, which is '\' if not defined. Another useful command is <LocalLeader>sR, which will start REPL in the namespace of the current buffer (it uses Clojure's "require"). In the previous example, this would start REPL in the "simple" namespace. REPL will be opened in a new window, where Clojure can be used as usual. Very convenient. Check the following image to see the previous tiny snippet in action.

There are other useful commands, which are all explained in ":help vimclojure". One of them is <LocalLeader>lw, which executes Clojure's "doc" to the word under the cursor. It's very helpful to check the documentation of a function. Also helpful is <LocalLeader>sw, which shows the source of the word under the cursor. The preview windows opened by these commands can be close via <LocalLeader>p. Check the Vimclojure's help for more.

Clojure In Practice : Yahoo! Weather

I have finished reading Clojure In Action. Now it's time for some hands-on exercises. There are some practical examples in the book, but they did not interest me. A while ago, when I was reading Maven By Example, I came across the Yahoo! Weather RSS Feed, and I thought that reading RSS feeds was an entertaining way to make samples. I'll try to make a simple example of how to read the Yahoo! Weather RSS Feed in Clojure. I'm still new to Clojure, so there may be better ways to do it. But at least, let's make something work.

I was thinking of what was needed in order to make use of the feed :

1. A way to send a request to the Yahoo! Weather RSS feed.
2. A way to interpret the feed (XML).
3. A convenient way to hold the weather data.

Sending a request to the Yahoo! Weather RSS feed

The url to the Yahoo! Weather RSS feed is http://weather.yahooapis.com/forecastrss. There are two request parameters which can be used to customize the type of weather report : the WOEID, which is the weather location, and the units for temperature (Celsius or Fahrenheit). (Check all details here).



The clojure.xml namespace has a convenient function to parse xml data from a stream or a URI: the "parse" function. Let's try it by getting a weather report where temperatures are in Celcius :

(ns kuriqoo.weather
  (:require [clojure.xml :as xml]))

(def yahoourl "http://weather.yahooapis.com/forecastrss?u=c&w=")

(defn get-feed [woeid]
  (xml/parse (str yahoourl woeid)))

Five lines of code in order to grab the feed !! Programming is dead :)



The "parse" function returns a struct-map which has bunch of :tag, :attrs, and :content keys to represent the content of the xml. Cool, but not too convenient to use. The next step will be to make it easier to get values from tags and attributes.

Interpreting the feed

The clojure.zip namespace has some interesting functions to navigate through trees. As we know, xml represent data in a tree of tags, and clojure.zip will help go through it. When I first saw that namespace, clojure.zip, I thought it had something to do with zip files. No, it contains helper functions related to zipper data structures. In brief (from here), a zipper is a data structure representing a location in a hierarchical data structure, and the path it took to get there. It provides down/up/left/right navigation, and localized functional 'editing', insertion and removal of nodes. Let's change the "get-feed" function to transform our data into a zipper :

(ns kuriqoo.weather
  (:require [clojure.xml :as xml]
            [clojure.zip :as zip]))

(def yahoourl "http://weather.yahooapis.com/forecastrss?u=c&w=")

(defn get-feed [woeid]
  (zip/xml-zip xml/parse (str yahoourl woeid)))

I've only added a required namespace and a call to "xml-zip". Programming is dead, really.



The clojure.zip namespace has some basic functions to help us read the feed. We can go up, down, left, right inside the tree of tags. There's an more convenient way to navigate through it : the zip-filter namespace of the Clojure-Contrib library. The doc says that zip-filter is a system for filtering trees and nodes generated by zip.clj in general, and xml trees in particular. There are some usage examples in the source file (xml.clj), so it's worth looking at. Let's use it. From Yahoo!'s documentation, we can see that the "rss" tag is the root tag of the xml document. It contains a "channel" tag which contains a few sub-elements.

By using the zip-filter namespace, we can easily access tags and attributes. For example, to get the city name of the weather report, we would use :

(ns kuriqoo.weather
  (:require [clojure.xml :as xml] 
            [clojure.zip :as zip]
            [clojure.contrib.zip-filter.xml :as zf]))

(-> (zf/xml1-> zipper :channel :yweather:location) (zf/attr :city))

where "zipper" is the data returned by our "get-feed" function. As you can see, it first grabs the "channel" tag, then the "yweather:location" tag, then the "city" attribute. The "xml->" function may returns a collection. Instead, the "xml1->" function returns the first element of that collection. It's more convenient to get tags and attributes values.

Gathering the weather data

There are several ways to hold data in Clojure. The way weather data is expressed in xml seems to fit with Clojure struct-maps. I'll make four of them : one for the units (temperate, distance...), one for the wind information, one for the astronomy informations (sunset, sunrise), and one for the actual weather conditions.

(defstruct units :temp :distance :pressure :speed)
(defstruct wind :chill :speed)
(defstruct astronomy :sunrise :sunset)
(defstruct condition :text :temp :date)

Then comes the function which grabs the data we're interested in :

(defn get-weather [woeid]
  (let [zipper (get-feed woeid)
        z-location (zf/xml1-> zipper :channel :yweather:location)
        z-units (zf/xml1-> zipper :channel :yweather:units)
        z-wind (zf/xml1-> zipper :channel :yweather:wind)
        z-astronomy (zf/xml1-> zipper :channel :yweather:astronomy)
        z-condition (zf/xml1-> zipper :channel :item :yweather:condition)]
    { 
    :city (zf/attr z-location :city)
    :country (zf/attr z-location :country)
    :units (struct-map units
                       :temp (zf/attr z-units :temperature)
                       :distance (zf/attr z-units :distance)
                       :pressure (zf/attr z-units :pressure)
                       :speed (zf/attr z-units :speed))
    :wind (struct-map wind
                       :chill (zf/attr z-wind :chill)
                       :speed (zf/attr z-wind :speed))
    :astronomy (struct-map astronomy
                       :sunrise (zf/attr z-astronomy :sunrise)
                       :sunset (zf/attr z-astronomy :sunset))
    :condition (struct-map condition
                       :temp (zf/attr z-condition :temp)
                       :text (zf/attr z-condition :text)
                       :date (zf/attr z-condition :date))
     }))

That's a big function. There must be a way to refactor it, but I'll leave that as my next exercise. Finally, let's make a function to print out a basic weather report:

(defn print-weather [info]
  (let [condition (:condition info)
        units (:units info)]
  (println "Weather at" (:date condition) "," 
           (:city info) "," (:country info))
  (println (condition :text) "," 
           (condition :temp)
           (units :temp))))

Now, let's see all this in action. I'm using the WOEID for Tokyo/Japan, which is 1118370.

user=> (use 'kuriqoo.weather :reload)
user=> (def info (get-weather 1118370))
user=> (println info)
{:city Tokyo, :country Japan, :units {:temp C, :distance km, :pressure mb, :speed km/h},
 :wind {:chill 8, :speed 16.09}, :astronomy {:sunrise 6:09 am, :sunset 5:36 pm},
 :condition {:text Light Rain Shower, :temp 10, :date Wed, 02 Mar 2011 12:29 pm JST}}
user=> (:country info)
"Japan"
user=> (-> info (:condition) (:text))
"Light Rain Shower"
user=> (print-weather info)
Weather at Wed, 02 Mar 2011 12:29 pm JST , Tokyo , Japan
Light Rain Shower , 10 C
nil

That's it. A simple weather report in Clojure. There are two things I'd like to do next. First, the feed contains a weather forecast for the following day, which I'd like to get. Then, I need to refactor the get-weather function and find a better way to get the data.

Book review: Clojure In Action

There are already a few books on Clojure. I decided to go for one of the latest, Clojure In Action (the book was still in early access, finished, but not published yet).

The first seven chapters teach the basics. They are easy to follow. I felt that the chapter on concurrency was not clear enough. A lot of text about common concurrency issues, but no practical examples of the different Clojure features. Maybe in the sections that I skipped ? (more on that next). I just wanted to see one example using multiple threads and Refs, Agents and others.

Leaving the basics, chapter 8 shows how to do mocking and stubbing in unit testing. Then come four chapters using third party libraries and frameworks. I'm ashamed to say that I skipped most of their content (50 pages). HBase, Redis, RabbitMQ, Hadoop... That would have been fine with small libraries which do not require configuration or knowledge. This does not reduce the value of the book though. It's an "In Action" book after all. It is clearly stated on the cover that it will use these frameworks. If you are interested to learn more than just the language, this book contains a vast panorama of "in action" examples. I may come back to these chapters later.

Then, back to the core. Protocols, macros, etc... The modus operandi thing in the Protocols chapter confused me more than anything, although it's a good exercise at making macros.

I haven’t read other books on Clojure so I can’t compare this one to others. I'm pleased with this book, although I was interested only in the core functionality. I think it will give you enough grounding to get going with Clojure.

Clojure clj-record library and Postgresql

I came across a problem when using the clj-record library with Postgresql. When creating a new record in a table, a java.lang.IllegalArgumentException is thrown, despite the fact that the record was successfully created. The "create" function inserts and retrieves the inserted record, so it looked like the problem happened when fetching the newly inserted record. By looking at the source code for the "create" function, I found out that the "insert" function called by "create" calls "id-query-for", which is a multimethod. For postgresql, the function is :

(defmethod id-query-for "postgresql" [_ table-name]
  (str "SELECT currval(pg_get_serial_sequence('" table-name "','id'))"))

Calling Postgresql's pg_get_serial_sequence means that the id column must be called "id", and that it must be "serial". My table layout looked like that :

CREATE TABLE users
(
  id integer NOT NULL DEFAULT nextval('seq_users'::regclass),
  "login" character varying(10),
  first_name character varying(20),
  last_name character varying(20),
  "password" character varying(10),
  email_address character varying(50),
  CONSTRAINT pk_users PRIMARY KEY (id)
)

The column id was properly named "id", but its type was not serial, which was the problem. pg_get_serial_sequence returned null, so the record could not be fetched. I changed my table layout to look like this :

CREATE TABLE users
(
  id serial NOT NULL,
  "login" character varying(10),
  first_name character varying(20),
  last_name character varying(20),
  "password" character varying(10),
  email_address character varying(50),
  CONSTRAINT pk_users PRIMARY KEY (id)
)

This worked as expected. I don't quite like the fact that the column id is fixed to "id", nor the logic to fetch the inserted record...

Using Clojure in Vim

There are already a few plugins for well known editors to use Clojure. Anybody familiar with Vim will want to use it with Clojure. My environment is not complete yet, but here is what I am using so far :

• The VimClojure plugin, for syntax highlighting.
• The TagList plugin. Very useful plugin which shows the content of tags files.
• Exuberant Ctags, used by the TagList plugin.

I'm currently using Vim 7.3, TagList 4.5 and Exuberant Ctags 5.8.

VimClojure plugin

The installation is straight forward, and is fully explained in a readme file in the archive. The following settings has to added in your .vimrc (_vimrc) :

syntax on
filetype plugin indent on

I've also added the following two settings:

let vimclojure#HighlightBuiltins=1 
let vimclojure#ParenRainbow=1

The rainbow parenthesis might be distracting, but I quite like it this way.


I'm leaving the plugin in offline mode, so I didn't configure the Nailgun server. This allows to use nice features like dynamic documentation lookup and Repl running in a Vim buffer, but I'll configure that when I get more familiar with Clojure. For the moment, I use Repl in another window and load the file I'm editing using (load "filename").

(UPDATE: the explanations for configuring the Nailgun client and server are here)

Exuberant Ctags

Nothing more than downloading the latest archive, extracting it, and setting the directory where the ctags executable is into the path.

TagList plugin

One problem: the taglist plugin relies on the filetype detected by Vim and passes the filetype to the exuberant ctags utility to parse the tags. With the VimClojure plugin on, the filetype is set to "clojure", which is not recognized by the taglist plugin. In order to tell the plugin to use the Lisp syntax, the following line should be added to the .vimrc (_vimrc) file :

let tlist_clojure_settings = 'lisp;f:function'

A useful setting is "let Tlist_Exit_OnlyWindow=1", which will close the TagList window if the last file was closed.

In Vim, the ":Tlist" command shows the tag list in a new window.

That's it. With a couple of plugins, using Clojure in Vim becomes much more fun.To generate the help documentation of both plugins, just type ":helptags ~/vimfiles/doc"(or another path where the docs are) in Vim. Take some time to go through the help and check useful settings can be used.

Using Exuberant Ctags on the command line with Clojure files

The following steps are not necessary in order to use the TagList plugin. It should be done only if you want to generate the tags file for Clojure files from the command line.

I downloaded the latest version (5.8). Unfortunately, Clojure is not supported yet. But lisp is. Exuberant Ctags autodetects a file type by looking at its file extension. We can for it to use a different language by using the --language-force option (--language-force=lisp), but it's annoying to do it all the time. What can we do then ? It is possible to override the file mappings via the --langmap option. Let's see how it's done:

Check the supported languages

ctags --list-languages

Ant
Asm
Asp
Awk
Basic
BETA
C
C++
C#
Cobol
DosBatch
...

Clojure is not there.

Check the file mappings

ctags --list-maps
...
Lisp     *.cl *.clisp *.el *.l *.lisp *.lsp
...

The extension above are associated to Lisp.

Adding Clojure file extensions to the Lisp mapping

New extensions can be added via the --langmap option:

ctags --langmap=lisp:+.clj --list-maps
...
Lisp     *.cl *.clisp *.el *.l *.lisp *.lsp *.clj
...

To set it permanently, we have to set this option in the CTAGS environment variable (e.g. set CTAGS=--langmap=lisp:+.clj), or in one of the setting files used by Exuberant Tags (e.g. $HOME/.ctags)

After everything was set, looking at a tags file generated by "ctags *.clj" will confirm that the Clojure extension is properly recognized.