Wednesday, March 2, 2011

Clojure In Practice : Yahoo! Weather

I have finished reading Clojure In Action. Now it's time for some hands-on exercises. There are some practical examples in the book, but they did not interest me. A while ago, when I was reading Maven By Example, I came across the Yahoo! Weather RSS Feed, and I thought that reading RSS feeds was an entertaining way to make samples. I'll try to make a simple example of how to read the Yahoo! Weather RSS Feed in Clojure. I'm still new to Clojure, so there may be better ways to do it. But at least, let's make something work.

I was thinking of what was needed in order to make use of the feed :
  1. A way to send a request to the Yahoo! Weather RSS feed.
  2. A way to interpret the feed (XML).
  3. A convenient way to hold the weather data.

Sending a request to the Yahoo! Weather RSS feed

The url to the Yahoo! Weather RSS feed is There are two request parameters which can be used to customize the type of weather report : the WOEID, which is the weather location, and the units for temperature (Celsius or Fahrenheit). (Check all details here).

The clojure.xml namespace has a convenient function to parse xml data from a stream or a URI: the "parse" function. Let's try it by getting a weather report where temperatures are in Celcius :

  (:require [clojure.xml :as xml]))

(def yahoourl "")

(defn get-feed [woeid]
  (xml/parse (str yahoourl woeid)))
Five lines of code in order to grab the feed !! Programming is dead :)

The "parse" function returns a struct-map which has bunch of :tag, :attrs, and :content keys to represent the content of the xml. Cool, but not too convenient to use. The next step will be to make it easier to get values from tags and attributes.

Interpreting the feed

The namespace has some interesting functions to navigate through trees. As we know, xml represent data in a tree of tags, and will help go through it. When I first saw that namespace,, I thought it had something to do with zip files. No, it contains helper functions related to zipper data structures. In brief (from here), a zipper is a data structure representing a location in a hierarchical data structure, and the path it took to get there. It provides down/up/left/right navigation, and localized functional 'editing', insertion and removal of nodes. Let's change the "get-feed" function to transform our data into a zipper :
  (:require [clojure.xml :as xml]
            [ :as zip]))

(def yahoourl "")

(defn get-feed [woeid]
  (zip/xml-zip xml/parse (str yahoourl woeid)))
I've only added a required namespace and a call to "xml-zip". Programming is dead, really.

The namespace has some basic functions to help us read the feed. We can go up, down, left, right inside the tree of tags. There's an more convenient way to navigate through it : the zip-filter namespace of the Clojure-Contrib library. The doc says that zip-filter is a system for filtering trees and nodes generated by zip.clj in general, and xml trees in particular. There are some usage examples in the source file (xml.clj), so it's worth looking at. Let's use it. From Yahoo!'s documentation, we can see that the "rss" tag is the root tag of the xml document. It contains a "channel" tag which contains a few sub-elements.

By using the zip-filter namespace, we can easily access tags and attributes. For example, to get the city name of the weather report, we would use :
  (:require [clojure.xml :as xml] 
            [ :as zip]
            [ :as zf]))

(-> (zf/xml1-> zipper :channel :yweather:location) (zf/attr :city))
where "zipper" is the data returned by our "get-feed" function. As you can see, it first grabs the "channel" tag, then the "yweather:location" tag, then the "city" attribute. The "xml->" function may returns a collection. Instead, the "xml1->" function returns the first element of that collection. It's more convenient to get tags and attributes values.

Gathering the weather data

There are several ways to hold data in Clojure. The way weather data is expressed in xml seems to fit with Clojure struct-maps. I'll make four of them : one for the units (temperate, distance...), one for the wind information, one for the astronomy informations (sunset, sunrise), and one for the actual weather conditions.

(defstruct units :temp :distance :pressure :speed)
(defstruct wind :chill :speed)
(defstruct astronomy :sunrise :sunset)
(defstruct condition :text :temp :date)
Then comes the function which grabs the data we're interested in :
(defn get-weather [woeid]
  (let [zipper (get-feed woeid)
        z-location (zf/xml1-> zipper :channel :yweather:location)
        z-units (zf/xml1-> zipper :channel :yweather:units)
        z-wind (zf/xml1-> zipper :channel :yweather:wind)
        z-astronomy (zf/xml1-> zipper :channel :yweather:astronomy)
        z-condition (zf/xml1-> zipper :channel :item :yweather:condition)]
    :city (zf/attr z-location :city)
    :country (zf/attr z-location :country)
    :units (struct-map units
                       :temp (zf/attr z-units :temperature)
                       :distance (zf/attr z-units :distance)
                       :pressure (zf/attr z-units :pressure)
                       :speed (zf/attr z-units :speed))
    :wind (struct-map wind
                       :chill (zf/attr z-wind :chill)
                       :speed (zf/attr z-wind :speed))
    :astronomy (struct-map astronomy
                       :sunrise (zf/attr z-astronomy :sunrise)
                       :sunset (zf/attr z-astronomy :sunset))
    :condition (struct-map condition
                       :temp (zf/attr z-condition :temp)
                       :text (zf/attr z-condition :text)
                       :date (zf/attr z-condition :date))
That's a big function. There must be a way to refactor it, but I'll leave that as my next exercise. Finally, let's make a function to print out a basic weather report:
(defn print-weather [info]
  (let [condition (:condition info)
        units (:units info)]
  (println "Weather at" (:date condition) "," 
           (:city info) "," (:country info))
  (println (condition :text) "," 
           (condition :temp)
           (units :temp))))

Now, let's see all this in action. I'm using the WOEID for Tokyo/Japan, which is 1118370.
user=> (use ' :reload)
user=> (def info (get-weather 1118370))
user=> (println info)
{:city Tokyo, :country Japan, :units {:temp C, :distance km, :pressure mb, :speed km/h},
:wind {:chill 8, :speed 16.09}, :astronomy {:sunrise 6:09 am, :sunset 5:36 pm},
:condition {:text Light Rain Shower, :temp 10, :date Wed, 02 Mar 2011 12:29 pm JST}} user=> (:country info) "Japan" user=> (-> info (:condition) (:text)) "Light Rain Shower" user=> (print-weather info) Weather at Wed, 02 Mar 2011 12:29 pm JST , Tokyo , Japan Light Rain Shower , 10 C nil

That's it. A simple weather report in Clojure. There are two things I'd like to do next. First, the feed contains a weather forecast for the following day, which I'd like to get. Then, I need to refactor the get-weather function and find a better way to get the data.

1 comment:

  1. Thanks for the share. Keep posting such kind of information on your blog. I bookmarked it for continuous visit.
    html5 media player