Monday, December 27, 2010

Unit testing Freemarker macros with JUnit and HtmlUnit

I recently had to modify a pagination macro in FreeMarker. That pagination system is a bit different from the ordinary ones, and the safest way to make sure that it was working properly was to unit test it. Making a lot of data to test it would have been to much hastle, so I looked for a way to test the FreeMarker macro out of the web container. I wanted to make sure that the generated page numbers were correct, and also that the previous and next arrows were displayed when needed. All pages are html anchors, all contained into a div tag. To test it, I had to :
  1. Make a unit test class for the macro.
  2. Make FreeMarker interprete the macro.
  3. Confirm that the expected html elements like divs and anchors were present.
Why using HtmlUnit ? Because it has some useful methods to retrieve html elements by their name or by their id.

Getting the necessary libraries

All I needed was : JUnit, FreeMarker and HtmlUnit. HtmlUnit has quite a lot of dependencies, but all the necessary libraries are part of the HtmlUnit archive. You can easy go to their respective homepage and download them yourself, or get them via Maven. Here are the necessary dependencies for the sample we'll make in this tutorial :
<dependencies>
    <dependency>
      <groupId>net.sourceforge.htmlunit</groupId>
      <artifactId>htmlunit</artifactId>
      <version>2.8</version>
    </dependency>
 <dependency>
     <groupId>org.freemarker</groupId>
     <artifactId>freemarker</artifactId>
     <version>2.3.16</version>
 </dependency>
 <dependency>
     <groupId>junit</groupId>
     <artifactId>junit</artifactId>
     <version>4.8.2</version>
 </dependency>
  </dependencies>

A basic macro

Let's make a basic pagination macro. We'll keep it simple. The page links won't point anywhere. We just want to show page numbers. The current page will be plain text, other page numbers will be html anchors. The macro parameters will be the total number of pages and the current page. It will also show previous and next if necessary.
<#macro doPagination totalPages currentPage >
 <#if (totalPages > 1)>
  <div id="pagination">
   <#-- Previous page -->
   <#if (currentPage > 1)>
    <a href="#">Prev</a>
   </#if>
   
   <#-- Page number -->
   <#list 1 .. totalPages as pageNumber>
    <#if (pageNumber == currentPage)>
     <div id="currentPage">${pageNumber}</div>
    <#else>
     <div><a href="#">${pageNumber}</a></div>
    </#if>  
   </#list>
   
   <#-- Next page -->  
   <#if (currentPage < totalPages)>
    <a href="#">Next</a>
   </#if>  
  </div>
 </#if>
</#macro>

Using a dummy template

We need a FreeMarker template to use the pagination macro. This template will just import the file where the macro is, and call the macro.
<#import "/main/webapp/templates/macros/pagination.ftl" as pagination/>
<html>
<body>
<@pagination.doPagination totalPages currentPage />
</body>
</html>

Preparing FreeMarker for the tests

We'll use the FreeMarker API to convert the dummy template into html. We'll make use of the following two classes : freemarker.template.Configuration and freemarker.template.Template. Let's set this up in the @Before method of our test class.
public class PaginationTest {

 /** Root directory where the FreeMarker templates are */
    private static final String TEMPLATE_ROOT = "src";
    /** Dummy pagination template location */
    private static final String PAGINATION_TEMPLATE = "test/java/com/kuriqoo/templates/macros/pagination_test.ftl";

    private Configuration cfg;
    private Template template;

    private Map<String, Object> rootMap;
    
    @Before
    public void setUp() throws Exception {
        cfg = new Configuration();
        cfg.setDirectoryForTemplateLoading(new File(TEMPLATE_ROOT));

        template = cfg.getTemplate(PAGINATION_TEMPLATE);
        
        rootMap = new HashMap<String, Object>();
    }
}
First, the TEMPLATE_ROOT. The macro is in "/src/main/webapp/templates/macros/pagination.ftl" and the dummy template is in "/src/test/java/com/kuriqoo/templates/macros/pagination_test.ftl". So the common template root is "src".

Then, the PAGINATION_TEMPLATE. This points to the dummy template. Note that the root ("src") is not included in it.


We create a Configuration and set its template root directory. Using that Configuration instance, we can build some templates, by calling its getTemplate method. Here, we are making our dummy template. Note that this template is not yet trnasformed into html. This will come after.

The HashMap call rootMap is used by FreeMarker when it generates the html code from the template. This map must contain all the necessary information used by our macro, like the total number of pages and the current page. This information will be set in the test methods.


Unit testing

Each test will be executed with the following steps:
  1. Set the information used by the pagination macro in the root map.
  2. Make FreeMarker generate html from our template.
  3. Feed HtmlUnit with our html to generate an HtmlPage instance.
  4. Declare some expectations, like the expected anchor tags representing page numbers.
  5. Get the anchor tags present in the pagination div and compare them to the expected ones.
  6. Assert that the current page is present and is not a link.

Set the information used by the pagination macro in the root map

Our simple macro only need to know the number of pages and the current page, so we'll set that information in the root map used by FreeMarker :
private void setPaginationAttributes(int totalPages, int currentPage) {
        rootMap.put("totalPages", totalPages);
        rootMap.put("currentPage", currentPage);
    }

Make FreeMarker generate html from our template and feed HtmlUnit with our html to generate an HtmlPage instance

This is the interesting part. Remember the Template instance we created above ? We'll use it to generate html. The method used for this is process(Map, Writer). We'll output the html content into a StringWriter. The reason is that we want to use the output to generate an HtmlUnit HtmlPage.
private HtmlPage getHtmlPageFromTemplate() throws TemplateException, IOException {

    // Process template
    StringWriter out = new StringWriter();
    template.process(rootMap, out);

    // Get HtmlPage
    WebClient client = new WebClient();
    PageCreator pageCreator = client.getPageCreator();
    WebResponseData responseData = new WebResponseData(out.toString().getBytes(), 200, "OK", new ArrayList<NameValuePair>());
    WebResponse response = new WebResponse(responseData, new URL("http://localhost:8080/"), HttpMethod.GET, 10);
    return (HtmlPage)pageCreator.createPage(response, client.getCurrentWindow());
}
There's a lot going on here, but most of it is dummy data. First, we call the process method of the Template method, and set the ouput into a StringWriter. Then, in order to create an HtmlPage, we setup a WebClient and generate a dummy response containing our html.

Declare some expectations

We want to check the following:
  1. All page numbers are anchors tags, except the current page
  2. The previous and next anchors tags are present when necessary
  3. The number of generated anchor tags equals the number of expected anchor tags
  4. The current page is present, and is not an anchor tag
Here is an example where there are only two pages, and the first one is the current page.
@Test
public void testPagination_twopages_currentisone() throws IOException, TemplateException {
    // Make FreeMarker template
 setPaginationAttributes(2, 1);
    HtmlPage page = getHtmlPageFromTemplate();

    // page links
    List<String> expectedContent = makeExpectedLinks(new String[]{"2"}, false, true); 
    List<HtmlElement> pageLinks = getPaginationLinks(page);
    Assert.assertEquals(expectedContent.size(), pageLinks.size());
    assertContainsLinks(expectedContent, pageLinks);
    assertCurrentPageIs(page, "1");
}
First, we set our FreeMarker information and generate the HtmlPage : two pages, page one is the current page. Then, we declare some expectations. makeExpectedLinks is a method generating the expected anchor links text. We pass an array of page numbers, and whether or not the previous and next links should be present:
private List<String> makeExpectedLinks(String[] links, boolean prev, boolean next) {
    List<String> expectedContent = new ArrayList<String>();
    expectedContent.addAll(Arrays.asList(links));
    if ( prev ) {
        expectedContent.add("Prev");
    }
    if ( next ) {
        expectedContent.add("Next");
    }
            
    return expectedContent;
}
The getPaginationLinks extracts the anchor tags from the html content. This is where HtmlUnit becomes useful. Here, we look for the pagination div and get all anchor elements.
private List<HtmlElement> getPaginationLinks(HtmlPage page) {
    HtmlDivision div = page.getHtmlElementById("pagination");
    return div.getElementsByTagName("a");
}
We make sure that the number of links meets our expectations : Assert.assertEquals(expectedContent.size(), pageLinks.size());. Then the content of the links are checked via the assertContainsLinks method :
private void assertContainsLinks(List<String> expectedLinks, List<HtmlElement> anchors) {
    List<String> anchorsText = new ArrayList<String>();
    for (HtmlElement htmlElement : anchors) {
        anchorsText.add(htmlElement.getTextContent());            
    }
    for( String expectedLink : expectedLinks ) {
        Assert.assertTrue("Expected page link["+expectedLink+"] was not found", anchorsText.contains(expectedLink));
    }
}

The current page is also checked via the assertCurrentPageIs method :
private void assertCurrentPageIs(HtmlPage page, String pageNo) {
 HtmlElement currentPage = getPaginationCurrent(page);
 Assert.assertTrue(!(currentPage instanceof HtmlLink));
    Assert.assertEquals(pageNo, currentPage.getTextContent());
}

From there, it's easy to make other tests for different page numbers and current pages:

@Test
    public void testPagination_twopages_currentistwo() throws IOException, TemplateException {
        // Make FreeMarker template
     setPaginationAttributes(2, 2);
        HtmlPage page = getHtmlPageFromTemplate();

        // page links
        List<String> expectedContent = makeExpectedLinks(new String[]{"1"}, true, false); 
        List<HtmlElement> pageLinks = getPaginationLinks(page);
        Assert.assertEquals(expectedContent.size(), pageLinks.size());
        assertContainsLinks(expectedContent, pageLinks);
        assertCurrentPageIs(page, "2");
    }

    @Test
    public void testPagination_threepages_currentistwo() throws IOException, TemplateException {
        // Make FreeMarker template
     setPaginationAttributes(3, 2);
        HtmlPage page = getHtmlPageFromTemplate();

        // page links
        List<String> expectedContent = makeExpectedLinks(new String[]{"1","3"}, true, true); 
        List<HtmlElement> pageLinks = getPaginationLinks(page);
        Assert.assertEquals(expectedContent.size(), pageLinks.size());
        assertContainsLinks(expectedContent, pageLinks);
        assertCurrentPageIs(page, "2");
    }

Running the tests under Eclipse shows the expected green bar :

Conclusion

Making FreeMarker macros and checking the content via a browser can be annoying and time wasting. The more branches you have in your logic, the more data you have to create. Changing the logic afterwards quickly becomes dangerous, unless you reuse the same data as before to make sure you've not broken the original logic. Unit testing the macro using JUnit, FreeMarker and HtmlUnit allows to prevent these problems.

Wednesday, December 22, 2010

Maven update remote indexes broken in IntelliJ10

I was trying to use Maven with IntelliJ10, but I could not update the indexes of the Maven Central Repository. After wasting a couple of hours, I found out that it was a know bug, IDEA-63043 Maven update remote indexes broken. Why do I keep running into all sorts of problems when using IntelliJ ? I love the interface and all the features it offers, but I wish I could use them at their full potential without having to look for bug reports.

Monday, December 20, 2010

JPA 2.0 with Spring 3.0 bug

I was trying to make a sample application using Spring3 and JPA2 on my new shiny IntelliJ IDEA 10, but stumbled quickly on a problem when I tried to run the application. Spring was complaining that the version attribute in my persistence.xml was wrong. It was set to "2.0", but Spring insisted that it could only be "1.0". I spent hours trying to figure out where the problem was. A wrong XMLSchema being picked up ? A problem with my JPA libraries ? A problem with my JPA provider ? No. The problem was coming from Spring itself (SPR-6711). Upgrading to Spring 3.0.5 fixed it. What a waste of time. It wouldn't have happened if I had used the latest libraries in the first place, instead of letting IDEA automatically import them (I guess version 3.0.0 were downloaded and added to my project).

Monday, November 8, 2010

Book review : Essential GWT: Building for the Web with Google Web Toolkit 2 (Developer's Library)

Essential GWT: Building for the Web with Google Web Toolkit 2 (Developer's Library)


Essential GWT is a misleading title. You could suppose, as I did, that it contains at least GWT basics. It doesn't. There is an overview of GWT, but nothing which will get you started if you're a beginner. So if you're expecting a tutorial about GWT development, widgets, etc..., pass your way. You should read another book or the online tutorial first. This book should have been named something like "Practical GWT Cookbook", as it contains some recipes about common web development topics like file uploading, security and much more.

I felt that the first half of the book was not very well structured. For example, I didn't understand why there is a paragraph on Code Generation in Chapter 4, Working with Browsers. Explanations are illustrated with code samples, but there are either too few, or too much. Too much, like the methods of the JDBC examples. Only one would have been enough. Too few, like the EJB example. Someone who knows EJB will know how to call a bean. Someone who doesn't will need much more information.

There are some annoying errors, especially in the MVP explanation. The same class gets three or four different names, making it very difficult to follow. MVP is an "essential" topic in this book, so it should have been carefully polished.

Nevertheless, the book still contains some interesting tips and techniques. I particularly enjoyed the speed measurement and the testing chapter. But overall I think it is falling short at explaining the essential.

Wednesday, November 3, 2010

Book review : Expert Oracle Database Architecture: Oracle Database Programming 9i, 10g, and 11g Techniques and Solutions, Second Edition

Expert Oracle Database Architecture: Oracle Database Programming 9i, 10g, and 11g Techniques and Solutions, Second Edition

No need to introduce Tom Kyte from "Ask Tom". Now imagine this great Oracle Database expert, sitting in front of you, lecturing about topics like table and index design or file management. Impossible you think ? Think twice. "Expert Oracle Database Architecture: Oracle Database Programming 9i, 10g, and 11g Techniques and Solutions" is Tom Kyte giving you a lecture. The book talks to you. The book forces you to open SQL*Plus and try yourself. This book is full of examples. Reading about databases might not be as fun as reading about programming, but you'll never get bored.

A chapter at the beginning of the book will help you create a proper test environment. Other chapters can be picked up in any order. In each chapter the author introduces some features, explains why you could be interested in them and how to use them. This includes tons of examples, common pitfalls and misuses.

I've been using Oracle Database for years, not as a DBA, but as a software developer. The amount of knowledge I have learnt with this book is tremendous. Tom Kyte has completely changed the way I see the database. Every developer using the Oracle Database should know more than just SQL. If you're one of them, grab this book as soon as possible.

Tuesday, October 5, 2010

Vimperator

Vimperator is a Firefox plugin which turns your browser into a kind of "Vi" for the web. If you like the Vi/Vim editor, you should give it a try. You can do most common tasks using the keyboard. After some experiment, I got addicted to it. Opening a link in the same tab, in a new tab, searching something with one of the installed search engines, filling forms, switching tabs... No need to reach for the mouse.

After installing it, the menu bar, navigation toolbar and bookmark toolbar will be hidden. This may be inconvenient at first, but you can display them again via ":set guioptions+=mTB". You can display the help screen at any time via the ":help" command.

Here is some of the commands I usually use :

  • Open a URL in the current tab : ":open ". Pressing tab while writing the url will show possible results.
  • Open a URL in a new tab : t
  • Open a link in the current tab : "f" followed by the number assigned to the link
  • Open a link in a new tab : "F" followed by the number assigned to the link
  • Click on a button, select a checkbox, move into a textarea... : "f" followed by the number assigned to the element
  • Close a tab : d
  • Open a previously closed tab : u
  • Use one of the search engines to search for some keywords : ":open ". The engine name is displayed in Firefox Search Engines dialog (:dialog searchengines). For example, ":open wikipedia something" will search for something on Wikipedia. To show the results in a new tab, use "t" or ":tabopen" instead of ":open"
  • Move around : h,j,k,l
  • Page Up/Down : Ctrl+f,Ctrl+b
  • Go to beginning/end of page : gg/G
  • Go forward/barckward in history : Ctrl+i/Ctrl+o
  • Go to next/previous tab : gt/gT
  • Reloading a page : r
  • Search in page : / followed by the keyword. After pressing enter, use "n"/"N" to go the next/previous occurence.
  • Copy the current URL in clipboard : y
  • Selecting text within a page : "i" to switch to CARET mode. This allows you to move at the beginning of the text you want to paste. "v" to switch to VISUAL mode to select the text, then "y" to yank it. "/" to search some text, before entering CARET mode is even faster.
  • Invoke gVim from a textarea : Ctrl+i. It helps if you have a long text to write.

There are other ways to browse efficiently. Check the help and find which one suits you best.

Tuesday, September 28, 2010

Book review : Eclipse Rich Client Platform, 2nd Edition

Eclipse Rich Client Platform will teach you how to create professional and redistributable RCP applications. Part I and II look like a tutorial, but the rest of the book goes far beyond. First, you will create a chat application, adding views, editors, actions, help, integrating a third party library... The style is clear, and the progression is logical. API details are left for Part III, where RCP indispensable components are discussed, and that is where the simple tutorial ends.

Part IV introduces more advanced features like p2, dynamic plug-ins, product configurations for various platforms, testing... I found that this part required more thinking than the rest of the book, but it is invaluable if you aim at making a professional application. The last part is a reference about Eclipse and OSGi.

If you plan to make an Eclipse RCP application, if you have some interest in it or in Eclipse plug-ins, if you just like the Eclipse IDE, this book is for you. You'll learn a lot about the Eclipse architecture, and you will learn it the easy way. I also own another book of the Eclipse Series, Eclipse Plug-ins, 3rd Edition, which I enjoyed a lot. I was not disappointed by Eclipse Rich Client Platform, 2nd Edition. I highly recommend it.