A recipe for Web Applications with Java, JSP, Hibernate, Velocity

Joe Desbonnet, 2 September 2008 Revision 0.1

The main ingredients

Programming language: Java Environment: JavaVM. OS - don't care! Hibernate for persistence Velocity for separation of logic and presentation JSP as glue between Core Classes, Velocity templates Hibernate Search for free text search Eclipse IDE, CVS Tomcat Servlet Container, Apache HTTP server Behind Hibernate a MySQL DB (but all popular DBs will work including PostgreSQL, Oracle, MSSQL) API documentation automatically generated with Javadoc YUI for client side user interface (layout, widgets, debuggin etc). Looking into Google GWT for future projects Libaries eg Batik (SVG), HTTPClient, JFreeChart etc

Java - business concerns

Free Formally defined & multiple independent implementations vendor lockin avoidable Very mature - first released 1995, widely deployed for server applications since about 1999 Huge range of development tools for free (eg Eclipse) and for fee (eg IntelliJ) Good availability of skilled programmers Certification programmes Cross platform OS (Linux, Windows, Solaris) and hardware(Intel, AMD, Sparc) Sun Java implementation available released under GPL licence

Java - technical concerns

Object Orientated from the onset - not bolted on as after thought Garbage Collection memory management model Compiled language (source distribution not required) Strong type checking Unicode directly supported API documentation automatically generated with Javadoc Scripting languages extensions available (Jython, JRuby) JavaVM security model facilitates tight control of OS resources Rich selection of APIs including image manipulation, cryptography, Email, XML, barcodes, Bluetooth, IO Fast (comparable to C++ for most applications) Scalable: can utilize multiple cores.

Javadoc
Javadoc is a documentation generator for generating API documentation in HTML format from Java source code. Javadoc is the industry standard for documenting Java classes. For Javadoc to work effectively, a specially formatted comment is placed before any class, method or property.

Javadoc - java comment example

/** * An utility class for Accounts System. A set of static methods * for performing operations on accounts system objects. */ public class AccountsUtils { /** * Get total of invoice * @param invoice The invoice to be totaled * @return total of invoice public static Float getInvoiceTotal (Invoice invoice) { // ... } }

Eclipse IDE
Modules for all apects of Java web app development, including Tomcat module Support for CVS, SVN repositories Smart editor: just hit CTRL-SPACE to see available methods etc Powerful refactoring tools: misspelled a class name? You are no longer stuck with your embarrassing mistake for life. Just open the refactor tool and rename the class, property, method - whatever. The system will rename all references including references in other projects. Documenation will be updated also. Not limited to Java: modules for C, C++, Python, PHP etc

Hibernate - Relational Persistence

Hibernate object/relational persistence and query service. Hibernate lets you develop persistent classes following objectoriented idiom - including association, inheritance, polymorphism, composition, and collections. Hibernate allows you to express queries in its own portable SQL extension (HQL), as well as in native SQL.

Database quirks (eg autoincrement) abstracted away.

Hibernate is a JBoss project (now part of RedHat) and is actively developed. LGPL licence. Commercial support available from RedHat.

Hibernate Search
Hibernate Search brings the power of full text search to Hibernate. Full text search engines like Apache Lucene allow applications to execute free-text search queries. However, it becomes increasingly more difficult to index a more complex object domain model - keeping the index up to date, dealing with the mismatch between the index structure and the domain model, querying mismatches, and so on.

Hibernate Search cont'

Hibernate Search abstracts you from these problems by solving: The structural mismatch: takes care of the object/index translation The duplication mismatch: manages the index, keeps changes synchronized with your database, and optimizes the index access transparently The API mismatch: lets you query the index and retrieve managed objects as any regluar Hibernate query would do Uses Apache Lucene internally. Depending on application needs, Hibernate Search works well in non-clustered and clustered mode, provides synchronous index updates and asynchronous index updates.

Hibernate example - a Java POJO class

public class Invoice { private Long id; private Date date; private String comment; private Account account; private Set<LineItem> lineItems;
// setter / getter methods for each object property public Long getId() { return this.id; } // etc... }

Hibernate example - mapping file (1/2)

<class name="ie.wombat.accounts.Invoice" table="invoice"> <cache usage="nonstrict-read-write" /> <id name="id" column="id" > <generator class="native"/> </id> <property name="date" column="invoice_date" /> <property name="comment" /> (continued ...)

Hibernate example - mapping file (2/2)

<many-to-one name="account" class="ie.wombat.accounts.Account" column="account_id" /> <set name="lineItems" table="invoice_lineitems" > <key column="invoice_id" /> <many-to-many column="lineitem_id" class="ie.wombat.accounts.LineItem" /> </set> </class>

Hibernate example - the db schema

invoice: id int primary key invoice_date date comment varchar(250) account_id int invoice_lineitems invoice_id lineitem_id lineitem id int primary key quantity ... etc

Hibernate example - loading an object

// have invoiceId (Long) from earlier Invoice invoice = (Invoice)hsession.load(Invoice.class,invoiceId);

Hibernate example - deleting an object

// want to delete object 'invoice' hsession.delete(invoice);

Hibernate example - making changes

// Change a property invoice.setComment ("my new comment!"); // Add a new line item LineItem newLineItem = new LineItem (); newLineItem.setQuantity(1); newLineItem.setDescription("My new line item"); newLineItem.setAmount (9.95);

hsession.save (newLineItem); invoice.getLineItems().add(newLineItem);

Hibernate example - queries

// Get all invoices int the last week and order by account name Calendar cal = Calendar.getInstance(); cal.add (Calendar.DAY, -7); Date oneWeekAgo = cal.getTime(); List<Invoice> invoices = hsession .createQuery("from Invoice as invoice " + " where invoice.date > :oneWeekAgo " + " order by invoice.account.name ") .setDate ("oneWeekAgo", oneWeekAgo) .list();

Hibernate HQL (1/2)

HQL (Hibernate Query Language) is similar in many ways to SQL, but unlike SQL it is fully object-oriented, understanding notions like inheritance, polymorphism and association. Example:

select cust from Product prod, Store store inner join store.customers cust where prod.name = 'widget' and store.location.name in ( 'Melbourne', 'Sydney' ) and prod = all elements(cust.currentOrder.lineItems)
... and in SQL this looks something like ...

Hibernate HQL (2/2)

SELECT cust.name, cust.address, cust.phone, cust.id, cust.current_order FROM customers cust, stores store, locations loc, store_customers sc, product prod WHERE prod.name = 'widget' AND store.loc_id = loc.id AND loc.name IN ( 'Melbourne', 'Sydney' ) AND sc.store_id = store.id AND sc.cust_id = cust.id AND prod.id = ALL( SELECT item.prod_id FROM line_items item, orders o

Hibernate Cache
Hibernate supports several caching systems including: EHCache, OSCache, SwarmCache, JBoss Tree Cache. Objects and query results can be cached.

Cache parameters is set in configuration files.

Hibernate Search Prior to Hibernate Search, you would need to add code to your business logic to update Lucene index whenever an object was modified. In response to a search Lucene would return record IDs which then need to be fetched separately. But no more...

Decide what fields you want indexed using Java annotations Perform free text queries using Lucene engine and get Java objects in response Index is automatically updated whenever an domain object is modified or deleted Index structure automatically generated from the domain model

Velocity Template Engine

Facilitates clear separation of logic and presentation. Simple to use: no Java knowledge required Facilitates customization and localization without changing the code base Fully supports unicode Mature (since 2000?) Apache Software Licence http://velocity.apache.org/

Velocity
Template + Context -> render -> output Context is just a bundle of name -> object pairs.

Velocity - Java code example

// Code to get util object with useful formatting methods // ... context.put ("util", util); // Code to get invoice object // ... context.put ("invoice", invoice); templates.merge (context, "/templates/helloworld.vm",out);

Velocity example - template code (1/2)

<html> <head> <title>Invoice #$invoice.getId()</title> </head> <body> <h1>Invoice $invoice.getId()</h1> Date: $util.formatDate($invoice.getDate())

<table> <thead> <tr><td>ID</td><td>Qty</td><td>Description</td><td>Amount </td></td></tr></thead>

Velocity example - template code (2/2)

<tbody> #foreach ($item in $invoice.getLineItems()) <tr> <td>$item.getId()</td> <td>$item.getQuantity()</td> <td>$item.getDescription()</td> <td>$util.formatCurrency($item.getAmount())</td> </tr> #end </body> <tfoot><tr> <td colspan="3" /><td>$util.formatCurrency($total)</td> </tr></tfoot>

Velocity - using #parse

The #parse directive can be used to include another template. This is particularly useful for enforcing consistency within a web application. For example: <html> #parse ("_html_head.vm") <body> #parse ("_page_header.vm") (page body goes here) #parse ("_page_footer.vm") </body> </html>

Velocity - #parse for fragment reuse

<body> <h1>Invoice #$invoice.getId()</h1> Date: $util.formatDate($invoice.getDate())

#parse ("_invoice_lineitem_table.vm")
</body>

JSP - the glue that binds it all together

JSP was originally conceived as a scripting language like ASP or PHP, where Java code is interspersed with HTML. This is very ugly.

Instead I use JSP as a glue that links HTTP GET/POSTs from the use with the core classes and Velocity templates.
Nice things about JSP: Can apply hot fixes to application without need to restart. Most post-commissioning bug fixes involve modifications to JSP and not core classes

JSP example - getting a edit form

invoice-edit.jsp: <%@include file="_header.jsp"%> // Expecting invoice ID in 'id' parameter. Load from DB. Long id = new Long ( request.getParameter("id")); // Put invoice in context context.put ("invoice", hsession.load(Invoice.class, id)); templates.merge ("invoice-edit.vm", context, out);

JSP example - edit form template

<form action="invoice-edit-submit.jsp" method="POST" >

ID: $invoice.getId() <br />

Date: <input type="text" name="date" size="16" value="$util.formatDate($invoice.getDate())" /> <br />

Comment: <input type="text" name="comment" size="32" value="$invoice.getComment()" /> ## etc <input type="submit" value="Save Invoice" /> </form>

JSP example - saving a form

invoice-edit-submit.jsp: <%@include file="_header.jsp"%> // Expecting invoice ID in 'id' parameter. Load from DB. Long id = new Long ( request.getParameter("id")); Invoice invoice = (Invoice)hsession.load(Invoice.class, id) invoice.setComment (request.getParameter("comment")); invoice.setDate (df.parse(request.getParameter("date"))); // etc // Back to invoice list page response.sendRedirect ("invoice-list.jsp");

JSP example - includes

I use JSP includes to perform actions that need to be preformed at the start of every action. These actions usually include: User authentication: has the user logged in? If not then redirect to login form Setup some handy variables: eg "user" - the logged in user object Create a Velocity context object and populate with standard objects, eg "user" - the logged in user, "utils" - a utility object with methods for formatting dates etc By convention prefix files that are not intended to be called directly by the user with an underscore. Each module has its own directory with it's own _header.jsp file.

ImageTable - the image problem

The problem: In most applications text needs to be accompanied by images. Historically relational databases are poor at storing binary blogs such as images. Therefore images are often stored in a file system, yet everything else is stored in a database. Problems association with storing images on filesystem: o two things to backup, not one o scaling more difficult: as file system must be shared or replicated Images are supplied in a wide range of shapes, sizes and format - yet we usually require a fixed dimensions when displaying content. Resizing on the browser yields poor results.

ImageTable - how it works

Images binaries are stored in a database along with image metadata. In addition to the original binary, an intermediate and thumbnail is also stored in the database. A local disk cache is maintained of recently accessed images, this reducing load on database Images are served using a servlet. Information in the path determines the size served: eg http://appserver/img/orig/1234.jpg returns original image, http://appserver/img/tn/1234.jpg returns a thumbnail for the same image. Filters provide alternative renderings of image eg http://appserver/img/200x200/1234.jpg returns a version of the image to fit 200x200px bounding box.

ImageTable - challenges
ImageTable uses Hibernate for image metadata, but goes direct to the JDBC layer to transfer image binary. It was found that both JDBC and Hibernate were not optimized for large binary objects (multiple copies were made in memory). Partial solution was to go direct to JDBC. The quality of ImageMagick scaling operations was found to be far superior to those generated from the Java Imaging API. JMagick provides a Java interface to ImageMagick, but it was found that a forking a process with a regular 'convert' command was just as fast and simpler to implement. May pose some portability issues.

ImageTable - future enhancements

More image filters eg: o Add border o Add watermark o Arbitary SVG transformation (drop shadows etc) Support for more formats (eg BMP, TIFF) Support for vector formas (SVG, AI)

YUI - Yahoo User Interface Library

This is a set of utilities and controls, written in JavaScript, for building richly interactive web applications using techniques such as DOM scripting, DHTML and AJAX. The YUI Library also includes several core CSS resources. All components in the YUI Library have been released as open source under a BSD license and are free for all uses. The YUI library is guaranteed to work identically across all "AGrade" browers. This is a periodically revised list. It currently includes IE 6, 7; Firefox 2, 3; Opera 9.5, Safari 3.1, Google Chrome?

YUI Modules
Grids: A CSS library which facilitates many common layouts. Connection Manager: send asynchronous (AJAX) requests Event utilities: attach events to DOM objects Calendars, Tabs, Menus, Trees, Color picker Rich text editor Charts (uses Flash for rendering) Animation (animate any property, including colors for fade effects etc) Drag & Drop File uploader (uses Flash) Debug logger Javascript compressor and much more...

Google GWT
Google GWT is an alternative approach to developing client side application. Whereas YUI facilitates writing client applications in Javascript, with GWT you write in Java. The Java is then compiled into Javascript. In effect Javascript becomes the "machine code" of the web browser. This has some huge advantages: You can develop in a "real" language and benefit from tools such as Eclipse IDE etc Product optimized Javascript: including only the bits that you need

Other Javascript frameworks

jQuery dojo script.aculo.us ext-js

Monitoring (1/2)

Monitoring (2/2)

Testing
JUnit - unit testing Monitoring app Selenium Manual testing

Other noteworthy tools:

Firebug for Javascript debugging [demo] YSlow for client side performance analysis [demo] JMeter for server performance evaluation Gears (formerly Google Gears). Useful for implementing an "offline" mode in web applications. Enhancements include: o LocalServer (local HTTP server) o WorkerPool Make your web applications more responsive by performing resource-intensive operations asynchronously o local database (with SQL!) o file upload o GPS integration JFreeChart for advanced charting [demo] Apache Derby: embeddable SQL database

Conclusions
The good bits: Productive development environment Predictable deployment environment: Quirks of OS are abstracted away. Automatic documenation with Javadoc Scalable Secure I18N Good skill availability Good tool and support availability Possible pitfalls: Use the right tools for the job. Most apps don't need high powered frameworks such as EJB. Java needs memory space. Don't skimp on memory for production setups: bigger resource footprint than, say PHP.

Possible deployment env spec

Require Sun Java version 1.5 or 6.0 (supplied on distribution CD or downloaded from http://java.sun.com) on approved Linux distribution, Windows Server 2003, Windows XP or Solaris.

Require (insert minimal hw spec: eg 3GHz CPU, 3GB RAM, 30GB HDD etc)

Installations
Galway.Net City Guide CMS: Galway.Net Xposed.ie IrelandAdventures.com

Other apps: Moore Institute Landed Estates Database http://www.landedestates.ie/ (name deleted) camera manufacturer: push software updates to camera software (FN Inc) Galway.Net billing system

More information (1/3)

Hibernate: http://www.hibernate.org Velocity: http://velocity.apache.org Tomcat: http://tomcat.apache.org

YUI: http://developer.yahoo.com/yui/
GWT: http://code.google.com/webtoolkit/

More information (2/3)

Gears http://gears.google.com/ YSlow: http://developer.yahoo.com/yslow/ Eclipse: http://www.eclipse.org/

JMeter: http://jakarta.apache.org/jmeter/
Firebug: http://getfirebug.com/

More information (3/3)

JFreeChart: http://www.jfree.org/ Apache Derby http://db.apache.org/derby/ Java API standards http://jcp.org/en/jsr/all News sites: http://ajaxian.com/ http://www.theserverside.com/ http://www.onjava.com/