7 2 Introduction 2.1 Objectives The main objective of this project is to build a portable and highly configurable embedded web server design based on a MicroBlaze soft processor core implemented on a Spartan-3E FPGA Starter Kit Board. The web server is intended to work as an interactive interface to lower level services in applications such as sensor network monitoring, process diagnostics, web services, remote information storage, just to name a few.
2.2 Motivation More and more commercial devices, such as mobile phones, digital cameras, printers, etc., are capable of connecting to wired or wireless networks in order to perform an increasingly wide range of applications. Most of them make use of the Internet to provide users with entertainment features, communication tools and so on. Other, more sophisticated devices, such as routers, may now be easily configured by means of a web interface. Actually HTTP (Hypertext Transfer Protocol) is one of the most popular communication protocols over TCP / IP networks. People spend a considerable time online, chatting or browsing the Internet. There is an ascending trend to migrate everything online or to be controlled remotely. Embedded systems designed on FPGAs or microcontrollers, by students and experts alike, should also embrace this trend.
8 2.3 Features-in-Brief 2.3.1 Automatic Configuration After an initial board setup, both the FPGA logic and the web server software are automatically configured on every power up or reset. Persistency is provided by the onboard Flash memories. The Ethernet IPv4 interface relies on DHCP (Dynamic Host Configuration Protocol) for connecting to the network (an IP, subnet mask and gateway are automatically acquired). For ease of use, the IP is displayed on the onboard LCD screen.
2.3.2 Model-View-Controller (MVC) Web Server Architecture At core of the embedded web server design lays a highly configurable C/C++ Model-View-Controller framework that provides an automatic management of the HTTP request/response flow. Developers are given access to use and also enhance concepts such as requests, responses, controllers, views, sessions, cookies and many others. Their functionality and behavior are based on the latest web development standards. The possibility of embedding snippets of C/C++ source code into HTML templates is one of the many available features.
9 2.3.3 Session Management and Authentication The web server manages user sessions by generating and keeping track of unique session identifiers which are stored in cookies on the client side. Authentication is one of the most important services that make use of this feature. Session related information storage and customized cookies are some of the other benefits.
2.3.4 File Systems The web server is as capable in providing dynamic content as with delivering static CSS (Cascading Style Sheets) or Javascript files, images, documents, etc., in binary or text format. Serving static content is possible due to the pluggable file system that is implementation-free. The web server framework includes a default memory-based implementation that makes use of the onboard Flash memory.
2.4 Project Summary The embedded web server design aims to provide other developers a portable and highly configurable framework to be used as an interactive web interface in other designs. It assembles both hardware and software components in an easy-to-setup architecture. 2.5 Tools Required 2.5.1 Spartan-3E Starter Kit Board While this project was developed on a Spartan-3E Starter Kit Board, it may be easily ported to any FPGA board that features an Ethernet physical layer interface. Also, at least one RS232 interface is recommended. 10 2.5.2 Xilinx Embedded Development Kit (EDK) The project was developed using EDK 10.1 SP3. The web servers hardware architecture was designed using the Xilinx Platform Studio (XPS), which is a part of the Xilinx Embedded Development Kit. The web server framework software was developed using the XPS Software Development Kit (SDK). 2.5.3 Xilinx ISE The Xilinx ISE was used for designing a couple of Intellectual Property (IP) cores to be used within EDK. One of them is the LCD core that connects to the MicroBlaze processor via the Processor Local Bus (PLB). However, Xilinx ISE is not required if the IP cores do not need to be modified. 2.5.4 Java Runtime Environment (JRE) The JRE is necessary for running a small tool that compiles HTML templates containing snippets of C/C++ code into views that are compatible to the web server framework. 2.6 Design Status The initial objectives of this project have been successfully achieved as may be concluded from the examples section of this document. However, during development, new ideas have emerged and, in some cases, they have been even put in practice. The further development section gathers some of the open ideas worth working on in the future. 3 Background 3.1 Why This Project After more than a couple of years developing VHDL systems and programming microcontroller applications, from simple to complex, moving to the Xilinx Embedded Development Kit was a step forward. It allows me to merge my hardware and software experience into projects that would have been almost impossible to do separately. As mentioned in my motivation at the beginning of this document, developing an embedded web server framework is a very useful initiative at the moment. While researching for previous attempts of the kind, several Xilinx application notes (see the References section) have proven to be very good starting points. One of the best commercial solutions is the web server developed by Treck Inc (www.treck.com). However, it does not address some of the objectives set for this project. Given the partnership with Xilinx it may become available along with the EDK sometime in the future. 3.2 Reference Material 3.2.1 Xilinx Embedded Development Kit The Xilinx Embedded Development Kit (EDK) is a suite of tools and Intellectual Property (IP) that enables the design of complete embedded processor systems for implementation in Xilinx Field Programmable Gate Array (FPGA) devices. The hardware and software components of an embedded design are merged 11 seamlessly into a unified system. The Xilinx Platform Studio (XPS) is the development environment for assembling the embedded hardware architecture, while the Xilinx Platform Studio Software Development Kit (SDK) manages the C/C++ software applications. The SDK is built on the popular Eclipse open-source framework. The EDK also consists of reusable IP cores, drivers, libraries, a GNU C/C++ compiler and debugger, etc. 3.2.1.1 The Base System Builder (BSB) The Base System Builder (BSB) is a wizard that quickly and efficiently establishes a working design which may be then customized. The BSB provides support (peripheral selection, automatic FPGA pinout match, initial test application, etc.) for a number of embedded processor development boards available from Xilinx. In the case of third-party boards additional boards support files need to be registered with the BSB. Usually these are also supplied by the board provider. The diagram below presents the work flow supported by the EDK (www.xilinx.com):
3.2.2 MicroBlaze The MicroBlaze embedded processor soft core is a Reduced Instruction Set Computer (RISC) optimized for implementation in Xilinx FPGAs and is included as a part of the EDK. The key benefits of using a soft processor include configurability to trade between price and performance, faster time to market, easy integration with the FPGA fabric, and avoiding obsolescence. MicroBlaze processor is parameterized to allow selective enabling of additional functionality. A typical MicroBlaze embedded system (www.xilinx.com): 12
3.2.2.1 Processor Local Bus (PLB) The PLB is a component of the IBM CoreConnect architecture. It is a high-performance synchronous bus designed for connecting processors to high-performance peripheral devices. It has mostly replaced the On-Chip Peripheral Bus (OPB) which is now deprecated. The currently available PLB version is 4.6. 3.2.3 Xilinx Micro-Kernel (XMK) Xilinx Micro-Kernel is a collective software system that includes the following components: Standard C Libraries (libc, libm) Standalone Board Support Package (BSP) Xilkernel - An embedded Kernel LibXil MFS - A Memory File System LibXil FATFS - A FAT file system LibXil Drivers - Device drivers for supported peripherals LibXil Flash - A library that provides read/write/erase/lock/unlock and device specific functionalities for parallel flash devices LibXil Isf - In-System-Flash library that supports the Xilinx In System Flash hardware XMK may be configured using the Software Platform Settings dialog in EDK. 3.2.4 Wireless Sensor Network The wireless sensor network was developed by Mihai Schitcu, a former masters degree student at the Technical University of Cluj-Napoca. The network is composed of a control base station and several remote sensor stations measuring parameters such as temperature and relative humidity. 13
All stations are designed on microcontroller architectures and operate with Nordic Semiconductor transceivers. The base station periodically updates the list of parameters received from the sensor stations. It also implements a simple command-based state machine that other devices, such as the embedded web server, can control in order to retrieve the latest parameter values. 4 Design 4.1 Features and Specifications The embedded web server is a unified system design based on a MicroBlaze soft processor architecture implemented on an FPGA board. The board must feature an Ethernet physical layer interface (OSI Layer 1 - Physical Layer). OSI Layer 2 (Data Link) is managed by a hardware MAC controller while the rest of the networking stack up to the HTTP protocol is software-oriented. The web server functionality is provided by a portable and highly configurable thread-based model-view-controller framework supported by an operating system kernel. 4.2 Design Overview The following paragraphs provide short overviews to both the hardware and software components of the embedded web server design. 4.2.1 System Architecture The embedded system design is assembled around a MicroBlaze processor by means of two buses: the Local Memory Bus (LMB) and the Processor Local Bus (PLB) (see the EDK-generated block diagram in Appendix 1). The processor has access to 16KB of Block RAM which store the initial bootloader, while using the onboard 64MB of DDR SRAM to run the web server software. An interrupt controller manages interrupts from the system timer and Ethernet controller. The two RS232 controllers handle the standard input / output streams and also the communication with a wireless sensor network. The onboard 16MB StrataFlash stores the application code and the memory file system when the board is powered off. Other IP cores include some GPIOs (General Purpose Input Output) for handling leds, switches and buttons. The following is a simplified conceptual version of the block diagram in Appendix 1: 14
4.2.2 Operating System The Xilinx Micro-Kernel (Xilkernel) is a small light-weight and easy to use kernel, providing features like scheduling, threads, IPC and synchronization with a POSIX subset interface:
(Xilinx Micro-Kernel documentation) 4.2.3 Network The Spartan-3E Starter Kit Board provides a 10 / 100 Mb Ethernet physical layer (PHY) interface. The Ethernet Media Access Controller (EMAC) is implemented in the FPGA logic by means of the IEEE 802.3 Std.-compliant XPS Ethernet Lite MAC (xps_ethernetlite) IP core. TCP/IP protocols are handled by the Lightweight IP (lwIP) open source networking stack. Finally, the HTTP protocol is implemented within the web server framework and is based on socket connections. The use of sockets requires a simple multithreaded kernel (like Xilkernel). 15 4.2.4 Web Server Framework The web server framework provides all the necessary components and concepts to easily configure and deploy a web server instance as well as develop all kinds of web applications with just a few lines of code. 4.2.5 File System The web server framework makes use of an interface to a file system in order to store and respond with static content (HTML, CSS, Javascript, etc.). A default Xilinx Memory File System (MFS) implementation is provided, but any other implementations bound to the required interface are pluggable into the framework. 4.3 Detailed Design Description 4.3.1 Base System Builder The easiest way to assemble the system is to use the Base System Builder (BSB). It allows you to easily set up a working design to start working on. Of course, components may be added and removed later, but that requires more effort due to the manual bus connection, port and address configurations and also the UCF (User Constraints File) entries:
4.3.2 System Architecture Most of the hardware components are assembled with their default configuration provided by the EDK. However, some particular constraints may apply. 16 4.3.2.1 MicroBlaze The processors reference clock frequency as well as the bus (PLB) clock frequency is 50MHz. It includes an on-chip hardware debug module and 16kB of local BRAM memory. The BRAM is initiated with a bootloader application that copies the rest of the executable code from Flash to the RAM memory and starts its execution. Caching, 2kB for both instructions and data, is enabled for a better performance. A floating point unit (FPU) is not required. 4.3.2.2 Multi-Port Memory Controller (MMPC) The multi-port memory controller handles the onboard 64MB of RAM memory. This is the systems main memory after the BRAM bootloader loads the executable code from the Flash memory. 4.3.2.3 XPS Multi-Channel External Memory Controller (XPS MCH EMC) The multi-channel external memory controller handles the onboard 16MB of Intel Flash PROM. The flash memory persists the web servers executable code during power off, but also holds the static content file system. 4.3.2.4 XPS UART Lite There are two UART cores on the FPGA fabric. One is used for providing the standard input / output and runs at 115200 bps with 8 data bits and no parity:
The second UART is used for communication with the wireless sensor network base station. It also runs at 115200 bps with 8 data bits, but requires even parity. 4.3.2.5 XPS Ethernet Lite MAC The MAC IP core implements the OSI Layer 2 (Data Link) providing support for higher level protocols such as TCP/IP. It may run at up to 100 Mbps in full duplex. 4.3.2.6 XPS GPIO The system also provides three GPIO cores that handle terminals such as the onboard leds, buttons and switches. 17 4.3.2.7 The LCD IP Core 4.3.2.7.1 Overview The Spartan-3E Starter Kit features a two-line by 16-character liquid crystal display (LCD). The LCD content is controlled by a 4-bit data interface. Although it is, somehow, difficult to set up initially, it becomes useful for displaying information in standard ASCII characters as well as other custom symbols. 4.3.2.7.2 Architecture Controlling the LCD is based on a predefined 8-bit command set. Due to the 4-bit data interface commands are sent in two steps. Taking also into consideration the interface initialization and execution delays, the control sequence needs to comply with strict timings. This is achieved by using a system of three synchronized state machines runing at the PLB bus clock (50MHz). One state machine handles the initialization sequence, one executes the commands and the last is used for synchronization. The LCD entity (lcd.vhd) provides the following external signals: Signal Name Mode Description clk in The clock signal sourced from the PLB bus (50MHz) reset in The reset signal SF_D(3:0) out LCD 4-bit data interface LCD_E out LCD enable signal (0 disable, 1 enable read/wr. ops.) LCD_RS out LCD register select LCD_RW out LCD read/write control (0 write to LCD, 1 read from LCD) start in Pulse signal that starts the command execution char(7:0) in The 8-bit character / command to write / execute com in 0 in the case of a command, 1 if char is a character 4.3.2.7.3 Bus connection In order for an LCD instance to be used in EDK, it needs to be connected to the PLB. Using the XPS Create and Import Peripheral Wizard one may simply generate a bus-compliant wrapper for the custom logic. 4.3.3 Software Libraries 4.3.3.1 Xilkernel Xilkernel is initiated by calling the xilkernel_main() method. First, it spawns all the user-defined static threads set in the Software Platform Settings (at least one such thread must exist, commonly named main_thread). The main thread then initiates lwIP and spawns another network configuration thread, named network_thread. This sets up the network interface (IP, subnet mark, gateway) and then spawns 18 a predefined thread, xemacif_input_thread, which actually handles the TCP/IP traffic to be stacked by lwIP. Threads are also used during the operation of the web server as each request gets to be processed in its own thread. Xilkernel is enabled and configured in the Software Platform Settings dialog:
Note that applications need to be linked with the lxilkernel flag (see screen-shots in section 3.3.2.4). 4.3.3.2 Lightweight IP (lwIP) 4.3.3.2.1 Overview Lightweight IP (lwIP, http://www.sics.se/~adam/lwip/) is an open source TCP/IP networking stack for use with embedded systems. Xilinx Embedded Development Kit (EDK) provides different lwIP library versions customized to run on embedded systems based on either a MicroBlaze or a PowerPC processor.
Lightweight IP provides support for the following protocols: Internet Protocol (IP) Internet Control Message Protocol (ICMP) User Datagram Protocol (UDP) Transmission Control Protocol (TCP) Address Resolution Protocol (ARP) Dynamic Host Configuration Protocol (DHCP)
19 The latest lwIP library deployed with EDK 10.1 SP3 is a port (version 1.00.a) of lwIP version 1.3.0. Being a standalone networking stack, lwIP has no operating system dependencies, although it may be used together with an OS / kernel (such as Xilkernel).
Developers may choose between two APIs: a raw API that provides access to the core of the stack (optimized for performance) and a socket API that provides a BSD (Berkeley Software Distribution, sometimes called Berkeley Unix) sockets style interface (optimized for ease of use). Although the latter is easier to use, it is inefficient both in performance and memory requirements due to the significant overhead. However, its performance is scheduled to be improved in subsequent releases of the library. More, one of its greatest advantages, which is also a requirement for the web server, is portability. All in all, the socket API has been chosen to provide the default implementation of the web server framework. 4.3.3.2.2 Functions and Features The Xilinx adapters provide some helper functions to simplify the use of the socket API: void lwip_init()
This is a single initialization function that replaces specific calls to initialize stats, memory, etc.
struct netif* xemac_add(struct netif*netif, struct ip_addr* ipaddr, struct ip_addr* netmask, struct ip_addr* gw, unsigned char* mac_ethernet_address unsigned mac_baseaddr) This function provides a unified interface to add any Xilinx EMAC core netif lwIP network interface structure ipaddr Holds the (static) IP address (e.g. 192.168.1.101). Set to 0 if using DHCP netmask Holds the (static) netmask address (e.g. 255.255.255.0). Set to 0 if using DHCP gw Holds the (static) gateway (e.g. 192.168.1.254). Set to 0 if using DHCP mac_ethernet_ address The MAC address (e.g. {0x00, 0x0a, 0x35, 0x00, 0x01, 0x02}) mac_baseaddr The base address of the EMAC core
void xemacif_input_thread(struct netif*netif)
In the socket mode, this thread must be launched separately in order for the application to receive the input packets. It receives data processed by the interrupt handlers and passes them to the lwIP tcpip_thread netif lwIP network interface structure 20
4.3.3.2.3 Hardware Configuration 4.3.3.2.3.1 Ethernet Media Access Controller (EMAC) Lightweight IP requires an Ethernet Media Access Controller (EMAC) core. It currently supports the XPS Ethernet Lite Media Access Controller (xps_ethernetlite) or XPS LocalLink Tri-mode Ethernet MAC (xps_II_temac) cores. The web server system uses a version 2.00b instance of the first (see the EDK- generated block diagram in Appendix 1). 4.3.3.2.3.2 Timer To maintain TCP timers, lwIP requires that certain functions are called at periodic intervals by the application. The web server system uses a version 1.00a instance of the XPS Timer / Counter. 4.3.3.2.4 Library Configuration Enabling lwIP, choosing the socket API along with tweaking with the many configuration parameters is easily done within the Software Platform Settings dialog:
Also note that applications need to be linked with the llwip4 flag:
21 4.3.3.2.5 Network Configuration 4.3.3.2.5.1 Dynamic Host Configuration Protocol (DHCP) DHCP is a network application protocol used by network devices (DHCP clients) to obtain configuration information for operation in an Internet Protocol network. This protocol reduces system administration workload, allowing devices to be added to the network with little or no manual intervention: network parameters are assigned to network devices from one or more DHCP servers. The web server is configured to act as a DHCP client in order to acquire the necessary network settings (IP address, subnet mask and gateway). This feature is handled by the network thread (see section 3.3.1 Xilkernel). Some of the most important steps in configuring the network interface are briefly explained below: Define a MAC address (00-0A-35-00-01-02): unsigned char mac_ethernet_address[] = { 0x00, 0x0a, 0x35, 0x00, 0x01, 0x02 }; Clear the network settings: IP4_ADDR(&ipaddr, 0, 0, 0, 0); IP4_ADDR(&netmask, 0, 0, 0, 0); IP4_ADDR(&gw, 0, 0, 0, 0); Set the network interface: xemac_add(netif, &ipaddr, &netmask, &gw, mac_ethernet_address, EMAC_BASEADDR) Start the xemacif_input_thread: sys_thread_new("xemacif_input_thread", (void(*)(void*)) xemacif_input_thread, netif, THREAD_STACKSIZE, DEFAULT_THREAD_PRIO); Start DHCP (client): dhcp_start(netif) The main Xilkernel thread (main_thread) waits until the network interface is set up and then proceeds to starting the server. 4.3.3.3 LibXil Memory File System (MFS) 4.3.3.3.1 Overview 22 LibXil MFS library provides a simple memory based file system, which allows easy access to data organized in files and folder hierarchies. The file system may reside in RAM, ROM or Flash memories and may be accessed directly or through MFS methods (operations are based on file and folder handles). MFS is easily enabled and configured within the Software Platform Settings dialog:
4.3.3.3.2 Functions and Features The following tables describe some most useful functions provided by the MFS library and used for the default implementation of the web server framework file system. Please consult the referenced documentation for more detailed information. void mfs_init_genimage(int numbytes, char* address, int init_type)
Initialize the memory file system (MFS) with an image generated by mfsgen (see the next section, 3.3.3.3 File System Images) numbytes The size, in bytes, of the file system image generated by mfsgen, plus an additional 4 bytes used by the library for internal usage (e.g. 798720 + 4 = 798724) address The starting address of the file system image in memory (e.g. 0x89F00000, while 0x89000000 is the starting address of the Flash memory) init_type Either MFSINIT_IMAGE or MFSINIT_ROM_IMAGE. The second represents a read-only file system. MFSINIT_IMAGE is used by default. However, note that, currently, the library does not support writing to file systems in Flash memories.
int mfs_exists_file(char* path) Return 0 if the specified path does not exist, 1 if it points to a file and 2 if it points to a folder path The path that points to a file or folder (e.g. css/layout.css)
int mfs_file_open(char* filepath, int mode) Opens the specified file 23 Return The index of the file in the array of open files, on success, -1 on failure filepath The path to the file to open mode One of MODE_READ, MODE_CREATE or MODE_WRITE. MODE_READ is used by default
long mfs_file_lseek(int fd, long offset, int whence) Return Returns offset from the beginning of the file to the current location on success. Typically used to retrieve the size of a file (e.g. mfs_file_lseek(, 0, MFS_SEEK_END)) fd The file descriptor returned by mfs_file_open offset The number of bytes to seek whence The position where to start seeking: one of MFS_SEE_END (the end of the file), MFS_SEEK_CURR (the current position) or MFS_SEEK_SET (the start of the file)
int mfs_file_read(int fd, char* buf, int buflen) Reads a number of bytes in the buffer starting from the current position Return The number of bytes read, on success, 0 on failure fd The file descriptor returned by mfs_file_open buf The destination buffer buflen The number of bytes to read
4.3.3.3.3 File System Images (mfsgen) In order to store predefined files and folders in memory, they must be converted into a file system image. This may be done by using the mfsgen utility within the EDK shell. It can merge an entire folder hierarchy on the development system into a MFS-ready image. The image may be downloaded to the on-board memories to be used as a pre-loaded file system.
The command syntax for using mfsgen is:
mfsgen -{c filelist| t | x} vsb num_blocks f mfs_filename
The best practice is to copy the folder hierarchy into a root folder, then navigate to the root folder, within the EDK shell, and run:
mfsgen -cvbfs ../image.mfs 1500 * 24
Image.mfs is the name of the new image, created outside of the root directory and 1500 is the number of blocks to be allocated. The * wildcard instructs that all files and folders in the root directory should be included.
Below is an example of the output generated by mfsgen:
Images may be downloaded to the on-board memories using XMD (Xilinx MicroProcessor Debugger):
dow data image.mfs 0x89F00000
0x89F00000 is the starting memory address where to copy the image to. Note that this location must not overlap with the executable code or with other previously reserved sections. When downloading the image to a flash memory, the best practice is to use the Program Flash Memory dialog in XPS (take great care when computing the Flash memory offset):
4.3.4 Web Server Framework 4.3.4.1 Objectives and Constraints While designing the web server hardware architecture and configuring the software platform settings is relatively straight-forward, developing a full featured embedded web server, similar to the ones 25 deployed on desktop computers / servers (see Apache Tomcat or Microsoft Internet Information Services, IIS) is a challenge. More, the ability to provide support for concepts like Model-View-Controller (MVC) should also be taken into consideration (see the Spring MVC framework). Finally, embedding C/C++ source code in HTML pages and using templates (see JSP/JSTL) would be a very useful feature.
In order for the web framework to be easily ported to any environment it was developed with as little dependencies as possible, though not entirely standalone. More, the source code has been well commented and documented hereby as to provide useful hints to anyone interested in using or contributing to its success.
The web server framework was developed mostly in C++. The power of object-oriented programming, although an overhead from the point of view of performance and memory, is undisputable. 4.3.4.2 Conceptual Diagram Below is a partial representation of how the web server framework maps onto the objectives indentified in the previous section.
4.3.4.3 Structure The portability requirement isolates most of the web server framework into a standalone (abstract) core wrapped by the rest of the components. The latter require certain external dependencies. The following paragraphs cover most of the web server framework starting with the core components. Most of these are basic C++ classes (see Appendix 2 for a list of all the source files). 26 4.3.4.3.1 Request This is a wrapper class for the raw HTTP request string received by the web server along the socket connection from the web client (browser / host). The following is a table view of a raw request to a page on www.digilentinc.com:
GET /nav1index.cfm?NavTop=2 HTTP/1.1 User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.59 Safari/525.19 Referrer: http://www.digilentinc.com/ Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,i mage/png,*/*;q=0.5 Accept-Encoding: gzip,deflate,bzip2,sdch Accept-Language: en-US, en Accept-Charset: ISO-8859-1,*,utf-8 Host: www.digilentinc.com Connection: Keep-Alive
Each row in the above table is a line in the raw request, always followed by a carriage return and a line feed (pseudocode <CR><LF> or \r\n in most computer programming languages, including C). The request ends with an empty line (it contains only the \r\n characters). Each line in the request, except for the first, has the form key: value. These tuples are called headers and mostly consist of predefined keys and values.
A web server that receives such a request will try to interpret it, perform some actions (commonly referred to as business logic) and respond accordingly. For example, the Connection: Keep-Alive header introduces a persistent connection which is a HTTP/1.1 specification. This connection type would allow the client to submit several requests along the same connection and wait for the responses to arrive in the same order. Currently this feature is not yet implemented by the web server and thus the equivalent connection response header is set to Close, instructing clients to issue each request on a different connection.
The first line of the request, also called the request line, indicates the request method (GET), the path to the requested resource (/nav1index.cfm?NavTop=2) and the version of the HTTP protocol (1.1). Note that before HTTP/1.0 the request line would only include the path to the resource.
There are currently eight request methods (sometimes called verbs), which indicate the desired action to be performed on the identified resource. By far the most used methods are GET and POST. GET requests the representation of the identified resource, for example, the HTML content of a web page, a CSS file (Cascading Style Sheet), JPEG image, etc. GET should not be used for operations that cause side- effects i.e. perform business logic on the server side. This is due to the fact that robots and crawlers, such as the ones used by search engines, may perform arbitrary requests.
A POST request submits data (usually a HTML form) to be processed by the specified resource. The data is included in the request body (last after the request line, header and the empty line). For example, during a web site registration the users information is sent to the web server where it is stored in a database.
27 The path to the requested resource (/nav1index.cfm?NavTop=2) may also contain request parameters. These are separated from the rest of the path by a question mark (?). The above example represents a request to /nav1index.cfm with one parameter, NavTop, having the value 2. Several request parameters are separated by & (e.g. /Products/Detail.cfm? NavTop=2 & NavSub=473 & Prod=PMOD-CLS).
Cookies are pieces of information that web sites store on the client-side that are used to keep track of sessions, user activity, etc. They are attached and set during HTTP responses. Subsequent requests to the same domain also include the previously stored cookies. Below is a table representation of the cookie request header:
In particular, the above example shows how the web server stores the users session identifier using a cookie. Every time the user makes a new request, the server knows that it is the same user due to the session ID. Cookies expire after a timeout period or, usually, when the browser is closed. Most browsers allow users to manually delete cookies or to view their content. There is also a limit as to how many cookies a browser may store on the client machine (approx. 300 in total, 20 per server), and their size (4kB).
During instantiation, a Request object parses the relevant information from the raw string request in order to provide easy access to the short and full request paths, the parameters and their values, the attached cookies, etc.
The following tables describe some of the most important methods in the Request class:
Request(char* requestBody) Constructor The constructors only parameter is the raw HTTP request string. It is parsed so that several pieces of information may be easily retrieved with the help of the following methods requestBody The raw request string
char* getFullPath() Return The full path to the requested resource, including parameters (e.g. /nav1index.cfm?NavTop=2)
char* getPath() Return The path to the requested resource, excluding parameters (e.g. /nav1index.cfm, while the full path is /nav1index.cfm?NavTop=2)
char* getParameter(const char* name) 28 Return The string representation of the parameter with the given name (e.g. given /nav1index.cfm?NavTop=2 as the full path, the value of parameter NavTop is 2) name The name of the request parameter
list<Cookie*> getCookies() Return The list (C++ library) of (references to) cookies attached to this request
Session* getSession() Return The session this request belongs to
RequestType getType(); Return The request method: RequestType::GET, POST or UNKNOWN
4.3.4.3.2 Response This is an abstract wrapper for the HTTP response string returned by the web server back to the web client (browser). It holds an internal buffer which is flushed when reaching its capacity. The flushing mechanism, i.e. the actual delivery of content to the web client, is only defined as an abstract method. Developers may subclass the abstract Response class and provide an implementation for this method. However, the default implementation, the SocketResponse class, is based on lwIP sockets. The raw response string is similar in structure to the raw request string: HTTP/1.1 200 OK Connection: close Date: Mon, 27 Apr 2009 16:16:10 GMT Server: Microsoft-IIS/6.0
Content-Type: text/html; charset=UTF-8
Each line of the response header is followed by a carriage return and a line feed. Except for the first, each line is a header having the form key: value. The response headers end with an empty line.
Response() (1) Response(Request* request) (2) 29 Constructors There are two overloaded constructors. The second constructor takes as parameter a reference to the request object which is used to pre-set parameters like the content type (given the file extension), etc. request (2) Reference to the request object
void setContentType(char* type)
Sets the type of content delivered to the client. It is converted into the corresponding response header (Content-Type) type The content type. Shortcuts like css, htm, html and some other common file extensions (see the class implementation for a complete list) are properly converted (e.g. css become text/css, htm and html become text/html, etc.)
void setContentLength(int length)
Sets the response body length, i.e. the actual data. It is converted to the corresponding response header (Content-Length) length The size in bytes of the response body
void addCookie(Cookie* cookie)
Attaches the given cookie to this response. It is converted to the corresponding response header (Set-Cookie) cookie A reference to the cookie to attach
void append(const char* content); (1) void append(const char* content, int length); (2) void append(string* content); (3) These are three overloaded methods that deliver content on the response stream content (1) Pointer to a character string terminated in a null character: \0 (2) Pointer to a character string (3) Pointer to a string object (C++ library) length (2) The number of characters to append from the given character string 30
void setCode(int code) Set the response code (e.g. 200 on success, 404 on missing resource, etc.) code The three-digit code
Cookies may be attached to requests by means of the Set-Cookie header:
The request contents are not bound to any format. Though usually HTML, it is also common to use Javascript Object Notation (JSON) when performing AJAX (Asynchronous Javascript + XML) operations. Some applications may handle XML (eXtensible Markup Language), like in the case of web services. 4.3.4.3.3 Controller This is an abstract core component of the web server framework being one of the most important Model-View-Controller concepts. After a request is received it is passed to a particular controller. Controllers are initially mapped to certain paths. Remember that the (short) request path is retrieved by calling getPath() on the Request object. The web server tries to match the request path to a registered controller and then call the Controllers handleRequest(Request*, Response*) method: void handleRequest(Request* request, Response* response) The web server expects implementations to generate the response request Pointer to the request object response Pointer to the response object
This method is defined as abstract in the Controller (super) class, thus it needs to be implemented by subclasses. Controllers also receive a reference to the Response object in order to be able to generate the response. Content may be delivered to the web client directly by the controller, but this practice is only recommended for short replies (e.g. AJAX calls). The best practice, however, is for the controller to perform its custom business logic, build a data model and pass it to a View object for rendering. Views are, thus, intended to render whole HTML (template) pages by filling placeholders with data from the model. If the request path does not match any controller, the web server will send the request to a default controller. By default, this controller is an instance of the FileSystemController. It interprets the request 31 path as a file path in the web servers file system. If no such file is found, a 404 not found reply is sent back to the web client. 4.3.4.3.4 View This abstract core component defines a single method: renderResponse(Request*, Response*, void*): void renderResponse(Request* request, Response* response, void* model)
The web server expects implementations to render the response, also making use of the model request Pointer to the request object response Pointer to the response object model Void pointer to a data structure representing the model
It is similar to the handleRequest method (signature) in the Controller class, but it receives an extra parameter, a void pointer to a data structure which is the data model previously built by the controller. View implementations may convert the void pointer to any particular data structure (primitive, C structure, class object, etc.). View implementations are not designed to be written manually as it would take too much time to render whole HTML pages. Instead, they are automatically generated by a portable Java-based tool that reads HTML pages and converts then into view implementations. Each HTML line in the original page is converted into response->append([line]) statements. This is only useful when the HTML page is a template that needs to be populated with some data or when the HTML page contains snippets of C/C++ code embedded within <% %> tags. In this case, the C/C++ code inside the snippets is copied to the corresponding View implementation as it is. However, if the HTML page is static, it is better to be kept in the file system in order to be faster processed by the FileSystemController. 4.3.4.3.5 FileSystem This abstract class is mostly used as an interface (it only defines some methods without any implementation). As the web server uses a reference to a FileSystem object, any implementation may be plugged into the web server. However, the web server framework comes with a default implementation based on the Xilinx Memory File System (MFS) library: the MemoryFileSystem. The following tables briefly describe the abstract methods defined in the FileSystem class: 32 int openFile(char* filepath, int mode) Opens a file in the file system by providing its path and opening mode Return A file descriptor to be used in association with other file system methods filepath The path to the file to open (e.g. css/layout.css) mode FileSystem::Read is currently the only expected value
int getFileSize(int fd) Return the size, in bytes, of the specified file fd The file descriptor retrieved when opening the file
int readFile(int fd, char* buffer, int count) Reads a number of bytes from the specified file into a buffer from the current position Return The number of bytes read on success or 0 on failure fd The file descriptor retrieved when opening the file buffer The buffer to hold the bytes count The number of bytes to read into the buffer
bool existsFile(char* filepath) Return Return true if the file system contains the file specified by the given path, false otherwise filepath The path to the file
int closeFile(int fd) Close a previously opened file Return 1 on success, 0 on failure 33 fd The file descriptor retrieved when opening the file
4.3.4.3.6 Cookie This class is a wrapper for HTTP cookies and holds the most basic information: a key and a value. It provides the appropriate constructors, getters and setters to set the key and value for a cookie. During construction, Request objects automatically parses the cookie header so that a list of Cookie references is always accessible by calling the getCookies() method. The web server will automatically scan through the cookies looking for the session identifier: csessionid. Cookies may also be attached to Response objects to be included in the response header. Again, the web server makes sure to set the session identifier cookie automatically. However, cookies may also be added manually by calling the addCookie(Cookie*) method in the Response class. 4.3.4.3.7 Session The web server keeps track of users by means of sessions. Each (anonymous) user has an associated session stored in memory and handled by a session manager. Each session is uniquely identified by a 128-bit ID (16 bytes, 32 hexadecimal characters) stored in a cookie on the users machine. The diagram below depicts the association of the requests issued by the same web client / user to a unique session:
Each Request object has access to a corresponding Session object. If the request does not hold valid session information, a new Session is spawn by the session manager and plugged into the Request object. Otherwise the session manager will retrieve the old Session. One of the most common uses for sessions is to store user-related information in the form of session attributes. These are key / value pairs stored in an internal map (C++ library). The key is a character string (attribute name) while the value is a void pointer (void*). This allows developers to attach any type of data structure to a name on the session. 34 Authentication and authorization are good examples of where this is very useful. After a user successfully enters his credentials via the login page of a web site, the proper security information is stored on his session allowing him access to any authorized resources without prompting for usernames and passwords at each request. 4.3.4.3.8 SessionManager This class is intended to group session-related logic into a separate component to be used remotely by the web server. It generates new sessions and unique session identifiers on request and stores them in a map to be retrieved when needed by the web server. 4.3.4.3.9 SocketResponse This is a subclass of Response which provides a lwIP socket-based implementation of the abstract flushBuffer() method. Its constructors require an extra parameter: the socket identifier. The implementation only needs to call lwip_write and then clear the buffer: lwip_write(sd, buffer->c_str(),buffer->size()); buffer->clear();
4.3.4.3.10 MemoryFileSystem This is a subclass of the FileSystem class that implements the required abstract methods by making use of methods in Xilinxs Memory File System library. 4.3.4.3.11 SocketServer Instances of this class are stand-alone web servers, bound to the same IP address, but on different ports. In most cases it is sufficient to deploy a single web server on the default port 80. The following tables briefly describe the most important methods in the SocketServer class that are used to configure and deploy a web server instance: void setController(const char* url, Controller* controller);
Maps a request path to the given controller. Any request matching the path is handled by this controller url The path to which the controller is bound to (e.g. /web/temp) controller The reference to the controller which will handle all requests matching the given path (e.g. a request to /web/temp?action=refresh with be a valid match) Controller* getController(const char* url) 35
Sets the controller that will handle requests that do not match any of the registered controllers defaultController A reference to a controller
void setSessionManager(SessionManager* sessionManager) Plugs in a reference to a session manager sessionManager A reference to a session manager
SessionManager* getSessionManager(); Return The session manager handling the sessions registered with this web server instance
void setFileSystem(FileSystem* fileSystem) Plugs in a reference to a file system (implementation) fileSystem A reference to a file system
Returns a reference to the controller registered to handle requests that match the given path. It is mostly used internally by the web server Return A reference to the controller registered to handle requests that map the given path or NULL if no such controller has been registered url The path that needs to be mapped to a controller (e.g. /web/temp?action=request) Controller* getDefaultController(); Return Returns a reference to the default controller registered to handle requests that have not matched any of the registered controllers. By default this is an instance of the FileSystemController. This method is mostly used internally by the web server 36
int start() Return Initiates the web server and returns 1 on success, 0 on failure
The following paragraphs exemplify the use of the above methods in a common attempt to configure and deploy a simple web server. Initiate the file system: FileSystem* fileSystem = new MemoryFileSystem(); fileSystem->init(); Define the controllers: Controller* ledsController = new LedsController(); Controller* testController = new TestController(); Controller* tempController = new TemperatureController(); Controller* writeController = new WriteController(fileSystem); Controller* switchController = new SwitchController(); Controller* clickController = new ClickController(); Controller* mapController = new MapController(); Define the default controller: Controller* defaultController = new FileSystemController(fileSystem); Define the session manager: SessionManager* sessionManager = new SessionManager(); Create a web server instance: SocketServer* server = new SocketServer(); Bind everything to the web server: server->setFileSystem(fileSystem);
server->setDefaultController(defaultController); server->setSessionManager(sessionManager); Start the web server: server->start(); Starting from this point the web server is ready to receive requests and deliver responses. 4.3.4.3.12 FileSystemController This particular controller interprets request paths as file system paths. It has an internal reference to a FileSystem object which is queried for the existence of the file specified in the request path. For example, in the case of a request to /css/layout.css, the controller tries to find a file named layout.css in the folder css at the top of the file system hierarchy. If the file is found, its content is delivered to the web client: if (fileSystem->existsFile(request->getPath() + 1)) { ... response->append(fcontent, fsize); ... } Otherwise, a 404 not found error is returned. 4.3.4.3.13 SensorNetwork This class is an interface to the wireless sensor network. It retrieves the latest parameter values from the control base station and provides them by means of several public methods. For example, getTemperature(0) returns the value of the first temperature sensor (there are 2). 4.3.5 C/C++ Scriptlets The web server framework provides the ability to embed C/C++ source code into HTML templates in order to fill placeholders or to manipulate the content. For example, the conditional structure below will insert a certain input button if the element at index i in the leds_array (an array stored in the model of the view) is positive. Otherwise, some other HTML content may be inserted at that location in the page. 38
4.3.5.1 Compilation Similar to the case of Java Server Pages (JPS) that are compiled into servlets, the HTML templates that contain C/C++ code snippets are compiled into implementations of the View class that override the renderResponse(Request*, Response*, void*) method. Basically, all the pieces (lines) of HTML code are converted into response->append([HTML]) calls, while the C/C++ code is copies as it is. The compilation result of the earlier example is printed below:
Of course, other C language structures such as for, while or do-while loops may be used to generate repetitive content like table rows and cells. 5 References 1. Spartan-3E FPGA Starter Kit Board User Guide (http://www.xilinx.com/support/documentation/boards_and_kits/ug230.pdf) 2. EDK Concepts, Tools, and Techniques (A Hands-On Guide to Effective Embedded System Design, http://www.xilinx.com/support/documentation/sw_manuals/edk_ctt.pdf) 3. MicroBlaze Processor Reference Guide (EDK 10.1 SP3*) 4. Xilinx Application Note 1026 (XAPP1026: LightWeight IP, lwIP, Application Examples by Siva Velusamy, http://www.xilinx.com/support/documentation/application_notes/xapp1026.pdf) 5. OS and Libraries Document Collection (EDK 10.1 SP3*, July 30, 2008) 6. lwIP 1.3.0 Library (v1.00.a) Documentation (EDK 10.1 SP3*) 7. LibXil Memory File System (MFS) Documentation (EDK 10.1 SP3*) 8. PLBv4.6 Slave Single Documentation (EDK 10.1 SP3*) 9. The Spring MVC Framework Reference Documentation (Chapter 13 of the Spring Framework Documentation, http://static.springframework.org/spring/docs/2.0.x/reference/mvc.html) 10. Hypertext Transfer Protocol (HTTP), Wikipedia article and references (http://en.wikipedia.org/wiki/HTTP) 39 11. Treck Inc. Web Server User Documentation (www.treck.com) 12. 13. Spring Web MVC Framework Documentation (http://static.springframework.org/spring/docs/2.0.x/reference/mvc.html) * These documents, and many others, may be found in the Xilinx EDK installation folder under the doc/usenglish sub-folder.
6 Examples The following subsections describe some of the most conclusive examples and applications provided with the web server. 6.1 Led Control 6.1.1 Overview This example shows how the state of the on-board leds may be controlled by means of the interactive web interface. The leds are powered on / off when clicking the corresponding buttons on the web page:
Clicking a button toggles the leds state (there are eight leds on the Spartan-3E Starter Kit Board). 6.1.2 LedsController This is the controller that handles requests to view the state of the leds, but also turns them on and off depending on some request parameters. It is registered to the path /web/leds such that a simple view request to the web server at address 192.168.1.101 would look like http://192.168.1.101/web/leds It shows the current state of the leds. Turning led no. 3 (from left to right) on is done by calling: http://192.168.1.101/web/leds?on=2 Note that the led index starts from 0. Thus led no. 3 has the index 2. It may be turned off with: http://192.168.1.101/web/leds?off=2 Internally, the controller holds the state of the leds in an integer variable (leds_state) which is modified if a led needs to be turned on or off. The controller checks the request parameters on and off using the getParameter(char*) method of the request object and converts the character value to an integer index: 40 if (request->getParameter("on") != NULL) { index = atoi(request->getParameter("on")); leds_state |= (1 << (7 - index)); } else if (request->getParameter("off") != NULL) { index = atoi(request->getParameter("off")); leds_state &= ((~(1 << (7 - index))) & 255); } The on-board leds are connected to a GPIO (General Purpose Input Output) core and are set by using the appropriate driver methods: XGpio leds; XGpio_Initialize(&leds, XPAR_LEDS_8BIT_DEVICE_ID); XGpio_SetDataDirection(&leds, 1, 0x00000000); XGpio_DiscreteWrite(&leds, 1, leds_state); Next, the HTML response needs to be rendered to show the (new) state of the leds. The controller creates an 8-element integer array (leds_array, one element for each led) with values 0 or 1 that represent the state of the led at each index: turned off or turned on. A void pointer to this array is passed as a model to an instance of LedsView that renders the response accordingly. 6.1.3 LedsView This is the subclass of View responsible for rendering the web page that shows the state of each led (see the screen-shot at the beginning of this example). It is generated from a HTML page with embedded C/C++ code snippets: <% int i; int* leds_array= (int*) model; %>
6.2 Click Count 6.2.1 Overview This common example focuses on the session management performed automatically by the web server. The Click me button shows the number of times the same user has pressed it. This number is attached to the users session and is incremented each time the user reloads the page. The web server makes sure that each request has a session, old or new, associated with it. 6.2.2 ClickController This controller handles requests of the form: http://192.168.1.101/web/click It retrieves the session object from the request and looks for a session attribute named count. If it doesnt exist, a pointer to a new integer is created, initiated to 0 and attached to the session under the form of a void pointer. However, if count has been previously set it is incremented. The following is a full transcription of the controllers handleRequest method. void ClickController: handleRequest(Request* request, Response* response) { Session* session = request->getSession(); int* count = (int*) session->getAttribute("count"); if (count == NULL) { count = new int; *count = 0; session->setAttribute("count", (void*) count); } else if (request->getParameter("action") != NULL && !strcmp(request- >getParameter("action"), "reset")) { *count = 0; } else { (*count)++; } View* clickView = new ClickView(); clickView->renderResponse(request, response, (void*) count); delete clickView; } 42 The controller sends the number of clicks to the view to be rendered. To reset the click count, the controller needs to receive a request parameter named action with the value reset: http://192.168.1.101/web/click?action=reset
6.2.3 ClickView The C/C++ code snippet that renders the number of clicks: <% int* count= (int*) model; %> <input type="button" value="Click me ( <% char buffer[4]; sprintf(buffer, "%d", *count); response->append(buffer); %> )" onclick="location.href='click'" /> <a href="click?action=reset">Reset</a> 6.3 Temperature measurement 6.3.1 Overview This example shows how the temperature value (degrees Celsius) acquired by the sensor network is presented to the user:
6.3.2 TemperatureController The temperature makes use of a SensorNetwork instance to update and retrieve the current temperature value which is then sent to the view to be rendered: SensorNetwork* sn = new SensorNetwork(); sn->update(); int* temp; *temp = sn->getTemperature(0); View* view = new TemperatureView(); view->renderResponse(request, response, (void*) temp);
43 6.3.3 TemperatureView As with other template pages that need to be filled with data, the temperature view uses C/C++ code snippets to insert the data at the right places. 6.4 Google Maps 6.4.1 Overview This example shows a balanced combination of using the Google Maps API together with background AJAX requests and session management. The user moves a marker along the map while his actions are recorded silently on the server side. The next time he loads the page, the marker will show exactly at the last coordinates:
The following screen shot of a data tampering tool shows how several AJAX requests are silently forwarded to the server with the current marker coordinates (latitude and longitude).
6.4.2 MapView The following image is a screen shot of the HTML template page loaded without being compiled. It can be seen how the view fills in the placeholders with data retrieved from the model by using response- >append() statements:
44 6.5 Configuration 6.5.1 Software The web server framework and custom business logic are compiled by the SDK into an ELF (Executable and Linkable Format) file. As the Block RAM is much too small (usually 8 to 16 kB) to accommodate the executable code, the processor runs the executable code from the bigger, onboard DDR memory (64MB DDR SDRAM). Getting the executable code into the RAM memory is done by running the following commands into the XMD (Xilinx Micro-Processor Debugger) shell: dow elf_file_name con , where dow downloads the executable code and sets the program counter address to 0 and con starts the program execution. This practice is extremely useful during development. However, a different approach should be followed when deploying the web server. As it would be highly inconvenient to manually download the executable code every time the board is powered up, it is easier to store it in an onboard Flash memory and use a bootloader to copy it to the DDR memory. The Xilinx Embedded Development Kit (EDK) provides a simple way to do this: the Program Flash Memory dialog in XPS. First, the ELF file needs to be selected along with the Auto-convert file to bootloadable SREC format when programming flash option. The Flash and DDR memories are selected (0 offset) by default so checking the Create Flash Bootloader Application option is the last thing to do. XPS creates a small bootloader application that resides in the Block RAM (select Mark to initialize BRAMs). It will copy the executable code from the Flash to the DDR memory and continue execution from there. 6.5.2 Hardware For deployment, the FPGA logic image is persisted on the on-board 4Mb serial Platform Flash PROM. When using the Master Serial mode configuration with the Spartan-3E Start Kit Board (check for the appropriate jumper settings), the image is automatically used to configure the FPGA at power up and reset. 6.5.2.1 iMPACT The FPGA image is generated and downloaded to Flash using iMPACT: Open iMPACT and select the default project Choose to Prepare a PROM File Make it a Xilinx PROM in MCS format and give it a file name Choose to use the Xilinx PROM in Serial Mode 45 Add the on-board Flash: xcf04s Generate the MCS file and do a boundary scan Download the MSC file to the Flash 7 Discussion 7.1 Problems Encountered The following paragraphs refer to some issues encountered during development which are not clearly stated in the referenced documentation: 7.1.1 lwIP 1. Heap size: lwIP will not work correctly if not assigned a greater heap size than the default value. One may set the heap size by modifying the linker script. A value of 0x400000 (4MB), although a lot more than required, was used for this project; 2. Netbuf count: due to the fact that lwIP uses Xilkernel services in socket mode, the number of semaphores chosen in the Xilkernel configuration must be equal or greater to the value set for the memp_num_netbuf (lwip_memory_options); otherwise socket connections may periodically be lost. 7.1.2 EDK 3. XMD: When using instances of cores defined in a global IP repository (for user-defined IP cores) it becomes difficult to start the hardware debugger (XMD), program the flash memory or other such utilities due to some path inconsistencies: the system cannot find the files related to your IP core; 8 Future Development 8.1 Security and Authorization The next versions of the web server framework could provide embedded security mechanisms like user roles hierarchies, URL (Universal Resource Locator) access authorization and secured content delivery. The web server would automatically store user authentication and authorization information on the session to be used internally for enforcing the security environment. Users that do not comply with required roles (e.g. when an admin role is required) are not allowed to access some resources. These features would be supported by abstract and default framework implementations, but would also allow for custom implementations.
46 9 Appendix 1: The EDK-generated block diagram:
47 10 Appendix 2: The web server framework files (SDK view)