Just-In-Time Compiler

MASSLAB

Contents
Java Virtual Machine
Runtime data areas Java instruction set pros & cons

Just-In-Time Compiler(JITC)
Architecture System Overview Multi-level recompilation Cost/Benefit model Profiling heuristics performance

Conclusion

Java Virtual Machine

An abstract computing machine Own instruction set (Java ISA) Memory manipulation at runtime
user class files
class loader
bytecodes

The Java APIs class file

JVM

execution engine
native method invocation

Host OS

Runtime Data Areas

PC register
each JVM thread has its own program counter the index in bytecode of the instruction

JVM stack
each JVM thread has its own stacks invoke method -> new frame is allocated on stack store operand stack, local variables, state variables

Operand Stack
used to pass arguments receive return results interpreter is responsible for transferring return result to the caller operand stack

Runtime Data Areas

Local variable
number of local variables in stack is determined at compile time local variables are numbered

Heap
one heap per one JVM created on VM start-up all class instance and arrays are allocated in heap

Constant pool
analogous to symbol table a collection of all symbolic data need by a class each constant pool is private to its class

Java Instruction Set

opcode is just one byte (256 combination possible) additional index bytes additional data bytes
opcode

opcode
opcode

index1 data

index2

Java Instruction Set

Data movement instr.
Stack manipulation instructions ex) bipush, pop, dup, Iconst_1, swap

Type conversion
convert one type of item on the stack to another ex) i2f, i2d, i2l, f2i, f2l

Functional instr.
operands are always taken from the stack and results are placed on the stack ex) iadd, iand

Control flow instr.

conditional branches ex) ifeq, if_icmpeq unconditional branches ex) jsr, jsr_w

Pros & cons of JVM

Pros
Write once, run everywhere!

Cons
Bytecode interpret at runtime Low performance

Solution: JITC & AOTC

JITC vs AOTC
Just-In-Time Compiler (JITC)
Compile bytecode to machine code at runtime Compile time are included in execute time make the maximum use of program execution information require some CPU and Memory resources ex) Jikes RVM, DSVM

Ahead-Of-Time Compiler (AOTC)

Compile bytecode to machine code before program execution Able to do heavy optimizations Cant know program execution information ex) LaTTe

Jikes RVM an example of JITC

developed at IBM T.J. Watson Research Center targeting server applications written in Java employs 2 strategies baseline compiler optimizing compiler (O0, O1, O2) A flexible online adaptive compilation infrastructure.

Jikes RVM

Architecture System Overview

3 components of Adaptive Optimization System (AOS)
Runtime measurements subsystem Controller Recompilation subsystem

Runtime Measurements Subsystem

Subsystem which produce raw profiling data
Monitor the performance of individual methods from a hardware performance monitor

Organizers periodically process and analyze the raw data

the processes data is used by the Controller

Controller
Orchestrates and conducts the other components of AOS system
build an compilation plan using profiling data to improve their performance

Determines whether it is profitable to recompile the method

To estimate cost of recompilation, linear model is used
this model is calibrated offline

use Cost/Benefit model to make this calculation

analytic model representing the cost and benefits of these tasks

Recompilation Subsystem
Recompilation occur
in separate threads from the application

3 components of compilation plan

Profiling data Optimization plan Instrumentation plan

Multi-Level Recompilation
Level 0 optimizations performed on-the-fly during the translation from bytecodes to the HIR
Register renaming for local variable, dead-code elimination

Level 1 addition local optimizations

CSE, array bound check elimination, redundant load elimination

flow-insensitive optimizations
copy and constant propagation, dead-assignment elimination

Scalar replacement of aggregates and short arrays Level 2 SSA-based flow sensitive optimization
SSA-PRE, SSA-CSE

Cost/Benefit model
Assumptions Sample data determines how long a method has executed Method will execute as much in the future as it has in the past Compilation cost and speedup are offline average

Cost/Benefit model
Definitions cur, current optimization level for method m T(j), expected future execution time at optimization level j C(j), compilation cost at optimization level j Choose j > cur that minimizes T(j) + C(j) If T(j) + C(j) < T(cur) recompile at optimization level j.

Estimated Cost/Benefit

Profiling heuristics
How to find candidates for optimization
counters call stack sampling

Counters
Insert method-specific counter on method entry and loop back edge. Optimize method that surpasses threshold. Very popular approach : Self, Hotspot, DSVM Issues
Overhead for incrementing counter can be significant simple, but hard to tune

Call stack sampling

Periodically record which method(s) are on the top of call stack. Approximates amount of time spent in each method Use Cost/Benefit model ex) Jikes RVM, JRocket Issues
low-overhead, but non-deterministic.

Performances

Conclusion
Java Virtual Machine has good features, but its performance is low. We can overcome low performance of JVM with Just-In-Time Compiler.