Birth, Life and Death of a Class
Volker Simonis [Фолкер Симонис], SAP / volker.simonis@gmail.com
https://simonis.github.io/JEEConf2019
The class File Format
- Code to be executed by the Java Virtual Machine
- Binary format (big-endian)
- Hardware- and operating system-independent
- Chapter 4 of the Java Virtual Machine Specification
The class File Format
ClassFile {
u4 magic; // 0xCAFEBABE
u2 minor_version;
u2 major_version; // >= 45
u2 constant_pool_count;
cp_info constant_pool[constant_pool_count-1];
u2 access_flags;
u2 this_class; // pointers into
u2 super_class; // the constant_pool
u2 interfaces_count;
u2 interfaces[interfaces_count];
u2 fields_count;
field_info fields[fields_count];
u2 methods_count; // Here's the
method_info methods[methods_count]; // real bytecode!
u2 attributes_count; // predefined and
attribute_info attributes[attributes_count]; // custom attributes
}
HelloWorld.java
package simonis;
public class HelloWorld {
public static void main(String[] args) {
System.out.println("Hello world!");
}
}
$ javac simonis/HelloWorld.java
$ javap -c -v simonis.HelloWorld
...
HelloWorld.class
public class simonis.HelloWorld
minor version: 0
major version: 53
flags: (0x0021) ACC_PUBLIC, ACC_SUPER
this_class: #1 // simonis/HelloWorld
super_class: #3 // java/lang/Object
interfaces: 0, fields: 0, methods: 2, attributes: 1
Constant pool:
#1 = Class #2 // simonis/HelloWorld
#2 = Utf8 simonis/HelloWorld
#3 = Class #4 // java/lang/Object
#4 = Utf8 java/lang/Object
...
#24 = Methodref #25.#27 // java/io/PrintStream.println:(Ljava/lang/String;)V
#25 = Class #26 // java/io/PrintStream
#26 = Utf8 java/io/PrintStream
#27 = NameAndType #28:#29 // println:(Ljava/lang/String;)V
#28 = Utf8 println
#29 = Utf8 (Ljava/lang/String;)V
...
public static void main(java.lang.String[]);
Code:
stack=2, locals=1, args_size=1
0: getstatic #16 // Field java/lang/System.out:Ljava/io/PrintStream;
3: ldc #22 // String Hello world!
5: invokevirtual #24 // Method java/io/PrintStream.println:(Ljava/lang/String;)V
8: return
LineNumberTable:
line 5: 0
line 6: 8
LocalVariableTable:
Start Length Slot Name Signature
0 9 0 args [Ljava/lang/String;
Class types
- Top-Level
- Nested
- Static Nested (Nested)
- Non-Static Nested (Inner)
- Local
- Anonymous
- VM-Anonymous
Static Nested Classes
public class TopLevel {
static class Nested {
public Nested() {}
}
public static void main(...) {
Nested nc = new Nested();
}
}
$ javac simonis/TopLevel.java
TopLevel.class
TopLevel$Nested.class
public class TopLevel {
static class Nested {
private Nested() {}
}
public static void main(...) {
Nested nc = new Nested();
}
}
$ javac simonis/TopLevel.java
TopLevel.class
TopLevel$Nested.class
TopLevel$1.class
$ java -jar ecj.jar simonis/TopLevel.java
TopLevel.class
TopLevel$Nested.class
What if we make the constructor of class Nested private ?
- Compile error
- Two classes as before
- Three classes
JAVAC
$ javac simonis/TopLevel.java
$ javap simonis.TopLevel$Nested
...
Compiled from "TopLevel.java"
class TopLevel$Nested {
private TopLevel$Nested();
flags: (0x0002) ACC_PRIVATE
Code:
...
TopLevel$Nested(TopLevel$1);
flags: (0x1000) ACC_SYNTHETIC
Code:
0: aload_0
1: invokespecial #1// "<init>"
4: return
}
ECJ
$ java -jar ecj.jar simonis/TopLevel.java
$ javap -c -p simonis.TopLevel$Nested
...
Compiled from "TopLevel.java"
class TopLevel$Nested {
private TopLevel$Nested();
flags: (0x0002) ACC_PRIVATE
Code:
...
TopLevel$Nested(TopLevel$Nested);
flags: (0x1000) ACC_SYNTHETIC
Code:
0: aload_0
1: invokespecial #12// "<init>"
4: return
}
Java 11 - JEP 181: Nest-Based Access Control
$ jdk11/bin/javac simonis/TopLevel.java
TopLevel.class
TopLevel$Nested.class
$ javap simonis.TopLevel
...
Compiled from "TopLevel.java"
public class simonis.TopLevel
...
public static void main(...);
// class TopLevel$Nested
0: new #13
3: dup
// method TopLevel$Nested.<init>()V
4: invokespecial #15
NestMembers:
class simonis/TopLevel$Nested
$ javap simonis.TopLevel$Nested
...
Compiled from "TopLevel.java"
class simonis.TopLevel$Nested
...
private simonis.TopLevel$Nested();
flags: (0x0002) ACC_PRIVATE
...
NestHost:
class simonis/TopLevel
Inner classes
public class TopLevel {
class Inner {
public Inner() {}
}
public static void main(String... args) {
Inner ic = new TopLevel() . new Inner();
}
}
Inner classes
$ javap -v simonis.TopLevel$Inner
...
Compiled from "TopLevel.java"
class simonis.TopLevel$Inner {
final simonis.TopLevel this$0;
flags: (0x1010) ACC_FINAL, ACC_SYNTHETIC
public simonis.TopLevel$Inner(simonis.TopLevel);
flags: (0x0001) ACC_PUBLIC
Code:
0: aload_0
1: aload_1
2: putfield #1 // Field this$0:Lsimonis/TopLevel;
5: aload_0
6: invokespecial #2 // Method java/lang/Object."<init>":()V
9: return
}
Anonymous and Local Classes
public class TopLevel {
public static void main(...) {
Runnable r = new Runnable() {
public void run() {}
};
}
}
$ javac simonis/TopLevel.java
TopLevel.class
TopLevel$1.class
public class TopLevel {
public static void main(...) {
class Local {
Local() {}
}
Local l = new Local();
}
}
$ javac simonis/TopLevel.java
TopLevel.class
TopLevel$1Local.class
VM-Anonymous classes
public class VmAnonymous {
public static void main(String... args) {
Runnable r = () -> {
new Exception("Hello World!") . printStackTrace();
};
System.out.println(r.getClass().getName());
r.run();
}
}
$ javac simonis/VmAnonymous.java
$ ll simonis/
-rw-rw-r-- 1 simonis simonis 1672 Okt 13 19:27 VmAnonymous.class
$ java simonis.VmAnonymous
simonis.VmAnonymous$$Lambda$1/88579647
java.lang.Exception: Hello World!
at simonis.VmAnonymous.lambda$0(VmAnonymous.java:9)
at simonis.VmAnonymous.main(VmAnonymous.java:12)
$ java -XX:+UnlockDiagnosticVMOptions -XX:+ShowHiddenFrames simonis.VmAnonymous
simonis.VmAnonymous$$Lambda$1/88579647
java.lang.Exception: Hello World!
at simonis.VmAnonymous.lambda$0(VmAnonymous.java:9)
at simonis.VmAnonymous$$Lambda$1/88579647.run(<Unknown>:1000000)
at simonis.VmAnonymous.main(VmAnonymous.java:12)
$ java -Djdk.internal.lambda.dumpProxyClasses=/tmp simonis.VmAnonymous
...
$ ls -l /tmp/*.class
...
-rw-rw-r-- 1 simonis simonis 336 Okt 13 19:43 VmAnonymous$$Lambda$1.class
Java 9 - Stack-Walking API (JEP 259)
public class VmAnonymous2 {
public static void main(String... args) {
Runnable r = () -> {
StackWalker sw = StackWalker.getInstance(Option.SHOW_HIDDEN_FRAMES);
sw.forEach(System.out::println);
};
r.run();
}
}
$ java simonis.VmAnonymous
simonis.VmAnonymous2.lambda$0(VmAnonymous2.java:14)
simonis.VmAnonymous2$$Lambda$1/88579647.run(Unknown Source)
simonis.VmAnonymous2.main(VmAnonymous2.java:22)
VM-Anonymous classes
public class VmAnonymous3 {
public static void main(String... args) {
Runnable r = () -> {
new Exception("Hello World!") . printStackTrace();
System.console().readLine();
};
r.run();
}
}
$ java -XX:+UnlockDiagnosticVMOptions -XX:+ShowHiddenFrames\
simonis.VmAnonymous3
java.lang.Exception: Hello World!
at simonis.VmAnonymous3.lambda$0(VmAnonymous3.java:7)
at simonis.VmAnonymous3$$Lambda$1/97730845.run(...)
at simonis.VmAnonymous3.main(VmAnonymous3.java:10)
$ jps
22436 VmAnonymous3
$ jhsdb hsdb
Class Identity
package simonis;
public class TwoLoaders {
public static void main(String[] args) throws ... {
TwoLoaders tl1 = new TwoLoaders();
ClassLoader cl = new URLClassLoader(
new URL[] { TwoLoaders.class.getResource("..") }, null);
Object tl2 = cl.loadClass("simonis.TwoLoaders").newInstance();
System.out.println(tl2.getClass().getName().equals(tl1.getClass().getName())
System.out.println(tl2 instanceof simonis.TwoLoaders);
tl1 = (TwoLoaders)tl2;
What's the output of
tl2.getClass().getName().equals(tl1.getClass().getName())
?
- true
- false
Whats the output of
tl2 instanceof simonis.TwoLoaders
?
- true
- false
What about "tl1 = (TwoLoaders)tl2" ?
- Compile error.
- Runtime error.
- Runs just fine.
$ java simonis.TwoLoaders
true // class names equality
false // instanceof
Exception in thread "main" // assignment
java.lang.ClassCastException:
simonis.TwoLoaders cannot be cast to simonis.TwoLoaders
at simonis.TwoLoaders.main(TwoLoaders.java:14)
$ sapjvm/bin/java simonis.TwoLoaders
...
java.lang.ClassCastException:
Cannot cast
class simonis.TwoLoaders
(loader java.net.URLClassLoader [id=8020,parents=<bootstrap>]) to
class simonis.TwoLoaders
(loader System [class=sun.misc.Launcher$AppClassLoader,urls=file:/tmp/])
at simonis.TwoLoaders.main(TwoLoaders.java:14)
$ jdk11/bin/java simonis.TwoLoaders
...
simonis.TwoLoaders cannot be cast to simonis.TwoLoaders
(simonis.TwoLoaders is in unnamed module of loader java.net.URLClassLoader
simonis.TwoLoaders is in unnamed module of loader 'app')
CHA - Class Hierarchy Analysis
package simonis;
public class CHA {
static class A {
void f() {}
}
static class B extends A {
void f() {}
}
static void g(A a) {
a.f();
}
...
...
public static void main (String[] args) {
A a = new A();
for (int i = 0; i < 20_000; i++) {
g(a); // JIT compile g()
}
B b = new B();
for (int i = 0; i < 20_000; i++) {
g(a);
}
}
}
$ java -Xbatch -XX:-TieredCompilation -XX:-UseOnStackReplacement \
-XX:+PrintCompilation -XX:+PrintInlining simonis.CHA
...
1020 4 b simonis.CHA::g (5 bytes)
@ 1 simonis.CHA$A::f (1 bytes) inline (hot)
1033 4 simonis.CHA::g (5 bytes) made not entrant
1035 6 b simonis.CHA::g (5 bytes)
@ 1 simonis.CHA$A::f (1 bytes) inline (hot)
\-> TypeProfile (11699/11699 counts) = simonis/CHA$A
$ java -Xbatch -XX:-TieredCompilation -XX:-UseOnStackReplacement \
-XX:+PrintCompilation -XX:+PrintInlining -Xlog:class+load simonis.CHA
...
[1,018s][info][class,load] simonis.CHA$A
1020 4 b simonis.CHA::g (5 bytes)
@ 1 simonis.CHA$A::f (1 bytes) inline (hot)
[1,033s][info][class,load] simonis.CHA$B
1033 4 simonis.CHA::g (5 bytes) made not entrant
1035 6 b simonis.CHA::g (5 bytes)
@ 1 simonis.CHA$A::f (1 bytes) inline (hot)
\-> TypeProfile (11699/11699 counts) = simonis/CHA$A
$ java ... -XX:CompileCommand="print simonis.CHA::g" \
-XX:+TraceDependencies simonis.CHA
...
// void simonis/CHA.g(simonis/CHA$A*)
//
000 B1:pushq rbp // Save rbp
subq rsp, #16 // Create frame
00c testq RSI, RSI // parm0:simonis/CHA$A*
00f je,s B3 // Null-pointer check
011 B2:addq rsp, 16 // Destroy frame
popq rbp
testl rax, [#poll_page]// Safepoint: poll for GC
01c ret
...
Dependencies:
Dependency of type unique_concrete_method
context = simonis.CHA$A
method = {method} {0x00007fd6ac477868} 'f' '()V' in 'simonis/CHA$A'
[nmethod<=klass]simonis.CHA$A
...
[1,006s][info][class,load] simonis.CHA$B
Failed dependency of type unique_concrete_method
context = simonis.CHA$A
method = {method} {0x00007fe759c00868} 'f' '()V' in 'simonis/CHA$A'
witness = simonis.CHA$B
Marked for deoptimization: Compiled method (c2) simonis.CHA::g (5 bytes)
...
// void simonis/CHA.g(simonis/CHA$A*)
//
...
00c movl R11, [RSI + #8] // compressed klass ptr (NullCheck RSI)
010 B2:cmpl R11, 0x7fe7440920a0 // narrowklass: precise klass simonis/CHA$A
017 jne,us B4
019 B3:addq rsp, 16 // Destroy frame
popq rbp
testl rax, [#poll_page] // Safepoint: poll for GC
024 ret
025 B4:call uncommon_trap(reason='class_check' action='maybe_recompile')
CHA - Class Hierarchy Analysis
package simonis;
public class CHA2 {
static class A {
void f() {}
}
static class B extends A {
void f() {}
}
static void g(A a) {
a.f();
}
...
...
public static void main (String[] args) {
A a = new A();
for (int i = 0; i < 20_000; i++) {
g(a); // JIT compile g()
}
A b = new B(); // was B b = new B();
for (int i = 0; i < 20_000; i++) {
g(a);
}
}
}
What happens if we change
B b = new B() to A b = new B()
?
- Same behaviour as before
- Same behaviour as before but without inlining of
f()intog() - No more recompilation
$ java -Xbatch -XX:-TieredCompilation -XX:-UseOnStackReplacement \
-XX:+PrintCompilation -XX:+PrintInlining -Xlog:class+load simonis.CHA2
...
[1,012s][info][class,load] simonis.CHA2$A
[1,018s][info][class,load] simonis.CHA2$B
1025 4 b simonis.CHA2::g (5 bytes)
@ 1 simonis.CHA2$A::f (1 bytes) inline (hot)
\-> TypeProfile (6700/6700 counts) = simonis/CHA2$A
$ java -Xbatch -XX:-TieredCompilation -XX:-UseOnStackReplacement -noverify \
-XX:+PrintCompilation -XX:+PrintInlining -Xlog:class+load simonis.CHA2
...
[0,993s][info][class,load] simonis.CHA2$A
995 4 b simonis.CHA2::g (5 bytes)
@ 1 simonis.CHA2$A::f (1 bytes) inline (hot)
[1,007s][info][class,load] simonis.CHA2$B
1007 4 simonis.CHA2::g (5 bytes) made not entrant
1008 6 b simonis.CHA2::g (5 bytes)
@ 1 simonis.CHA2$A::f (1 bytes) inline (hot)
\-> TypeProfile (11699/11699 counts) = simonis/CHA2$A
What happens if we use -noverify ?
- Same behaviour as before (just one compilation)
- Initial behaviour (compilation / deoptimization / compilation)
CHA - Class Hierarchy Analysis
package simonis;
public class CHA2 {
static class A {
void f() {}
}
static class B extends A {
void f() {}
}
static void g(A a) {
a.f();
}
...
...
public static void main (String[] args) {
A a = new A();
for (int i = 0; i < 20_000; i++) {
g(a); // JIT compile g()
}
A b = new B(); // was B b = new B();
for (int i = 0; i < 20_000; i++) {
g(a);
}
}
}
Class Instrumentation / Redefinition
package java.lang.instrument;
public interface Instrumentation {
void addTransformer(ClassFileTransformer transformer);
public boolean isRetransformClassesSupported();
public void retransformClasses(Class<?>... classes);
public boolean isRedefineClassesSupported();
public void redefineClasses(ClassDefinition... definitions);
}
public interface ClassFileTransformer {
public byte[] transform(ClassLoader cl, String className
Class<?> classBeingRedefined, ProtectionDomain pd,
byte[] classfileBuffer)
...
}
Creating an Instrumentation Agent
Manifest-Version: 1.0
Premain-Class: simonis.InstAgent
Can-Retransform-Classes: true
Can-Redefine-Classes: true
public class InstAgent {
public static void premain(String args, Instrumentation inst) {
...
}
}
$ jar cvfm InstAgent.jar manifest.mf simonis/InstAgent.class
$ java -javaagent:InstAgent.jar='arguments' ApplicationClass
public class InstAgent {
static Instrumentation inst;
static String pattern;
static int count = 0;
public static void premain(String args, Instrumentation inst) {
InstAgent.inst = inst;
pattern = args;
inst.addTransformer(new MethodInstrumentorTransformer(), true);
}
static class MethodInstrumentorTransformer implements ClassFileTransformer {
@Override
public byte[] transform(.., String className, .., byte[] classfileBuffer) {
if (!className.startsWith(pattern)) return null; // No transformation,
... // else transform to print a message at every method entry and exit
}
}
public static Instrumentation getInst() {
count++;
return inst;
}
public class Instrument {
public static void main(String[] args) {
Runnable r = () -> {
new Exception("Hello World!") . printStackTrace();
};
r.run();
}
}
$ java -javaagent:InstAgent.jar=simonis -XX:+ShowHiddenFrames simonis.Instrument
=> transformimg simonis/Instrument
0 -> simonis.Instrument::main([Ljava/lang/String;)V
0 -> simonis.Instrument::lambda$0()V
java.lang.Exception: Hello World!
at simonis.Instrument.lambda$0(Instrument.java:7)
at simonis.Instrument$$Lambda$1/1556595366.run(<Unknown>:1000000)
at simonis.Instrument.main(Instrument.java:9)
0 <- simonis.Instrument::lambda$0()V
0 <- simonis.Instrument::main([Ljava/lang/String;)V
public class Instrument2 {
static CountDownLatch stop = new CountDownLatch(1);
public static class A {
public static Runnable getRunnable() {
return () -> {
try { stop.await(); } catch (Exception e) {};
};
}
}
public static void main(String[] args) throws ... {
Runnable r = A.getRunnable();
for (int i = 0; i < 10; i++) {
InstAgent.getInst().retransformClasses(Instrument2.A.class);
new Thread(r).start();
}
System.in.read();
...
$ java -javaagent:InstAgent.jar='simonis/Instrument2$A' simonis.Instrument2
=> transformimg simonis/Instrument2$A
0 -> simonis.Instrument2$A::getRunnable()
0 <- simonis.Instrument2$A::getRunnable()
=> re-transforming simonis/Instrument2$A
1 -> simonis.Instrument2$A::lambda$0()V
=> re-transforming simonis/Instrument2$A
2 -> simonis.Instrument2$A::lambda$0()V
=> re-transforming simonis/Instrument2$A
...
9 -> simonis.Instrument2$A::lambda$0()V
$ jcmd <pid> GC.class_stats InstBytes,KlassBytes,MethodCount,Bytecodes
Index InstBytes KlassBytes Bytecodes ClassName
320 16 528 9 simonis.Instrument2$A$$Lambda$1/985922955
768 0 496 80 simonis.Instrument2
769 0 496 70 simonis.Instrument2$A
770 0 496 70 simonis.Instrument2$A
... . ... .. .....................
776 0 496 70 simonis.Instrument2$A
777 0 496 70 simonis.Instrument2$A
How many versions of class
simonis.Instrument2$A
do we have ?
- One version
- Several versions
- 10 versions
Class representation in the HotSpot VM
Class unloading
- It's an optimization (not required by the spec)
- Can't simply unload/reload on request:
- JVMS §5.3:
"Given the same name, a good class loader
should always return the same Class object" - Static initializers
- JVMS §5.3:
"Given the same name, a good class loader
- GC dependant (happens at safepoint)
- Interacts with object finalization (JVMLS §12.7)
public class Unload {
public static class X {}
public static void main(String[] args) throws ... {
ClassLoader cl = new URLClassLoader(
new URL[] { Unload.class.getResource("..") }, null);
Object o = cl.loadClass("simonis.Unload$X").newInstance();
o = null;
cl = null;
systemGC();
$ java -Xlog:class+load,class+unload simonis.Unload | grep simonis
[info][class,load ] simonis.Unload
[info][class,load ] simonis.Unload$X
simonis: -> Calling System.gc() ...
[info][class,unload] unloading class simonis.Unload$X
simonis: <- System.gc() done
Class unloading and finalization
public class Unload {
static Object keepAlive;
public static class Y {
protected void finalize() throws Throwable {
Unload.keepAlive = this;
}
}
public static void main(String[] args) throws ... {
ClassLoader cl = new URLClassLoader(...);
o = cl.loadClass("simonis.Unload$Y").newInstance();
o = null;
cl = null;
systemGC();
keepAlive = null;
systemGC();
When will class simonis.Unload.Y be unloaded ?
- During the first System.gc()
- During the second System.gc()
- Never
$ java -Xlog:class+load,class+unload simonis.Unload | grep simonis
[class,load ] simonis.Unload
[class,load ] simonis.Unload$Y
simonis: -> Calling System.gc() ...
simonis: <- System.gc() done
<RETURN>
simonis: -> Calling System.gc() ...
[class,unload] unloading class simonis.Unload$Y
simonis: <- System.gc() done
Class unloading and finalization
public class Unload {
static Object keepAlive;
public static class Y {
protected void finalize() throws Throwable {
Unload.keepAlive = this;
}
}
public static void main(String[] args) throws ... {
ClassLoader cl = new URLClassLoader(...);
o = cl.loadClass("simonis.Unload$Y").newInstance();
o = null;
cl = null;
systemGC();
// keepAlive = null;
systemGC();
What happens when we remove keepAlive = null ?
- No unloading any more
- Nothing changes
public static class Y {
protected void finalize() throws Throwable {
Unload.keepAlive = this;
}
}
public static void main(String[] args) throws ... {
ClassLoader cl = new URLClassLoader(..., null);
o = cl.loadClass("simonis.Unload$Y").newInstance();
systemGC();
systemGC();
$ java -Xlog:class+load=debug,class+unload=debug simonis.Unload | grep simonis
[class,load ] simonis.Unload klass: 0x100060030 'ClassLoaders$AppClassLoader'
[class,load ] simonis.Unload$Y klass: 0x100074430 'java/net/URLClassLoader'
simonis: -> Calling System.gc() ...
simonis: <- System.gc() done
[class,load ] simonis.Unload klass: 0x100074628 'java/net/URLClassLoader'
<RETURN>
simonis: -> Calling System.gc() ...
[class,unload] unloading class simonis.Unload 0x100074628
[class,unload] unloading class simonis.Unload$Y 0x100074430
public class Unload {
public static class Z {
public static void run() throws InterruptedException {
Thread.sleep(3_000);
System.out.println("simonis: exiting Z::run()");
}
}
public static void main(String[] args) throws ... {
ClassLoader cl = new URLClassLoader(...);
Class<?> c = cl.loadClass("simonis.Unload$Z");
// Call Z::run() reflectively
new MyThread(c.getDeclaredMethod("run")).start();
c = null;
cl = null;
systemGC();
systemGC();
When will the class simonis.Unload.Z be unloaded ?
- During the first System.gc()
- During the second System.gc()
- It depends on the time between the first and second System.gc()
- Never
$ java -Xlog:class+load,class+unload simonis.Unload | grep simonis
[class,load] simonis.Unload
[class,load ] simonis.Unload$Z
simonis: -> Calling System.gc() ...
simonis: <- System.gc() done
<RETURN> // quick
simonis: -> Calling System.gc() ...
simonis: <- System.gc() done
simonis: exiting Z::run()
$ java -Xlog:class+load,class+unload simonis.Unload | grep simonis
[class,load] simonis.Unload
[class,load ] simonis.Unload$Z
simonis: -> Calling System.gc() ...
simonis: <- System.gc() done
simonis: exiting Z::run()
<RETURN> // wait for more than 3 seconds ...
simonis: -> Calling System.gc() ...
[class,unload] unloading class simonis.Unload$Z
simonis: <- System.gc() done