© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.

FireCRaCer:
The Best Of Both Worlds

Volker Simonis (@volker_simonis)
Principal Software Engineer
AWS

https://simonis.github.io/jProfessionals2023/

About me

  • Principal Software Engineer @ Amazon Corretto
  • OpenJDK Member, Committer & Reviewer
  • OpenJDK Build, Hotspot, Porters & Vulnerability Group
  • Java Community Process Executive Committee
  • SAP JVM and SapMachine @ SAP
  • OpenJDK Build PowerPC/AIX & s390x Project Lead
  • JCP Expert Groups for Java SE 9 to 13
  • OpenJDK Governing Board (2022/23)

About Firecracker

  • Minimalistic virtual machine monitor (VMM)
  • KVM backed
  • Limited devices: block/network (virtio), vsock, serial
  • REST-based configuration
  • Written in Rust (based on crosvm, now rust-vmm)
  • MicroVM-Metadata Service (JSON/IMDS)
  • Sandboxing (cgroup, chroot, seccomp)
  • Apache 2 licensed
 
 
 
 
 
 

Firecracker Demo

Firecracker vs. Docker

  • Full KVM virtualization
  • VMs completely isolated (ballooning device)
  • More secure
  • Snapshot support
  • cgroup/namespace isolation
  • Sharing between containers (COW/page cache)
  • Lightweight
  • Checkpoint/Restore with CRIU

CRaC & CRIU

  1. Create a standard checkpoint/restore notification API
    • Use org.crac as a wrapper for:
      jdk.crac / javax.crac / -Dorg.crac.Core.Compat=..
  2. Make the JDK "snapsafe":
    • i.e. act gracefully on checkpoint/restore & clone events
    • state, secrets, time, resources, ...
  3. Integrate with checkpoint/restore mechanisms:
    • Currently only CRIU

  • Serializes single processes to the file system

Using org.crac

  •                   
    import org.crac.Context;
    import org.crac.Core;
    import org.crac.Resource;
    
    @Component
    public class MyCheckpointRestoreListener implements Resource {
    
      @Autowired
      public MyCheckpointRestoreListener(...) {
        Core.getGlobalContext().register(this);
        logger.info("=> MyCheckpointRestoreListener: Registering in {}", Core.getGlobalContext());
      }
      @Override
      public void beforeCheckpoint(Context<? extends Resource> context) {
        logger.info("=> MyCheckpointRestoreListener: beforeCheckpoint");
      }
      @Override
      public void afterRestore(Context<? extends Resource> context) {
        logger.info("=> MyCheckpointRestoreListener: afterRestore");
      }
    }
                      
                    

CRaC & CRIU DEMO

FireCRaCer

  • Checkpointing plain/CRaC JDK with Firecracker:
    • No need to worry about file descriptors, network, shared memory,..
  • Can't checkpoint a Firecracker VM from within the guest:
    • Need host cooperation
  • Maximum security and speed
  • No COW memory sharing (need Ballooning/ Samepage Merging)

FireCRaCer DEMO

  1. "Unattended" checkpoint/restore with unmodified JDK
     
  2. Using a "CRaC"-enhanced JDK inside Firecracker
     
  3. Using a "JVMTI-agent" implementing the org.crac API

FireCRaCer DEMO

  1. "Unattended" checkpoint/restore with unmodified JDK
     
  2. Using a "CRaC"-enhanced JDK inside Firecracker
     
  3. Using a "JVMTI-agent" implementing the org.crac API

FireCRaCer DEMO

  1. "Unattended" checkpoint/restore with unmodified JDK
     
  2. Using a "CRaC"-enhanced JDK inside Firecracker
     
  3. Using a "JVMTI-agent" implementing the org.crac API

Java-agent implementing org.crac

  • Injected with -Dorg.crac.Core.Compat=io.simonis.crac
  •                   
    package io.simonis.crac.impl;
    
    public class FirecrackerContext extends Context<Resource> {
    
      private final ArrayList<WeakReference<Resource>> resources = new ArrayList<>();
    
      @Override
      public void register(Resource resource) {
        resources.add(new WeakReference<Resource>(resource));
      }
    
      @Override
      public void beforeCheckpoint(Context<? extends Resource> context) throws CheckpointException {
        for (var iterator = resources.listIterator(resources.size()); iterator.hasPrevious();) {
          Resource r = iterator.previous().get();
          r.beforeCheckpoint(this);
          ...
      @Override
      public void afterRestore(Context<? extends Resource> context) throws RestoreException {
        for (var iterator = resources.listIterator(); iterator.hasNext();) {
          ...
                      
                    

JVMTI-agent controlling threads and GC

  • Use JVMTI to suspend/resume threads before/after checkpointing:
  •                   
    jint Java_io_simonis_SuspendResumeAgent_suspendThreads(JNIEnv *env, jclass cls) {
      jint threads_count;
      jthread *threads;
      jvmti->GetAllThreads(&threads_count, &threads);
      ...
      jvmtiError error = jvmti->SuspendThreadList(threads_count - 1, threads, errors);
    
    jint Java_io_simonis_SuspendResumeAgent_resumeThreads(JNIEnv *env, jclass cls) {
      jint threads_count;
      jthread *threads;
      jvmti->GetAllThreads(&threads_count, &threads);
      ...
      jvmtiError error = jvmti->ResumeThreadList(threads_count, threads, errors);
                      
                    
  • Use JVMTI to perform a full GC before checkpointing:
  •                   
    jint Java_io_simonis_SuspendResumeAgent_forceGC(JNIEnv *env, jclass cls) {
      jvmtiError error = jvmti->ForceGarbageCollection();
                      
                    

userfaultfd

  • Handling page faults in user space
  • Faulting and fault-handling thread can be in different processes
  • A process creates a userfaultfd object
  • The userfaultfd object can be used (via ioctl()) to:
    • Registers memory ranges
    • Handle page faults
  • Useful for:
    • Virtual Machine live migration
    • Implementing GCs (faster than mprotect()/SIGSEGV handler)

FireCRaCer/userfaultfd   DEMO

References

  • Firecracker: Lightweight Virtualization for Serverless Applications
  • Firecracker Design
  • Firecracker internals: a deep dive inside the technology powering AWS Lambda
  • Firecracker snapshot support
  • Firecracker ballooning support
  • OpenJDK Project: Coordinated Restore at Checkpoint
  • CRIU
  • Userfaultfd kernel documentation
  • Kernel Samepage Merging (KSM)
     
  • FireCRaCer scripts and UffdVisualizer: https://github.com/simonis/fireCRaCer

https://simonis.github.io/jProfessionals2023/