I wrote about the class loader area last time, but I wasn’t quite satisfied, so I’ll investigate a bit more. I didn’t understand the mechanism of Java’s ClassLoader. Let’s start from Stack Overflow’s How does clojure class reloading work?.

Question

I’ve been investigating code and documentation about how class reloading works in Clojure. Many websites, such as http://tutorials.jenkov.com/java-reflection/dynamic-class-loading-reloading.html, indicate that loading a class essentially involves obtaining a byte sequence from any data structure, converting it to an instance of class Class using defineClass, and resolving (linking) that class using resolveClass. (Does defineClass implicitly call resolveClass?) Any classloader limits linking a class to only once. If you try to link to an existing class, nothing happens. This presents a problem where you can’t link a new class instance, so you need to create a new classloader instance each time you reload a class.

Looking at Clojure, there are multiple ways to define classes:

Anonymous classes: reify proxy
Named classes: deftype defrecord (internally uses deftype) gen-class

Code related to these ultimately reaches clojure/src/jvm/clojure/lang/DynamicClassLoader.java. DynamicClassLoader/defineClass creates a class instance and caches it. When loading and using a class, forName is called, which in turn calls DynamicClassLoader/findClass. DynamicClassLoader/findClass searches the cache and then searches the base class. (Normally, classloaders search the base class first, which is the opposite.)

The source of confusion is:

  • The documentation states that forName returns before linking the class
  • This means existing DynamicClassLoaders can’t reload classes
  • Instead, you must create a new instance of DynamicClassLoader, but I can’t find that code

Since proxy and reify are anonymous classes, it’s not a problem even if the class names differ. However, for named classes, that’s not the case.

Please explain the mechanism of DynamicClassLoader. Ultimately, I want to load and reload .class files compiled with javac.

Answer

Not all of the following use the same technique.

  • proxy

The proxy macro creates a named class from the base class and interfaces. Each method is treated as a Clojure fn within the instance, so regardless of whether the macro internals are the same, the same proxy class is used when the same interface is inherited. Actual class reloading does not occur.

  • reify

In the case of reify, the method body is compiled directly into the class, so the proxy trick can’t be used. Instead, a new class is created each time the form is compiled, so even if you edit and reload the body, a completely new class (with a new name) is created. This also doesn’t involve actual class reloading.

  • gen-class

With gen-class, you specify the class name, so it differs from proxy and reify. The gen-class macro only has a specification (spec) and no method body. This means it references a Clojure fn, similar to proxy. However, unlike proxy, the class name is tied to the spec, so you can’t modify the body and reload. Therefore, gen-class is only available with AOT and requires JVM restart.

  • deftype and defrecord

These actually perform dynamic class reloading. When compiling code containing the class or when forName is called, such as when the class name needs to be resolved, DynamicClassLoader/findClass is called.

(deftype T [a b])  ; define an original class named T
(def x (T. 1 2))   ; create an instance of the original class
(deftype T [a b])  ; load a new class by the same name
(cast T x)         ; cast the old instance to the new class -- fails
; ClassCastException   java.lang.Class.cast (Class.java:2990)

For a new class definition, Clojure creates a new DynamicClassLoader at the top-level form. This is not only for deftype and defrecord, but also for reify and fn.

(.getClassLoader (class x))
;=> #<DynamicClassLoader clojure.lang.DynamicClassLoader@337b4703>

(.getClassLoader (class (T. 3 4)))
;=> #<DynamicClassLoader clojure.lang.DynamicClassLoader@451c0d60>

If you haven’t defined a new T class, they have the same classloader.

(.getClassLoader (class (T. 4 5)))
;=> #<DynamicClassLoader clojure.lang.DynamicClassLoader@451c0d60>

Summary

Furthermore, this discussion continued on the ML, and I was able to understand that an instance of DynamicClassLoader is created each time a class is loaded. Actually, I can see the stack structure in the DynamicClassLoader code, so it wasn’t that mysterious. Still, having so many ways to define classes is confusing. I’m starting to think it would be faster to buy a book. Something like “Programming Clojure”.

(ns test.core
  (:use [clojure.stacktrace])
  (:require [clojure.contrib.io :as io]))

(def class-name "hello")
(def file-path "src/test/hello.class")

(defn get-byte-array
  [file-path]
  (io/to-byte-array
    (io/file file-path)))

(defn reload-class
  [class-name file-path]
  (.defineClass (clojure.lang.DynamicClassLoader.)
                class-name
                (get-byte-array file-path)
                nil))

;(reload-class class-name file-path)

(def x (reload-class class-name file-path))
(.getName x)
(def xo (.newInstance x))

;; Modify hello.java and recompile

(def y (reload-class class-name file-path))
(.getName y)
(def yo (.newInstance y))

(.hi xo)
(.hi yo)