Cannot deserialize continuation with classes originating from a different classloader

[jgomer2001]

Hi,

I'm making use of serializable continuations (version 1.7.14)

I have several Java classes that belong to a classloader other than the main classloader of my application. Sometimes, instances that belong to such classes are part of a computation that I need to pause and resume later.

Serialization of continuations work but when deserializing, I get a ClassNotFoundException at the custom implementation of readObject in NativeJavaObject class:

Caused by: java.lang.ClassNotFoundException: my.class.name
	at java.net.URLClassLoader.findClass(URLClassLoader.java:471) ~[?:?]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:589) ~[?:?]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:522) ~[?:?]
	at org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:544) ~[?:?]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:522) ~[?:?]
	at java.lang.Class.forName0(Native Method) ~[?:?]
	at java.lang.Class.forName(Class.java:398) ~[?:?]
	at java.io.ObjectInputStream.resolveClass(ObjectInputStream.java:756) ~[?:?]
	at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1995) ~[?:?]
	at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1862) ~[?:?]
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2169) ~[?:?]
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1679) ~[?:?]
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2464) ~[?:?]
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2358) ~[?:?]
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2196) ~[?:?]
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1679) ~[?:?]
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:493) ~[?:?]
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:451) ~[?:?]
	at org.mozilla.javascript.NativeJavaObject.readObject(NativeJavaObject.java:882) ~[rhino-1.7.14.jar:1.7.14]

I tried to implement a "wrapping" serialization strategy by subclassing ObjectOutputStream and ObjectInputStream, and overriding replaceObject​ and resolveObject​ respectively so that NativeJavaObject's writeObject and readObject are not called and instead some custom code is executed.

I verified my custom code was actually called, but for my surprise readObject was still being called at a later stage:

Caused by: java.lang.NullPointerException
	at org.mozilla.javascript.ScriptableObject.getTopLevelScope(ScriptableObject.java:1941) ~[rhino-1.7.14.jar:1.7.14]
	at org.mozilla.javascript.ScriptableObject.getTopScopeValue(ScriptableObject.java:2415) ~[rhino-1.7.14.jar:1.7.14]
	at org.mozilla.javascript.ClassCache.get(ClassCache.java:72) ~[rhino-1.7.14.jar:1.7.14]
	at org.mozilla.javascript.JavaMembers.lookupClass(JavaMembers.java:769) ~[rhino-1.7.14.jar:1.7.14]
	at org.mozilla.javascript.NativeJavaObject.initMembers(NativeJavaObject.java:63) ~[rhino-1.7.14.jar:1.7.14]
	at org.mozilla.javascript.NativeJavaObject.readObject(NativeJavaObject.java:892) ~[rhino-1.7.14.jar:1.7.14]
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
	at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
	at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1175) ~[?:?]
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2325) ~[?:?]
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2196) ~[?:?]
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1679) ~[?:?]
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2464) ~[?:?]
	at java.io.ObjectInputStream.defaultReadObject(ObjectInputStream.java:629) ~[?:?]
	at org.mozilla.javascript.Slot.readObject(Slot.java:53) ~[rhino-1.7.14.jar:1.7.14]
	at jdk.internal.reflect.GeneratedMethodAccessor6.invoke(Unknown Source) ~[?:?]
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
	at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
	at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1175) ~[?:?]
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2325) ~[?:?]
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2196) ~[?:?]
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1679) ~[?:?]
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:493) ~[?:?]
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:451) ~[?:?]
	at org.mozilla.javascript.ScriptableObject.readObject(ScriptableObject.java:2641) ~[rhino-1.7.14.jar:1.7.14]

I'd be grateful if you can suggest a workaround for my case.

Thank you.

[p-bakker]

@szegedi @michbarsinai Happen to know the answer to this question?

[michbarsinai]

Not sure how you'd go about this inside Rhino (e.g. pre-populate the class cache to the like).
Using streams only, and assuming that's possible, you could use stubs. Upon serialization, create a stub that contains all the info needed to re-create the non-serializable object (that's the part I'm not sure about). On de-serialization, re-create these objects from the serialized stubs.

[szegedi]

It's always been one of the problems with serialization that upon deserialization, you need to be able to resolve class names to class objects – a typical job for a class loader. I can only recommend that you come up with a solution where class loader hierarchy used in deserialization is compatible to that of the one used in serialization, and you try to ensure that all your serialized runtime classes can be dereferenced by name in it. If you can't do that, you can still decide to write a custom class loader that consults several independent class loaders when loading a class (effectively, having multiple parents.) This might or might not cause some additional unwanted consequences.

FWIW, back in 2005-2010 timeframe I wrote a continuation-based serializing distributed script executor that was driving hundreds of thousands of concurrently running long-executing workflow scripts at any given time, and the system even had a feature for on-demand loading and unloading of different coexisting versions of the code (so we could evolve our platform but still allow app developers to upgrade to newer versions of it when it suited them). This was all accomplished with judicious use of phantom references and ordinary simple hierarchies of class loaders. Even for all that sophisticated usage, I still never had to resort to more exotic class loader architectures. I'm not saying "if it was enough for me it should be enough for you", but want to highlight that these can still get you very far, and that if you end up having some much more exotic class loader setups, you might want to look into whether you can reduce it back to something simpler.

[jgomer2001]

Really appreciate your feedback guys! I'm already following a strategy of stubs like Michael pointed out. I'll dive deeper into this and share results here.

[p-bakker]

@michbarsinai @szegedi Tnx for your input!

FWIW, back in 2005-2010 timeframe I wrote a continuation-based serializing distributed script executor that was driving hundreds of thousands of concurrently running long-executing workflow scripts at any given time, and the system even had a feature for on-demand loading and unloading of different coexisting versions of the code (so we could evolve our platform but still allow app developers to upgrade to newer versions of it when it suited them).

@szegedi is https://github.com/szegedi/spring-web-jsflow what you're referring to here?

As part of #954 I'd also like to add a page about Rhino's support for continuations. If any of you is willing to contribute some (basic) documentation, please let me know. Otherwise, I hope I can ask you to review whatever I've come up to when I get around to writting something down related to Continuations

[p-bakker]

@jgomer2001 am leaving this case open as a reminder to add something about this to our documentation, but besides that I don't think there's anything actionable in here, in the sense that I don't see a feature request or bug report in it, correct?

[jgomer2001]

Right, I didn't raise this as a direct request because I know serialization is a pain point and maybe it is more convenient for the project maintainers to re-define how serialization should work in the future as in #704 . Traditional Java serialization feels pretty uncomfortable.

I'll dive deeper into this and share results here.

I was able to make something work, see: https://gist.github.com/jgomer2001/7acbe479bdffba6bcce8bfbb7122e314

not too beautiful but useful for me by now. This would actually allow me to use another serialization technology (eg. kryo) to some extent