jsr367-experts@jsonb-spec.java.net

[jsr367-experts] Re: [jsonb-spec users] Re: Re: Re: Re: Re: Java polymorphism support

From: Romain Manni-Bucau <rmannibucau_at_tomitribe.com>
Date: Thu, 24 Mar 2016 12:09:31 +0100

Am I right saying it uses a proprietary API to work?

If so I think my JsonbGenerator/JsonbReader are a standardization of such
API and what Mark proposed as well. Then maybe we can discuss their
integration in the spec?
Le 24 mars 2016 12:03, "Roman Grigoriadi" <roman.grigoriadi_at_oracle.com> a
écrit :

> Hi,
>
> @Romain - I added a test -
> org.eclipse.persistence.json.bind.adapters.PolymorphismAdapterTest.
> It uses these pojos:
>
> public static class Animal {
> public String name;}
> public static class Dog extends Animal {
> public String dogProperty;}
> public static class Cat extends Animal {
> public String catProperty;}
> public static class Pojo {
> public Animal animal; public List<Animal> listOfAnimals = new ArrayList<>();}
>
> and generates this JSON for accordingly instantiated pojo:
>
> {
> "animal":{
> "className":"org.eclipse.persistence.json.bind.adapters.PolymorphismAdapterTest$Dog",
> "instance":{
> "name":"Ralph",
> "dogProperty":"Property of a Dog."
> }
> },
> "listOfAnimals":[
> {
> "className":"org.eclipse.persistence.json.bind.adapters.PolymorphismAdapterTest$Cat",
> "instance":{
> "name":"Snowball",
> "catProperty":"Property of a Cat."
> }
> },
> {
> "className":"org.eclipse.persistence.json.bind.adapters.PolymorphismAdapterTest$Dog",
> "instance":{
> "name":"Ralph",
> "dogProperty":"Property of a Dog."
> }
> }
> ]
> }
>
> An org.eclipse.persistence.json.bind.internal.unmarshaller.TypeWrapperItem
> class is responsible for
> *"how "Jsonb handles such event with creating instance of appropriate
> type"". *Please see it at eclipselink repo
> <http://git.eclipse.org/c/eclipselink/eclipselink.runtime.git>, master
> branch. The Jsonb project is in eclipselink-runtime/jsonb folder.
>
> I don't know what are the internals of Johnzon, but I suppose it will also
> include some custom handling for specified attribute in case of TypeWrapper
> class. Maybe I just missed to specify explicitly in previous communication,
> that I have to add this custom handling in JSONB. Of course it will never
> work without it..
>
> Regards,
> Roman
>
> On 03/23/2016 02:26 PM, Romain Manni-Bucau wrote:
>
> Will wait for the test cause I really don't see how "Jsonb handles such
> event with creating instance of appropriate type" but wanted to share the
> read before write solution works even if not elegant using the
> typewrapper/adatper we spoke about:
>
>
> public void ploymorphism() { // we run it since it checked list/item
> conversion
>
> final Bar bar = new Bar(); // ~your Animal
>
> bar.value = 11;
>
>
> final Bar2 bar2 = new Bar2();
>
> bar2.value = 21;
>
> bar2.value2 = 22;
>
>
> final Polymorphism foo = new Polymorphism(); // the pojo with a
> collection of bars
>
> foo.bars = new ArrayList<>(asList(bar, bar2));
>
>
> final Jsonb jsonb = JsonbBuilder.create(
>
> new JsonbConfig()
>
> .setProperty("johnzon.readAttributeBeforeWrite", true) // NOT STANDARD,
> enforces to use the getter to determine the type to deserialize
>
> .withPropertyOrderStrategy(PropertyOrderStrategy.LEXICOGRAPHICAL) /*
> assertEquals() order */);
>
>
> final String toString = jsonb.toJson(foo);
>
> assertEquals("{\"bars\":[" +
>
> "{\"type\":\"org.apache.johnzon.jsonb.AdapterTest$Bar\",\"value\":{\"value
> \":11}}," +
>
> "{\"type\":\"org.apache.johnzon.jsonb.AdapterTest$Bar2\",\"value\":{\"
> value\":21,\"value2\":22}}]}", toString);
>
>
> final Polymorphism read = jsonb.fromJson(toString, Polymorphism.class);
>
> assertEquals(2, read.bars.size());
>
> assertEquals(11, read.bars.get(0).value);
>
> assertTrue(Bar.class == read.bars.get(0).getClass());
>
> assertEquals(21, read.bars.get(1).value);
>
> assertTrue(Bar2.class == read.bars.get(1).getClass());
>
> assertEquals(22, Bar2.class.cast(read.bars.get(1)).value2);
>
> }
>
>
>
>
> 2016-03-23 14:18 GMT+01:00 Roman Grigoriadi <roman.grigoriadi_at_oracle.com>:
>
>>
>>
>> On 03/23/2016 02:02 PM, Romain Manni-Bucau wrote:
>>
>>
>>
>> 2016-03-23 13:39 GMT+01:00 Roman Grigoriadi <roman.grigoriadi_at_oracle.com>
>> :
>>
>>> I think [A] is misunderstood..
>>>
>>>
>>>
>>> *[A] Think I get it: the adapter will return Foo and then on this
>>> instance Jsonb will populate fields. * That is not what I ment. An
>>> adapter is at the end of the process, T instance is already populated both:
>>> when adapter adapts it returns adapted result.
>>> Above statement is maybe enough, but for sake of clarity let me revisit
>>> whole process.
>>>
>>> 1. Lets have JSON document for a TypeWrapper<T>
>>> {
>>> "animal":{
>>> "className":"com.foo.Dog",
>>> "instance":{
>>> "name":"Doggie",
>>> "dogProperty":"Property of a Dog."
>>> }
>>> }
>>> }
>>>
>>> During deserialization what happens first is a completed instance of a
>>> TypeWrapper with a a fully populated T inside. This happens in following
>>> steps:
>>> 1. Jsonb encounters deserializing a TypeWrapper<T>, and processes it
>>> custom TypeWrapperDeserializer.
>>> 2. TypeWrapperDeserializer (implemented by us), creates an empty
>>> instance of TypeWrapper.
>>> 3. Jsonb reads and sets property "className" into TypeWrapper
>>> 4. Jsonb loads a class by this classname and creates instance with
>>> default construcotor in our case Dog or Cat and sets it to TypeWrapper
>>> instance.
>>>
>>
>> That's where I'm lost. Only way it could work - without putting this
>> behavior in the spec - is if TypeWrapper overrides the setter of classname
>> to also set the instance:
>>
>> public void setClassName(final String className) {
>> this.className= className;
>> try {
>> this.value = Animal.class.cast(Thread.currentThread().getContextClassLoader().loadClass(className).newInstance());
>> } catch (final InstantiationException | IllegalAccessException | ClassNotFoundException e) {
>> throw new IllegalArgumentException(e);
>> }
>> }
>>
>> Nope, it is only the className set as a String in this moment.
>> Next JsonParser enconters a JsonObject and pushes ObjectStarted event
>> into JSONB. Jsonb handles such event with creating instance of appropriate
>> type. That type is normally read from ClassModel (which is read from actual
>> Field or ParameterizedType argument of a Collection). But in our case this
>> type is swapped with a Class loaded dynamically by name.
>>
> So now we have a correct instance ready to be populated by common
>> reflection deserialization.
>>
>> And then Jsonb before creating the Animal instance reads the field to
>> extract its type which is likely not what is done since the type is deduced
>> statically from the model class (field type) - this part is important to
>> avoid surprises and open security issues.
>>
>> No need to create instance now, it is already fully populated. Just
>> extract it from TypeWrapper as is.
>>
>>
>> If I'm still wrong do you care writing a small test case showing that in
>> action please?
>>
>> Sure I will implement a test and push it to our eclipselink git which
>> hosts the jsonb-ri.
>>
>>
>> 5. Jsonb parses a "ClassModel" for Dog or Cat class (reading all @Jsonb
>>> annots etc if needed).
>>> 6. Jsonb goes deeper in json document parsing json object for "instance"
>>> property and populating all fields in Dog or Cat (doesn't really care what
>>> is it as it is common reflection based process for all properties inside).
>>> 7. After above is completed an instance of TypeWrapper with fully
>>> deserialized T inside is passed to an adapter method "T
>>> adaptFromJson(TypeWrapper<T> obj)".
>>> 8. Result from adapter is than set to a field, setter, collection, map,
>>> array or whatever else is declared in domain model to unmarshall into.
>>>
>>> What returns an adapter is already ready to be set into model as is with
>>> polymorphism working this way.
>>>
>>> Now I suppose this is exactly what:
>>> [A]....*"Thought adapter should return a populated instance otherwise
>>> no way to convert to business model properly IMO"* means ?
>>>
>>> Hope this helps and will bring some light.
>>>
>>> Thanks,
>>> Roman
>>>
>>>
>>>
>>> On 03/23/2016 12:49 PM, Romain Manni-Bucau wrote:
>>>
>>>
>>> 2016-03-23 12:29 GMT+01:00 Roman Grigoriadi <roman.grigoriadi_at_oracle.com
>>> >:
>>>
>>>> You load a Dog or Cat class by name, which appears in json and
>>>> instantiate it with default constructor. This is what I suppose a custom
>>>> deserializer for TypeWrapper would do.
>>>>
>>>>
>>> [A] Think I get it: the adapter will return Foo and then on this
>>> instance Jsonb will populate fields. Thought adapter should return a
>>> populated instance otherwise no way to convert to business model properly
>>> IMO (if you don't have an explicit model you maybe don't want a setter or
>>> field to be used). That's why I was kind of blocked there.
>>>
>>>
>>>> If you want to polymorphically adapt each animal in a collection you
>>>> would have instead annotate Animal.class with @JsonbTypeAdapter, or pass an
>>>> instance of AnimalAdapter to JsonbConfig.
>>>>
>>>> You are right, you can't add @JsonbTypeAdapter(AnimalAdapter.class) to
>>>> List<Animal> as it would than have to be declared as: "extends
>>>> TypeAdapter<List<Animal>>" and parent implementation would not work than.
>>>>
>>>>
>>> Well adapter for items instead of the field in case of a collection is
>>> something we can implement - we do in Johnzon - and which is useful IMO.
>>> That said it doesn't solve the polymorphism issue. If [A] is solved it can
>>> work but today we don't have any way to know if the adapter builds the
>>> instance or just instantiate it if I got you right.
>>>
>>>
>>>> Did I get your point?
>>>>
>>>>
>>> Think so.
>>>
>>>
>>>> Roman.
>>>>
>>>>
>>>> On 03/23/2016 12:04 PM, Romain Manni-Bucau wrote:
>>>>
>>>> I don't get this part "Jsonb looks at animal field model and gets
>>>> "toType" from annotated adapter".
>>>>
>>>> How do you create a Dog in your example instead of an Animal.
>>>>
>>>> Keep in mind you don't know all types from jsonb point of view. The
>>>> real use case is more obvious on a collection - at least for me:
>>>>
>>>> public static class Pojo {
>>>> @JsonbTypeAdapter(AnimalAdapter.class)
>>>> public Collection<Animal> animals; // Dog, Cow, Cat, Horse...
>>>> }
>>>>
>>>>
>>>>
>>>> Romain Manni-Bucau
>>>> @rmannibucau
>>>> http://www.tomitribe.com
>>>> http://rmannibucau.wordpress.com
>>>> https://github.com/rmannibucau
>>>>
>>>> 2016-03-23 11:51 GMT+01:00 Roman Grigoriadi <
>>>> roman.grigoriadi_at_oracle.com>:
>>>>
>>>>> You don't instantiate X in adapter you already have it. Here is
>>>>> implementation of such adapter:
>>>>>
>>>>> class TypeAdapter<T> implements JsonbAdapter<T, TypeWrapper<T>> {
>>>>>
>>>>> @Override
>>>>> public TypeWrapper<T> adaptToJson(T obj) throws Exception {
>>>>> TypeWrapper<T> wrapper = new TypeWrapper<>();
>>>>> wrapper.setClassName(obj.getClass().getName());
>>>>> wrapper.setInstance(obj);
>>>>> return wrapper;
>>>>> }
>>>>>
>>>>> @Override
>>>>> public T adaptFromJson(TypeWrapper<T> obj) throws Exception {
>>>>> return obj.getInstance();
>>>>> }
>>>>> }
>>>>>
>>>>> class AnimalAdapter extends TypeAdapter<Animal> {}
>>>>>
>>>>> Than you have a pojo like this:
>>>>>
>>>>> public static class Pojo {
>>>>> @JsonbTypeAdapter(AnimalAdapter.class)
>>>>> public Animal animal;
>>>>> }
>>>>>
>>>>> And json like this:
>>>>>
>>>>> {
>>>>> "animal":{"className":"com.foo.Dog",name":"Doggie","dogProperty":"Property of a Dog."}}
>>>>> }
>>>>>
>>>>>
>>>>> What happens during deserialization in our implementation is:
>>>>> JsonParser generates an event for started object at "animal" property.
>>>>> Jsonb looks at animal field model and gets "toType" from annotated adapter.
>>>>> Deserialization than happens for TypedWrapper as for common arbitrary user
>>>>> Type, but with custom deserializer, reading class property first and
>>>>> creating instance from it. After TypedWrapper<Animal> is fully
>>>>> deserialized, it is passed to adapter instance, to retrieve an Animal,
>>>>> which is finally set to Animal field.
>>>>>
>>>>> Regards,
>>>>> Roman
>>>>>
>>>>>
>>>>> On 03/23/2016 11:20 AM, Romain Manni-Bucau wrote:
>>>>>
>>>>> Hi Roman,
>>>>>
>>>>> can you fill some "..." please? typically at the moment "I know the
>>>>> type is X". How with an adapter do you instantiate X and then let jsonb
>>>>> populate it (without having another Jsonb instance)? That was the main
>>>>> missing part of the puzzle.
>>>>>
>>>>>
>>>>> Romain Manni-Bucau
>>>>> @rmannibucau
>>>>> http://www.tomitribe.com
>>>>> http://rmannibucau.wordpress.com
>>>>> https://github.com/rmannibucau
>>>>>
>>>>> 2016-03-23 11:14 GMT+01:00 Roman Grigoriadi <
>>>>> roman.grigoriadi_at_oracle.com>:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> @Romain - what you and Dmitry suggested with adapters, will actully
>>>>>> work in our case. We will have to to add a custom deserializer handler for
>>>>>> a "TypedWrapper<T>" (similar to collection / array or reflection
>>>>>> deserializers), which will read a class property first and on
>>>>>> encountering a json object for T will create a correct type instance. The
>>>>>> only condition would be, that class property must appear before inner
>>>>>> instance in json object for "TypedWrapper<T>" as Eugen suggested:
>>>>>>
>>>>>> {"classProperty":"com.foo.Dog","instance":{...}}
>>>>>>
>>>>>> Adapters will than work in a common way to extract T from
>>>>>> TypedWrapper and set it to a class model be it a field or a
>>>>>> collection/array.
>>>>>>
>>>>>> I would also add required enumeration of allowed subtypes into
>>>>>> TypedAdapter:
>>>>>>
>>>>>> class TypedAdapter<T> implements JsonbAdapter<T, TypedWrapper<T>> {
>>>>>> TypedAdapter(Class[] allowedSubtypes) {...}
>>>>>> ...
>>>>>> }
>>>>>>
>>>>>> So we don't allow loading of any arbitrary class suggested by
>>>>>> incoming json in case of not trusted third party.
>>>>>>
>>>>>> @Romain - adding support for custom serializers / deserializers in
>>>>>> fashion of fasterxml with access to JsonGenerator/JsonParser would also
>>>>>> solve the problem in similar fashion, but it would be more complicated for
>>>>>> a client to implement logic driven by JsonParser events, than by providing
>>>>>> predefined adapter.
>>>>>>
>>>>>> Thanks,
>>>>>> Roman
>>>>>>
>>>>>> On 03/21/2016 09:36 PM, Romain Manni-Bucau wrote:
>>>>>>
>>>>>> Hi Dmitry,
>>>>>>
>>>>>> was my thought as well but it doesn't really work cause in adapter
>>>>>> you can't relaunch the deserialization chain with the same jsonb instance.
>>>>>> The serialization side is the easy one but the deserialization makes
>>>>>> adapters API not enough to have a consistent behavior accross the
>>>>>> deserialization. Making the adapter "JsonbAware" would work. We can
>>>>>> probably use @Inject (CDI if there or just the JSR 330 as an optional
>>>>>> dependency if not)
>>>>>>
>>>>>> Side note: I'd like we enable this impl but we don't provide them in
>>>>>> the spec since it is likely dependent on your use case and saw cases where
>>>>>> type was not enough and we will likely not handle all possible models.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Romain Manni-Bucau
>>>>>> @rmannibucau
>>>>>> http://www.tomitribe.com
>>>>>> http://rmannibucau.wordpress.com
>>>>>> https://github.com/rmannibucau
>>>>>>
>>>>>> 2016-03-21 21:27 GMT+01:00 Dmitry Kornilov <
>>>>>> dmitry.kornilov_at_oracle.com>:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> This topic has been discussed already. Current version of the spec
>>>>>>> is clear about polymorphism: *“**Deserialization into polymorphic
>>>>>>> types is not supported by default mapping. [JSB-3.8-1]”. *The spec
>>>>>>> is almost finished and I am strongly against going back and changing some
>>>>>>> basic things now.
>>>>>>>
>>>>>>> On the other hand I understand that this use case exists. There is a
>>>>>>> possible workaround to it using adapters. Sorry if there are any mistakes,
>>>>>>> I am writing code without checking.
>>>>>>>
>>>>>>> We can create a generic wrapper to any type holding its type info
>>>>>>> like this:
>>>>>>>
>>>>>>> class TypedWrapper<T> {
>>>>>>> String type;
>>>>>>> T obj;
>>>>>>> }
>>>>>>>
>>>>>>> We can create a generic adapter which converts type to wrapper and
>>>>>>> vice versa. I am not putting a full implementation here, I hope that the
>>>>>>> idea is clear:
>>>>>>>
>>>>>>> class TypedAdapter implements JsonbAdapter<T, TypedWrapper<T>> {
>>>>>>> TypedWrapper adaptToJson(T obj) {
>>>>>>> TypedWrapper<T> t = new TypedWrapper<T>();
>>>>>>> t.type = obj.getClass().getName();
>>>>>>> t.obj = obj;
>>>>>>> return t;
>>>>>>> }
>>>>>>>
>>>>>>> T adaptFromJson(TypedWrapper<T> t) {
>>>>>>> // Use reflection API to create an instance of object based
>>>>>>> on t.type field value
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>> Now lets imagine that we have the following classes:
>>>>>>>
>>>>>>> class Animal {
>>>>>>> }
>>>>>>>
>>>>>>> class Dog extends Animal {
>>>>>>> }
>>>>>>>
>>>>>>> class Cat extends Animal {
>>>>>>> }
>>>>>>>
>>>>>>> class Foo {
>>>>>>> Animal animal;
>>>>>>> }
>>>>>>>
>>>>>>> To properly handle serialization/deserialization of Foo we just need
>>>>>>> to create an adapter which extends TypedAdapter from ab
>>>>>>>
>>>>>> ...