jsr367-experts@jsonb-spec.java.net

[jsr367-experts] Re: [jsonb-spec users] Re: Re: Re: Re: Re: Re: Java polymorphism support

From: Roman Grigoriadi <roman.grigoriadi_at_oracle.com>
Date: Thu, 24 Mar 2016 12:14:51 +0100

It depends if we include TypeWrapperAdapter into spec and API. If not,
than yes it would be a proprietary of jsonb-ri.

Roman.

On 03/24/2016 12:09 PM, Romain Manni-Bucau wrote:
>
> Am I right saying it uses a proprietary API to work?
>
> If so I think my JsonbGenerator/JsonbReader are a standardization of
> such API and what Mark proposed as well. Then maybe we can discuss
> their integration in the spec?
>
> Le 24 mars 2016 12:03, "Roman Grigoriadi" <roman.grigoriadi_at_oracle.com
> <mailto:roman.grigoriadi_at_oracle.com>> a écrit :
>
> Hi,
>
> @Romain - I added a test -
> org.eclipse.persistence.json.bind.adapters.PolymorphismAdapterTest.
> It uses these pojos:
>
> public static class Animal {
> public Stringname; }
>
> public static class Dogextends Animal {
> public StringdogProperty; }
>
> public static class Catextends Animal {
> public StringcatProperty; }
>
> public static class Pojo {
> public Animalanimal; public List<Animal>listOfAnimals =new ArrayList<>(); }
>
> and generates this JSON for accordingly instantiated pojo:
>
> { "animal":{
> "className":"org.eclipse.persistence.json.bind.adapters.PolymorphismAdapterTest$Dog",
> "instance":{ "name":"Ralph", "dogProperty":"Property of a Dog." }
> }, "listOfAnimals":[ {
> "className":"org.eclipse.persistence.json.bind.adapters.PolymorphismAdapterTest$Cat",
> "instance":{ "name":"Snowball", "catProperty":"Property of a Cat."
> } }, {
> "className":"org.eclipse.persistence.json.bind.adapters.PolymorphismAdapterTest$Dog",
> "instance":{ "name":"Ralph", "dogProperty":"Property of a Dog." }
> } ] }
>
> An
> org.eclipse.persistence.json.bind.internal.unmarshaller.TypeWrapperItem
> class is responsible for /"how "Jsonb handles such event with
> creating instance of appropriate type"".
> /Please see it at eclipselink repo
> <http://git.eclipse.org/c/eclipselink/eclipselink.runtime.git>,
> master branch. The Jsonb project is in eclipselink-runtime/jsonb
> folder.
> /
> /I don't know what are the internals of Johnzon, but I suppose it
> will also include some custom handling for specified attribute in
> case of TypeWrapper class. Maybe I just missed to specify
> explicitly in previous communication, that I have to add this
> custom handling in JSONB. Of course it will never work without it..
>
> Regards,
> Roman
>
> On 03/23/2016 02:26 PM, Romain Manni-Bucau wrote:
>> Will wait for the test cause I really don't see how "Jsonb
>> handles such event with creating instance of appropriate type"
>> but wanted to share the read before write solution works even if
>> not elegant using the typewrapper/adatper we spoke about:
>>
>>
>> public void ploymorphism() { // we run it since it checked
>> list/item conversion
>>
>> final Bar bar = new Bar(); // ~your Animal
>>
>> bar.value = 11;
>>
>>
>>
>> final Bar2 bar2 = new Bar2();
>>
>> bar2.value = 21;
>>
>> bar2.value2 = 22;
>>
>>
>>
>> final Polymorphism foo = new Polymorphism(); // the pojo with a
>> collection of bars
>>
>> foo.bars = new ArrayList<>(asList(bar, bar2));
>>
>>
>>
>> final Jsonb jsonb = JsonbBuilder.create(
>>
>> new JsonbConfig()
>>
>> .setProperty("johnzon.readAttributeBeforeWrite", true) // NOT
>> STANDARD, enforces to use the getter to determine the type to
>> deserialize
>>
>> .withPropertyOrderStrategy(PropertyOrderStrategy.LEXICOGRAPHICAL) /* assertEquals() order */);
>>
>>
>>
>> final String toString = jsonb.toJson(foo);
>>
>> assertEquals("{\"bars\":[" +
>>
>> "{\"type\":\"org.apache.johnzon.jsonb.AdapterTest$Bar\",\"value\":{\"value\":11}}," +
>>
>> "{\"type\":\"org.apache.johnzon.jsonb.AdapterTest$Bar2\",\"value\":{\"value\":21,\"value2\":22}}]}", toString);
>>
>>
>>
>> final Polymorphism read = jsonb.fromJson(toString,
>> Polymorphism.class);
>>
>> assertEquals(2, read.bars.size());
>>
>> assertEquals(11, read.bars.get(0).value);
>>
>> assertTrue(Bar.class == read.bars.get(0).getClass());
>>
>> assertEquals(21, read.bars.get(1).value);
>>
>> assertTrue(Bar2.class == read.bars.get(1).getClass());
>>
>> assertEquals(22, Bar2.class.cast(read.bars.get(1)).value2);
>>
>> }
>>
>>
>>
>>
>> 2016-03-23 14:18 GMT+01:00 Roman Grigoriadi
>> <roman.grigoriadi_at_oracle.com <mailto:roman.grigoriadi_at_oracle.com>>:
>>
>>
>>
>> On 03/23/2016 02:02 PM, Romain Manni-Bucau wrote:
>>>
>>>
>>> 2016-03-23 13:39 GMT+01:00 Roman Grigoriadi
>>> <roman.grigoriadi_at_oracle.com
>>> <mailto:roman.grigoriadi_at_oracle.com>>:
>>>
>>> I think [A] is misunderstood..
>>>
>>> /[A] Think I get it: the adapter will return Foo and
>>> then on this instance Jsonb will populate fields.
>>>
>>> / That is not what I ment. An adapter is at the end of
>>> the process, T instance is already populated both: when
>>> adapter adapts it returns adapted result.
>>> Above statement is maybe enough, but for sake of clarity
>>> let me revisit whole process.
>>>
>>> 1. Lets have JSON document for a TypeWrapper<T>
>>> {
>>> "animal":{
>>> "className":"com.foo.Dog",
>>> "instance":{
>>> "name":"Doggie",
>>> "dogProperty":"Property of a Dog."
>>> }
>>> }
>>> }
>>>
>>> During deserialization what happens first is a completed
>>> instance of a TypeWrapper with a a fully populated T
>>> inside. This happens in following steps:
>>> 1. Jsonb encounters deserializing a TypeWrapper<T>, and
>>> processes it custom TypeWrapperDeserializer.
>>> 2. TypeWrapperDeserializer (implemented by us), creates
>>> an empty instance of TypeWrapper.
>>> 3. Jsonb reads and sets property "className" into
>>> TypeWrapper
>>> 4. Jsonb loads a class by this classname and creates
>>> instance with default construcotor in our case Dog or
>>> Cat and sets it to TypeWrapper instance.
>>>
>>>
>>> That's where I'm lost. Only way it could work - without
>>> putting this behavior in the spec - is if TypeWrapper
>>> overrides the setter of classname to also set the instance:
>>>
>>> public void setClassName(final String className) {
>>> this.className= className;
>>> try {
>>> this.value = Animal.class.cast(Thread.currentThread().getContextClassLoader().loadClass(className).newInstance());
>>> }catch (final InstantiationException | IllegalAccessException | ClassNotFoundException e) {
>>> throw new IllegalArgumentException(e);
>>> }
>>> }
>> Nope, it is only the className set as a String in this moment.
>> Next JsonParser enconters a JsonObject and pushes
>> ObjectStarted event into JSONB. Jsonb handles such event with
>> creating instance of appropriate type. That type is normally
>> read from ClassModel (which is read from actual Field or
>> ParameterizedType argument of a Collection). But in our case
>> this type is swapped with a Class loaded dynamically by name.
>>
>> So now we have a correct instance ready to be populated by
>> common reflection deserialization.
>>> And then Jsonb before creating the Animal instance reads
>>> the field to extract its type which is likely not what is
>>> done since the type is deduced statically from the model
>>> class (field type) - this part is important to avoid
>>> surprises and open security issues.
>> No need to create instance now, it is already fully
>> populated. Just extract it from TypeWrapper as is.
>>>
>>> If I'm still wrong do you care writing a small test case
>>> showing that in action please?
>> Sure I will implement a test and push it to our eclipselink
>> git which hosts the jsonb-ri.
>>>
>>> 5. Jsonb parses a "ClassModel" for Dog or Cat class
>>> (reading all @Jsonb annots etc if needed).
>>> 6. Jsonb goes deeper in json document parsing json
>>> object for "instance" property and populating all fields
>>> in Dog or Cat (doesn't really care what is it as it is
>>> common reflection based process for all properties inside).
>>> 7. After above is completed an instance of TypeWrapper
>>> with fully deserialized T inside is passed to an adapter
>>> method "T adaptFromJson(TypeWrapper<T> obj)".
>>> 8. Result from adapter is than set to a field, setter,
>>> collection, map, array or whatever else is declared in
>>> domain model to unmarshall into.
>>>
>>> What returns an adapter is already ready to be set into
>>> model as is with polymorphism working this way.
>>>
>>> Now I suppose this is exactly what:
>>> [A]..../"Thought adapter should return a populated
>>> instance otherwise no way to convert to business model
>>> properly IMO"/ means ?
>>>
>>> Hope this helps and will bring some light.
>>>
>>> Thanks,
>>> Roman
>>>
>>>
>>>
>>> On 03/23/2016 12:49 PM, Romain Manni-Bucau wrote:
>>>>
>>>> 2016-03-23 12:29 GMT+01:00 Roman Grigoriadi
>>>> <roman.grigoriadi_at_oracle.com
>>>> <mailto:roman.grigoriadi_at_oracle.com>>:
>>>>
>>>> You load a Dog or Cat class by name, which appears
>>>> in json and instantiate it with default
>>>> constructor. This is what I suppose a custom
>>>> deserializer for TypeWrapper would do.
>>>>
>>>>
>>>> [A] Think I get it: the adapter will return Foo and
>>>> then on this instance Jsonb will populate fields.
>>>> Thought adapter should return a populated instance
>>>> otherwise no way to convert to business model properly
>>>> IMO (if you don't have an explicit model you maybe
>>>> don't want a setter or field to be used). That's why I
>>>> was kind of blocked there.
>>>>
>>>> If you want to polymorphically adapt each animal in
>>>> a collection you would have instead annotate
>>>> Animal.class with @JsonbTypeAdapter, or pass an
>>>> instance of AnimalAdapter to JsonbConfig.
>>>>
>>>> You are right, you can't add
>>>> @JsonbTypeAdapter(AnimalAdapter.class) to
>>>> List<Animal> as it would than have to be declared
>>>> as: "extends TypeAdapter<List<Animal>>" and parent
>>>> implementation would not work than.
>>>>
>>>>
>>>> Well adapter for items instead of the field in case of
>>>> a collection is something we can implement - we do in
>>>> Johnzon - and which is useful IMO. That said it doesn't
>>>> solve the polymorphism issue. If [A] is solved it can
>>>> work but today we don't have any way to know if the
>>>> adapter builds the instance or just instantiate it if I
>>>> got you right.
>>>>
>>>> Did I get your point?
>>>>
>>>>
>>>> Think so.
>>>>
>>>> Roman.
>>>>
>>>>
>>>> On 03/23/2016 12:04 PM, Romain Manni-Bucau wrote:
>>>>> I don't get this part "Jsonb looks at animal field
>>>>> model and gets "toType" from annotated adapter".
>>>>>
>>>>> How do you create a Dog in your example instead of
>>>>> an Animal.
>>>>>
>>>>> Keep in mind you don't know all types from jsonb
>>>>> point of view. The real use case is more obvious
>>>>> on a collection - at least for me:
>>>>>
>>>>> public static class Pojo {
>>>>> @JsonbTypeAdapter(AnimalAdapter.class)
>>>>> public Collection<Animal> animals; // Dog, Cow, Cat, Horse...
>>>>> }
>>>>>
>>>>>
>>>>> Romain Manni-Bucau
>>>>> @rmannibucau
>>>>> http://www.tomitribe.com
>>>>> http://rmannibucau.wordpress.com
>>>>> https://github.com/rmannibucau
>>>>>
>>>>> 2016-03-23 11:51 GMT+01:00 Roman Grigoriadi
>>>>> <roman.grigoriadi_at_oracle.com
>>>>> <mailto:roman.grigoriadi_at_oracle.com>>:
>>>>>
>>>>> You don't instantiate X in adapter you already
>>>>> have it. Here is implementation of such adapter:
>>>>>
>>>>> class TypeAdapter<T> implements JsonbAdapter<T, TypeWrapper<T>> {
>>>>>
>>>>> @Override
>>>>> public TypeWrapper<T> adaptToJson(T obj) throws Exception {
>>>>> TypeWrapper<T> wrapper = new TypeWrapper<>();
>>>>> wrapper.setClassName(obj.getClass().getName());
>>>>> wrapper.setInstance(obj);
>>>>> return wrapper;
>>>>> }
>>>>>
>>>>> @Override
>>>>> public T adaptFromJson(TypeWrapper<T> obj) throws Exception {
>>>>> return obj.getInstance();
>>>>> }
>>>>> }
>>>>>
>>>>> class AnimalAdapter extends TypeAdapter<Animal> {}
>>>>>
>>>>> Than you have a pojo like this:
>>>>>
>>>>> public static class Pojo {
>>>>> @JsonbTypeAdapter(AnimalAdapter.class)
>>>>> public Animal animal;
>>>>> }
>>>>>
>>>>> And json like this:
>>>>>
>>>>> {
>>>>> "animal":{"className":"com.foo.Dog",name":"Doggie","dogProperty":"Property of a Dog."}}
>>>>> }
>>>>>
>>>>>
>>>>> What happens during deserialization in our
>>>>> implementation is: JsonParser generates an
>>>>> event for started object at "animal" property.
>>>>> Jsonb looks at animal field model and gets
>>>>> "toType" from annotated adapter.
>>>>> Deserialization than happens for TypedWrapper
>>>>> as for common arbitrary user Type, but with
>>>>> custom deserializer, reading class property
>>>>> first and creating instance from it. After
>>>>> TypedWrapper<Animal> is fully deserialized, it
>>>>> is passed to adapter instance, to retrieve an
>>>>> Animal, which is finally set to Animal field.
>>>>>
>>>>> Regards,
>>>>> Roman
>>>>>
>>>>>
>>>>> On 03/23/2016 11:20 AM, Romain Manni-Bucau wrote:
>>>>>> Hi Roman,
>>>>>>
>>>>>> can you fill some "..." please? typically at
>>>>>> the moment "I know the type is X". How with
>>>>>> an adapter do you instantiate X and then let
>>>>>> jsonb populate it (without having another
>>>>>> Jsonb instance)? That was the main missing
>>>>>> part of the puzzle.
>>>>>>
>>>>>>
>>>>>> Romain Manni-Bucau
>>>>>> @rmannibucau
>>>>>> http://www.tomitribe.com
>>>>>> http://rmannibucau.wordpress.com
>>>>>> https://github.com/rmannibucau
>>>>>>
>>>>>> 2016-03-23 11:14 GMT+01:00 Roman Grigoriadi
>>>>>> <roman.grigoriadi_at_oracle.com
>>>>>> <mailto:roman.grigoriadi_at_oracle.com>>:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> @Romain - what you and Dmitry suggested
>>>>>> with adapters, will actully work in our
>>>>>> case. We will have to to add a custom
>>>>>> deserializer handler for a
>>>>>> "TypedWrapper<T>" (similar to collection
>>>>>> / array or reflection deserializers),
>>>>>> which will read a class property first
>>>>>> and on encountering a json object for T
>>>>>> will create a correct type instance. The
>>>>>> only condition would be, that class
>>>>>> property must appear before inner
>>>>>> instance in json object for
>>>>>> "TypedWrapper<T>" as Eugen suggested:
>>>>>>
>>>>>> {"classProperty":"com.foo.Dog","instance":{...}}
>>>>>>
>>>>>> Adapters will than work in a common way
>>>>>> to extract T from TypedWrapper and set it
>>>>>> to a class model be it a field or a
>>>>>> collection/array.
>>>>>>
>>>>>> I would also add required enumeration of
>>>>>> allowed subtypes into TypedAdapter:
>>>>>>
>>>>>> class TypedAdapter<T> implements
>>>>>> JsonbAdapter<T, TypedWrapper<T>> {
>>>>>> TypedAdapter(Class[] allowedSubtypes) {...}
>>>>>> ...
>>>>>> }
>>>>>>
>>>>>> So we don't allow loading of any
>>>>>> arbitrary class suggested by incoming
>>>>>> json in case of not trusted third party.
>>>>>>
>>>>>> @Romain - adding support for custom
>>>>>> serializers / deserializers in fashion of
>>>>>> fasterxml with access to
>>>>>> JsonGenerator/JsonParser would also solve
>>>>>> the problem in similar fashion, but it
>>>>>> would be more complicated for a client to
>>>>>> implement logic driven by JsonParser
>>>>>> events, than by providing predefined
>>>>>> adapter.
>>>>>>
>>>>>> Thanks,
>>>>>> Roman
>>>>>>
>>>>>> On 03/21/2016 09:36 PM, Romain
>>>>>> Manni-Bucau wrote:
>>>>>>> Hi Dmitry,
>>>>>>>
>>>>>>> was my thought as well but it doesn't
>>>>>>> really work cause in adapter you can't
>>>>>>> relaunch the deserialization chain with
>>>>>>> the same jsonb instance. The
>>>>>>> serialization side is the easy one but
>>>>>>> the deserialization makes adapters API
>>>>>>> not enough to have a consistent behavior
>>>>>>> accross the deserialization. Making the
>>>>>>> adapter "JsonbAware" would work. We can
>>>>>>> probably use @Inject (CDI if there or
>>>>>>> just the JSR 330 as an optional
>>>>>>> dependency if not)
>>>>>>>
>>>>>>> Side note: I'd like we enable this impl
>>>>>>> but we don't provide them in the spec
>>>>>>> since it is likely dependent on your use
>>>>>>> case and saw cases where type was not
>>>>>>> enough and we will likely not handle all
>>>>>>> possible models.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Romain Manni-Bucau
>>>>>>> @rmannibucau
>>>>>>> http://www.tomitribe.com
>>>>>>> http://rmannibucau.wordpress.com
>>>>>>> https://github.com/rmannibucau
>>>>>>>
>>>>>>> 2016-03-21 21:27 GMT+01:00 Dmitry
>>>>>>> Kornilov <dmitry.kornilov_at_oracle.com
>>>>>>> <mailto:dmitry.kornilov_at_oracle.com>>:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> This topic has been discussed
>>>>>>> already. Current version of the spec
>>>>>>> is clear about polymorphism:
>>>>>>> /“//Deserialization into polymorphic
>>>>>>> types is not supported by default
>>>>>>> mapping. [JSB-3.8-1]”. /The spec is
>>>>>>> almost finished and I am strongly
>>>>>>> against going back and changing some
>>>>>>> basic things now.
>>>>>>>
>>>>>>> On the other hand I understand that
>>>>>>> this use case exists. There is a
>>>>>>> possible workaround to it using
>>>>>>> adapters. Sorry if there are any
>>>>>>> mistakes, I am writing code without
>>>>>>> checking.
>>>>>>> /
>>>>>>> /
>>>>>>> We can create a generic wrapper to
>>>>>>> any type holding its type info like
>>>>>>> this:
>>>>>>>
>>>>>>> class TypedWrapper<T> {
>>>>>>> String type;
>>>>>>> T obj;
>>>>>>> }
>>>>>>>
>>>>>>> We can create a generic adapter
>>>>>>> which converts type to wrapper and
>>>>>>> vice versa. I am not putting a full
>>>>>>> implementation here, I hope that the
>>>>>>> idea is clear:
>>>>>>>
>>>>>>> class TypedAdapter implements
>>>>>>> JsonbAdapter<T, TypedWrapper<T>> {
>>>>>>> TypedWrapper adaptToJson(T obj) {
>>>>>>> TypedWrapper<T> t = new
>>>>>>> TypedWrapper<T>();
>>>>>>> t.type = obj.getClass().getName();
>>>>>>> t.obj = obj;
>>>>>>> return t;
>>>>>>> }
>>>>>>>
>>>>>>> T adaptFromJson(TypedWrapper<T> t) {
>>>>>>> // Use reflection API to
>>>>>>> create an instance of object based
>>>>>>> on t.type field value
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>> Now lets imagine that we have the
>>>>>>> following classes:
>>>>>>>
>>>>>>> class Animal {
>>>>>>> }
>>>>>>>
>>>>>>> class Dog extends Animal {
>>>>>>> }
>>>>>>>
>>>>>>> class Cat extends Animal {
>>>>>>> }
>>>>>>>
>>>>>>> class Foo {
>>>>>>> Animal animal;
>>>>>>> }
>>>>>>>
>>>>>>> To properly handle
>>>>>>> serialization/deserialization of Foo
>>>>>>> we just need to create an adapter
>>>>>>> which extends TypedAdapter from ab
>>>>>>>
> ...
>