On 03/23/2016 02:02 PM, Romain Manni-Bucau wrote:
>
>
> 2016-03-23 13:39 GMT+01:00 Roman Grigoriadi
> <roman.grigoriadi_at_oracle.com <mailto:roman.grigoriadi_at_oracle.com>>:
>
> I think [A] is misunderstood..
>
> /[A] Think I get it: the adapter will return Foo and then on this
> instance Jsonb will populate fields.
>
> / That is not what I ment. An adapter is at the end of the
> process, T instance is already populated both: when adapter adapts
> it returns adapted result.
> Above statement is maybe enough, but for sake of clarity let me
> revisit whole process.
>
> 1. Lets have JSON document for a TypeWrapper<T>
> {
> "animal":{
> "className":"com.foo.Dog",
> "instance":{
> "name":"Doggie",
> "dogProperty":"Property of a Dog."
> }
> }
> }
>
> During deserialization what happens first is a completed instance
> of a TypeWrapper with a a fully populated T inside. This happens
> in following steps:
> 1. Jsonb encounters deserializing a TypeWrapper<T>, and processes
> it custom TypeWrapperDeserializer.
> 2. TypeWrapperDeserializer (implemented by us), creates an empty
> instance of TypeWrapper.
> 3. Jsonb reads and sets property "className" into TypeWrapper
> 4. Jsonb loads a class by this classname and creates instance with
> default construcotor in our case Dog or Cat and sets it to
> TypeWrapper instance.
>
>
> That's where I'm lost. Only way it could work - without putting this
> behavior in the spec - is if TypeWrapper overrides the setter of
> classname to also set the instance:
>
> public void setClassName(final String className) {
> this.className= className;
> try {
> this.value = Animal.class.cast(Thread.currentThread().getContextClassLoader().loadClass(className).newInstance());
> }catch (final InstantiationException | IllegalAccessException | ClassNotFoundException e) {
> throw new IllegalArgumentException(e);
> }
> }
Nope, it is only the className set as a String in this moment.
Next JsonParser enconters a JsonObject and pushes ObjectStarted event
into JSONB. Jsonb handles such event with creating instance of
appropriate type. That type is normally read from ClassModel (which is
read from actual Field or ParameterizedType argument of a Collection).
But in our case this type is swapped with a Class loaded dynamically by
name.
So now we have a correct instance ready to be populated by common
reflection deserialization.
> And then Jsonb before creating the Animal instance reads the field to
> extract its type which is likely not what is done since the type is
> deduced statically from the model class (field type) - this part is
> important to avoid surprises and open security issues.
No need to create instance now, it is already fully populated. Just
extract it from TypeWrapper as is.
>
> If I'm still wrong do you care writing a small test case showing that
> in action please?
Sure I will implement a test and push it to our eclipselink git which
hosts the jsonb-ri.
>
> 5. Jsonb parses a "ClassModel" for Dog or Cat class (reading all
> @Jsonb annots etc if needed).
> 6. Jsonb goes deeper in json document parsing json object for
> "instance" property and populating all fields in Dog or Cat
> (doesn't really care what is it as it is common reflection based
> process for all properties inside).
> 7. After above is completed an instance of TypeWrapper with fully
> deserialized T inside is passed to an adapter method "T
> adaptFromJson(TypeWrapper<T> obj)".
> 8. Result from adapter is than set to a field, setter, collection,
> map, array or whatever else is declared in domain model to
> unmarshall into.
>
> What returns an adapter is already ready to be set into model as
> is with polymorphism working this way.
>
> Now I suppose this is exactly what:
> [A]..../"Thought adapter should return a populated instance
> otherwise no way to convert to business model properly IMO"/ means ?
>
> Hope this helps and will bring some light.
>
> Thanks,
> Roman
>
>
>
> On 03/23/2016 12:49 PM, Romain Manni-Bucau wrote:
>>
>> 2016-03-23 12:29 GMT+01:00 Roman Grigoriadi
>> <roman.grigoriadi_at_oracle.com <mailto:roman.grigoriadi_at_oracle.com>>:
>>
>> You load a Dog or Cat class by name, which appears in json
>> and instantiate it with default constructor. This is what I
>> suppose a custom deserializer for TypeWrapper would do.
>>
>>
>> [A] Think I get it: the adapter will return Foo and then on this
>> instance Jsonb will populate fields. Thought adapter should
>> return a populated instance otherwise no way to convert to
>> business model properly IMO (if you don't have an explicit model
>> you maybe don't want a setter or field to be used). That's why I
>> was kind of blocked there.
>>
>> If you want to polymorphically adapt each animal in a
>> collection you would have instead annotate Animal.class with
>> @JsonbTypeAdapter, or pass an instance of AnimalAdapter to
>> JsonbConfig.
>>
>> You are right, you can't add
>> @JsonbTypeAdapter(AnimalAdapter.class) to List<Animal> as it
>> would than have to be declared as: "extends
>> TypeAdapter<List<Animal>>" and parent implementation would
>> not work than.
>>
>>
>> Well adapter for items instead of the field in case of a
>> collection is something we can implement - we do in Johnzon - and
>> which is useful IMO. That said it doesn't solve the polymorphism
>> issue. If [A] is solved it can work but today we don't have any
>> way to know if the adapter builds the instance or just
>> instantiate it if I got you right.
>>
>> Did I get your point?
>>
>>
>> Think so.
>>
>> Roman.
>>
>>
>> On 03/23/2016 12:04 PM, Romain Manni-Bucau wrote:
>>> I don't get this part "Jsonb looks at animal field model and
>>> gets "toType" from annotated adapter".
>>>
>>> How do you create a Dog in your example instead of an Animal.
>>>
>>> Keep in mind you don't know all types from jsonb point of
>>> view. The real use case is more obvious on a collection - at
>>> least for me:
>>>
>>> public static class Pojo {
>>> @JsonbTypeAdapter(AnimalAdapter.class)
>>> public Collection<Animal> animals; // Dog, Cow, Cat, Horse...
>>> }
>>>
>>>
>>> Romain Manni-Bucau
>>> @rmannibucau
>>> http://www.tomitribe.com
>>> http://rmannibucau.wordpress.com
>>> https://github.com/rmannibucau
>>>
>>> 2016-03-23 11:51 GMT+01:00 Roman Grigoriadi
>>> <roman.grigoriadi_at_oracle.com
>>> <mailto:roman.grigoriadi_at_oracle.com>>:
>>>
>>> You don't instantiate X in adapter you already have it.
>>> Here is implementation of such adapter:
>>>
>>> class TypeAdapter<T> implements JsonbAdapter<T, TypeWrapper<T>> {
>>>
>>> @Override
>>> public TypeWrapper<T> adaptToJson(T obj) throws Exception {
>>> TypeWrapper<T> wrapper = new TypeWrapper<>();
>>> wrapper.setClassName(obj.getClass().getName());
>>> wrapper.setInstance(obj);
>>> return wrapper;
>>> }
>>>
>>> @Override
>>> public T adaptFromJson(TypeWrapper<T> obj) throws Exception {
>>> return obj.getInstance();
>>> }
>>> }
>>>
>>> class AnimalAdapter extends TypeAdapter<Animal> {}
>>>
>>> Than you have a pojo like this:
>>>
>>> public static class Pojo {
>>> @JsonbTypeAdapter(AnimalAdapter.class)
>>> public Animal animal;
>>> }
>>>
>>> And json like this:
>>>
>>> {
>>> "animal":{"className":"com.foo.Dog",name":"Doggie","dogProperty":"Property of a Dog."}}
>>> }
>>>
>>>
>>> What happens during deserialization in our
>>> implementation is: JsonParser generates an event for
>>> started object at "animal" property. Jsonb looks at
>>> animal field model and gets "toType" from annotated
>>> adapter. Deserialization than happens for TypedWrapper
>>> as for common arbitrary user Type, but with custom
>>> deserializer, reading class property first and creating
>>> instance from it. After TypedWrapper<Animal> is fully
>>> deserialized, it is passed to adapter instance, to
>>> retrieve an Animal, which is finally set to Animal field.
>>>
>>> Regards,
>>> Roman
>>>
>>>
>>> On 03/23/2016 11:20 AM, Romain Manni-Bucau wrote:
>>>> Hi Roman,
>>>>
>>>> can you fill some "..." please? typically at the moment
>>>> "I know the type is X". How with an adapter do you
>>>> instantiate X and then let jsonb populate it (without
>>>> having another Jsonb instance)? That was the main
>>>> missing part of the puzzle.
>>>>
>>>>
>>>> Romain Manni-Bucau
>>>> @rmannibucau
>>>> http://www.tomitribe.com
>>>> http://rmannibucau.wordpress.com
>>>> https://github.com/rmannibucau
>>>>
>>>> 2016-03-23 11:14 GMT+01:00 Roman Grigoriadi
>>>> <roman.grigoriadi_at_oracle.com
>>>> <mailto:roman.grigoriadi_at_oracle.com>>:
>>>>
>>>> Hi,
>>>>
>>>> @Romain - what you and Dmitry suggested with
>>>> adapters, will actully work in our case. We will
>>>> have to to add a custom deserializer handler for a
>>>> "TypedWrapper<T>" (similar to collection / array or
>>>> reflection deserializers), which will read a class
>>>> property first and on encountering a json object
>>>> for T will create a correct type instance. The only
>>>> condition would be, that class property must appear
>>>> before inner instance in json object for
>>>> "TypedWrapper<T>" as Eugen suggested:
>>>>
>>>> {"classProperty":"com.foo.Dog","instance":{...}}
>>>>
>>>> Adapters will than work in a common way to extract
>>>> T from TypedWrapper and set it to a class model be
>>>> it a field or a collection/array.
>>>>
>>>> I would also add required enumeration of allowed
>>>> subtypes into TypedAdapter:
>>>>
>>>> class TypedAdapter<T> implements JsonbAdapter<T,
>>>> TypedWrapper<T>> {
>>>> TypedAdapter(Class[] allowedSubtypes) {...}
>>>> ...
>>>> }
>>>>
>>>> So we don't allow loading of any arbitrary class
>>>> suggested by incoming json in case of not trusted
>>>> third party.
>>>>
>>>> @Romain - adding support for custom serializers /
>>>> deserializers in fashion of fasterxml with access
>>>> to JsonGenerator/JsonParser would also solve the
>>>> problem in similar fashion, but it would be more
>>>> complicated for a client to implement logic driven
>>>> by JsonParser events, than by providing predefined
>>>> adapter.
>>>>
>>>> Thanks,
>>>> Roman
>>>>
>>>> On 03/21/2016 09:36 PM, Romain Manni-Bucau wrote:
>>>>> Hi Dmitry,
>>>>>
>>>>> was my thought as well but it doesn't really work
>>>>> cause in adapter you can't relaunch the
>>>>> deserialization chain with the same jsonb
>>>>> instance. The serialization side is the easy one
>>>>> but the deserialization makes adapters API not
>>>>> enough to have a consistent behavior accross the
>>>>> deserialization. Making the adapter "JsonbAware"
>>>>> would work. We can probably use @Inject (CDI if
>>>>> there or just the JSR 330 as an optional
>>>>> dependency if not)
>>>>>
>>>>> Side note: I'd like we enable this impl but we
>>>>> don't provide them in the spec since it is likely
>>>>> dependent on your use case and saw cases where
>>>>> type was not enough and we will likely not handle
>>>>> all possible models.
>>>>>
>>>>>
>>>>>
>>>>> Romain Manni-Bucau
>>>>> @rmannibucau
>>>>> http://www.tomitribe.com
>>>>> http://rmannibucau.wordpress.com
>>>>> https://github.com/rmannibucau
>>>>>
>>>>> 2016-03-21 21:27 GMT+01:00 Dmitry Kornilov
>>>>> <dmitry.kornilov_at_oracle.com
>>>>> <mailto:dmitry.kornilov_at_oracle.com>>:
>>>>>
>>>>> Hi,
>>>>>
>>>>> This topic has been discussed already. Current
>>>>> version of the spec is clear about
>>>>> polymorphism: /“//Deserialization into
>>>>> polymorphic types is not supported by default
>>>>> mapping. [JSB-3.8-1]”. /The spec is almost
>>>>> finished and I am strongly against going back
>>>>> and changing some basic things now.
>>>>>
>>>>> On the other hand I understand that this use
>>>>> case exists. There is a possible workaround to
>>>>> it using adapters. Sorry if there are any
>>>>> mistakes, I am writing code without checking.
>>>>> /
>>>>> /
>>>>> We can create a generic wrapper to any type
>>>>> holding its type info like this:
>>>>>
>>>>> class TypedWrapper<T> {
>>>>> String type;
>>>>> T obj;
>>>>> }
>>>>>
>>>>> We can create a generic adapter which converts
>>>>> type to wrapper and vice versa. I am not
>>>>> putting a full implementation here, I hope
>>>>> that the idea is clear:
>>>>>
>>>>> class TypedAdapter implements JsonbAdapter<T,
>>>>> TypedWrapper<T>> {
>>>>> TypedWrapper adaptToJson(T obj) {
>>>>> TypedWrapper<T> t = new TypedWrapper<T>();
>>>>> t.type = obj.getClass().getName();
>>>>> t.obj = obj;
>>>>> return t;
>>>>> }
>>>>>
>>>>> T adaptFromJson(TypedWrapper<T> t) {
>>>>> // Use reflection API to create an
>>>>> instance of object based on t.type field value
>>>>> }
>>>>> }
>>>>>
>>>>> Now lets imagine that we have the following
>>>>> classes:
>>>>>
>>>>> class Animal {
>>>>> }
>>>>>
>>>>> class Dog extends Animal {
>>>>> }
>>>>>
>>>>> class Cat extends Animal {
>>>>> }
>>>>>
>>>>> class Foo {
>>>>> Animal animal;
>>>>> }
>>>>>
>>>>> To properly handle
>>>>> serialization/deserialization of Foo we just
>>>>> need to create an adapter which extends
>>>>> TypedAdapter from above and annotate /animal/
>>>>> field:
>>>>>
>>>>> class AnimalAdapter extends
>>>>> TypedAdapter<Animal, TypedWrapper<Animal>> {
>>>>> }
>>>>>
>>>>> class Foo {
>>>>> @JsonbTypeAdapter(AnimalAdapter.class)
>>>>> Animal animal;
>>>>> }
>>>>>
>>>>> We can include TypedWrapper and TypedAdapter
>>>>> in reference implementation.
>>>>>
>>>>> Thanks,
>>>>> Dmitry
>>>>>
>>>>>
>>>>>
>>>>>> On 21 Mar 2016, at 19:31, Eugen Cepoi
>>>>>> <cepoi.eugen_at_gmail.com
>>>>>> <mailto:cepoi.eugen_at_gmail.com>> wrote:
>>>>>>
>>>>>> Not sure if it should or not be part of
>>>>>> jsonb. Though it is very likely people will
>>>>>> want that feature. On the other side, it is a
>>>>>> good way to have different impls distinguish
>>>>>> them self by providing such advanced features
>>>>>> (I mean as part of the impl and not at all
>>>>>> present in the spec). In that case I would
>>>>>> find it fine to have people configure
>>>>>> directly the impl api.
>>>>>>
>>>>>> When we think polymorphism, at first it seems
>>>>>> enough to handle it only for json objects,
>>>>>> but it can also happen for other types, esp.
>>>>>> ones that are serialized as literals. Then
>>>>>> where do you store the type? Same for json
>>>>>> arrays. This is just one aspect of the
>>>>>> introduced complexity when one wants to
>>>>>> handle polymorphism. The produced json starts
>>>>>> getting more and more ugly.
>>>>>>
>>>>>> Then comes the moment when you want to change
>>>>>> the name of the serialized property or the
>>>>>> format you use.
>>>>>>
>>>>>> In some other situations (depending on how it
>>>>>> has been implemented) you might want to have
>>>>>> access to a dom structure that holds the
>>>>>> properties so you can extract from them what
>>>>>> is related to the type and then "convert"
>>>>>> that structure to the target type. This
>>>>>> happens if you allow the type property to
>>>>>> appear in any order.
>>>>>>
>>>>>> There are many other things that one could
>>>>>> want for polymorphic support which would make
>>>>>> all that even more complex.
>>>>>>
>>>>>>
>>>>>> In genson I first handled it only for what
>>>>>> gets serialized as a json object and imposed
>>>>>> that the type property must appear first in
>>>>>> the json (so we don't need to read the full
>>>>>> structure before knowing the target type).
>>>>>> Then recently I added support for all types
>>>>>> (literal and array) by wrapping them in a
>>>>>> json object (yeah not very nice, but has the
>>>>>> merit of being relatively simple).
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2016-03-21 2:22 GMT-07:00 Sebastian Daschner
>>>>>> <java_at_sebastian-daschner.de
>>>>>> <mailto:java_at_sebastian-daschner.de>>:
>>>>>>
>>>>>> Hi guys,
>>>>>>
>>>>>> The idea was that this "javaType" (name
>>>>>> is subject to change) wouldn't be
>>>>>> included in all cases. Default behaviour
>>>>>> should be to serialize as small / easy as
>>>>>> possible. Only if the type information is
>>>>>> needed to determine the actual type, an
>>>>>> @JsonbTypeInfo annotation could be added
>>>>>> to the corresponding type. Configuring a
>>>>>> "forceTypeInfo" boolean for the
>>>>>> programmatic JSONB config could be
>>>>>> possible though.
>>>>>>
>>>>>> @Przemyslaw:
>>>>>> Even if you solely use DTOs for
>>>>>> serialization (I won't agree that that's
>>>>>> a must for all situations) you can still
>>>>>> have situations with subclasses and more
>>>>>> complex hierarchies -- we had that case
>>>>>> in several projects, also when using
>>>>>> transfer objects to map the domain
>>>>>> objects. "Flattening" the type
>>>>>> hierarchies as workaround is not always
>>>>>> desired.
>>>>>>
>>>>>> But the longer I think about your
>>>>>> approach that enums could be used to
>>>>>> derive the type information (like JPA,
>>>>>> yes) the more I like this idea as it
>>>>>> would be more generic (more difficult /
>>>>>> more effort to configure, but easier to
>>>>>> apply to different technologies). We
>>>>>> should investigate further (with some
>>>>>> examples) what approach could be the best
>>>>>> fit for JSONB.
>>>>>>
>>>>>> But I highly recommend that some
>>>>>> functionality to determine polymorphic
>>>>>> types should be included in the spec.
>>>>>>
>>>>>>
>>>>>> @Romain:
>>>>>> The spec should in fact require the impl
>>>>>> to check for "suitable" types then, i.e.
>>>>>> only types to which the actual property
>>>>>> type can be assigned to should be
>>>>>> allowed, etc.
>>>>>>
>>>>>> I like your idea to add a "Wrapper" to
>>>>>> unwrap instances but let JSONB still do
>>>>>> the (de-)serialization. This could be
>>>>>> helpful for more control how the
>>>>>> containing type (the List in my example)
>>>>>> should be instanciated. But it doesn't
>>>>>> fully solve the polymorphism example in
>>>>>> all cases, as the implementation of the
>>>>>> actual type (Customer or VipCustomer) can
>>>>>> still not determined from the nested JSON
>>>>>> objects if there is no such thing as a
>>>>>> JSON meta type information.
>>>>>>
>>>>>>
>>>>>> Cheers,
>>>>>> Sebastian
>>>>>> On 03/21/2016 08:32 AM, Przemyslaw
>>>>>> Bielicki wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> I'm not a fan of the original idea (and
>>>>>>> it's an euphemism). Remember that JSONB
>>>>>>> is used to serialize/deserialize DTOs,
>>>>>>> not domain objects. You should have an
>>>>>>> additional adapter layer to convert DTOs
>>>>>>> into application's business model/domain
>>>>>>> objects and back.
>>>>>>> Polymorphism is broken (yeah, it's
>>>>>>> another topic and more philosophic) and
>>>>>>> I would discourage its application in
>>>>>>> DTO layer. If you have to, you can use
>>>>>>> an enum to mark what's the target class
>>>>>>> (check JPA for additional inspirations).
>>>>>>> "//javaType": "x.y.z" is really bad idea
>>>>>>> IMO - what if you use the same JSON to
>>>>>>> communicate with applications developed
>>>>>>> in .net, python, php, c++ at the same
>>>>>>> time? (yes, it happens). Would you
>>>>>>> define a type for each platform?
>>>>>>>
>>>>>>> If you use DTOs in business layer,
>>>>>>> you're doing something wrong :)
>>>>>>>
>>>>>>> @Romain, your idea looks interesting,
>>>>>>> can you show and example or
>>>>>>> deserialization process where
>>>>>>> polymorphism is involved?
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Przemyslaw
>>>>>>>
>>>>>>> On Sun, Mar 20, 2016 at 10:52 PM, Romain
>>>>>>> Manni-Bucau <rmannibucau_at_tomitribe.com
>>>>>>> <mailto:rmannibucau_at_tomitribe.com>> wrote:
>>>>>>>
>>>>>>> Hi guys
>>>>>>>
>>>>>>> Why not allowing adapters to wrap
>>>>>>> and unwrap instances but keep
>>>>>>> serializer mecanism. For
>>>>>>> serialization no issue but for
>>>>>>> deserialization we need to switch
>>>>>>> the deserialized type and then post
>>>>>>> process the instance - or do it
>>>>>>> automatically with an Unwrappable
>>>>>>> interface?
>>>>>>>
>>>>>>> Overall idea for this particular use
>>>>>>> case would be:
>>>>>>>
>>>>>>> X ----TypedWrapper(X)----> JSON
>>>>>>> JSON----TypedWrapper.class---->
>>>>>>> TypedWrapped ----unwrap----> X
>>>>>>>
>>>>>>> Parts in arrows are adapter/jsonb
>>>>>>> integration.
>>>>>>>
>>>>>>> An alternative is to provide to
>>>>>>> adapters the stream and jsonp
>>>>>>> instances to do it themself with
>>>>>>> probably a mapper reference to
>>>>>>> re(de)serialise an object directly.
>>>>>>>
>>>>>>> Wdyt?
>>>>>>>
>>>>>>> ---------- Message transféré ----------
>>>>>>> De : "Romain Manni-Bucau"
>>>>>>> <rmannibucau_at_tomitribe.com
>>>>>>> <mailto:rmannibucau_at_tomitribe.com>>
>>>>>>> Date : 19 mars 2016 21:49
>>>>>>> Objet : Re: [jsonb-spec users] Java
>>>>>>> polymorphism support
>>>>>>> À : <users_at_jsonb-spec.java.net
>>>>>>> <mailto:users_at_jsonb-spec.java.net>>
>>>>>>> Cc :
>>>>>>>
>>>>>>> Hi Sebastian
>>>>>>>
>>>>>>> I am not fan of that - Mark knows ;)
>>>>>>> - and in particular now I'm sure we
>>>>>>> shouldnt do it: the 0-day
>>>>>>> vulnerability is still there and you
>>>>>>> open the door to the same issue or a
>>>>>>> complicated config adding this
>>>>>>> feature in something as generic as
>>>>>>> jsonb.
>>>>>>>
>>>>>>> Le 19 mars 2016 21:28, "Sebastian
>>>>>>> Daschner"
>>>>>>> <java_at_sebastian-daschner.de
>>>>>>> <mailto:java_at_sebastian-daschner.de>>
>>>>>>> a écrit :
>>>>>>>
>>>>>>> Hi experts,
>>>>>>>
>>>>>>> I don't know whether this has
>>>>>>> been discussed in the mailing
>>>>>>> list before
>>>>>>> but a needed functionality would
>>>>>>> be to specify the Java type of
>>>>>>> properties in the serialized JSON.
>>>>>>>
>>>>>>> As JSON doesn't standardize
>>>>>>> comments or other "attributes"
>>>>>>> (like XML
>>>>>>> does) a "magic property" could
>>>>>>> be added.
>>>>>>>
>>>>>>> Please see the proposal
>>>>>>> (explored by Mark Struberg,
>>>>>>> Reinhard Sandtner
>>>>>>> and myself) on my blog:
>>>>>>> https://blog.sebastian-daschner.com/entries/json_mapping_polymorphism_support
>>>>>>>
>>>>>>> WDYT?
>>>>>>>
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Sebastian
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>