users@jersey.java.net

[Jersey] Re: ChunkedInput with Protobuf messages

From: <michael.lewis_at_barclays.com>
Date: Tue, 10 Dec 2013 17:45:09 +0000

Apologies for Spam: but just in case anyone runs into this 'problem' with their own types - you have to pass the default delimiter on ChunkedOutput.
e.g. final ChunkedOutput<Protobuf.Counterparty> output = new ChunkedOutput<>(Protobuf.Counterparty.class,"\r\n");
i.e. Add "\r\n" and the ChunkedInput on the client will work fine.

RTFM helps too ;)


From: Lewis, Michael: Markets (LDN)
Sent: Tuesday, December 10, 2013 3:56 PM
To: users_at_jersey.java.net<mailto:users_at_jersey.java.net>
Subject: [Jersey] ChunkedInput with Protobuf messages

Hi,

I hope someone can help. I have an issue with ChunkedInput parsing. I have a service that is returning Protobuf messages via ChunkedOutput,
as follows:

    @GET
    @Path("{date}")
    @Produces("application/x-protobuf")
    public ChunkedOutput<Protobuf.Counterparty> getAllCounterparties(final @PathParam("date") String asOfDateString) throws
final ChunkedOutput<Protobuf.Counterparty> output = new ChunkedOutput<>(Protobuf.Counterparty.class);
              ...
    }

This invokes a method that writes out the Protobuf messages:


        while (resultSet.next()) {
            Protobuf.Counterparty c = Protobuf.Counterparty.newBuilder()
                    .setCounterpartyId(resultSet.getString(2))
              .....
                    .build();
            output.write(c);
        }

This appears to work: in a browser it will slowly download all the messages in chunks.

The problem I have is with the client. I register the message body reader, and try to read the chunks as follows:

final ChunkedInput<Protobuf.Counterparty> chunkedInput
                    = response.readEntity(new GenericType<ChunkedInput<Protobuf.Counterparty>>() {});
chunkedInput.setChunkType("application/x-protobuf");

Protobuf.Counterparty chunk;
while ((chunk = chunkedInput.read()) != null) {
       ...
}

The chunkedInput.read() is failing - it is trying to read all the data into a single Protobuf message and I get an exception (Protobuf message too large).

Do I need to supply my own ChunkedParser? Any ideas on how to the best way to resolve this?

Regards,
Mike Lewis

The code for the message body reader is pretty simple:

@Provider
@Consumes("application/x-protobuf")
public class ProtobufMessageBodyReader implements MessageBodyReader<Message> {

   /**
   */
    public boolean isReadable(Class<?> type, Type genericType, Annotation[] annotations, MediaType mediaType) {
        return Message.class.isAssignableFrom(type);
    }

    /**
     */
    public Message readFrom(Class<Message> type,
                            Type genericType,
                            Annotation[] annotations,
                            MediaType mediaType,
                            MultivaluedMap<String,String> httpHeaders,
                            InputStream entityStream) throws java.io.IOException, WebApplicationException {
        try {
            Method newBuilder = type.getMethod("newBuilder");
            GeneratedMessage.Builder<?> builder = (GeneratedMessage.Builder<?>) newBuilder.invoke(type);
            return builder.mergeFrom(entityStream).build();
        }
        catch (Exception e) {
            throw new WebApplicationException(e);
        }
    }

}



_______________________________________________

This message is for information purposes only, it is not a recommendation, advice, offer or solicitation to buy or sell a product or service nor an official confirmation of any transaction. It is directed at persons who are professionals and is not intended for retail customer use. Intended for recipient only. This message is subject to the terms at: www.barclays.com/emaildisclaimer<http://www.barclays.com/emaildisclaimer>.

For important disclosures, please see: www.barclays.com/salesandtradingdisclaimer<http://www.barclays.com/salesandtradingdisclaimer> regarding market commentary from Barclays Sales and/or Trading, who are active market participants; and in respect of Barclays Research, including disclosures relating to specific issuers, please see http://publicresearch.barclays.com.

_______________________________________________

_______________________________________________

This message is for information purposes only, it is not a recommendation, advice, offer or solicitation to buy or sell a product or service nor an official confirmation of any transaction. It is directed at persons who are professionals and is not intended for retail customer use. Intended for recipient only. This message is subject to the terms at: www.barclays.com/emaildisclaimer.

For important disclosures, please see: www.barclays.com/salesandtradingdisclaimer regarding market commentary from Barclays Sales and/or Trading, who are active market participants; and in respect of Barclays Research, including disclosures relating to specific issuers, please see http://publicresearch.barclays.com.

_______________________________________________