users@jersey.java.net

Re: [Jersey] Problem saving binary data using Jersey multipart

From: Geoff Sallee <geoff.sallee_at_gmail.com>
Date: Mon, 9 Feb 2009 11:40:23 -0800

Okay. We ended up using Apache Commons fileupload which works for us.
Thanks for your help though.

On Wed, Feb 4, 2009 at 11:36 PM, Craig McClanahan
<Craig.McClanahan_at_sun.com>wrote:

> Craig McClanahan wrote:
>
> Geoff Sallee wrote:
>
> Unfortunately that did not work. I also attempted actually changing the
> value of the threshold variable in the source code for MultiPartConfig and
> recompiled the whole library (thinking that maybe the properties file was
> not being picked up by my app for some reason) but there was still no
> change.
>
> Hmm ... the mystery deepens.
>
> Unfortunately, I'm going to be in the air (and out of communication for a
> while) flying from Europe back to the US, but I will be able to take a look
> at this in detail early next week.
>
> OK, we're going to have to color me officially puzzled. I just added
> (r1939, should be available soon in a Hudson build, or available immediately
> if you download and build from the respository) some unit tests that copy
> files up to 1.6mb in size up and down the HTTP pipe, and things seem to work
> as expected. As far as I can tell, jersey-multipart is faithfully serving
> up every single byte in the body part, whether or not the size exceeds the
> buffer threshold. So, I don't *think* jersey-multipart is causing this
> issue for you.
>
> Offhand, I cannot see anything obvious in the code you've shown us that
> would cause a truncation issue like this ... but I sure can't see where
> jersey-multipart is doing it.
>
> Craig
>
>
> On Fri, Jan 30, 2009 at 3:20 AM, Craig McClanahan <
> Craig.McClanahan_at_sun.com> wrote:
>
>> Geoff Sallee wrote:
>>
>>> Hello,
>>>
>>> I just integrated the jersey-multipart-1.0.1 library into my Jersey
>>> project in order to more easily handle file uploads. The problem is that
>>> when I attempt to save the file using the InputStream retrieved from
>>> multiPart.getBodyParts().get(0) (there is only one part in my test form) it
>>> only saves part of the file to disk. For example, if I attempt to upload a
>>> 92KB JPEG image it only gets saved as an 8KB JPEG, which only contains the
>>> top portion of the image. Here is the code I am using:
>>>
>>> public Response addUserAsset(
>>> @PathParam("userName") String userName,
>>> MultiPart multiPart) {
>>> String relativePath = null;
>>> String mimeType = null;
>>> String fileName = null;
>>> try {
>>> InputStream inputStream = null;
>>> for(BodyPart bodyPart : multiPart.getBodyParts()) {
>>> mimeType = bodyPart.getHeaders().getFirst("Content-Type");
>>> if(!StringUtils.isEmpty(mimeType) &&
>>> mimeType.startsWith("image")) { String metaData =
>>> bodyPart.getHeaders().getFirst("Content-Disposition");
>>> Matcher m = fileNamePattern.matcher(metaData);
>>> if(m.matches()) {
>>> fileName = m.group(1);
>>> }
>>> BodyPartEntity bpe = (BodyPartEntity)
>>> bodyPart.getEntity();
>>> inputStream = bpe.getInputStream();
>>> break;
>>> }
>>> }
>>> if(inputStream != null) {
>>> String baseDirPath =
>>> servletContext.getInitParameter("USER_DIGITAL_ASSET_DIR_PATH");
>>> DigitalAssetAccess access = new
>>> DigitalAssetAccessImpl(baseDirPath);
>>> relativePath = access.store(userName, fileName, mimeType,
>>> inputStream); //THIS METHOD ACTUALLY STORES THE FILE TO DISK
>>> }
>>> else {
>>> result.setSuccess(false);
>>> result.addMessage(INVALID_VALUE);
>>> return ResponseUtil.buildResponse(Status.BAD_REQUEST,
>>> result);
>>> }
>>> }
>>> catch(IOException ioe) {
>>> result.setSuccess(false);
>>> result.addMessage(INTERNAL_SERVER_ERROR);
>>> return
>>> ResponseUtil.buildResponse(Status.INTERNAL_SERVER_ERROR, result);
>>> }
>>> catch(Exception e) {
>>> result.setSuccess(false);
>>> result.addMessage(INTERNAL_SERVER_ERROR);
>>> return
>>> ResponseUtil.buildResponse(Status.INTERNAL_SERVER_ERROR, result);
>>> }
>>> }
>>>
>>> My store method uses the InputStream as follows:
>>>
>>> public String store(String userLoginId, String fileName,
>>> String mimeType, InputStream inputStream) throws IOException {
>>> File parentDir = new File(baseDirPath + "/" + userLoginId);
>>> if (!parentDir.exists()) {
>>> parentDir.mkdirs();
>>> }
>>>
>>> String assetFileName = getUniqueName(fileName, mimeType);
>>>
>>> File assetFile = new File(parentDir, assetFileName);
>>> if (assetFile.exists()) {
>>> throw new IllegalStateException(
>>> "File already exists with the name:" + assetFileName);
>>> } else {
>>> assetFile.createNewFile();
>>> }
>>>
>>> OutputStream outputStream = new BufferedOutputStream(
>>> new FileOutputStream(assetFile), BUFFER_SIZE); //
>>> BUFFER_SIZE is 1024 * 1024 * 2 (2MB)
>>>
>>> int b;
>>> while ((b = inputStream.read()) != EOF) { // EOF is -1
>>> outputStream.write(b);
>>> }
>>> inputStream.close();
>>> outputStream.flush();
>>> outputStream.close();
>>>
>>> return userLoginId + "/" + assetFileName;
>>> }
>>>
>>> I have also tested storing the file using the javax.ImageIO package with
>>> the same results, so I can't imagine it's due to the way the file is being
>>> stored. Am I missing something important? I did notice the mention of
>>> calling multipart.cleanup() but that had no effect in my case. Right now it
>>> looks like the files that are being stored are exactly 8KB so I'm thinking
>>> it has something to do with the threshold value, but I'm not sure what I can
>>> do to fix it. Any help is appreciated.
>>>
>>> That is definitely a red flag that points to a threshold value issue
>> (although the default buffer size is 4k, so it's definitely odd). Could you
>> do me the favor of one experiment to nail this down farther? I'd like you
>> to try setting the threshold value to something larger than the size of the
>> file (to ensure that it stays buffered in memory and not to the local disk
>> file that would otherwise be used). If the app works in this configuration,
>> then we know for sure that something in the jersey-multipart buffering logic
>> is loopy. If it still doesn't work, we'll need to keep digging.
>>
>> To set this configuration, create a "jersey-multipart-config.properties"
>> file (in WEB-INF/classes if this is a webapp, or on the classpath
>> otherwise), with an entry like this:
>>
>> bufferThreshold = 128000
>>
>> This will set the buffer size to ~128k, which should be big enough to
>> avoid the disk buffering for a 92kb image file.
>>
>> Regards,
>>> Geoff
>>>
>>
>> Craig
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe_at_jersey.dev.java.net
>> For additional commands, e-mail: users-help_at_jersey.dev.java.net
>>
>>
>
>
>