Hi Geoff,
Glad you managed to find a work around. If you have the time and
inclination would it be possible to send a zip of a simple project
that reproduces the issues?
Thanks,
Paul.
On Feb 9, 2009, at 8:40 PM, Geoff Sallee wrote:
> Okay. We ended up using Apache Commons fileupload which works for
> us. Thanks for your help though.
>
> On Wed, Feb 4, 2009 at 11:36 PM, Craig McClanahan <Craig.McClanahan_at_sun.com
> > wrote:
> Craig McClanahan wrote:
>>
>> Geoff Sallee wrote:
>>>
>>> Unfortunately that did not work. I also attempted actually
>>> changing the value of the threshold variable in the source code
>>> for MultiPartConfig and recompiled the whole library (thinking
>>> that maybe the properties file was not being picked up by my app
>>> for some reason) but there was still no change.
>> Hmm ... the mystery deepens.
>>
>> Unfortunately, I'm going to be in the air (and out of communication
>> for a while) flying from Europe back to the US, but I will be able
>> to take a look at this in detail early next week.
>>
> OK, we're going to have to color me officially puzzled. I just
> added (r1939, should be available soon in a Hudson build, or
> available immediately if you download and build from the
> respository) some unit tests that copy files up to 1.6mb in size up
> and down the HTTP pipe, and things seem to work as expected. As far
> as I can tell, jersey-multipart is faithfully serving up every
> single byte in the body part, whether or not the size exceeds the
> buffer threshold. So, I don't *think* jersey-multipart is causing
> this issue for you.
>
> Offhand, I cannot see anything obvious in the code you've shown us
> that would cause a truncation issue like this ... but I sure can't
> see where jersey-multipart is doing it.
>
>> Craig
>>>
>>> On Fri, Jan 30, 2009 at 3:20 AM, Craig McClanahan <Craig.McClanahan_at_sun.com
>>> > wrote:
>>> Geoff Sallee wrote:
>>> Hello,
>>>
>>> I just integrated the jersey-multipart-1.0.1 library into my
>>> Jersey project in order to more easily handle file uploads. The
>>> problem is that when I attempt to save the file using the
>>> InputStream retrieved from multiPart.getBodyParts().get(0) (there
>>> is only one part in my test form) it only saves part of the file
>>> to disk. For example, if I attempt to upload a 92KB JPEG image it
>>> only gets saved as an 8KB JPEG, which only contains the top
>>> portion of the image. Here is the code I am using:
>>>
>>> public Response addUserAsset(
>>> @PathParam("userName") String userName,
>>> MultiPart multiPart) {
>>> String relativePath = null;
>>> String mimeType = null;
>>> String fileName = null;
>>> try {
>>> InputStream inputStream = null;
>>> for(BodyPart bodyPart : multiPart.getBodyParts()) {
>>> mimeType = bodyPart.getHeaders().getFirst("Content-
>>> Type");
>>> if(!StringUtils.isEmpty(mimeType) &&
>>> mimeType.startsWith("image")) { String
>>> metaData = bodyPart.getHeaders().getFirst("Content-Disposition");
>>> Matcher m = fileNamePattern.matcher(metaData);
>>> if(m.matches()) {
>>> fileName = m.group(1);
>>> }
>>> BodyPartEntity bpe = (BodyPartEntity)
>>> bodyPart.getEntity();
>>> inputStream = bpe.getInputStream();
>>> break;
>>> }
>>> }
>>> if(inputStream != null)
>>> { String baseDirPath =
>>> servletContext.getInitParameter("USER_DIGITAL_ASSET_DIR_PATH");
>>> DigitalAssetAccess access = new
>>> DigitalAssetAccessImpl(baseDirPath);
>>> relativePath = access.store(userName, fileName,
>>> mimeType, inputStream); //THIS METHOD ACTUALLY STORES THE FILE TO
>>> DISK
>>> }
>>> else {
>>> result.setSuccess(false);
>>> result.addMessage(INVALID_VALUE);
>>> return
>>> ResponseUtil.buildResponse(Status.BAD_REQUEST, result);
>>> }
>>> }
>>> catch(IOException ioe) {
>>> result.setSuccess(false);
>>> result.addMessage(INTERNAL_SERVER_ERROR);
>>> return
>>> ResponseUtil.buildResponse(Status.INTERNAL_SERVER_ERROR, result);
>>> }
>>> catch(Exception e) {
>>> result.setSuccess(false);
>>> result.addMessage(INTERNAL_SERVER_ERROR);
>>> return
>>> ResponseUtil.buildResponse(Status.INTERNAL_SERVER_ERROR, result);
>>> }
>>> }
>>>
>>> My store method uses the InputStream as follows:
>>>
>>> public String store(String userLoginId, String fileName,
>>> String mimeType, InputStream inputStream) throws
>>> IOException {
>>> File parentDir = new File(baseDirPath + "/" +
>>> userLoginId);
>>> if (!parentDir.exists()) {
>>> parentDir.mkdirs();
>>> }
>>>
>>> String assetFileName = getUniqueName(fileName, mimeType);
>>>
>>> File assetFile = new File(parentDir, assetFileName);
>>> if (assetFile.exists()) {
>>> throw new IllegalStateException(
>>> "File already exists with the name:" +
>>> assetFileName);
>>> } else {
>>> assetFile.createNewFile();
>>> }
>>>
>>> OutputStream outputStream = new BufferedOutputStream(
>>> new FileOutputStream(assetFile), BUFFER_SIZE); //
>>> BUFFER_SIZE is 1024 * 1024 * 2 (2MB)
>>>
>>> int b;
>>> while ((b = inputStream.read()) != EOF) { // EOF is -1
>>> outputStream.write(b);
>>> }
>>> inputStream.close();
>>> outputStream.flush();
>>> outputStream.close();
>>>
>>> return userLoginId + "/" + assetFileName;
>>> }
>>>
>>> I have also tested storing the file using the javax.ImageIO
>>> package with the same results, so I can't imagine it's due to the
>>> way the file is being stored. Am I missing something important?
>>> I did notice the mention of calling multipart.cleanup() but that
>>> had no effect in my case. Right now it looks like the files that
>>> are being stored are exactly 8KB so I'm thinking it has something
>>> to do with the threshold value, but I'm not sure what I can do to
>>> fix it. Any help is appreciated.
>>>
>>> That is definitely a red flag that points to a threshold value
>>> issue (although the default buffer size is 4k, so it's definitely
>>> odd). Could you do me the favor of one experiment to nail this
>>> down farther? I'd like you to try setting the threshold value to
>>> something larger than the size of the file (to ensure that it
>>> stays buffered in memory and not to the local disk file that would
>>> otherwise be used). If the app works in this configuration, then
>>> we know for sure that something in the jersey-multipart buffering
>>> logic is loopy. If it still doesn't work, we'll need to keep
>>> digging.
>>>
>>> To set this configuration, create a "jersey-multipart-
>>> config.properties" file (in WEB-INF/classes if this is a webapp,
>>> or on the classpath otherwise), with an entry like this:
>>>
>>> bufferThreshold = 128000
>>>
>>> This will set the buffer size to ~128k, which should be big enough
>>> to avoid the disk buffering for a 92kb image file.
>>>
>>> Regards,
>>> Geoff
>>>
>>> Craig
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe_at_jersey.dev.java.net
>>> For additional commands, e-mail: users-help_at_jersey.dev.java.net
>>>
>>>
>>
>
>