jsr369-experts@servlet-spec.java.net

[jsr369-experts] Re: HTTP Push, URI and header mutations

From: Greg Wilkins <gregw_at_intalio.com>
Date: Fri, 5 Dec 2014 11:52:01 +0100

I've been working with Push a little bit more and have an alternative
proposal for the API.

I've started work on this, but thought I'd seek some feedback before I
commit too much effort. The idea is that often there will be many
resources to push and that a lot of the work needed to work out headers,
sessions and cookies is common to all the pushes. So we need a new
PushBuilder instance that does all this work once. We would obtain a
PushBuilder from a new method on the request:

    /** Get a PushBuilder associated with this request initialized as
follows:<ul>
     * <li>The method is initialized to "GET"</li>
     * <li>The headers from this request are copied to the Builder, except
for:<ul>
     * <li>Conditional headers (eg. If-Modified-Since)
     * <li>Range headers
     * <li>Expect headers
     * <li>Authentication headers
     * <li>Referrer headers
     * </ul></li>
     * <li>If the request was Authenticated, an Authentication header will
     * be set with a container generated token that will result in
equivalent
     * authentication</li>
     * <li>The query string from {_at_link #getQueryString()}
     * <li>The {_at_link #getRequestedSessionId()} value, unless at the time
     * of the call {_at_link #getSession(boolean)}
     * has previously been called to create a new {_at_link HttpSession}, in
     * which case the new session ID will be used as the PushBuilders
     * requested session ID.</li>
     * <li>The source of the requested session id will be the same as for
     * this request</li>
     * <li>The builders Referer header will be set to {_at_link
#getRequestURL()}
     * plus any {_at_link #getQueryString()} </li>
     * <li>If {_at_link HttpServletResponse#addCookie(Cookie)} has been called
     * on the associated response, then a corresponding Cookie header will
be added
     * to the PushBuilder, unless the {_at_link Cookie#getMaxAge()} is <=0, in
which
     * case the Cookie will be removed from the builder.</li>
     * <li>If this request has has the conditional headers
If-Modified-Since or
     * If-None-Match then the {_at_link PushBuilder#isConditional()} header is
set
     * to true.
     * </ul>
     *
     * <p>Each call to getPushBuilder() will return a new instance
     * of a PushBuilder based off this Request. Any mutations to the
     * returned PushBuilder are not reflected on future returns.
     * @return A new PushBuilder or null if push is not supported
     */
    public PushBuilder getPushBuilder()
    {
        ...
    }

The javadoc for that method expands as

Get a PushBuilder associated with this request initialized as follows:

   - The method is initialized to "GET"
   - The headers from this request are copied to the Builder, except for:
      - Conditional headers (eg. If-Modified-Since)
      - Range headers
      - Expect headers
      - Authentication headers
      - Referrer headers
   - If the request was Authenticated, an Authentication header will be set
   with a container generated token that will result in equivalent
   authentication
   - The query string from getQueryString()
   <eclipse-javadoc:%E2%98%82=jetty-server/src%5C/main%5C/java%3Corg.eclipse.jetty.server%7BRequest.java%E2%98%83Request~getPushBuilder%E2%98%82%E2%98%82getQueryString%E2%98%82>
   - The getRequestedSessionId()
   <eclipse-javadoc:%E2%98%82=jetty-server/src%5C/main%5C/java%3Corg.eclipse.jetty.server%7BRequest.java%E2%98%83Request~getPushBuilder%E2%98%82%E2%98%82getRequestedSessionId%E2%98%82>
    value, unless at the time of the call getSession(boolean)
   <eclipse-javadoc:%E2%98%82=jetty-server/src%5C/main%5C/java%3Corg.eclipse.jetty.server%7BRequest.java%E2%98%83Request~getPushBuilder%E2%98%82%E2%98%82getSession%E2%98%82boolean>
    has previously been called to create a new HttpSession
   <eclipse-javadoc:%E2%98%82=jetty-server/src%5C/main%5C/java%3Corg.eclipse.jetty.server%7BRequest.java%E2%98%83Request~getPushBuilder%E2%98%82HttpSession>,
   in which case the new session ID will be used as the PushBuilders requested
   session ID.
   - The source of the requested session id will be the same as for this
   request
   - The builders Referer header will be set to getRequestURL()
   <eclipse-javadoc:%E2%98%82=jetty-server/src%5C/main%5C/java%3Corg.eclipse.jetty.server%7BRequest.java%E2%98%83Request~getPushBuilder%E2%98%82%E2%98%82getRequestURL%E2%98%82>
    plus any getQueryString()
   <eclipse-javadoc:%E2%98%82=jetty-server/src%5C/main%5C/java%3Corg.eclipse.jetty.server%7BRequest.java%E2%98%83Request~getPushBuilder%E2%98%82%E2%98%82getQueryString%E2%98%82>
   - If HttpServletResponse.addCookie(Cookie)
   <eclipse-javadoc:%E2%98%82=jetty-server/src%5C/main%5C/java%3Corg.eclipse.jetty.server%7BRequest.java%E2%98%83Request~getPushBuilder%E2%98%82HttpServletResponse%E2%98%82addCookie%E2%98%82Cookie>
    has been called on the associated response, then a corresponding Cookie
   header will be added to the PushBuilder, unless the Cookie.getMaxAge()
   <eclipse-javadoc:%E2%98%82=jetty-server/src%5C/main%5C/java%3Corg.eclipse.jetty.server%7BRequest.java%E2%98%83Request~getPushBuilder%E2%98%82Cookie%E2%98%82getMaxAge%E2%98%82>
    is <=0, in which case the Cookie will be removed from the builder.
   - If this request has has the conditional headers If-Modified-Since or
   If-None-Match then the PushBuilder.isConditional()
   <eclipse-javadoc:%E2%98%82=jetty-server/src%5C/main%5C/java%3Corg.eclipse.jetty.server%7BRequest.java%E2%98%83Request~getPushBuilder%E2%98%82PushBuilder%E2%98%82isConditional%E2%98%82>
    header is set to true.

Each call to getPushBuilder() will return a new instance of a PushBuilder
based off this Request. Any mutations to the returned PushBuilder are not
reflected on future returns.
The PushBuilder itself allows most of the initialisation to be overridden
if need be, but hopefully it wont:

public interface PushBuilder
{
    public String getMethod();
    public void setMethod(String method);

    public Enumeration<String> getHeaderNames();
    public String getHeader(String name);
    public void setHeader(String name,String value);

    public String getQueryString();
    public void setQueryString(String query);

    public boolean isRequestedSessionIdValid();
    public boolean isRequestedSessionIdFromCookie();
    public boolean isRequestedSessionIdFromURL();
    public String getRequestedSessionId();

    public void setRequestedSessionId(String id);

    public Cookie[] getCookies();
    public void removeCookie(String name);
    public void addCookie(Cookie cookie);

    public boolean isConditional();
    public void setConditional(boolean conditional);

    /* ------------------------------------------------------------ */
    /** Push a resource.
     * Push a resource based on the current state of the PushBuilder. If
{_at_link #isConditional()}
     * is true and an etag or lastModified value is provided, then an
appropriate conditional header
     * will be generated. If an etag and lastModified value are provided
only an If-None-Match header
     * will be generated. If the builder has a session ID, then the pushed
request
     * will include the session ID either as a Cookie or as a URI parameter
as appropriate. The builders
     * query string is merged with any passed query string.
     * @param uriInContext The URI within the current context of the
resource to push.
     * @param etag The etag for the resource or null if not available
     * @param lastModified The last modified date of the resource or null
if not available
     * @throws IllegalArgumentException if the method set expects a request
     * body (eg POST)
     */
    void push(String uriInContext,String etag,long lastModified);
}

So normal usage of the push builder should be pretty simple, but it is
flexible if need be. The key aspect of the API is that it separates
out the two phases of customising push request: firstly the customisation
for session, authentication etc that are common to all pushed
requests; secondly each pushed request can be customised with conditional
headers, session URI parameters and query string.

 Here is an example usage in Jetty's PushCacheFilter:

        // Are there associated resources to push?
        if (!primaryResource.getAssociated().isEmpty())
        {
            // Let's make sure we push them to a valid session
            request.getSession(true);

            // Is push actually supported?
            PushBuilder builder = baseRequest.getPushBuilder();
            if (builder!=null)
            {
                // Yes, lets set a header as a demo
                builder.setHeader("Via","Push Example");

                // If query strings are not relevant, then clear
                if (!isPreservingQueries())
                    builder.setQueryString(null);

                // Push each resource
                for (AssociatedResource resource :
primaryResource.getAssociated())
                {

builder.push(resource.getURI(),resource.getETag(),resource.getLastModified());
                }
            }
        }

Thoughts?





On 5 December 2014 at 19:56, Greg Wilkins <gregw_at_intalio.com> wrote:

>
> All,
>
> another thing that I think would be helpful for implementing push is an
> event for when a response is committed, but before the content has been
> sent.
>
> This is the perfect time for a push algorithm to extract information about
> associated resources (size, etag, modified, session, set-cookies etc.) and
> also a great time to start doing pushed.
>
> If we try to implement push before commit, then we may not see a newly
> valid session or some authority token set in a cookie that needs to be used
> when creating the associated requests. If we delay later, then the
> client will have already received content and may have already issued
> requests for the associated resources.
>
> Would it be possible to add something like:
>
> interface ServletResponseListener
> {
> void responseCommitted(ServletResponseEvent event);
> }
>
> class ServletResponseEvent
> {
> ServletRequest getServletRequest() {...}
> ServletResponse getServletResponse() {...}
> }
>
> thoughts?
>
>
>
> On 28 November 2014 at 16:27, Greg Wilkins <gregw_at_intalio.com> wrote:
>
>>
>> Hi all,
>>
>> I've had a few more thoughts on a HTTP Push API that benefit from a
>> little more experience and experimentation.
>>
>> One of the proposals discussed was for an API that looked something like:
>>
>> response.push(uri)
>>
>> The issue with this are:
>>
>> 1. Cannot mutate headers and container will have to work out all the
>> headers needed
>> 2. Does not work for cross context pushing (not a huge loss, but
>> breaks orthogonality of container)
>>
>> The alternative API that I proposed was
>>
>> servletContext.getRequestDispatcher(uri).push(request)
>>
>> which solves both the problems identified above as cross context can be
>> used and a wrapped request to be pushed in if the headers need to be
>> mutated.
>>
>> So this led me to start thinking what mutations are needed when
>> generating the psuedo request that will be in the push promise and which is
>> used to generate the pushed resource. There are actually quiet a few
>> that I can think of:
>>
>> - If the base request contains a session ID in it's URI, then the
>> pushed URIs need to be updated with a session ID.
>> - If the requested session ID is for an invalid session, but
>> getSession() returns a new valid session, then any session ID in the pushed
>> request (either as cookie or uri param) should be the new session ID and
>> not the old invalid one.
>> - Conditional headers need to be removed/mutated as any etags or
>> modified dates will relate to the base request and not the pushed resource.
>> - Referer header should be set as the base resource, not the base
>> resources referrer
>> - Any digest authentication headers need to be removed as they apply
>> only to the base request. We then need to decide if pushed requests need
>> to be reauthenticated and/or reauthorised?
>> - Perhaps the method needs to be mutated? Can we push GETs from POSTS?
>> - Expect 100 headers removed
>> - Range headers removed
>> - Upgrade headers removed
>>
>>
>> So this is a lot of mutating and filtering required to make a good pushed
>> request and I'm sure there are complexities and interactions yet to be
>> discovered that need to be handled. Thus I'm starting to think that
>> passing in the request as the template is not really the best way to go:
>>
>> - as it will be a bit clumsy to have to wrap the request and then do
>> all this mutating.
>> - if left to the application, it is likely that many of these things
>> will not be correctly handled
>>
>> Thus I'm leaning back to a simpler API where just the uri is passed and
>> the container is responsible for doing all these kinds of mutations.
>> Perhaps some kind of optional additional headers could be passed in?
>>
>> Anyway, in short, I think this mechanism is going to be a lot more
>> complex and tricky to get right than I originally thought.
>>
>> cheers
>>
>>
>>
>>
>>
>>
>>
>>
>> --
>> Greg Wilkins <gregw_at_intalio.com> @ Webtide - *an Intalio subsidiary*
>> http://eclipse.org/jetty HTTP, SPDY, Websocket server and client that
>> scales
>> http://www.webtide.com advice and support for jetty and cometd.
>>
>
>
>
> --
> Greg Wilkins <gregw_at_intalio.com> @ Webtide - *an Intalio subsidiary*
> http://eclipse.org/jetty HTTP, SPDY, Websocket server and client that
> scales
> http://www.webtide.com advice and support for jetty and cometd.
>



-- 
Greg Wilkins <gregw_at_intalio.com>  @  Webtide - *an Intalio subsidiary*
http://eclipse.org/jetty HTTP, SPDY, Websocket server and client that scales
http://www.webtide.com  advice and support for jetty and cometd.