I have searched and searched for an answer to the problem as and I can't find anything I am guessing it isn't a problem for anyone else and therefore something stupid I am doing.
Looking at these two resources it seems easy enough to create a custom class that implements javax.naming.spi.ObjectFactory which I have done (pretty much copied the example in the first post below for JNDIPropertiesFactory), packaged and added this jar to [glassfish]/domains/domain1/lib/ext (also tried [glassfish]/lib and as a single class to [glassfish]/domains/domain1/lib/classes/package/structure/JNDIPropertiesFactory)
http://forums.java.net/jive/thread.jspa?messageID=316228
http://blogs.sun.com/chengfang/entry/how_to_create_custom_resources
Then creating custom resource using the admin console of Glassfish, using resource type of java.util.Properties and resource factory of package.structure.JNDIPropertiesFactory. This shows as being created no problem in the console logs.
In my app and looking up JNDI resource in normal way context.lookup(....) and casting to Properties object. With some logging I can see that GlassFish is performing the lookup correctly but when it comes to create the correct object i.e. Properties, it doesn't and just returns the javax.naming.Reference object from JNDI directly - which in turn throws a ClassCastException (unsurprisingly).
In my custom ObjectFactory implementation getObjectInstance method I have as a test just thrown an exception but this doesn't even happen. I can only assume that this isn't registered with the server in some way (other than as a factory resource type when defining the custom resource).
The question is this - how then does the custom objectfactory implementation get called? Or any other pointers would be helpful.
snippet of log file when doing the lookup...
[#|2009-04-28T12:09:41.343+0100|FINE|sun-appserver9.1|javax.enterprise.system.core.naming|_ThreadID=20;_ThreadName=httpSSLWorkerThread-8080-1;ClassName=com.sun.enterprise.naming.SerialContext;MethodName=<init>;_RequestID=37c5dc8a-fc3e-41ed-ae68-8a2391199777;|SerialContext ==> SerialContext instance created : com.sun.enterprise.naming.SerialContext_at_1748654|#]
[#|2009-04-28T12:09:41.359+0100|FINE|sun-appserver9.1|javax.enterprise.system.core.naming|_ThreadID=20;_ThreadName=httpSSLWorkerThread-8080-1;ClassName=com.sun.enterprise.naming.SerialContext;MethodName=lookup;_RequestID=37c5dc8a-fc3e-41ed-ae68-8a2391199777;|SerialContext ==> doing lookup with com.sun.enterprise.naming.SerialContext_at_1748654|#]
[#|2009-04-28T12:09:41.359+0100|FINE|sun-appserver9.1|javax.enterprise.system.core.naming|_ThreadID=20;_ThreadName=httpSSLWorkerThread-8080-1;ClassName=com.sun.enterprise.naming.SerialContext;MethodName=lookup;_RequestID=37c5dc8a-fc3e-41ed-ae68-8a2391199777;|SerialContext ==> looking up : java:comp/env/param/viewDuties/properties|#]
[#|2009-04-28T12:09:41.359+0100|FINE|sun-appserver9.1|javax.enterprise.system.core.naming|_ThreadID=20;_ThreadName=httpSSLWorkerThread-8080-1;ClassName=com.sun.enterprise.naming.java.javaURLContext;MethodName=lookup;_RequestID=37c5dc8a-fc3e-41ed-ae68-8a2391199777;|In javaURLContext.lookup, name = java:comp/env/param/viewDuties/properties serialcontext...null|#]
[#|2009-04-28T12:09:41.359+0100|FINE|sun-appserver9.1|javax.enterprise.system.core.naming|_ThreadID=20;_ThreadName=httpSSLWorkerThread-8080-1;ClassName=com.sun.enterprise.naming.NamingManagerImpl;MethodName=lookup;_RequestID=37c5dc8a-fc3e-41ed-ae68-8a2391199777;|serialcontext in NamingManagerImpl..null|#]
[#|2009-04-28T12:09:41.359+0100|FINE|sun-appserver9.1|javax.enterprise.system.core.naming|_ThreadID=20;_ThreadName=httpSSLWorkerThread-8080-1;ClassName=com.sun.enterprise.naming.NamingManagerImpl;MethodName=lookup;_RequestID=37c5dc8a-fc3e-41ed-ae68-8a2391199777;|NamingManagerImpl : looking up name : java:comp/env/param/viewDuties/properties|#]
[#|2009-04-28T12:09:41.375+0100|FINE|sun-appserver9.1|javax.enterprise.system.core.naming|_ThreadID=20;_ThreadName=httpSSLWorkerThread-8080-1;ClassName=com.sun.enterprise.naming.NamingManagerImpl;MethodName=getMangledIdName;_RequestID=37c5dc8a-fc3e-41ed-ae68-8a2391199777;|Component Id: D:-workspace-.metadata-.plugins-org.eclipse.wst.server.core-tmp0-viewDuties_/viewDuties|#]
[#|2009-04-28T12:09:41.375+0100|INFO|sun-appserver9.1|javax.enterprise.system.stream.out|_ThreadID=20;_ThreadName=httpSSLWorkerThread-8080-1;|getPropertiesFromJNDI 6721141 [httpSSLWorkerThread-8080-1] DEBUG uk.police.kent.util.sap.ConnectionFactory - object from jndi => javax.naming.Reference
|#]
[#|2009-04-28T12:09:41.390+0100|WARNING|sun-appserver9.1|javax.enterprise.system.stream.err|_ThreadID=20;_ThreadName=httpSSLWorkerThread-8080-1;_RequestID=37c5dc8a-fc3e-41ed-ae68-8a2391199777;|java.lang.ClassCastException: javax.naming.Reference
at uk.police.kent.util.sap.ConnectionFactory.getPropertiesFromJNDI(ConnectionFactory.java:68)
[Message sent by forum member 'conradcrampton' (conradcrampton)]
http://forums.java.net/jive/thread.jspa?messageID=344125