We’re late to the XPages game and it’s only in the last few weeks I’ve felt confident enough to push XPages into our system’s templates. I only allow new Domino tech to be deployed when I’m confident that the technology works because we run on a very constrained 32-bit Solaris Domino install. I was ready to use XPages on 8.5.2 but we ran into issues pushing out a XPage Single Copy Design to websites. I tried using standard Domino single copy design templates but whenever we updated an XPage we received various errors. So I waited until 8.5.3 as this version was meant to fix our issue with XSCD.

Although we can now push our XSCD’s through our templates, unfortunately moving to XSCD has not solved our issue. We are still unable to reliably deploy changes to XPages to all servers in our cluster without receiving the error:

HTTP Web Server: Command Not Handled Exception

In the XPages error log:

Exception Thrown
 java.lang.NullPointerException
 at com.ibm.xsp.webapp.FacesServletEx.service(FacesServletEx.java:87)
 at com.ibm.xsp.webapp.DesignerFacesServlet.service(DesignerFacesServlet.java:103)
 at com.ibm.designer.runtime.domino.adapter.ComponentModule.invokeServlet(ComponentModule.java:576)
 at com.ibm.domino.xsp.module.nsf.NSFComponentModule.invokeServlet(NSFComponentModule.java:1267)
 at com.ibm.designer.runtime.domino.adapter.ComponentModule$AdapterInvoker.invokeServlet(ComponentModule.java:847)
 at com.ibm.designer.runtime.domino.adapter.ComponentModule$ServletInvoker.doService(ComponentModule.java:796)
 at com.ibm.designer.runtime.domino.adapter.ComponentModule.doService(ComponentModule.java:565)
 at com.ibm.domino.xsp.module.nsf.NSFComponentModule.doService(NSFComponentModule.java:1251)
 at com.ibm.domino.xsp.module.nsf.NSFService.doServiceInternal(NSFService.java:598)
 at com.ibm.domino.xsp.module.nsf.NSFService.doService(NSFService.java:421)
 at com.ibm.designer.runtime.domino.adapter.LCDEnvironment.doService(LCDEnvironment.java:341)
 at com.ibm.designer.runtime.domino.adapter.LCDEnvironment.service(LCDEnvironment.java:297)
 at com.ibm.domino.xsp.bridge.http.engine.XspCmdManager.service(XspCmdManager.java:272)

This appears irrespective of the complexity of the XPages. It doesn’t matter if the XPages included complex code or simply says ‘Hello World’. The server the code is replicated to will 95% of the time display the error message, and the other servers in the cluster fail around 50% of the time. At the point that the new XPages fails. Every XPages in the database fails.  The odd thing is that after an hour the page starts to work. Which suggests some sort of cache is being flushed.
 

Things I’ve tried unsuccessfully:

1. Make a minor change and re-replicate.
2. Make a minor change to the XSCD template directly on the server.
3. Use the ‘Build’ function to rebuild the database and replicate.
4. Rebuild directly on the server.
5. Re-sign the database and replicate
6. Re-sign the database on the server.
7. Delete page and redeployed
8. HTTP refresh
9. Accessing an XPages immediately after the JVM purge period has expired.
10. Changed the page persistence from disk to ‘keep pages in memory’
11. Changed the page persistence to ‘keep current page in memory’
12. Changed the minimum supported release to 8.5.3
13. Uninstalled Notes client and reinstalled, build, re-sign and re-deploy.

Things I’ve tried successfully:

1. Restarting HTTP server (unacceptable with 900 active web users per server, many of whom are students assignments)

Because we run on Solaris and IBM refuse to move Solaris to a 64-bit platform, our JVM is set to 256mb and is set to use OS memory for the HTTP task otherwise our server collapses due to a lack of headroom. So we do not have much leverage to change this figure.

Has anyone experienced this problem?

We’ve just implemented a Dojo Date picker to replace an old calendar tool we were using.

In testing, the date picker correctly defaulted to the European date format of dd/mm/yyyy. However, all this testing was on PCs. Testing the same code on my Mac in Safari, the date format has switched to US format. My Mac is setup as a UK machine with UK date format, but it’s still defaulted to incorrect date format.

As we’re a University with many international staff and students, assuming that their computers will be set to European format would not be sensible and as the date is being stored in Domino and Oracle we need control over the date formats to ensure the correct date objects are created.

Luckily there’s a very simple solution. If you’re using html markup and the dojo parser to generate the date picker, you simply have to add:

constraints={datePattern:’dd/MM/yy’}

to your input box’s html and it will default to the European format.

We’ve just upgraded a server to 8.5.3. We have an agent that runs as a ‘web user’ but has ‘full admin rights’. The agent basically reads a view with documents that have readers fields and assembles them for display to the user. The agent needs to grab info from a protected database so has ‘full administration rights’. Previously the documents were read with the ‘users’ access permissions. So readers fields were obeyed. Various versions of this code has run this way for 10 years. Under 8.5.3 this is not the case. The readers fields are being ignored. Has anyone else experienced this issue?

I’m posting this on behalf of my colleague David Harding (@dharding) who’s been investigating XPages Single Copy Design. I think it’s best I leave it to Dave to explain:

We are looking to roll out XPages Single Copy Design to several thousand databases.  So far, we have not been able to find a recommended practice for rolling out the XPages Single Copy Design flag and template path to all of these databases.

Documentation on the Lotus Notes and Domino Application Development wiki (http://www-10.lotus.com/ldd/ddwiki.nsf/dx/Single_Copy_XPage_Design#Does+employing+SCXD+typically+have+a+positive+or+negative+affect+on+speed+of+performance%3F) suggests that setting the options in the template should propagate to databases that inherit from that template.

Quote from page:

“Is it true that selecting or de-selecting SCXD in a template does not propagate to applications already created with that template?

This is not the case. If a template is set to use a SCXD database and an application is created from this template then the new application will have the XPages from the SCXD database. And changing the template to say another SCXD or no SCXD database set will reflect in the database that inherits its design from the template.”

In testing, we have not found this to be the case.  Testing adding in the Xpage template path and checking the flag in the template*, combined with all combinations of HTTP Restarts and Design updates do not seem to visibly update the database.properties file in the inheriting database to include the $XPageSharedDesign and $XpageSharedDesignTemplate items.

*When I say Template here, I am referring to the ntf file that the database inherits from, not the Xpages source database that will be the path in the $XpageSharedDesignTemplate item in the database.properties file.

In an attempt to find an alternative, reliable method of deploying this setting we have looked at using a NotesNoteCollection to access the database.properties file and adding in the values.  There doesn’t seem to be an obvious way to locate this file, though tests have indicated that it is the only one in the database with a $Daos item, so we have been able to get a hold of it by filtering for that in the NotesNoteCollection.

So the questions are, is there a ‘best practice’ for rolling out the Single Copy XPage flag? Is the NotesNoteCollection method a no-go area?  Why are our experiences with the template not in line with the claims in the aforementioned quote?

We’re in the process of planning our move to DAOS and we’re confused about whether or not to use LZ1 compression. I’m wondering if anyone can help clarify how compression works for web content.

In our environment, we have no Notes clients, everything is web-based. All the recommendations I’ve read say that we should switch on LZ1 compression. However, when we switch on compression for web uploads, our test files appear to be compressed using Huffman and not LZ1. So when we upload an existing file, we end up with two files in the DAOS vault as the compression types are different.

Should this be happening? Or have we missed some obvious configuration option that ensure web uploads are compressed as LZ1?

If you use Domino’s HTTP GZIP compression, you may have issues with your Google Search Appliance reporting:

Malformed HTTP header: empty content

It appears that the Google box will tell Domino that it accepts compressed content in the HTTP headers even though it is unable to process the content. Luckily I found an easy solution on the Google Search Appliance support forum.

The solution is to add headers to the Google Box’s http requests.

1. Open the GSA admin interface.
2. Select ‘Crawl and Index‘ -> ‘HTTP Headers‘
3. In the ‘Additional HTTP Headers for Crawler’ enter:

Accept-Encoding: *;q=0

If you are one of the few Solaris users and use the ‘Dir$’ function to read directories in your administrative routines to process databases in a directory. Be aware that it doesn’t work for databases over 2GBs in size. They’re omitted from list.

We PMRed it. But it’s already been closed. Apparently it’s a code limitation and won’t be getting fixed. I expect it will be all that 32-bit Solaris code we’re still having to use. Don’t you just love having your hardware tied down by software?

The amazing Watson playing and beating human competitors at Jeopardy.

Some photos taken at Lotusphere

With no mention of Lotus during the conference I thought ‘That’s Not My Name’ by the ‘Ting Tings’ would be appropriate.

All the photos have been added to the Flickr ‘Lotusphere 2011 – Official Group’

We’ve been suffering from various http memory related issues. We run on Solaris and Domino on Solaris is 32bit (and apparently staying this way). So despite having 64GB of memory in our servers, we’re constrained by the 4GB shared memory limit.

The issues seem to relate to Java. At various times agents stop working due to memory issues. Despite going through every Java agent with a fine tooth comb, the issues still re-occur. Increasing the Java pool size doesn’t help because this causes a different issue. Because we allow staff to upload files up to 45MB in size, we get file upload issues as the memory fragments and Domino cannot assign a contiguous memory block. (Setting our one 8.5 server to use the ‘buffer’ doesn’t appear to work either).

The problem has been with Lotus for some considerable time and we’ve been going though the process of adding more and more debug information. These settings are generating GBs of logs every week. So far Lotus Support have been unable to pinpoint which agent is causing the memory leak.

For the last two weeks we’ve been running with some new settings suggested by Lotus Support. The new settings corresponded to a peak period, end of Semester A exams. Despite the increased load, both servers have running without memory issues for the full week between backups. So I thought I’d share the settings:

The HTTP setting is the important one as we don’t use our Domino setup for email, and seems to move the http process out of the shared memory heap and therefore allows us to increase the Java heap without side effects. It would be nice to have more information as to what this setting means. For example, how much memory can the HTTP server consume? Is it the full 32 bit address space? Hopefully somebody at Lotusphere will know. But it does seem to be an incredibly useful setting that could be enabled by default on 32bit Domino servers with large amounts of memory.