Unable to reliably deploy XPages using XSCD
We’re late to the XPages game and it’s only in the last few weeks I’ve felt confident enough to push XPages into our system’s templates. I only allow new Domino tech to be deployed when I’m confident that the technology works because we run on a very constrained 32-bit Solaris Domino install. I was ready to use XPages on 8.5.2 but we ran into issues pushing out a XPage Single Copy Design to websites. I tried using standard Domino single copy design templates but whenever we updated an XPage we received various errors. So I waited until 8.5.3 as this version was meant to fix our issue with XSCD.
Although we can now push our XSCD’s through our templates, unfortunately moving to XSCD has not solved our issue. We are still unable to reliably deploy changes to XPages to all servers in our cluster without receiving the error:
HTTP Web Server: Command Not Handled Exception
In the XPages error log:
Exception Thrown java.lang.NullPointerException at com.ibm.xsp.webapp.FacesServletEx.service(FacesServletEx.java:87) at com.ibm.xsp.webapp.DesignerFacesServlet.service(DesignerFacesServlet.java:103) at com.ibm.designer.runtime.domino.adapter.ComponentModule.invokeServlet(ComponentModule.java:576) at com.ibm.domino.xsp.module.nsf.NSFComponentModule.invokeServlet(NSFComponentModule.java:1267) at com.ibm.designer.runtime.domino.adapter.ComponentModule$AdapterInvoker.invokeServlet(ComponentModule.java:847) at com.ibm.designer.runtime.domino.adapter.ComponentModule$ServletInvoker.doService(ComponentModule.java:796) at com.ibm.designer.runtime.domino.adapter.ComponentModule.doService(ComponentModule.java:565) at com.ibm.domino.xsp.module.nsf.NSFComponentModule.doService(NSFComponentModule.java:1251) at com.ibm.domino.xsp.module.nsf.NSFService.doServiceInternal(NSFService.java:598) at com.ibm.domino.xsp.module.nsf.NSFService.doService(NSFService.java:421) at com.ibm.designer.runtime.domino.adapter.LCDEnvironment.doService(LCDEnvironment.java:341) at com.ibm.designer.runtime.domino.adapter.LCDEnvironment.service(LCDEnvironment.java:297) at com.ibm.domino.xsp.bridge.http.engine.XspCmdManager.service(XspCmdManager.java:272)
This appears irrespective of the complexity of the XPages. It doesn’t matter if the XPages included complex code or simply says ‘Hello World’. The server the code is replicated to will 95% of the time display the error message, and the other servers in the cluster fail around 50% of the time. At the point that the new XPages fails. Every XPages in the database fails. The odd thing is that after an hour the page starts to work. Which suggests some sort of cache is being flushed.
Things I’ve tried unsuccessfully:
- Make a minor change and re-replicate.
- Make a minor change to the XSCD template directly on the server.
- Use the ‘Build’ function to rebuild the database and replicate.
- Rebuild directly on the server.
- Re-sign the database and replicate
- Re-sign the database on the server.
- Delete page and redeployed
- HTTP refresh
- Accessing an XPages immediately after the JVM purge period has expired.
- Changed the page persistence from disk to ‘keep pages in memory’
- Changed the page persistence to ‘keep current page in memory’
- Changed the minimum supported release to 8.5.3
- Uninstalled Notes client and reinstalled, build, re-sign and re-deploy.
Things I’ve tried successfully:
- Restarting HTTP server (unacceptable with 900 active web users per server, many of whom are students assignments)
Because we run on Solaris and IBM refuse to move Solaris to a 64-bit platform, our JVM is set to 256mb and is set to use OS memory for the HTTP task otherwise our server collapses due to a lack of headroom. So we do not have much leverage to change this figure.
Has anyone experienced this problem?