Unable to reliably deploy XPages using XSCD

We’re late to the XPages game and it’s only in the last few weeks I’ve felt confident enough to push XPages into our system’s templates. I only allow new Domino tech to be deployed when I’m confident that the technology works because we run on a very constrained 32-bit Solaris Domino install. I was ready to use XPages on 8.5.2 but we ran into issues pushing out a XPage Single Copy Design to websites. I tried using standard Domino single copy design templates but whenever we updated an XPage we received various errors. So I waited until 8.5.3 as this version was meant to fix our issue with XSCD.

Although we can now push our XSCD’s through our templates, unfortunately moving to XSCD has not solved our issue. We are still unable to reliably deploy changes to XPages to all servers in our cluster without receiving the error:

 

HTTP Web Server: Command Not Handled Exception

 

In the XPages error log:

Exception Thrown
 java.lang.NullPointerException
 at com.ibm.xsp.webapp.FacesServletEx.service(FacesServletEx.java:87)
 at com.ibm.xsp.webapp.DesignerFacesServlet.service(DesignerFacesServlet.java:103)
 at com.ibm.designer.runtime.domino.adapter.ComponentModule.invokeServlet(ComponentModule.java:576)
 at com.ibm.domino.xsp.module.nsf.NSFComponentModule.invokeServlet(NSFComponentModule.java:1267)
 at com.ibm.designer.runtime.domino.adapter.ComponentModule$AdapterInvoker.invokeServlet(ComponentModule.java:847)
 at com.ibm.designer.runtime.domino.adapter.ComponentModule$ServletInvoker.doService(ComponentModule.java:796)
 at com.ibm.designer.runtime.domino.adapter.ComponentModule.doService(ComponentModule.java:565)
 at com.ibm.domino.xsp.module.nsf.NSFComponentModule.doService(NSFComponentModule.java:1251)
 at com.ibm.domino.xsp.module.nsf.NSFService.doServiceInternal(NSFService.java:598)
 at com.ibm.domino.xsp.module.nsf.NSFService.doService(NSFService.java:421)
 at com.ibm.designer.runtime.domino.adapter.LCDEnvironment.doService(LCDEnvironment.java:341)
 at com.ibm.designer.runtime.domino.adapter.LCDEnvironment.service(LCDEnvironment.java:297)
 at com.ibm.domino.xsp.bridge.http.engine.XspCmdManager.service(XspCmdManager.java:272)

 

This appears irrespective of the complexity of the XPages. It doesn’t matter if the XPages included complex code or simply says ‘Hello World’. The server the code is replicated to will 95% of the time display the error message, and the other servers in the cluster fail around 50% of the time. At the point that the new XPages fails. Every XPages in the database fails.  The odd thing is that after an hour the page starts to work. Which suggests some sort of cache is being flushed.
 

Things I’ve tried unsuccessfully:

  1. Make a minor change and re-replicate.
  2. Make a minor change to the XSCD template directly on the server.
  3. Use the ‘Build’ function to rebuild the database and replicate.
  4. Rebuild directly on the server.
  5. Re-sign the database and replicate
  6. Re-sign the database on the server.
  7. Delete page and redeployed
  8. HTTP refresh
  9. Accessing an XPages immediately after the JVM purge period has expired.
  10. Changed the page persistence from disk to ‘keep pages in memory’
  11. Changed the page persistence to ‘keep current page in memory’
  12. Changed the minimum supported release to 8.5.3
  13. Uninstalled Notes client and reinstalled, build, re-sign and re-deploy.

 

Things I’ve tried successfully:

  1. Restarting HTTP server (unacceptable with 900 active web users per server, many of whom are students assignments)

 

Because we run on Solaris and IBM refuse to move Solaris to a 64-bit platform, our JVM is set to 256mb and is set to use OS memory for the HTTP task otherwise our server collapses due to a lack of headroom. So we do not have much leverage to change this figure.

Has anyone experienced this problem?

5 Comments Add yours

  1. David Leedy says:

    I know that at my company we use this feature HEAVILY… something like 60,000 nsf’s give or take. I’m not that involved in that project but I do THINK we need to “Restart Task HTTP” server when a change is made…

    I’m not sure what the issue is… but you’re not alone at least.

  2. Michael says:

    Are you sure the option to recompile xpages manually is set ? (I don’t speak about Eclipse automatic build, which should be disabled too !)

  3. Paul says:

    Thanks for the feedback.

    @David Leedy – the old IT solution of ‘switching it on an off’ never fails. But it’s disappointing that so many Domino releases require this solution.

    @Michael – thanks for the suggestion but the option is set.

  4. Paul,
    We found an issue that happens when an application times out, leading to a null pointer exception. This happens with both 852 and 853, and it should be fixed in 853 FP1, at least we’re pushing to get the fix in.

  5. Paul says:

    Thanks. Hopefully it will fix the problem so that we make wide scale use of XPages within our system.

Leave a Reply