Denial of service caused by Google Search Appliance OneBox module

We use a Google Search Appliance to index our Domino websites. We use Google’s OneBox module to execute FT searches to return results from a database we don’t want Google to index (basically 80,000 journal records we don’t want to be counted against our license total).

Yesterday we upgraded our Search Appliance to the latest version of Google’s software (6.2.0.G14). About an hour after switching it on, one of our Domino servers was hit by a sustained denial of service. The Agent called by the Google OneBox module had saturated all our HTTP threads. The Google box was making continuous requests to the search agent at a rate of nearly 200,000 per hour. The odd thing was, Google’s OneBox module passes across the end-user’s IP address as part of the search query and all requests were coming from 216.239.43.1 – a Google IP address.

It appears that there’s a known issue with the OneBox module that can cause this. (Bug report #2368523). Google immediately applied the patch, and after an hour the requests had stopped.

If you intend to upgrade to 6.2. I suggest you remove all OneBox modules from your front-ends before upgrading

So related to this problem, I find it easy to create a DOS on a Domino server. Calling any agent that takes a second to return results continuously creates a DOS (it might simply be a case of holding F5). Our server’s are set up as per Lotus’s recommendations. But does anyone have any tips for optimising the Domino http stack (Solaris) to avoid DOS?

4 Comments Add yours

  1. john says:

    Interesting, Lucene/Solr do not have a “license count” for the number of documents indexed — so there’s no penalty for growth or scaling.http://www.lucidimagination.com/

  2. Julie says:

    I’m in the process of implementing a GSA to index Domino databases. As a matter of interest, why did you opt to use OneBox to return the results you want the GSA to index.

    I’m currently looking at the Lotus Notes Connector to feed the GSA the documents I want the GSA to index so just wondered what the difference between using the Connector and OneBox is.

    Does using a OneBox adhere to Notes security (ACL access, readers, authors fields etc)?

    TIA

    Julie

  3. Paul says:

    I use the onebox module to return content generated from an Agent elsewhere in the system. We do not index our entire site and we have existing code that the onebox could query. Essentially, we use the OneBox functionality to add a ‘related’ section on the right hand side of the page specifically for content that we do not want the Search Appliance to crawl.

  4. Paul says:

    @Julie We index public content using a crawler. But there is an index of external resources (journal subscriptions) that we don’t index. We execute a OneBox query that runs an Agent that FTSearch’s the content in the database. It then returns via the required xml. We don’t have any queries that use Notes security.

Leave a Reply