Archive

Posts Tagged ‘Domino’

HTTP Redirect buffer overflow

February 24th, 2010

We use the Domino redirect [url] to direct to users to different web pages.

One of our systems handles the Athens devolved authentication used many by UK Universities to provide access to journal subscriptions to students off-campus.

Athens basically works by a set of redirects and can be used by users either pre journal search or post. If they choose not to pre-authenticate. When a user accesses a journal they’re directed to Athen’s login page. They can then identify their host institution. At this point a series of redirects takes place.

Basically Athens sends a long URL to a Java agent on Domino. This url has a long MD5 session hash and information to pass back to Athens so that it can then direct the user to the correct page on publisher’s website.

We process the hash, validate it and then using a private key create a new hash that includes information about the user such as a unique id number and any permission sets that apply to them.

Today, it stopped working.

A publisher recently restructured their website and this created much longer url hashes. (2200 characters long)

Every time the agent was called from the Athens, Domino generated a http error – ‘Buffer Overflow’.

After some debugging. We identified that the error was generated by the in-built Domino redirect. It appears that there is a 2048 character limit on Java redirects.

The solution is simple to implement. You just need to return url string as part of the header via the http ‘Location’. But it’s odd that Domino will accept 4kb’s long URLs but less for redirects.

Technote: ‘Buffer Overflow Exception’ error using URL redirection with a string longer than 2048

Domino , ,

Two Web Application Development Consultants posts with a focus on ‘Learning Technologies’

November 24th, 2009

Sorry, for posting a job advert. But it might be useful to any out of work Domino developers.

2 x Applications and Development Consultant (with a focus on Learning Technology)

We are looking for experienced web-based software development professionals with substantial programming experience (preferably with expertise in Lotus Domino, Lotuscript, Java but additional languages such as .NET would be of interest) to join our web and learning applications team; developing and supporting the University’s intranet and on-line learning and business support environments.

More details can be found on The University of Hertfordshire’s website

Domino ,

Server loosing permissions

June 16th, 2009

Our Domino servers (7.03) run on Solaris and we’ve recently moved them onto new hardware. As part of the move, they have been moved in Solaris zones. Unfortunately since the changes we’ve experienced a serious issue that appears to be related to server permissions. Users are loosing the ability to upload files as it appears that the server is rewriting the permissions to the ID file so root access is required. Effectively the server is loosing access to it’s own ID file. So far we’ve been unable to identify why this happens.

Domino ,

Odd Notes problems

April 27th, 2009

We all experience issues with all software and Domino / Notes is not immune but it’s really infuriating when two issues strike at once.

We’ve been moving server’s around and my Design (8.02) client stopped replicating. Not in the normal way, an invalid connection document, but one where any replication event caused an immediate NSD. After much hair pulling – and at my age that’s something you want to avoid in case it doesn’t grow back – the only solution appears to be a complete reinstall.

The second more serious issue was that one of our servers lost permissions to it’s own ID file. Although HTTP continued to serve files anything requiring the ID file stopped functioning. A quick chmod solved the problem and all functionality returned. But this is the first live server we’ve run in a Solaris zone and the how zones function is beyond my basic unix skills. Hopefully this is a simple unix configuration issue and not some fundamental problem with domino on solaris (it wouldn’t be the first time we’ve run into solaris issues). One for our Domino admin to sort out tomorrow.

Domino ,

Line Input crashes when importing Unicode

April 14th, 2009

We have a facility for staff that allows them to download an Excel marking scheme, enter student marks and then to upload it as a tab delimited file (or xml file) back into the system. Last night a member of staff using the facility crashed both servers in our cluster.

On further inspection (and a lot of development server crashes!) I finally narrowed the problem down to a single line of code

Line Input #fileID, strLine

On further investigation it appears that the user exported their marking scheme from Excel using the ‘Unicode (.txt)’ rather than ‘ Text (.txt)’ and it’s the import of Unicode that was causing Domino to crash. My quick solution to stop the server crash is to use ‘Dos2Unix’ convert the file before importing. Although it seems to alter the format so that it no long imports correctly, it’s stopped the server from crashing.

Domino , , ,

Domino issues with latest Google Search Appliance software (5.2.0.G32)

April 11th, 2009

A word of warning if you use a Google Search Appliance to index your Domino content. The latest version of the appliance software (5.2.0.G32) has a bug which means it is now case sensitive when checking for Domino rewrite and ignore rules. For example it will only exclude agents if they are correctly capitalised. i.e OpenAgent. If you have agents which might cause you some problems (say agents that delete content or send emails), make sure you add ignore statements to the exception lists before starting the index. Luckily our Google box runs with ‘student’ access so couldn’t do any damage!

Domino ,

Screen reader survey

February 21st, 2009

A survey by WebAIM on how users of screen readers interact with web pages is worth a read. It gives a small insight into how screen reader users navigate around pages and some of the problems they face. The survey failed to clarify the term Web 2.0 so the survey hasn’t helped to understand if AJAX based sites disenfranchise partially sighted users but it does confirm some of the advice accessibility experts make. It also reinforces some of the point I tried to make to the Domino developers at Lotusphere about where the new XPage technology fails to meet basic accessibility criteria, namely that screen reader users use the page’s semantics to navigate around the page. A good practice that XPage authoring using the visual interface fails to support.

When html was originally specified it was intended to be a docuument markup language and the tags selected define the page’s content – headings, paragraphs, lists, etc. As web developers we all know this, it’s really basic stuff.

Since headings are more important than paragraphs, a significant percentage of screen reader users use headings to quickly navigate around sections of the page and this confirms the advice accessibility experts have been giving for years. Define a readable document and then style it to look like an application. XPages gets this fundamentally wrong and provides no visual tools to generate standard page markup, a glaring omission that no other editor that I’m aware off fails to include.

All’s not lost. If you want to develop a semantic page it’s just a case of switching to code view and you can add normal markup and it wouldn’t take much for Lotus to add some simple page editing tools.

Hopefully surveys like this help developers see how some simple changes to their pages can make page browsing easier for a small forgotten number of web users

Accessibility, Domino, Web Design, XPages , , ,

Templates – why are so many settings not inherited?

February 10th, 2009

Ok, I’ve been using Domino long enough to know that templates don’t do what you think they should do so I shouldn’t really be surprised.

I had a problem with document locking. Some documents couldn’t be edited because there were some temporary locks being set and my code didn’t unlock them. Okay, my code was at fault, I’d accidentally allowed temporary locks when the administration server was unavailable, but since our servers only get rebooted once a week about 2 in the morning and the documents in question were created during the day, I shouldn’t have been seeing temporary locks.

On further investigation it appears that reason temporary locks were being set was because there was no administration server set in the ACL. This was despite the fact I had set one in the template the database was created from.

I’ve never understood why some features are inherited at creation and others aren’t. If the tick option for document locking is inherited and designed to use the administration server to ensure locking in a cluster, is it wrong of me to think that this should also be inherited from the template?

Domino

Domino 8.5 – reflections

January 21st, 2009

From my personal perspective the big feature in 8.5 is XPages, but 8.5 is not a one trick pony. There have been a number of other improvements, such as the ID Vault (something not applicable in our institution) and Domino Attachment and Object Service (DAOS).

DAOS

DAOS is described elsewhere. But basically, DAOS allows you to store the attachments in your Domino databases outside of the database, reducing the size of the nsf. 

Big figures are being thrown around; reducing storage size by 40-60%, removal of duplicate attachments and DAOS invisible to Domino apps

DAOS can be enabled per-database or across the entire server and it’s possible to undo the change at a later stage, which is excellent news. Also since it’s implemented at the API level, it’s invisible to the end user or developer.

Reflecting on our system, DAOS  offers:

  • Quicker backups. DAOS are stored as files on the file system so can be incrementally backed up.
  • NSF are smaller. Since these are loaded into the server’s cache for web delivery, this means more apps can stay resident in memory and so less disk IO.
  • We create 4000 modules (databases) per year and carry across teaching resources from previous years. Often resources are uploaded as attachments, and don’t change year on year, so we should see another significant space saving

Other 8.5 features and benefits

  • 8.5 contains 400 bug fixes to the nsf
  • 50% reduction in CPU for transaction logging
  • ID Vault (easier password change)
  • Domino configuration tuner for identifying performance issues with your Domino setup. This will be updated as and when, rather than being tied to a Domino release, another positive step from Lotus. 

Developments post 8.5

There will finally be a new Eclipse based lotuscript editor release with 8.5.1

Further ahead the team are investigating how XPages can consume SOAP, REST, and XML (data sources). New controls for XPages such as menu bar, toolbar, outline and general performance improvements.

They are also working on directory independence such as using user info from ldap or active directory to improve single sign-on

Domino, XPages , ,

Designer 8.5 – XPages first impressions…..

January 6th, 2009

…. confusing! I’m sure it will make sense once I start to use it. 

Had a brief look at the 8.5 discussion template and it appears all that all link properties are blank so no effort has been made to generate alternative views when Javascript is unavailable. Bad design. 

I’m not sure when I’ll be able to use XPages in anger. Our learning environment has 22,000 users with 14,000 individuals logging in daily. It’s completely web based – no Notes clients, and with the environment heavily used in teaching, we have to ensure the software is reliable before upgrading. We’ve experienced reliability issues with Domino on Solaris pre X.02/X.52 release so we tend to wait for these releases. We’re also are limited to making big system upgrades to the month of August. So it may even be 2010 before I see 8.5 live on our systems. Let’s hope 8.5 proves to be reliable and we can go sooner.

Domino ,

8.5 Discussion template

January 6th, 2009

Ok, now this is beyond a joke. Sorry, but Domino 8.5 is going to be an accessibility nightmare. I know the discussion template is used many many companies and I’ve seen some University’s base their discussion facilities on this template (not us, we wrote our own), but the new template makes NO ATTEMPT to be accessible, both from point of view of handling javascript or by using semantic HTML.

Sean Cull has kindly left his 8.5 discussion template  open for us to play with.

If you’ve got Firefox with the web developer tool bar installed. Switch off css

discuss

Notice how the page just collapses into a mess. None of the generally accepted conventions for semantic pages have been obeyed (such as marking up related links as lists).

It’s even worse if you switch off javascript. Although many of the advanced screen readers such as JAWS, integrate with Internet Explorer and deal with certain types of Javascript events, it’s generally accepted in accessibility circles that wherever possible, web developers should attempt to make as much of the page work without javascript as possible. In fact nothing in the discussion template works without javascript. All the links are to ‘#’ with the clicks being handled through events. Why, when there’s a document stored in the database is it now possible to show the document when javascript is switched off.

If the template demonstrated significant usability improvements over the existing template it would almost be acceptable, but the template is very basic. Where’s the different view modes such as viewing discussions as linear threads or the ability to filter discussions to the current conversation (i.e. limiting the view to the direct reply hierarchy). Where’s the facility to view the message you’re writing your reply to? Where’s the lookup when tags are added?

A core template such as this, should be accessible. If I was evaluating this product for use within the University it would not make it past the tender. It would fail on every accessibility point.  Sorry, it’s simply that bad.

Domino, Web Design , , ,

Lotusphere online

January 6th, 2009

Lotusphere online has been available for several days. Although I’ve still not been sent an email informing me that it’s live (just like last year), unlike last year my email and confirmation number work. 

A number of features are created using XPage technology and this is the first application I’ve seen that uses this functionality. I’ve mentioned that I’m concerned that XPages will provide Domino designers with tools that they might use that caused them to embed an accessibility time-bomb into their company’s intranets. So I thought I’d take a brief look at how accessible this IBM site is and knocked up a quick video walk through.

Domino, Lotusphere , , ,

“We Are Unable to Accept Your Lotusphere 2009 Birds-of-a-Feather abstract”

December 4th, 2008

This year I put in two Lotusphere abstracts, a BOF and presentation on ‘accessible web design‘. To be honest neither abstract was very good having been put together at the last minute. If I’m honest, I’m relieved that I won’t have the worry of presenting and can just relax and enjoy the week. But I’m always uncomfortable attending conferences when I’m not presenting, especially at such an expensive conference (why doesn’t IBM sponsor educational establishments – most academic conferences are half the price of this one?)

Accessibility is extremely important in my job. A large proportion of ‘Web 2.0′ sites are designed without any real consideration of accessible design and I have a concern that Domino 8.5 will encourage the Domino community into adopting Web 2.0 technologies without understanding the problems they might be creating.

When I was a design student I visited the ‘Royal National Institute for the Blind’ and met one of their designers. He explained that the RNIB believed that spending some time with a student designer to explain some of the of the issues surrounding designing accessible products, would be repaid many times over, if that designer adopted sympathetic practices in industry. So I’ve always tried to show as much consideration as possible when designing websites, for the issues faced by people with disabilities.

At Lotusphere 2008 there were lots of wizzy AJAX being demonstrated. Not once in any of the session I attended, did a presenter discuss the accessibility issues created by the inappropriate adoption of AJAX. For example, the excellent ‘Great Code Giveaway’, showed developers how to make views exciting through AJAX. The fact that if you had Javascript disabled, you got nothing, wasn’t mentioned. Looking at the code afterwards, there were simple approaches that could have allowed the code to degrade to a more basic accessible design. I imagined Domino designers rushing back creating a whole host of inaccessible websites.

After chatting to some of the Designer and Domino teams, I wasn’t filled with confidence that Lotus are really tackling accessibility in Designer 8.5. So although I’m relieved I don’t have to present, it’s clear to me at least, that Lotus should have scheduled one of the accessibility sessions.

I hope it doesn’t come across that I’m bitter for not getting my sessions on to the agenda, I’m really am not. There was a far better accessibility abstract posted to IdeaJam that also hasn’t been included in the programme. It’s just I know how frustrated I was at last year’s Lotusphere that accessiblity issues were completely missing from the agenda, I’ll suspect this year I’ll be be doubly so.

Domino, Web Design , , ,

A nasty little change in Domino 8

November 28th, 2008

We recently upgraded one of our development servers to Domino 8 and ran into problem with subforms. It appears that you can no longer include a subform twice on the same page. So for example if you’ve included ‘next’ buttons using a the same subform at the top and bottom of a discussion page, the page will no longer display.

I guess this has been a bug within Domino that I know I’m not the only person to exploit, but I really can’t understand why Lotus would decide to fix this bug. Why fix something that actually improved functionality? Surely they should have been building on this ‘bug’ maybe parametrising it, turning it into a useful new feature instead of removing it.

This is going to cause me a significant amount of work when I upgrade our main system to V8.

Domino ,

WordPress not Domino

November 27th, 2008

I’ve been heavily using Domino for development for 8 years and though I’ve had many issues with it (anything relating to Java and the Web on Solaris), I’ve found it really easy to build complex web applications.

Although I have a number of Domino servers available to me, work regs mean that I’m not allowed to use them for a personal web presence. So despite having developed blogging facilities that are used within the intranet,  I need to find another blogging solution. After a brief play with some of the alternatives I’ve plumped for good old WordPress hosted on a Mediatemple Grid Service.

For $20 per month (I got it for $17 – after searching for a discount code), I get what appears so far (after two hours of playing) a really comprehensive service, offering php, mysql, and gives me something that I’ve been itching to play with, Rails.

I find it disappointing that not only isn’t there any real options for cheap personal Domino hosting, but that IBM don’t appear to be making any effort at pushing Domino to new developers by making ‘personal’ Domino server available for free. How is Domino meant to compete against the likes of php and developing languages such as Rails?

Domino

Document locking

November 27th, 2008

StudyNet provides students with access to an online website for everyone of their modules. Each module provides staff with tools to create resources, podcasts, video, discussions, etc. One feature that’s steadily developed is our module ‘wiki’ facility. 

This year there’s been a significant number of staff using the facility with large cohorts and this has led to a problem when students edit a page at the same time requiring staff to manually merge the contributions.

There appeared to be two solutions. To provide better tools to merge contributions across a number of documents or to lock pages when somebody edited the document. Locking appeared to be the best and easiest solution. It was how to achieve this in a cluster.

I’d been considering implementing my own locking system but after a web search it was clear that the ‘document locking’ facility introduced with Domino 6 that I’d discounted because it ‘doesn’t work on the web’, did via WQO and WQS agents and it was just Lotus’ usual flaky documentation that was wrong.

For those who have been as slow as me, here’s a brief rundown of some of the issues I’ve dug up and issues I’ve faced with our implementation

The first thing to be aware is that the locking option is switched off by default. To enable it you need to assign a master lock server, by assigning an administration server to your database. (Advanced option in the ACL). When a document is edited, the lock is recorded on this server. You will also need to make sure that the ‘enforce a consistent ACL across replicas’ is also set. Once this is done you can enable locking via the database properties.

Locking is actually pretty simple to implement. There’s three main methods, document.lock, document.unlock and document.lockhaolders. When you set a lock, one of two types can be set, permanent and semi-permeant lock. In a web cluster, semi-permeant locks are set if the admin server is unavailable. I found some references to bugs where the semi-permeant locks aren’t released and become permeant locks, so in our case I’ve avoided them. If our admin server is down, the error is caught and locking is disabled. 

Once a document is locked, no one else can save the document until the lock is released. Unfortunately, these are permanent locks. You have to clear the lock otherwise no one, including managers, can save the document. It seems that many people use a scheduled agent to free the locks overnight. For us, this isn’t an appropriate solutions.

StudyNet provides a website (domino database) for every module of every course that run at the University, approximately 4000 each year. We also keep live 5 years worth of websites so that students can refer back to their previous modules – totalling over 20,000 active websites. It’s just not practical to run overnight jobs checking all the documents on all these databases for locks. So I’ve implemented a self-clearing lock system.

When a student edits the page, a lock is set via a web query open agent. The page is locked for 30 minutes after which the lock will be released by the next student viewing the page (also through a WQO agent). A javascript autosave timeout is also set on the edit page for 25 minutes. The save executes a web query save agent clearing the lock. Any other student viewing the page will see a message warning them that the page is unavailable for editing until a particular time. After 30 minutes is a student hasn’t saved or has navigated away the lock will be cleared. Though it won’t be fool proof, it should significantly reduce the number of conflicts.

The locking method adds two fields to the edited document on the master lock server. These are ‘$Writers‘ array and datetime ‘$WritersDate’

Although it’s possible to return the $Writers array using the document.lockholders function, there isn’t a method to return the $writesdate option. This appears to be an obvious oversight but a extremely annoying one because these fields are not replicated out through the cluster – why would they, the whole point of assigning a single server to handle locks is to get round the replication delay. Since my code is designed to free up locks after 30 minutes, I’m checking the ACL.adminstrationserver property and then (if necessary) opening the version of the document being viewed on the other server to check the lock date. 

When a document is locked you cannot delete the document (or copy?). In our case we have agents that handle delete wiki documents, so it was just a case of checking for a lock before deleting.

Another important point is that the lock option is not inherited from templates. So in our case, we got 20,000 websites that do not have the admin or locking option set. So you’ll need to update them using an agent or do what we’ve done and include in your lock code a check to see if the current database’s ‘IsDocumentLockingEnabled’.

Domino ,