Beyond 2.0

A few of my colleagues at work have been twittering about doing something post Web 2.0.  So it kind of got me thinking what would be next (aside from the obvious Web 3.0 + Semantic Web).  Then I remembered about the talk of the long tail.

My guess is the Web 3.0 + Semantic web stuff as time passes would end up in the left part where the dominating market is.  In that domain, you would also have all the social networking, on-line applications, tagging and other on-line applications.  These used to be in the long tail, but has rapidly been shifting to be the dominant market.

What's in the long tail?

So what goes on the long tail now? One may think that the traditional off-line tools and products are going to the long tail, but that's not necessarily the case.  It is true that they are losing the market share, but they are still pretty dominant.

So what does that leave us?  Do we just give up and go fishing, go on a cruise, or go flying off for a vacation?  Well if we do chances are we'd be disconnected from the world (especially when some of those places do not have cell phone or Wi-Fi).

It is nice to be disconnected once in a while to recollect your thoughts, but it would also be nice to share those thoughts back to the world once we come back from our hiatus.

That's what I think the new long tail would be.

The "Partially On-line" application

In our world today there are many applications out there on the web that help us connect with others or share our knowledge.  However, they all require us to be connected which is not guaranteed.

Some common business scenarios are:

• Workplace inspections on oil rigs.
• Child safety intake in the boonies.
• Deep sea floor analysis.
• Satellite inspections.
• Decentralized version control.

All of those scenarios require someone to capture or process information but not have any access to the Internet or even cell phone towers.

What is being done

This problem is nothing new as there have been much work on these already.  Here are some examples of what is out there and my comments about them.

Technology	Comments
Lotus Notes/Domino	This is the product that leads the pack in terms of adoption within many businesses.  However, it is ridiculously heavy on resources (especially on version 8).  It is also forgetting its roots and requiring more connectivity (it spits out many dialogs and messages when you don't have connection to the servers).
Google Gears	This is gaining traction now as a de-facto implementation for partially on-line applications.  However, it requires a lot of re-work and the off-line application does not work the same way as an on-line application. It is also not based on any standards unlike...
HTML5	This is the standard way of storing database data within the browser.  However, WebKit is the only popular toolkit that uses this so far. Also it has the same problem as Google Gears where the off-line application is different from the on-line application.
GIT	One of the most popular decentralized version control systems out there. Unfortunately, it is only dealing with files.  Not necessarily an application. However, using this as the basis of application data storage could be the future.
Synchronization Technologies	There are many out there but the concept is to synchronize the data between various devices. But these are just data not applications.

What else can be done

As I specified above there have been some work already.  However, for each of them I had comments on because they can still be improved.

Ideally what I'd like to see are:

• A small application server that runs on the local machine.
• The server can be used as a proxy so if we request a URL and it is one of those disconnected apps it will show correctly on our browser (or other UI).
• It has a data synchronization thread constantly running in the background.

The data store would have the following properties.

• A document based data store like Lotus Notes, this provides the simplest method of managing data.
• Field level comparison support for each document.  That way developers can provide an easy interface for users to compare conflicts.
• XQuery or SQL support on querying the data.
• Document Lock support
• Automatic merge/updates like most version control systems.
• Local version history (so you can go back in time with changes that occurred locally).
• Archiving support (allows you to push data to a separate store for things that you don't use often).
• Partial document support (allows you to break the document into smaller parts and prioritize which fields to download first).
• The URL of the application is part of the namespace and is used in conjunction with the GUIDs to ensure uniqueness across applications.

Deployment would have the following properties.

• A simple EAR or other deployment archive can get downloaded an installed onto the local application server.
• The deployment archive would have a descriptor that specifies the URL to use and such.
• For security reasons, the URL in the descriptor must match the URL that it the application was downloaded from.

More stuff

My ideas above will provide disconnected support for the application and the data.  However, if we look further down the long tail again, we see one other use case that I have bypassed so far.  The feature with the dominant market and with the solutions I have outlined above has one slightly nasty limitation.  A single point of failure known as the server.

So what do we do about it?

Well I am thinking that we would take GIT's approach of creating a "web of trust" so we only get data from people that we trust.  The difference would be with GIT it is just code.  What if we pushed it so it has data and the application?

Then we'd have a social network based on local trusts rather than something that is run on a server (which can be hacked or sold off).

It is not that much further off from what I have written above.  The main difference would be the deployment, which instead of a URL would use some sort of secure certificate.

Unfortunately, the intellect needed to use these applications are not a majority in this current generation of people.  We have not hit a bubble where facebook, friendster, twitter, google or linkedin all get hacked and their databases are broadcast out to the Internet or sold to the highest bidder.  Though that would just take time especially with quantum computing and faster processors coming out trying to break existing public key encryption standards.

So why do we care about the long tail?

The long tail provides us with primarily untapped markets.  It saves us from having to compete against the big guys and get away with less risk.

Also your ideas to capture these long tail markets would not have to be as innovative because all you are trying to do is satisfy the need, rather than blowing off the competition.

Anyway hopefully this gives you readers ideas on what to do, I am too lazy to actually build these things but I would really like them to exist. :)

Disabling autocomplete on every single field in Curam

Like disabling the cache, disabling autocomplete is one of the other things that may be requested for Curam applications.  Theoretically this is simple if you control everything in a web application as you only need to put in autocomplete="off" in every form.  However, you can't really do that easily in Curam as you don't control the output JSPs nor do they provide you with a facility to do it.

Modifying gen-jsp.xsl

The first inclination would be to modify the generators just like I did when I disabled the cache. Unfortunately, Curam has its own form tag which does not accept the autocomplete attribute. So changing gen-jsp.xsl will not work.

Curam Implementation

Instead of fixing the output from the generators, we can use JavaScript that gets executed to update all the forms to disable autocomplete.  Of course to do that you need to know how to handle the differences between browsers for the onload and the setAttribute functionality.

However, we can get around it by using the standard Curam extension mechanism.  If you look at the generated code, then you will notice that there are a few JavaScript files that are always loaded, one of the relatively stable ones that have not really changed is omega3-util.js so we can theoretically use that.

First we make a copy of CuramCDEJ\lib\curam\web\jscript\omega3-util.js to components\custom\WebContent\CDEJ\jscript.  That will allow us to separate our customizations from the core files.

Now we write the following script at the end of the file.

function disableautocomplete() {

  var forms = document.getElementsByTagName('form')

  for (var i = 0; i < forms.length; ++i) {

    if (forms[i].setAttribute) {

      forms[i].setAttribute('autocomplete', 'off')

    } else {

      forms[i].autocomplete = 'off'

    }

  }

}

if (window.addEventListener) {

  window.addEventListener('load', disableautocomplete, true)

} else if (window.attachEvent) {

  window.attachEvent('onload', disableautocomplete)

} else {

  window.onload = disableautocomplete

}

Customizing gen-page-footer.xsl does not work

Although this change is be done using JavaScript and the scripts can be placed anywhere including gen-page-footer.xsl, this will not be sufficient.  The gen-page-footer.xsl file is only used by the standard Curam pages, but pop-ups do not use this file so if the pop-up has a form, the data would be saved.

Disabling cache in Curam

This is better than disabling the back button, even if it is one of the 10 ten web design mistakes.  However, for Curam applications and most web applications, this is something that would really need to be done in order to ensure that the user gets the most current data when they look at a page and to satisfy the requirement of nothing persisted on the client machine.

Technical overview

There are two things that need to be done.  The first is to add the following lines to the HTTP header:

Cache-Control: no-store

Pragma: no-cache

Expires: 0

And the following in the <head> block of your web page.

<meta http-equiv="Expires" content="0" />

<meta http-equiv="Cache-Control" content="no-cache" />

<meta http-equiv="Pragma" content="no-cache" />

This will make most web browsers not cache the page.

The second thing you need to do is to work around the Internet Explorer bug which disregards the no-cache header information if your page is larger than 64k.  To do that add

<head>

<meta http-equiv="Pragma" content="NO-CACHE" />

</head>

Near the end of your file.

Curam implementation

Unlike the process of disabling the back button, you cannot simply make the changes on gen-page-header.xsl or gen-page-footer.xsl.  The changes need to be directly done on CuramCDEJ\lib\curam\xml\xsl\gen-jsp.xsl. 

Note: If there are any upgrades to Curam, the changes need to be redone on the file.

In gen-page-header.xsl, add the following XML after the first instance of <head> in the jsp.xsl

<jsp:scriptlet>

response.setHeader("Cache-Control", "no-store");

response.setHeader("Pragma", "no-cache");

response.setHeader("Expires", "0");

</jsp:scriptlet>

<meta http-equiv="Expires" content="0" />

<meta http-equiv="Cache-Control" content="no-cache" />

<meta http-equiv="Pragma" content="no-cache" />

You also need to add the following XML after the scriptlet that outputs </body>:

<head>

<meta http-equiv="Expires" content="0" />

<meta http-equiv="Cache-Control" content="no-cache" />

<meta http-equiv="Pragma" content="no-cache" />

</head>

Browser compatibility

This has only been tested on Firefox and Internet Explorer 6, but it should work with most browsers.

Disabling the Back button in Curam

First let me say this. DISABLING THE BACK BUTTON IS A NOT A GOOD IDEA. Especially for web applications as this is a button that helps people navigate through your web site quickly. However, if you really have to remove the functionality. You can use the simple approach of adding the following JavaScript in every single page:

history.forward()

In current versions of Curam, they provide a facility for us to do this without actually modifying the CDEJ directly. First, make a copy of CuramCDEJ\lib\curam\xml\xsl\gen-page-header.xsl and put it somewhere in your components\custom folder. Then add:

<script type="text/javascript"><xsl:comment>
history.forward();
//</xsl:comment></script>

just before

<table class="page-header" summary="">

Then do a clean build.

But just let me iterate one more time... DISABLING THE BACK BUTTON IS A NOT A GOOD IDEA.