Beyond 2.0

A few of my colleagues at work have been twittering about doing something post Web 2.0.  So it kind of got me thinking what would be next (aside from the obvious Web 3.0 + Semantic Web).  Then I remembered about the talk of the long tail.

My guess is the Web 3.0 + Semantic web stuff as time passes would end up in the left part where the dominating market is.  In that domain, you would also have all the social networking, on-line applications, tagging and other on-line applications.  These used to be in the long tail, but has rapidly been shifting to be the dominant market.

What's in the long tail?

So what goes on the long tail now? One may think that the traditional off-line tools and products are going to the long tail, but that's not necessarily the case.  It is true that they are losing the market share, but they are still pretty dominant.

So what does that leave us?  Do we just give up and go fishing, go on a cruise, or go flying off for a vacation?  Well if we do chances are we'd be disconnected from the world (especially when some of those places do not have cell phone or Wi-Fi).

It is nice to be disconnected once in a while to recollect your thoughts, but it would also be nice to share those thoughts back to the world once we come back from our hiatus.

That's what I think the new long tail would be.

The "Partially On-line" application

In our world today there are many applications out there on the web that help us connect with others or share our knowledge.  However, they all require us to be connected which is not guaranteed.

Some common business scenarios are:

• Workplace inspections on oil rigs.
• Child safety intake in the boonies.
• Deep sea floor analysis.
• Satellite inspections.
• Decentralized version control.

All of those scenarios require someone to capture or process information but not have any access to the Internet or even cell phone towers.

What is being done

This problem is nothing new as there have been much work on these already.  Here are some examples of what is out there and my comments about them.

Technology	Comments
Lotus Notes/Domino	This is the product that leads the pack in terms of adoption within many businesses.  However, it is ridiculously heavy on resources (especially on version 8).  It is also forgetting its roots and requiring more connectivity (it spits out many dialogs and messages when you don't have connection to the servers).
Google Gears	This is gaining traction now as a de-facto implementation for partially on-line applications.  However, it requires a lot of re-work and the off-line application does not work the same way as an on-line application. It is also not based on any standards unlike...
HTML5	This is the standard way of storing database data within the browser.  However, WebKit is the only popular toolkit that uses this so far. Also it has the same problem as Google Gears where the off-line application is different from the on-line application.
GIT	One of the most popular decentralized version control systems out there. Unfortunately, it is only dealing with files.  Not necessarily an application. However, using this as the basis of application data storage could be the future.
Synchronization Technologies	There are many out there but the concept is to synchronize the data between various devices. But these are just data not applications.

What else can be done

As I specified above there have been some work already.  However, for each of them I had comments on because they can still be improved.

Ideally what I'd like to see are:

• A small application server that runs on the local machine.
• The server can be used as a proxy so if we request a URL and it is one of those disconnected apps it will show correctly on our browser (or other UI).
• It has a data synchronization thread constantly running in the background.

The data store would have the following properties.

• A document based data store like Lotus Notes, this provides the simplest method of managing data.
• Field level comparison support for each document.  That way developers can provide an easy interface for users to compare conflicts.
• XQuery or SQL support on querying the data.
• Document Lock support
• Automatic merge/updates like most version control systems.
• Local version history (so you can go back in time with changes that occurred locally).
• Archiving support (allows you to push data to a separate store for things that you don't use often).
• Partial document support (allows you to break the document into smaller parts and prioritize which fields to download first).
• The URL of the application is part of the namespace and is used in conjunction with the GUIDs to ensure uniqueness across applications.

Deployment would have the following properties.

• A simple EAR or other deployment archive can get downloaded an installed onto the local application server.
• The deployment archive would have a descriptor that specifies the URL to use and such.
• For security reasons, the URL in the descriptor must match the URL that it the application was downloaded from.

More stuff

My ideas above will provide disconnected support for the application and the data.  However, if we look further down the long tail again, we see one other use case that I have bypassed so far.  The feature with the dominant market and with the solutions I have outlined above has one slightly nasty limitation.  A single point of failure known as the server.

So what do we do about it?

Well I am thinking that we would take GIT's approach of creating a "web of trust" so we only get data from people that we trust.  The difference would be with GIT it is just code.  What if we pushed it so it has data and the application?

Then we'd have a social network based on local trusts rather than something that is run on a server (which can be hacked or sold off).

It is not that much further off from what I have written above.  The main difference would be the deployment, which instead of a URL would use some sort of secure certificate.

Unfortunately, the intellect needed to use these applications are not a majority in this current generation of people.  We have not hit a bubble where facebook, friendster, twitter, google or linkedin all get hacked and their databases are broadcast out to the Internet or sold to the highest bidder.  Though that would just take time especially with quantum computing and faster processors coming out trying to break existing public key encryption standards.

So why do we care about the long tail?

The long tail provides us with primarily untapped markets.  It saves us from having to compete against the big guys and get away with less risk.

Also your ideas to capture these long tail markets would not have to be as innovative because all you are trying to do is satisfy the need, rather than blowing off the competition.

Anyway hopefully this gives you readers ideas on what to do, I am too lazy to actually build these things but I would really like them to exist. :)

Integrated Google

After playing around a bit with some of Google's services it would be nice if things are more integrated rather than looking like a hodge podge of different websites. Single Sign-on and Profile I think number one thing they need to do is to get single sign on (SSO) working. SSO is different from just having a common user ID and password like Google accounts, though that is a requirement. Google should just have one sign in page that would allow you to go to any of Google's services without having to log in again. Right now Orkut requires a separate sign-in in order to access it. Along with the single sign-on Google should get rid of each application having their own profile and unify it into one single place. Although provide the facility to use different profiles with whatever service. Somewhat like Yahoo, but hopefully done properly. Drive space Next is the unification of all our space. I think this only applies to PicasaWeb, Blogger, YouTube/GoogleVideo, Google Documents and GMail. But I think it would be a good idea to just merge the space you would normally get with GMail into one pool rather than having a separate area for it. All our pictures including profile ones should be moved to PicasaWeb's space as well. Contact management Another one would be contact management. Right now we have GMail's address book, which by far is the simplest of any service I have used, but it does not provide good ways of managing your contacts. I think they should leverage Orkut and move all the contact information into Orkut (with privacy settings of course, you don't want to give out information about a friend that they don't want people to know about). We should also allow other people to see what we write about them in the contact info (in case we want to make sure we have everything up to date) within limits we set. How would they do it? Of course this sort of endeavor would be a lot of work. If I were in their shoes I would try to leverage a lot of the projects and specifications that are out there now. For example, single sign-on could be done through JEE's architecture already. The authentication token can be passed down to existing apps without having to recode them in Java. The Google Account can then be converted to use the OpenID specification as well. All our profile information can be stored in LDAP servers. Profile pictures would be rendered pointers to PicasaWeb's area so it would be more efficient on the LDAP servers. If all the space is pooled into GMail, then they only have to manage one data store rather than multiple. And since it is all pooled, we may end up with 4Gmail rather than the current 2G.

What I would look for in a requirements management tool

While looking through my RSS feeds I caught a glimpse of JRequisite. From the web site, it looks like a very far cry from any other requirements management tool you may have used at work (aside from Excel, Word and e-mails that is). That being said, it is still a good start. I wouldn't use it just yet, but it did get me to thinking what I would want in a requirements management tool -- for Eclipse that is. Here's a quick run through of the requirements I would have for my requirements management tool:

• Integration with the Eclipse Team API for versioning the requirement documents.
• There are requirement document types:
• General -- which is the catch all and contains an ID field, a title field, a tags field and one big entry field plus a list of attachments.
• Roles and Responsibilities -- A list of possible roles in the project and the task they are required to perform.
• Glossary -- an alphabetical list of business terms used to provide a common language.
• Each requirement is in its own text file and a zip file containing any attachments that the requirement may have.
• Requirements are linked together by levels of refinement. So if I modify a parent of a requirement the other requirements are tagged as needing review.
• Modifications are just tracked by the date/time stamp of the requirements file set.
• The general requirement editor is Wiki based so non-developers can have an easier time hyperlinking requirements to each other.
• Copying and pasting a picture in the editor field should paste a link to the picture. The picture is saved in the attachments zip.
• Holding down Ctrl and hovering a word that is in the glossary will show a hyperlink that links to the glossary entry.
• There is only one Glossary, but each entry is its own file.
• Modifications to glossary entries will trigger a scan (or can be done manually or skipped altogether) on any document that has a word that is written in the modified glossary entry and flag them as needed for review.
• Hovering over a word that is in the glossary shows a tooltip containing the definition.
• History of the requirement is tracked through the Eclipse Team API.
• A way to export the requirement tree to a PDF or HTML or as an Eclipse Help Plugin for external reviewers.
• For performance a local database (may be derby or hsqldb) is used to hold all "index" data that needs to be looked up. Any changes to any of the Requirements documents trigger an update to the database. This is easier than having to create a custom database backend to do this for us. And we can use standard SQL queries to the database and create new ones if needed.
• No access control as that should be handled by the Team API implementation.
• To keep things simple, requirements is a project nature and requirements documents are not usable outside of a requirements project.
• The editor form would have 3 tabs: Edit, Preview, Change History
• Preview should be a rendered HTML containing images and links as necessary. Links will open up other requirement documents in a new editor view.
• The Preview embeds the change history at the bottom as part of the document.
• Cycles are not allowed between requirement documents.
• Saves should be quick, scans may be triggered, but are run in the background using the Eclipse Job API.
• An ID clean up function to help reorganize the documents. Basically set the ID to a defined format e.g. [primary tag] [number] == UCS 1.1
• IDs normally do not get changed though so a warning is shown.
• Open source

Of course this requirements list is going to be vastly different from what JRequisite will be doing anyway, since they are after non textual based information. It would be interesting to see.