Showing posts with label java. Show all posts
Showing posts with label java. Show all posts

Tuesday, March 04, 2008

Reading from a bunch of strings

This extends a java.io.Reader which will be reading from a bunch of strings.  The purpose of this is to give the basis of a reader that would read chunks of data which are already be a Strings, but not allocate the whole string information in memory.  For example, reading XML from VARCHARs that can span more than one record.

Test case

The test case shows a base cases and the general case.

public class BunchOfStringsReaderTest {
    @Test
    public void readFromBunchOfStrings() throws Exception {
        final Reader p = new BunchOfStringsReader("FooBar", "mew mew",
                "pikachu");
        final char[] buf = new char[1000];
        int c;
        int i = 0;
        do {
            c = p.read(buf, i, 10);
            i += c;
        } while (c >= 0);
        final String s = new String(buf, 0, i + 1);
        Assert.assertEquals("FooBarmew mewpikachu", s);
 
    }
 
    @Test
    public void readFromNothing() throws Exception {
        final Reader p = new BunchOfStringsReader();
        final char[] buf = new char[20];
        final int c = p.read(buf, 0, 1);
        Assert.assertEquals(c, -1);
    }
 
    @Test
    public void readFromOneString() throws Exception {
        final Reader p = new BunchOfStringsReader("FooBar");
        final char[] buf = new char[20];
        final int c = p.read(buf, 0, 1);
        Assert.assertEquals(c, 1);
        Assert.assertEquals('F', buf[0]);
 
    }
 
}

The actual code

The following shows the code to be implemented to satisfy the test case.

public class BunchOfStringsReader extends Reader {

    private StringReader currentReader = null;
    private int i = 0;
    private final String[] strings;

    public BunchOfStringsReader(final String... strings) {
        this.strings = strings;
        if (strings.length > 0) {
            currentReader = new StringReader(strings[0]);
        }
    }

    @Override
    public void close() throws IOException {

    }

    public StringReader getNextString() {
        return new StringReader(strings[++i]);
    }

    private boolean hasNextString() {
        return i + 1 < strings.length;
    }

    @Override
    public boolean markSupported() {
        return false;
    }

    @Override
    public int read(final char[] cbuf, final int off, final int len)
            throws IOException {
        if (currentReader == null) {
            return -1;
        }
        int c = currentReader.read(cbuf, off, len);
        if (c == -1 && hasNextString()) {
            currentReader = getNextString();
            c = currentReader.read(cbuf, off, len);
        }
        return c;
    }
}

Of course this is just a sample of what can be done.  A more realistic implementation such as something that will get the data from a database record would do the following:

  • modify the constructor to get a database connection/rowset/etc. passed onto it.
  • modify hasNextString() and getNextString() to get the check if there is a next record and get the data from the database.
  • modify close() to close the database connection/rowset/etc.

Further work

The performance can be further improved by not relying on the StringReader and managing it via code.  However, the resulting code would be more complicated, so it is left as an exercise for the reader.

Tuesday, December 11, 2007

Programming vs Scripting

http://www.perl.com/pub/a/2007/12/06/soto-11.html

Normally I would agree with Larry's comments, but after working a few years in both scripting (Perl, Ruby) and programming (Java, C, C++) in many different language variations.  I think if I had to choose between one or the other, it would be it depends.

If I had a choice, it basically boils down to who is going to code and maintain it.  If the answer is anyone but me, myself and I, then I would go for a programming language rather than a scripting language.

Easy Onramps

Though I have to admit, the amount of effort to learn Perl myself is quite shallow, I learned to do most of the things I need to do within the first week of using Perl just because I code-build-run cycle is just one command, perl.

When I have to deal with other people, they would already know how to do the javac-java or at least use the Run As Java Application command on their IDE.  Without those, I don't think we'd even get them into our development teams.

Java/C# programmers are generally a dime a dozen.  Though you do get what you pay for.

So providing an easy onramp is good and all, but it is not a reason for me to get other people who are already trained as Java programmers to switch over.

TIMTOWTDI

"There is always more than one way to do it," is a very useful feature when I am coding for myself.  Not just coding, but I think well developed UIs should also follow that approach which I alluded to when I talked about Input Context Efficiency.

Having the option to do things differently reduces the amount of time I have to think about to get something working and focus more on getting what I need done.

When you are working in a large enough team, that freedom can backfire onto you.  Unless you are significantly diligent in your code reviews, TIMTOWDI can create a maintenance nightmare because everyone codes things their own way.

In Java, we have one conditional keyword: if.  In Perl, we have if and unless.

In Java, we have one way of writing "and": &&.  In Perl, we have && and and.

In Java, we have only one way of structuring our code: package containing classes containing methods.  In Perl, we can have an entire method in one file, or multiple classes of different names in the same file, multiple modules in a single file.

In Java, we have to explicitly define variables that we use.  In Perl, we have $_.

Granted that most of these problems can be handled automatically by some code review tools that are available in Perl and Java, I haven't found one for Perl that would integrate with the IDE as Checkclipse does.

early binding / late binding

Most scripting languages have a concept of late binding.  Which means that unlike Java, classes are not "final" and you can add new methods dynamically.

Although if I write code I may need to use that feature.  I would delay finding out about any errors until run-time.

If it were just me, then I would be creating some unit tests that would prove that the methods I call do work correctly.  Or sometimes since I would write something one off, I just let it fail at run-time when I run a test because it is easy enough to find the problem.

However, keeping track of 1000 methods in your head can be quite daunting too.  Not to mention typing it correctly.  Copy and paste may be safer, but it forces me to switch to my mouse to point to what I need to copy and paste.

Auto-completes do help out, but if one just thinks he typed something in correctly, and it fails at runtime.  Then the repercussions could be quite bad if it had been deployed to a production environment where change management processes are enforced.

I'd rather have something fail as early as possible rather than later.  Unit testing does help out with this one, but finding those kinds of developers are more difficult.  Especially if you have to deal with global resourcing.

Conclusion

As I have stated before, it basically boils down to is it for myself or for others.

If I have to write for others, I would use a programming language that would limit what I can do, and have almost everything defined including how to indent and casing.  Because that is easier for me than explaining how I do something in a language most people won't understand.

If I have to write for myself, I'd use whatever tool makes it easier for me.  Which in 90% of the time is Perl.

Thursday, October 25, 2007

util-2.0.ear released

One of my other open source projects that I am not diligently working on called twiff contains small project called util.  Unfortunately, I haven't been able to update the documentation, but I have made some a number of changes to it structurally.  Some of the major changes are

  • Switched to Maven 2
  • Added a getfile.jsp to retrieve any file and browse through the file system
  • Added multi-line text field for db.jsp

This app is primarily for application developers who may require a backdoor to their application during development and testing.  Aside from databases and files, the users of this can check JMS, JNDI and the environment/session data.  Most of the code is written in single file JSPs (normally a bad coding practice) so it can be extracted and placed in any web application to get more data (especially for env.jsp which shows session information).

You can go to the File Release site to get the files.

Saturday, September 22, 2007

Application and Reference Data Upgrades

This is a rethink of my previous data retention strategies with some application deployment considerations thrown in.

This approach should be ok with most DBAs as using DMXes to retain data may be too foreign a concept for most managed operations.  The deployment will not be automated using tools and procedures that are not developed by the deployment team so they can reuse what they have been doing in other applications and not have something different for your application.

The approach should work on any application, but Curam applications would benefit from the approach anyway.

Objectives

The following are the objectives I am trying to achieve in my approach.

  • Data is retained from one build to the next.
  • Support changes in "Reference Data" loads.
  • Application Deployment should be performed using standard methods and not some specialized batch file file not coming from the deployment team.
  • Infinitely repeatable in terms of number of environment supported.

Requirements

  • You require at least three deployment environments to test on.
  • You have a tool that would compare two different databases for both structure and data.

Environments

Environment 0 - This environment is where your developers perform their Integration Test.  This is the first shared environment that the application would get deployed to.  Data does not get retained in this environment.  Automatic application deployment and database resets are performed only to this level.  In Curam speak this is the only place where DMX files are used.

Environment 1 - This environment is where builds that are defined as stable and you have your test team performing their tests on.  Data is retained in this environment.

Environment 2 - This can be your final environment or your user acceptance environment.  Obviously data better be retained in this environment.  Applications and database changes fully tested on Environment 1 are the only ones moved over to this environment.

Environment n - As this approach is infinitely repeatable, we can take as many environments each taking application and database changes from Environment n - 1.

Approach

The first thing you need to perform is set a base line.  This tasks can be performed when at least two environments are put in place.  Before deploying the code base from environment 0 to environment 1 for the first time, the environment 0 data is reset to only contain the reference data.  The DBA then makes a backup or snapshot of the database state at that point - we call the backup Database Snapshot Build 0.  The deployer also takes the EAR of the current application to prepare it for Environment 1 - we call the EAR file EAR Build 0.

Environment 1 then gets it's database restored from Database Snapshot Build 0.  It then has EAR Build 0 deployed to it.

Now we have our base line ready for the next build.

So let's say the next build, Build z (where z > 0), is ready to be tested and deployed by the testers in Environment 1.  The following things happen:

  1. Developers smoke test Build z on environment 0.
  2. The database is reset to only contain reference data in environment 0.
  3. DBA takes a Database Snapshot Build z.
  4. DBA uses his database difference tool to determine what has changed between Database Snapshot Build z and Database Snapshot Build z-1.  The tool would generate the proper SQL change scripts.
  5. DBA restores the database in environment 0 to Database Snapshot Build z-1.
  6. DBA applies the SQL change scripts against the database in environment 0.
  7. The DBA verifies that the table changes relatively match Database Snapshot z.  I say relatively as there may be slight differences especially since the SQL change scripts may only alter the table and not be able to drop columns.
  8. The DBA then makes a backup of the database of environment 1.
  9. The DBA applies their SQL change script against the database.  If it works out, great, if not the SQL change script may need to be hand extended to support further data changes.
  10. EAR Build z is deployed on Environment 1.
  11. A smoke test on environment 1 is performed.  If there is data corruption, the database is reverted to its backup and the SQL change script may need to be hand extended to support further data changes.
  12. At this point Environment 1 has been updated and its data has been retained.

Now let's say we want to deploy Build z (where z > 0)the to the final environment, let's call it environment n (where n > 1) as it can be repeated as many times as possible.

  1. In Environment n we perform a database backup.
  2. In Environment n we apply the database change script from Environment n-1.  If it works out, great, if not the SQL change script may need to be hand extended to support further data changes.  Most likely reason would be Primary Key IDs for reference data may be clashing in which case we need to remap the clashing IDs to other IDs while maintaining referential integrity.
  3. EAR Build z is deployed on Environment n.
  4. A smoke test on environment n is performed.  If there is data corruption, the database is reverted to its backup and the SQL change script may need to be hand extended to support further data changes.  Most likely the scripts need to be modified to perform data cleanup tasks.
  5. At this point Environment n has been updated to build z and its data has been retained.

Conclusion

I think I have covered the objectives I have set out earlier.  With the exception of Environment 0 which is the environment were we bootstrap all the changes.

  • Any data entered is retained from one build to the next due to the fact we only apply patch changes to the database rather than resetting the whole thing.
  • The "Reference Data" is loaded up properly as the patch changes contain the reference data changes as well.
  • Application Deployment is performed using standard deployment procedures, there are no specialized scripts required.
  • There is no technical limit as to the number of environment levels that you can deploy as the deployment logic is recursive.

The toughest part may be locating a tool that would perform the database diffs that your DBA team would be comfortable with.  It is best to let the DBA choose the tool and they can get pretty pricey, but it will all be worth it come your second release to production.

Tuesday, April 03, 2007

LDAP Logon and Password Management (part 2)

This is in continuation of my previous post. The PostLogonFilter Well today I am starting up with the filters. So I added the necessary entries to my web.xml file now. As I was writing it, I think I would be creating a separate java only project soon so I can keep the web app as customizable as possible. But for now all will be in the same project. So the PostLogonFilter would be mapped against /j_security_check and it will be invoked on a successful run of the validation. I was thinking of putting in a JUnit test to validate this, but I don't really know how to verify j_security_check on HttpUnit. Oh well, the app is very simple I think it would be way too overkill to automate the unit test on this one. Ugh, I feel dirty just saying that. Of course since you are going to be using servlet API, you really should include the javax.servlet servlet-api dependency. Grr, they should add that to the archetype as well. When I finish with this I am going to be writing a patch for that webapp archetype. Apparently I am using version 2.4 of the servlet API not the one I originally intended. I love maven, just one switch on the POM and poof we're on version 2.4 now. For now I put in a System.out in the filter, but I didn't see it on the console. I think it is because the jetty just eats it up. So I registered commons-logging and put in the message using commons logging. But that still didn't work. Found out the reason. j_security_check filters only work on WebSphere. Fortunately for me that is my target platform. Unfortunately I have no way of testing without deploying to that platform, I'd rather use Jetty because of speed. Anyway I used the Listener instead. Yes it is not ideal because it will get invoked for every request, but since the application is small and simple it shouldn't be too too bad.

public void requestDestroyed(final ServletRequestEvent sre) {
final HttpServletRequest httpServletRequest = (HttpServletRequest) sre
  .getServletRequest();
String mapping = httpServletRequest.getRequestURI().substring(
  httpServletRequest.getContextPath().length());
if ("/j_security_check".equals(mapping)
  && httpServletRequest.getUserPrincipal() != null) {
    System.err.println("yo " +
      httpServletRequest.getUserPrincipal().getName());
}
}
Is what I put in my code. Seems to work, getting the mapping is going to be a bit slow, but I'll work on optimizing it later. I will be changing the "yo" message with the expiration checks in a bit. However, all of the above does not work well. So how do I do it now? Well I made a Servlet that does the login instead, but forwards to j_security_check. The LogonServlet The logon servlet works suprisingly well. Of course it means that you have to use a non-standard servlet rather than j_security_check, but so what? Really it does not matter since this is a separate application and we can customize it as much as we want. So here is how it flows: Checking for expiration of passwords I took some concepts from a devWorks article on how to do this and for the most part it is fine, except that the netscape packages are not in an obvious place. However, I did find the Netscape LDAP SDK eventually and things just worked out for the best from that point. One gotcha that you have to note is when you reset the password, it will not expire in Tivoli, but rather it will return an error when you try to login. So rather than calling the method "check if password expired", I basically called it "checkPasswordChangeRequired" Further work Once I got the main flows all worked out, I just put struts around it to make the UI code more cleaner rather than having a lot of spaghetti JSP and servlets with embedded Java code. It also supports i18n when we have to port it to French. However, I am not going to cover that topic.

Monday, April 02, 2007

LDAP Logon and Password Management

I am just about to start off a new subproject which does logon and password management for LDAP. These are just my notes. Since this is for work, I can't post the full source code in the open, sorry. However, most of my readers should be competent enough to build it if they have to. Some background I don't like to create any unnecessary work for myself. Early on, I realized that when you log into one application in the application server, you are logged onto every application on the application server. This is with WebSphere and Tomcat. So I find it annoying when I go into a new project and I (or someone else) has to rebuild the logon use case with the change and manage password. Unfortunately, the proper enterprise solution tends to get expensive and is a separate project on its own. The proper enterprise solution would be an Identity Management solution such as Tivoli Identity Manager. Because of time constraints, I am going to do the YAGNI approach so everything would be in one uber project. I'll split it off later. My other goal is to make what I build into an archetype so we can use it with other projects as an asset. Some notes to get me started I have to hold the LDAP contexts so I do not use up any grace logins if I have to change the password on expiration. I do not intend to put in any detailed error messages aside from "you got an error". There is a servlet filter that after j_security_check it does the expiration checks and forwards to the right pages. Starting things off To start things off I am going to use Maven's webapp archetype: mvn archetype:create -DarchetypeArtifactId=maven-archetype-webapp -DgroupId=org.xxx -DartifactId=XxxLogon Getting into the IDE And since I am an Eclipse user, I just typed the following in the directory mvn eclipse:eclipse I chose this approach over using the Eclipse plug-in, because I find that the maven command line plugin gets updated more regularly than that of the Eclipse plug-in. Unfortunately, the archtetype does not support WTP. This isn't much of a surprise for me since I had issues with Maven and WTP a year before. It is disappointing that these were not integrated with the archetype. Anyway I just added the plugin information and it seems to register as a webapp when I imported it into Eclipse. I then put in the maven jetty plug-in and test it out by going to http://localhost:8080/XxxLogon and "Hello world" comes out successfully. Nice I recommend this approach rather than using the servers that run on WTP, because (after experimenting) JSP changes get applied automatically, no need for a seperate publish step. Security Now I have to make it so it forces me to do a logon. However, when I put in the security-constraints (except for transport-guarantee which I know should not work because I didn't set up SSL) it gives me the error "No realm". It turns out I need to set up a realm, but I couldn't find any Realms for Jetty that uses LDAP so for now I used the HashUserRealm. I did this by adding the userRealms blob as shown in Maven2 Jetty Plugin: Configuration and created my realms.properties file that looks like trajano: passw0rd,everyone Once I did that I can log in properly. SSO test I decided to deploy to WebSphere Application Serer rather than Jetty since I haven't found out how to do multiple app deployments using the Jetty plugin. No biggie. I need to deploy it there in the end anyway. Ran the XxxLogon app entered my credentials then went to the URL of the real app. Works like a charm! Since I have eliminated the integration risk, I don't have to deal with that for a while now. Ya gotta like RUP's elaboration phase. What's next? In the next few days I am going to do the expiration handling and change of password.

Sunday, February 25, 2007

Why Trails is better than Rails

I haven't used much of the new "Rails"type development kits yet. However, if I have to choose amongst them, I would be choosing Trails. The main problem with Grails (and Rails) is you are adding a new language to the mix. If all you have are highly skilled developers and not planning to do Global Resourcing then that's all cool. However, if you are building an application with a lot of use cases, its best to rely on as little changes to the development environment as possible. It is better to move your top developers off to new customers rather than repeating drudgery with each use case (you get cheaper GR resources for that). The problems with Ruby on Rails ... I've seen presentations and demonstrations of Rails and for the most part it is a great sell for those who are keen enough to learn new tools. Having a smaller LoC metric is good and all, but just because you can reduce something into a single line does not imply that it is generally readable.
I've done many shell and Perl one liners when I need to get things done but it does not mean I would really want to subject anyone to what I've written.
Although I would be sold and would probably use Rails if I needed to get something out quickly and don't expect people to manage what I write. That hasn't happened for a while, most of the applications I tend to build is part of a large team and not a solo flight. It may change sooner than later because they have relatively good Ruby IDE support now. What critical things haven't I seen with RoR (and why they are important).
  • XA transaction support. I need to be a bit more specific with this one, as most people think that XA only involves databases. XA can support different resource types such as message queues and enterprise connectors. For the most part, in order to communicate with some legacy systems, a JMS type system may be needed. The message transport tends to be part of a larger transaction which may involve database activity as well.
  • Attribute oriented web page development (like Tapestry). The only way that I have seen HTML customization done in the demonstrations I've seen are through RHTML. RHTML is basically HTML with Ruby code (shades of model-1 JSPs and old ASP pages). RHTML, JSP, ASP or ASP.Net pages are usually not viewable by the browser. I alluded to when I talked about my ideal web project. Although I hear there is an add-on for this.
  • I18n support. Being in Canada we have two official languages English and French. So if you have to develop applications for the government or companies that deal with Quebec, you bet i18n support is important. Although I hear there is an add on for this too.
  • Single sign-on. This is more the JAAS thing. If you've been in the industry long enough, you would have heard "How many times must we do the logon use case?" quite often. Although I have an extenstion to that which is "How many times must we do the user management use case?" Personally having to do that again isn't my idea of fun, and I'd rather delegate that to a product like Tivoli Identity Manager which I just get someone to customize the screens and leave all the backend user management stuff to the product. If you have a large enough project, you would probably save more money buying the product rather than waiting for it to be reimplemented again for your application.
  • Database migration. From what I saw with Rails, updates to the database tables are done manually by the Ruby developer. This would be quite a chore if there is a large number of table updates that need to be done. Hibernate provides Java developers with "schema-update" which generates the SQL needed to migrate existing database tables as needed (though you should still get your DBA to review it).
So with the exception of Tapestry and Hibernate, the JEE environment already provides us with the necessary infrastructure out of the box. One key thing that Rails has over Java language solutions is the on the hot code change support. That may ween a lot of people over, myself included. But, I like to have as much of the infrastructure done for me when I do work and I would like to delegate work out to experts as needed. Much ado about Grails ... So far my experience with Grails has been limited to just some well done screencasts. However, most of it was quite a bore (not because of the presenter, he did a really good job), but because it looks the same just replace Rails with Grails and Ruby with Groovy. So as far as innovation goes it does not really help much. As I was perusing through the screencasts, I found some things that I didn't really like:
  • Batch file/ant based application generation. After using Maven for a while, I haven't found a better way of structuring your source files than they did. It may be overkill, but at least it works well (with the exception of web content). An older version of Trails did this as well, thankfully they outgrew that and moved to Maven.
  • Groovy Server Pages (GSP) It looks a lot like JSP files, which I would avoid because it cannot be sent back to the web development team once the Java guys put in all their tags and what not.
  • Groovy Yet another programming language.
  • Batch file to create new domain object. Rather than just creating a new class.
  • Ant based build targets. Rather than leveraging more standard targets we have a new set of targets to deal with. This could easily be fixed by moving the development to Maven.
  • I have to enter the domain class twice? Not sure why.
Like Rails, Grails support the on the hot code change support. Since Grails runs on top of a JEE stack, that may ween a lot more people over, myself included. However, it is not without its warts which I have listed above. So why Trails? Trails, although it is still in its infancy, is better geared for application development with a larger team and multiple roles (i.e., web developer, application developer) and can more easily leverage existing skills of Java developers. Here are the things that Trails has it going for:
  • Maven integration. This means dependency management, builds and project layouts are more consistent. You also get the standard site reports as you need without having to deal with a lot of low level Ant.
  • A smaller code base. Because it seems to delegate a lot of its work to other opensource packages like Tapestry and Spring rather than reinventing the wheel.
  • Tapestry. So we can utilize web developers with their Dreamweaver or whatever other tool more effectively rather than forcing them to use a JSP editor which may not offer them enough artistic flexibility.
  • Hibernate. So we get "schema-update"
  • JEE based. So we get i18n and XA out of the box rather than an afterthought.
However, ... Trails is far from perfect Trails is the only one I have worked with the longest among the three so I do have some issues with it.
  • Not too many users. I think a good indication of this is the activity on the Trails IRC channel. I am usually the only one there when I go on.
  • Their web site does not render correctly in Internet Explorer. I normally use Firefox, but there are still a number of people that are using Internet Explorer. Having their side navigation going all the way down to the bottom of the page does not look very professional.
  • No update on their website since September 7, 2006
  • They use Maven, but their website seems to be hand coded HTML. Rather than using something like a Mavenized report. Yes it may look more plain, but at least it is consistent. It took me a bit to figure out what the subversion URL is.
  • No hot code replace. Not really possible you need to regenerate and compile every time as well. This would make things go slower and slower as your application grows.
  • Seems to be only one maintainer right now. Since I started Trails again, which is just a few weeks ago, it seems that only one person is actually maintaining the framework now.
  • Still in SNAPSHOT. There is no 1.0 release of Trails yet. Right now all I've been working on is the SNAPSHOT version. Although the SNAPSHOT seems stable enough. They should make a roadmap they can say it is ready and people can plan to use it.
  • An updated presentation. Trails used to have a 12 minute presentation that shows a many to many relationship. Someone needs to update this presentation. Maybe I'll do that myself if I have to present Trails to my peers.

Sunday, April 16, 2006

AJAX Queue

I was thinking of doing a simple AJAX framework for myself using some concepts of GUI programming. Actually just one, the event queue. Please note I haven't done the implementation yet, this is just an idea at the moment. The advantange to performing this approach is that it allows a separation of event handling from the actual page. The development of the AJAX enabled page would be similar to that of a standard GUI application. Components:

  • one JMS queue that resides on the presentation tier.
  • one servlet to retrieve the queue data from the JMS queue and convert it to XML
  • one servlet to retrieve a secure random number.
  • a Javascript API to retrieve the queue data from the servlet and process the queue events.
  • a JavaScript API to send data to the Servlet tier
  • a Java API to put things into the JMS queue.
JMS Queue All the event information is stored in a JMS queue. A JMS queue is required in order to ensure clusterabilty. On my initial drafts I put the queue as part of the session. Once I started to type out some rough implementation, I realized it would not be thread safe, and putting in the synchronized block prevents it from being clusterable. So I ended up with using a JMS queue on the presentation tier.
For those that don't understand the concept of "on the presentation tier," the basic idea is you would have a separate set of resources (which might be cheaper alternatives) that is used only to support whatever presentation tasks you might have. This is to allow separation of business concerns from presentation concerns. For example, a presentation tier database stores things that are too big or cumbersome to let session management handle. In the same way a temp file is used to put information rather than store things in memory and let the virtual memory manager handle the data. The presentation tier resources tend to be cheaper and less persistent, i.e., on application restart, all the temporary runtime data stored in there are deleted.
In order to prevent queues from overflowing, each message in the queue should have an expiration set. The expiration should be no more than one minute depending on how long you expect clients to process the next event. The messages should also be mark as non-persisting, since we don't need them to stay if the queues happen to restart (since the application might be restarted as well anyway). QueueServlet The QueueServlet is an HttpServlet that retrieves event data from the JMS queue and transforms the data into an XML stream (possibly via JDOM or manual StringBuffers). This is the only place for the AJAX side to get responses from the server side. The servlet will implement the doPost() method and uses the following logic:
public void doPost(HttpServletRequest request, HttpServletResponse response) {
 // requestId is set by the RequestIdServlet
 String requestId = request.getHeader("requestId");
 String sessionId = request.getSession(false).getId();
 List queueData = new LinkedList(); // we don't do random access to the list

 do {
   // get the message from the JMS queue non blocking
   Message m =
      jmsQueue.getMessage("requestId = '" + requestId + "' and " +
                          "sessionId = '" + sessionId + "'");
   if (m == null) {
     break;
   }
   queueData.add(new AjaxEvent(m));
 }
 // Write the queue data as XML
 writeQueueDataXml(queueData,response);
}
This approach ensures that the largest possible chunk is sent for every request. This solves the issue of too many requests getting a small chunk of data. RequestIdServlet This servlet performs a SecureRandom call to get a random number to pass back to the browser. The purpose of the id is to allow multiple pages to the same website to be open at the same time. As the QueueServlet, it only supports the POST method. AjaxQueue JavaScript The AjaxQueue JavaScript API is a JS file provided by the framework. It is responsible for creating/managing the XmlHttpRequest object and dealing with infrastructure based tasks. The typical Ajax enabled page would look like this:
<script src="ajaxQueue.js" type="text/javascript"></script>
<script>
// define event handlers
function handleFooEvent(eventDomTree) {
 // do some DOM manipulation
}

// register events
registerEventHandler("Foo", handleFooEvent);
registerEventHandler("Bar", handleBarEvent);
registerErrorEventHandler(handleErrorEvent);

// start up the queue
startQueue("QueueHandlerUrl", "RequestIdUrl");
</script>
In earlier attempts of implementing this, I tried to use a more object oriented fashion. However, since the xmlHttpRequest.onreadystatechange function cannot have parameters passed into it, things just ended up in global variables. The global variables in ajaxQueue.js are:
  • xmlHttp - an XmlHttpRequest object built based on the browser
  • eventHandlers - an associative array that maps event name to a handler
  • errorEventHandler - a function reference that gets called when there is an error
  • requestId - a unique number representing the requests made from the current page
  • queueUrl - the URL to the QueueServlet
  • queueTimeout - the time to sleep before the next queue query
The xmlHttp.onreadystatechange function is implemented something like this.
xmlHttp.onreadystatechange = function () {
   if (xmlHttp.readyState == 4) {
       if (xmlHttp.status == 200) {
           var events = xmlHttp.responseXML.getElementsByTagName("event");
           for (var i = 0; i < events.length; ++i) {
              var eventName = events[i].name
             if (eventName == "history") {
                  // special event to create a virtual entry in
                  // the history to handle the back button
                  doHistoryEvent(events[i]);
              }
              eventHandler = eventHandlers[eventName];
              if (eventHandler) {
                  eventHandler(events[i]);
              }
           }
           window.setTimeout("processQueue()", queueTimeout);
       } else {
           errorEventHandler(xmlHttp.statusText);
       }
   }
}
The startQueue function would look like this:
function startQueue(url, requestIdUrl, timeout) {
   if (xmlHttp == null) {
       // don't do anything if AJAX is not supported.
       return;
   }
   requestId = getRequestId(requestIdUrl);
   queueUrl = url;
   queueTimeout = timeout
   // create invisible iframe to handle history events
   createHistoryIFrame();

   processQueue();
}
The processQueue() function that sends the request to the servlet would look like this:
function processQueue() {
   xmlHttp.open("POST", queueUrl, true);
   xmlHttp.setRequestHeader("requestId", requestId);
   xmlHttp.send(null);
}
One way to further reduce the load on the servers is to alter the queueTimeout with a more intelligent algorithm rather than a fixed amount. This would have to be application specific though. Sending messages to the server Requests to the server side always need to have the "requestId" so they know what to put into the queue. And there will be several pieces of information that is going to get passed into the backend for each request. So the following function is provided:
function sendMessage(url, messageData) {
   if (xmlHttp == null) {
       // ensure that nothing happens if AJAX is not supported.
       return;
   }
   // create a new xmlHttpRequest for sending data
   var sendXmlHttp = createXmlHttpRequest();
   sendXmlHttp.open("POST", url);
   sendXmlHttp.setRequestHeader("requestId", requestId);
   sendXmlHttp.send(messageData);
   // error handling goes here.
}
The requests for the message sends are synchronous rather than asynchronous. This ensures that errors are captured properly if there are any. Putting events into the queue To put a message into the queue, the following code gets invoked:
AjaxQueue queue = null;
try {
 queue = new AjaxQueue(servletContext, request);
 queue.putEvent(e);
} finally {
 if (queue != null) {
   queue.close();
 }
}
Of course that code block is too big and there will be a lot of copy and paste if we just left it like that. So a utility class called AjaxQueueUtil with a method
AjaxQueueUtil.putEvent(ServletContext context,
 HttpServletRequest request,
 Event events...)
that sends a list of events into the queue. This is provided to shield users from copying and pasting so many lines of code. Conclusion As specified earlier the advtange to performing this approach is that it allows a separation of event handling from the actual page. The development of the AJAX enabled page would be similar to that of a standard GUI application. If the browser does not support AJAX or even scripting, its okay since the pattern is non-invasive to HTML and developers can AJAXify things as needed. This pattern takes advantage of facilities within the J2EE spec and supports clustered environments and multiple windows on the same session. A large chunk of the logic is hidden within the API implementation to make things easier for the developers. However, there is no concrete implementation of this pattern yet.

Thursday, March 23, 2006

J2EE with Maven 2 + WTP

Today I decided to revisit WTP, since they came out with a new version 1.0.1, I thought I'd give it the good old college try again. This time blog the experience too. Along with this, I decided to take maven2 out for a spin again with J2EE projects. I will avoid trying to be mean like the BileBlog, because I don't think there is a need to. As expected, Maven 2 is lacking in documentation on how to set up a J2EE project with Eclipse. Granted they need to get it working first which they might've but I am not looking at the SVN version of their project, just the 2.0.2 that was released along with the plugins that come with it. If you are going to attempt something new with Maven 2, I suggest you have three things open:

  1. CGI:IRC client pointing to #maven
  2. Maven's JIRA
  3. Google
First off, the eclipse plugin that comes with Maven release doesn't work correctly so you should at least try to get the latest release version of the plugin by adding to your POM. It does not autoupdate either with the -up or -cpu command paramter of mvn as I noted in MECLIPSE-82:
<build>
 <plugins>
   <plugin>
     <groupid>org.apache.maven.plugins</groupid>
     <artifactid>maven-eclipse-plugin</artifactid>
     <version>2.1</version>
     <configuration>
       <workspace>${basedir}</workspace>
       <!--<downloadsources>true</downloadsources>-->
     </configuration>
   </plugin>
 </plugins>
 ...
</build>
This will ensure that you get the 2.1 release of the eclipse plugin. Now that's on my root project POM. I plan to create a multiproject Maven setup so one project for WAR, domain objects, EAR and maybe one for EJB later. So I create additional projects and POMs for the sub project, for now I only did a war, a jar, and an ear. No EJBs yet. In the WAR pom.xml make sure you add the following in the build section in order to create the proper Eclipse WTP meta data.
<plugin>
 <groupId>org.apache.maven.plugins</groupId>
 <artifactId>maven-eclipse-plugin</artifactId>
 <configuration>
   <wtpversion>1.0</wtpversion>
 </configuration>
</plugin>
I also created a simple JSP to test if everything works. Using Vincent's instructions with Cargo and a slight tweak that is to change dir to home. I also think ${basedir}/src/main/webapp can be changed to something else like ${maven.war.src.webapp} or something, but I haven't found it yet. Anyway you then just type in mvn cargo:start to get things going. Then go to the browser and go to http://localhost:8080/webapp/ivt.jsp. Cool so I can run things on Cargo environment with Jetty. I have to see how to do it with Geronimo later. Now, I do a mvn eclipse:eclipse on the root project (btw, it has the packing of pom. As expected it does not generate any .project files for the root project, but generates them for the subprojects. However, it does seem to not know how to handle ear packaged projects and treats it like a standard Java project. The others went correctly. I had trouble earlier with eclipse:add-maven-repo and I haven't really pushed forward with it much. For now I added the M2_REPO variable to my pristene Eclipse workspace built from wtp-all-in-one-sdk-R-1.0.1-200602171228-win32.zip. Anyway, I just imported the projects and everything went well for the most part (the EAR project is busted as it is treated like a Java project). However everything else is okay. Now using Geronimo devtools and the devworks article as a guideline to use this the Geronimo Eclipse plugin, I got everything installed. Quite nice. Just had to point it to a freshly downloaded Geronimo installation directory. Tried to start the server. Works beautifully and I can access the admin console. However.... I tried to install my WAR (since the EAR was not available) no dice when I tried to load it. And when I checked the console, it wasn't even there. So I tried Tomcat 5.5 which I also had installed. Everything works. However, since I want to build an EAR eventually this will not do. Then I thought what if I used the other Geronimo that uses tomcat rather than Jetty. Start up time was noticibly slower. However, no luck still with my webapp. So for the time being I think I am stuck with Tomcat. Or use RAD 6.0 which has an integrated test environment for WebSphere.

Friday, February 24, 2006

Commons-logging in WebSphere Application Server 6.0

WebSphere Application Server 6.o uses commons-logging and is loaded by the application server's class loader. Normally logging would configured on the server level through the administration console. Go to Troubleshooting, click Logging and Tracing > server name > Change Log Detail Levels and configure what needs to be logged there. However, there are times when the WebSphere logging is not enough and applications need something more powerful such as Log4j to do the logging. It is recommended that the class loaders are left to the default policies rather than switching to PARENT_LAST or using an APPLICATION rather than MODULE policy. This will ensure you are coding to spec and make it easier for you to switch application servers if needed (e.g. Geronimo on the development workstations and WebSphere Application Server for the production servers). It also prevents leaks when restarting applications as the class loader and all its data are unloaded cleanly. To configure Log4j to work on an application deployed in WebSphere 6.0, the application module must have the commons-logging.jar and log4j.jar file within its classpath. It is then a matter of invoking the LogFactory.setAttribute() method on initialization. For web modules, create an initialization servlet as follows:

public class InitServlet extends GenericServlet implements Servlet {
public void init() {
  LogFactory.getFactory().setAttribute(
      LogFactoryImpl.LOG_PROPERTY,
      Log4JLogger.class.getName());
  DOMConfigurator.configure(
      getClass().getClassLoader().getResource("log4jconfig.xml"));
}
public void service(ServletRequest request, ServletResponse response) {
}
public void destroy() {
  LogFactory.releaseAll();
}
}
Some notes:
  • The DOMConfigurator line specifies where to find the Log4J configuration to use.
  • The destroy() method is used to release all the Log objects that were created by the LogFactory within the current class loader in order to prevent leaks when restarting.
  • A log statement may be put into the service() method, to indicate successful configuration of the LogFactory.
For EJB modules, there is no such thing as an initialization EJB within the spec so it has to be done in the ejbCreate() and ejbRemove() methods. Since this can be repetitive, it is best that the EJBs extend a base class that sets this up. An example of an abstract EJB base class is as follows:
public abstract class BaseSessionBean implements SessionBean {
public void ejbCreate() throws CreateException {
   LogFactory.getFactory().setAttribute(
       LogFactoryImpl.LOG_PROPERTY,
       Log4JLogger.class.getName());
   DOMConfigurator.configure(
       getClass().getClassLoader().getResource("log4jconfig.xml"));
}
public void ejbRemove() {
   LogFactory.releaseAll();
}
}
This is similar to the InitServlet except that it uses the EJB constructs rather than that of Servlets. This base class has to be extended by all EJBs in the project to ensure that the LogFactory gets configured correctly. However, this is very intrusive to the application since it introduces the Fragile Base Class antipattern. To solve the antipattern, two things need to be done: create a custom LogFactory and let the module know to use the custom LogFactory. The custom log factory looks like this:
public class CustomLogFactory extends LogFactoryImpl {
 private static final String LOGGER = Log4JLogger.class.getName();
 protected String getLogClassName() {
     setAttribute(LOG_PROPERTY, LOGGER);
     return super.getLogClassName();
 }
 protected Constructor getLogConstructor() throws LogConfigurationException {
     setAttribute(LOG_PROPERTY, LOGGER);
     return super.getLogConstructor();
 }
}
Some notes:
  • The getLogClassName() and getLogConstructor() methods are called by LogFactory in order to create the actual log.
  • The setAttribute() method for the two methods in order to ensure that the Log wanted is the Log4J logger. Putting the setAttribute() in the constructor does not work.
The other thing to do is register the new LogFactory. To do this, create a file in the META-INF/services directory called org.apache.commons.logging.LogFactory and put in the fully qualified name of the custom LogFactory. This has to be done for every module, but at least it is away from the application class hierarchy and can even be kept in a separate jar. However, this will still leak since the releaseAll() method is not called and there is no way of implementing this without putting it as part of the class heirarchy.

Monday, August 08, 2005

EJB Pattern - Service Locator with Remote Method Invocation

To start, I'm quite sure there are several implementation of this pattern already. However, this is just one of three parts. The first part introduces an implementation of the Service Locator pattern that performs remote method invocation. The second part is short and puts optimizations into the implementation. The final part involves converting it into a PicoContainer ComponentAdapter.

Context

Services are implemented as Java objects that can be run and tested outside the context of a container, but using an EJB to provide separation of the service implementation from clients.

Problem

The use of services as Java Objects rather than EJBs allows for easier testability without using a container. The problem comes when using EJBs as a transport and ending up with several EJBs that only perform transport functionality as this will yield in duplicate code and a lot of configuration done through the EJB descriptors and manifests. Using the Service Locator pattern in conjunction with Dynamic Proxies would reduce the amount of EJBs needed to one.

Forces

  • Reduce the number of EJBs that perform the same function of transporting data from client to the service implementation.
  • Prevent cut and paste coding from the service interface to the EJB Remote interface (that require java.rmi.RemoteException thrown).
  • Remove the need of hand-coded proxies for every service.

Solution overview

Create an EJB that instantiates implementations of services and stores them in a map. The EJB will have an exposed method

Serializable
invoke(Class serviceClassInterface,
    String methodName,
    Serializable[] args)
that would invoke the method on the service class implementation instance. This single EJB would be used for all services.

Depending on the application server. we are no longer taking full advantage of the EJB Cache provided by the as there is only one entry. Also, the application server might not provide the capability to create a pool of stateless session beans.

You will also lose access to metrics based on EJB usage since it all goes to one. This is similar to the Struts framework where everything goes into one servlet.

A service locator to performs the EJB lookup functionality and retrieve an RMI client stub for the clients that can connect to the server.

Solution Step 1: Creating the service test case

In keeping with the Test First Development methodology, a test case is made that shows how we plan to use the service. Starting simple makes things easier to understand so the first test case shows how the service would be used without a container.

public void testUpperServiceNoContainer() {
  UpperService upperService = new UpperServiceImpl();
  assertEquals("ABCDE", upperService.upper("Abcde"));
}
The implementation class implements the method as follows:
public String upper(String s) {
  return s.toUpperCase();
}

Solution Step 2: Introducing the service locator

The service locator pattern is pretty simple, you send a request for a service and it will give you an object that you can use to manipulate the service. Unlike the Core J2EE pattern the implementation discussed here avoids the Singleton antipattern.

public void testServiceLocator() {
  ServiceLocator locator = new MapServiceLocator();
  UpperService upperService =
    (UpperService) locator.getService(UpperService.class);
  assertEquals("ABCDE", upperService.upper("Abcde"));
}

At this point the application would now fail to run since we do not have ServiceLocator implemented yet. The next few sections discuss how to build the ServiceLocator class.

Solution Step 3: Creating the Service Locator class

We start the service locator using a simple map that instantiates the implementation. This will make the test case pass.

public class MapServiceLocator implements ServiceLocator {
  private Map serviceMap;
  public MapServiceLocator() {
    serviceMap = new HashMap();
    serviceMap.put(UpperService.class, new UpperServiceImpl());
  }
  public Object getService(Class clazz) {
    return serviceMap.get(UpperService.class);
  }
}

Our next step would be to remove knowledge of UpperServiceImpl on the MapServiceLocator implementation.

Solution Step 4: Introducing the Invoker

Currently the ServiceLocator has knowledge of the actual implementation classes. Separating the knowledge of the implementation would allow packaging code so interfaces and transfer objects are the only ones provided to the client and the implementation can be put elsewhere. An approach will use Java's reflection API and use it as an indirect way of invoking a method. By using this approach it is possible to invoke a method based on information obtained at runtime rather at compile time. The following test case shows how an invoker method would be used.

public void testInvoke() {
Invoker invoker = new Invoker();
assertEquals("ABCDE",
invoker.invoke(UpperService.class,
 "upper", new Serializable[] { "Abcde" }));
}
As shown, the client has no concept of the actual implementation. Similar to that of the ServiceLocator. The implementation of Invoker is as follows:
public class Invoker {
  private Map map;
  public Invoker() {
    map = new HashMap();
    map.put(UpperService.class, new UpperServiceImpl());
  }
  public Serializable invoke(Class clazz, String mName, Serializable[] args) {
    Object impl = map.get(clazz);
    Class[] parameterTypes = new Class[params.length];
    for (int i = 0; i < args.length; ++i) {
     parameterTypes[i] = args[i].getClass();
    }
    Method method = clazz.getMethod(mName, parameterTypes);
    return (Serializable)method.invoke(impl, args);
  }
}

Implementing the method above would make the test case pass.

Solution Step 5: Changing the service method to use the invoker

The java.lang.reflect.Proxy class allows us to create a custom invocation handler that would intercept method calls and perform an action based on it. In our case, it should simply invoke the implementation method. The following test case demonstrates how clients would use it. Care has to be taken to ensure that only Serializable data gets transmitted otherwise if the service is running on a remote server it would cause problems. The following service locator has been converted to use the invoker. An invocation handler acts similar to the testInvoke method. The locator creates a proxy that uses the invocation handler which invokes the actual method.

public class InvokerServiceLocator implements ServiceLocator {
private Invoker invoker = new Invoker();
public Object getService(Class clazz) {
InvocationHandler handler = new InvocationHandler() {
 public Object invoke(Object proxy, Method method, Object[] args){
   Serializable[] serializableArgs = new Serializable[args.length];
   System.arraycopy(args, 0, serializableArgs, 0, args.length);
   return invoker.invoke(clazz, method.getName(), args);
 }
};
return Proxy.newProxyInstance(clazz.getClassLoader(),
 new Class[] { clazz }, handler);
}
}

When creating a new test case using the updated implementation, it should pass as is. The Invoker itself can be separated so an interface can be put into the client and the implementation can be put into the EJB tier. The next step would be to change Invoker to an EJB.

Solution Step 6: Creating a sample client

Creating a simple client that invokes the remote service would help in testing. Initially, the local Invoker would be used rather a remote one.

protected void init() {
  getServletContext().setAttribute(ServiceLocator.class.getName(),
  new InvokerServiceLocator());
}

protected void doGet(HttpServletRequest req, HttpServletResponse resp) {
  ServiceLocator locator =
    (ServiceLocator) getServletContext().getAttribute(
      ServiceLocator.class.getName());
  UpperService upperService =
    (UpperService) locator.getService(UpperService.class);
  resp.getWriter().println(upperService.upper("Abcde"));
}

Since this eventually would use an EJB call, it this client should have to be tested on a full J2EE container rather than using something like ServletUnit. It is suggested that an HttpUnit test is created for this one to ensure that it will work in an automated fashion.

Solution Step 7: Converting to an EJB

Converting the Invoker to be an EJB is a purely mechanical process. The constructor is moved to the ejbCreate() method and the boiler plate interfaces, descriptors and manifests are created.

public class InvokerBean implements SessionBean {
private Map map;
public void ejbCreate() throws CreateException {
map = new HashMap();
map.put(UpperService.class, new UpperServiceImpl());
}
public Serializable invoke(Class clazz, String mName, Serializable[] args) {
Object impl = map.get(clazz);
Class[] parameterTypes = new Class[args.length];
for (int i = 0; i < args.length; ++i) {
 parameterTypes[i] = args[i].getClass();
}
Method method = clazz.getMethod(mName, parameterTypes);
return (Serializable)method.invoke(impl, args);
}
}

The interfaces can be implemented as Local or Remote or both. Its recommended that the local interface is used rather than remote, not because of efficiency (some application servers can convert to use pass-by-reference automatically if the EJB is in the same server), but because of security. Local interfaces ensure that the invoker gets used only by the application and not remote clients. Its a choice between not needing to change the code if they want remote or security.

Solution Step 8: Converting InvokerServiceLocator to use the EJB

The final step is to create an implementation of the InvokerServiceLocator to use the EJB instead of a local version.

public Object getService(Class clazz) {
InvocationHandler handler = new InvocationHandler() {
public Object invoke(Object proxy, Method method, Object[] args) {
 Serializable[] serializableArgs = new Serializable[args.length];
 System.arraycopy(args, 0, serializableArgs, 0, args.length);
 Context context = new InitialContext();
 InvokerLocalHome =
   (InvokerLocalHome) context.lookup("java:comp/env/ejb/Invoker"));
 InvokerLocal invoker = home.create();
 return invoker.invoke(clazz, method.getName(), serializableArgs);
}
};
return Proxy.newProxyInstance(clazz.getClassLoader(),
new Class[] { clazz }, handler);
}

When invoking the test client it should yield the same results.

Passing in additional data

On multi-tier applications, sometimes it is required to pass some contextual data stored in a ThreadLocal or other variable from one tier to another. The pattern allows to make the invoke method pass data that is not inside the method signature and send it over the wire. The EJB implementation class can create a new ThreadLocal to store the data or call a separate interface method that would store the contextual data. Although Dependency Injection would be better, ThreadLocal is an easy way of transmitting contextual data around without having to do any of the leg-work. The invoker method on the EJB tier can wrap the setting and clearing of ThreadLocal data as follows:

public Serializable invoke(Serializable contextData,
  Class clazz, String mName, Serializable[] args) {
  try {
    setThreadLocalData(contextData);
    Object impl = map.get(clazz);
    Class[] parameterTypes = new Class[args.length];
    for (int i = 0; i < args.length; ++i) {
      parameterTypes[i] = args[i].getClass();
    }
    Method method = clazz.getMethod(mName, parameterTypes);
    return (Serializable)method.invoke(impl, args);
  } finally {
    clearThreadLocalData();
  }
}

Please note that if dependency injection is used or the services contain attributes, you might not use the map to store the services but instantiate them as needed.

Further Work

The implementation provided in this article has not been optimized for performance. There are several ways of improving this such as creating a cache of methods and caching the JNDI lookup. The list of implementation classes are also hard coded into the Maps and those can be made property file driven. All these techniques would be described in the next article (which I apologize that it had never materialized).