Open Source Projects Blog

Since java.util.logging has a variety of problems in real-world scenarios, we simply can't use it.  In light of that, we depend on SLF4J as a logging framework which allows multiple options for the logging implementation, so that no matter what you do with our code, you can have logging react how you need it to.  However, when building projects which include layers of dependency like we do, you run into the problem of managing those dependencies properly and unless you're the one that set up the chain, it's easy to get frustrated trying to figure out what libraries are of what type.  In this "logging framework separate from logging implementation" setup, this can get confusing.

Hopefully this post will serve to clear that up and be referenced when one starts up a project and asks, "Wait, what should I be putting in my dependency list and how?"

As I've already mentioned, SLF4J is our choice of logging framework.  This is what the code will depend on directly and the classes you write your logging statements with.  Since that is a direct dependency, you would add this to your dependency lists as you would any other library you directly depend on.  In Maven terms, that would be 

but since that's default, your entry would just look like this:

The usual logging implementation we use is Logback.  If you are writing a library, which this blog post is mostly aiming at, then you DO NOT want to depend on this in your released (compiled) code, because it is an implementation detail.  You would be forcing the users of your library to use this logging implementation, which negates the purpose of using SLF4J.  If you are releasing an executable program of some sort, however, it would be fine to do so.  For a library or similar, you would only want to depend on Logback for testing or running test programs, so your scope would be "test", so that the library is NOT packaged with or depended on by any dependency chain.  Your entry would look like this:

 

Often our projects will have multiple Maven modules.  If you follow the usual pattern of a parent pom for the project which defines project-wide dependencies in a <dependencyManagement> block, and you set the scope properly there, in your modules, you can safely just have the simplified entry like this:

Keeping the module dependencies streamlined and managed well from the main project pom.xml.

Maven Build Updates

Since my last posting here, most of the issues referred to in that post have been resolved.  With Root POM 1.0.4 (and Open POM 1.0.4), there should no longer be any issues with Maven 3, the release plugin, the site plugin, etc.  Due to the Maven 3 requirement forced on us by the Android plugin, we are now fully ready for Maven 3.

Note that Root and Open 1.1.0 will be released very shortly.

Annotations

With the release of Guava 11, it now depends on jsr305.  This creates a problem: for years, we've been using jcip-annotations to indicate @Immutable or @Threadsafe status, etc. on our classes.  Recently, we've started to use Guava a lot more (and will likely continue to do so).  Guava has eliminated a great deal of code from dcs-util, for example, and that's a tremendous benefit.  But now we have two different versions of the annotations on the classpath, and that just doesn't make sense.

Findbugs produces the JSR 305 jar file, which places the annotations in javax.annotations.  Unfortunately, JSR 305 is pretty much dead–whatever that means for a JSR that probably has run to its logical completion.  However, JCIP Annotations has been dead (okay, not really: it really is the heart of JSR 305) for a long time too.

Given these changes, the next version of the Root POM (1.1.0) will remove the depedency management entry for JCIP annotations and not replace it.  Users should migrate to the jsr 305 annotations, but they will not be specified in dependency management in the parent POM at this time.

Conversion is a simple swapping of imports, so we should update quickly.  DCS Util 1.1.7 will be converted, and this will lead to changes throughout.  Findbugs supports both the old and the new annotations, so that should transition well.

We will eventually explore the other 'stuff' in JSR 305 as time goes by.

Removal of JUnit Dependency Management

We're not leaving JUnit; we're just not going to manage the version from the Root POM anymore.  Android requires an earlier version of JUnit and other projects may have different needs.  Projects like Minotaur SA should manage this for their children, not in a global corporate parent POM. 

This removes all dependency management from the Root POM.  The Root POM will now be focused on Maven versions and build reproducibility.

Semantic Versioning

We've followed the Linux kernel convention for a long time, except that I'm pretty certain we've never really understood it.  Too often, we've been confused about what version should come next.  The trick is in remembering that odd versions are unstable betas, and even versions are feature frozen releases.  It's pretty unintuitive.

The leading scheme for versioning these days is Semantic Versioning.  It is relatively easy to use with Maven, RPM, OSGi and other common tools that care about version numbers.  The main difference is that pre-releases after 1.0.0 are named with a -alpha.1, or a -beta.1, etc.

It's time to change.  You'll notice that the Root POM and Open POM are going to 1.1.0 instead of 1.0.5.  This will not be an unstable release, but merely the next minor version.  Arguably, we should probably go to 2.0.0, but that feels like too much, and I'm not sure what binary compatibility in terms of XML build artifacts means, so, we'll stick to the minor.  Other projects will switch with their next unstable release.  New projects should start using this scheme.

Maven Build Updates

Root POM 1.0.2 and Open POM 1.0.2 have been released. These releases update a variety of things, but mainly focus on making mercurial releases work. A few notes:

  • The default tag name suggested by the release:prepare task will ALWAYS BE WRONG! You must correct this when doing a release. This should hopefully be fixed soon as it is a regression introduced in 2.2 which just came out. Unfortunately, we need 2.2 to support Mercurial.
  • The site plugin was not updated like the others, because the new version of the site plugin is massively broken in how it attempts to 'correct' URLs, leading to just utter failures on releases. It probably won't be fixed until some indeterminate version of Maven 3. Continue doing site stuff as before.
  • For Mercurial projects, you must not override the developer connection scm url. The one you get from your parent pom (also in hg) will be correct. Just set the connection and the url as usual.

Please update your SVN projects as appropriate.

As of today, I'm migrating the codebase away from Substance and to Insubstantial. This will start in the latest versions of DCS Util, Harmonice, and SA and likely filter out from there. The new libraries are in a separate group id.

Speling Errors has more details (the new maintainers blog).

Please use the new stuff when spinning up a new project.

In order to make it easier to execute Java programs and to stop copying large amounts of antrun code all around from project to project, we've created a maven plugin for this purpose, called Dark Exec. Check out the documentation and please start using 1.0.0 in your projects. Remember to put your configuration in either:

  • src\main\config - Things reread at runtime (logback.groovy, config.xml)
  • src\main\exec - General configuration/scripts (run.sh, run.bat)

Enjoy!

Our open source (and other) projects are switching to Logback. To this end, I'm going to talk about why we are making this change and the details of how to effect the change.

A Brief History of Our Logging

By the time DCS started putting together software, java.util.logging had come into JDK 1.4, and was therefore available to all of our code which consistently targeted the latest JDK release.  Because of this, we never really considered log4j as an alternative.  Log4j had come into existence to address the complete lack of a useful logging platform for JDK 1.3 and earlier.  In response, Sun added their own version of logging into the core.

JDK logging had several advantages over log4j.  The first is that you didn't need to tote an external library around with you.  The second, and probably more important, advantage was a simpler configuration mechanism.  Log4j, however, continued to innovate and added logging to remote sinks (useful in many deployments) and other newer features.  In the community at large, no consensus on which platform was better could be reached, and this led to the atrocity that was Jakarta Commons Logging.

Commons logging brought a whole host of problems to users in a wide variety of scenarios.  This article goes into detail about many of the issues, so I won't elaborate here.  Needless to say, the wide acceptance of commons logging clearly indicated the need from a library point of view at least, for a common way to access whatever logging mechanism the application developer wanted to use.

At DCS, we exclusively used commons logging, but as we developed libraries we intended to be available for other users, we felt the need to address this issue.  SLF4J came to our attention by way of Hibernate, which switched to using it a couple of years ago.  Finding it useful, we began migrating our projects to using SLF4J instead of java.util.logging directly, even on an application basis.  The real advantage of SLF4J is it's {} notation for filling in parameters, which is much simpler than the equivalent (which most aren't even aware of) under java.util.loggging.

Now, we're changing out java.util.logging for logback.

Why Logback?

Here's an example of logback output from SA Viewer:

See the third column?  It's the thread the logging statement came from.  That's why Logback.

In addition to that one killer feature, the cleaner single line output is also highly desirable.  We wrote a custom formatter for java.util.logging to achieve the same thing, but it didn't come into wide use in our projects.  We simply got used to the double line format.

Logback also implements SLF4J natively, without the need for a translation layer, which is slightly more performant.

So How Do We Switch?

Declare a Dependency on Logback Classic

DCS projects should declare a runtime dependency on the logback classic project.  Currently, this looks like this:

We make it runtime to avoid any accidental dependencies on logback details.
Note you should remove dependencies on slf4j-jdk14.

Upgrade to DCS Util 1.1.2

In earlier versions of DCS Utilities, we did some explicit initialization of JDK logging in DcsUtil.getIntance().initialize() that we no longer wish to do.  That initialization is removed starting in 1.1.2 (which will be released today).  That initialization did something very useful, however: it redirected output to the appropriate platform specific location for logging.  How do we do that in logback?

Create Your Logback Configuration

The trick is to use the Groovy configuration options for logback to do what we used to do in the initialization.  This is more flexible and pretty cool.  Note that since Harmonice has a compile time dependency on Groovy, we almost always have Groovy on our classpath anyway, so we don't need to put it there explicitly.  Here's a sample configuration for Minotaur Viewer:

To reference this file, put it on the classpath.  For example, here is how we run Minotaur Viewer:

We have a directory of content in 'src/main/exec' which includes our .bat files and .sh for starting Minotaur Viewer, and we have a 'src/main/config' directory with the logback.groovy file.  We don't put that in the 'src/main/exec' folder to avoid having it copied.  We should include both directories in our assemblies for distribution.

Note that we can modify the src/main/config/logback.groovy file while our program is running and get the modifications to take effect without restarting the application.  This is done in a highly optimized way without an external thread to leak while handling this (a problem with the same functionality in log4j).

Conclusion

Logback gives us some powerful new capabilities, yielding better insight into what's going on inside our software, and frankly that's all you can ask for from a logging library. It's frankly reassuring that even such low level capabilities continue to evolve for Java developers. Please comment below with any questions.

To quote the Android documentation:

A good way to test your application's ability to restore its state is to simply rotate the device so that the screen orientation changes. When the screen orientation changes, the system destroys and recreates the activity in order to apply alternative resources that might be available for the new orientation. For this reason alone, it's very important that your activity completely restores its state when it is recreated, because users regularly rotate the screen while using applications.

Head this lesson well! Your activity is going to be destroyed and recreated a lot, rather you like it or not. All real work must be done in a service.

Android is the most MVC platform, EVER!

DCS Utilities 1.0.0

DCS Util 1.0.0 has been released. This is stable release that we intend to support with bug fixes (if any are needed) at least through Fall 2011, where we hope to replace it with 2.0.0. As a transition aid, we'll eventually release a 1.0.1 with a number of methods marked as deprecated, mainly those that have good alternatives in Google Guava, which we intend to avoid duplicating. Things like SimpleFileParser are actually implemented in much the same way we did in the Files class, so it makes sense to drop it.

There will not be a 1.2.0 release.

We've also created a wiki space just for DCS Utilities and updated the documentation a bit in there.  We'll be cleaning up the documentation for the rest of our open source projects in a similar way.

DCS Utilities 2.0.0

The road to DCS Utilities 2.0.0 has begun.  See our Roadmap for more information on the overall plan.  DCS Util 1.1.0 will be dropping tonight into our Maven repository, featuring support for Android on the core and the fragmentation of the utility classes into a number of different modules.  1.1.0 is very unstable right now; we expect some significant changes before we release 2.0.0.

We've started to use a lot more UUIDs in our code to identify the source of various things, and need to persist that data into our databases. We usually use Hibernate, so here's a quick trick on how to do that.

To allow a UUID field to be automatically persisted and loaded form the db without having to split it into longs, you can annotate it like this:

UUIDs and Hibernate

So the Minotaur SA Router Plugin Minotaur Analysis Plugin (how's that for an official name?) has this bit of code:

which used to work very well for mapping a key/value map of basic types. But when Minotaur Analysis tried loading these records to process them, it threw exceptions about 'MAPKEY' not being defined in the table. This used to work...

The Culprit - Hibernate Versions

Minotaur SA (creator of the database in question) is running Hibernate 3.5.3. Minotaur Analysis (consumer of the database) is running the version of Hibernate that comes with JBoss 5.1.0, a disappointing Hibernate 3.3.1. Between these two versions, two things happened:

  • @CollectionOfElements was deprecated
  • The default name of the key column was changed

In the new world of Hibernate 3.5.3, this kind of collection should be mapped with the JPA 2.0 annotation @ElementCollection. The name of the column holding the key will now be name of the collection variable with _KEY appended to it. It used to be MAPKEY.

When I tried to load the mapping in the older version of Hibernate, it looked for the older column name (MAPKEY), which wasn't there, and was therefore very confused. The solution to this is actually fairly simple:

The trick here is to just specify the name of the key column using the (also deprecated by the way) @MapKey annotation. This is the annotation from the Hibernate namespace, not the java.util.persistence one. This annotation works equally well in both older and newer versions of Hibernate.

The real solution is to update Minotaur Analysis (and JBoss) to a more recent version of Hibernate. Not sure what to do about that, but for now, this is the required workaround.

If you're getting errors with JXTables when you upgraded from SwingX 1.0 (or before) to SwingX 1.6 (or later) even though you've changed no code, here's why (as found on java.net forums):

The difference between SwingX 1.0 and 1.6 is target jdk version: the latter uses core sort/filtering mechanism

The difference between JXTable and core JTable sort/filter is that the former's autoCreateRowSorter defaults to true (plus it controls the sorte's properties) which causes the exception

To disable:

JXTable table = new JXTable();
table.setAutoCreateRowSorter(false);
table.setRowSorter(null);
table.setModel(myModel);

Found here: http://forums.java.net/jive/thread.jspa?threadID=152412&tstart=0

JXTables that depend solely on the user selecting which column to sort by (common in my case) would not have ever used these methods previously since the default used to be what I needed. Somehow the sorter created by default does NOT get the proper row count from the model in some cases, so it being created and somehow misinformed causes an ArrayIndexOutOfBoundsException when trying to display or update a specified row. Why that's the case is another item to investigate...

Oracle sues Google over Android

Here we go again... Going to party like it's 1999...

Interestingly, it's just been announced that Google has shut down development on Google Wave (CNET News Story ). They cited a lack of adoption by customers, and a lack of understanding of what the product actually is.

Google attempted to create a paradigm for communication that makes a lot of sense, particularly on mobile devices, but really is a new method of communicating outside of what we normally think of. Email is just letters you send on your computer. IM is simple enough to be understood by almost everyone. But when you combine features of both with wikis, and the ability to embed outside content, it just goes over almost everyone's head. Wikis are pretty hard for people to get their heads around as it is--Word and Track Changes (ugh) would be dead by now if that wasn't the case.

The other real problem with this is 'what is it for?' Google never really answered that--it was a solution in search of a problem. The funny thing is that I think we've figured out the problem it is a solution for: tactical collaboration. It just wasn't one Google ever though of as they don't work in that space. Funny enough, another problem it could solve is similar to the developer collaboration features added into NetBeans.

The notion of collaborating on content while being able to IM and keeping the full history is a powerful solution. It's just not a problem that people have realized they actually have, even though they actually do. Faced with a market like that, it's no wonder they tossed in the towel. I do not think this is the end for Wave though. We'll just have to see.

SwingWorker is Broken

This is no exaggeration: the core SwingWorker is completely broken.

The Problem

In JDK 1.6 u16, Sun introduced a fix to clean up SwingWorkers in multiple applets in the same JVM. Previously shutting down an applet would not shut down all the associated swing workers until the JVM terminated in some situations (this was actually a regression introduced along the way). The problem is that the fix to this bug (6799345 on the bug parade) has an introduced a much more severe problem: all SwingWorkers now run in a single background thread. Every SwingWorker in your application runs in a single background thread. So if one of your SwingWorkers is a long running task that periodically updates your GUI, it essentially hoses every other SwingWorker from ever running. (6880336 )

This came up in our SA Viewer product, which as soon as I added a plugin that queried an independent web service, I stopped seeing the core updates coming from an independent Swing Worker.

Simple Workaround

There is a simple workaround, however: just switch to the SwingWorker from the SwingX project. Almost all of our Swing projects already have that on the classpath, so it's just a simple import change. Their SwingWorker was meant to be a backport to 1.5, but they have failed to track all of the recently introduced regressions...

Other workarounds are described in detail on the bugs above.

A Disturbing Pattern

The delay in shipping JDK 7 has led some at Oracle (formerly Sun) to backport changes from JDK 7 to JDK 6. A number of these backported changes have introduced significant regressions. A nasty array out of bounds error in the built in web server had us stuck on JDK 1.6 u14 all the way through JDK 1.6 u19. This one is simpler to work around, but I worry about quality control, and I worry about them telling us the honest truth. The minor updates (uXX) should be isolated to security issues. If we have to update to fix security issues, we don't expect core APIs to start blowing up on us. These changes should have driven the platform to 1.6.1, not still 1.6.0. I suppose Oracle doesn't want to fork their code and maintain all the branches. There are two alternatives: 1) fix bugs like this very quickly when they are caught (within two weeks); 2) amp up your internal testing to catch regressions before they start, or pre-release to those who can help you test.

The problem is that Oracle is forcing that burden on us, the consumers of the platform. For consumer deliverables, rather than enterprise deliverables, the burden is even worse.

DCS Note

All DCS team members are instructed to remove all uses of the core SwingWorker from any internal code (switching to the SwingX version). We'll probably switch back slowly after the upgrade, but any use of SwingWorker from the core right now is a regression for us.

Netbeans 6.9 went live today, so I quickly upgraded. I opened up a project I'm working on and promptly got a warning that I had declared a method both static and final, and that this was useless. Huh?

Language Effects of Static Methods

Say you have a parent class:

Parent

and a child class:

Child

If you call call Parent.blah(), you'll get 'Parent' in your output. If you call Child.blah(), you'll get 'Child' in your output. The method is resolved by the class reference. If you call it from an instance, say:

Don't Do This

you will get 'Parent'.

This behavior is likely to be confusing. It's one of the reasons our coding conventions say not to use instances to call static methods (preferring the Parent.blah() or Child.blah() forms) as you are less likely to make an error.

Adding in Final

If you declare the blah() method in Parent final, you will not be permitted to declare the method in Child with the same signature. The above code would not compile. This is actually different than the meaning of final on a regular method, however. The final in this case prevents shadowing, not overriding. You cannot actually override a static method.

Netbeans 6.9, by default (yes, you can turn this off), thinks that this is not useful. Let's say you have a main method in a child and a parent (again, don't do this), they can live independently perfectly happily. Why forbid this? I learned (more than a decade ago at this point) to write:

and the final just rolls off my fingers after the static. Besides, I write a lot of thread-sensitive code, so static non-final always makes me twinge.

The Question

So should we avoid the declaration since it doesn't really mean what many will think it means? Or should we use it to prevent someone from maybe doing something bad? I'm inclined to keep the final (perhaps more motivated by inertia than anything else), but I'm open to thoughts.