20050920

A Disk Full of JAR Files

I am not bragging, but I have 12698 JAR Files on my local hard disk. I am a heavy Java user and developer, but surely there must be something wrong with the platform if you get this kind of proliferation. I don't expect it to be any kind extraordinary example - certainly no record.

I know that if I used a better OS I could hard link them and get a few gig back, but I should not have to do that. I am also aware that quite a few of these jars are copies produced by various application server instances and ant builds.

To illustrate the scale of the problem I scanned for Jakarta's commons-logging.jar. I found over 200 of them - 4 different versions. That is not too bad because each jar is only around 30k. But log4j is ten times the size and when I searched for that jar I found about the same number. Some commonly used (and growing) jars are very large, like Xalan and jaxrpc.

All this disk usage makes the Java platform/technology the single biggest user of disk space on my machine. Java is already the biggest user of my cpu cycles and memory. And this annoys me, that bloat could be easily avoided.

So that is my suggestion for the next version of Java: solve the bloat problem.

My own suggestion for a solution would be to make the classloader use an inbuilt cache of jars. So that way jars are imported into the VM environment rather than just loaded by reference. Now that most manifest files have the 'Class-Path:' setting then it would be possible to extend this behaviour to a build guid style reference model.

There are obvious problems with this solution that I am aware of. But even a slightly better system would alleviate the issue.

5 comments:

Matt Raible said...

Heh, I just checked and I have 23,271. I would say 1/2 of those are in "build" directories - so the number isn't very accurate.

I guess that's what I get for having 123 open source projects in my "source" directory. ;-)

Anonymous said...

It could be worse. :-) You could have a registry where all of the jars form a spider web of references using GUIDs upon GUIDs.

Anonymous said...

The proliferation of redundant jar libaries is one of the problems being solved by the jpackage.org project, for LSB compliant Linux distributions. Red Hat is now officially cooperating with JPackage to ensure compatibility. I use it on Fedora Core 4. Works great.

It really makes Java feel like part of the OS, not bolted on. Highly recommended.

Anonymous said...

It's exactly the problem that is aimed to be addressed by the JSR 277. But you'll have to wait for dolphin to see it be part of the standard JRE...

straun said...

Thanks for some good comments:

matt: that is pretty good going, I wonder if there is anyone out there with more.

jpackage.org: is something I use on another FC4 machine that I have. But I find it a little inflexible for being able to develop effectively against. Great for deployment though.

xavier: I will keep an eye on the JCP for that one.