Thomas Keller – Page 3

jazzlib – an alternative for reading ZIP files in Java

Java had zip-reading capabilities for a long time, naturally because `jar` files are simply compressed zip files with some meta data. The needed classes reside in the `java.util.zip` namespace and are `ZipInputStream` and `ZipEntry`.

Recently, however, `ZipInputStream` gave me a huge headache. My use case was as simple as

* read the zip entries of a list of zip files (each varying in size, but usually around 20MB)
* skip to the zip entry that has a certain name (a single text file with only two bytes of contents)
* read the contents of this zip entry and close the zip

Doing this for about 25 files took my Pentium D (2GHz) with 3GB of RAM roughly **20 seconds**. Wow, 20 seconds really? I created a test case and profiled the code in question separately with [YourKit](http://www.yourkit.com) (which is a really great tool, by the way!):

It got stuck quite a bit in `java.util.zip.Inflater.inflateBytes` – but that seemed to use native code, so I couldn’t profile any further.

So I went on and searched for an alternative of `java.util.zip` – and luckily I found one with [jazzlib](http://jazzlib.sourceforge.net), which provides a pure Java implementation for ZIP compression and decompression. This library is GPL-licensed (with a small exception clause to prevent the pervasiveness of the GPL) and comes in two versions, one that duplicates the single library classes underknees `java.util.zip` (as a drop-in replacement for JDK versions where this is missing) and one that comes in its own namespace, `net.sf.jazzlib`.

After I went for the second version, I restarted my test and it only took about **7 seconds** this time. At first I thought that there must be some downside to this approach, so I checked the timings for a complete decompression of the archive, but the timings here were on par with the ones from `java.util.zip` (roughly 5 seconds for a single 20MB file).

I haven’t tested compression speed, because it doesn’t matter much for my use case, but the decompression speed alone is astonishing. I wonder why nobody else stumbled upon these performance problems before…

Indefero 1.1 released

I’m pleased to announce the immediate release of [Indefero 1.1](http://projects.ceondo.com/p/indefero). This release features support for another version control system, [monotone](http://www.monotone.ca), and comes with tons of smaller improvements and bug fixes.

A full list of changes can be found in the [News](http://projects.ceondo.com/p/indefero/page/News/) document.

Many thanks to all the contributors!

mtn-browse 0.72 and accompanying Perl library released

Tony Cooper announced a new release of his [Monotone Browser](http://www.coosoft.plus.com/software.html) software and also a new version of the underlying [Monotone::AutomateStdio](http://search.cpan.org/~aecooper/Monotone-AutomateStdio-0.12/lib/Monotone/AutomateStdio.pod) Perl library.

Both packages are now compatible with the most recent version of monotone, 0.99.1. Additionally, mtn-browse also supports all the new selector functions introduced in monotone 0.99 and is able to restrict revision and file histories to specific branches.

Many thanks to Tony for his outstanding work! I’ll update the MacPorts packages in a few…

One framework to rule them all

I have a love/hate relationship with PHP; while I have [talked in great length about all the badness](http://www.thomaskeller.biz/blog/2010/03/23/fuck-php/) that comes with it, I still find myself quite often writing and / or contributing to PHP applications. Most of this work was done with [Symfony](http://www.symfony-project.org) and, to a smaller degree, recently also with a relatively unknown framework called [Pluf](http://www.pluf.org), which is the main building block for [Indefero](http://www.indefero.net), the project and source code management tool I’m contributing to.

The author of Pluf, Loic d’Anterroches, has released a new framework to the wild today and its called [**Photon**](http://www.photon-project.com). It got its name from the particle, so naturally the tag line is *”…because nothing is faster than a photon”*.

So you are by now probably asking “Ok, but **why should I care** about that? There are dozens if not hundreds of PHP frameworks out there, they all claim to be fast, so this is probably just a complete waste of time…”

Well, Photon is very different, because it achieves its performance with a break of the traditionally known request-response model: Much like a real **application server** you know from other languages you can process requests asynchronously now in pure PHP, i.e. start and stop processes on request and delegate long-running work for later execution!

PHP developers did all kinds of bad tricks to emulate this stuff in the past. Most of these tricks employ a cache table and a cron entry that regularily executes some PHP script. This is buggy, doesn’t scale well and is of course plain ugly.

I won’t go into more details – just hop over to [www.photon-project.com](http://www.photon-project.com), read the docs and grab the pre-release while its hot!

Nazi-Überfall auf Wohnprojekt in Dresden

Ich bin eigentlich gegen jede Form von Gewalt, egal ob sie von links oder rechts kommt. Nach [diesen schockierenden Bildern vom letzten Wochenende](http://www.spiegel.de/video/video-1110518.html) aber rücke ich von meiner Position ab.

Unbelehrbaren Menschen muss Einhalt geboten werden – einer Horde von etwa 200 wildgewordenen Nazis, die 15 Minuten lang ein Wohnhaus in Dresden mit Steinen zu bewerfen, erst recht! Wie kann es sein, dass die Polizei – wenige Meter enfernt vom Ort des Geschehens positioniert – unfähig ist, Warnschüsse abzufeuern oder auf eine andere Art einzugreifen, um diesen Treiben ein Ende zu bereiten?

Wenn der Gesetzgeber und die Polizei weiterhin so unwillens sind, nationalsozialistisch gesinnte Gewalttäter wirksam zu bekämpfen, dann sehe ich die autonome Linke in Zukunft als die letzte Bastion vor den marodierenden Rechtsradikalismus in diesem Land. Dann haben sie mich als Unterstützer, denn dann bin ich ganz klar **für Selbstjustiz**.

Understanding the Chinese

We have to face more competition from the east and should start being less arrogant and ignorant. And we should prepare that the dominance of the western civilization will end sooner than we think.

Moments of love

There are certain moments in my life where I feel this love. Love and satisfaction. In these moments I do not regret at all to still work for an “underdog” version control system like [monotone](http://www.monotone.ca) and have not converted to the dark side of the moon (read: git). One of these moments was today, when I did a merge between two branches.

See, many people working with Subversion or even git are annoyed of its merge behaviour. Sometimes they even fear to merge because hell breaks loose if the source branch moved files or even directories (yikes!) around. I cannot tell how often I had to resolve tree conflicts by hand in Subversion and I also cannot tell how often git forced me to clean my workspace from uncommitted or stashed changes before I was able to pull new stuff in.

Today now with monotone, I propagated the latest changes from the [main branch](https://code.monotone.ca/p/monotone/source/tree/h:net.venge.monotone/) to my little [source-tree-cleanup branch](https://code.monotone.ca/p/monotone/source/tree/h:net.venge.monotone.source-tree-cleanup/). The latter moves many files and even whole directories around, so one would expect merge chaos in an ordinary version control system if files inside these moved directories would have been changed, or even new files would have been added, in the main branch in the meantime, right?

Not so with monotone. It not only tracks the file identity and rename history for files, but also for directories, so the merge went flawlessly without manual intervention:

$ mtn propagate net.venge.monotone net.venge.monotone.source-tree-cleanup
mtn: propagating net.venge.monotone -> net.venge.monotone.source-tree-cleanup
mtn: [left] 4f505ed0817cef6a34e1cc404b0511426a15fef9
mtn: [right] ec9269fa9ddd7b7e2d4716610241a7577159ba9d
mtn: [merged] 1b5c865085537cebd09b5e8481fba1eebfd63a03

And the update afterwards was no exception:

$ mtn update
mtn: updating along branch 'net.venge.monotone.source-tree-cleanup'
...
mtn: adding test/func/diff_order hinzu
mtn: adding test/func/diff_order/__driver__.lua hinzu
...
mtn: updating NEWS
...
mtn: updating src/cmd_diff_log.cc
mtn: updating src/cmd_list.cc
...

(some context: `./test/func` was `./tests` before and `./src` was `./`)

Call me fanboy, but I _love_ this version control system!

monotone activity roundup

If you remember my earlier post(s) you probably ask where the highly anticipated version _one dot zero_ remains. Originally we planned to release it by the end of 2010, but we couldn’t really meet that date because we really wanted to polish a couple of things before releasing it. So what is the new plan then?

Well, we agreed that we want to release not later than by the end of Q1 2011 and if you look at our [RoadMap](http://wiki.monotone.ca/RoadMap/) you see that most stuff is already worked on, but you also see that we have quite a lot to do.

#### Core Monotone

Stephen did a great job over the last couple of weeks and reviewed our manual: He restructured the commands section, included many cross references, reworked and clarified many sections and purged old contents.

Richard, Stephen and Lapo worked on the wiki and tagged contents for certain user groups, i.e. users and developers. They also reviewed and cleaned up many pages which are often looked after and accessed. While all this is not quite finished yet, it already looks a bit better than before.

Richard and me fixed [a couple bugs](https://code.monotone.ca/p/monotone/issues/label/399/open/) that have been reported since 0.99. Right now the tracker looks more or less green with only two issues left open.

Beside the issues I’ve been working privately on the [source tree cleanup branch](https://code.monotone.ca/p/monotone/source/tree/h:net.venge.monotone.source-tree-cleanup/) and got some help from Richard here as well. This branch basically moves tests and sources around and puts them into a proper directory structure, so we do not clutter the main directory with sources, documentation, utilities and other stuff. I hope I can finish this branch next – only some work on the documentation is missing – and can merge that soon to the trunk.

#### Tools

Three others news are not about monotone itself, but related to it: [usher](https://code.monotone.ca/p/contrib/source/tree/h:net.venge.monotone.contrib.usher/), [indefero](http://projects.ceondo.com/p/indefero/) and [guitone](http://guitone.thomaskeller.biz).

Richard is currently preparing usher’s sources for packaging. We learned a lot about a proper usher setup when we configured usher with indefero on our own server instance, [code.monotone.ca](https://code.monotone.ca), and while Richard used that knowledge to bring usher forward, I reworked the [sources](http://projects.ceondo.com/p/indefero/source/tree/develop/src/IDF/Plugin/SyncMonotone.php) and [documentation](http://projects.ceondo.com/p/indefero/page/InstallationScmMonotone/) of the SyncMonotone plugin in indefero.

Beside that I’m also working on including a graphical branch view to guitone. Work has gotten somewhere already as you can imagine from the screenshot, but there are a couple of subleties to fix and clean before I’ll release that actually. Packaging this version will be kind-of fun; yesterday for example I stumbled upon a crash in libgvc after I moved the Graphviz libraries into the Mac OS X application bundle (which is needed for easy deployment), Windows packaging hasn’t even be started in this regard and I also need to investigate further if and how Graphviz’ Common Public License (CPL) allows me to distribute its binaries with my binaries at all.

#### Roundup

So you see we’re not lazy, but we all have so many things to do that it will just take a little longer. So stay tuned and thanks for your continued support!

Debugging Java WebStart applications with Eclipse

The last couple of days I went through a little nightmare: I needed to debug a Java application which showed some weird behaviour only when loaded via WebStart, but not when executed within Eclipse.

[Multiple](http://www.jacoozi.com/index.php?option=com_content&task=view&id=119&Itemid=134) [internet](http://javaswamy.blogspot.com/2008/10/debugging-jws-with-eclipse.html) [resources](http://www.ibm.com/developerworks/java/library/os-eclipse-javadebug/index.html) told me to use the `JAVAWS_VM_ARGS` environment variable to tell the Java VM to either create a debug socket itself or connect to some socket started from Eclipse and then simply set a breakpoint there and wait for the meal. But nothing worked out for me, until I found two very important issues nobody wrote about so far:

* Under Java 6 `JAVAWS_VM_ARGS` seems to be completely ignored by `javaws`, on Linux and on Windows. The way to go was to use separate `-J` options in the call to `javaws` – and given a running Eclipse debug server you finally saw the threads of your running application.
* Wait, were these threads really belonging to _your_ running application? This was the second issue. Apparently `javaws` directly forks away as soon as it started and runs your application in a separate VM, which _of course_ did not get the original debug options passed. Again, some internet resources said it would be enough to load the jnlp file from the net and start locally, but actually this did not work out. The magic option to apply to `javaws` here was `-Xnofork` and finally, on the next run I saw my application’s threads in Eclipse and my breakpoints magically worked as expected.

Here is the complete command line example again (given a running Eclipse debug server on port 8000):

javaws -Xnofork -J-Xdebug -J-Xnoagent \
-J-Xrunjdwp:transport=dt_socket,server=n,address=8000 \
path/to/app.jnlp

Hope that helps someone 🙂

monotone docathon on Sunday, 18:00 UTC

The monotone team holds a “docathon” – a small documentation sprint – next Sunday, Dec 19th 2010, starting at 18:00 UTC for approx. 6hrs. During this time we meet on IRC and want to collaboratively tidy the [wiki pages](http://wiki.monotone.ca), further improve the [manual](http://www.monotone.ca/docs), update [INSTALL](https://code.monotone.ca/p/monotone/source/tree/h:net.venge.monotone/INSTALL) and cleanup other text-based files and more for the upcoming 1.0 release.

So if you want to lend a helping hand to your favourite version control system, just drop by on [IRC](irc://irc.oftc.net/#monotone), say hello and we’ll quickly get in contact with you 🙂

See you on Sunday!