Qt itemviews trouble

Whoever digged a little deeper into Qt’s item views and models in the past knows that they’re quite tricky beasts sometimes. Two problems where driving me nuts recently:

1. How can I auto-expand the root node of a `QTreeView` whenever the view is resetted?
2. How can I restore the expansion / root state of a `QTreeView` when a model filter cleared all the items out before, effectively resetting the view as in 1.?

After a lot of struggling and lots of debugging I think I found a solution for both problems, which are slightly connected.

To get the auto-expansion feature, I connected the `layoutChanged()` signal of the model to a timer in my view, whose `timeout()` signal then again called the following slot:

void MyTreeView::delayedLayoutChanged()
{
    QModelIndex index = model()->index(0, 0, QModelIndex());
    expand(index); // or alternatively: setRootIndex(index);
}

This simply takes the first element of the model and expands it. Why the trick with the timer? Well, apparently the `layoutChanged()` signal is already processed internally by `QAbstractItemView` or `QTreeView` and its implementation resets the view state completly – which is of course problematic if it gets executed _after_ our own slot. A single-shot timer with an interval of 0 ms is enough here to make Qt use the next event loop cycle and do the right thing for us.

Now, things get more complex if you have a `QSortFilterProxyModel` attached to your tree view – imagine one of these fancy find-as-you-type inputs. If a user enters a search word which clears out all items from your current view and then resets the input again to see the original view, the view eventually gets a `layoutChanged()` signal as well, but because of the clearing you may have lost the expansion level in which the user was before!

`QPersistentModelIndex` is here for the rescue. All you have to do is remember the model index you previously used to expand your view and use that instead of the static one if it is still valid:

QPersistentModelIndex MyTreeView::lastExpansion;
void MyTreeView::expandSomething(const QModelIndex & index)
{
    lastExpansion = static_cast(model())
        ->mapToSource(index);
    expand(index);
}
void MyTreeView::delayedLayoutChanged()
{
    QModelIndex index;
    // is the item still part of the source model? 
    // (important for dynamic source models)
    if (lastExpansion.isValid())
    {
        index = static_cast(model())
              ->mapFromSource(lastExpansion);
    }
    else
    {
        index = model()->index(0, 0, QModelIndex());
    }
    expand(index); // or alternatively: setRootIndex(index);
}

While `QPersistentModelIndex` and `QModelIndex` are two distinct classes, the Trolls made them easily interchangable without special casting magic.

Note that you always need to save the source index and never the proxy index which might be invalidated if it is removed from the view. Things get of course even more complex if you have not one, but two proxy models which filter your view…

Fuck php

Seriously, fuck it. Not only for it long-standing inconsistencies in the “user API”, no, I’ve rarely seen a piece of source code crap with such a low comment / code ratio.

What I’m trying to do? Debugging `SoapClient` from `ext/soap` and figure out why it ignores my `typemap`. Yes, there is a trace mode – but this will only fill the “private” `__getLastXXX()` methods, so I’m doing `printf()` debugging, within PHP’s source, as if it would be 1996 again. An amazing blaze from the past.

Oh, and in case you wonder if PHP can finally fully interoperate with other standard SOAP servers like Apache’s Axis, no, it still can’t. The developer of ext/soap is busy with other tasks these days, quoting him “[…] your WSDL uses overloaded functions […] and ext/soap doesn’t support them. I hardly believe it’ll support them in the future, in case nobody provide a patch.”, but hey, the aforementioned bug is only open for 5 or so years, right?

Apparently those people who give PHP the “enterprise ready” notion now are just waiting until SOAP died completly and everything has been replaced by the next best thing or what? Stupid morons, they should look at the source code of their “product”, think about it for ten seconds and finally run away screaming loud and beg the lord for forgiveness.

monotone 0.47 released

The monotone team is proud to announce the release 0.47 of our beloved version control system! This release features many bugfixes, a new translation (Portuguese) and major performance improvements for projects with larger trees. A complete list of changes can be found in NEWS.

You can grab it at the usual location – binaries are posted there as they come in.

Thanks to all people who made this possible!

Symfony development

Last week the second incarnation of Symfony Live came to an end and I just had the time to check a couple of shared slides from the event.

Definitely interesting stuff going on there, especially the preview release of Symfony 2.0 whose code is available on GitHub since a couple of weeks and which makes major changes to the “good old way” one used to write symfony applications (actions are now “controllers” and extensions “bundles” and well, a dozen of other things changed as well of course… you can read everything in detail here).

Also, Doctrine 2.0 seems to be the first PHP ORM which decouples the modelling approach from the actual database abstraction layer, skips the need for base classes and enables the model definition via annotations. Also, they seem to fight against the overly complex magic from Doctrine 1.x (one of my top complaints on Doctrine in comparison to, f.e. Propel) – maybe I’ll revisit Doctrine again when the next version gets stable.

The guys at Sensio labs do really have a fast development pace and I get more and more the impression that the Symfony ecosystem is the major competitor for the Zend framework. Community-wise I think Symfony is already much bigger than any other PHP framework.

guitone 1.0rc1 released

I’m proud to announce the immediate release of guitone-1.0rc1. This is the first release in a series of smaller releases which aims at the stabilization of the guitone codebase. Many (if not most) of the features one would consider needed for a “1.0” release have been implemented, a couple of outstanding bugs (noticable FS#39, FS#41 and
FS#42) have to get fixed beforehand though before that happens. Please test and report bugs if possible.

Outstanding news of this release:

  • Synchronization with other monotone nodes is no possible
  • Workspace action implementation almost feature-complete (add, drop, revert, rename, ignore, unignore and update finally work)
  • guitone can now create new monotone databases and setup new projects
  • Much improved key management with the ability to change the passphrase of keys, filter the key output and also drop keys
  • Many more bugfixes and smaller improvements as well as stabilization of the codebase

For a complete list of changes please check the NEWS file. I’ll try and prepare a Windows binary in the next couple of days, but probably won’t get to that before Wednesday, so if you want to help out, drop me
a note. A binary for Mac OS X should arrive shortly as well.

Never trust doctrine:data-dump…

…and especially not if you get the impression that the dump will afterwards be readable by the `doctrine:data-load` command of symfony.

It was a costly lesson today when I tried to reimport a dump of a couple of Sympal tables. One of them, the one which models the menu items, has a nested set behaviour, and apparently this one cannot be restored properly by doctrine:

[Doctrine_Record_UnknownPropertyException]                                    
  Unknown record property / related component "children" 
  on "sfSympalMenuItem"

Apparently this particular issue popped up a couple of times in the past for other people as well (Google for it) and while the help of `doctrine:data-dump` still (Doctrine 1.2) blatantly states

The doctrine:data-dump task dumps database data:

./symfony doctrine:data-dump

The task dumps the database data in data/fixtures/%target%.

The dump file is in the YML format and can be reimported
by using the doctrine:data-load task.

./symfony doctrine:data-load

(with the emphasis of “can be reimported”)

the author of Doctrine, Jonathan Wage, told me today on Sympal’s IRC (shortened):

<jonwage> we don’t want people to think you can dump and then restore
<jonwage> that is not what the data fixtures are for
<jonwage> b/c dumping and then loading will never work
<jonwage> an ORM modifies data on the way and and the way out
<me> I mean the least thing doctrine could do there is that if it detects the nested set behaviour it should error out clearly on dump
<jonwage> so you can’t dump the data through an ORM and then try and reload it
<jonwage> i.e. hashed passwords
<me> if dumping is “never” going to work – why do you support dumping into yaml at all?!
<jonwage> if we do that then we would have to throw errors in sooooooo many other cases too
<jonwage> because it is at least a little bit of a convenience
<me> its like a half-baked feature then
<jonwage> we dump the raw data
<jonwage> and you can tweak it
<jonwage> thats my point though, it will ALWAYS be a half baked feature thats why we document it that way
<jonwage> it can NEVER work 100% the way you want it to
<jonwage> so if we fix that one thing, a million other things will be reported that we cannot fix
<jonwage> bc an ORM is not a backup and restore tool
<jonwage> it is impossible

Now I know that as well. My only problem was that I struggled “what is wrong with my fixtures” the whole time and never dared to ask “what is wrong with doctrine”…

Tip: Logging with Symfony >= 1.2

Imagine you have a business method in your model which needs to be accessed by two environments: once from a symfony task and once from the web. So far so good, now what if this business method should be able to log contents somewhere visibly, in case of the command line task to console and to a file and in case of the web application to the default logging mechanisms used there?

Getting the logger in web context is easy, all you have to do is

$logger = sfContext::getInstance()->getLogger();

but its a little harder to do for the command line task.

By default no symfony context is created for a command line task and even if it is created, the above call returns an instance of sfNoLogger. Logging in command applications happens through the sfTask::logSection() method, which basically throws an event at the created dispatcher in SYMFONYDIR/lib/command/cli.php. There you can also see that an instance of sfCommandLogger is created, but there is no way to get your fingers at this instance, because its purely local.

So what can we do? Parametricizing the business method with the sfTask instance and using the logSection() is obviously no solution, because this would break in web context where no such sfTask instance exists…

My solution was a bit more straight forward – I simply decided to not use the task-supplied logging schema at all, but created my own logger like this:

$dispatcher = new sfEventDispatcher();
$logger = new sfAggregateLogger($dispatcher);
$logger->addLogger(new sfCommandLogger($dispatcher));
// optionally add another file logger
if ($logToFile)
{
    $logger->addLogger(
        new sfFileLogger($this->dispatcher, ...)
    );
}

Hope this helps somebody.

monotone 0.46 released

The monotone developers are proud to announce the release of version 0.46. The highlights in this release are bisection support – thanks to Derek Scherger! – and the possibility to call the automation interface over the network – thanks to Timothy Brownawell!

Please note that stdio interface has been changed in an backwards-incompatible way. More information can be found in the documentation and in an earlier blog post of me.

Thanks again to everybody who made this release possible! Grab it while its hot – MacPorts already has the new version and other binaries should follow shortly after this announcement.

Doctrine Horror

My latest Symfony project uses Doctrine as ORM, which is considered to be a lot better than Propel by many people…

Well, not by me. Doctrine seems to have a couple of very good concepts, amongst them built-in validators, a powerful query language, and last but not least, an easy schema language. (Though to be fair, Propel will gain most of these useful things in the future as well or already has, f.e. with its `PropelQuery` feature.)

But Doctrine also fails in many areas; the massive use of overloads everywhere makes it very hard to debug and even worse, it tries to outsmart you (the developer) in many areas, which makes it even more hard to debug stuff which Doctrine doesn’t get right.

A simple example – consider this schema:

Foo:
  columns:
     id: { type: integer(5), primary: true, autoincrement: true }
     name: { type: string }

Bar:
  columns:
     id: { type: integer(5), primary: true, autoincrement: true }
     name: { type: string }

FooBarBaz:
  columns:
     foo_id: { type: integer(5), primary: true }
     bar_id: { type: integer(5), primary: true }
     name: { type: string }

(I’ll skip the relation setup here, Doctrine should find them all with an additional `detect_relations: true`)

So what do you expect you see when you call this?

$obj = new FooBarBaz();
print_r($obj->toArray());

Well, I expected to get an empty object, with a `NULL`ed `foo_id` and `bar_id`, but I didn’t! For me `foo_id` was filled with a 1. Wait, where does this come from?

After I digged deep enough in Doctrine_Record, I saw that this was automatically assigned in the constructor, coming from a statically incremented `$_index` variable. I could revert this by using my own constructor and call `assignIdentifier()` like this:

class FooBarBaz extends BaseFooBarBaz 
{
   public function __construct()
   {
      parent::__construct();
      $this->assignIdentifier(false);
   }
}

but now this object could no longer be added to a `Doctrine_Collection` (which is a bummer, because if you want to extend object lists with “default” empty objects, you most likely stumble upon a Doctrine_Collection, which is the default data structure returned for every SQL query).

So you might ask “Why the hell does all this impose a problem for you?”

Well, if you work with the `FooForm` created by the doctrine plugin for you in Symfony and you want to add `FooBarBazForm` via `sfForm::embedFormForEach` a couple of times (similar to the use case described here), you suddenly have the problem that your embedded form for the appended new `FooBarBaz` object “magically” gets a foo_id of a wrong (maybe not existing) `Foo` object and you wonder where the heck this comes from…

I have my lesson learned for the last one and a half days. I promise I’ll never *ever* create a table in Doctrine with a multi-key primary key again and I’m returing back to Propel for my next project.

monotone automate stdio overhauled [Update]

Yesterday my “automate-out-of-band” branch finally made it into monotone’s trunk. This is a prerequisite for the support of netsync commands in guitone I’ve blogged about earlier, as it makes in-stream informational, error and ticker messages possible, even for remote connections!

While I was at it, several other small things have been changed in stdio – f.e. the error code is now only issued once as payload of the ‘l’ stream-, which mean unfortunately that monotone 0.46 will break compatibility with clients which only understand the pre-0.46 output format. To avoid another hard break like this in the future, a new header section has been added to both stdio’s and the first header which is issued there is the “format-version” header:

$ mtn au stdio
format-version: 2

[...actual output...]

`stdio-version` is promised to stay constant as long as the output format doesn’t change and will be incremented by ‘1’ if there is any other major dealbreaker in the future. This is actually different from the `interface_version` number we also have in the automate interface, whose major number will raise everytime an incompatible change is made to any automate command. We’ll possibly change this behaviour to something more client-friendly in the future, but there is no ETA on this yet, as the current system is still good enough.

All changes for (remote) stdio will be clearly documented in the manual once 0.46 is out. Before this release happens though, I plan to finish the “automate-netsync” branch as well… Holiday time is hacking time 🙂

[Update: It was decided to name the version header “format-version” instead of “stdio-version”; I’ve updated my example accordingly]