In case you have to access a very restricted SSH server which only accepts a single key (ie. the one which is set up in `~/.ssh/authorized_keys`) and otherwise fails, its the easiest to set the specific key in your local `~/.ssh/config` file as follows:
Host very.secure.server IdentityFile ~/.ssh/id_dsa IdentitiesOnly true
The second entry, `IdentitiesOnly`, forces SSH to only use known identity files and not look for more available identities f.e. from a running `ssh_agent` instance (which are always tried in first instance as it seems).
Thanks for the tip, Tommy! This helps me a lot with github!