Trouble with Cmd-H and Cmd-Q in VB 3.1.2

[jsdza]

Firstly, I wish to express my sincere thanks and gratitude to the developers of Virtual Box. This is an awesome product.

That having been said, I would like to address a keyboard problem that has arisen for me in release 3.1.2, which I believe stems from a fix for bug #5684 http://www.virtualbox.org/ticket/5684

The short story is that I am no longer able to type the Alt+H or Alt+Q keystrokes into my Windows VM; those keystrokes are effectively dead for me. Even when the keyboard is captured, those keystrokes still do not reach the VM.
My keyboard mappings (about which more below) are such that the Cmd key is used as the Alt key in my Windows VM.

Installing the 3.1.2 Guest Additions in the VM does not make a difference.

The release note for VB 3.1.2 states "Mac OS X hosts: re-enabled CMD+Key combinations, even if the Host-Key isn't CMD" which implies that there is now something "magic" about the CMD key which prevents it from being cleanly remapped within a guest. Perhaps I should rephrase that: it prevents the Host key from being cleanly changed to something other than CMD.

I'm guessing (haven't looked at the code yet) that the bug fix has hard-coded keystrokes for Cmd-H and Cmd-Q and looks for those specifically; plus it does so even when the keyboard has been captured.

IMHO this fix, which is essentially about making OS X menu shortcut keys work, should
(a) be disabled when keyboard capture is active, and
(b) read the OS X menu shortcut keys from the menu itself
rather than being hard coded as Cmd-H and Cmd-Q,
thereby honouring changes made via NSUserKeyEquivalents

The rest of this note (a bit of a ramble, sorry about that) explains my position in more detail.

I want a keyboard layout that places the PC Alt key where my fingers expect it to be, namely next to the space bar, which would be the CMD key on my Mac. By default, this is the Windows key within a VM.

In my Windows VM, I use a registry setting (HKLM\System\CCS\Control\KeyboardLayout) to swap the Alt (Alt/Option on Mac KB) and Windows (Cmd on Mac KB) keys. IIRC the default Host key is Left Cmd; I set it in VB preferences to the Left Option key, so that Alt-keypresses in the VM are not executed as VB Host Key actions.

So far so good, except for one final problem, namely the OS X menu shortcut keys in the Virtual Box VM menu, specifically Cmd-H and Cmd-Q. These would invoke the VM Hide and Quit menu items when what I really wanted was to key in Alt+H or Alt+Q to the VM itself. One way around that is to capture the keyboard (tap Host key or use Auto-Capture) but even better, I found I could change the OS X menu shortcut keys to something I was less likely to type under normal circumstances in the VM (by using 'defaults write org.virtualbox.app.VirtualBoxVM NSUserKeyEquivalents ...')

This has worked well up to now: I could have the PC VM's Alt key where I expected it to be, a Cmd-H keystroke in the VM would send an Alt+H to the VM instead of hiding the VM window, and Cmd-Q would send an Alt+Q to the VM instead of raising the VB VM Quit dialogue.

Unfortunately it's broken in VB 3.1.2: Cmd-H and Cmd-Q do nothing at all for me. The VM doesn't see these keystrokes because the Virtual Box VM sends them straight up to its Carbon/Cocoa framework, and they don't do anything there because they are not menu shortcuts (thanks to my NSUserKeyEquivalents changes).

[jsdza]

I've now had a look at the code, and I think the following minor changes should suffice:

In VBox/Frontends/VirtualBox/src/VBoxConsoleView.cpp, at line 142 change:

if (VBoxCocoaApplication_isApplicationCommand(pvCocoaEvent)) 
 	return false;

to:

// Perform app menu shortcuts unless keyboard is captured
    if ( ( ! view->mKbdCaptured ) && VBoxCocoaApplication_isApplicationCommand(pvCocoaEvent) ) 
 	return false;

In VBox/Frontends/VirtualBox/src/darwin/VBoxCocoaApplication.m, rewrite VBoxCocoaApplication_isApplicationCommand thus:

bool VBoxCocoaApplication_isApplicationCommand(const void *pvEvent)
{
    NSEvent     *pEvent = (NSEvent *)pvEvent;
    NSEventType  eEvtType = [pEvent type];

    if ( (eEvtType == NSKeyDown || eEvtType == NSKeyUp)
        && [[NSApp mainMenu] performKeyEquivalent:pEvent] )
        return true;     // Global application menu shortcut command key handled
    else
        return false;    // Not a global application command key
}

[JamesMcLean]

I would like to second this request.

From my perspective, the "fix" in http://www.virtualbox.org/ticket/5684 was actually a "break". It also seems to me that those calling for 5684 want to "have their cake and eat it too." If one is going to move the host key away from Cmd, then you can't expect Cmd-Q to do something at the host level.

If you move the host key to something other than Cmd, that is presumably because you want to use the Cmd key as something else in the VM. (I want my Cmd to operate as Ctl in the Windows VM, for instance, so that common shortcuts involve the same key presses for both VM and host.) That being the case, it is highly unlikely that you won't want Cmd-Q and Cmd-H to go to the guest.

The solution proposed by jsdza is OK, but not very user friendly, as it requires changing menu key bindings through the command line with NSUserKeyEquivalents. (Normally, one could use System Preferences > Keyboard > Keyboard Shortcuts, but that doesn't seem to be an option in this case since the Virtualbox VM application can't be called up there.)

I would call for the default behavior to be that all menu shortcuts for the Virtualbox VM use the host key. An option could be provided to satisfy the requests in 5684, but I can't see a justification for making that the default, let alone the only functionality.

[jsdza]

Thank you James! I was beginning to despair that I was the only person who cared about this.

> The solution proposed by jsdza is OK, but not very user friendly, as it requires changing menu key
> bindings through the command line with NSUserKeyEquivalents.
> Normally, one could use System Preferences > Keyboard > Keyboard Shortcuts, but that doesn't seem
> to be an option in this case since the Virtualbox VM application can't be called up there.)

Agreed. That irked me too, and it was only after much trial and error that I stumbled upon
NSUserKeyEquivalents, which of course is anything but obvious. I don't even regard it as a solution
to the 5684 problem (and I agree, the "fix" was indeed a "break"), but it solved what for me was
a major problem for ages, namely that when I typed Cmd-H (meaning Alt+H in the VM) the effect
was to hide the entire VM window.

> I would call for the default behavior to be that all menu shortcuts for the Virtualbox VM use the host key.
I think that is an excellent suggestion

[MarkCranness]

Thank you James! I was beginning to despair that I was the only person who cared about this.

You might have more feedback if you post this to the developers mailing list:
http://www.virtualbox.org/mailman/listinfo/vbox-dev

[jsdza]

You might have more feedback if you post this to the developers mailing list:
http://www.virtualbox.org/mailman/listinfo/vbox-dev

Thanks Mark. I did that at the same time as I posted here:
http://vbox.innotek.de/pipermail/vbox-d ... 02225.html

Alas, no response to that.

[DanShockley]

(Normally, one could use System Preferences > Keyboard > Keyboard Shortcuts, but that doesn't seem to be an option in this case since the Virtualbox VM application can't be called up there.)

I just started really using VirtualBox, and it is definitely frustrating (7 years after this thread!) that the guest OS still won't capture command-Q and command-H.

However, you CAN use the System Preferences interface to modify keyboard shortcuts: just scroll down the app list to "Other…", command-click in the Dock on the running VirtualBox VM to open it's location in the Finder (it is at /Applications/VirtualBox.app/Contents/Resources/VirtualBox VM.app), and then drag that into the app-selector window.