The problem is that XInput2 smooth scroll events are not being received by the guest.
To see this, listen for XInput2 events on the host and the guest by running
When scrolling, only the
ButtonPress and
ButtonRelease events generated by the host will be received by the guest. The smooth scroll
Motion events are never received. This is why scrolling vigorously allows some scrolling to work, since this is the only time that
ButtonPress and
ButtonRelease events are generated by the host.
To work around this issue, you can run VirtualBox with the QT_XCB_NO_XI2=1 environment variable set. This disables QT's usage of XInput2 and has it fall back to using vanilla XInput scrolling behavior. In other words, to work around this issue, from a terminal, you can run
This works because for vanilla XInput clients, the X server is required to emulate smooth scrolling using only
ButtonPress and
ButtonRelease events, which are the ones, at present, capable of being received by the VirtualBox guest.