This almost worked, except that it sometimes left hovering on after leaving
an AoI, which then caused a subsequent drag to be handled as hover_click.
This caused the history list sometimes to be closed when trying to drag.
We had input state scattered all over the place. This cleans things up.
We also merge the functions of left and (previously) middle button.
This breaks history panning.