Gestures on the desktop

This is a rambling discussion about programming an app to use gestures on a desktop computer.  It says that your instances of the Command class are more complicated when gestures can drive the command instances.

Apple HIG suggests that a ‘good’ desktop app should support gestures.

It wasn’t always this way; even today few desktop computers have multi-touch input devices.  And multi-touch is the important word.  To be clear, this Apple recommendation is about supporting those users who have Apple Trackpads and Magic Mouse ( which support multi-touch.)  I don’t know how rare those are these days, whether Apple even ships new Macs with Magic Mouse standard.

To implement this recommendation requires that you look at certain commands (actions) in a different way:

  • they might need to be cancelable
  • they might need to be combined

Consider as an example using pinch and swipe gestures to control a view.

In my experience, both these gestures are active at the same time.  That is, when a user places two fingers, both gestures start.  Then a user can pinch their fingers, or pan their fingers, or both simulateneously, and the system delivers both gestures (as  updates).  Finally, the user lifts both fingers and the system delivers both gestures ascompleted.  (See my post about testing gestures, to see how I determined that the above is more or less accurate.)  Or, if the user places a third finger down, the system delivers both gestures as canceled.

Combined

In the usual app, there is a command to pan the view and a separate command to zoom the view.  With gestures, those commands should be combined.  Here I am using command in the Command pattern sense: some unit of action (which is often undoable, except that view change commands are not usually undoable.)

It is no big deal, after all both commands are simply changes to a viewing transform, and both zoom and pan can be combined in one transforming command.  I’m just saying it is more general than separate commands, and many apps are not written to be general, but specific for separate view change commands.

Cancelable

In the usual app, a command to change the view is not cancelable.  The action of the command is usually fed back (ghosted) as the user is performing the command.  For example, as the user moves a scroll bar, the view (what the user sees) changes in real time (not after the user releases the scroll bar gadget.)

Using gestures, which might be canceled, you must be prepared to cancel the view change.  This suggests a transactional command: one that seems to happen in real time but that can be rolled back.  (in database parlance?: the command gives a consistent view to all users, and can be rolled back.)

For example, if a user puts two fingers down, moves them, and then puts another finger down, that might cancel the pinch and swipe gestures.  If you were already ghosting the view change, your app now needs to roll it back. (And maybe handle the new three finger gesture?)

It is more complicated if the command is also undoable: then you need a transactional, undoable command.

The bottom line is: to support gestures adequately requires various flavors of command classes: combined, transactional, and undoable.

Advertisements

One thought on “Gestures on the desktop

  1. Pingback: Deploying PyQt Python Qt apps cross platform using pyqtdeploy | plashless

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s