Modern desktop environments still predominantly use the keyboard/mouse interaction model, but were extended to deal with various types of trackpads, drawing tablets, touch screens and alike, which brings them nearer to interaction concepts of mobile devices.
The "translation" of the pointer device concepts (or the aforementioned additional input methods) of the remote client (which the user is interacting with on their device) to the virtual environment has some implications. Today there are multiple types of devices around: traditional mouse, touch pads in various formats and touch screens often with multi finger / touch gestures. The same applies for the keyboards: Traditional desktops work with a PC102 or reduced standardized laptop keyboards with a certain layout and minimal key configuration (number and existence of all relevant function keys and modifiers). This is typically rather different in the onscreen overlay keyboards shown on the individual devices (some environments like IOS explicitly prohibit certain keycodes). It would be helpful if specialized keyboard layouts for IOS / Android could be configured, chosen to mimic at least the some halfway compact laptop keyboard.
Several issues:
- Actually, the mouse can be shown or not (via the little overlay icon upper right corner). The form of interaction is not entirely clear, e.g. if not shown, activated icons on the desktop are displayed but "double click" doesn't work.
- If the pointer is shown it is tedious that the mouse needs to be dragged from its given position and cannot just "jump" or is dragged to the last position where e.g. a touch happened.
Here, it should be discussed which modes of interaction are desirable, configurable:
- Simple touchpad mode - jump the cursor to the actual position when a single touch happened
- Use / define (multi)touch gestures to use in interaction
- Differentiate the various hardware use modes of the device (full screen touchpad - meaning an external display is attached where the remote screen is shown vs. single shared screen with virtual keyboard and mouse)
- Generally, mouse hardware and/or behavior configurable (in the app)
- Touch pad or mouse like behavior
- Possibility to change the mouse wheel / touchpad behaviour (vertical scroll direction as in Apple vs. Windows)
Modern desktop environments still predominantly use the keyboard/mouse interaction model, but were extended to deal with various types of trackpads, drawing tablets, touch screens and alike, which brings them nearer to interaction concepts of mobile devices.
The "translation" of the pointer device concepts (or the aforementioned additional input methods) of the remote client (which the user is interacting with on their device) to the virtual environment has some implications. Today there are multiple types of devices around: traditional mouse, touch pads in various formats and touch screens often with multi finger / touch gestures. The same applies for the keyboards: Traditional desktops work with a PC102 or reduced standardized laptop keyboards with a certain layout and minimal key configuration (number and existence of all relevant function keys and modifiers). This is typically rather different in the onscreen overlay keyboards shown on the individual devices (some environments like IOS explicitly prohibit certain keycodes). It would be helpful if specialized keyboard layouts for IOS / Android could be configured, chosen to mimic at least the some halfway compact laptop keyboard.
Several issues:
Here, it should be discussed which modes of interaction are desirable, configurable: