User interface libraries contain controls for editing data. For example, text boxes, radio buttons and buttons.
They combine data visualization with user input, which seems convenient for developers.
“Rendering” presents content to the user. Input processing handles the feedback the user gives back to the controls.
Obviously, these are two completely different concepts:
I assume that the the combination of input processing and rendering in “controls” is seriously flawed and causes user interface development to be perceived as hard.
Input processing needs to be separated from control rendering because they differ in so many ways:
In time
Rendering needs to be available at any time. As soon the presentation model changes, it needs to be invoked.
User input needs to be processed at the time a control is focused.
On the surface
To update only parts of the screen, rendering should be partial; it may modify only parts on the screen at a time. Visually, it is expected to work in parallel.
User input processing happens in one screen area at a time.
Of performance
Rendering must be fast to keep the eye pleased. It may be parallelized to run on multiple cores.
Input handling may be processed much slower in a single thread.
In implementation complexity
Rendering is conceptually simple: structured data is rendered to a bitmap.
Input processing is more complex, it maintains focus states, may handle text input and processes different input devices.
Of relations
Rendering just reads the data model and writes to the screen bitmap.
Input handling reads and writes the data model, it even modifies what is temporarily rendered and how (the view model). Additionally, it listens to events and maintains a set of internal states.
Of states
Rendering does not need to maintain temporary states.
Input controls often implement large state machines.
Because of these huge differences, a number of essential conflicts are present in all frameworks I know:
- The complexity of event processing and calling back into the application binds all controls to the UI thread, and so they may not respond fast enough for rendering. Therefore, incremental rendering is often impossible, rendering can not be parallelized, output must be cached and a lot of memory is wasted.
- Input controls maintain a lot of states, even when they are invisible the waste memory.
- Virtualization (the selective instantiation / rendering of controls depending on the visible area) is hard, because event handling and focus processing is not separated.
- Separation of the domain model and the view model gets hard, because the roles of the controls are not clearly defined.
To avoid these conflicts, input processing and rendering should be separated:
My suggestion to user interface framework developer:
Introduce the concept of sensor fields that have no rendered representation and instantiate the required input (control / logic) at the time they get used.
My suggestion for the user interface developer:
Render output-only “components” and overlay edit controls at times when the user actually requires them. For high performance applications, use a fast rendering library instead and overlay edit controls from your favorite UI framework when required.
yours
armin