Apple computer files another patent application for touch screen interaction

YAAP! (Yet another Apple Patent app)

Apple has applied for another patent for a touch screen user interface- this time it’s how the virtual keyboard would appear on screen.

It’s an interesting problem: how do you get a virtual keyboard to pop out on the screen without making the rest of the user interface entirely useless (i.e. how do you make sure the large (judging from the illustrations) virtual keyboard doesn’t obscure UI elements like buttons, tabs and text boxes- or more importantly content like movies and webpages- the problem is figuring out what screen content is important and then choosing the best way to display the keyboard so those elements that are significant are no obscured. Depending on the circumstance Apple suggests a keyboard could appear in a pop-up-box, sliding from the bottom and either obscuring or causing the rest of the display to squash up. In another patent we even saw little keyboards pop up in an iTunes pane to help select a song. What ever the case the ‘intelligence’ of the system in decided what elements of a screen are important will be the key to the usability of these features.

Will this flow of patent applications never cease? Does a million patent applications make a product or are they merely trying to get some licensing returns from end-of-line projects?

The Nitty Gritty

The full patent application is here: Virtual input device placement on a touch screen user interface
Filed: September 16, 2005
Inventors (all also on other ‘touch sensitive input devices’ patent apps):
Imran Chaudhri
Greg Christie
Bas Ording

This patent deals with how the OS would decide how to make a keyboard appear on screen- it would be very context dependent. The OS would try and not obscure ‘significant’ portions of the display by making the virtual keyboard slide up (like the Dashboard more widgets button), or it would simply pop on top or even squeeze the rest of the display…

In response to a virtual input device initiation event, initial characteristics of the virtual input device display are determined. Based on characteristics of the application display and the characteristics of the virtual input device display, initial characteristics of a composite display image are determined including the application display and the virtual input device display. The composite image is caused to be displayed on the touch screen.

I think this boils down to if there is a small window at the top of the screen that you are trying to type in then the virtual keyboard may obscure the bottom half of the screen- conversely if the typing is in a full screen application (iPhoto) then the screen may be squeezed up so all of the screen is still visible.

It (of course) gets more complicated than this with discussion of what a ‘significant’ portion of the display is when the keyboard is being used: “That is, an input field of an application display is typically determined to be significant because it may represent the portion of the application with which the user is interacting via the virtual input device. However, other portions of the application display may be determined to be significant. For example, a portion of the application display that is directly affected by input via the virtual input device may be determined to be significant. In some examples, there may not even be an input field of the application display. ”

A cute idea is just by laying your hands on the screen in a typing position a keyboard pops up: As yet another example, the keyboard initiation event may be triggered by the user putting her fingers on the touch screen (for example, a multipoint touch screen) in a “typing” position.”

Other patent applications mentioned in this filing: “ACTIVATING VIRTUAL KEYS OF A TOUCH-SCREEN VIRTUAL KEYBOARD” and “OPERATION OF A COMPUTER WITH TOUCH SCREEN INTERFACE” though these have not been released.