Pen interfaces are computer input devices (or set of devices) that allow users to interact with computer systems. Usually, a pen interface comprises of two separate parts: a stylus (or pen), and a ‘tablet’. The stylus in such interfaces is the digital equivalent of a pen, or a pencil, and the tablet is similarly equivalent to a writing or drawing surface. Common marketing terms for pen interfaces generally drop the word ‘stylus’ or ‘pen’, and refer to these interfaces as ‘graphic tablets’ or simply ‘tablet’ interfaces.
Pen interfaces are most commonly used by graphic artists and designers who use drawing programs like Paint, Photoshop, and Illustrator to create digital art. Not uncommonly, pen interfaces are also used by computer users as replacements for traditional mice and keyboards. In addition to computers, where their usage is most prominent, pen interfaces are also found in many mobile devices – although that is more an exception than a norm. Almost all mobile devices today have touch screens, but most mobile users do not require a stylus. Typical uses of a mobile device like a smartphone require only the use of fingers, and no more precision is called for unless the user wishes to use a drawing app on their device. Since the use of pen interfaces (or styluses to be precise) in mobile devices is uncommon, I will focus on their use as interfaces for traditional personal computers (desktop or handheld).
A Brief History of Pen Interfaces
According to Wikipedia, the first attempt at making an electronic device for handwriting was the ‘Telautograph’ in 1888 (see figure) which was an elaborate setup consisting of a sending device and a receiving device. The sender would send electrical impulses corresponding to ‘pen’ gestures, and the receiver would decode those impulses and ‘draw’ out
the received patterns using a pen attached to a servo-motor based setup. While similar devices were invented at regular intervals after the Telautograph, the first such device that could be used to input data to a computer dates back to 1956. In 1985, a Massachusetts based company called Pencept came up with a general-purpose input device for MS-DOS based computers. There have since been countless improvisations on the pen interface, but the basic design comprising of a stylus and a tablet has remained the same. Some recent products like the Wacom Bamboo series include touch sensitivity in their tablets which allows users to work with both styluses and fingers — the fingers often being used to ‘pinch-zoom’ or rotate the digital canvases or artboards they are working on on-screen.
Types of Pen Interfaces
As described earlier, the pen interface has evolved into a few categories of interfaces, and while most categories have coexisted at various times, interfaces which have touch-enabled tablets have come to dominate the market. Here are the major categories of pen interfaces that are available today:
- stylus and tablet, commonly marketed as “graphic tablets”,
- stylus and touch enabled tablet, commonly marketed as “pen and touch tablets”, and
- stylus and touch enabled display, commonly marketed as just “tablet computers” (for example, the iPad)
I will limit the scope of this paper to only type 1, as tablet computers are (as the name implies) not just interfaces but tremendously capable computers and therefore belong to a different product category altogether. Also, type 2 interfaces have been around for only a few years and are still finding their way to potential users.
In the next few sections, I will present an analysis on how pen interfaces are typically used, and how we can weigh their effectiveness. Effectiveness is not proposed in this paper as a quantitative metric, but as a qualitative variable to gauge how usable pen interfaces are as a whole. But before that let us briefly define what our definition of usability specific to pen interfaces looks like.
Usability Considerations for Pen Interfaces
Usability is defined broadly as “the ease of use and learnability of a human-made object”. Therefore, simply put, the usability of pen interfaces can be defined as the ease with which humans can (learn to) use a pen interface for their needs as computer users — for creating digital art/design, and for typing and pointing on a screen. I will weigh usability of pen interfaces on the following factors:
- setup time
- the time taken to learn and control the interface
- the tactile feedback offered by the interface (important for almost all applications other than as a pointing device)
- software interface (this factor though not strictly related to the interface hardware, is anyway very important as I will explain in this paper).
Guided by the above considerations, let us now consider typical uses of pen interfaces for digital art/design and typing/pointing.
Usage of Pen Interfaces for Digital Art/Design
In order to weigh the usability of pen interfaces for digital art, let us consider a comparison between the tools an artist or designer uses for his/her creations in the paper-and-ink or canvas-and-paint world, vs what he/she uses in the digital world.
The expressiveness found in art is arguably because of the artist’s ability to materialize his/her creativity unhindered onto a piece of paper or canvas. This unhindered connection between the artist’s brain and the canvas is facilitated by an artist’s most basic tools — paints, brushes, and canvas/paper. This expression is made possible because the outcomes of an artist’s strokes are in essence the records of what brush size was used, how much pressure was applied, how quickly was the stroke made, or how wide or ‘free-handed’ a stroke was, and what the texture of the paper/canvas was among other factors. Let us now examine how pen interfaces try to facilitate the same expressiveness digitally.
Most pen interfaces come with a single stylus, and therefore there is no possibility of changing the size of the stylus — more specifically the part of the stylus that touches the tablet, called the tip. In most cases the tip is about 1mm thick, far smaller than what might feel ‘right’ to an artist trying to use it for the first time. Since neither the form factor of the stylus-tablet combination, nor the size of the tip matches with real paintbrushes, it is expected that a new artist would need to spend some time learning and adapting to the interface. Moreover, a pen interface needs a basic ‘setup’ routine, which commonly refers to installation of software ‘drivers’ specific to the particular make of the interface into the computer. Therefore, there are additional steps (i.e. more time & effort) involved before a stylus can be put to use.
Texture of painting surface
The surface of the tablet is the next most important thing in a pen interface. The most important thing to consider is that it is considerably harder than paper or canvas. That, in addition to the hardness of the tip of the stylus, also steepens the learning curve for a user. The very idea of applying more (or less) pressure to achieve the desired artistic effect seems to get grounded on first exposure to such pen interfaces. In addition to the hardness, the smoothness of the surface makes it difficult to tame the strokes, and therefore it is also a factor that requires learning effort from a first time user. Therefore, in addition to adjustments with respect to the nature of surfaces, there is extra effort required to adapt to pen interfaces.
As mentioned in the previous point, the tablet surface as well as the stylus tip are hard, and therefore a user does not feel the area of contact between the brush and the surface grow larger with increased pressure (or vice-versa). While software drivers and programs do visually create ‘broader’ or ‘thicker’ strokes when the stylus is pressed harder onto the tablet surface, there is no physical manifestation of this broadening. Therefore, in a strictly intuitive sense, usability suffers at least until a user’s learning phase is complete. Usability also suffers when the user does press the stylus harder on to the tablet as the sound or perceived ‘feel’ of the hard tip of the stylus rubbing against the surface of the tablet can be unpleasant to some users.
Speed and sweep size of stroke
When a real brush dipped in paint is used to make a stroke on canvas, less and less paint remains on the brush tip as it continues to remain in contact with the canvas. This is another facet of expression that pen interfaces try to mimic — but again, it is only made possible by the software running on the computer. The tablet surface does not provide any visual feedback, or tactile feedback of a drying brush rubbing against the canvas.
Throughout the listing above, we see a significant learning curve which is not avoidable for any new hardware interface — especially for those interfaces for which users have already employed a ‘mental model’ for long periods of time. Let us now consider pen interfaces as replacements for keyboards and mice.
Pen Interfaces as Text Input and Pointing Devices
Although not common, text input and pointing form another category of use of pen interfaces. While keyboards and mice are themselves as abstract as pen interfaces when it comes to the act of ‘writing’ or pointing at stuff, people have become used to them. This again implies a steep learning curve if a computer user wishes to adapt his or her mental model of interacting with a computer to using pen interfaces.
Barring the lack of direct visual feedback, writing or inputting text with a pen interface may not prove to be very difficult for most people (as the stylus very much looks and feels like a real pen). But pointing with pen interfaces does involve some leaps in learning. This is so because mice (generally) function properly only when they are moved on a two dimensional hard surface. And since most computer users have a hard surface to work on, mice are very easy to control especially when the visual feedback is coming from a separate entity (the monitor screen). Also mice stay where you leave them on the desk (so does the pointer on the screen). But when you put the stylus of a pen interface away from the tablet, it is not easy to restore the pointer to the same place as you left it on the screen. This is one of the main problems experienced by users when using pen interfaces as pointing devices. I refer to this as the ‘first touch’ problem, and will describe how interface makers have tried to mitigate this problem further ahead in the upcoming sections.
When used for text input, tablet interfaces present a similar problem. The hardness of the surface of the tablet and stylus tip reduces the friction needed to control the strokes for calligraphy or normal text input. Normal text input is generally achieved through text recognition software that mediate the interaction between the pen interface/user and a word processing program. These text recognition programs read what the user squiggles on the tablet with his/her stylus, and tries to convert those squiggles into text characters that the word processor can understand.
Pen Interfaces: More Usability Considerations
Apart from the discussion above, let us consider a few more factors that impact the usability of pen interfaces. I specifically consider the direction of these impacts (positive or negative), and the effect of learning on their usability.
As I mentioned in the section on use of pen interfaces for digital art/design, there is a significant amount of effort involved to setup the interfaces. Typical setup procedures include installation of drivers, and configuration of the interface. Elaborating more on configuration, there are two general options users can choose from when it comes to operating pen interfaces: a ‘mouse’ mode, and a ‘tablet’ mode. As the name tells, in the mouse mode the interface behaves just like a mouse as the computer sees it, that is the position of the pointer on the screen is in no strict sense related to the position of the stylus tip on the tablet. In mouse mode, a user can leave the mouse pointer at one point on the screen, put the pen down, pick it up again, press it against the tablet at a different place and yet resume at the same point on the screen as previously.
In contrast, in the ‘tablet’ mode, the tablet becomes a virtual, scaled-down map of the screen. So the position of the a stylus tip on the tablet is in lock-step with the position of the pointer on the screen. But even then, the ‘first touch’ issue I mentioned earlier is a challenge for most users. Even after habitual use of a stylus, a user can find it awkward to put the stylus down at the ‘right’ place on the tablet.
The thickness of a stylus can be a factor for people who prefer either thinner or thicker pens/pencils in real life. The problem solely arises because of the fact that a user cannot just buy another stylus that is comfortable for a him/her to grip and use. This can be a limitation for many artistic uses like calligraphy, where the thickness of the stylus can interfere with the economy of motion of the fingers or wrist of the user.
Tablet orientation/hand-eye coordination
Unlike real life pen and paper, there is no visual feedback about what the actual stroke looks like from the tablet itself. One has to look at the screen and draw on the tablet placed on a table or in a user’s lap. This ‘disconnected’ action-feedback interplay causes the problem of stroke alignment. For example, if a user wants to draw a horizontal line in his/her artwork using the pen interface, and he/she begins stroking the stylus to draw it, the line would not turn out to be horizontal on the screen unless it was also horizontally drawn on the tablet. In simpler words, if one were to move the tablet even slightly, the user would have to adjust his/her stroking direction accordingly to achieve the desired effect. This is one of the most important drawbacks of pen interfaces as far as usability is concerned.
One factor that often hampers the usability of many pen interfaces is USB connectivity. The fact that a USB cable tethers the tablet, and hence the user, to the computer obviously restricts the user’s freedom of movement. In most cases the user is forced to work on a desk and cannot keep his/her tablet in the lap when the desire to do so strikes. In any case, the short length (less than a metre) of the cable forces desktop users to sit really close to the computer, and in case the monitor is large (say more than 20”) this can cause great discomfort especially when the tablet itself is down on the desk. The use of cables also adds to the ‘setup’ time mentioned above, infact if the user stores his/her tablet separate from the computer, this setup time (however small) is present at all times the user wants to start using the pen interface.
But many manufacturers are now doing away with cable interfaces, and providing wireless connectivity (either via Bluetooth, or WiFi) even in budget devices. This is a welcome step, and it truly liberates the user who can now choose to use their pen interface from any vantage point that suits their needs or circumstances. Therefore, it can be said that at least the implementation of wireless connectivity serves to improve the usability of pen interfaces as far as the user’s freedom of movement is concerned.
Let us now consider the size of the tablet and the role it plays in usability. Since tablets are meant to be lightweight and portable, their active surface area is usually no larger than 8”x11”.
By active area is meant that portion of the surface of the tablet which is sensitive to the touch of the stylus. This area is usually demarcated with visible etch marks on the tablet to guide the users to keep their stylus within that area. As I discussed earlier in the paper, most tablets provide two modes of operation: the mouse mode, and the tablet mode. In the mouse mode, the active area is just that — the area over which the stylus will be sensed. In the tablet mode, however, each corner of the active area is mapped to the respective corners of the display, i.e. the active area represents a scaled down map of the display itself. Each point on the active area always corresponds to the same area on the display.
While the tablet mode is a very important improvement over the mouse mode as far as pointing or artistic use is concerned, two problems still remain. Firstly, the ‘first touch’ issue still does not get addressed. Secondly, since the active area of the tablet is in almost all cases smaller than the display, any stroke of the stylus on the active area corresponds to a larger stroke on the screen (assuming that the paint software’s digital canvas is scaled to fit the screen). This may not be an insurmountable problem, but it does take some learning to get used to.
While it is important to study the usability of a device when a new user is exposed to it, it is equally important to evaluate its usability once the initial learning phase is over. At the same time it can not be overemphasized that if a device continues to present challenges to a user even after the user has learnt to operate it, then the device needs a redesign.
Having used many graphic tablets over the last few years, my personal opinion is that most modern tablets have high usability once I have learnt to ‘control’ the basics — getting used to the active area, and getting comfortable with tablet orientation primarily. As mentioned at the beginning of the paper pen interfaces with ‘touchable’ and clickable tablet surfaces have become fairly common. These interfaces allow users to map shortcuts to often-used features of their favorite design/paint software programs to gestures and clickable buttons on the tablets and styluses.
Taking a broader perspective, pen interfaces have certainly come a long way, and there is an entire world of digital artists and designers who use these interfaces almost exclusively. That pen interfaces provide features like pressure sensitivity has helped many users overcome or ignore some of the other usability problems like a wired connection, or the ‘first touch’ effect. The ‘first touch’ effect, and alignment issues are dimensions where a lot more design work needs to be done to improve usability. The biggest challenge, however, is the amount of hand-eye coordination that these interfaces demand, with the tablet lying typically down on a desk, and the user viewing the resulting strokes on a screen in front (at head level, say). While modern tablet computers combine the display and active areas in tablets (the most ubiquitous example being the iPad) and thereby mitigate the demanding nature hand-eye coordination, the pressure sensitivity afforded by these tablets is limited in comparison to traditional pen interfaces.
The future definitely presents fertile grounds for pen interface innovators since there is a lot of ground to cover before these interfaces can truly remove any barriers between the artists’ brains and expression. I have earmarked some areas for improvement in this white paper, and I have also tried to provide a set of dimensions along which innovators can focus on.