Windows 10 Pen Woes, Part 2

May 25, 2018

Last time I wrote about the Windows 10 update that effectively crippled the usefulness of pens for drawing. For better or worse, Microsoft provided a registry hack that kinda-sorta-but-not-quite reverted pen functionality to what it was prior to the update. While it was good to be able to draw with the pen again without holding down any additional buttons after applying the registry hack, it was the kinda-sorta nature of the “fix” that still tripped me up. Having a Windows programming background I set about the task of seeing if I could get the pen working much more appropriately.

Warning: This blog post delves into certain Windows programming techniques so may not be appropriate for all audiences.

The Problem

Windows-based pens typically have what amounts to three buttons. For an official Windows Surface pen these are the tip of the pen, a button on the side of the pen barrel, and an eraser button on the backend of the pen. For my ASUS Pen the eraser button is actually a second barrel mounted button, which means that the pen doesn’t need to be turned over to erase.

The basic idea with a pen is that the tip should operate similarly to the left button on a mouse and the barrel mounted button on the pen should operate similarly to the right button on a mouse. The eraser button is a special button that needs to be detected by “pen aware” applications. All well and good until you consider how the Windows 10 update, even with the registry hack in place, subtly but critically changed the behavior of the pen.

A brief bit of background for non-programmers that have wandered into these murky waters. Windows operating systems work by sending messages to applications to inform them when important events have happened. For example, when the left mouse button is pressed down Windows will send a left button down message to the application the mouse was positioned over. When the button is released, Windows then sends a left button up message. Similar messages are generated for other mouse buttons and for keyboard keys.

Pen support came along sometime after mouse support. This is important because an application can only do stuff with messages that it’s been programmed to listen for. So if a pen only sent pen messages then any application that wasn’t programmed to “listen” for pen messages would just ignore them. To make pens more generally usefully when pen input happens Windows sends both pen and mouse messages. Modern applications can then just listen for the pen messages while older applications get the pseudo-mouse messages that, in theory, allow them to treat pens as if they were mouses. Or mice. Whatever.

Now, prior to the Windows 10 update the pen behaved like so:

  • When the pen was touched to the surface without the barrel button being held a left mouse button down message would be sent. A corresponding left mouse button up message would be sent when the pen was lifted from the surface.
  • Similarly, when the pen was touched to the surface with the barrel button held a right mouse button down message would be sent. A corresponding right mouse button up message would be sent when the pen was lifted from the surface.
  • In either case, if the pen was moved across the surface between being touched to the surface and being released a mouse move message would be sent to inform the application the position of the mouse had changed.

All well and good as this exactly matches how a mouse itself behaves. However, after the Windows 10 update occurred the pen changes to behave like so:

  • When the pen is touched to the surface without the barrel button being held down a pen down message is generated but the left mouse button down message is not generated right away. Instead the pen must be moved or released first. In the case of being moved, the left mouse button message is sent followed by any number of mouse move events. If the pen is removed from the surface right away then both a left mouse button down and a left mouse button up event are sent (and a pen up message is also sent).
  • When the pen is touched with the barrel button being held down is where things get interesting. Again Windows does not send a right mouse button down event immediately. Instead it waits to see if the pen moves or is released. If the pen is released then both a right mouse button down message and right mouse button up message are generated. BUT if the pen is moved before being released then Windows generates a left mouse button down message, a number of mouse move messages, and finally a left mouse button up message when the pen is removed from the surface.

Although the details of the pen being touched to the surface without the barrel button being held are different in terms of when messages are generated, they still generate the appropriate messages to be treated as though the left mouse button were being used.

However, the behavior of the pen when the barrel button is pressed is remarkably and destructively different from how a right mouse button works. Especially so in the painting software I use where one painting color can be assigned to the left mouse button and a second color can be assigned to the right mouse button. In this case, as long as I only wanted to draw a single dot by “right tapping” with the pen (holding the barrel button down, touching the surface, and then immediately lifting the pen) everything was fine. But if I actually wanted to draw, say, a line with the secondary color by holding the barrel button down and moving the pen across the surface I was out of luck. Why? Because Windows, after the update, always interprets pen interactions that involve any movement of the pen as left mouse button interactions, which would cause the color assigned to the left mouse button, not the right, to be used when drawing that line.

This was a significant impact on my drawing productivity. Most of what I draw is line art based, so I typically have my line color assigned to the left mouse button and the background canvas color assigned to the right mouse button. This allows me to quickly draw my desired line with the line color but then to easily correct any minor wobbles or mistakes by simply drawing over them with the background canvas color. As the song goes, you don’t know what you’ve got ’til it’s gone. This was definitely the case with being able to properly use the barrel button to simulate the right button of a mouse.

The Fix

So while the simulated left mouse button interactions for the pen work for non-pen dedicated software, the simulated right mouse button interactions do not. This was the starting point for me for making the pen work in Windows the way it did prior to the update (a.k.a. the correct way).

As I mentioned, in order to more or less support both dedicated pen applications and applications not specifically designed for pens, Windows sends both pen messages and mouse messages. And while the mouse messages wait until the pen is either moved or lifted from the surface, the pen messages do not, meaning that as soon as the pen touches the surface it is possible not only to detect that occurrence but to check to see if the barrel button is pressed as well.

In order to correctly interpret the messages Windows sends, it is necessary to put into place, from the programming side of things, two message hook routines. Message hooks are routines that receive different types of Windows message before those messages are passed onto specific applications. The general idea is to catch and interpret the appropriate pointer messages and convert them into the matching mouse messages at the appropriate times. Kind of like updated Windows tries to do but fails at.

The message hook that is needed is a WH_GETMESSAGE hook. As with all hooks this is installed by calling the SetWindowsHookEx routine. An important caveat here is that in order for this to work globally throughout Windows the hook routine itself must be in a DLL, not within an application.

This WH_GETMESSAGE hook must listen for two messages in particular as follows:

WM_POINTERDOWN

This message is received when a button on any pointer is pressed. A pointer could be a mouse, pen, or finger.

When this message is received the LPARAM that accompanies it is actually a pointer to a MSG structure, which contains the message specific information that will be passed onto the application.

Windows provides several macros for interpreting information from this structure for WM_POINTERDOWN messages. In particular the get_pointerid_wparam macro can be called with MSG.wparam to get the unique pointer ID for the pointer that generated the message. This can then be passed to GetPointerType to determine if the specific type of pointer is a pen or not.

If the pointer is a pen then the MSG structure can be checked to see more specifically if it is a pen with the barrel button pressed. This is done by calling several additional macros:

  • is_pointer_inrange_wparam – makes sure the pen is in range of the surface
  • is_pointer_incontact_wparam – makes sure the pen is touching the surface
  • is_pointer_secondbutton_wparam – makes sure the barrel button is pressed

If all of these conditions are true then a simple call to mouse_event(MOUSEVENTF_RIGHTDOWN, 0, 0, 0, 0) can be used to simulate the right mouse button being pressed down.

At the same time, it’s important to store in a global variable in the DLL that the right mouse button is now considered down. The reason for this will become apparent shortly.

WM_POINTERUP

For a fully robust solution, it would be best to store the pointer ID from the WM_POINTERDOWN message and ensure that when the pointer up message is encountered that the IDs match. However, in my case I just went for the simple expedient of treating any pointer up as the matching up to the down I used to generate the start of the right mouse button being down.

Here, if the global variable tracking if the right mouse button is down is set to TRUE, you would then call mouse_event(MOUSEEVENTF_RIGHTUP, 0, 0, 0, 0) to indicate the right mouse button has been released. You would also clear the global variable so the DLL no longer considers the mouse button as down.

The above work within a DLL is sufficient to detect when the pen touches and is removed from the surface with the barrel button pressed in order to interpret those messages into the correct right mouse button down and up messages. However, there are a couple of additional messages that must be dealt with via a second message hook, which must appear not in a DLL but as part of an actual application with a message queue attached to it.

The second hook to be installed is WH_MOUSE_LL hook for dealing with low level mouse messages. Remember that it’s not just our WM_POINTERDOWN and WM_POINTERUP hook that’s interpreting messages. Windows is doing it too. So while our first hook is converting a barrel button press, move, and release as right mouse button interactions, Windows itself treats those as left mouse button interactions. In order to prevent those incorrect left mouse button interactions from causing problems we need a low level mouse hook that will block them from reaching the destination application.

The low level mouse hook is a special type of hook that if the hook routine returns 1 indicates that the message has already been dealt with and shouldn’t be passed on to anything else, which is exactly what we want.

In particular we need to watch for two messages:

WM_LBUTTONDOWN and WM_LBUTTONUP

In both cases we want to check the global variable inside our DLL (which our low level mouse hooking application should use) to see if we consider the right mouse button as currently down from the WM_POINTERDOWN event. If it is, then we want the low level mouse hook function to return 1 to block the left mouse button messages from reaching any applications.

To quickly sum up, first we need a DLL that:

  • Has a global variable to track if the right mouse button is considered down or not.
  • Installs a WH_GETMESSAGE hook.
  • Converts appropriate (i.e. barrel button pressed) WM_POINTERDOWN messages into WM_RBUTTONDOWN messages via mouse_event and sets the global variable.
  • Converts WM_POINTERUP messages into WM_RBUTTONUP messages via mouse_event if the global variable is set, and then clears the global variable.

We also need a standalone application that:

  • Uses the DLL to turn on the WH_GETMESSAGE hook.
  • Installs a WH_MOUSE_LL hook.
  • Has the mouse hook return 1 if WM_LBUTTONDOWN or WM_LBUTTONUP messages are encountered and the DLL’s global variable is set. Otherwise, the mouse hook should return the value of the next hook in line via CallNextHookEx.

Once all of that’s in place, when you run the standalone application, the appropriate hooks will be installed globally (i.e. across all of Windows) for your current Windows session. That means that any applications you have running will now properly receive Windows messages from the pen as if it were a mouse, completing the restoration of pen functionality that the registry hack, which is also needed for all of this work, does not.

Caveat

Windows hooks only operate on application of the same bitness. Meaning that in order for the hooks to work for all Windows applications you actually need two versions of the DLL and two versions of the standalone application: a 32-bit version and a 64-bit version. Both versions can run side by side without interfering with another.

For best results, place the two applications (or at least shortcuts to the two applications) into the Windows “Startup” folder. That way, whenever Windows starts these two applications will also start so the pen behaves as expected right from the get go.

Now, I’ve already created a 32-bit version of the DLL and standalone application generally described here. I’m still experimenting with a few things, like trying to detect if the eraser button is pressed (its pressed detection is handled differently from either the tip or barrel button), so I don’t have a fully production ready version yet. What I have is good enough for my needs but not quite good enough to release to the world. I’ll keep working on it as time and my own inclination permits and perhaps in the future I’ll release it as a little bit of freeware. Who knows? Keep checking back. It might just pop up on this website someday.