Monday, December 19, 2016

libinput touchpad pointer acceleration analysis

A long-standing criticism of libinput is its touchpad acceleration code, oscillating somewhere between "terrible", "this is bad and you should feel bad" and "I can't complain because I keep missing the bloody send button". I finally found the time and some more laptops to sit down and figure out what's going on.

I recorded touch sequences of the following movements:

  • super-slow: a very slow movement as you would do when pixel-precision is required. I recorded this by effectively slowly rolling my finger. This is an unusual but sometimes required interaction.
  • slow: a slow movement as you would do when you need to hit a target several pixels across from a short distance away, e.g. the Firefox tab close button
  • medium: a medium-speed movement though probably closer to the slow side. This would be similar to the movement when you move 5cm across the screen.
  • medium-fast: a medium-to-fast speed movement. This would be similar to the movement when you move 5cm across the screen onto a large target, e.g. when moving between icons in the file manager.
  • fast: a fast movement. This would be similar to the movement when you move between windows some distance apart.
  • flick: a flick movement. This would be similar to the movement when you move to a corner of the screen.
Note that all these are by definition subjective and somewhat dependent on the hardware. Either way, I tried to get something of a reasonable subset.

Next, I ran this through a libinput 1.5.3 augmented with printfs in the pointer acceleration code and a script to post-process that output. Unfortunately, libinput's pointer acceleration internally uses units equivalent to a 1000dpi mouse and that's not something easy to understand. Either way, the numbers themselves don't matter too much for analysis right now and I've now switched everything to mm/s anyway.

A note ahead: the analysis relies on libinput recording an evemu replay. That relies on uinput and event timestamps are subject to a little bit of drift across recordings. Some differences in the before/after of the same recording can likely be blamed on that.

The graph I'll present for each recording is relatively simple, it shows the velocity and the matching factor.The x axis is simply the events in sequence, the y axes are the factor and the velocity (note: two different scales in one graph). And it colours in the bits that see some type of acceleration. Green means "maximum factor applied", yellow means "decelerated". The purple "adaptive" means per-velocity acceleration is applied. Anything that remains white is used as-is (aside from the constant deceleration). This isn't really different to the first graph, it just shows roughly the same data in different colours.

Interesting numbers for the factor are 0.4 and 0.8. We have a constant acceleration of 0.4 on touchpads, i.e. a factor of 0.4 "don't apply acceleration", the latter is "maximum factor". The maximum factor is twice as big as the normal factor, so the pointer moves twice as fast. Anything below 0.4 means we decelerate the pointer, i.e. the pointer moves slower than the finger.

The super-slow movement shows that the factor is, aside from the beginning always below 0.4, i.e. the sequence sees deceleration applied. The takeaway here is that acceleration appears to be doing the right thing, slow motion is decelerated and while there may or may not be some tweaking to do, there is no smoking gun.


Super slow motion is decelerated.

The slow movement shows that the factor is almost always 0.4, aside from a few extremely slow events. This indicates that for the slow speed, the pointer movement maps exactly to the finger movement save for our constant deceleration. As above, there is no indicator that we're doing something seriously wrong.


Slow motion is largely used as-is with a few decelerations.

The medium movement gets interesting. If we look at the factor applied, it changes wildly with the velocity across the whole range between 0.4 and the maximum 0.8. There is a short spike at the beginning where it maxes out but the rest is accelerated on-demand, i.e. different finger speeds will produce different acceleration. This shows the crux of what a lot of users have been complaining about - what is a fairly slow motion still results in an accelerated pointer. And because the acceleration changes with the speed the pointer behaviour is unpredictable.


In medium-speed motion acceleration changes with the speed and even maxes out.

The medium-fast movement shows almost the whole movement maxing out on the maximum acceleration factor, i.e. the pointer moves at twice the speed to the finger. This is a problem because this is roughly the speed you'd use to hit a "mentally preselected" target, i.e. you know exactly where the pointer should end up and you're just intuitively moving it there. If the pointer moves twice as fast, you're going to overshoot and indeed that's what I've observed during the touchpad tap analysis userstudy.


Medium-fast motion easily maxes out on acceleration.

The fast movement shows basically the same thing, almost the whole sequence maxes out on the acceleration factor so the pointer will move twice as far as intuitively guessed.


Fast motion maxes out acceleration.

So does the flick movement, but in that case we want it to go as far as possible and note that the speeds between fast and flick are virtually identical here. I'm not sure if that's me just being equally fast or the touchpad not quite picking up on the short motion.


Flick motion also maxes out acceleration.

Either way, the takeaway is simple: we accelerate too soon and there's a fairly narrow window where we have adaptive acceleration, it's very easy to top out. The simplest fix to get most touchpad movements working well is to increase the current threshold on when acceleration applies. Beyond that it's a bit harder to quantify, but a good idea seems to be to stretch out the acceleration function so that the factor changes at a slower rate as the velocity increases. And up the acceleration factor so we don't top out and we keep going as the finger goes faster. This would be the intuitive expectation since it resembles physics (more or less).

There's a set of patches on the list now that does exactly that. So let's see what the result of this is. Note ahead: I also switched everything from mm/s which causes some numbers to shift slightly.

The super-slow motion is largely unchanged though the velocity scale changes quite a bit. Part of that is that the new code has a different unit which, on my T440s, isn't exactly 1000dpi. So the numbers shift and the result of that is that deceleration applies a bit more often than before.


Super-slow motion largely remains the same.

The slow motions are largely unchanged but more deceleration is now applied. Tbh, I'm not sure if that's an artefact of the evemu replay, the new accel code or the result of the not-quite-1000dpi of my touchpad.


Slow motion largely remains the same.

The medium motion is the first interesting one because that's where we had the first observable issues. In the new code, the motion is almost entirely unaccelerated, i.e. the pointer will move as the finger does. Success!


Medium-speed motion now matches the finger speed.

The same is true of the medium-fast motion. In the recording the first few events were past the new thresholds so some acceleration is applied, the rest of the motion matches finger motion.


Medium-fast motion now matches the finger speed except at the beginning where some acceleration was applied.

The fast and flick motion are largely identical in having the acceleration factor applied to almost the whole motion but the big change is that the factor now goes up to 2.3 for the fast motion and 2.5 for the flick motion, i.e. both movements would go a lot faster than before. In the graphics below you still see the blue area marked as "previously max acceleration factor" though it does not actually max out in either recording now.


Fast motion increases acceleration as speed increases.

Flick motion increases acceleration as speed increases.

In summary, what this means is that the new code accelerates later but when it does accelerate, it goes faster. I tested this on a T440s, a T450p and an Asus VivoBook with an Elantech touchpad (which is almost unusable with current libinput). They don't quite feel the same yet and I'm not happy with the actual acceleration, but for 90% of 'normal' movements the touchpad now behaves very well. So at least we go from "this is terrible" to "this needs tweaking". I'll go check if there's any champagne left.

16 comments:

Shiba said...

That's a really huge improvement for my thouchpad, thank you very much!

Lapo said...

Awesome

Arti Zirk said...

Touchpad on my Dell Vostro 3360 is now much more usable. Thank you!

Hugh Redelmeier said...

Thanks very much for working on this.

I'm a little confused. Surely a human will adjust their touchpad action based on screen pointer feedback from the system. How can you treat touchpad sequences as invariant even when the feedback is being changed?

(I recently read a book about Doug Englebart (the inventor of the mouse). His goal was to have users and systems "co-evolve": each changing and improving together. In the case of the touchpad, users will evolve to handle whatever the touchpad software does.)

Perhaps the first requirement is consistency. I've sometimes felt touchpad response a little inconsistent on my Linux systems, especially on ones that are overloaded. That problem might be in Firefox rather than libinput.

Peter Hutterer said...

Hugh: I'm not treating it as completely invariant but you'll find that when you move your finger on a touchpad, you have a fairly clear idea of what's supposed to happen. If that doesn't happen then you get higher cognitive load because now you have to correct for it. This is particularly true for movements with a pre-selected target, e.g. an icon close to the pointer.

Personally, I also notice this a lot more on a touchpad than a mouse, the feedback loop on a mouse seems to be tighter than on a touchpad. But that's based on a sample size of 1 (i.e. me :)

Nine H said...

I've been messing around with the accelerometers on my phone in sailfish OS, I was wondering what tool you used to render your graphs, and what your scripts look like, because it looks heaps cleaner and smarter than what I'm doing (printfs and wild guesses).

Peter Hutterer said...

Nine: they're just gnuplot graphs. I used printfs in the libinput code, then a python script to convert from the raw data into the bits I needed. The python script writes out a bunch of gnuplot files (plus matching data files). the source to reproduce is here https://github.com/whot/libinput/commits/wip/touchpad-pointer-accel-v3

Nine H said...

reading the gnuplot docs and your sourcecode right now. thanks ledge.

Shark said...

Hi there, Peter! Nice work! Any expectations as to when it might land in Fedora's repos?

Peter Hutterer said...

Shark: probably not until I can cut a 1.6 RC, it's too many patches to pull in, sorry. But at least that one shouldn't be too far off.

MatMaul said...

Hi Peter, I installed master on my Fedora 25 to test the new code. Seems fine, but I find the max speed too slow for me. While it was ok before with the gnome setting at around 80%, I am now struggling even at 100%. Is it possible to change a multiplier somewhere to be able to reach higher speed ? Thanks !

Peter Hutterer said...

MatMaul: file a bug please, this blog's format isn't useful when it comes to debugging things.

André Brait said...

Where should one report bugs related to libinput?

Peter Hutterer said...

https://wayland.freedesktop.org/libinput/doc/latest/reporting_bugs.html

André Brait said...

Thanks a lot!

Christopher Carr said...

From the perspective of someone coming from Chromebooks -- I haven't had a laptop running Linux in several years:

Picked up a ThinkPad X1 Carbon 4th Gen recently, and installed Antergos with Gnome. Compared to my old Toshiba Chromebook 2 (Swanky), the cursor feels extremely squirrely -- with the biggest issue being that I overshoot targets every single time. On my Chromebook, cursor comes to a screeching halt exactly where I want it, after both long and short movements. My ThinkPad's touchpad feels pretty horrible by comparison, to be frank, and I've been using it for about a week now. (As a separate matter, palm detection doesn't seem to actually work).

I know this isn't the place for bug reports, just thought I would offer my impressions coming from Chrome OS -- not a comparison to previous versions of libinput. It's almost maddening.

I wish I could use the Chrome OS input stack.