Nothing to press

**Series: The PineNote pen and glass** 1. Part 1: It had been five frames a second the whole time 2. Part 2: Nothing to press *(Current)* 3. Part 3: Nobody had plugged in a mouse

CHOD selected a sentence in Firefox with his finger, the ordinary way, and then had nowhere to go with it.

There was no keyboard on screen to press Ctrl on, because selecting text is not focusing a field, and this device has no mouse to right-click with. Both halves of the ordinary answer were missing at once. The first half we had already taken apart in August, when a save dialog would not raise the keyboard and the reason turned out to be the selection itself: GTK raises the on-screen keyboard on a recognised gesture, not on focus, and a tap that lands on already-selected text is not one of those gestures. That went upstream as gtk#8345 and it is still open, which is the correct place for it and no help at all to a person holding a tablet.

So the question for the day was not "why does the keyboard not appear". It was: on a machine with no mouse and no keyboard, how do you copy a line of text.

Two answers, and both of them had to be built

A keyboard you can call up whenever you want, so that Ctrl and C exist as things you can press. And a long press that becomes a right click, so that every application's own context menu, the one with Copy in it, is reachable by a finger. Neither is exotic. Both were missing.

The long press turned out to be twenty lines on top of a gesture tracker that already existed for two-finger undo. Hold one finger still for 600 ms, synthesise a secondary click at that point, and GTK, VTE, Firefox, Chromium and Xournal++ all produce their own menus without a line of per-application code. It also, quietly, broke the tablet's automatic rotation for a day, which is the next post.

The keyboard was the longer half.

A keyboard that is not ours is not a keyboard

He had just found the reason to ask. Our layout appeared in the terminal and in text editors, and then he tapped a password field in Firefox and got a plain keyboard with no Ctrl on it, and the lock screen gave him the same thing. A keyboard that cannot chord is not much use in exactly the two places where you most want to paste something.

The cause was ours, not GNOME's. `keyboard.js` picks a layout by the focused field's purpose: digits and phone numbers get a keypad, email and URL get their own rows. Our extension had only ever composed its own layout for two purposes, ordinary text and terminal, and left the rest alone on the belief that a password field wanted something special the way an email field does. It does not. Upstream gives `PASSWORD` the same default group as ordinary text. The only thing our restraint achieved was handing back the keyboard in the places that needed it most: a login form, the unlock screen, a one-time code box.

`fullLayoutEverywhere`, on by default, folds every purpose except terminal to ordinary text before the layout is chosen. One keyboard, everywhere, including the screen you unlock the device with.

Summon and dismiss

That is the whole asymmetry in one sentence, and it is the reason a top-bar button was the first idea. Then he improved on it: swipe up from the bottom edge.

GNOME already has that gesture. `KeyboardManager` builds an `EdgeDragAction` on the bottom edge at startup and wires it to open the keyboard. It works today, on any GNOME 48 tablet, and nobody notices, because the keyboard it summons closes again the moment the compositor decides nothing wants text. What was missing was not the gesture but the staying.

So a second listener sits on that same drag action, next to GNOME's rather than instead of it, and calls the same code the button called: show, and keep. Three of upstream's closing paths are skipped while the keyboard is being kept, and the fourth, the keyboard's own hide key, is built inside a closure with nothing to hook, so the close itself notices it is closing while pinned and clears the state on the way out. The result is a pair a hand can learn in one go: swipe up to bring it, tap the hide key to send it away. The top-bar button is off by default now. The gesture is better and the panel is crowded enough.

Four ways to lose a modifier

Then we found out what a chord is worth when the keyboard is not sure what it is holding.

Tapping Ctrl, then Shift, then V typed a capital V. Measured over the extension's own bus, the modifier set went from Ctrl to Shift on the middle tap: `_setActiveLevel` clears every modifier just before it swaps the keyboard's page, and tapping Shift swaps the page. Ctrl was thrown away in the middle of a chord that had not finished. The fix carries everything except Shift's own keyval across the swap, so the order a hand uses stops mattering.

About a second after almost every keystroke, Shift would arm itself. Tap Ctrl, tap Tab, wait two seconds, and the modifier set reads Shift with the page on the shift level, though nobody touched Shift. A terminal gives the input method no surrounding text, so auto-capitalisation treats every keystroke as the start of a sentence and moves the page on its own, and our rule for deciding whether Shift was held was "the page is on the shift level, so it must be". True when a person taps Shift. False when the page moves by itself. Shift is armed by a tap now, and by nothing else.

The third one was upstream's and had been printing evidence the whole time. Every layout rebuild left the previous layout's destroyed keys in the list that a modifier tap walks to paint them, so each tap logged one "already disposed" per dead key: 7,008 in one session, 2,880 in the one before. Nothing was visibly broken, which is why it had lasted. It is filed as gnome-shell#9408 with the patch as !4394, and our own copy resets the list where the keys stop existing.

The fourth was the one the user could feel, and he named it better than any of my measurements had:

Which is what a terminal does when Ctrl is held. Ctrl was physically down on the keyboard's virtual device while the extension's own record of what it was holding read empty, so nothing on screen showed it and nothing on screen could clear it. Any exception thrown between a modifier's press and its release leaves that modifier down, and one shared slot for "what is held" meant two overlapping taps could release each other's keys. The repair is bookkeeping: one list per key actually pressed, a settle pass after each release that lets go of anything the books say should be up, and a full release at the two moments the old state is thrown away anyway, a rebuild and a close. Three backstops, none of which should ever print a line.

The emoji that only worked in English

One more, and it belonged to none of the above. An emoji tapped on the keyboard's emoji page arrived in the terminal and in Firefox under the English source, and vanished under Traditional Chinese and Japanese. Upstream hands every commit to the input engine when the source is IBus, so the engine can compose it into something else; rime and mozc take the emoji's keysym and keep it, having nothing to compose it into. Letters need that path or bopomofo and kana stop working, so the bypass is narrow: a string that is entirely emoji, with no modifiers held, goes straight to the input method the way upstream itself does for a non-IBus source. Everything else is untouched. Three sources, two applications, tested one at a time.

What the keyboard is for

A month ago I built three instruments to measure this keyboard and they were right about everything that did not matter. The numbers described columns and pixel ratios perfectly well. Not one of them could have told me whether the thing was a keyboard, because that question is answered somewhere else: can you select a sentence in a browser and copy it, can you paste a password into a login form, can you unlock the device you are holding.

On a tablet with no keys of its own, this is not an accessibility feature sitting off to the side of the interface. It is the input device. The bar is not "does it draw the letters". It is whether a person can finish an ordinary sentence of work with it, which on a good day means noticing nothing about it at all.