Blues typing out fixed texts on both QMK and Arduino Leonardo-based macro keyboards

The short version is that a 17 ms delay between each key action (press or release, normal key or modifier key) for an Arduino-based keyboard will solve most problems. At this point, there isn’t any known solution for USB 3 ports, short of reinventing part of the firmware, for a QMK-based macro keyboard, except finding a computer system that is less prone to the problem, for example using USB 2 ports instead of USB 3 ports.

The simplest macro key on a macro keyboard would seem to be one that just types out some fixed text (with a few or without any use of modifier keys, notably the Shift key). For example,

Fixed the weird syntax highlighting (as a result, the diff looks more extensive than it really is - use view "Side-by-side Markdown" to compare).

The problem

But it is not straightforward. For two macro keyboards, one based on QMK and another on custom firmware on an Arduino Leonardo, at least one key press is missed in up to about 50% of the cases. This is particularly important when the text must be accurate, for instance, a URL. If it is more than typing out characters, the result could be catastrophic (but the typing rate is usually much lower as the macro keyboard in most cases has to wait for programs to respond).

Strange combinations

Strangely, the problem can not be isolated to a single factor. On one system it worked in Geany, but not in Firefox. On another system, it worked in both.

Isolating the problem

We don’t know exactly where the problem is. It could be all way from the electronic signal level (e.g., bad USB cables or bad USB ports/hubs) all the way up to the application level, with the BIOS, and the operating system and its drivers inbetween.

Different applications

This can actually make a difference… In a particular combination, the problem was present in an edit window in a web browser (and in two different web applications), but not in a text editor (Geany).

Different Linux flavours, incl. GNOME vs. better alternatives

A candidate culprit is GNOME.

For instance, GNOME is known to be buggy and to introduce new bugs in fundamental functionality. For instance, they broke the clipboard with Ubuntu 19.10 (Eoan Ermine) (it worked fine in Ubuntu 19.04 (Disco Dingo)).

Here is one data point from a Ubuntu MATE 20.04 (Focal Fossa) system:

Results:

Logged into MATE: A 2% error rate was observed (7 / 290 lines).
Logged into Cinnamon: no errors were observed in 444 typed out lines
For Windows 10 Home 64 bit under VirtualBox: no errors were observed in 1680 typed out lines.

This indicates the problem could be with GNOME. MATE is based on GNOME 2 and the error is definitely present in GNOME 3 (for example, on Ubuntu 18.04 (Bionic Beaver). The measured error rate varies, but it is in the range 3% – 15%).

Test conditions:

Arduino Leonardo-based macro keyboard.
Nominal key delay of 2 ms (known to cause problems on some systems), both for key presses and key releases. The actual type out speed was measured to be about 150 per second (cycle time of about 7 ms).
Macro keyboard connected to a powered D-Link USB hub, on a non-extra power port
GA-970A-DS3P motherboard
Base operating system Ubuntu MATE 20.04 (Focal Fossa) with Cinnamon used as the default GUI.
Typing a 144 character-long fixed text line into a Firefox edit window (repeating lines automatically by the macro keyboard): “Fixed the weird syntax highlighting (as a result, the diff looks more extensive than it really is – use view “Side-by-side Markdown” to compare).”
There is a 1 second delay before Enter is hit at the end of the line. And a 2-second delay before the next line is typed out. This is to allow the system to catch up if needed, come to rest, and be closer to real-world use.
For Windows (running under VirtualBox), several restarts of Firefox and of Windows itself were performed before the test in order for updates to be performed, etc.
NoScript version 11.4.6 was active in Firefox, both on Linux and Windows.
The keyboard layout was set to Danish in both native Linux and on Windows.

Note that the hardware (USB cable, USB hub, USB port, and motherboard), BIOS, base layers of the operating system, and version of Firefox (102.0) are exactly the same.

Operating systems

Windows vs. Linux may make a difference.

An alternative interpretation of the results from section “Different Linux flavours” (though most are unlikely due to the different result for MATE and Cinnamon (VirtualBox and Windows not involved at all)):

Differences in the configuration of the two Firefox instances (in Windows and on Linux).
VirtualBox could buffer and/or time space keyboard events.
Difference in behaviour of Firefox on Linux and Windows (even for the exact same version). For instance, in the experiment we used the 64 bit version on Linux and 32 bit in Windows.
Differences in screen resolution and Firefox zoom levels.

Different versions of the same Linux distribution (with the same window manager)

It may play a role.

BIOS

BIOS settings related to USB. BIOS type and versions.

More than one keyboard connected

The system (e.g., the operating system) may get confused or overloaded if there is more than one keyboard connected.

We could disconnect any keyboard but the macro keyboard during the test to exclude this factor.

AVR microcontroller keyboard vs. commercial keyboards

Even though QMK and the Arduino Leonardo macro keyboards have completely different code bases, they are both based on the same AVR microcontroller, ATmega32U4. The QMK keyboard is implemented with another Arduino board, Arduino Micro, but it is the exact same microcontroller.

In one combination, where both keyboards were connected to the same USB hub on the exact same system (Ubuntu 20.04 MATE system with an edit field in Firefox), only the QMK keyboard exhibited the problem. The type out rate of the QMK keyboard was measured to be about 50 characters per second and the type out rate of the Arduino Leonardo keyboard was matched to that. So the problem is not only dependent on the type out rate. (The use of the USB ports in the USB hub for the two keyboards were also swapped to rule out any influence of the physical USB ports.)

USB 2 vs. USB 3

For the QMK keyboard, there seems to be a robust difference!

On the GA-970A-DS3P system, the problem can be reproduced (about 50% error rate) with connecting one of the USB 3 ports (with or without a USB hub), but not when using one of the USB 2 ports. Or at least the error rate is much lower for USB 2 ports. 154 lines were output without any error.

Further tests are needed to rule out the motherboard itself. That is, this difference should also be demonstrated on another motherboard.

Note that this motherboard had trouble using USB 2 and USB 3 at the same time, at least on Linux. It is not known if this has an influence or not.

USB hub vs. direct connection

There could be a difference. There are also various combinations, e.g., a USB 2 hub connected to a USB 3 port on the motherboard. But in one combination, a USB 2 hub on a USB 3 port, it didn’t make any difference.

The USB hub may be powered (own power supply) or powered by the host system.

There are various kinds of USB hubs.

The particular USB hub in use may have a hardware problem.

USB ports

The particular USB port may have a hardware problem or a problem delivering enough power under some conditions. For instance, there could be some problem (say, partially working) with one USB 3 port, but not the other USB 3 port. This could confuse test results and/or be the real cause.

The supply voltage may be marginal.

USB cable

The problem could be due to a bad USB cable in the QMK-based keyboard.

The resistance in the cable, particular for the supply voltage, could be too high. Some dubious USB cables are known to exist where the resistance is four times higher than normal (probably copper replaced by iron/steel).

Bad microcontroller

The problem could be due to bad Arduino microcontroller hardware in the QMK-based keyboard (a one-off problem). Or a general hardware problem (design problem) with the Arduino Micro board. Or a general hardware or software problem (design problem) with all Arduino boards using the ATmega32U4 microcontroller.

One particular board could be more sensitive to a marginal supply voltage than another.

The decoupling of the power supply (transient) on a board (in general or for a particular board) could be deficient.

Spell checker in Firefox

It is conceivable that key presses could be missed if the entire Firefox application is locked up for extended periods of time by the spell checker. The effect is especially pronounced on some Wikipedia pages (100% CPU consumption can be observed at times, but it is not known if it actually locks up the application).

However, the missed keys have also been observed in Stack Overflow edit summary fields where the spell checker is apparently not active (it may only be activated when the focus is in HTML ‘textarea’ form fields?).

A nearly-working solution

Previous experience, that is, empirical knowledge, from the development of the Arduino-based macro keyboard has shown that a delay between even short sequences of keys to be necessary. This was tried on Linux-based systems. It remains to be seen if it is different on Windows.

For instance, just for the two keys Alt + Tab to work reliably, 8 ms between them was too fast. The threshold was found to be 17 ms, at least at a glance (it is not known if the error falls to 0 above that or if there is just a lower probability of error).

However, the error rate does not fall to zero. Key strokes are still missed from time to time. The problem is more like reduced by 1-2 orders of magnitude. This is OK for most applications, but not when perfection is required. For instance, missing a key or modifier key in a complex keyboard macro is potentially disastrous.

Automating the testing

An error rate of about 50% (for every typed out string) is noticeable, but it quickly becomes tedious to do. And we also want to be able to detect lower error rates and get statistics about where the error occurs, for instance, depending on how long the string is or where in the string a character is.

It would also be interesting to measure if there is any difference between the two types of macro keyboards.

We can’t just let it type into a text document, save it, and analyse it. The problem could be masked as we know at least one instance where the application (Firefox vs. Geany) makes it behave differently.