A hack to use mouse actions in Via macros

The short version: A relatively simple hack enables mouse actions in Via. Seldomly used keycodes, e.g., for F23 and F24, accepted by Via, are converted to the keycodes for left click and right click just before macro execution.

This breaks the last barrier for Via to be used for serious use of macros, not just prototyping.

Introduction

By default, Via has a number of limitations that prevents it from being used for more than prototyping macros, as preparation for implementing macros as, e.g., classic QMK macros or for a macro execution engine that enables cancelling macros in progress (among other things):

Space for macros is limited.
The number of macros is limited to 16
Mouse actions are not supported in macros
Macros only have a number to identify them, not a name
Macro execution can’t be interrupted (other than physically disconnecting the keyboard (or otherwise deprive it of power)). Via macros have this missing feature in common with traditional QMK macros (whereas many commercial offerings have the capability). It requires a whole different way of executing macros, but there is a way.
Via can only show the United States keyboard layout (the physical layout of keys on it is often referred to as “ANSI”) interpretation of keycodes, not, for example, a Nordic keyboard layout on an ISO keyboard layout (the ISO layout has an extra key, between Shift and what is interpreted as “Z” on many language key layouts, compared to the ANSI layout). This has nothing to do with macros, but it may cause confusion when using Via (it doesn’t hinder configuration of an ISO keyboard in any way; it is just that the Via display (an interpretation) does not correspond to what actually happens when operating the keyboard).
Some key codes or their aliases are not accepted (in Via macros and/or keymappings). Typically, only one alias is accepted, not the full keycode or other aliases (though sometimes it is the other way around: the full name is accepted and not any aliases (for example, mouse scrolling))

Via also has a number of usability problems:

The startup animation may suggest it is hanging at startup (after a while), but it is not. It is still possible to select the DESIGN tab and select the JSON file. A sample
After assigning a key, Via automatically moves to the next key. This makes it easy to make inadvertent changes, as the increment to the next key is unexpected (and may not be noticed, looking at possible keys to map and not the keyboard display at the top). It may be great if making a lot of changes in succession, but definitely not when making keymapping adjustments to single keys. It is particularly confusing when it advances to a knob… A sample. Vial has the same problem; it has probably been copied from Via, without thinking about how problematic it is. Fortunately, it can be disabled in Via: SETTINGS (the last tab on the top) → Fast Key Mapping (turn it off; inner rectangle to the left)

1. has been addressed previously, and it is possible to increase to be essentially unlimited. 2. is also possible to increase. (The exact same solution can be used with Vial to increase the number of macros from the default 16.). 4. needs to be addressed by maintaining some sort of external document. 5. is practically impossible, though an idea is, like the hack here, to intercept Via macro execution, save all key actions into a buffer, and use the same idea of piecemeal execution as described in The basis for a way to cancel QMK macros in progress—piecemeal in order to check between issuing each key action if a key has been pressed to stop macro execution.

3. is addressed here. The problem is that Via, for unknown reasons, doesn’t accept the QMK keycodes for mouse actions, e.g., KC_MS_BTN2 for right-click:

“Whoops! Invalid keycodes detected inside{}: KC_MS_BTN2”

It is a Via problem as QMK doesn’t distinguish between key codes for keyboard actions and mouse actions. For example, mouse actions work perfectly fine in traditional QMK macros (if QMK has been set up at compile time to support it (e.g., by default for most Keychron keyboards)).

Note that it isn’t the problem of Via only accepting aliases instead of the full keycodes; using the alias “KC_BTN2” will give the same result.

So the idea is to use QMK keycodes that Via accepts and then convert them to mouse keycodes on the fly, during macro execution. Macro execution is in function dynamic_keymap_macro_send() where the stored macros in EEPROM memory are interpreted and converted to a format that function send_string_with_delay() expects. See this blog post for details on function send_string_with_delay().

Select keycodes to represent the mouse actions

This should be keycodes that are unlikely to be used in any macro.

For example:

KC_F23 (key code 114 (decimal)) for mouse left click (keycode KC_MS_BTN1)

KC_F24 (key code 115 (decimal)) for mouse right click (keycode KC_MS_BTN2)

Source code

The hack can be implemented as follows.

At the end of function “dynamic_keymap_macro_send()” (in file quantum/dynamic_keymap.c), just before the call of function “send_string_with_delay(data, DYNAMIC_KEYMAP_MACRO_DELAY);” (approximately line 384), add:

        // Hack to enable mouse actions in Via macros
        //
        // We use some uncommon keycodes (F23 and F24) and
        // replace them with mouse actions on the fly
        if (
               (data[0] == SS_QMK_PREFIX) &&

               // Both key press and key release
               ((data[1] == SS_DOWN_CODE) || (data[1] == SS_UP_CODE))
           )
        {
            if (data[2] == KC_F23)
            {
                // Replace the KC_F23 keycode with the
                // keycode for ***mouse left click***
                //
                data[2] = KC_MS_BTN1; // Alias: KC_BTN1
            }

            if (data[2] == KC_F24)
            {
                // Replace the KC_F24 keycode with the
                // keycode for ***mouse right click***
                //
                data[2] = KC_MS_BTN2; // Alias KC_BTN2
            }
        }

The firmware needs to be build using this changed source code and the keyboard flashed.

Procedure in Via

Save off the Via configuration (to a JSON file), using “SAVE + LOAD” → “Save Current Layout” → “Save”. This is to be able to revert changes, especially inadvertent changes (there isn’t any undo functionality in Via).
Recording a macro: Use the up arrow key to represent a left click. Use the down arrow key to represent a right click. Or if those two are used in the macro, choose some other set of keys, for example, F7 and F8.
Stop recording.
Save (crucial for the next step, as it isn’t really a view (as one would expect (bad UI design)), but a read from the keyboard, thus wiping out the recording if not saved first)
Switch to code/JSON view by clicking the “</>” icon
Replace “KC_UP” with “KC_F23” (effectively left click with this hack) – both press and release (“+KC_F23” and “-KC_F23“)
Replace “KC_DOWN” with “KC_F24” (effectively right click with this hack) – both press and release (“+KC_F24” and “-KC_F24“)
Again, save (crucial for the next step, as it isn’t really a view (as one would expect (bad UI design)), but a read from the keyboard, thus wiping out the change if not saved first)
Switch back to the normal view (the icon to the left of the “</>” icon)
Make other adjustments, like changing the timing. All non-critical timing can be changed to 17 ms to speed up the macro.
Save changes
Assign the new macro to some key on some layer
Test the macro. If needed, adjust the timings.
Save off the Via configuration (to a JSON file). This is to have a backup as the Via configuration is easily wiped out or easily accidentally changed.

Testing

I have now used this solution (hack) for a few weeks without any ill effects. It really makes a difference to automate simple, but very frequently used, keyboard and mouse actions.

For instance, it is very tedious to change to Markdown mode every single time when using the comment editor for Reddit comments. These keyboard actions and a single mouse left click, enabled by the hack, works for that:

Left click (the mouse cursor is expected to be over the “…“)
Wait 300 ms (verified OK timing. 200 ms may also work.)
Arrow down (the focus will change to “Save”. It is not 100% reliable as it seems to remember the state for each comment.)
Arrow up (the focus will change to “Edit comment”)
Enter
Wait 600 ms (verified OK timing. 400 ms is too fast; sometimes it works and sometimes not. Though 600 ms is also sometimes too fast; 800 ms may work better)
Ctrl + Home (To make it more robust when there is code or a list at the end of the comment (content is changed by Tabs in those cases)).
3 x Tab (the focus will change to “T”. Though the number of tabs depend on the particular subreddit; some enable posting images and this increases the required number of tabs to four or five)
Enter (The editing bar, with “B” … “Markdown editor”, will become visible. Note: The focus in the text will change to the very end, no matter the original focus in the other mode… (“Rich Text Editor”))
Wait 200 ms
1 x arrow left (the focus will change to “Markdown editor”)
Enter.
An extension: wait 200 ms (for updating the screen after Enter), Ctrl + Home, type a space (it can be any character) and delete it again with Backspace; this will automatically expand the view to include all of the comment (at least in Firefox)
Another extension: enable spell-checking in Firefox (for unknown reasons, it is always turned off, even if enabled in a previous session for the same comment). Wait 200 ms (for updating the screen after the previous step), right-click, wait 200 ms, 3 times arrow down, and Enter. Though the mouse cursor may or may not be in the right location for this work (for example, depending on the scroll position and the size of the comment (number of lines)).
A variation (a slightly different macro): After the above, jump to a particular place, specified by a search string in the clipboard: Ctrl + F, Ctrl + V (paste into the search box), Esc (to change focus back to the edit pane), and arrow left. The arrow left is to change the focus, so the place is actually visible (to reduce confusion); it can also be done manually, but macros are for saving on this sort of thing. To really make it convenient, perhaps do a five times arrow down plus five times arrow up to center it vertically (as the initial result is often focused at the very bottom of the screen).

Note that in this example, F7 should be used to represent the left click, not arrow up (as arrow up is part of the macro itself).

Note: It seems the behaviour changed at Reddit in late May 2024; the Markdown state now seems to be remembered (in most cases). Thus steps 7-12 aren’t necessary anymore (and must also be removed for the (Via) macro to work correctly).

Note: It may not work (“Edit Comment” will not be selected) at high zoom levels, as the behaviour changes if “Edit Comment” is not visible due to the zoom level…

Snippets for mouse actions in Via macros

Alternatively, instead of recording and substituting key codes, use (copy-paste, mix, and match) these snippets of Via macro source:

Left-click: {+KC_F23}{17}{-KC_F23}{17}
Right-click: {+KC_F24}{17}{-KC_F24}{17}
Double click: {+KC_F23}{17}{-KC_F23}{100}{+KC_F23}{17}{-KC_F23}{17}

More examples

Copy the link under the cursor in Firefox

Right click (the mouse cursor is expected to be over the link)
Wait 400 ms
L (for menu item “Copy Link”

Via macro source: {+KC_F24}{17}{-KC_F24}{400}{+KC_L}{17}{-KC_L}

Note: {+KC_L}{17}{-KC_L} can not be reduced to {KC_L}, even if it is only a single letter; it is too fast… (at least on some systems) Thus, the rule holds that there should be at least 17 ms between keystrokes, at least for normal keys (it may or may not be less restrictive for modifier keys).

Open the link under the cursor in a new tab in Firefox

Right click (the mouse cursor is expected to be over the link)
Wait 400 ms
T (for menu item “Open Link in New Tab”

Via macro source: {+KC_F24}{17}{-KC_F24}{400}{+KC_T}{17}{-KC_T}

Note that a (faster) alternative way is Ctrl + left click:

{+KC_LCTL}{17}{+KC_F23}{17}{-KC_F23}{17}{-KC_LCTL}{17}

But here it is demonstrated that delays are often crucial, for example, when a key action results in a screen update.

A variation is to move the newly opened tab to the left (by moving the current tab one to the right), so it does not occupy the right-most tab: Add Shift + Ctrl + PgDn.

Epilogue

Mouse actions are actually technically supported in Via macros, but not from a user perspective (it seems to be a bug in the Via client).

So practically speaking, the solution described here is still required.