PsychoPy3 PsychHID based Keyboard API

Background

In an earlier post we used a MilliKey response box to test the time stamping accuracy of three different Python accessible keyboard event APIs when receiving 1000 Hz MilliKey USB keyboard events:

  • psychopy.event.getKeys()
  • psychopy.iohub…keyboard.getPresses()
  • Psychtoolbox Python PsychHID()

and found that Psychtoolbox PsychHID Python interface provided the most accurate keyboard event time stamps across all three OS’es tested, particularly for macOS.

PsychoPy3 keyboard.Keyboard

In this post we use the MilliKey response box to test the latest development version of PsychoPy3 and the new keyboard event handling module added by Jon Pierce, psychopy.hardware.keyboard.Keyboard.

psychopy.hardware. keyboard.Keyboard uses the Psychtoolbox PsychHID Python interface to provide accurate cross platform 1000 Hz keyboard event time stamping.

Results

psychopy.hardware. keyboard.Keyboard

PsychoPy3’s psychopy.hardware. keyboard.Keyboard module takes full advantage of the PsychHID engine to provide accurate 1000 Hz MilliKey keyboard event time stamping on Windows, Linux, and macOS. Average event latency is ~1 msec across all operating systems.

Very nice.

Comparision

In comparison, lets look at the performance of other PsychoPy keyboard event APIs.

psychopy.iohub..keyboard.getPresses()

Python3 psychopy.iohub keyboard event timing is not as consistent across operating systems. On Windows, and in particular Linux, iohub comes close to matching the performance of PsychHID, but macOS causes extra delay compared to other operating systems.

psychopy.event.getKeys()

Results for psychopy.event.getKeys() are similarly poor across operating systems. Remember that in this test keyboard events were being created during calls to win.flip(). The timing results show how big of an impact this has on time stamping error when a 60Hz monitor is used.


Happy Hacking!

Share:

PsychHID Rules!

Updated March 19th, 2019 with Linux results.

I spent some time last weekend using the MilliKey response box to test the keyboard event time-stamping accuracy of the PsychToolbox3 PsychHID library written by Mario Kleiner. Given my love of Python, I had to try out the psychtoolbox Python wrapper of PsychHID and compare it to psychopy.event.getKeys() and psychopy.iohub keyboard.getPresses().

This test also makes things a bit harder for the experiment software by having the display updated on each frame of a trial and using MilliKey to generate keyboard events between 2 and 14 msec into a retrace.

Screen capture of test script running using PsychHID. Results are amazing, especially given in this case a screen cap app was also running!

Results

We’ll start with the results and then give the source code later. All results were collected using PsychoPy 3.0.6 with Python 3.7.1.

Regardless of operating system, the computer monitor was running at 60 Hz.

1000 key press events were generated by the MilliKey device for each test, with the key event occurring between 2 and 14 msec into a window flip().

Windows 10

psychopy.event.getKeys

As would be expected, the time stamp accuracy obtained from
psychopy.event.getKeys depends on when the key press event actually occurred relative to win.flip(), with an average error of 7 msec ranging from 0.5 msec to 16.9 msec.

psychopy.iohub keyboard.getPresses()

In contrast, iohub and PsychHID perform keyboard event time-stamping asynchronously to the experiment process, so average time-stamp error is much better.

With that said, iohub does not match the raw speed of PsychHID:

Python PsychHID() KbQueue*

Linux

psychopy.event.getKeys

As was seen on Windows, the time stamp accuracy of keyboard events using event.getKeys() is totally dependent on when the key press event actually occurred relative to win.flip()

psychopy.iohub keyboard.getPresses()

iohub performance on Linux seems very good, even on Python 3.

Python PsychHID() KbQueue*

PsychHID just takes top place on Linux as well, with iohub not far behind (maybe not at all statistically).

macOS 10.13.6

Results from macOS are of particular interest because of the poor 1000 Hz USB keyboard event time-stamping results that seem to exist when using standard macOS keyboard event libraries, even in best case test conditions.

psychopy.event.getKeys

psychopy.iohub keyboard.getPresses()

The iohub results for macOS further illustrate that the extra event delay seen compared to other operating systems is likely because of macOS keyboard API overhead, even when the macos accessibility API is used. To be fair, this maybe a Python + macOS interaction, I have not tried timing a C program for example.

Python PsychHID() KbQueue*

On macOS, PsychHID is the only framework I have tested that provides 1 msec average keyboard event latency. At a very high level, I think this is because PsychHID uses HID event sniffing to detect keyboard events on macOS, bypassing the standard macOS keyboard handling APIs, and their associated delays, all together.

Python Source Code

The same test script was used to test all three event frameworks. The KB_BACKEND variable specified which backend to use for the test:

  • ‘psychopy’: Use psychopy.event.getKeys(timeStamped=True).
  • ‘iohub’: Start an iohub server and use io.devices.keyboard.getPresses().
  • ‘psychhid’: Start a Psychtoolbx HID KbQueue and use KbQueueGetEvent.
# -*- coding: utf-8 -*-
from __future__ import absolute_import, division, print_function
from psychopy import core, visual
import string
import serial
import numpy

KB_BACKEND = 'psychhid' # 'iohub', or 'psychhid'
# Serial Port of MilliKey.
mK_serial_port = 'COM3'
#mK_serial_port = '/dev/cu.usbmodem2998881'
# Number of keyboard events to generate.
kgen_count = 100
# min, max msec duration for generated keypresses.
min_dur, max_dur = 175,  300
# min, max msec MilliKey will wait before issuing the requested key press event.
min_delay, max_delay = 2, 14
possible_kgen_chars = string.ascii_lowercase + string.digits

getKeys = None
getTime = core.getTime
if KB_BACKEND == 'psychhid':
    from psychtoolbox import GetSecs
    getTime = GetSecs

if KB_BACKEND == 'psychopy':
    from psychopy import event
    def getKeys():
        return event.getKeys(timeStamped=True)
elif KB_BACKEND == 'iohub':
    global io
    from psychopy.iohub import launchHubServer
    io = launchHubServer()
    def getKeys():
        keys = io.devices.keyboard.getPresses()
        return [(k.key,k.time) for k in keys]
elif KB_BACKEND == 'psychhid':
    from psychtoolbox import PsychHID
    keys = [1] * 256
    PsychHID('KbQueueCreate', [], keys)
    PsychHID('KbQueueStart')
    # Seems like on Windows this must be called before first kb event
    # or first event has a time stamp error of ~ 20 msec.
    _ = GetSecs()
    def getKeys():
        keys = []
        while PsychHID('KbQueueFlush'):
            evt = PsychHID('KbQueueGetEvent')[0]
            if evt['Pressed']:
                K = chr(int(evt['CookedKey'])).lower()
                keys.append((K, evt['Time']))
        return keys

results = numpy.zeros(kgen_count, dtype=numpy.float64)

mk_sconn = serial.Serial(mK_serial_port, baudrate=128000, timeout=0.1)

win = visual.Window([800, 400])#, fullscr=True, allowGUI=False)
txt1 = "MilliKey Generating Key Press: [%s]\nOffset: %.1f msec. Duration: %d msec.\n%d of %d events."
msg = visual.TextStim(win, text=txt1)
dotPatch = visual.DotStim(win, color=(0.0, 1.0, 0.0), dir=270,
                          nDots=223, fieldShape='sqr',
                          fieldPos=(0.0, 0.0), fieldSize=1.5,
                          dotLife=50, signalDots='same',
                          noiseDots='direction', speed=0.01,
                          coherence=0.9)
msg.draw()
dotPatch.draw()
win.flip()

count = 0
while count < kgen_count:
    kchar = possible_kgen_chars[count % (len(possible_kgen_chars))]
    press_duration = int(numpy.random.randint(min_dur, max_dur))
    delay_evt_usec = int(numpy.random.randint(min_delay, max_delay))*1000
    evt_delay_sec = delay_evt_usec/1000.0/1000.0
    msg.setText(txt1%(kchar, delay_evt_usec/1000.0, press_duration,
                            count+1, kgen_count))
    dotPatch.draw()
    msg.draw()
    win.flip()

    # Instruct MilliKey device to:
    #  - generate a key press delay_evt_usec after receiving the KGEN command
    #  - generate key release event press_duration after press event is sent.
    kgen_cmd = "KGEN {} {} {}\n".format(kchar,
                                            press_duration,
                                            delay_evt_usec).encode()
    mk_sconn.write(kgen_cmd)
    mk_sconn.flush()
    # stime is the time the KGEN command was sent to the MilliKey device.
    # plus the event offset the device is using.
    stime = getTime()+evt_delay_sec

    # If next 3 lines are not used, PsychHID getKeys() seems to loose
    # occational keypress event on Windows 10. I could be using PsychHID()
    # incorrectly, need to investigate....
    dotPatch.draw()
    msg.draw()
    win.flip()

    # Keep checking for key press events until one is received
    kb_presses = getKeys()
    while not kb_presses:
        dotPatch.draw()
        msg.draw()
        win.flip()
        kb_presses = getKeys()

    kpress, ktime = kb_presses[0]
    if not kb_presses:
        raise RuntimeError("KGEN Timeout Error: No Key Press Event Detected.")

    # Ensure we got the key we were expecting.....
    if kchar == kpress:
        results[count] = ktime-stime
        count += 1
    else:
        txt = "Keyboard Key != Key Press Issued ([{}] vs [{}]). "
        txt += "Was a keyboard key or MilliKey button pressed during the test?"
        raise RuntimeError(txt.format(kchar, kpress))

    # Wait until after MilliKey has issued associated key release event.
    ctime = getTime()
    while getTime() - ctime < (press_duration/1000.0)*2:
        dotPatch.draw()
        msg.draw()
        win.flip()
        getKeys()

# Done test, close backend if needed
if KB_BACKEND == 'psychhid':
    PsychHID('KbQueueStop')
elif KB_BACKEND == 'iohub':
    io.quit()

mk_sconn.close()
win.close()

# Print Results
# Convert times to msec.
evt_results = results[:count] * 1000.0
print("%s Timestamp Accuracy Stats"%(KB_BACKEND))
print("\tCount: {}".format(evt_results.shape))
print("\tAverage: {:.3f} msec".format(evt_results.mean()))
print("\tMedian: {:.3f} msec".format(numpy.median(evt_results)))
print("\tMin: {:.3f} msec".format(evt_results.min()))
print("\tMax: {:.3f} msec".format(evt_results.max()))
print("\tStdev: {:.3f} msec".format(evt_results.std()))

# Plot Results
def createHistogram(data, title=None, xlabel="Time stamp Error (msec)",
                    ylabel="Probability density", nbins = 50):
    try:
        import matplotlib.pyplot as plt
        mu, sigma = data.mean(), data.std()
        # the histogram of the data
        n, bins, patches = plt.hist(data, nbins, density=1,
                                    facecolor='green', alpha=0.75)
        # add a 'best fit' line
        y = ((1 / (numpy.sqrt(2 * numpy.pi) * sigma)) *
             numpy.exp(-0.5 * (1 / sigma * (bins - mu))**2))
        plt.plot(bins, y, '--')
        plt.xlabel(xlabel)
        plt.ylabel(ylabel)
        if title is None:
            title = r"Histogram"
        title = r'$\mathrm{%s:}\ \mu=%.3f,\ \sigma=%.3f$'%(title,mu,sigma)
        plt.title(title)
        plt.grid(True)
        plt.tight_layout()
        plt.show()
    except:
        print("Could not create Histogram:\n\ttitle: {}\n\tlabel: {}\n\tylabel: {}".format(title, xlabel, ylabel))

createHistogram(evt_results,"%s\ Time\ Stamp\ Accuracy"%(KB_BACKEND))

core.quit()

Conclusions

Hopefully this post provides some useful information about the time stamping accuracy you should expect to achieve when using a 1000 Hz USB keyboard with different operating systems and keyboard event APIs. I hope it also illustrates how useful the MilliKey response box can be in testing 1000 Hz USB keyboard event delay and time stamping error.

If you are using macOS and want to really get events with 1 msec delay, then you must really use PsychHID, no matter what the response box maker claims (that includes us!).

When using Windows 10, psychopy.event.getKeys() provides acceptable time stamp accuracy when your experiment script is only checking for keyboard events during the trial. If the experiment updates the display frequently during a trial, psychopy.iohub or PsychHID should be used.

Looking forward to PsychHID based keyboard event monitoring being added to PsychoPy in the future.


PsychHID is really quite amazing; nice job Mario!

Happy Hacking!

Share:

Test Visual Stimulus Onset Timing using Keyboard Events !?

In this post we look at how to use the MilliKey DeLux light sensor upgrade to test visual stimulus onset timing using PsychoPy Coder by checking for a keyboard event.

MilliKey DeLux Light Sensor Upgrade

The MilliKey DeLux is an affordable and easy to use light sensor attachment for MilliKey response boxes. The MilliKey DeLux can generate light level driven 1000 Hz USB serial or keyboard events that are read by the same experiment software controlling visual stimulus presentation.

In a previous post about Testing PsychoPy event.waitKeys() Event Timing, we demonstrated how to use the MilliKey to test the keyboard event time stamping accuracy of the experiment software. This was achieved by sending USB serial messages from the experiment software instructing the MilliKey to generate keyboard events and then comparing the key press time to the time the serial message was sent. We found that, at least on Windows and Linux, PsychoPy can accurately time stamp MilliKey 1000 Hz keyboard events with one millisecond average delay and sub-millisecond variability.

One of the unique features of the MilliKey DeLux is that it can be configured to generate a letter ‘l’ key press event when the light sensor brightness crosses a trigger threshold level. This allows you to assess visual stimulus onset timing of an experiment by using the standard keyboard event handling functionality of the software being used.

PsychoPy Coder Test Script

Lets start by creating a simple PsychoPy Coder demo that repeatedly:

  • clears the screen to black
  • draws a white screen
  • waits for a keyboard press event
  • prints time difference between reported keyboard event and stimulus onset (white screen) times.
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from __future__ import absolute_import, division, print_function
from psychopy import visual, event, core

# Number of dark -> light display change tests to run
test_count = 10

# Create a full screen window and two full screen stim.
win = visual.Window(fullscr=True, screen=0)
white_stim = visual.ShapeStim(win, lineColor='white', fillColor='white',
                              vertices=((-1,-1),(1,-1),(1,1),(-1,1)))
dark_stim = visual.ShapeStim(win, lineColor='black', fillColor='black',
                             vertices=((-1,-1),(1,-1),(1,1),(-1,1)))

# draw / setup both stim
white_stim.draw()
dark_stim.draw()
win.flip()

# Manually press any key to start test
event.waitKeys(timeStamped=True)

print("trial\tRT\tkey_time\tflip_time\tkey")
# run test_count loops
count=0
while count < test_count:
    # Draw black screen and display it for about 0.5 seconds.
    dark_stim.draw()
    win.flip()
    core.wait(0.5)
	# Throw away any possible junk keys
	# prior to changing display 
    event.getKeys()
	
    # Draw white (target) stim and flip to it,
    # getting time flip occurred (stim onset)
    # time from psychopy.
    white_stim.draw()
    ftime = win.flip()

	# Key keyboard event and print time difference between
	# flip() time and keypress time.
    k, ktime = event.waitKeys(timeStamped=True)[0]
    kdelay = ktime - ftime    
    print("%d\t%.6f\t%.6f\t%.6f\t%s"%(count, kdelay, ftime, ktime, k))

    core.wait(0.25)
    count+=1

We have basically created the most simple of manual reaction time tests. Run the above script from PsychoPy Coder and press a key as quickly as you can each time the screen turns white.

For example, running the example and manually pressing the space key, I get output like:

trial	RT	key_time	flip_time	key
0	0.368140	1.687949	2.056089	space
1	0.171302	2.570518	2.741820	space
2	0.197275	3.256913	3.454187	space
3	0.198487	3.967356	4.165843	space
4	0.196070	4.679771	4.875841	space
5	0.215323	5.390571	5.605894	space
6	0.197549	6.118132	6.315682	space
7	0.193179	6.828715	7.021894	space
8	0.166193	7.535785	7.701978	space
9	0.193039	8.214774	8.407813	space

So far this has all been kind of boring right?

What if we could use this exact script to test the visual onset timing of the white screen? Lets do it using the MilliKey DeLux.

MilliKey DeLux Setup

For this test to work we first need to make sure the MilliKey DeLux is configured to:

  1. use a sensible threshold level for the monitor being Generate Keyboard Events.
  2. auto generate light sensor triggered keyboard events.

Lets do this using the LabHackers’ Device Manager application.

LabHackers' Device Manager DeLux Setting Tab

MilliKey DeLux light sensor settings are found on the DeLux tab of the LabHackers’ Device Manager application.

For this example use the following settings:
Trigger Type: ‘Keyboard Event’
Trigger Delay: 5 msec
Enable Background Processing: Checked / True

The Trigger Threshold setting can be set using the auto threshold feature of the Light Sensor Dialog. The following video is a screen capture from a Windows 10 computer running the Labhackers’ Device Manager application. It illustrates using the auto threshold feature to set the light sensor threshold to an appropriate level for the monitor being tested. The MilliKey DeLux hardware, which can not be seen in the video, was placed over the white rectangle of the test dialog on the LCD monitor.

Using LabHackers’ Device Manager to Configure a MilliKey DeLux light sensor.

After you have configured the MilliKey DeLux to generate ‘offline’ keyboard events, and have exited the LabHackers’ Device Manager, the MilliKey will generate a ‘l’ key press event 5 milliseconds (Trigger Delay) after screen brightness crosses the threshold level set. Each key press lasts 123 msec, at which time a key release event is sent by the MilliKey.

Note: Once the MilliKey DeLux is armed and ready to create keyboard events, if the light level hitting the is not controlled you may get a bunch of letter ‘l’s being typed on your computer. If you want to abort this test and get the MilliKey to stop generating keyboard events, unplug the MilliKey USB cable from the computer and reconnect it. This turns off the Enable Background Processing setting, which is disabled by default when you first connect the MilliKey to a computer.

Results

For this test we used a Windows 7 computer connected to a CRT monitor running at 60Hz. We used a CRT monitor because there is effectively no delay between when a video card starts a screen retrace and when the top left corner of the CRT monitor starts displaying update. In contrast LCD / TFT monitors all have a response time, which can range between 1 and 10+ msec depending on the monitor model.

trial   RT      key_time        flip_time       key
0       0.004892        4.412357        4.417248        l
1       0.005464        5.188762        5.194226        l
2       0.005237        5.976979        5.982216        l
3       0.005049        6.765145        6.770194        l
4       0.004792        7.553340        7.558132        l
5       0.005583        8.341535        8.347118        l
6       0.005390        9.129753        9.135143        l
7       0.005198        9.917925        9.923123        l
8       0.005766        10.694352       10.700118       l
9       0.005555        11.482574       11.488128       l

Since a CRT monitor was used, the time difference between win.flip() and when the CRT actually starts being updated should be 0.0 msec. Remember that we set the MilliKey DeLux to generate the key press event 5 msec after the stimulus onset was detected by the device hardware, so if win.flip() is accurate, the delay calculated by the test should be 0.0 after we subtract the 5 msec event delay specified in the device settings.

As the following table shows, they are:

00.0049-0.0001
10.00550.0005
20.00520.0002
30.00500.0000
40.0048-0.0002
50.00560.0006
60.00540.0004
70.00520.0002
80.00580.0008
90.00560.0006

It is important to note that using keyboard events as light sensor triggers is only suggested on systems that provide accurate keyboard event timing. Therefore we can not suggest this method for PsychoPy testing on macOS at this time. macOS users should use the Serial event mode, which we will demonstrate in a future post.

Next

The MilliKey DeLux keyboard event mode is very useful for testing stimulus onset timing in experiment software that you know provides accurate keyboard event timing and when you can not, or do not, want to add custom code to your experiment.

If the keyboard event time stamping accuracy of the software being used is poor, or you would rather not get light sensor triggers as keyboard events, then you can use USB Serial events instead. In the next MilliKey DeLux post, we will look at using the Serial trigger type to test stimulus onset timing in a PsychoPy Coder experiment.

Happy Hacking!

Share:

Testing PsychoPy event.waitKeys() Event Timing

As discussed in my last post, the LabHackers’ MilliKey button box includes the ability to generate 1000 Hz USB keyboard events after receiving a KGEN serial command. (BTW, the USB2TTL8 also supports the KGEN command.)

One of the main uses of the KGEN command is to test keyboard event timing from within your experiment software without the need to use extra external hardware. In this post we look at how to test the time stamping accuracy of the PsychoPy event.waitKeys() function using KGEN.

Why test?

You might be wondering why, given the MilliKey is a 1000 Hz response box that generates keyboard events with 1 msec average latency, it is important to be able to test keyboard event timing from within your experiment software? Why can’t we just subtract one msec from key press time stamps and assume everything is OK? Two reasons:

  1. 1000 Hz USB HID Keyboard latency can be influenced by the operating system and other USB hardware connected to the computer.
  2. Keyboard events are time stamped by the experiment software being used, not the response box hardware itself.

Therefore it is not safe to rely solely on the device specifications; timing really should be tested and verified within the experiment software being used.

Python Example

#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
IMPORTANT: Set the mK_serial_port variable below to the correct
           serial port string for the MilliKey to be tested.

Example of testing the latency of MilliKey keyboard events and event time 
stamping accuracy when using core.waitKeys() within a PsychoPy script.

While the demo runs, keyboard events are generated by the
MilliKey device based on serial commands sent from the script.

**DO NOT PRESS ANY KEYS OR MILLIKEY BUTTONS DURING THE TEST.**
"""
from __future__ import absolute_import, division, print_function

from psychopy import visual, event, core
import serial
import string
import numpy

# Serial Port of MilliKey.
mK_serial_port = 'COM138'

# Number of keyboard events to generate.
kgen_count = 100

# min, max msec duration for generated keypresses.
min_dur, max_dur = 175,  300

# Usec MilliKey will wait before issuing the requested key press event.
delay_evt_usec = 5000

possible_kgen_chars = string.ascii_lowercase + string.digits
results = numpy.zeros(kgen_count, dtype=numpy.float64)
mk_sconn = serial.Serial(mK_serial_port, baudrate=128000, timeout=0.1)

win = visual.Window([600, 400])
msg = visual.TextStim(win, text='Creating Key Press: [] (duration of {} msec)')
msg.draw()
win.flip()

evt_delay_sec = delay_evt_usec/1000.0/1000.0
kb_presses = ['']
count = 0
run = True
txt1 = "Generating Key Press: [{}] (duration of {} msec)\n{} of {} events."
while run is True and count < kgen_count:
    kchar = possible_kgen_chars[count % (len(possible_kgen_chars))]
    press_duration = int(numpy.random.randint(min_dur, max_dur))
    msg.setText(txt1.format(kchar, press_duration, count+1, kgen_count))
    msg.draw()
    win.flip()
    kgen_cmd = "KGEN {} {} {}\n".format(kchar,
                                            press_duration,
                                            delay_evt_usec).encode()
    mk_sconn.write(kgen_cmd)
    stime = core.getTime()+evt_delay_sec
    kb_presses = event.waitKeys(maxWait=1.0)
    ktime = core.getTime()
    if not kb_presses:
        raise RuntimeError("KGEN Timeout Error: No Key Press Event Detected.")
    for kbp in kb_presses:
        if kbp in ['escape', 'esc']:
            run = False
            break
    if run is False:
        continue
    kpress = kb_presses[0]
    if kchar == kpress:
        results[count] = ktime-stime
        core.wait((press_duration/1000.0)*1.2)
        count += 1
    else:
        txt = "Keyboard Key != Key Press Issued ({} vs {}). "
        txt += "Was a keyboard key or MilliKey button pressed during the test?"
        raise RuntimeError(txt.format(kchar, kpress))

mk_sconn.close()
win.close()

# Convert times to msec.
results = results[:count] * 1000.0

print("MilliKey Keyboard Press Event Delay (Timestamp Accuracy) Stats")
print("\tCount: {}".format(results.shape))
print("\tAverage: {:.3f} msec".format(results.mean()))
print("\tMedian: {:.3f} msec".format(numpy.median(results)))
print("\tMin: {:.3f} msec".format(results.min()))
print("\tMax: {:.3f} msec".format(results.max()))
print("\tStdev: {:.3f} msec".format(results.std()))

core.quit()

Test Results

Windows 10

Running PsychoPy 3.0.5.

MilliKey Keyboard Press Event Delay (Timestamp Accuracy) Stats
    Count: (100,)
    Average: 1.012 msec
    Median: 0.982 msec
    Min: 0.509 msec
    Max: 1.633 msec
    Stdev: 0.303 msec

Linux (Mint 18.3)

 MilliKey Keyboard Press Event Delay (Timestamp Accuracy) Stats
	Count: (100,)
	Average: 0.899 msec
	Median: 0.898 msec
	Min: 0.350 msec
	Max: 1.499 msec
	Stdev: 0.309 msec

When a PsychoPy experiment script, running on Windows or Linux, can simply wait for MilliKey keyboard events, there is an average difference of ~ 1.0 msec between when the KGEN command is sent by the script and when the keyboard event is received by PsychoPy. If we subtract the time it takes to send a KGEN command, which is 0.3 msec on average, then the average MilliKey key press latency in this test would be about 1.0 – 0.3 = 0.7 msec.

macOS 10.13.6

Running PsychoPy 3.0.5.

MilliKey Keyboard Press Event Delay (Timestamp Accuracy) Stats
Count: (100,)
Average: 4.755 msec
Median: 4.608 msec
Min: 3.679 msec
Max: 18.594 msec
Stdev: 1.445 msec

The average keyboard delay from the same script running on macOS is longer and also more variable than other operating systems tested. It seems, for example, that the first keyboard event received after the python interpreter starts is delayed by an extra 10 or more milliseconds. Results with first event removed:

 MilliKey Keyboard Press Event Delay (Timestamp Accuracy) Stats
    Count: (99,)
    Average: 4.615 msec
    Median: 4.597 msec
    Min: 3.679 msec
    Max: 5.939 msec
    Stdev: 0.394 msec

Remember that the USB serial latency on macOS is under 1 msec, as would be expected, so the additional delay in receiving keyboard events seems to be specific to processing done by the macOS operating system or the underlying keyboard event handling libraries used.

This pattern of results does not seem to be specific to PsychoPy event.waitKeys(). I have also tested using pygame, glfw, pyqt4 and pyqt5 and all show a similar pattern of results across operating systems.

If you have a LabHackers device, please let us know if you your results are consistent with this or not. For that matter, if you have done actual keyboard event delay testing using any method on macOS, it would be great to see the results.

Notes

It is important to recognize that using KGEN to test keyboard event timing has the following caveats:

  1. Timing results can only be generalized to other 1000 Hz USB HID Keyboard hardware that has a 0 millisecond debounce time.
  2. Given the sub millisecond jitter that will occur when sending each serial KGEN command, test results should be considered accurate to within 0.5 msec, even though the resolution of the timing is much greater.

Even with these in mind, we think KGEN is pretty useful and hope you do too.

Happy Hacking!

Share:

Generating Keyboard Events using USB Serial Commands

LabHackers’ MilliKey and USB2TTL8 devices can generate a USB Keyboard event after receiving a USB serial command from the software it is connected to.

This post explains the LabHackers’ KGEN command and illustrates how to use it in a simple Python script.

KGEN Serial Command

Both the MilliKey and USB2TTL8 support the KGEN serial command. The KGEN command tells the device to generate a key press event for the specified key and duration. The USB keyboard event is identical an event generated from an actual MilliKey button press – release.

Format

KGEN key duration [offset]\n

where:

  • key: The key to use for the event. A, B, C, …
  • duration: Number milliseconds between when device sends key press and release events.
  • offset: Optional number of microseconds device will wait before sending the key press event. 0 = send as soon as KGEN command is received / processed. Default is 0 usec.

For example, send the following serial message to your LabHackers’ device to have it generate a ‘z’ key press for 123 milliseconds:

KGEN Z 123\n

or, to press the Up arrow key for 300 milliseconds, 2.5 milliseconds after receiving the command:

KGEN UP 300 2500\n

Python Example

To keep this example as short as possible, it uses KGEN to generate keyboard events but does /not/ have code to collect them.

To ‘see’ the keyboard events generated from this example, start the example and immediately switch focus to a text editor window so that you can see the keyboard events in action.

#!/usr/bin/env python
# -*- coding: utf-8 -*-
from __future__ import division, print_function

import serial
import time

# Serial Port of LabHackers' Device.
serial_address = 'COM138'
#serial_address = '/dev/cu.usbmodem3775821'

# press duration
press_duration = 250

# seconds to sleep between key press
iti = 0.5

# List of characters to use in KGEN.
key_list = "these key events are KGEN generated."

# Open USB Serial connection to LabHackers' device
try:
    s = serial.Serial(serial_address, baudrate=128000, timeout=0.1)
except serial.SerialException as e:
    print("Check SERIAL_PORT variable is correct for device being tested.")
    raise e

# Give user 5 seconds to switch to a text editor program.
time.sleep(5.0)

for key in key_list:
    # KGEN uses 'SPACE' to indicate ' ' key.
    if key == ' ':
        key = 'SPACE'

    # for each character in key_list, tell the LabHackers device to immidiately
    # generate a press event for that key lasting press_duration msec.
    s.write("KGEN {} {} {}\n".format(key, press_duration, 0).encode())

    # wait iti seconds before issueing next KGEN command
    time.sleep(iti)

# close serial connection
s.close()

In an upcoming post we will use the KGEN command to test keyboard event latency and event time stamping accuracy using PsychoPy.

Happy Hacking!

Share:

Testing round trip USB Serial latency using Python

One important consideration when using a 1000 Hz USB Serial device interface is the delay from when a Serial message is sent to the device to the time when the Serial reply is received by the program. We call this the round trip, or end to end, USB Serial latency of the device.

Here is a simple Python script that tests the round trip USB Serial latency of the first detected MilliKey or USB2TTL8 device.

This example uses a fixed Serial port address that needs to be manually changed to the correct Serial port of the device being tested. For an example of how to programatically detect your LabHackers’ device serial port from within Python checkout this post.

#!/usr/bin/env python
# -*- coding: utf-8 -*-
from __future__ import division, print_function

import serial
import numpy
from timeit import default_timer as getTime

SERIAL_PORT = 'COM138'
ITERATION_COUNT = 10000

try:
    sconn = serial.Serial(SERIAL_PORT, baudrate=128000, timeout=0.1)
except serial.SerialException as e:
    print("Check SERIAL_PORT variable is correct for device being tested.")
    raise e
    
#clear anything in serial rx
while sconn.readline():
    pass

# Test time it takes to send a serial message to the labhackers device
# and receive the serial reply.
results = numpy.zeros(ITERATION_COUNT, dtype=numpy.float64)
for i in range(ITERATION_COUNT):
    tx_time = getTime()
    sconn.write(b"PING\n")
    r = sconn.readline()
    rx_time = getTime()
    if r:
        results[i] = rx_time-tx_time
    else:
        raise RuntimeError("Serial RX Timeout.")

sconn.close()

# Convert times to msec.
results = results * 1000.0

print("LabHackers' USB Serial Rx - Tx Latency Stats")
print("\tCount: {}".format(results.shape))
print("\tAverage: {:.3f} msec".format(results.mean()))
print("\tMedian: {:.3f} msec".format(numpy.median(results)))
print("\tMin: {:.3f} msec".format(results.min()))
print("\tMax: {:.3f} msec".format(results.max()))
print("\tStdev: {:.3f} msec".format(results.std()))

Here is the output from the script when run on Windows, Linux and macOS, showing that the average round trip USB serial delay of a LabHackers’ device is well under 1 msec on all operating systems.

Windows 10

Run using PsychoPy 3.0.5 Coder IDE

LabHackers' USB Serial Rx - Tx Latency Stats
    Count: (10000,)
    Average: 0.391 msec
    Median: 0.379 msec
    Min: 0.352 msec
    Max: 1.840 msec
    Stdev: 0.055 msec

Linux (Mint 18.3)

LabHackers' USB Serial Rx - Tx Latency Stats
	Count: (10000,)
	Average: 0.320 msec
	Median: 0.313 msec
	Min: 0.253 msec
	Max: 0.583 msec
	Stdev: 0.026 msec
	

macOS 10.13.6

Run using PsychoPy 3.0.5 Coder IDE

LabHackers’ USB Serial Rx – Tx Latency Stats
Count: (10000,)
Average: 0.708 msec
Median: 0.719 msec
Min: 0.559 msec
Max: 1.501 msec
Stdev: 0.059 msec

Happy Hacking!

Share:

Detecting LabHackers’ Serial port addresses using Python

LabHackers’ MilliKey and USB2TTL8 devices have a USB Serial interface that is assigned a unique address by the operating system the first time the device is connected to a computer. The LabHackers’ Device Manager application can be used to view the serial port address assigned to a device.

However, the same LabHackers’ device will likely be assigned a different serial port address when it is connected to a different computer, so it is also useful to be able to identify LabHackers’ device serial port addresses from within your Python script.

Finding available Serial ports

If we want to connect to a LabHackers’ device USB Serial interface, but do not know the device’s serial port address, the first thing we need to do is find it.

Lets start by finding the serial port address for all serial devices connected to the computer.

import serial
import os

def get_serial_ports():
    """
    Return list of serial port addresses that have be openned.
    """
    if os.name == 'nt':  # Windows
        available = []
        for i in range(1, 512):
            try:
                sport = 'COM%d'%(i)
                s = serial.Serial(sport, baudrate=128000)
                available.append(sport)
                s.close()
            except (serial.SerialException, ValueError):
                pass
        return available
    else:  # macOS and Linux
        from serial.tools import list_ports
        return [port[0] for port in list_ports.comports()]

get_serial_ports() returns a list of all serial port addresses on the computer that have a device connected to them. For example:

serial_ports = get_serial_ports()
print(serial_ports)

run on a Windows computer with two connected serial ports, will return something like:

 ['COM2', 'COM9']

Note: get_serial_ports() returns all the serial devices connected to the computer.

Detecting LabHackers’ device Serial ports

Next we need to find which, if any, of the serial addresses are connected to a LabHackers’ MilliKey or USB2TTL8 device. Building on get_serial_ports() , we can get a list of the LabHackers’ ports by checking the responds of each serial port when “PING\n” is sent.

 def get_labhackers_ports():
    """
    Return list of connected LabHackers' device serial port addresses.
    """
    devices = []
    for p in get_serial_ports():
        s = serial.Serial(p, baudrate=128000, timeout=0.1)
        s.write(b"PING\n")
        rx = s.readline()
        if rx:
            rx = str(rx)
            if rx.find('MilliKey')>=0 or rx.find('USB2TTL8')>=0:
                devices.append(p)
        s.close()
    return devices

For example:

labhacker_ports = get_labhackers_ports()
print(labhacker_ports)

run on a Windows computer with one LabHackers’ device connected will return one serial port address:

['COM9']

Putting it together

Here is a complete version of the code.

#!/usr/bin/env python
# -*- coding: utf-8 -*-
from __future__ import print_function

import serial
import os

def get_serial_ports():
    """
    Return list of serial port addresses that have be openned.
    """
    if os.name == 'nt':  # Windows
        available = []
        for i in range(1, 512):
            try:
                sport = 'COM%d'%(i)
                s = serial.Serial(sport, baudrate=128000)
                available.append(sport)
                s.close()
            except (serial.SerialException, ValueError):
                pass
        return available
    else:  # macOS and Linux
        from serial.tools import list_ports
        return [port[0] for port in list_ports.comports()]

def get_labhackers_ports():
    """
    Return list of connected LabHackers' device serial port addresses.
    """
    devices = []
    for p in get_serial_ports():
        s = serial.Serial(p, baudrate=128000, timeout=0.1)
        s.write(b"PING\n")
        rx = s.readline()
        if rx:
            rx = str(rx)
            if rx.find('MilliKey')>=0 or rx.find('USB2TTL8')>=0:
                devices.append(p)
        s.close()
    return devices

if __name__ == '__main__':
    print(get_labhackers_ports())

Happy Hacking!

Share: