{"id":134,"date":"2020-12-13T13:11:51","date_gmt":"2020-12-13T17:11:51","guid":{"rendered":"http:\/\/blog.labhackers.com\/?p=134"},"modified":"2020-12-21T12:25:04","modified_gmt":"2020-12-21T16:25:04","slug":"testing-psychopy-event-waitkeys-event-timing","status":"publish","type":"post","link":"https:\/\/blog.labhackers.com\/?p=134","title":{"rendered":"Testing PsychoPy event.waitKeys() Event Timing"},"content":{"rendered":"\n<p><sup>Originally posted on February 27, 2019.<\/sup><\/p>\n\n\n\n<figure class=\"wp-block-embed is-type-rich is-provider-embed-handler wp-block-embed-embed-handler wp-embed-aspect-4-3 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"Testing PsychoPy event.waitKeys() timing with a MilliKey response box\" width=\"750\" height=\"563\" src=\"https:\/\/www.youtube.com\/embed\/eV2YVOG2YHM?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen><\/iframe>\n<\/div><\/figure>\n\n\n\n<p>As discussed&nbsp;<a href=\"https:\/\/blog.labhackers.com\/?p=355\" target=\"_blank\" rel=\"noreferrer noopener\">in my last post<\/a>, the LabHackers\u2019&nbsp;<a href=\"https:\/\/www.labhackers.com\/millikey.html\" target=\"_blank\" rel=\"noreferrer noopener\">MilliKey&nbsp;<\/a>button box includes the ability to generate 1000 Hz USB keyboard events after receiving a KGEN serial command. (BTW, the&nbsp;<a href=\"https:\/\/www.labhackers.com\/usb2ttl8.html\" target=\"_blank\" rel=\"noreferrer noopener\">USB2TTL8&nbsp;<\/a>also supports the KGEN command.)<\/p>\n\n\n\n<p>One of the main uses of the KGEN command is to test keyboard event timing from within your experiment software without the need to use extra external hardware. In this post we look at how to test the time stamping accuracy of the&nbsp;<a href=\"http:\/\/www.psychopy.org\/\" target=\"_blank\" rel=\"noreferrer noopener\">PsychoPy<\/a>&nbsp;event.waitKeys() function using KGEN.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Why test?<\/h2>\n\n\n\n<p>You might be wondering why, given the MilliKey is a 1000 Hz response box that generates keyboard events with 1 msec average latency, it is important to be able to test keyboard event timing from within your experiment software? Why can\u2019t we just subtract one msec from key press time stamps and assume everything is OK? Two reasons:<\/p>\n\n\n\n<ol class=\"wp-block-list\"><li>1000 Hz USB HID Keyboard latency can be influenced by the operating system and other USB hardware connected to the computer.<\/li><li>Keyboard events are time stamped by the experiment software being used, not the response box hardware itself.<\/li><\/ol>\n\n\n\n<p>Therefore it is not safe to rely solely on the device specifications; timing really should be tested and verified within the experiment software being used.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Python Example<\/h2>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"python\" class=\"language-python\">#!\/usr\/bin\/env python\n# -*- coding: utf-8 -*-\n\"\"\"\nIMPORTANT: Set the mK_serial_port variable below to the correct\n           serial port string for the MilliKey to be tested.\n\nExample of testing the latency of MilliKey keyboard events and event time \nstamping accuracy when using core.waitKeys() within a PsychoPy script.\n\nWhile the demo runs, keyboard events are generated by the\nMilliKey device based on serial commands sent from the script.\n\n**DO NOT PRESS ANY KEYS OR MILLIKEY BUTTONS DURING THE TEST.**\n\"\"\"\nfrom __future__ import absolute_import, division, print_function\n\nfrom psychopy import visual, event, core\nimport serial\nimport string\nimport numpy\n\n# Serial Port of MilliKey.\nmK_serial_port = 'COM138'\n\n# Number of keyboard events to generate.\nkgen_count = 100\n\n# min, max msec duration for generated keypresses.\nmin_dur, max_dur = 175,  300\n\n# Usec MilliKey will wait before issuing the requested key press event.\ndelay_evt_usec = 5000\n\npossible_kgen_chars = string.ascii_lowercase + string.digits\nresults = numpy.zeros(kgen_count, dtype=numpy.float64)\nmk_sconn = serial.Serial(mK_serial_port, baudrate=128000, timeout=0.1)\n\nwin = visual.Window([600, 400])\nmsg = visual.TextStim(win, text='Creating Key Press: [] (duration of {} msec)')\nmsg.draw()\nwin.flip()\n\nevt_delay_sec = delay_evt_usec\/1000.0\/1000.0\nkb_presses = ['']\ncount = 0\nrun = True\ntxt1 = \"Generating Key Press: [{}] (duration of {} msec)\\n{} of {} events.\"\nwhile run is True and count &lt; kgen_count:\n    kchar = possible_kgen_chars[count % (len(possible_kgen_chars))]\n    press_duration = int(numpy.random.randint(min_dur, max_dur))\n    msg.setText(txt1.format(kchar, press_duration, count+1, kgen_count))\n    msg.draw()\n    win.flip()\n    kgen_cmd = \"KGEN {} {} {}\\n\".format(kchar,\n                                            press_duration,\n                                            delay_evt_usec).encode()\n    mk_sconn.write(kgen_cmd)\n    stime = core.getTime()+evt_delay_sec\n    kb_presses = event.waitKeys(maxWait=1.0)\n    ktime = core.getTime()\n    if not kb_presses:\n        raise RuntimeError(\"KGEN Timeout Error: No Key Press Event Detected.\")\n    for kbp in kb_presses:\n        if kbp in ['escape', 'esc']:\n            run = False\n            break\n    if run is False:\n        continue\n    kpress = kb_presses[0]\n    if kchar == kpress:\n        results[count] = ktime-stime\n        core.wait((press_duration\/1000.0)*1.2)\n        count += 1\n    else:\n        txt = \"Keyboard Key != Key Press Issued ({} vs {}). \"\n        txt += \"Was a keyboard key or MilliKey button pressed during the test?\"\n        raise RuntimeError(txt.format(kchar, kpress))\n\nmk_sconn.close()\nwin.close()\n\n# Convert times to msec.\nresults = results[:count] * 1000.0\n\nprint(\"MilliKey Keyboard Press Event Delay (Timestamp Accuracy) Stats\")\nprint(\"\\tCount: {}\".format(results.shape))\nprint(\"\\tAverage: {:.3f} msec\".format(results.mean()))\nprint(\"\\tMedian: {:.3f} msec\".format(numpy.median(results)))\nprint(\"\\tMin: {:.3f} msec\".format(results.min()))\nprint(\"\\tMax: {:.3f} msec\".format(results.max()))\nprint(\"\\tStdev: {:.3f} msec\".format(results.std()))\n\ncore.quit()<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">Test Results<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Windows 10<\/h3>\n\n\n\n<p>Running PsychoPy 3.0.5.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">MilliKey Keyboard Press Event Delay (Timestamp Accuracy) Stats\n    Count: (100,)\n    Average: 1.012 msec\n    Median: 0.982 msec\n    Min: 0.509 msec\n    Max: 1.633 msec\n    Stdev: 0.303 msec<\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Linux (Mint 18.3)<\/h3>\n\n\n\n<pre class=\"wp-block-preformatted\"> MilliKey Keyboard Press Event Delay (Timestamp Accuracy) Stats\n\tCount: (100,)\n\tAverage: 0.899 msec\n\tMedian: 0.898 msec\n\tMin: 0.350 msec\n\tMax: 1.499 msec\n\tStdev: 0.309 msec<\/pre>\n\n\n\n<p>When a PsychoPy experiment script, running on Windows or Linux, can simply wait for MilliKey keyboard events, there is an average difference of ~ 1.0 msec between when the KGEN command is sent by the script and when the keyboard event is received by PsychoPy. If we subtract the time it takes to send a KGEN command, which is 0.3 msec on average, then the average MilliKey key press latency in this test would be about 1.0 \u2013 0.3 = 0.7 msec.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">macOS 10.13.6<\/h3>\n\n\n\n<p>Running PsychoPy 3.0.5.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">MilliKey Keyboard Press Event Delay (Timestamp Accuracy) Stats\nCount: (100,)\nAverage: 4.755 msec\nMedian: 4.608 msec\nMin: 3.679 msec\nMax: 18.594 msec\nStdev: 1.445 msec<\/pre>\n\n\n\n<p>The average keyboard delay from the same script running on macOS is longer and also more variable than other operating systems tested. It seems, for example, that the first keyboard event received after the python interpreter starts is delayed by an extra 10 or more milliseconds. Results with first event removed:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"> MilliKey Keyboard Press Event Delay (Timestamp Accuracy) Stats\n    Count: (99,)\n    Average: 4.615 msec\n    Median: 4.597 msec\n    Min: 3.679 msec\n    Max: 5.939 msec\n    Stdev: 0.394 msec<\/pre>\n\n\n\n<p>Remember that the&nbsp;<a href=\"http:\/\/blog.labhackers.com\/?p=25\" target=\"_blank\" rel=\"noreferrer noopener\">USB serial latency on macOS<\/a>&nbsp;is under 1 msec, as would be expected, so the additional delay in receiving keyboard events seems to be specific to processing done by the macOS operating system or the underlying keyboard event handling libraries used.<\/p>\n\n\n\n<p>This pattern of results does not seem to be specific to PsychoPy event.waitKeys(). I have also tested using pygame, glfw, pyqt4 and pyqt5 and all show a similar pattern of results across operating systems.<\/p>\n\n\n\n<p>If you have a LabHackers device, please let us know if you your results are consistent with this or not. For that matter, if you have done actual keyboard event delay testing using any method on macOS, it would be great to see the results.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Notes<\/h2>\n\n\n\n<p><strong>I<\/strong>t is important to recognize that using KGEN to test keyboard event timing has the following caveats:<\/p>\n\n\n\n<ol class=\"wp-block-list\"><li>Timing results can only be generalized to other 1000 Hz USB HID Keyboard hardware that has a 0 millisecond debounce time.<\/li><li>Given the sub millisecond jitter that will occur when sending each serial KGEN command, test results should be considered accurate to within 0.5 msec, even though the resolution of the timing is much greater.<\/li><\/ol>\n\n\n\n<p>Even with these in mind, we think KGEN is pretty useful and hope you do too.<\/p>\n\n\n\n<p>Happy Hacking!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Originally posted on February 27, 2019. As discussed&nbsp;in my last post, the LabHackers\u2019&nbsp;MilliKey&nbsp;button box includes the ability to generate 1000 Hz USB keyboard events after receiving a KGEN serial command. (BTW, the&nbsp;USB2TTL8&nbsp;also supports the KGEN command.) One of the main uses of the KGEN command is to test keyboard event timing from within your experiment&hellip; <a class=\"more-link\" href=\"https:\/\/blog.labhackers.com\/?p=134\">Continue reading <span class=\"screen-reader-text\">Testing PsychoPy event.waitKeys() Event Timing<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":"","footnotes":""},"categories":[17,4],"tags":[16,8,18,10,14,11],"class_list":["post-134","post","type-post","status-publish","format-standard","hentry","category-keyboard-event-timing","category-millikey-response-box","tag-keyboard-events","tag-millikey","tag-psychopy","tag-python","tag-timing","tag-usb-serial","entry"],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/blog.labhackers.com\/index.php?rest_route=\/wp\/v2\/posts\/134","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.labhackers.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.labhackers.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.labhackers.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.labhackers.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=134"}],"version-history":[{"count":6,"href":"https:\/\/blog.labhackers.com\/index.php?rest_route=\/wp\/v2\/posts\/134\/revisions"}],"predecessor-version":[{"id":446,"href":"https:\/\/blog.labhackers.com\/index.php?rest_route=\/wp\/v2\/posts\/134\/revisions\/446"}],"wp:attachment":[{"href":"https:\/\/blog.labhackers.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=134"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.labhackers.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=134"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.labhackers.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=134"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}