{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Connecting browser events to widgets with `Event`"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## What are browser events?\n",
"\n",
"In a nutshell, they are keyboard or mouse events in the browser. They are also called _DOM events_, and DOM is short for *Document Object Model*, the representation of the components of a web page displayed in a browser. When you interact with something in a browser (like a notebook cell) events corresponding to keypresses and mouse clicks are generated by the browser and handled by the notebook's JavaScript. \n",
"\n",
"## The `Event` widget\n",
"\n",
"`Event` is a widget that represents and provides a Python interface to mouse and keyboard interactions with the widgets, like `Image` or `Label`, that are displayed in the browser.\n",
"\n",
"Unlike most other widgets, the `Event` is not itself displayed. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from ipywidgets import Label, HTML, HBox, Image, VBox, Box, HBox\n",
"from ipyevents import Event \n",
"from IPython.display import display"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Short example\n",
"\n",
"The example below attaches a listener that responds to mouse clicks and key down on an `Label` widget and displays the event information in a `HTML` widget.\n",
"\n",
"Note that the information returned in the event depends on whether the event was a mouse or a keyboard event."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"l = Label('Click or type on me!')\n",
"l.layout.border = '2px solid red'\n",
"\n",
"h = HTML('Event info')\n",
"d = Event(source=l, watched_events=['click', 'keydown', 'mouseenter', 'touchmove'])\n",
"\n",
"def handle_event(event):\n",
" lines = ['{}: {}'.format(k, v) for k, v in event.items()]\n",
" content = ' '.join(lines)\n",
" h.value = content\n",
"\n",
"d.on_dom_event(handle_event)\n",
" \n",
"display(l, h)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### If you watch keyboard events then key presses are *not* passed on to the notebook \n",
"\n",
"This is to prevent the user from, e.g., inadvertently cutting a cell by pressing `x` over a widget that is catching key events."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Adding a new view of a widget attaches listeners to the view\n",
"\n",
"Clicking on the label below will update the HTML widget above, just like clicking (or typing) on the first view of the widget did."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"l"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### You can get some information about the HTML element to which the event listener is attached\n",
"\n",
"One of the items returned in the event information dictionary is called `target` and contains the name, ID and classes on the HTML element to which the event listener is attached."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"l2 = HTML('''\n",
" \n",
" \n",
" \n",
" \n",
"''')\n",
"\n",
"h2 = HTML('Event info')\n",
"d2 = Event(source=l2, watched_events=['click'])\n",
"\n",
"def handle_event(event):\n",
" h2.value = (f\"You clicked a {event['target']['tagName']} with \"\n",
" f\"ID {event['target']['id']} \" \n",
" f\"and classes: {event['target']['className']}\")\n",
"\n",
"d2.on_dom_event(handle_event)\n",
"\n",
"display(l2, h2)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Watchable events"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can get a list of the events to which a `DOMListener` can respond from the `supported_key_events` and `supported_mouse_events` properties."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"print('Key events: ', d.supported_key_events)\n",
"print('Mouse events: ', d.supported_mouse_events)\n",
"print('Touch events:', d.supported_touch_events)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### More about the watchable events\n",
"\n",
"For more about these events, see the [MDN list of DOM events](https://developer.mozilla.org/en-US/docs/Web/Events)\n",
"\n",
"**Keyboard events**\n",
"\n",
"The keyboard events are triggered only if the mouse is over the widget.\n",
"+ `keydown` is triggered when a key is pressed down or if the key is held down and repeats.\n",
"+ `keyup` is triggered when a key is released.\n",
"\n",
"**Mouse events**\n",
"\n",
"Mouse events trigger only when the mouse is over the watched widget. \n",
"\n",
"+ `click` currently, triggered if any mouse button is pushed, but in the near future will trigger only when the primary mouse button is clicked.\n",
"+ `auxclick` triggers when a non-primary mouse button is clicked.\n",
"+ `dblclick` triggers when there is a double click on the element.\n",
"+ `mouseenter` triggers when the mouse enters the element.\n",
"+ `mouseleave` triggers when the mouse leaves the element.\n",
"+ `mousedown` triggers when any mouse button is depressed.\n",
"+ `mouseup` triggers when any mouse button is released.\n",
"+ `mousemove` triggers every time the mouse moves over the widget. *Enabling this will generate a large volume of events between the front end and the back end.*\n",
"+ `wheel` trigger when the user scrolls in any direction. *Note that watching this disables scrolling of the notebook while the mouse is over the watched widget.*\n",
"\n",
"**Touch events**\n",
"\n",
"Touch events trigger on touch-enabled devices when something (e.g. a finger or a stylus) touches the screen. Touch events include information about all of the touch points on the screen, not just touch points within the watched widget. The touch event interface assumes that there may be multiple touch points.\n",
"\n",
"+ `touchstart` triggers when the watched element is touched.\n",
"+ `touchend` triggers when the touch point that triggered `touchstart` is removed from the screen or if the touch moves off the screen.\n",
"+ `touchmove` triggers when one or more of the touch points changes even if the touch is no longer over the target widget.\n",
"+ `touchcancel` triggers when something happens to inteerupt the touch (e.g. a dialog box pops up or there are too many touches on the screen)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Optional filter useful if watching keyboard and mouse events\n",
"\n",
"The `DOMListener` widget has a property, `ignore_modifier_key_events`, which, if set to `True`, causes key events for the modifier keys `Shift`, `Control`, `Alt`, and `Meta`, to be ignored. This is useful if you want to watch keyboard events and mouse events, and you want to handle mouse events with modifier keys pressed differently than plain mouse events.\n",
"\n",
"Normally, an action like Shift-click generates *two* events: a `keydown` for the `Shift` followed by a `click` (with its `shiftKey` set to `True`). While the unwanted `Shift` event could be filtered out on the python side, `ignore_modifier_key_events` provides a convenient way to avoid the extra events."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Choose wisely\n",
"\n",
"Think carefully about which events to monitor and experiment frequently. If, for example, you want to monitor both single and double mouse clicks, you probably do *not* want to watch for both `click` and `dblclick`. The reason is that when you double click a long string of events is generated:\n",
"\n",
"```\n",
"mousedown\n",
"mouseup\n",
"click\n",
"mousedown\n",
"mouseup\n",
"click\n",
"dblclick\n",
"```\n",
"\n",
"If you watch both `click` and `dblclick` then you will get three events per double click. \n",
"\n",
"One way to handle this case would be to watch only for `click`, and to check the `timeStamp` property of the event dictionary to see how close together in time the clicks are. If they are close enough, consider two consecutive clicks to be a double click, otherwise consider it to be two separate clicks. `timeStamp` is measured in milliseconds."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## The `event` dictionary\n",
"\n",
"The single argument passed to a callback registered with `on_dom_event` is a dictionary. The entries in the dictionary depend on what kind of event occurs, but always contains the key `type` whose value is the type of event , e.g. `'click'` or `'keydown`', and the key `target` whose value includes information about the DOM element to which the event watcher is attached.\n",
"\n",
"### Common to all events\n",
"\n",
"All events have these keys in the event dictionary:\n",
"\n",
"+ `altKey`: `bool`, `True` if the `alt` key was down when the event occurred, otherwise `False`.\n",
"+ `ctrlKey`: `bool`, `True` if the `control` key was down when the event occurred, otherwise `False`.\n",
"+ `metaKey`: `bool`, `True` if the `meta` key (the Windows key on Windows, the command key (cloverleaf) on the mac) was down when the event occurred, otherwise `False`.\n",
"+ `shiftKey`: `bool`, `True` if the `Shift` key was down when the event occurred, otherwise `False`.\n",
"+ `type`: `str`, type of event (e.g. `'click'`) that occurred.\n",
"+ `timeStamp`: `float`, the time (in milliseconds) at which the event occurred. \n",
"+ `target`: `dict` with the ID, CSS classes, and the name of the HTML tag of the element to which the listener is attached."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### For keyboard events only\n",
"\n",
"A key-related event has these keys in addition to the common ones above:\n",
"\n",
"+ `key`: `str`, the key that was pressed. Note that if the shift key was held down when the key was pressed, and the key pressed was a letter, this will be an upper case letter.\n",
"+ `code`: `str`, also indicates the key pressed. the value begins with the string `Key`. This may contain additional information about the keypress, for example whether it was the left or right key pressed if the key appears multiple times on the keyboard.\n",
"+ `location`: `int`, the location of the key on the device (typically `0`).\n",
"+ `repeat`: `bool`, `True` if this event is the result of a key being held down.\n",
"\n",
"For more details, see the [MDN documentation of HTML keyboard events](https://developer.mozilla.org/en-US/docs/Web/API/KeyboardEvent#Properties)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### For mouse events only\n",
"\n",
"A mouse-related event has these keys in addition to the common ones above. They are broken into groups below to simplify the discussion. For more details about these properties, see the [MDN documentation of mouse events](https://developer.mozilla.org/en-US/docs/Web/API/MouseEvent).\n",
"\n",
"#### Which mouse button was pressed\n",
"+ `button`: The button that was pushed or released for an event related to a mouse button push.\n",
"+ `buttons`: The buttons that were depressed when a mouse-related event occurs whether or not the event was a click. For example, when the mouse moves onto a widget being watched for `mouseenter`, then `buttons` indicates which, if any, mouse buttons are being help down.\n",
"\n",
"### Location of the mouse event\n",
"\n",
"All locations are integers.\n",
"\n",
"These attributes are standard properties of HTML mouse events.\n",
"\n",
"+ `screenX`, `screenY`: Coordinates of the event relative to the user's whole screen.\n",
"+ `clientX`, `clientY` and `x`, `y`: Coordinates of the event relateive to the visible part of the web page in the browser. \n",
"+ `pageX`, `pageY`: Coordinates of the event relative to the top of the page currently being displayed.\n",
"+ `offsetX`, `offsetY`: Offset of this DOM element relative to its parents.\n",
"+ `relativeX`, `relativeY`: The location of the mouse relative to the current element (i.e. widget). This emulates the functinoality of the `layerX/Y` attributes implemented in many browsers but not part of any standard.\n",
"\n",
"### Additional location for some types of widgets\n",
"\n",
"+ `dataX`, `dataY`: Position of the mouse event in the underlying data object. For example, a click on an `Image` widget returns as `dataX` and `dataY` the coordinates of the click in the image. This location should be regarded as the *approximate* location the user intended given the difficulty of clicking accurately at the pixel level.\n",
"\n",
"For widgets (like `Label`) for which there is no underlying array object these properties are not returned.\n",
"\n",
"### Touch events only\n",
"\n",
"Touch events do not store locations directly. Instead, they store a list of touch points, each of which has location information. The top-level attributes unique to a touch event are\n",
"\n",
"+ `changedTouches`: a list of the touch points which have changed.\n",
"+ `targetTouches`: a list of all of the touch points which are still on the surface and which originated in the target widget.\n",
"+ `touches`: a list of all of the touch points on the surface, regardless of whether they have changed or originated in the target widget.\n",
"\n",
"Each of the touches in those lists contains:\n",
"\n",
"+ `identifier`: A unique identifier of the touch point.\n",
"+ The [position attributes for mouse events](#Location-of-the-mouse-event) and the [additional position information](#Additional-location-for-some-types-of-widgets) when it is available.\n",
"\n",
"### Information about the on-screen size of the widget to which the `Event` is attached\n",
"\n",
"The properties below are useful for things like computing the position of the mouse in some set of coordinates different than that provided by default. Using them is an alternative to writing your own widget that includes JavaScript for computing a custom mouse position and returning it in `dataX` and `dataY`, described above.\n",
"\n",
"From the [MDN documentation of bounding rectangle](https://developer.mozilla.org/en-US/docs/Web/API/Element/getBoundingClientRect), the bounding rectanlge \"is the smallest rectangle which contains the entire element\"; all values are in pixels.\n",
"\n",
"The positions are relative to the part of the page currently visible in the browser window, which are the same coordinates as `clientX` and `clientY` for the mouse position.\n",
"\n",
"+ `boundingRectWidth`, `boundingRectHeight`: Width and height of the element in pixels.\n",
"+ `boundingRectTop`, `boundingRectLeft`, `boundingRectBottom`, `boundingRectRight`: Top, left, bottom and right positions of the upper left and lower right corners of the element, in the same coordinates as the mouse position `clientX` and `clientY`."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Mouse scroll events only\n",
"\n",
"If the event type is `wheel` then these keywords are also part of the event dictinoary:\n",
"\n",
"+ `deltaX`: Horizontal scroll amount.\n",
"+ `deltaY`: Vertical scroll amount.\n",
"+ `deltaZ`: Scroll amount in z-direction.\n",
"+ `deltaMode`: Units of the scroll amount (0 is pixels, 1 is lines, 2 is pages)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Preventing default actions for DOM events\n",
"\n",
"Some DOM events, like a right-click on a mouse, have a default action that usually occurs; in the case of a right-click a context menu is typically displayed. If you wish to suppress the default action while watching a widget, set the `prevent_default_action` property to `True`.\n",
"\n",
"Note that the DOM event for a right-click is `contextmenu`"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"label_no_default = Label('Right-click here and NO contextmenu is displayed')\n",
"label_no_default.layout.border = '2px solid red'\n",
"\n",
"label_default = Label('Right-click here and contextmenu IS displayed')\n",
"label_default.layout.border = '2px solid blue'\n",
"\n",
"h2 = HTML('Event info')\n",
"\n",
"dom_no_default = Event(source=label_no_default, \n",
" watched_events=['contextmenu'], \n",
" prevent_default_action=True)\n",
"\n",
"dom_default = Event(source=label_default, \n",
" watched_events=['contextmenu'], \n",
" prevent_default_action=False)\n",
"\n",
"def handle_event_default_demo(event):\n",
" lines = ['{}: {}'.format(k, v) for k, v in event.items()]\n",
" content = ' '.join(lines)\n",
" h2.value = content\n",
"\n",
"dom_no_default.on_dom_event(handle_event_default_demo)\n",
"dom_default.on_dom_event(handle_event_default_demo)\n",
" \n",
"display(label_no_default, h2, label_default)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Why is there a blue box around widgets attached to ipyevents (sometimes)?\n",
"\n",
"For `keydown` events to be captured by ipyevents and not by a notebook running in JupyterLab, ipyevents needs to grab the `focus` of the browser, i.e. set the `source` for the `Event` widget to be where input is currently happening in the browsers. Browsers will typically highlight the selected element by highlighting it with a blue border.\n",
"\n",
"The upside of of this is that it is perhaps easier to to tell *why* keyboard events and are not being processed by the notebook in the usual way.\n",
"\n",
"The downside is a blue border that may not fit well with the rest of your styling. To allow you to control the way the border looks, `ipyevents` adds a class, `ipyevents-watched` to the element to which it gives the focus. The CSS to select the element focused by ipyevents is below (along with an example of making the outline thinner):\n",
"\n",
"```css\n",
".ipyevents-watched:focus {\n",
" outline-width: '1px'\n",
"}\n",
"```\n",
"\n",
"For more information about the options for styling an outline see the [MDN documentation on outline](https://developer.mozilla.org/en-US/docs/Web/CSS/outline)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Limiting the rate at which events are passed from browser to Python\n",
"\n",
"In some cases you may want to limit the rate at which events are passed from the browser to Python. In others, you may want to impose a minimum time between consecutive events. \n",
"\n",
"Both can be done with `Event`. In both cases the amount of time, in milliseconds, between messages passed to Python is `Event.wait`. If `wait` is zero then there is no limit imposed.\n",
"\n",
"The first type of rate limiting is called *throttling*. It passes consecutive events only once per `wait` milliseconds no matter how many times the event happens in the browser. One case where this is useful is when watching `mousemove` or `wheel` events if they generate slow operations on the Python side. \n",
"\n",
"The second type of rate limiting is called *debouncing*. It passes an event to Python when event happens in the browser if at least `wait` milliseconds have passed since the last time that event happened in the browser.\n",
"\n",
"Select the type of rate limiting with with `Event.throttle_or_debounce`.\n",
"\n",
"The three boxes below all watch the `wheel` event but differ in their rate limiting. Mouse over each then scroll to see the difference. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"no_rate_box = Label(\"No rate limit (all scroll events passed)\", layout={'border': '2px solid red', 'width': '100%'}) \n",
"throttle_box = Label(\"Throttle (no more than 0.5 event per sec)\", layout=no_rate_box.layout)\n",
"debounce_box = Label(\"Debounce (wait 0.5 sec between scrolls)\", layout=no_rate_box.layout)\n",
"\n",
"event_no_rate = Event(source=no_rate_box, watched_events=['wheel'])\n",
"\n",
"# If you omit throttle_or_debounce it defaults to throttle\n",
"event_throttle = Event(source=throttle_box, watched_events=['wheel'], wait=500)\n",
"\n",
"event_debounce = Event(source=debounce_box, watched_events=['wheel'], wait=500, throttle_or_debounce='debounce')\n",
"\n",
"wheel_info = HTML(\"Waiting for a scroll...\")\n",
"\n",
"def update_display(event):\n",
" lines = [\n",
" 'timeStamp: {}'.format(event['timeStamp']),\n",
" 'deltaY: {}'.format(event['deltaY'])\n",
" ]\n",
" content = ' '.join(lines)\n",
" wheel_info.value = content\n",
" \n",
"event_no_rate.on_dom_event(update_display)\n",
"event_throttle.on_dom_event(update_display)\n",
"event_debounce.on_dom_event(update_display)\n",
"\n",
"instructions = HTML(\"Mouse over each box then scroll to see response\")\n",
"\n",
"VBox([instructions, HBox([no_rate_box, throttle_box, debounce_box], layout={'width': '100%'}), wheel_info])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Mouse location on an `Image` widget\n",
"\n",
"The `Image` widget provides the location of mouse events in \"array\" or image coordinates as the `arrayX` and `arrayY` entries in the event dictionary.\n",
"\n",
"Those entries take into account the size of the underlying image and the displayed width/height (including border and/or padding). \n",
"\n",
"Consider the image below, which has an intrinsic width of 600 pixels and height of 300 pixels. It is a gaussian centered in the image, i.e. with center coordinates 300, 150 in the original image.\n",
"\n",
"\n",
"\n",
"First, let's load the image into an `Image` widget and attach a `DOMListener` to it. \n",
"\n",
"The `dragstart` event is added just to prevent the image from being dragged if a user clicks, holds the mouse, and moves it."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"with open('images/gaussian_600_x_300.png', 'rb') as f:\n",
" value = f.read()\n",
"\n",
"image = Image(value=value, format='png')\n",
"\n",
"# The layout bits below make sure the image display looks the same in lab and classic notebook \n",
"image.layout.max_width = '100%'\n",
"image.layout.height = 'auto'\n",
"\n",
"im_events = Event()\n",
"im_events.source = image\n",
"im_events.watched_events = ['click']\n",
"\n",
"no_drag = Event(source=image, watched_events=['dragstart'], prevent_default_action = True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The image is displayed in three different ways, none of which match the original image dimensions, and for all of which the aspect ratio is incorrect (the images are placed inside a `Box` to control their size independently).\n",
"\n",
"Clicking on the center of *any* of the views of the image returns the same coordinate."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"vbox = VBox()\n",
"vbox.layout.width = \"100%\"\n",
"header = HTML('
'.format(event['dataX'], event['dataY'])\n",
" \n",
"im_events.on_dom_event(update_coords)\n",
"\n",
"vbox"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Enabling x, y coordinates as a trait\n",
"\n",
"There is a trait, called `xy`, that can be enabled so that the coordinates of the cursor are sent to the event widget as a trait.\n",
"\n",
"It is enabled by setting the attribute `xy_coordinate_system` to one of the coordinate systems the `Event` widget understands. Those are \n",
"\n",
"+ `array` -- \"natural\" coordinates for the widget (e.g. image)\n",
"+ `client` -- Relative to the visible part of the web page\n",
"+ `offset` -- Relative to the padding edge of widget\n",
"+ `page` -- Relative to the whole document\n",
"+ `relative` -- Relative to the widget\n",
"+ `screen` -- Relative to the screen\n",
"\n",
"If the attribute is set to `None` then the trait is disabled. That option will eliminate message traffic to/from the browser that is not useful if you are not interested in cursor position."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"no_drag.xy_coordinate_system = 'data'"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Once the attribute has been set, then you can access the value in the usual way. You will need to click on any of images above after you have run the cell above setting the coordinate system or the list of coordinates will be empty."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"no_drag.xy"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.4"
}
},
"nbformat": 4,
"nbformat_minor": 4
}