Interactive online version: .

Connecting browser events to widgets with Event

What are browser events?

In a nutshell, they are keyboard or mouse events in the browser. They are also called DOM events, and DOM is short for Document Object Model, the representation of the components of a web page displayed in a browser. When you interact with something in a browser (like a notebook cell) events corresponding to keypresses and mouse clicks are generated by the browser and handled by the notebook’s JavaScript.

The Event widget

Event is a widget that represents and provides a Python interface to mouse and keyboard interactions with the widgets, like Image or Label, that are displayed in the browser.

Unlike most other widgets, the Event is not itself displayed.

from ipywidgets import Label, HTML, HBox, Image, VBox, Box, HBox
from ipyevents import Event
from IPython.display import display

Short example

The example below attaches a listener that responds to mouse clicks and key down on an Label widget and displays the event information in a HTML widget.

Note that the information returned in the event depends on whether the event was a mouse or a keyboard event.

l = Label('Click or type on me!')
l.layout.border = '2px solid red'

h = HTML('Event info')
d = Event(source=l, watched_events=['click', 'keydown', 'mouseenter', 'touchmove'])

def handle_event(event):
    lines = ['{}: {}'.format(k, v) for k, v in event.items()]
    content = '<br>'.join(lines)
    h.value = content

d.on_dom_event(handle_event)

display(l, h)

If you watch keyboard events then key presses are not passed on to the notebook

This is to prevent the user from, e.g., inadvertently cutting a cell by pressing x over a widget that is catching key events.

Adding a new view of a widget attaches listeners to the view

Clicking on the label below will update the HTML widget above, just like clicking (or typing) on the first view of the widget did.

l

You can get some information about the HTML element to which the event listener is attached

One of the items returned in the event information dictionary is called target and contains the name, ID and classes on the HTML element to which the event listener is attached.

l2 = HTML('''
    <button id=eeny class="success special">eeny</button>
    <button id=meeny class="caution">meeny</button>
    <button id=miny>miny</button>
    <button id=moe>moe</button>
''')

h2 = HTML('Event info')
d2 = Event(source=l2, watched_events=['click'])

def handle_event(event):
    h2.value = (f"You clicked a {event['target']['tagName']} with "
                f"ID {event['target']['id']} "
                f"and classes: {event['target']['className']}")

d2.on_dom_event(handle_event)

display(l2, h2)

Watchable events

You can get a list of the events to which a DOMListener can respond from the supported_key_events and supported_mouse_events properties.

print('Key events: ', d.supported_key_events)
print('Mouse events: ', d.supported_mouse_events)
print('Touch events:', d.supported_touch_events)

Key events:  ['keydown', 'keyup']
Mouse events:  ['click', 'auxclick', 'dblclick', 'mouseenter', 'mouseleave', 'mousedown', 'mouseup', 'mousemove', 'wheel', 'contextmenu', 'dragstart', 'drag', 'dragend', 'dragenter', 'dragover', 'dragleave', 'drop']
Touch events: ['touchstart', 'touchend', 'touchmove', 'touchcancel']

More about the watchable events

For more about these events, see the MDN list of DOM events

Keyboard events

The keyboard events are triggered only if the mouse is over the widget. + keydown is triggered when a key is pressed down or if the key is held down and repeats. + keyup is triggered when a key is released.

Mouse events

Mouse events trigger only when the mouse is over the watched widget.

• click currently, triggered if any mouse button is pushed, but in the near future will trigger only when the primary mouse button is clicked.

• auxclick triggers when a non-primary mouse button is clicked.

• dblclick triggers when there is a double click on the element.

• mouseenter triggers when the mouse enters the element.

• mouseleave triggers when the mouse leaves the element.

• mousedown triggers when any mouse button is depressed.

• mouseup triggers when any mouse button is released.

• mousemove triggers every time the mouse moves over the widget. Enabling this will generate a large volume of events between the front end and the back end.

• wheel trigger when the user scrolls in any direction. Note that watching this disables scrolling of the notebook while the mouse is over the watched widget.

Touch events

Touch events trigger on touch-enabled devices when something (e.g. a finger or a stylus) touches the screen. Touch events include information about all of the touch points on the screen, not just touch points within the watched widget. The touch event interface assumes that there may be multiple touch points.

• touchstart triggers when the watched element is touched.

• touchend triggers when the touch point that triggered touchstart is removed from the screen or if the touch moves off the screen.

• touchmove triggers when one or more of the touch points changes even if the touch is no longer over the target widget.

• touchcancel triggers when something happens to inteerupt the touch (e.g. a dialog box pops up or there are too many touches on the screen).

Optional filter useful if watching keyboard and mouse events

The DOMListener widget has a property, ignore_modifier_key_events, which, if set to True, causes key events for the modifier keys Shift, Control, Alt, and Meta, to be ignored. This is useful if you want to watch keyboard events and mouse events, and you want to handle mouse events with modifier keys pressed differently than plain mouse events.

Normally, an action like Shift-click generates two events: a keydown for the Shift followed by a click (with its shiftKey set to True). While the unwanted Shift event could be filtered out on the python side, ignore_modifier_key_events provides a convenient way to avoid the extra events.

Choose wisely

Think carefully about which events to monitor and experiment frequently. If, for example, you want to monitor both single and double mouse clicks, you probably do not want to watch for both click and dblclick. The reason is that when you double click a long string of events is generated:

mousedown
mouseup
click
mousedown
mouseup
click
dblclick

If you watch both click and dblclick then you will get three events per double click.

One way to handle this case would be to watch only for click, and to check the timeStamp property of the event dictionary to see how close together in time the clicks are. If they are close enough, consider two consecutive clicks to be a double click, otherwise consider it to be two separate clicks. timeStamp is measured in milliseconds.

The event dictionary

The single argument passed to a callback registered with on_dom_event is a dictionary. The entries in the dictionary depend on what kind of event occurs, but always contains the key type whose value is the type of event , e.g. 'click' or 'keydown’, and the key target whose value includes information about the DOM element to which the event watcher is attached.

Common to all events

All events have these keys in the event dictionary:

• altKey: bool, True if the alt key was down when the event occurred, otherwise False.

• ctrlKey: bool, True if the control key was down when the event occurred, otherwise False.

• metaKey: bool, True if the meta key (the Windows key on Windows, the command key (cloverleaf) on the mac) was down when the event occurred, otherwise False.

• shiftKey: bool, True if the Shift key was down when the event occurred, otherwise False.

• type: str, type of event (e.g. 'click') that occurred.

• timeStamp: float, the time (in milliseconds) at which the event occurred.

• target: dict with the ID, CSS classes, and the name of the HTML tag of the element to which the listener is attached.

For keyboard events only

A key-related event has these keys in addition to the common ones above:

• key: str, the key that was pressed. Note that if the shift key was held down when the key was pressed, and the key pressed was a letter, this will be an upper case letter.

• code: str, also indicates the key pressed. the value begins with the string Key. This may contain additional information about the keypress, for example whether it was the left or right key pressed if the key appears multiple times on the keyboard.

• location: int, the location of the key on the device (typically 0).

• repeat: bool, True if this event is the result of a key being held down.

For more details, see the MDN documentation of HTML keyboard events

For mouse events only

A mouse-related event has these keys in addition to the common ones above. They are broken into groups below to simplify the discussion. For more details about these properties, see the MDN documentation of mouse events.

Which mouse button was pressed

• button: The button that was pushed or released for an event related to a mouse button push.

• buttons: The buttons that were depressed when a mouse-related event occurs whether or not the event was a click. For example, when the mouse moves onto a widget being watched for mouseenter, then buttons indicates which, if any, mouse buttons are being help down.

Location of the mouse event

All locations are integers.

These attributes are standard properties of HTML mouse events.

• screenX, screenY: Coordinates of the event relative to the user’s whole screen.

• clientX, clientY and x, y: Coordinates of the event relateive to the visible part of the web page in the browser.

• pageX, pageY: Coordinates of the event relative to the top of the page currently being displayed.

• offsetX, offsetY: Offset of this DOM element relative to its parents.

• relativeX, relativeY: The location of the mouse relative to the current element (i.e. widget). This emulates the functinoality of the layerX/Y attributes implemented in many browsers but not part of any standard.

Additional location for some types of widgets

• dataX, dataY: Position of the mouse event in the underlying data object. For example, a click on an Image widget returns as dataX and dataY the coordinates of the click in the image. This location should be regarded as the approximate location the user intended given the difficulty of clicking accurately at the pixel level.

For widgets (like Label) for which there is no underlying array object these properties are not returned.

Touch events only

Touch events do not store locations directly. Instead, they store a list of touch points, each of which has location information. The top-level attributes unique to a touch event are

• changedTouches: a list of the touch points which have changed.

• targetTouches: a list of all of the touch points which are still on the surface and which originated in the target widget.

• touches: a list of all of the touch points on the surface, regardless of whether they have changed or originated in the target widget.

Each of the touches in those lists contains:

• identifier: A unique identifier of the touch point.

• The position attributes for mouse events and the additional position information when it is available.

Information about the on-screen size of the widget to which the Event is attached

The properties below are useful for things like computing the position of the mouse in some set of coordinates different than that provided by default. Using them is an alternative to writing your own widget that includes JavaScript for computing a custom mouse position and returning it in dataX and dataY, described above.

From the MDN documentation of bounding rectangle, the bounding rectanlge “is the smallest rectangle which contains the entire element”; all values are in pixels.

The positions are relative to the part of the page currently visible in the browser window, which are the same coordinates as clientX and clientY for the mouse position.

• boundingRectWidth, boundingRectHeight: Width and height of the element in pixels.

• boundingRectTop, boundingRectLeft, boundingRectBottom, boundingRectRight: Top, left, bottom and right positions of the upper left and lower right corners of the element, in the same coordinates as the mouse position clientX and clientY.

Mouse scroll events only

If the event type is wheel then these keywords are also part of the event dictinoary:

• deltaX: Horizontal scroll amount.

• deltaY: Vertical scroll amount.

• deltaZ: Scroll amount in z-direction.

• deltaMode: Units of the scroll amount (0 is pixels, 1 is lines, 2 is pages).

Preventing default actions for DOM events

Some DOM events, like a right-click on a mouse, have a default action that usually occurs; in the case of a right-click a context menu is typically displayed. If you wish to suppress the default action while watching a widget, set the prevent_default_action property to True.

Note that the DOM event for a right-click is contextmenu

label_no_default = Label('Right-click here and NO contextmenu is displayed')
label_no_default.layout.border = '2px solid red'

label_default = Label('Right-click here and contextmenu IS displayed')
label_default.layout.border = '2px solid blue'

h2 = HTML('Event info')

dom_no_default = Event(source=label_no_default,
                       watched_events=['contextmenu'],
                       prevent_default_action=True)

dom_default = Event(source=label_default,
                    watched_events=['contextmenu'],
                    prevent_default_action=False)

def handle_event_default_demo(event):
    lines = ['{}: {}'.format(k, v) for k, v in event.items()]
    content = '<br>'.join(lines)
    h2.value = content

dom_no_default.on_dom_event(handle_event_default_demo)
dom_default.on_dom_event(handle_event_default_demo)

display(label_no_default, h2, label_default)

Why is there a blue box around widgets attached to ipyevents (sometimes)?

For keydown events to be captured by ipyevents and not by a notebook running in JupyterLab, ipyevents needs to grab the focus of the browser, i.e. set the source for the Event widget to be where input is currently happening in the browsers. Browsers will typically highlight the selected element by highlighting it with a blue border.

The upside of of this is that it is perhaps easier to to tell why keyboard events and are not being processed by the notebook in the usual way.

The downside is a blue border that may not fit well with the rest of your styling. To allow you to control the way the border looks, ipyevents adds a class, ipyevents-watched to the element to which it gives the focus. The CSS to select the element focused by ipyevents is below (along with an example of making the outline thinner):

.ipyevents-watched:focus {
   outline-width: '1px'
}

For more information about the options for styling an outline see the MDN documentation on outline.

Limiting the rate at which events are passed from browser to Python

In some cases you may want to limit the rate at which events are passed from the browser to Python. In others, you may want to impose a minimum time between consecutive events.

Both can be done with Event. In both cases the amount of time, in milliseconds, between messages passed to Python is Event.wait. If wait is zero then there is no limit imposed.

The first type of rate limiting is called throttling. It passes consecutive events only once per wait milliseconds no matter how many times the event happens in the browser. One case where this is useful is when watching mousemove or wheel events if they generate slow operations on the Python side.

The second type of rate limiting is called debouncing. It passes an event to Python when event happens in the browser if at least wait milliseconds have passed since the last time that event happened in the browser.

Select the type of rate limiting with with Event.throttle_or_debounce.

The three boxes below all watch the wheel event but differ in their rate limiting. Mouse over each then scroll to see the difference.

no_rate_box = Label("No rate limit (all scroll events passed)", layout={'border': '2px solid red', 'width': '100%'})
throttle_box = Label("Throttle (no more than 0.5 event per sec)", layout=no_rate_box.layout)
debounce_box = Label("Debounce (wait 0.5 sec between scrolls)", layout=no_rate_box.layout)

event_no_rate = Event(source=no_rate_box, watched_events=['wheel'])

# If you omit throttle_or_debounce it defaults to throttle
event_throttle = Event(source=throttle_box, watched_events=['wheel'], wait=500)

event_debounce = Event(source=debounce_box, watched_events=['wheel'], wait=500, throttle_or_debounce='debounce')

wheel_info = HTML("Waiting for a scroll...")

def update_display(event):
    lines = [
        'timeStamp: {}'.format(event['timeStamp']),
        'deltaY: {}'.format(event['deltaY'])
    ]
    content = '<br>'.join(lines)
    wheel_info.value = content

event_no_rate.on_dom_event(update_display)
event_throttle.on_dom_event(update_display)
event_debounce.on_dom_event(update_display)

instructions = HTML("Mouse over each box then scroll to see response")

VBox([instructions, HBox([no_rate_box, throttle_box, debounce_box], layout={'width': '100%'}), wheel_info])

Mouse location on an Image widget

The Image widget provides the location of mouse events in “array” or image coordinates as the arrayX and arrayY entries in the event dictionary.

Those entries take into account the size of the underlying image and the displayed width/height (including border and/or padding).

Consider the image below, which has an intrinsic width of 600 pixels and height of 300 pixels. It is a gaussian centered in the image, i.e. with center coordinates 300, 150 in the original image.

First, let’s load the image into an Image widget and attach a DOMListener to it.

The dragstart event is added just to prevent the image from being dragged if a user clicks, holds the mouse, and moves it.

with open('images/gaussian_600_x_300.png', 'rb') as f:
    value = f.read()

image = Image(value=value, format='png')

# The layout bits below make sure the image display looks the same in lab and classic notebook
image.layout.max_width = '100%'
image.layout.height = 'auto'

im_events = Event()
im_events.source = image
im_events.watched_events = ['click']

no_drag = Event(source=image, watched_events=['dragstart'], prevent_default_action = True)

The image is displayed in three different ways, none of which match the original image dimensions, and for all of which the aspect ratio is incorrect (the images are placed inside a Box to control their size independently).

Clicking on the center of any of the views of the image returns the same coordinate.

vbox = VBox()
vbox.layout.width = "100%"
header = HTML('<h2>Same image, three views</h2>')

images = HBox()
i1 = Box()
i1.layout.height = '100px'
i1.children = [image]

i2 = Box()
i2.children = [image]
i2.layout.width = '100px'

i3 = Box()
i3.layout.border = '5px red solid'
i3.layout.padding = '10px'
i3.layout.width = '300px'
i3.children = [image]

images.children = [i1, i2, i3]

coordinates = HTML('<h3>Click an image to see the click coordinates here</h3>')

vbox.children = [header, images, coordinates]

def update_coords(event):
    coordinates.value = '<h3>Clicked at ({}, {})</h3>'.format(event['dataX'], event['dataY'])

im_events.on_dom_event(update_coords)

vbox

Enabling x, y coordinates as a trait

There is a trait, called xy, that can be enabled so that the coordinates of the cursor are sent to the event widget as a trait.

It is enabled by setting the attribute xy_coordinate_system to one of the coordinate systems the Event widget understands. Those are

• array – “natural” coordinates for the widget (e.g. image)

• client – Relative to the visible part of the web page

• offset – Relative to the padding edge of widget

• page – Relative to the whole document

• relative – Relative to the widget

• screen – Relative to the screen

If the attribute is set to None then the trait is disabled. That option will eliminate message traffic to/from the browser that is not useful if you are not interested in cursor position.

no_drag.xy_coordinate_system = 'data'

Once the attribute has been set, then you can access the value in the usual way. You will need to click on any of images above after you have run the cell above setting the coordinate system or the list of coordinates will be empty.

no_drag.xy

[]