neuronode context & effect: multi-modal switching for

NeuroNode Context & Effect:Multi-modal Switching for

Communication and Control

James Schorey, Chief Technology Officer, Control Bionics Inc.

1

Speaker Disclosures/Session Disclosure

James Schorey is employed by and receives a salary from Control Bionics Inc. He is

their Chief Technology Officer.

Control Bionics has developed and patented products that are trademarked as

the NeuroNode and Context & Effect (C&E). Because there are no other like-

kind products available, course offerings will only cover information that pertains

to the effective and safe use of the above-named products.

This presentation will focus exclusively on the NeuroNode with Context & Effect

and will not include information on other similar or related products.

2

Learning Objectives

• Compare three access methods available for communication and control

systems.

• Discuss how one user-interface and multiple access methods can be used for

three outcomes: toy, robotic, and game control.

• Demonstrate how thoughtful design can allow a single switch input to control

multiple outputs.

3

Session Evaluation and CEUs

• Virtual Presentation Survey▪ Your feedback is very important to us. Please be sure to complete the survey

after the presentation. You will be automatically redirected to the survey at end of the presentation. Upon completion of the survey, your certificate of completion will be posted to your learner profile.

• CEUs▪ ATIA 2021: AT Connected education presentations are ACVREP, AOTA, ASHA, CRC

and IACET CE approved. Please note that sessions are reviewed for specialty CEU eligibility, but not all sessions are approved for specialty CEUs.

▪ You will receive a CEU Certificate after submitting an Assessment. The certificate will be posted to your learner profile. You may submit more than one Assessment for different CEUs for the same education session e.g. ASHA and IACET.

4

The Context & Effect: What is it?

• A device used with the NeuroNode to control robots, toys, and gaming devices

• An accompanying App for building a communication and control “ecosystem” that can map a single user switch to many switch outputs

• A system that allows user-activated transitions among access methods, for example, to move from direct switch control of a dining robot to switch scanning for communication related to activities of daily living (dining, in this case)

5

Context & Effect: Major Components

• The C&E Box has 8 switched outputs (standard 3.5mm jacks) for direct connection to toys, robots, and gaming devices.

• The C&E Box connects to a Windows 10 computer via USB. Communication between the App and the C&E Box takes place over this USB link.

• An RFID reader is built in to the C&E box so that entire communication and control ecosystems are available with the swipe of card.

6

Example 1 of 2: The Obi Dining Robot

• Two switch outputs required• Obi control can be made with a single

user activation site using “short press” and “long press” or with a single switch in switch scanning

• In this example, a short activation with the NeuroNode triggers the switch 1 input on the Obi to select the food (move the spoon over the desired food compartment), and a long press triggers Obi switch input 2 to scoop the food and deliver it to a preprogrammed position (the diner’s mouth)

7

Other Panels Comprising the C&E Dining Ecosystem

• Other panels can be included to form the dining ecosystem

• Here a scanned panel allows the diner to make choices apart from the actual delivery of food

• Timeouts form an important part of the ecosystem. For example, if the diner hasn’t invoked the feeding robot in some user-settable time, the panel can automatically switch to some higher-level panel to allow for communication related to the task at hand: “I’m done” or “more.”

8

Other Panels Comprising the C&E Dining Ecosystem

• Although the primary purpose of the C&E is controlling devices, most activities require communication as part of that control

• This dining “help panel” allows the user to communicate a number of needs related to the task at hand. “Wipe my face,” for example.

9

Video: Controlling the Obi Dining Robot

• An important part of the C&E experience is designing the communication and control ecosystem so the user does not have to look at the App window. Instead, she can concentrate on dining, gaming, etc.

10

Video: Controlling the Obi Dining Robot

• Short Activation = Select Food

• Long Activation = Deliver Food

11

Example 2 of 3: Xbox Powerstar Golf

• Six switch outputs required• Microsoft’s Xbox Adaptive Controller

(XAC) is used as the interface between C&E and the Xbox

• The entire game is controlled with a single NeuroNode activation

12

Powerstar Golf: Swinging the Club

• Powerstar shots are made with the Xbox ‘A’ button on the controller

• If other controls are needed, the user can allow the C&E countdown timer to expire, causing a panel transition to a menu for access to more Xbox controls

13

Power Star Golf: Adjusting a Putt

• Part of a winning round of golf is being able to adjust the stance of the golfer in relationship to the hole. Every hole will have wind and a unique slope of the green to contend with.

• This scanned menu provides putter stance adjustment with the left and right Xbox controls, a navigation-only button to return to the Play Menu, and a navigation-only button to return to the Main Menu.

14

Video 1 of 2: Powerstar Golf

• In important part of C&E is creating an ecosystem where the user can play long periods of time without intervention (i.e., give him access to all necessary Xbox controls)

• Of course, long periods of play = lots of motor skills practice

15

Video: Powerstar Golf

16


• This short example demonstrates an 18-year-old male with spinal muscular atrophy (SMA) nearly parring a hole after just a few minutes of practice…

• He had not yet taken advantage of the sub-menu for adjusting his putting angle

17


18

Example 3: Xbox Pinball - Flippers

• This C&E pinball ecosystem has two primary panels, one for the flippers, and one for a plunger to launch the next ball into play

• “Combine Switch 1 and 2” is checkmarked within the Direct Switch access method so a single switch can be tied to both flipper controls on the Xbox

19

Example 3: Xbox Pinball - Plunger

• If the flippers aren’t activated within some preprogrammed time, the C&E will automatically transition to a plunger panel

• The plunger uses the same NeuroNode activation; however, instead of switching the controls for the flippers, the output is rerouted to the Xbox control for activating the plunger

20

Example 3: Xbox Pinball – Switch Design

• Each panel has its own independent settings for assigning “switch in” and “switch out” mappings for each button.

• This allows a single NeuroNode activation to control multiple Xbox controls.

21

Video: Xbox Pinball

• This short example demonstrates an 18-year-old male with SMA using the NeuroNode to play pinball with three Xbox controls

22

Video: Xbox Pinball

23

Context & Effect: A Communication and Control System

• There are eight programmable wired switch outputs. This gives us utility in toy control, robotics control, and advanced gaming.

o

24


• The system allows immediate selection among the three access methods (and variations of these). That is, the “Cause” (the access method) can be changed in seconds while keeping the “Effect” (e.g., animation of toys, control of an Xbox) constant.

• Most importantly, the access method can change with each panel transition, moving, say, from direct switching for control of a robot to switch scanning for communication needs.

25


• Panels can be linked by context.• For example, if the client is using a

dining robot with C&E and he hasn’t invoked the feeding robot in some user-settable time, the panel can automatically switch to some higher-level panel to allow for communication related to the task at hand: “I’m done” or “more.”

o

26


• Panels can be switched by user control and with RFID cards. A swipe of a card can invoke a C&E ecosystem for dining, or an Xbox interface, or any number of aids for learning, activities of daily living, or game playing.

o

27

Building a C&E Ecosystem: Intermediate Xbox Pinball

• This C&E ecosystem is comprised of two panels: plunger and flippers

• One user input (the NeuroNode) is translated into three output switches for control of the Xbox pinball game via the Xbox Adaptive Controller

28

Context & Effect (C&E) Summary

• The C&E is a

device used with

the NeuroNode

(or other switch

input) to control

robots, toys, and

gaming devices

29


• The C&E allows the access method to

change on a panel by panel basis.

• “Scan Buttons” initiates a button by button

scan, and selection is made when the

desired choice is highlighted.

• “Direct Switch” maps a button directly to a

switch input with the ability to add “long

press” and the combination of buttons.

• “Direct Select” is the typical mouseover and

click type of selection method. This would

be the choice when eye gaze is desired.

30


• The C&E includes an accompanying App for building a

communication and control “ecosystem” that can map a

single user switch (or a few switches) to many switch

outputs

• Panels are linked by button activations or by context

(countdowns, time of day, etc.)

• Ultimately, the C&E allows ecosystem designs that allow the

user to perform a task (communication and control) without

caregiver intervention

31

NeuroNode and the Context & Effect

Thank you!

• [email protected]

• (513) 453-4848

• controlbionics.com

32

mailto:[email protected]

NeuroNode Context & Effect:Multi-modal Switching for Communication and Control

NeuroNode and “Context & Effect” are trademarks of Control Bionics Inc. These products are protected by patents and patents pending.

Powerstar Golf is a golf video game developed by Zoë Mode and published by Microsoft Studios for the Xbox One game console. It was released on November 22, 2013.*

Other trademarks are property of their respective owners.

*Wikipedia

33

neuronode context & effect: multi-modal switching for

Documents