doc.: IEEE 802.15-19-0421-02-0dep
Submission
Project: IEEE P802.15 Working Group for Wireless Personal Area Networks (WPANs)Submission Title: [WNG Brain-Machine Interface based on Electrocorticography using high speed UWB wireless body area network]Date Submitted: [18 July 2019]Source: [Takafumi Suzuki1, Masayuki Hirata2] [1; Center for Information and Neural Networks(CiNet), National Institute of Information and Communication Technology(NICT), 2; Dept. of Neurological Diagnosis and Restoration, Graduate School of Medicine, Osaka University] Address [1; 1-4 Yamadaoka, Suita, Osaka, 565-0871
2; 2-2 Yamadaoka, Suita, Osaka, 565-0871]Voice:[1; +81-80-9098-3264, 2;+81-6-6210-8429], FAX: [+81-6-6210-8430], Email:[1; [email protected], 2;[email protected]] Re: []Abstract: [A important use case of dependable body area network(WBAN) for implanted devices is introduced to perform reliable and massive data for ECoG-based Brain machine interface to require amendment for IEEE802.15.6 wireless medical body area network.]Purpose: [information]Notice: This document has been prepared to assist the IEEE P802.15. It is offered as a basis for discussion and is not binding on the contributing individual(s) or organization(s). The material in this document is subject to change in form and content after further study. The contributor(s) reserve(s) the right to add, amend or withdraw material contained herein.Release: The contributor acknowledges and accepts that this contribution becomes the property of IEEE and may be made publicly available by P802.15.
Slide 1 Takafumi Suzuki(NICT), Masayuki Hirata(Osaka University)
September 2019
doc.: IEEE 802.15-19-0421-02-0dep
Submission
Brain-Machine Interface based on Electrocorticography using high speed
UWB wireless body area network
Slide 2
September 2019
Takafumi Suzuki(NICT), Masayuki Hirata(Osaka University)
Takafumi SuzukiCenter for Information and Neural Networks, NICT
Masayuki HirataDepartment of Neurological Diagnosis and Restoration,
Osaka University Graduate School of Medicine
doc.: IEEE 802.15-19-0421-02-0dep
Submission Slide 3 Takafumi Suzuki(NICT), Masayuki Hirata(Osaka University)
September 2019
Outline■ECoG-BMI system
■1st Genaration 128ch system: Clinical ECoG-BMI system→ Clinical test in 2020
■2nd Generation 4096ch system → Next generation system・Flexible electrode technology ・UWB wireless technology
■BMI (system evaluation)・Real-time decoding・Robotic arm control and cortical adaptation flexible ECoG electrode-mesh 8:1 Multiplxer and logic
15 mm
20.7 mm
14 m
m 19 mm50μmPt Black
UWB-T(FPGA)
Controller
ZigBee
Qi
Battery
UWB transmitter andOperation control block
Wireless power supply
Outside body
128 Mbps(wireless)
UWB receiver andPC connection block
6.4Mbps(8 x 800kbps)
x81~64ch
512ch base unit No.8
Recording
No.165~128ch
Recording
No.1
3585~4096ch
Recording
No.64
MUX-A
(FPGA)
1~64ch
512ch base unit No.1
Recording
No.165~128ch
Recording
No.1
449~512ch
Recording
No.8
MUX-A
(FPGA)800kbps
MUX-B
(FPGA)
Multi-channel recording and digital multiplexing block
51.2 Mbps(64 x 800kbps)
UWB-R(FPGA)
LED(time)
ZigBee
USBPCZigBee
Inside body
4.9 mm2
doc.: IEEE 802.15-19-0421-02-0dep
Submission Slide 4 Takafumi Suzuki(NICT), Masayuki Hirata(Osaka University)
September 2019
grasp open
Real time control of external devices just by thinkingA promising therapy for severely disabled patients
communication robot control
ALS, spinal cord injury, amputated limb, stroke, ……..
Thank you !
Robot arm
BMI Project (Osaka U & NICT)
doc.: IEEE 802.15-19-0421-02-0dep
Submission Slide 5 Takafumi Suzuki(NICT), Masayuki Hirata(Osaka University)
September 2019
ECoG: ElectrocorticogramGood balance (Information rate, invasiveness, long-term stability)
④Spike or LFP
×High Invasive◎Info:high×Short-term
② EEG
◎No invasive△Info : low(On/Off(1bit)
selection slowly)○Long-term
③ECoG(Electrocorticogram)
△Low invasive○Info: middle(Robotic hand
control in RT)○Long-term
①fMRI, MEG
◎No invasive◎Info: high×Huge system
Recording methods for clinical BMI
EEG
ECoG
x10info
doc.: IEEE 802.15-19-0421-02-0dep
Submission Slide 6 Takafumi Suzuki(NICT), Masayuki Hirata(Osaka University)
September 2019
ECoG signalWaldart, et al, J.Physiology, 2009
SUA: Single Unit ActivityMUA: Multi Unit Activity
ECoG(50~180Hz)sync with motion= informative
doc.: IEEE 802.15-19-0421-02-0dep
Submission Slide 7 Takafumi Suzuki(NICT), Masayuki Hirata(Osaka University)
September 2019
Clinical Research (using wired ECoG-BMI system)
Grasp(Yanagisawa, Ann Neurol)
ALS patient (1st Clinical Research)(NHK news)
(by Osaka Univ.)
Implantable system is necessary for daily life support by BMI system ⇒ “Wireless” is the key
doc.: IEEE 802.15-19-0421-02-0dep
Submission Slide 8 Takafumi Suzuki(NICT), Masayuki Hirata(Osaka University)
September 2019
1st generation ECoG BMIsystem
- 128ch (# of electrodes)- ISM band (2.4GHz)- Clinical test in 2020- Communication
device for ALS- Robot arm control
for Paralysis
doc.: IEEE 802.15-19-0421-02-0dep
Submission
A fully implantable wireless BMI system
Prototype 64/128ch systemsix months in a monkey (2013)
4.9m
m
4.9mm
32ch
LN
A
32ch
LN
AADC x2
IF
64ch ECoG Electrode
titanium head casing
Wi-Fi transmitter andwireless power supply unit (epoxy embedded)
wireless data / power control board
coil
rechargeable Li polymer battery
64ch neural recording LSI
Pre-clinical test in 2017-⇒ Clinical test in 2020
September 2019
Slide 9 Takafumi Suzuki(NICT), Masayuki Hirata(University of Osaka)
doc.: IEEE 802.15-19-0421-02-0dep
Submission
1st Generation 128ch system (Improvement for clinical use)■Abdomen unit
→ Integrated into head unitSmaller system can decrease various risks.
Current version: Casing, non-touch energy supply
・LSI improvement → Lower noise, safety, etc.
TSMC CMOS0.25μm(7.1mm×7.3mm)
September 2019
Slide 10Takafumi Suzuki(NICT), Masayuki Hirata(University of Osaka)
Prototype 64/128ch systemsix months in a monkey (2013)
4.9m
m
4.9mm
32ch
LN
A
32ch
LN
AADC x2
IF
64ch ECoG Electrode
titanium head casing
Wi-Fi transmitter andwireless power supply unit (epoxy embedded)
wireless data / power control board
coil
rechargeable Li polymer battery
64ch neural recording LSI
doc.: IEEE 802.15-19-0421-02-0dep
Submission
1st Generation 128ch system (Improvement for clinical use)
TSMC CMOS 0.25μm (7.1mm×7.3mm)-32ch x 4chips-Noise (input) 3μVpp-Capable of High-γ
band recording
Inside
Receiver Coil forenergy supply
Surface electrodearray withbio-compatiblesilicone
Wireless transmitter (2.4 GHz ISM Band) ~1.9Mbps - GLP test (bio-compatiblilty )
- Implant test (animal)
September 2019
Slide 11Takafumi Suzuki(NICT), Masayuki Hirata(University of Osaka)
doc.: IEEE 802.15-19-0421-02-0dep
Submission Slide 12 Takafumi Suzuki(NICT), Masayuki Hirata(Osaka University)
September 2019
2nd generation ECoG BMIsystem
- 4096ch~ (# of electrodes)- UWB band (7.9GHz)- Clinical test in 2030?- Robot arm control
for Paralysiswith individual finger control
doc.: IEEE 802.15-19-0421-02-0dep
Submission
Next generation multi-channel BMI system more than 4,000 channels
Issue of multi-channel system: volume of dataex. ECoG, 1kS/s, 12bit-ADC⇒ ~1Mbps@100ch, ~100Mbps@10,000ch
Our targetimplantable, distributed, and wireless
For more accurate estimation of movement intentions a large number of recording channels recording at several regions simultaneously
September 2019
Slide 13 Takafumi Suzuki(NICT), Masayuki Hirata(University of Osaka)
doc.: IEEE 802.15-19-0421-02-0dep
Submission
Improvement for next generation (128ch-> 4096ch) Issues to be solved(1): High density electrode array
Silicone + Pt array Parylene-C + Pt (or Au)
(2): LSI (amplifier + ADC)32ch x 4 chips 64ch x 64chips
(3): Wireless transmittingISM (1.9Mbps) UWB(128Mbps)
September 2019
Slide 14Takafumi Suzuki(NICT), Masayuki Hirata(University of Osaka)
doc.: IEEE 802.15-19-0421-02-0dep
Submission
2nd Generation(4096ch ECoG-BMI system)
Flexible electrode technology
September 2019
Slide 15Takafumi Suzuki(NICT), Masayuki Hirata(University of Osaka)
doc.: IEEE 802.15-19-0421-02-0dep
Submission
Electrode Array
3D-shapeIndividutl MRI Data1st Generation
Up: High Density(IED: 2.5mm)
Down:Clinical
(Safety, High density, Stability)
(Hirata M, IEICE Trans Commun, 2011)
3D-double surfacefor intra-sulcus
Flexible electrode array・Parylene-C、 High Density: IED
50μm・Relationship between intracortical
and ECOG signal
2nd Generation
September 2019
Slide 16Takafumi Suzuki(NICT), Masayuki Hirata(University of Osaka)
doc.: IEEE 802.15-19-0421-02-0dep
Submission
Flexible Electrode array for ECoG
・50μm x 50μm・Platinum black
1mm
Toda, Neuroimage(2011)
GoldParylene-CSilicon Wafer
Aluminum Mask
Oxygen plasmaetching
Platinum black
Parylene-C
(a)
(b)
(c)
(d)Monkey128ch
- > 2.5 years- into sulcus
(Hasegawa Lab, Niigata Univ.)
September 2019
Slide 17 Takafumi Suzuki(NICT), Masayuki Hirata(University of Osaka)
doc.: IEEE 802.15-19-0421-02-0dep
Submission
2nd Generation(4096ch ECoG-BMI system)
Wireless technology⇒UWB(Ultra-Wide Band)
September 2019
Slide 18Takafumi Suzuki(NICT), Masayuki Hirata(University of Osaka)
doc.: IEEE 802.15-19-0421-02-0dep
Submission
Super multi-channel system using UWB(4096ch~)
Electrode array+ LSI(64ch x 64units)
BMI- Robotic hand control- Feedback
Large-scaleRecording
Real-timeDecoding
Wireless transmitting
(UWB)
1st Generation (128ch)-ISM band (2.4GHz) [1.9Mbps]12bit x 1kHz x 128ch = 1.5 Mbps
2nd Generation (>4000ch)12bit x 1kHz x 4096ch = 49Mbps UWB(Ultra Wide Band)
+ Distributed system
September 2019
Slide 19Takafumi Suzuki(NICT), Masayuki Hirata(University of Osaka)
doc.: IEEE 802.15-19-0421-02-0dep
Submission
UWB system (4096ch) -Prototype-
waterproof package
UWB transmitter
UWB receiver
flexible ECoG electrode-meshNeural recording LSI board(64ch-LNA, MUX, 12b-ADC)
8:1 Multiplexer and logic
15 mm
20.7 mm
14 m
m
19 mm
UWB-T(FPGA)
Controller
ZigBee
Qi
Battery
UWB transmitter andOperation control block
Wireless power supply
Outside body
128 Mbps(wireless)
UWB receiver andPC connection block
[email protected] GHz ±0.625 GHz
6.4Mbps(8 x 800kbps)
x8
1~64ch
512ch base unit No.8
RecordingNo.1
65~128ch Recording
No.1
3585~4096ch
RecordingNo.64
MUX-A(FPGA)
1~64ch
512ch base unit No.1
RecordingNo.1
65~128ch Recording
No.1
449~512ch
RecordingNo.8
MUX-A(FPGA)
800kbps
MUX-B(FPGA)
Multi-channel recording and digital multiplexing block
51.2 Mbps(64 x 800kbps)
UWB-R(FPGA)
LED(time)
ZigBee
USB
PCZigBee
Inside body
4.9 mm2
128 Mbps(wireless)
September 2019
Slide 20 Takafumi Suzuki(NICT), Masayuki Hirata(University of Osaka)
doc.: IEEE 802.15-19-0421-02-0dep
Submission
UWB transmitter and receiver
RF board
back view
front view BPF SMT for Antenna
base board
batteryconnectorAC/DCconnector
Magnetic powerswitch connector
Zigbee board
antenna
rechargeable battery(430mAh, 3.7V)
back view
front view Zigbee boardsampling timeLED [ms]
base boardRF board
debug port
~100mm
USB port
Including UWB transmitter, BPF, antenna, Zigbee, Li-ion polymer battery
-Including UWB receiver, antenna, Zigbee unit-Connected to PC by USB2.0-Real-time Graphical view
helical antenna
September 2019
Slide 21 Takafumi Suzuki(NICT), Masayuki Hirata(University of Osaka)
doc.: IEEE 802.15-19-0421-02-0dep
Submission
UWB system (Specification)Number of channels 64~4096 ch
Supply voltage 3.3 V
Total power of implant devices 2.03 W @4096ch (version 2014)
UWB bandwidth 7.275~8.525 GHz for internationally usage
UWB data rate 128 Mbps
MUX-A and MUX-B (common hardware)
Xilinx Spartan6 XC6SLX16, FPC connector (11pin)
UWB transmitter (water proof casing)
RF, BPF and ZigBee board
LVDS receiver (51.2Mbps), Magnetic Power SW (20mm range)and rechargeable Li polymer battery (400mAh)
UWB receiver
RF board, base board (Xilinx vertex4, USB2.0) and Zigbee board
PC
Core i7 3820 Win7 (USB2.0, GUI application)
September 2019
Slide 22 Takafumi Suzuki(NICT), Masayuki Hirata(University of Osaka)
doc.: IEEE 802.15-19-0421-02-0dep
Submission
Evaluation of UWB in human equivalent liquid phantom
Measured characteristics of received signal strength.
(a) Diagram and (b) photograph of the measurement setup.
Packet error rate with respect to the phantom depth.
OK
NG
too near
too far
128 Mbps UWB wireless communication is available below 20 mm between inside to outside body.
20 dB dropin body
September 2019
Slide 23 Takafumi Suzuki(NICT), Masayuki Hirata(University of Osaka)
doc.: IEEE 802.15-19-0421-02-0dep
Submission
Summary
Slide 24 Takafumi Suzuki(NICT), Masayuki Hirata(Osaka University)
September 2019
Our collaborative project 1st and 2nd generation ofECoG BMI are introduced.
Medical applications will needhigh speed (100Mbps~) and securewireless communication.
doc.: IEEE 802.15-19-0421-02-0dep
Submission
September 2019
Thank you for your attention
Slide 25 Takafumi Suzuki(NICT), Masayuki Hirata(University of Osaka)