welcome [sercuarc.org]...2017/02/01 · serc talks february 1, 2017 1 today’s session will be...

SERC Talks February 1, 2017 1

Today’s session will be recorded.

An archive of today’s talk will be available at: www.SERCuarc.org

Use the Q&A box to queue questions, reserving the chat box for comments, and

questions will be answered during the last 5-10 minutes of the session.

If you are connected via the dial-in information only, please email questions or

comments to Ms. Mimi Marcus at [email protected].

Any issues? Use the chat feature for any technical difficulties or other comments, or

email Ms. Mimi Marcus at [email protected].

WELCOME

What is the Self?

Grady Booch, IBM Almaden Research Laboratory

February 1, 2017 | 1:00 pm ET

http://www.sercuarc.org/

mailto:[email protected]

mailto:[email protected]


The Systems Engineering Research Center (SERC) is a federally funded University Affiliated Research Center managed by Stevens Institute of Technology.

Any views, opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the United States Department of Defense, ASD(R&E), nor the SERC.

No Warranty. This Stevens Institute of Technology Material is furnished on an “as-is” basis. Stevens Institute of Technology makes no warranties of any kind, either expressed or implied, as to any matter including, but not limited to, warranty of fitness for purpose or merchantability, exclusivity, or results obtained from use of the material. Stevens Institute of Technology does not make any warranty of any kind with respect to freedom from patent, trademark, or copyright infringement.

This material has been approved for public release and unlimited distribution.

© 2017 IBM Corporation 1

What Is The Self?

Grady Booch


Imagine unleashing Watson in the physical world. Give it eyes, ears, and touch, then let it act in that world with hands and feet and a face, not just as an action of force but also as an action of influence. This is embodied cognition: by placing the cognitive power of Watson in a robot, in an avatar, an object in your hand, or even in the walls of an operating room, conference room, or spacecraft, we take Watson's ability to understand and reason and draw it closer to the natural ways in which humans live and work. In so doing, we augment individual human senses and abilities, giving Watson the ability see a patient's complete medical condition, feel the flow of a supply chain, or orchestrate the tasks in a day in the life of an individual.


Embodied Cognition Defined

An embodied cognition must– Be in and of the world– Reason– Learn– Have identity

An embodied cognition is not– Simply an STT -> NLC -> action pipeline


Embodiment

Raw curiosity

More computational

power

Advances in AI algorithms and architectures

Better/smaller sensors and

actuators

Better batteries

3D printing

Advances in neural implants

New military doctrine

Demographic changes

Economic need

Advances in neural implants Technology push

Demand pullRaw curiosity


An Agent May Be Embodied In A Robot…


Dependent Autonomous


Fixed place Mobile


Thing-facing Human-facing


Predictable Responsive Social


Architecturally, robots may be classified along one of several dimensions


Biologically-organized

Symbolically-organized


Thin Client Thick Client


Thin Client Thick Client

By way of context, Watson is largely a symbolically-organized thin client cognitive system

Biologically-organized

Symbolically-organized


An Avatar


A Space


Or An Object


Concierge/Greeter

Where is the elevator?

Home

Did I turn off the stove?

Hotel

Bring me some fresh towels

Retail

Where can I find this in my size?

Elder Care

Don’t forget to take your pills

Cobot

Get me a screwdriver

Fabrication

Watch me do this

Boardroom

Help me decide

Companion

Let’s play a game

Tutor

How do I solve this problem?

Get me a screwdriver

Transportation

Use Cases


Embodied Cognition Differentiated

Embodiment offers far more than request/response behavior Embodiment is more than just conversation Agency Personalization Multi-modal Learns (not programed)


Concierge/Greeter

Avatar, Robot, Device

Home

Space, Avatar, Robot, Device

Hotel


Retail


Elder Care

Space, Robot, Device

Cobot

Robot,Avatar

Fabrication

Robot

Boardroom

Space, Avatar, Device

Companion

Avatar, Robot, Device

Tutor

Avatar, Device

Space, Avatar

Transportation

Form Factors


Self

20

Architecture


Who said that?

Sensory Fusion

I feel sad.

Theory of Mind

I need the Phillips head screwdriver.

Context

Give me that one; no, I mean that one!

Embodied Conversation

It’s good to see you again, Sandia!

Learning

I’ve just picked up a fault in the AE35 unit.

Devices

In what room was Alyssa working yesterday?

Knowledge Representation

Technical Issues

You don’t have permission to do that

Security

Room 806 has a light out

Scale


Intu Deployment

Access and configure Bluemixservices (via web)

Manage all Project Intu instances (via desktop/laptop)

SDK on devices (via Avatar, Raspberry Pi, Robots, Desktop/Laptop)

Project Intu Gateway Project Intu Manager Project Intu Kernel


Research Portfolio

Core Technology Topic Project NamePlatform Self

Goals & Planning

Planning

Embodied Action

Multi-modal Contextual Interaction

Embodied Decision Support Systems

Models of the World, Others, and Self

Spatial Intelligence

People Intelligence

Spatial Awareness

Theory of Mind

Learning Learning by Example

Sensors & Perception/ActuatorsPhysical Interaction

Edge Intelligence

Platform & Meta Management Tooling

23


Background

In November 2015, Rob High, Ray Chancey, and I were brainstorming about a suitable grand challenge for the 2016 Embodied Cognition GTO.

We settled on a very grand one: build a system to support mission control, flight operations, and the scientific agenda of NASA’s Mission to Mars; the requirements of this mission lead to the concept of Watson in the walls (an embodied cognition for the Orion spacecraft and the Mars habitat) together with robots and other devices powered by Watson (namely, Robonaut and its peers).

Recognizing this was a 20+ year research effort, I set out to devise an architecture that would not only support this Mission to Mars but the journey of which would yield a middleware that would attend to a variety of use cases, from concierge to retail to elder care and many others.

Ray named this architecture “Self”.


Self Basic Principles

Augment human capabilities

Learn, don’t program

Theory of mind and social intelligence

Self understanding

Platform agnostic

Embodiment as a robot, an avatar, a device, or a space

Deployment as middleware with microservices in the cloud


Self Significant Design Decisions

Self is a hybrid architecture, encompassing explicit symbolic computation in the center together with neural networks at the edges.

Inspired by Minsky’s Society of Mind, behavior takes place in the context of multiple concurrent agents that communicate opportunistically via blackboards and deterministically via peer to peer connections.

Inspired by Brooks’s subsumption architecture, behavior takes place in a hierarchy of cognition, from involuntary reflexes to voluntary skills to goals and planning.

We maintain a clear separation of concerns among perception, actuating, models, and behavior.

As much as possible, behavior is either taught or is learned, not programmed.

As much as possible – driven by these separation of concerns, the needs of packaging, and performance – all components are made manifest as RESTful microservices.

As much as possible, plans, skills, and reflexes are extensible.

Self is intentionally full of strange loops: components of Self are also parts of the models of itself.

Self is intentionally fractal: an instance of Self may have models of others, which themselves are other instances of Self.


Self Markitecture

IBM Confidential 27

Sensors & Perception

Actuators

Agents

Goals & Planning

Platform Management Infrastructure

Meta Management

Models

Voluntary Behavior & Skills

Involuntary Behavior & Reflexes

Others

World

Self


Embodied cognition represents the next generation of Watson


Embodied cognition represents the extension of IT into the physical world, bringing cognition to the edge of the Internet of Things


At this touch point we encounter the intersection of big data and human presence, mediated by cognitive systems that reason


Text

Watson can be the back end to a variety of embodied cognitive systems.


Text + Audio


Text + Audio

This is the current state of an embodied Watson(the low hanging fruit, wherein platforms such as Pepper serve as animated speakers)


Text + Audio

There is much, much more that is possible(if we extent the embodiment to encompass greater perception and action in the world)


Text + Audio + Vision


Text + Audio + Vision + Emotion


Text + Audio + Vision + Emotion + Motion


Text + Audio + Vision + Emotion + Motion + Physical interaction


Text + Audio + Vision + Emotion + Motion + Physical interaction

Socially-intelligent cooperative robot


Can Graphical Models Provide a Sufficient Basis for General Intelligence?Dr. Paul S. Rosenbloom, Institute for Creative Technologies, University of Southern CaliforniaApril 5, 2017 | 1:00 pm ET

UPCOMING TOPICS:

What Are Cyber-Social Learning Systems And How Will We Form Them?

Dr. Kevin Sullivan, University of Virginia

June 7, 2017 | 1:00 pm ET

Thank you for joining us! Please check back on the SERC website for today’s recording and future SERC Talks information!

http://www.sercuarc.org/serc-talks/

welcome [sercuarc.org]...2017/02/01 · serc talks february 1, 2017 1 today’s session will be...

Documents