slides set number 1.. class 478/578: general 1 1.my name is marek perkowski 2.you can call my marek,...
Post on 22-Dec-2015
221 views
TRANSCRIPT
SLIDES SET NUMBER 1.
Class 478/578: General 11. My name is Marek Perkowski
2. You can call my Marek, or Dr. Perkowski or whatever you like.
3. This class is fun, at least for me.
4. I hope that you will have fun also.
5. We build practical robots – embedded systems
6. Class is graded based on practical achievements, a little bit similar to Capstone Project.
7. You can find all information on my webpage, find me through Google.
Class 478/578: General 21. If you are a graduate student your project is more
difficult, otherwise the same.
2. Two homeworks and Project
3. No exam.
4. Student presentations (related to homeworks or projects)
5. I expect high quality of reports (many graduate students had publications based on these reports)
6. Robots connected to Internet (demo and explanation next Thursday).
Class 478/578: grading1. Homework 1 – 10 % (evolutionary algorithms and
foraging)
2. Homework 2 – 10 % (any subset of your project)
3. Presentation – 10 %
4. Project – 70 %
5. Groups – 1 to 5 students, group leader.
6. In final report, each student has a separate part to demonstrate his/her work.
7. Each student presents a separate presentation of his work.
Class 478/578: book1. Braunl.
– You can find slides to this book on internet– Book was ordered early but it must be reprinted “on
demand”.– If you have no book, do not worry. All is in my slides.– Somebody told me that PDF of all text is also on
internet
• Slides of my class on my webpage – look for “Embedded Robotics” on my main webpage.
• To find my webpage do search on Google “Marek Perkowski”
Class 478/578: your background
1. Programming– Matlab– C– C++– Java
2. Some basic digital design and interfacing experience (only in some projects)
3. Some basic math, Boolean Algebra, probability.
4. Digital Signal Processing, Image Processing (for some projects, will be covered in debth in ECE 479 next quarter)
Class 478/578: your background review
1. Boolean functions, gates and circuits
2. Finite State Machines
3. Probabilistic State Machines
4. Grammars
5. Linked Lists
6. Arduino
Class 478/578: your background information
Please give me today the following information:
1.Your first name, last name and contact (email, phone)
2.Do you want to be on my Facebook – send me message on Facebook.
3.Programming classes you have taken.
4.Programming projects you have done.
5.Robot projects you have done. Please write more.
6.Any hardware projects you have done, like fixing a radio or a computer, building a FPGA controller etc.
7.Your background (hardware, software, art, physics, math, biology, etc)
8.Are you a graduate or undergraduate student.
9.For each of three areas: theory, programming and practical robot building, write percentages of your project’s grade (I am not sure I will be able to take this into account in every case)
10.Do you prefer to work alone or in a team for this class?
Class 478/578: your background information
Please give me today the following information:
1.Your first name, last name and contact (email, phone)
2.Do you want to be on my Facebook – send me message on Facebook.
3.Programming classes you have taken.
4.Programming projects you have done.
5.Robot projects you have done. Please write more.
6.Any hardware projects you have done, like fixing a radio or a computer, building a FPGA controller etc.
7.Your background (hardware, software, art, physics, math, biology, etc)
8.Are you a graduate or undergraduate student.
9.For each of three areas: theory, programming and practical robot building, write percentages of your project’s grade (I am not sure I will be able to take this into account in every case)
10.Do you prefer to work alone or in a team for this class?
Class 478/578: Projects and Lab
1. Meeting with Chris Clark
2. Meeting with class TA
3. Webpages with previous projects
4. Interfacing to internet
5. Lab keys (cards)
Class 478/578: Projects for this year
1. Dancing hexapods
2. Foraging hexapods
3. Robot Theatre
4. Sustainable Robot for advertising
5. Robot Guide for PSU
6. Robots controlled by iPhones, Ipads, etc.
7. Advanced theories for robotics (only for individual graduate students)
EMBEDDED EMBEDDED SYSTEMSSYSTEMS
• Textbook:• T. Bräunl Embedded Robotics, Springer
2003
Plan of class
• Week 1:– Servo programming– Evolutionary algorithms
• Week 2:– Humanoid Robots– Models of robotics
• Mapping, grammars, automata, probabilistic, Braitenberg Vehicles, natural language, logic based learning.
1.1 Definition
• Definition for: embedded system
• A combination of hardware and software which together form a component of a larger machine.
• An example of an embedded system is a microprocessor
that controls an automobile engine.
• An embedded system is designed to run on its own without human intervention, and may be required to respond to events in real time.
• Source: www.computeruser.com/resources/dictionary
Applications Applications AreasAreas
Application Areas• TV• stereo• remote control• phone / mobile phone• refrigerator• microwave• washing machine• electric tooth brush• oven / rice or bread cooker• watch• alarm clock• electronic musical instruments• electronic toys (stuffed animals,handheld toys, pinballs, etc.)• medical home equipment (e.g. bloodpressure, thermometer)• …• [PDAs?? More like standard computer system]
Consumer Products
Application Areas
• Medical Systems– pace maker, patient monitoring systems, injection systems,
intensive care units, …
• Office Equipment– printer, copier, fax, …
• Tools– multimeter, oscilloscope, line tester, GPS, …
• Banking– ATMs, statement printers, …
• Transportation – (Planes/Trains/[Automobiles] and Boats)
• radar, traffic lights, signalling systems, …
Application Areas• Automobiles
– engine management, trip computer, cruise control, immobilizer, car alarm,
– airbag, ABS, ESP, …
• Building Systems– elevator, heater, air conditioning, lighting, key
card entries, locks, alarm systems, …
• Agriculture– feeding systems, milking systems, …
• Space– satellite systems, …
Application Areas
• Facts:– 1997: The average U.S. household has over 10
embedded computers (source: www.it.dtu.dk/~jan)• 1998: 90% Embedded Systems vs. 10%
Computers– (source: Frautschi, www.caliberlearning.com)
• 2001: The Volvo S80 has 18 embedded controllers and 2 busses (source: Volvo)
AutomobilesAutomobiles
Robot Robot Metaphors Metaphors and Modelsand Models
Animatronic “Robot” or Animatronic “Robot” or devicedevice
brain effectors
Perceiving “Robot”Perceiving “Robot”
brainsensors
Reactive Robot Reactive Robot is the is the simplest behavioral robotsimplest behavioral robot
Brain is a
mapping
sensors
This is the simplest robot that satisfies the definition of a robot
effectors
Reactive Robot in environmentReactive Robot in environment
brainsensors
This is the simplest robot that satisfies the definition of a robot
effectors
ENVIRONMENT is a feedback
Braitenberg Braitenberg Vehicles and Vehicles and Quantum Quantum Automata RobotsAutomata Robots
Another Example: Another Example: Braitenberg Braitenberg Vehicles and Quantum BVVehicles and Quantum BV
Braitenberg VehiclesBraitenberg Vehicles
Braitenberg Vehicles: Braitenberg Vehicles: Homework 1 ideaHomework 1 idea
1. Can you think about other robot behaviors?
2. Can you develop software for robots with other mechanics/kinematics but the same emergent principles?
3. Design circuits for switchable behaviors: like sound that switches from shy to aggressive robot.
Emotional Robot Emotional Robot has a has a simple form of memory or statesimple form of memory or state
Brain is a
Finite State
Machine
sensors
This is the simplest robot that satisfies the definition of a robot
effectors
Behavior as an interpretation of a string
• Newton, Einstein and Bohr.
• Hello Professor
• Hello Sir
• Turn Left . Turn right.
behavior
Behavior as an interpretation of a tree
• Newton, Einstein and Bohr.
• Hello Professor
• Hello Sir
• Turn Left . Turn right.
behavior
Grammar. Derivation. Alphabets.
Our First Base Our First Base Robot Robot Architecture Architecture and Designsand Designs
Fig. 1. Learning Behaviors as Mappings fromFig. 1. Learning Behaviors as Mappings fromenvironment’s features to interaction proceduresenvironment’s features to interaction procedures
AutomaticAutomaticsoftwaresoftwareconstructionconstructionfrom examplesfrom examples
(decision tree, (decision tree, bibi--decomposition,decomposition,AshenhurstAshenhurst, DNF), DNF)
Speech frommicrophones
Image featuresfrom cameras
Sonars and othersensors
Emotions andknowledge memory
probability Verbal responsegeneration (textresponse and TTS).Stored sounds
Headmovementsand facialemotionsgeneration
Neck and shouldersmovement generation
Neck and upper body movement
generation
Robot Head Construction, 1999Robot Head Construction, 1999
Furby head with new controlFurby head with new control JonasJonas
We built and animated various kinds of humanoid heads with from 4 to 20 DOF, looking for comical and entertaining values.
High school summer camps, hobby roboticists, undergraduates
Mister ButcherMister Butcher
4 degree of freedom neck
Latex skin from Hollywood
Robot Head Construction, 2000Robot Head Construction, 2000
SkeletonSkeleton Alien
We use inexpensive servos from Hitec and Futaba, plastic, playwood and aluminum.
The robots are either PC-interfaced, use simple micro-controllers such as Basic Stamp, or are radio controlled from a PC or by the user.
AdamAdamMarvin the Crazy RobotMarvin the Crazy Robot
Technical Construction, 2001 Technical Construction, 2001 DetailsDetails
Virginia WoolfVirginia Woolf
heads equipped with microphones, USB cameras, sonars heads equipped with microphones, USB cameras, sonars and CDS light sensorsand CDS light sensors
20012001
MaxMax
Image processing and pattern recognition uses software developed at PSU, CMU and Intel (public domain software available on WWW). Software is in Visual C++, Visual Basic, Lisp and Prolog.
BUG (Big Ugly Robot)BUG (Big Ugly Robot)
20022002
Visual Feedback and Learning based on Visual Feedback and Learning based on Constructive InductionConstructive Induction
20022002Uland Wong, 17 years old
Professor Perky Professor Perky
1 dollar latex skin 1 dollar latex skin from Chinafrom China
• We compared several commercial speech systems from Microsoft, Sensory and Fonix. •Based on experiences in highly noisy environments and with a variety of speakers, we selected Fonix for both ASR and TTS for Professor Perky and Maria robots.
• We use microphone array from Andrea Electronics.
Professor Perky with automated Professor Perky with automated speech recognition (ASR) and speech recognition (ASR) and text-to-speech (TTS) capabilitiestext-to-speech (TTS) capabilities
2002, Japan
Maria, Maria, 2002/20032002/2003
20 DOF
Construction Construction details of Mariadetails of Maria
location of location of controlling controlling rodsrods
location location of head of head servosservos
location location of remote of remote servosservosCustom
designed skin
skull
Animation of eyes and eyelidsAnimation of eyes and eyelids
Cynthia, 2004, June
Currently the hands
are not moveable.
We have a separate hand design project.
Software/Hardware Architecture•Network- 10 processors, ultimately 100 processors.
•Robotics Processors. ACS 16
•Speech cards on Intel grant
•More cameras
•Tracking in all robots.
•Robotic languages – Alice and Cyc-like technologies.
Face detection localizes the person and is the first step for feature and face recognition.
Acquiring information about the human: face detection and recognition, speech recognition and sensors.
Face features recognition and visualization.
Use of Multiple-Valued (five-valued) variables Smile, Mouth_Open and Eye_Brow_Raise for facial feature and face recognition.
HAHOE KAIST ROBOT THEATRE, KOREA, SUMMER 2004
Sonbi, the Confucian Scholar Paekchong, the bad butcher
Czy znacie dobra sztuke dla teatru robotow?
Editing movementsEditing movements
Yangban the
Aristocrat and Pune
his concubine
The Narrator
The Narrator
We base all our robots on inexpensive radio-controlled servo technology.
We are familiar with latex and polyester technologies for faces
Martin Lukac and Jeff Allen wait for your help, whether you want to program, design behaviors, add muscles, improve vision, etc.
New Silicone Skins
A simplified diagram of software explaining the principle of using machine learning based on constructive induction to create new interaction modes of a human and a robot.
Probabilistic Probabilistic and Finite State and Finite State MachinesMachines
Probabilistic State Machines to describe Probabilistic State Machines to describe emotionsemotions
Happy state
Ironic state
Unhappy state
“you are beautiful”
/ ”Thanks for a compliment”
“you are blonde!”
/ ”I am not an idiot”
P=1
P=0.3
“you are blonde!”
/ Do you suggest I am an idiot?”
P=0.7
Facial Behaviors of MariaFacial Behaviors of Maria
Do I look like younger than twenty three?Maria asks:Maria asks:
“yes”
“no” “no”
0.30.7
Response:Response:
Maria smilesMaria smilesMaria frownsMaria frowns
Probabilistic Grammars for performancesProbabilistic Grammars for performances
Who?
What?
Where?
Speak ”Professor Perky”, blinks eyes twice
Speak “In the classroom”, shakes head
P=0.1
Speak “Was drinking wine”
P=0.1
P=0.3
P=0.5
Speak ”Professor Perky”
Speak ”Doctor Lee”
Speak “in some location”, smiles broadly
Speak “Was singing and dancing”
P=0.5 P=0.5
P=0.1 P=0.1
….
P=0.1
Human-controlled modes of Human-controlled modes of dialog/interactiondialog/interaction
Robot asks
Human teaches
Human commandsHuman asks
Robot performs
“Hello Maria”
“Thanks, I have a question”
“Thanks, I have a lesson”
“Thanks, I have a command”
“Lesson finished”
“Questioning finished”
“Command finished”
“Stop performance”
“Question”
Dialog and Dialog and Robot’s Robot’s KnowledgeKnowledge
Robot-Receptionist Initiated Robot-Receptionist Initiated ConversationConversation
Robot
What can I do for you?What can I do for you?
Human
Robot asksThis represents operation mode
Robot-Receptionist Initiated Robot-Receptionist Initiated ConversationConversation
Robot
What can I do for you?What can I do for you?
Human
I would like to order a I would like to order a table for twotable for two
Robot asks
Robot-Receptionist Initiated Robot-Receptionist Initiated ConversationConversation
Robot
Smoking or non-Smoking or non-smoking?smoking?
Human
Robot asks
Robot-Receptionist Initiated Robot-Receptionist Initiated ConversationConversation
Robot
Smoking or non-Smoking or non-smoking?smoking?
Human
I do not understandI do not understand
Robot asks
Robot-Receptionist Initiated Robot-Receptionist Initiated ConversationConversation
Robot
Do you want a table in a Do you want a table in a smoking or non-smoking smoking or non-smoking section of the restaurant? section of the restaurant?
Non-smoking section is Non-smoking section is near the terrace.near the terrace.
Human
Robot asks
Robot-Receptionist Initiated Robot-Receptionist Initiated ConversationConversation
Robot
Do you want a table in a Do you want a table in a smoking or non-smoking smoking or non-smoking section of the restaurant? section of the restaurant?
Non-smoking section is Non-smoking section is near the terrace.near the terrace.
Human
A table near the terrace, please
Robot asks
Human-Initiated ConversationHuman-Initiated Conversation
Robot Human
Hello Maria
Robot asks
initialization
Human-Initiated ConversationHuman-Initiated Conversation
Robot Human
Hello MariaWhat can I do for you?
Robot asks
Human-AskingHuman-Asking
Robot Human
Question
Human asksQuestionRobot asks
Human-AskingHuman-Asking
Robot Human
Question
Human asks
Yes, you ask a question.
Human-AskingHuman-Asking
Robot Human
What book wrote Lee?
Human asks
Yes, you ask a question.
Human-AskingHuman-Asking
Robot Human
What book wrote Lee?
Human asks
I have no sure information.
Human-AskingHuman-Asking
Robot Human
Try to guess.
Human asks
I have no sure information.
Human-AskingHuman-Asking
Robot Human
Try to guess.
Human asks
Lee wrote book “Flowers”.
Human-AskingHuman-Asking
Robot Human
This is not true.
Human asks
Lee wrote book “Flowers”.
Human-TeachingHuman-Teaching
Robot Human
Questioning finished
Human teaches“Questioning finished” Robot asks
Human asks
Thanks, I have a lesson
Human endsHuman endsquestioningquestioning
Human-TeachingHuman-Teaching
Robot Human
Questioning finished
Human teaches“Questioning finished” Robot asks
Human asks
Thanks, I have a lesson
Robot enters Robot enters asking modeasking mode
What can I do for you?What can I do for you?
Human-TeachingHuman-Teaching
Robot Human
Thanks, I have a lesson
Human teaches“Questioning finished” Robot asks
Human asks
Thanks, I have a lesson
Human starts Human starts teachingteaching
What can I do for you?What can I do for you?
Human-TeachingHuman-Teaching
Robot Human
Thanks, I have a lesson
Yes
Human teaches
Human-TeachingHuman-Teaching
Robot Human
I give you question-answer patternYes
Human teaches
Human-TeachingHuman-Teaching
Robot Human
Question pattern:
What book Smith wrote?Yes
Human teaches
Robot Human
Answer pattern:
Smith wrote book “Automata Theory”
Yes
Human teaches
Human-TeachingHuman-Teaching
Human-TeachingHuman-Teaching
Robot Human
Checking question:
What book wrote Smith?Yes
Human teaches
Human-TeachingHuman-Teaching
Robot Human
Checking question:
What book wrote Smith?Smith wrote book “Automata Theory”
Human teaches
Human-TeachingHuman-Teaching
Robot Human
I give you question-answer patternYes
Human teaches
Human-TeachingHuman-Teaching
Robot Human
Question pattern:
Where is room of Lee?Yes
Human teaches
Human-TeachingHuman-Teaching
Robot Human
Answer pattern:
Lee is in room 332Yes
Human teaches
Human-Checking what robot Human-Checking what robot learnedlearned
Robot Human
Lesson finished
Human asksQuestionRobot asks
Human teaches“Lesson finished”
Human-Checking what robot Human-Checking what robot learnedlearned
Robot Human
Lesson finished
Human asksQuestionRobot asks
Human teaches“Lesson finished”
What can I do for you?
Human-Checking what robot Human-Checking what robot learnedlearned
Robot Human
Question
Human asksQuestionRobot asks
Human teaches“Lesson finished”
What can I do for you?
Human-AskingHuman-Asking
Robot Human
Question
Human asksQuestionRobot asks
Human teaches“Lesson finished”
Yes, you ask a question.
Human-AskingHuman-Asking
Robot Human
What book wrote Lee?
Human asks
Yes, you ask a question.
Human-AskingHuman-Asking
Robot Human
What book wrote Lee?
Human asks
I have no sure information.
Human-AskingHuman-Asking
Robot Human
Try to guess.
Human asks
I have no sure information.
Human-AskingHuman-Asking
Robot Human
Try to guess.
Human asks
Lee wrote book “Automata Theory”
Observe that robot found similarity between Smith and Lee and generalized (incorrectly)
Behavior, Dialog and LearningBehavior, Dialog and Learning
• The dialog/behavior has the following components:
– (1) Eliza-like natural language dialogs based on pattern matching and limited parsing.
• Commercial products like Memoni, Dog.Com, Heart, Alice, and Doctor all use this technology, very successfully – for instance Alice program won the 2001 Turing competition.
– This is a “conversational” part of the robot brain, based on pattern-matching, parsing and black-board principles.
– It is also a kind of “operating system” of the robot, which supervises other subroutines.
• (2) Subroutines with logical data base and natural language parsing (CHAT).
– This is the logical part of the brain used to find connections between places, timings and all kind of logical and relational reasonings, such as answering questions about Japanese geography.
Behavior, Dialog and LearningBehavior, Dialog and Learning
• (3) Use of generalization and analogy in dialog on many levels. – Random and intentional linking of spoken language,
sound effects and facial gestures. – Use of Constructive Induction approach to help
generalization, analogy reasoning and probabilistic generations in verbal and non-verbal dialog, like learning when to smile or turn the head off the partner.
Behavior, Dialog and LearningBehavior, Dialog and Learning
• (4) Model of the robot, model of the user, scenario of the situation, history of the dialog, all used in the conversation.
• (5) Use of word spotting in speech recognition rather than single word or continuous speech recognition.
• • (6) Continuous speech recognition (Microsoft)
• (7) Avoidance of “I do not know”, “I do not understand” answers from the robot. – Our robot will have always something to say, in the worst case,
over-generalized, with not valid analogies or even nonsensical and random.
Behavior, Dialog and LearningBehavior, Dialog and Learning
Questions to students1. Present a concept of a robot with architecture based on combinational logic
mapping. Design a function from gates.
2. Present a concept of a robot with architecture based on deterministic Finite State Machine. Show a graph or table of the machine. You can also draw a flowchart.
3. Present a concept of a robot with architecture based on probabilistic Finite State Machine. Show a graph or table of the machine.
4. Present a software internet robot for natural language conversation, similar to receptionist robot from this set of slides. The robot should discuss Intelligent Robotics Laboratory, its research, faculty and students. What are the “states of robot”? What are the key-words to transit from state to state, draw a diagram.
5. Analyze four different Braitenberg Vehicles based on a robot with kinematics of a standard car. Two of them can be similar to Shy and Aggressive robots from class.
6. Analyze four different “Braitenberg-like robots” that have a head and one hand. Two of them can be similar to Shy and Aggressive robots from class
This is not a homework, just to test your knowledge. You do not have to give it to me but you may if you want.