Machine Learning:

Download Machine Learning:

Post on 10-May-2015




0 download

Embed Size (px)


<ul><li>1.ELEVATOR L GUEST LAUNDRY </li></ul><p>2. Schedule09:00 Registration, poster set-up, and continental breakfast 09:30 Welcome 09:45 Invited Talk: Machine Learning in SpaceKiri L. Wagstaff, N.A.S.A. 10:15 A general agnostic active learning algorithmClaire Monteleoni, UC San Diego 10:35 Bayesian Nonparametric Regression with Local ModelsJo-Anne Ting, University of Southern California 10:55 Coffee Break 11:15 Invited Talk: Applying machine learning to a real-worldproblem: real-time ranking of electric componentsMarta Arias, Columbia University 11:45 Generating Summary Keywords for Emails Using Topics.Hanna Wallach, University of Cambridge 12:05 Continuous-State POMDPs with Hybrid DynamicsEmma Brunskill, MIT 12:25 Spotlights 12:45 Lunch 14:20 Invited Talk: Randomized Approaches to Preserving PrivacyNina Mishra, University of Virginia 14:50 Clustering Social NetworksIsabelle Stanton, University of Virginia 15:10 Coffee Break15:30Invited Talk: Applications of Machine Learning to ImageRetrievalSally Goldman, Washington University 16:00 Improvement in Performance of Learning Using ScalingSoumi Ray, University of Maryland Baltimore County 16:20 Poster Session 17:10 Panel/ Open Discussion 17:40 Concluding Remarks 3. Invited TalksMachine Learning in Space Kiri L. Wagstaff, N.A.S.A.Remote space environments simultaneously present signicant challenges to the machine learning community and enormous opportunities for advancement. In this talk, I present recent work on three key issues associated with machine learning in space: on-board data classication and regression, on-board prioritization of analysis results, and reliable computing in high-radiation environments. Support vector machines are currently being used on-board the EO-1 Earth orbiter, and they are poised for adoption by the Mars Odyssey orbiter as well. We have developed techniques for learning scientist preferences for which subset of images is most critical for transmission, so that we can make the most use of limited bandwidth. Finally, we have developed fault-tolerant SVMs that can detect and recover from radiation-induced errors while performing on- board data analysis.About the speaker:Kiri L. Wagstaff is a senior researcher at the Jet Propulsion Laboratory in Pasadena, CA. She is a member of the Machine Learning and Instrument Autonomy group, and her focus is on developing new machine learning methods that can be used for data analysis on-board spacecraft. She has applied these techniques to data being collected by the EO-1 Earth-orbiting spacecraft, Mars Odyssey, and Mars Pathnder. She has also worked on crop yield prediction from orbital remote sensing observations, the fault protection system for the MESSENGER mission to Mercury, and automatic code generation for the Electraradio used by the Mars Reconnaissance Orbiter and the Mars Science Laboratory. She is very interested in issues such as robustness (developing fault-tolerant machine learning methods for high-radiation environments) and infusion (how can machine learning be used to advance science?). She holds a Ph.D. in Computer Science from Cornell University and is currently working on an M.S. in Geology from the University of Southern California. 4. Applying machine learning to a real-world problem: real-time ranking of electric components Marta Arias, Columbia UniversityIn this talk, I will describe our experience with applying machine learning techniques to a concrete real-world problem: the generation of rankings of electric components according to their susceptibility to failure. The system's goal is to aid operators in the replacement strategy of most at-risk components and in handling emergency situations. In particular, I will address the challenge of dealing with the concept drift inherent in the electrical system and will describe our solution based on a simple weighted-majority voting scheme.About the speaker:Marta Arias received her bachelor's degree in ComputerScience from the Polytechnic University of Catalunya(Barcelona, Spain) in 1998. After that she worked for ayear at Incyta S.A. (Barcelona, Spain), a companyspecializing in software products for Natural LanguageProcessing applications. She then enrolled in the graduatestudent program at Tufts University, recieving her PhD inComputer Science in 2004. That same year she joined theCenter for Computational Learning Systems of ColumbiaUniversity as an Associate Research Scientist. Dr. Arias' research interest include the theory and application of machine learning. 5. Randomized Approaches to Preserving Privacy Nina Mishra, University of Virginia, Microsoft ResearchThe Internet is arguably one of the most important inventions of the last century. It has altered the very nature of our lives -- the way we communicate, work, shop, vote, recreate, etc. The impact has been phenomenal for the machine learning community since both old and newly created information repositories, such as medical records and web click streams, are readily available and waiting to be mined. However, opposite these capabilities and advances is the basic right to privacy: On the one hand, in order to best serve and protect its citizens, the government should ideally have access to every available bit of societal information. On the other hand, privacy is a fundamental right and human need, which theoretically is served best when the government knows nothing about the personal lives of its citizens. This raises the natural question of whether it is even possible to simultaneously realize both of these diametrically opposed goals, namely, information transparency and individual privacy. Surprisingly, the answer is yes and I will describe solutions where individuals randomly perturb and publish their data so as to preserve their own privacy and yet large-scale information can still be learned. Joint work with Mark Sandler.About the speaker: Nina Mishra is an Associate Professor in the Computer Science Department at the University of Virginia. Her research interests are in data mining and machine learning algorithms as well as privacy. She previously held joint appointments as a Senior Research Scientist at HP Labs, and as an Acting Faculty member at Stanford University. She was Program Chair of the International Conference on Machine Learning in 2003 and has served on numerous data mining and machine learning program committees. She also serves on the editorial Boards of Machine Learning, IEEE Transactions on Knowledge and Data Engineering, IEEE Intelligent Systems and theJournal of Privacy and Condentiality. She is currently on leave in Search Labs at Microsoft Research. She received a PhD in Computer Science from UIUC. 6. Applications of Machine Learning to Image Retrieval Sally Goldman, Washington UniversityClassic Content-Based Image Retrieval (CBIR) takes a single non-annotated query image, and retrieves similar images from an image repository. Such a search must rely upon a holistic (or global) view of the image. Yet often the desired content of an image is not holistic, but is localized. Specically, we dene Localized Content-Based Image Retrieval as a CBIR task where the user is only interested in a portion of the image, and the rest of the image is irrelevant. We discuss our localized CBIR system, Accio!, that uses labeled images in conjunction with a multiple-instance learning algorithm to rst identify the desired object and re-weight the features, and then to rank images in the database using a similarity measure that is based upon individual regions within the image. We will discuss both the image representation and multiple-instance learning algorithm that we have used in the localized CBIR systems that we have developed. We also look briey at ways in which multiple-instance learning can be applied to knowledge-based image segmentation.About the speaker:Dr. Sally Goldman is the Edwin H. Murty Professor ofEngineering at Washington University in St. Louis and theAssociate Chair of the Department of Computer Scienceand Engineering. She received a Bachelor of Science inComputer Science from Brown University in December1984. Under the guidance of Dr. Ronald Rivest at theMassachusetts Institute of Technology, Dr. Goldmancompleted her Master of Science in ElectricalEngineering and Computer Science in May 1987 and herPh.D. in July 1990. Dr. Goldman's research is in the areaof algorithm design and analysis and machine learningwith a recent focus on applications to the area of content-based image retrieval. Dr. Goldman has received manyteaching awards and honors including the Emerson Electric Company Excellence in Teaching Award in 1999, and the Governor's Award for Excellence in Teaching in 2001. Dr. Goldman and her husband, Dr. Ken Goldman, have just completed a book titled, A Practical Guide to Data Structures and Algorithms using Java. 7. TalksA General Agnostic Active Learning Algorithm Claire Monteleoni, UC San DiegoWe present a simple, agnostic active learning algorithm that works for any hypothesis class of bounded VC dimension, and any data distribution. Most previous work on active learning either makes strong distributional assumptions, or else is computationally prohibitive. Our algorithm extends a scheme due to Cohn, Atlas, and Ladner to the agnostic setting (i.e. arbitrary noise), by (1) reformulating it using a reduction to supervised learning and (2) showing how to apply generalization bounds even for the non-i.i.d. samples that result from selective sampling. We provide a general characterization of the label complexity of our algorithm. This quantity is never more than the usual PAC sample complexity of supervised learning, and is exponentially smaller for some hypothesis classes and distributions. We also demonstrate improvements experimentally.This is joint work with Sanjoy Dasgupta and Daniel Hsu. Currently in submission, but for a full version, please see UCSD tech report: Nonparametric Regression with Local Models Jo-Anne Ting, University of Southern CaliforniaWe propose a Bayesian nonparametric regression algorithm with locally linear models for high-dimensional, data-rich scenarios where real- time, incremental learning is necessary. Nonlinear function approximation with high-dimensional input data is a nontrivial problem. An application example is a high-dimensional movement system like a humanoid robot, where real-time learning of internal models for compliant control may be needed. Fortunately, many real-world data sets tend to have locally low dimensional distributions, despite having high dimensional embedding (e.g., Tenenbaum et al. 2000, Roweis &amp; Saul, 2000). A successful algorithm, thus, must avoid numerical problems arising potentially from redundancy in the input data, eliminate irrelevant input dimensions, and be computationally efcient to allow for incremental, online learning.Several methods have been proposed for nonlinear function approximation, such as Gaussian process regression (Williams &amp; Rasmussen, 1996), support vector regression (Smola &amp; Schlkopf, 1998) and variational Bayesian mixture models (Ghahramani &amp; Beal, 2000). However, these global methods tend to be unsuitable for fast, incremental function approximation. Atkeson, Moore &amp; Schaal (1997) have shown that in such scenarios, learning with spatially localized models is more appropriate, particularly in the framework of locally weighted learning. 8. In recent years, Vijayakumar &amp; Schaal (2000) have introduced a learningalgorithm designed to fulll the fast, incremental requirements of locally weightedlearning, specically targeting high-dimensional input domains through the use oflocal projections. This algorithm, called Locally Weighted Projection Regression(LWPR),performs competitively in its generalization performance with state-of-the-art batch regression methods. It has been applied successfully tosensorimotor learning on a humanoid robot for the purpose of executing fast,accurate movements in a feedforward controller. The major issue with LWPR is that it requires gradient descent (with leave-one-out cross-validation) to optimize the local distance metrics in each localregression model. Since gradient descent search is sensitive to the initial values,we propose a novel Bayesian treatment of locally weighted regression withlocally linear models that eliminates the need for any manual tuning of metaparameters, cross-validation approaches or sampling. Combined with variationalapproximation methods to allow for fast, tractable inference, this Bayesianalgorithm learns the optimal distance metric value for each local regressionmodel. It is able to automatically determine thesize of the neighborhood data(i.e., the ``bandwidth) that should contribute to each local model. A Bayesianapproach offers error bounds on the distance metrics and incorporates thisuncertainty in the predictive distributions. By being able to automatically detectrelevant input dimensions, our algorithm is able to handle high- dimensional datasets with a large number of redundant and/or irrelevant input dimensions and alarge number of data samples. We demonstrate competitive performance of ourBayesian locally weighted regression algorithm with Gaussian Processregression and LWPR on standard benchmark sets. We also explore extensionsof this locally linear Bayesian algorithm to a real-time setting, to offer aparameter-free alternative for incremental learning in high-dimensional spaces.Generating Summary Keywords for Emails Using Topics. Hanna Wallach, University of Cambridge Email summary keywords, used to concisely represent the gist of an email, canhelp users manage and prioritize large numbers of messages. Previous work onemail keyword selection has focused on a two-stage supervised learning systemthat selects nouns from individual emails using pre-dened linguistic rules [1]. Inthis work we present an unsupervised learning framework for selecting emailsummary keywords. A good summary keyword for an email message is not bestcharacterized as a word that is unique to that message, but a word that relatesthe message to other topically similar messages. We therefore use latentrepresentations of the underlying topics in a user's mailbox to nd words thatdescribe each message in the context of existing topics rather than selectingkeywords based on a single message in isolation. We present and compareseveral methods for selecting email summary keywords, based on two well- 9. known models for inferring latent topics: latent semantic analysis (LSA) andlatent Dirichlet allocation (LDA). Summary keywords for an email message are generated by selecting thewords that are most topically similar to the words in the email. We use twoapproaches for selecting these words, one based on query-document similarity,and the other based on word association. Each approach may be used inconjunction with either LSA or LDA. We evaluate keyword quality by...</p>