“as we may think” (vannevar bush, 1945) source: w3.org/proposal.html
DESCRIPTION
Source: w3.org/Proposal.htmlTRANSCRIPT
![Page 1: “As we may think” (Vannevar Bush, 1945) Source: w3.org/Proposal.html](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1ad97f8b9ab0599743f9/html5/thumbnails/1.jpg)
From Digital Library to Digital LibrarianA case study on Microsoft Academic ServicesKuansan WangDirector and Principal ResearcherInternet Service Research CenterMicrosoft Research, Redmond WA, USA
![Page 2: “As we may think” (Vannevar Bush, 1945) Source: w3.org/Proposal.html](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1ad97f8b9ab0599743f9/html5/thumbnails/2.jpg)
“As we may think” (Vannevar Bush, 1945)
![Page 3: “As we may think” (Vannevar Bush, 1945) Source: w3.org/Proposal.html](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1ad97f8b9ab0599743f9/html5/thumbnails/3.jpg)
Source: http://w3.org/Proposal.html
www.webat25.org
![Page 4: “As we may think” (Vannevar Bush, 1945) Source: w3.org/Proposal.html](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1ad97f8b9ab0599743f9/html5/thumbnails/4.jpg)
Lance Ulanoff, Mashable.com, December 4 2015
![Page 5: “As we may think” (Vannevar Bush, 1945) Source: w3.org/Proposal.html](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1ad97f8b9ab0599743f9/html5/thumbnails/5.jpg)
Hiro: Who worshipped Asherah?Librarian: Everyone who lived between India and
Spain, from the second millennium B.C. up into the Christian era. With the exception of the Hebrews, who only worshipped her until the religious reforms....
Hiro: I thought the Hebrews were monotheists….Librarian: Monolatrists. They did not deny the
existence of other gods. Asherah was venerated as the consort of Yahweh.
Hiro: I don't remember anything about God having a wife in the Bible.
Librarian: The Bible didn't exist at that point. Judaism was just a loose collection of Yahwistic cults, each with different shrines and practices.
Hiro and the Librarian, Chapter 30, Snow Crash (1992) , Neal Stephenson
![Page 6: “As we may think” (Vannevar Bush, 1945) Source: w3.org/Proposal.html](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1ad97f8b9ab0599743f9/html5/thumbnails/6.jpg)
In Ugarit
In EgyptIn Israel and
Judah
In Arabia
![Page 7: “As we may think” (Vannevar Bush, 1945) Source: w3.org/Proposal.html](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1ad97f8b9ab0599743f9/html5/thumbnails/7.jpg)
Semantic Web(Tim Berners-Lee, 2000)
“The intelligent agent that people have touted
for ages will finally materialize.”
![Page 8: “As we may think” (Vannevar Bush, 1945) Source: w3.org/Proposal.html](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1ad97f8b9ab0599743f9/html5/thumbnails/8.jpg)
http://www.w3.org/2000/Talks/1206-xml2k-tbl/slide10-0.html
![Page 9: “As we may think” (Vannevar Bush, 1945) Source: w3.org/Proposal.html](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1ad97f8b9ab0599743f9/html5/thumbnails/9.jpg)
![Page 10: “As we may think” (Vannevar Bush, 1945) Source: w3.org/Proposal.html](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1ad97f8b9ab0599743f9/html5/thumbnails/10.jpg)
Knowledge WebSemantic Web• Human readable vs machine
readable contents
• Human defines standard for data formats and models
• Explicit and precise specification of knowledge representation that everyone has to agree upon
• Machine reads human readable contents
• Machine learns to conflate different formats of the same thing
• Latent and fuzzy representation of knowledge learned by mining big data
![Page 11: “As we may think” (Vannevar Bush, 1945) Source: w3.org/Proposal.html](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1ad97f8b9ab0599743f9/html5/thumbnails/11.jpg)
TRADITIONALWEB SEARCH
Paradigm Shift in Web Search (the “Librarian”)
KNOWLEDGE WEB SEARCH
Index Keywords in Documents
Digest World’s Knowledge
Match Keywords in Queries
Match User Intent
Relevance of “10 blue links”
Dialog Experience
1. “Bing Dialog Model: Knowledge, Intent and Dialog”, MSR Faculty Summit, July 20102. “Introducing the Knowledge Graph: things, not strings”, Official Google Blog, May 20123. “Chinese Search Engine – Baidu’s Practice”, SIRIP, SIGIR 2014, July 2014
![Page 12: “As we may think” (Vannevar Bush, 1945) Source: w3.org/Proposal.html](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1ad97f8b9ab0599743f9/html5/thumbnails/12.jpg)
“Dialog Acts” in Bing/Cortana• Answer• Confirmation• Disambiguation• Suggestion• Progressive: Refinement• Digressive: Recommendation (reactive + proactive)
• Key difference from human-to-human dialog• Not limited to anthropomorphic natural language dialogs• Each dialog turn can present multiple acts• Can overload back channel communications
![Page 13: “As we may think” (Vannevar Bush, 1945) Source: w3.org/Proposal.html](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1ad97f8b9ab0599743f9/html5/thumbnails/13.jpg)
ConfirmationAnswer
Progressive Suggestion
Digressive Suggestion
![Page 14: “As we may think” (Vannevar Bush, 1945) Source: w3.org/Proposal.html](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1ad97f8b9ab0599743f9/html5/thumbnails/14.jpg)
Answer
Confirmation
![Page 15: “As we may think” (Vannevar Bush, 1945) Source: w3.org/Proposal.html](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1ad97f8b9ab0599743f9/html5/thumbnails/15.jpg)
Closed-loop Dynamic Bayesian Inference
+
Behavior Model
+-
Previous Inference
s(K, It - 1)
InferredIntent (It)
InferredIntent (It )
InferredAction (At)
User Behavior (Ut)
Intent Model
Interaction Model
Knowledge+ History
Bayesian Minimum Risk It = arg max P(I | Ut, K, It-
1) At = arg min E[Cost(A, It )]
Expected Behavior (U’t)
Expected Behavior (U’t)
![Page 16: “As we may think” (Vannevar Bush, 1945) Source: w3.org/Proposal.html](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1ad97f8b9ab0599743f9/html5/thumbnails/16.jpg)
Digital Librarian for Researchers: How far are we?A Case Study on Microsoft Academic Search
![Page 17: “As we may think” (Vannevar Bush, 1945) Source: w3.org/Proposal.html](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1ad97f8b9ab0599743f9/html5/thumbnails/17.jpg)
![Page 18: “As we may think” (Vannevar Bush, 1945) Source: w3.org/Proposal.html](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1ad97f8b9ab0599743f9/html5/thumbnails/18.jpg)
Predictive Completion and Disambiguation
![Page 19: “As we may think” (Vannevar Bush, 1945) Source: w3.org/Proposal.html](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1ad97f8b9ab0599743f9/html5/thumbnails/19.jpg)
Progressive Suggestion: Topic recognition, not just
keyword extraction
![Page 20: “As we may think” (Vannevar Bush, 1945) Source: w3.org/Proposal.html](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1ad97f8b9ab0599743f9/html5/thumbnails/20.jpg)
Progressive Suggestion
![Page 21: “As we may think” (Vannevar Bush, 1945) Source: w3.org/Proposal.html](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1ad97f8b9ab0599743f9/html5/thumbnails/21.jpg)
Knowledge Driven Suggestions
![Page 22: “As we may think” (Vannevar Bush, 1945) Source: w3.org/Proposal.html](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1ad97f8b9ab0599743f9/html5/thumbnails/22.jpg)
Research Challenges• How to complete never
foreseen academic queries?
• How to rank completion suggestions?
• How to avoid making completions leading no search results?
![Page 23: “As we may think” (Vannevar Bush, 1945) Source: w3.org/Proposal.html](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1ad97f8b9ab0599743f9/html5/thumbnails/23.jpg)
More on Intent Inference• Generative model approach:
• Dynamic ranking, , score depending on user behavior (e.g., query)
• Static ranking, , score determined by knowledge and dialog history
![Page 24: “As we may think” (Vannevar Bush, 1945) Source: w3.org/Proposal.html](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1ad97f8b9ab0599743f9/html5/thumbnails/24.jpg)
Special Case 1: Static Rank at Onset• Given knowledge graph, find for all entity types• Journal, article, conference, author, institution
• Journal impact factor: E. Garfield, Science, 1972
• Page Rank: A paper is important if cited by important papers• G. Pinski and F. Narin, Information Processing and Management, 1976• N. Geller, Information Processing and Management, 1978• Rediscovery of Perron-Frobenius theorem (1904)
• How to make better use of heterogeneity of the graph?
![Page 25: “As we may think” (Vannevar Bush, 1945) Source: w3.org/Proposal.html](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1ad97f8b9ab0599743f9/html5/thumbnails/25.jpg)
Static Rank of a Paper• Inaugural WSDM Cup, Autumn 2015• Industry organizer: MSR and Elsevier
• http://www.wsdm-conference.org/2016/wsdm-cup.html
![Page 26: “As we may think” (Vannevar Bush, 1945) Source: w3.org/Proposal.html](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1ad97f8b9ab0599743f9/html5/thumbnails/26.jpg)
Institution (20K)
Paper (> 100M) Venue (> 23K) Event (> 46K)
Field of Study (> 50K)
Author (> 40M)
Microsoft Academic Graph
Citations (billions)
![Page 27: “As we may think” (Vannevar Bush, 1945) Source: w3.org/Proposal.html](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1ad97f8b9ab0599743f9/html5/thumbnails/27.jpg)
Microsoft Academic Graph (MAG)• Data Releases on Azure
• Free Azure accesses for research
• http://research.microsoft.com/MAG
• Web Service API coming!
• Community properties
![Page 28: “As we may think” (Vannevar Bush, 1945) Source: w3.org/Proposal.html](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1ad97f8b9ab0599743f9/html5/thumbnails/28.jpg)
Special Case 2: “Zero-query” Suggestion• Digital librarian to notify me new materials I should read
• Find whenever the knowledge graph grows
• Best if • Tailored to user based on interests inferred from aggregated behaviors• Following user wherever, whenever and whatever
• Cortana: intelligent personal assistant• Windows, Android, IOS
![Page 29: “As we may think” (Vannevar Bush, 1945) Source: w3.org/Proposal.html](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1ad97f8b9ab0599743f9/html5/thumbnails/29.jpg)
![Page 30: “As we may think” (Vannevar Bush, 1945) Source: w3.org/Proposal.html](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1ad97f8b9ab0599743f9/html5/thumbnails/30.jpg)
![Page 31: “As we may think” (Vannevar Bush, 1945) Source: w3.org/Proposal.html](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1ad97f8b9ab0599743f9/html5/thumbnails/31.jpg)
Summary• Intelligent agent at web scale (“digital librarian”): • From keyword matching to intent/knowledge understaning• One year old for academic services!
• Conduct interactive dialog or forage on behalf of users behind the scene• Albeit w/o anthropomorphic façade
• Microsoft Academic Services:• Search (reactive)• Cortana notification (proactive)• Data and Intelligent API • We want to build a community