M-CAFETopicTaggingWithWatson

Dataset§ M-CAFEforIEOR115:16

weeksin Aug - Dec,2015• Studentcount:115• Ideacount:106

§ 106ideaswithtagsaresplitrandomlyintotrain(86ideas)andtest(20ideas).

WatsonNaturalLanguageClassifier

Train&Test Sets• Train:86 ideaswith topicstagged.• Test:20ideaswithouttopicstagged.

Screencaptureofthe.csvfilefortrainingset

Code• curl-i -u"896090f0-631f-4745-b02a-47b6417140d6":"xuDyj6lD9USr"-Ftraining_data=@/Users/apple/Desktop/mcafe_watson_train.csv -Ftraining_metadata="{\"language\":\"en\",\"name\":\"McafeClassifier\"}""https://gateway.watsonplatform.net/natural-language-classifier/api/v1/classifiers"

• curl-G-u"896090f0-631f-4745-b02a-47b6417140d6":"xuDyj6lD9USr""https://gateway.watsonplatform.net/natural-language-classifier/api/v1/classifiers/3AE103x13-nlc-1276/classify"--data-urlencode "text=testData"

TestResult:80%Accuracy!Outofthe20testsamples,16werecorrectedclassified.

Idea TopicSlowerpace. Lectures

AddLectureoverview ResourcesIwantmorepracticewithRelationalAlgebraandeventuallySQL. HomeworkThelastfewlectureshavebeenverymathematicallyprecisein

notationwhichcanmakeitabittrickytowrapyourheadaround.Specificquestions/examples(likewhatmightbeonhw)wouldbegreattohelpusmakesureweunderstanditmovingforward.

Lectures

Theprojectseemsalittlestopandgo.Wehaven'tbeenabletoworkonitforaweekorsobutIfeellikewe'llsoonbeexpectedtodoabunchofworkforDP2.Itwouldbehelpfulifwecouldhavethetoolstohaveamoreconstantlevelofworkonthe

project.

Projects

Pleasetryandpostthelabsearliersothatwecangetaheadstartreadingandunderstandingthem. Labs

Homework2onlyhasdatabasequestions,maybeputsomeconnectives? Homework

Incorporateashortquestionandanswerperiodmidwayoflecturetoassessparticipatingstudents'understandingofthe

lecture/topicsbeingpresented.Lectures

Examplesofideaswhicharecorrectlyclassified:

Misclassifications• Thetruetagisamongthetoptwotagssuggestedbytheclassifier.• Misclassificationoccurswhenanideaisarbitrarilytaggedorwithlackofcontext.

Idea TrueTag Pred Tag Confidence

1.slowdownalittlebit Lectures Resources Resources:0.288;Lectures:0.224

2.Itwouldbegreatifyoucouldprovide

outsideresourcesonrulesandguidelinesforthingslikeERdiagramsthatyouthinkareworth

ourtime.

Resources Lectures Lectures:0.879;Resources:0.130

Idea TrueTag Pred Tag Confidence

3.Iwouldlikehavesomeimplantationproblems

usingSQLHomework NewTopics

NewTopics:0.803;

Homework:0.076

4.MorehandsonexperiencesonDatabases Homework NewTopics

NewTopics:0.786;

Homework:0.117

MisclassificationsContd…• Thetruetagisamongthetoptwotagssuggestedbytheclassifier.• Misclassificationoccurswhenanideaisarbitrarilytaggedorwithlackofcontext.

QuestionsforIBM• 1.Howistheclassifiertrained?Whatistheclassificationmethod?• 2.Isthereaversionoftheclassifierthatcanreturnthepredictedtopicforthetestset?• 3.Thisessentiallyasupervisedclassificationproblem,doesWatsonhaveanunsupervisedversionavailable,justproviderawtextanditwouldassigntags?