filtering twitter noise
DESCRIPTION
The problem, and some solutions using TalkingPuffin and Scala. Related blog post and screencast: http://briccetti.blogspot.com/2009/11/talkingpuffin-075-released-with-better.htmlTRANSCRIPT
![Page 1: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/1.jpg)
Filtering Twitter Noise
The problem, and some solutions using TalkingPuffin and Scala.
Dave Briccetti@dcbriccetti
![Page 2: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/2.jpg)
What is Twitter noise?
![Page 3: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/3.jpg)
![Page 4: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/4.jpg)
How can I mute everything from Foursquare?
![Page 5: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/5.jpg)
![Page 6: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/6.jpg)
Nothing more from Foursquare.
If we wanted, we could just exclude Foursquare tweets from individual users.
![Page 7: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/7.jpg)
What about noisy retweeters?
![Page 8: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/8.jpg)
![Page 9: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/9.jpg)
![Page 10: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/10.jpg)
No more retweets from this user. This leaves her original, primary tweets, which we value.
![Page 11: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/11.jpg)
How does TalkingPuffin store these filters?
Let’s look at the filters dialog.
![Page 12: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/12.jpg)
![Page 13: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/13.jpg)
These are “compound filters.”
![Page 14: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/14.jpg)
![Page 15: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/15.jpg)
Contains a List of these:
![Page 16: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/16.jpg)
![Page 17: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/17.jpg)
Tags
![Page 18: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/18.jpg)
Tags may be assigned to people, and tweets may be included or excluded by tags.
Tags can be exported to Twitter lists.
![Page 19: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/19.jpg)
Here we just want to look at Twitter employees.
![Page 20: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/20.jpg)
![Page 21: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/21.jpg)
![Page 22: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/22.jpg)
Tags are managed by a TagUsers object.
![Page 23: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/23.jpg)
![Page 24: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/24.jpg)
CompoundFilters and tags are stored in InOutFilters.
![Page 25: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/25.jpg)
![Page 26: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/26.jpg)
InOutFilters are stored in a FilterSet, along with some general, boolean options.
![Page 27: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/27.jpg)
![Page 28: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/28.jpg)
Detecting retweets with regular expressions
![Page 29: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/29.jpg)
![Page 30: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/30.jpg)
A shared repository of regular expressions or other filters that can be used in a community to
identify and filter out noise.
External Noise Filters
![Page 31: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/31.jpg)
General Filters Pane
![Page 32: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/32.jpg)
Delivered Via a Web Service on Google App Engine
![Page 33: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/33.jpg)
Loaded into a List of regular expressions
![Page 34: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/34.jpg)
And used to find noise tweets
![Page 35: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/35.jpg)
Summary
● There is a lot of noise on Twitter● With the right tools you can control it● TalkingPuffin is one such tool● Scala is a great language for creating such
tools
![Page 36: Filtering Twitter Noise](https://reader034.vdocuments.net/reader034/viewer/2022052522/554a6344b4c905522f8b5748/html5/thumbnails/36.jpg)
More
● Learn more about ● TalkingPuffin at TalkingPuffin.org● Scala at scala-lang.org
● Consider joining the TalkingPuffin open source development effort
● Contact● Dave Briccetti, @dcbriccetti