iwt paris project 16 nov
TRANSCRIPT
PARIS Demonstrators
Yucheng Jin, Joris Klerkx, Erik Duval
Augment groupHCI @ KU Leuven May 2, 2023
Outline• 2nd demonstrator
o Design and implementationo Evaluationo Limitations
• Final demonstratoro Designo Related worko Technical requirementso Implementation plan
2nd demonstrator - offline adaptation
Design
This proof-of-content demonstrator focuses on the retrieval of relevant advertisements and off-line adaptation of the advertisements .
The context of personalized ads is online movie.
The adaptation depends on four aspects:
• The types of movie• Basic user profile on Facebook (Age, gender)• Posts on Facebook wall (personality)• Advertisement preferences
data
Advertiser: value presented by user data
User: value realized through personalization
What should we do toward such a trade-off?
Trade-off
Research hypothesisWe hypothesize that quality and effectiveness of personalized ads can be increased by empowering users to explore and steer the selection process.
To verify our assumption, we investigate the impact of Transparency (T) and User control (UC) on four key aspects:
• Quality: interest match, context match, attractiveness and annoyance?
• Behavioral intention: willingness to click, purchase, and use?• Understanding: understand why and how a particular ad is
selected?• Attitude: satisfaction, confidence and trust of ads?
https://www.facebook.com/business/products/ads
Transparency and user control of advertisement on Facebook
https://www.facebook.com/business/products/ads
Research methodologyIterative design and rapid prototyping
Design
ImplementationEvaluate
Prototype design. Refine the features of prototype
Implement the prototypeEvaluate the prototype with users in diverse settings.
Workflow of demonstrator
http://paris-ad.evennode.com/paris/
Transparency
User control
ImplementationAn web app of Facebook
RESTful API for accessing to user data and advertisement data
GET http://paris-ad.evennode.com/paris/api/ads?ageLevel=2
GET http://paris-ad.evennode.com/paris/api/user?id=564133123727385PUT http://paris-ad.evennode.com/paris/api/user?id=564133123727385data:{ gender : ”male”} All data in JSON format
Schema of user data Schema of ad data
Filtering by
EvaluationWe conducted a between-subjects study on Amazon Mechanical Turk (MTurk) where we recruited 200 subjects who have above 80% lifetime approval rate for HITs. Compensation was $1 for each study and average study completion time was around 11 minutes.
We created four experimental conditions:• Condition 1 (C1): (No-T & No-UC) base condition.• Condition 2 (C2): (T & No-UC).• Condition 3 (C3): (No-T & UC).• Condition 4 (C4): (T & UC)
We used the user-centric evaluation framework of recommender system and tailored the questionnaire to evaluate four aspects of targeted advertisement: Quality, Behavioral intention, Understanding and Attitude.
As a result, we created four post-study questionnaires QueA, QueB, QueC and QueD to assess the effect of T and UC in different conditions.
~80% subjects noticed online targeted ads.~10% subjects configured targeted ads.
Pu, Pearl, Li Chen, and Rong Hu. "A user-centric evaluation framework for recommender systems." Proceedings of the fifth ACM conference on Recommender systems. ACM, 2011.
Common statements shown in four questionnaires.
Specific statements and optional questions regarding users’ perception of T (QueB), UC (QueC) and T & UC (QueD).
5-point Likert scale, Strongly agree - Strongly disagree
Evaluation steps
1. Introduce web app to subjects
2. Log in to the app with their Facebook accounts.
3. Play movie trailer and show ads.
4. During the trailer, subjects could rate the ads and configure ads.
5. After watching the trailer, subjects were asked to complete the
questionnaire.
What is the result ?
Common statements
Kruskal-Wallis Dunn post hocSTM1(Interest match)
(H=14.49, df=3, p=.002) C1 (Median: 3) and C4 (Median: 4), (p=.001)
STM2(Willing. to click)
(H=11.42, df=3, p=.010) C1 (Median: 3) and C4 (Median: 4) (p=.014).
STM3 (Willing. to see)
(H=11.74, df=3, p=.008) C1 (Median: 3) and C4 (Median: 4), (p=.018)
C1 (Median: 3) and C2 (Median: 4), (p=.03)
STM2 (Understanding)
(H=13.68, df=3, p=.003) C1 (Median: 3) and C4 (Median: 4), (p=.009)
C1 (Median: 3) and C3 (Median: 4), (p=.010).
Statistical analysis result
Specific statements
Configuration Quality Behavioral intention
Understanding Attitude
No-T & No-UC
T & No-UC ★
No-T & UC ★
T & UC ★ ★ ★
Submitted as a full paper to IUI 2016 (Rank A)
Deliverables 9.3 and 9.4
Limitations
• 70 elements of seven ad categories. Not a real data set of ads.
• The algorithm for selecting appropriate ads is not validated.• More advanced adaptive features based on vision
technology are not implemented.• Not building directly on PARIS technology (yet).
Final demonstrator –online adaptation
DesignThis demonstrator will online analyze, adapt, and integrate content and advertisements.
The same objects and attributes as used in the first two demonstrators will be targeted, but now the linguistic or/and visual processing should be very efficient timewise to ensure effective interaction.
A database of templates is queried and the advertisement template is created in real-time adapted to the preferences of its user.
ScenarioFurniture advertisements
https://thetake.com/
Related work
Technical requirements
12
3
4
Object recognitionAd retrieval
Advertisement adaptation
$349Buy
Link to webshops
Second screen (mobile) apps with TV
1. Object recognition
We need to recognize objects that appeared in a frame when the user pauses the video.
• (The time when a particular object appears in a movie)• The position where the recognized object appear in a key frame. (Recognized objects should be labelled such as a rectangles)• The description (query terms to webshops) of recognized object
(what is it? (chair, table), color, brand, etc.)
Discussed with VISICS
2. Advertisement retrieval
• A valid data set of ads, each ad contains meaningful annotation such as brand, color, categoryo VLERICK, LIIR
• A defined user model for advertising (age, gender, personality …)o CWI -> API
• An model for selecting advertisement for a targeted usero CWI, LIIR
• A set of valid adaptive rules for showing personalized adso VLERICK
3. Advertisement adaptation
• Modify a particular object according to the user profile. o the object coloro the object orientationo the object positiono the background of object-> VISICS
• Maybe show these adaptations by using AR
4. Link to webshops
• Issue query terms to webshops to find related furniture to the identified object in the video (e.g., “EKTORP, Chair, Idemo red”)
https://developer.sears.comhttp://docs.72lux.com/product-api-v1.htmlAmazon Product Advertising APIEtc.
• Exact matching difficult• Not too many products of furniture in these shopping APIs
Implementation planIterative design and rapid prototyping
• Time:~ January 2016 (1st version)~ February 2016 (Evaluation for the 1st version)~ March 2016 (2nd version with integrating other PARIS technology)
• Performance:Online adaptation, (almost in) real time.
Might build on external APIs for some modules to speed up development until PARIS technology is ready
Technical support PARIS partner Proposed date
Obj. recognition VISICS D4.4 Software for robust recognition of object classes in video (M30)
Data set of ads VLERICK, LIIR
User modelling for ads CWI D6.1 Software for inferring demographic profile (M18)D6.3 Software for learning product preferences from user generated content (M24)
Model for selecting ads CWI, LIIR D7.1 Software for ad selection model (M45)
Adaptive rules VLERICK D8.1 Report on the design of personalized advertisements (M24)
Obj. replacement VISICS D8.2 Software for object replacement in images and video (M42)
