geostyle: discovering fashion trends and eventsrallies, festivals and sporting events automatically...

GeoStyle: Discovering Fashion Trends and Events Utkarsh Mall 1 Kevin Matzen 2 Bharath Hariharan 1 Noah Snavely 1 Kavita Bala 1 Pipeline Results Future Work Contributions ● Extending the method for other domains and datasets. ● Analysis at a ﬁne-grained level. ● Looking at methods to alleviate bias due to dataset. Detecting and predicting fashion trends over space and time ● A fully-automated framework for fashion trend discovery. ● Accurately model and forecast long-term trends, like seasonal cycles. ● Discover and group short-term spikes caused by events, like rallies, festivals and sporting events ● Automatically discover the reasons for these events. Stage II - Characterizing Trends Stage III - Discovering Events Stage IV - Mining Reasons for Events 44 cities 3 years (June 2013-May 2016) 7.7 million images Instagram+Flickr100m Bangkok Seattle Bangkok Chicago Rio de Janeiro Yellow Color No Sleeves T-shirt Red Color Yellow Color 2014 Dec 2015 Dec 2014 Oct 2014 Apr 2014 Jun 2014 Jun 2014 Jul dad father halloween freaknight songkran festival cup stanleycup worldcup brasil Discovers events even in held-out cities from a different domain (Flickr). ● Classify attributes from StreetStyle [1]. ● Aggregate probability values at each time & city. ● Use TF-IDF on captions to identify words most representative of an event. ● Find outliers using hypothesis testing framework. ● Group outliers based on proximity and periodicity. 4th, july, pride, america, 4thofjuly halloween, freaknight, freaknight2014 happyhalloween, halloween2014 1 Cornell University 2 Facebook Stage I - Recognition Acknowledgm e nt This work was funded by NSF (CHS: 1617861 and CHS:1513967) and an Amazon Research Award. Accurate, ﬁne-grained prediction of trends D iscovered new events unknown to the authors newyear, winter, christmas, holidays, year Attribute Recognition & Style Discovery Take-away ● Modelling trends using prior understanding results in more accurate and interpretable forecasting models. ● 20% improvement in prediction error rate v/s VAR. ● A total of 725 events discovered of which 456 are interpretable as seasonal sporting events, festivals or political rallies. ● Text combined with visual features helps in explaining events. ● 100% of the top 35 events are explainable by captions. “Catalan way” 2013 Sep in Barcelona Lower is better Popular events around the world using representative styles ● Interpretable, expressive parametric model capturing linear and cyclical trends. (I) (I I) (III) (IV) t-SNE of dataset images References [1] K. Matzen, K. Bala, and N. Snavely. Streetstyle: Exploring world-wide clothing styles from millions of photos. CoRR, 2017 Parameter Interpretable meaning r Trade-off between linear and cyclic trend m lin Rate of long-term change in popularity m cyc Amplitude and sign of cyclical spikes k Spikiness of cyclical spikes Styles are discovered by clustering in embedding space [1].

Upload: others

Post on 30-May-2020

2 views

Category:

Documents

0 download

Report

Download

Embed Size (px):

TRANSCRIPT

Page 1: GeoStyle: Discovering Fashion Trends and Eventsrallies, festivals and sporting events Automatically discover the reasons for these events. Stage II - Characterizing Trends Stage III

GeoStyle: Discovering Fashion Trends and EventsUtkarsh Mall1 Kevin Matzen2 Bharath Hariharan1 Noah Snavely1 Kavita Bala1

Pipeline

Results

Future Work

Contributions

● Extending the method for other domains and datasets.● Analysis at a fine-grained level.● Looking at methods to alleviate bias due to dataset.

Detecting and predicting fashion trends over space and time● A fully-automated framework for fashion trend discovery.● Accurately model and forecast long-term trends, like seasonal

cycles.● Discover and group short-term spikes caused by events, like

rallies, festivals and sporting events● Automatically discover the reasons for these events.

Stage II - Characterizing Trends

Stage III - Discovering Events

Stage IV - Mining Reasons for Events

44 cities3 years (June 2013-May 2016)7.7 million imagesInstagram+Flickr100m

Bangkok Seattle Bangkok Chicago Rio de Janeiro

Yellow Color No Sleeves T-shirt Red Color Yellow Color

2014 Dec 2015 Dec

2014 Oct 2014 Apr 2014 Jun 2014 Jun2014 Jul

dadfather

halloweenfreaknight

songkranfestival

cupstanleycup

worldcupbrasil

Discovers events even in held-out cities from a different domain (Flickr).

● Classify attributes from StreetStyle [1].● Aggregate probability values at each time & city.

● Use TF-IDF on captions to identify words most representative of an event.

● Find outliers using hypothesis testing framework.● Group outliers based on proximity and periodicity.

4th, july, pride, america, 4thofjuly

halloween, freaknight, freaknight2014 happyhalloween, halloween2014

1Cornell University 2Facebook

Stage I - Recognition

AcknowledgmentThis work was funded by NSF (CHS: 1617861 and CHS:1513967) and an Amazon Research Award.

Accurate, fine-grained prediction of trends Discovered new events unknown to the authors

newyear, winter, christmas, holidays, year

Attribute Recognition&

Style Discovery

Take-away● Modelling trends using prior understanding results in more

accurate and interpretable forecasting models.● 20% improvement in prediction error rate v/s VAR.● A total of 725 events discovered of which 456 are interpretable

as seasonal sporting events, festivals or political rallies. ● Text combined with visual features helps in explaining events.

● 100% of the top 35 events are explainable by captions.

“Catalan way”2013 Sep in Barcelona

Lower is better

Popular events around the world using representative styles

● Interpretable, expressive parametric model capturing linear and cyclical trends.

(I)

(II)

(III)

(IV)

t-SNE of dataset images

References[1] K. Matzen, K. Bala, and N. Snavely. Streetstyle: Exploring world-wide clothing styles from millions of photos. CoRR, 2017