importing and synchronizing content using feeds

56
Importing and syncing content using Feeds Peter Vanhee 2nd October 2011 - DrupalCamp Spain drupal 7

Upload: peter-vanhee

Post on 15-Jan-2015

12.331 views

Category:

Technology


2 download

DESCRIPTION

Presentation from DrupalCamp Spain 2011 about Feeds in Drupal 7.

TRANSCRIPT

Page 1: Importing and synchronizing content using Feeds

Importing and syncing content using Feeds

Peter Vanhee2nd October 2011 - DrupalCamp Spain

drupal7

Page 2: Importing and synchronizing content using Feeds

Who am I?

‣ Consumer and contributor to Drupal for over 5 years

‣ Co-founder Youth Agora @youthagora and Nuvole @nuvoleweb

‣ Web consultant in Barcelona @pvhee

Page 4: Importing and synchronizing content using Feeds
Page 5: Importing and synchronizing content using Feeds

Content is aggregated and synchronized using Feeds

Page 6: Importing and synchronizing content using Feeds

Use Cases

‣ Build Activity streams using feeds from Twitter, Facebook, ...

‣ Batch import sets of users from file

‣ Import Commerce Products

‣ ...

Page 7: Importing and synchronizing content using Feeds

What is Feeds? 1

Page 8: Importing and synchronizing content using Feeds

Feeds is...

‣ Import / aggregate framework

‣ Create and update drupal entities out of the box: nodes, users, taxonomy

‣ Real-time feeds support

Page 9: Importing and synchronizing content using Feeds

Related

‣ Migrate‣ Table Wizard‣ User import‣ Node import

‣ Aggregator

Page 10: Importing and synchronizing content using Feeds

Feeds as the standard for piping content

Page 11: Importing and synchronizing content using Feeds

Feeds usage

Statistics from drupal.org

drupal 7

drupal 6

statistics from drupal.org

Page 12: Importing and synchronizing content using Feeds

Fetcher

Parser

Processor

Page 13: Importing and synchronizing content using Feeds

Fetcher

‣ Http

‣ File upload

A complete list is at http://drupal.org/node/856644

Page 14: Importing and synchronizing content using Feeds

‣ CSV, RSS1, RSS2, Atom

‣ OPML, Sitemap XML

‣ XPath for XML/HTML

‣ JSONPath for JSON

‣ YouTube, Flickr, Slideshare

‣ iCal

Parser

A complete list is at http://drupal.org/node/856644

Page 15: Importing and synchronizing content using Feeds

‣ Node

‣ User

‣ Taxonomy

‣ Commerce Products

Processor

A complete list is at http://drupal.org/node/856644

Page 16: Importing and synchronizing content using Feeds

Example #12011.drupalcamp.es my drupal site

Import and sync RSS Feeds

Page 17: Importing and synchronizing content using Feeds

Feeds importers are configurations of

Fetcher Parser Processor

Page 18: Importing and synchronizing content using Feeds
Page 19: Importing and synchronizing content using Feeds
Page 20: Importing and synchronizing content using Feeds
Page 21: Importing and synchronizing content using Feeds

create nodes of type Feed item

Page 22: Importing and synchronizing content using Feeds

fields in Feed itemfields in RSS

Page 23: Importing and synchronizing content using Feeds

And now... importing!

Page 24: Importing and synchronizing content using Feeds

/import

Page 25: Importing and synchronizing content using Feeds
Page 26: Importing and synchronizing content using Feeds

We can do this with Drupal Core Aggregator too!

Page 27: Importing and synchronizing content using Feeds

Let’s try spawning

User 1

Feed A

Feed B

Feed C

User 2 Feed X

User

User 1

Feed A

Feed B

Feed C

FeedsAggregator (core)

Page 28: Importing and synchronizing content using Feeds

Spawning

Page 29: Importing and synchronizing content using Feeds

Add Feed = Add Node

Page 30: Importing and synchronizing content using Feeds

Example #2Import images

Page 31: Importing and synchronizing content using Feeds

A node of type Story will hold a Flickr image

Page 33: Importing and synchronizing content using Feeds

Parse XML with XPath expressions

//item

description

media:thumbnail/@url

geo:latgeo:long

titlelink

Page 34: Importing and synchronizing content using Feeds

Fetcher

Parser

Processor

http

XPath Parser

Map targets from XPathNode processor

Page 35: Importing and synchronizing content using Feeds

select XPath XML parser

Page 36: Importing and synchronizing content using Feeds

source is XPath Result fields in Story

Page 37: Importing and synchronizing content using Feeds
Page 38: Importing and synchronizing content using Feeds
Page 39: Importing and synchronizing content using Feeds

Importing

Page 40: Importing and synchronizing content using Feeds
Page 41: Importing and synchronizing content using Feeds

!

We have problems

Tags are not imported correctly

Page 42: Importing and synchronizing content using Feeds

Feeds Tamper

http://drupal.org/project/feeds_tamper

Page 43: Importing and synchronizing content using Feeds

Enter Tampering

Input Field Output Field

drupal feeds sevilla drupalfeedssevilla

Me</a>posted a photo:</p>

<a href=”.”>Me</a>posted a photo

explode

html_encode

Page 44: Importing and synchronizing content using Feeds

Configure Tampering

Page 45: Importing and synchronizing content using Feeds

More tamper actions

Page 46: Importing and synchronizing content using Feeds

Add your own tamper plugins

snippet from html_entity_decode.inc

Page 47: Importing and synchronizing content using Feeds

After importing... tags are now correctly split

Page 48: Importing and synchronizing content using Feeds

Importing Big Stuff

‣ Feeds has Batch support

‣ Feeds can run on cron jobs in the background

Page 49: Importing and synchronizing content using Feeds

Field support‣ Text‣ File‣ Image‣ Link‣ Geofield‣ Taxonomy‣ Password‣ Date‣ Integer

‣ Email‣ Multimedia‣ Address

Tip: check Feeds issue queues for patches

Page 50: Importing and synchronizing content using Feeds

Feeds as a Framework

‣ Supports import/sync of Commerce Products (via Commerce Feeds)

‣ Straightforward to add custom fetchers, patchers and processors using Feed hooks (check feeds.api.php)

Page 51: Importing and synchronizing content using Feeds

Finally... when not to use Feeds

‣ Content does not need to be stored by Drupal

‣ Complex content migration tasks

Page 52: Importing and synchronizing content using Feeds

Are we there yet?

Now... Real-Time?

Publisher Subscriber

Are we there yet?

Are we there yet?

Are we there yet?

Yes. Here you are.

Slide taken from @alexb presentation on Feeds

Page 53: Importing and synchronizing content using Feeds

Enter PubSubHubBub

Publisher SubscriberHub

What does this solve?1. Notifications to Subscribers managed by Hub2. Send only what is changed and when it gets changed

Page 54: Importing and synchronizing content using Feeds

Euhm.. so what does Feeds do exactly?

Feeds makes Drupal a PSHB Subscriber

Update feeds in real-time from PSHB Hubs e.g. Blogger, Wordpress, Superfeedr, ...

Page 55: Importing and synchronizing content using Feeds

Thanks!

Peter Vanhee@pvhee

Page 56: Importing and synchronizing content using Feeds

Questions?

Peter Vanhee@pvhee