migrating from drupal to plone with transmogrifier
Post on 29-Nov-2014
6.019 Views
Preview:
DESCRIPTION
TRANSCRIPT
Clayton Parker, Senior Developer
Migrating From Drupal to Plone with Transmogrifier
PLONE SYMPOSIUM EAST 2011
PLONE SYMPOSIUM EAST 2011Who Am I?
• claytron
• Python dev since 2003
• Plone Core Committer
• Foundation Member
PLONE SYMPOSIUM EAST 2011What will we learn?
• Transmogrifier basics
• Migration Planning Process
• Creating a pipeline
PLONE SYMPOSIUM EAST 2011Migrations
• One off scripts
• In multiple places
• Little to no re-usability
PLONE SYMPOSIUM EAST 2011Transmogrifier
• A framework for migrations
• Re-usable parts
PLONE SYMPOSIUM EAST 2011Basics
• Pipeline
• Blueprints
• Sources
• Section
PLONE SYMPOSIUM EAST 2011Sources
• A blueprint
• First item in your pipeline
PLONE SYMPOSIUM EAST 2011
<html><body><h3>Code Sample</h3><p>Replace this text!</p></body></html>
Example Pipeline[transmogrifier]pipeline = csv_file constructor schemaupdater
[csv_file]blueprint = collective.transmogrifier.sections.csvsourcefilename = my.migration.import:my_items.csv
[constructor]blueprint = collective.transmogrifier.sections.constructor
[schemaupdater]blueprint = plone.app.transmogrifier.atschemaupdater
PLONE SYMPOSIUM EAST 2011my_items.csv
_path , _type , title , description/folder1 , Folder , First Folder , This is folder One/folder2 , Folder , Second Folder , This is folder Two/folder1/foo , Document , One Foo , A document named foo/folder2/foo , Document , Two Foo , Another doc named foo
PLONE SYMPOSIUM EAST 2011The Result
PLONE SYMPOSIUM EAST 2011Items
• Each item is a mapping
• Keys are fields
• Keys with a leading underscore are controllers
PLONE SYMPOSIUM EAST 2011Example Item{'_id': 'a-stronger-connection-to-out-customers', '_path': 'content/stronger-connection-out-customers', '_status': 1L, '_text_mimetype': 'text/html', '_transitions': 'publish', '_type': 'Document', 'allowDiscussion': False, 'creation_date': '2011/05/14 9:20:50', 'effectiveDate': '2011/05/14 9:20:50', 'modification_date': '2011/05/18 9:22:22', 'subject': 'better\ninteresting\nstronger', 'text': '<p>this is some text</p>\r\n', 'title': 'A stronger connection to out customers'}
PLONE SYMPOSIUM EAST 2011Migration Strategy
• Investigate the source
• Prepare the destination
• Find Transmogrifier blueprints
• Write the pipeline
PLONE SYMPOSIUM EAST 2011Write your own
• Missing blueprint
• Write one
• Contribute it back
PLONE SYMPOSIUM EAST 2011GenericSetup
• Make migration part of your release
• Ability to package migrations
PLONE SYMPOSIUM EAST 2011My Migration
• Drupal backed by MySQL
• transmogrify.sqlalchemy
• Plone 4.0.5
• collective.blog.star
• plone.app.discussion
PLONE SYMPOSIUM EAST 2011Package Layoutmy.migration!"" my# !"" __init__.py# %"" migration# !"" __init__.py# !"" config# # !"" articles.cfg# # !"" base.cfg# # !"" blogs.cfg# # !"" comments.cfg# # %"" pages.cfg# !"" configure.zcml# %"" profiles# %"" default# !"" metadata.xml# %"" transmogrifier.txt!"" setup.cfg%"" setup.py
PLONE SYMPOSIUM EAST 2011Registering Configs
<transmogrifier:registerConfig name="my.migration.base" title="My migration base config" description="Base settings for all transmogrifier imports" configuration="config/base.cfg" />
PLONE SYMPOSIUM EAST 2011Registering Configs
<transmogrifier:registerConfig name="my.migration.pages" title="Drupal pages" description="Import pages from Drupal into Plone" configuration="config/pages.cfg" />
PLONE SYMPOSIUM EAST 2011Registering Configs
<transmogrifier:registerConfig name="my.migration.articles" title="Drupal articles" description="Import articles from Drupal into Plone" configuration="config/articles.cfg" />
PLONE SYMPOSIUM EAST 2011Registering Configs
<transmogrifier:registerConfig name="my.migration.blogs" title="Drupal blog entries" description="Import blog entries from Drupal into Plone" configuration="config/blogs.cfg" />
PLONE SYMPOSIUM EAST 2011Registering Configs
<transmogrifier:registerConfig name="my.migration.comments" title="Drupal comments" description="Import comments from Drupal into Plone" configuration="config/comments.cfg" />
PLONE SYMPOSIUM EAST 2011transmogrifier.txt
my.migration.pagesmy.migration.articlesmy.migration.blogsmy.migration.comments
PLONE SYMPOSIUM EAST 2011Package Layoutmy.migration!"" my# !"" __init__.py# %"" migration# !"" __init__.py# !"" config# # !"" articles.cfg# # !"" base.cfg# # !"" blogs.cfg# # !"" comments.cfg# # %"" pages.cfg# !"" configure.zcml# %"" profiles# %"" default# !"" metadata.xml# %"" transmogrifier.txt!"" setup.cfg%"" setup.py
PLONE SYMPOSIUM EAST 2011base.cfg pipeline[transmogrifier]pipeline = drupal portal_type url_normalizer path publication_state text_mimetype mimetype_encapsulator folders constructor commenting comments schema_update workflow reindex_object
[settings]# Path to use if there isn’t one givenbase_path = other-content# Have to escape python string formatting for when # this gets passed into sqlalchemydate_format = %%Y/%%m/%%d %%k:%%i:%%s
PLONE SYMPOSIUM EAST 2011drupal section
[drupal]blueprint = transmogrify.sqlalchemydsn = mysql://user:password@localhost/drupal-transmog
PLONE SYMPOSIUM EAST 2011articles.cfg[transmogrifier]include = my.migration.base
[drupal]query = SELECT node.title, node.status AS status, GROUP_CONCAT(tag_data.name SEPARATOR '\n') AS subject, FROM_UNIXTIME(node.created, "${settings:date_format}") AS creation_date, FROM_UNIXTIME(node.created, "${settings:date_format}") AS effectiveDate, FROM_UNIXTIME(node.changed, "${settings:date_format}") AS modification_date, body_data.body_value AS text url_alias.alias AS _path FROM node INNER JOIN field_data_field_tags AS tag_field ON tag_field.entity_id = node.nid INNER JOIN taxonomy_term_data AS tag_data ON tag_data.tid = tag_field.field_tags_tid INNER JOIN field_data_body AS body_data ON body_data.entity_id = node.nid INNER JOIN url_alias ON url_alias.source = CONCAT("node/", node.nid) GROUP BY node.title, node.status, node.created, node.changed, body_data.body_value, url_alias.alias;
[portal_type]value = string:Document
PLONE SYMPOSIUM EAST 2011pages.cfg[transmogrifier]include = my.migration.base
[drupal]query = my.migration.base SELECT node.title, node.status AS _status, FROM_UNIXTIME(node.created, "${settings:date_format}") AS creation_date, FROM_UNIXTIME(node.created, "${settings:date_format}") AS effectiveDate, FROM_UNIXTIME(node.changed, "${settings:date_format}") AS modification_date, body_data.body_value AS text, url_alias.alias AS _path FROM node INNER JOIN field_data_body AS body_data ON body_data.entity_id = node.nid INNER JOIN url_alias ON url_alias.source = CONCAT("node/", node.nid) WHERE node.type = "page" GROUP BY node.title, node.status, node.created, node.changed, body_data.body_value, url_alias.alias;
[portal_type]value = string:Document
PLONE SYMPOSIUM EAST 2011blogs.cfg[transmogrifier]include = my.migration.base
[drupal]query = SELECT node.title, node.satus AS _status, FROM_UNIXTIME(node.created, "${settings:date_format}") AS creation_date, FROM_UNIXTIME(node.created, "${settings:date_format}") AS effectiveDate, FROM_UNIXTIME(node.created, "${settings:date_format}") AS modification_date, body_data.body_value AS text, url_alias.alias AS _path FROM node INNER JOIN field_data_body AS body_data ON body_data.entity_id = node.nid INNER JOIN url_alias ON url_alias.source = CONCAT("node/", node.nid) WHERE node.type = "blog" GROUP BY node.title, node.status, node.created, node.changed, body_data.body_value, url_alias.alias;
[portal_type]value = string:BlogEntry
[commenting]value = python:True
PLONE SYMPOSIUM EAST 2011comments.cfg[transmogrifier]include = my.migration.base
[drupal]query = my.migration.base SELECT comment.subject AS title, FROM_UNIXTIME(comment.created, "${settings:date_format}") AS published, FROM_UNIXTIME(comment.changed, "${settings:date_format}") AS updated, comment.name AS author_name, body_data.comment_body_value AS text, url_alias.alias AS _parent_path FROM comment INNER JOIN field_data_comment_body AS body_data ON body_data.entity_id = comment.cid INNER JOIN node ON node.nid = comment.nid INNER JOIN url_alias ON url_alias.source = CONCAT("node/", node.nid) GROUP BY comment.subject, comment.created, comment.changed, comment.name, body_data.comment_body_value, url_alias.alias;
[portal_type]# Override the portal type to use the "comment_type"key = string:_comment_typevalue = string:plone.app.discussion
PLONE SYMPOSIUM EAST 2011base.cfg overrides[path]blueprint = collective.transmogrifier.sections.inserter# only add a path if one does not existcondition = python:'_path' not in item and not '_parent_path' in itemkey = string:_path# Add the value in the extended configurationvalue = string:${settings:base_path}/${item/_id}
[portal_type]blueprint = collective.transmogrifier.sections.inserterkey = string:_type# We will add the value in the extended config, but we need a# default set herevalue = string:
[commenting]blueprint = collective.transmogrifier.sections.inserterkey = string:allowDiscussion# default to falsevalue = python:False
PLONE SYMPOSIUM EAST 2011Content Creation
[comments]blueprint = transmogrify.comments
[folders]blueprint = collective.transmogrifier.sections.folders
[constructor]blueprint = collective.transmogrifier.sections.constructor
[schema_update]blueprint = plone.app.transmogrifier.atschemaupdater
[workflow]blueprint = plone.app.transmogrifier.workflowupdater
[reindex_object]blueprint = plone.app.transmogrifier.reindexobject
PLONE SYMPOSIUM EAST 2011Item Modification[publication_state]blueprint = collective.transmogrifier.sections.insertercondition = python:'_status' in item and item['_status'] == 1key = string:_transitionsvalue = string:publish
[text_mimetype]# This could probably be taken from the database as wellblueprint = collective.transmogrifier.sections.inserterkey = string:_text_mimetypevalue = string:text/html
[mimetype_encapsulator]blueprint = plone.app.transmogrifier.mimeencapsulatorkey = textmimetype = python:item.get('_%s_mimetype', key)field = keycondition = mimetype
PLONE SYMPOSIUM EAST 2011
DEMO
PLONE SYMPOSIUM EAST 2011Links
• collective.transmogrifier
http://pypi.python.org/pypi/collective.transmogrifier/
• plone.app.transmogrifier
http://pypi.python.org/pypi/plone.app.transmogrifier/
PLONE SYMPOSIUM EAST 2011Useful Sources and Blueprints• plone.app.transmogrifier
• transmogrify.filesystem
• transmogrify.sqlalchemy
• transmogrify.webcrawler
• quintagroup.transmogrifier
• transmogrify.comments
Check out
sixfeetup.com/demos
top related