everything you wanted to know about trove but didn't know whom to ask!
Post on 01-Jul-2015
352 Views
Preview:
DESCRIPTION
TRANSCRIPT
Everything you wanted to know about Trove but didn't know
whom to ask!Nikhil Manchanda [HP]
PTL, TroveEmail: nikhil.manchanda@hp.com
IRC: SlickNikTwitter: @SlickNik@hphelioncloud
Amrith Kumar [Tesora]Core Reviewer, TroveEmail: a@tesora.com
IRC: amrithTwitter: @amrithkumar
@tesoracorp
Tweet about this presentation #openstack #trove
What is Trove• Trove Mission
“To provide scalable and reliable Cloud Database as a Service provisioningfunctionality for both relational and non-relational database engines, and to continue to improve its fully-featured and extensible open source framework.”
https://wiki.openstack.org/wiki/Trove
• Provisioning and de-provisioning.• Administration.• Configuration.• Backup and Restore.• Clustering & Replication.
• Create and manipulate tables, collections, namespaces.• Select, Insert, Update, Delete Data.
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 20142
An Overview of Trove
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 20143
Trove Architecture [1]
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 20144
Trove Architecture [2]
Keysto
ne
Cinder VolumeCinder VolumeCinder Volume
No
va-Netw
orkin
g/N
eutro
nTrove API
Message Bus
Trove Conductor
Trove Task Manager
Compute Instance
Guest Agent SQL/NoSQL
Backup
Data
BackupBackups
DB
BackupBackup
Guest Images
DataData
Nova
Cinder
Swift
Glance
Trove
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 20145
Trove Architecture [3]
Trove Guest Instance
Operating System
Database Management System
Trove Guest Agent
Trove Controller
Trove API
Message Bus
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 20146
Trove Taskmanager
Trove Conductor
Where Can I Get Trove?• Use the source, Luke!
• OpenStack Trove Project• https://git.openstack.org/cgit/openstack/trove• Mirrored at - http://github.com/openstack/trove.git
• Trove python client binding and command line client• https://git.openstack.org/cgit/openstack/python-troveclient• Mirrored at – http://github.com/openstack/python-troveclient.git
• Trove design specifications for blueprints• https://git.openstack.org/cgit/openstack/trove-specs• Mirrored at – http://github.com/openstack/trove-specs
• Trove scripts for installation and testing, and elements for building guest images• https://git.openstack.org/cgit/openstack/trove-integration• Mirrored at – http://github.com/openstack/trove-integration.git
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 20147
trove/guestagent/strategies
backup/
[…]
replication/
restore/
storage/
trove/guestagent/
[…]
backup/
common/
datastore/
[…]
strategies/
[…]
trove/guestagent/strategies/backup
base.py
base.pyc
couchbase_impl.py
couchbase_impl.pyc
__init__.py
__init__.pyc
mysql_impl.py
mysql_impl.pyc
postgresql_impl.py
grep class guestagent/strategies/backup/mysql_impl.py
class MySQLDump(base.BackupRunner):
class InnoBackupEx(base.BackupRunner):
class InnoBackupExIncremental(InnoBackupEx):
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 20148
Trove Modularity
Getting Started with Trove
• As a Trove user• OpenStack distribution that includes Trove (such as HP Helion Dev Platform)
• http://www8.hp.com/us/en/cloud/hphelion-openstack.html• Tesora DBaaS platform, a Trove packaging tailored for the enterprise
• http://tesora.com/download-tesora-dbaas-platform-enterprise-edition
• As a Trove developer$ git clone http://github.com/openstack/trove-integration$ cd trove-integration/scripts$ ./redstack install$ ./redstack kick-start mysql
• On top of DevStack• Add to localrc:
• ENABLED_SERVICES+=,trove,tr-api,tr-tmgr,tr-cond• Swift should also be enabled for Backup and Restore.
• ENABLED_SERVICES+=,s-proxy,s-object,s-container,s-account• SWIFT_HASH=<swift-hash-here>
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 20149
Provisioning a Trove Instance
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 201410
$ trove create
<instance-name> <flavor-id>
--size <volume-size>
• Support for flavors
• Support for volumes using Cinder
• Optional parameters to create• Image per ‘datastore type’ and ‘version’
• Support AZs using --availability_zone
• Support for Neutron using --nic
{"instance": {
"status": "BUILD","updated": "2014-06-05T19:33:46","name": "test","links": [
{"href": "https://region-a.geo-
1.database.hpcloudsvc.com/v1.0/39745227274204/instances/ce629494-a64d-41ed-b73c-04cb28bb33bb",
"rel": "self"},{
"href": "https://region-a.geo-1.database.hpcloudsvc.com/instances/ce629494-a64d-41ed-b73c-04cb28bb33bb",
"rel": "bookmark"}
],"created": "2014-06-05T19:33:46","id": "ce629494-a64d-41ed-b73c-04cb28bb33bb","flavor": {
"id": "1002","links": [
{"href": "https://region-a.geo-
1.database.hpcloudsvc.com/v1.0/39745227274204/flavors/1002","rel": "self"
},{
"href": "https://region-a.geo-1.database.hpcloudsvc.com/flavors/1002",
"rel": "bookmark"}
]},"datastore": {
"version": "5.5","type": "mysql"
}}
}
Managing the Trove Database
• Resize flavor
• Resize volume
• Datastore specific extensions:• Create Database / Schema
• Create Users
• Grant Users Permissions on Databases
• Enable a Root User
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 201411
Trove is Tuned
• Trove sets up sane defaults• InnoDB only• Disable load data infile and select into outfile
• Ability to automatically tune settings by flavor – e.g. for my.cnf• Buffer Pool Size, • Log file size• max_connections, etc.
• API to programmatically target configuration groups for settings
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 201412
Trove is Secure
• Removes anonymous user
• Removes non-localhost users
• Removes local file access
• Mangles root user password
• Sets up Security Groups to manage access (via API)
• User SSH access to datastore not required
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 201413
Backup and Restore
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 201414
$ trove backup-create <backup-name> <instance-id>
• Optional params:• Description through --description
• Incremental backups using –parent
• To Restore backup use create:
$ trove create
<instance-name> <flavor-id>
--size <volume-size>
--backup <backup-id>
{
"backup": {
"instance_id": "9a31e420-aee6-4329-92f7-4b6b3790bcd4",
"status": "NEW",
"updated": "2014-06-05T19:43:28",
"locationRef": null,
"name": "DEMO_instance_test_backup",
"created": "2014-06-05T19:43:28",
"size": null,
"id": "f811a9cf-f397-4e07-a21f-7134094976f8",
"description": null
}
}
Backups in Trove
• Fully Managed
• Triggered and Tracked via API
• Streamed to Swift (OpenStack Object Storage)
• Incremental as well as Full backups
• Multiple formats per datastore supported via strategies:
• XtraBackup (Percona)
• mysqldump
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 201415
Replica Instances
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 201416
$ trove create
<instance-name> <flavor-id>
--size <volume-size>
--replica_of <instance-id>
• Support for async mysql replication (mysql slave instances)
• Manual detach using
$ trove update <instance-id>
--detach-replica-source
{"instance": {
"status": "BUILD","updated": "2014-06-05T19:33:46","name": "test","links": [
{"href": "https://region-a.geo-
1.database.hpcloudsvc.com/v1.0/39745227274204/instances/ce629494-a64d-41ed-b73c-04cb28bb33bb",
"rel": "self"},{
"href": "https://region-a.geo-1.database.hpcloudsvc.com/instances/ce629494-a64d-41ed-b73c-04cb28bb33bb",
"rel": "bookmark"}
],"created": "2014-06-05T19:33:46","id": "ce629494-a64d-41ed-b73c-04cb28bb33bb","flavor": {
"id": "1002","links": [
{"href": "https://region-a.geo-
1.database.hpcloudsvc.com/v1.0/39745227274204/flavors/1002","rel": "self"
},{
"href": "https://region-a.geo-1.database.hpcloudsvc.com/flavors/1002",
"rel": "bookmark"}
]},"datastore": {
"version": "5.5","type": "mysql"
}}
}
Clusters
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 201417
$ trove cluster-create <datastore> <ds-version>
• Optional parameters to cluster-create• --instance <flavor_id=flavor_id,volume=volume>
• Specify multiple times to create multiple instances for your cluster
• Initial support in Juno added for MongoDB Clusters• Sets up mongo config server, and mongo query routers
• Transparent and driven by configuration options
• Support for adding shards to existing cluster for horizontal scale out.
Juno and Kilo
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 201418
Completed in Juno
• Async MySQL replication (master-slave)
• Clusters for MongoDB
• Neutron Support
• Support for PostgreSQL
• Config-groups enhancements• Configuration groups per datastore / version
• Config-groups for MongoDB
• Backups for Cassandra and Couchbase
• Additional Tempest Tests
Planned for Kilo
• Building out clusters -- including Semi Synchronous (Galera) mysql clusters
• Async Replication v2 – failover support
• Associate flavors with datastores
• Access datastore logs via API
• Removing deprecated oslo-incubator code
• Upgrade testing through grenade
• YOUR IDEA HERE!
Your Idea Here
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 201419
• Growing community of contributors• 128 contributors from 28 companies• 1723 commits, and 231829 lines of code
• Open to new ideas, and code
• Lots of room for improvement
• Find us at #openstack-trove on FreeNode.
Thank You!
Got Questions?
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 201420
Nikhil Manchanda [HP]PTL, Trove
Email: nikhil.manchanda@hp.comIRC: SlickNik
Twitter: @SlickNik@hphelioncloud
Amrith Kumar [Tesora]Core Reviewer, TroveEmail: a@tesora.com
IRC: amrithTwitter: @amrithkumar
@tesoracorp
Tweet about this presentation #openstack #trove
top related