storage is not virtualized enough - part 2: let's do service chaining

Storage Is Not Virtualized EnoughPart 2: Let’s do service chaining !

Howard Huang@HuaweiAccela Zhao@EMC

Bio

Accela Zhao: Technologist at EMC OCTO (Office of CTO), 3 year experience in Openstack and distributed storage, active Cinder contributor, former presenter in Tokyo summit.

Howard Huang: Standard Engineer and open source community operation manager from Huawei, involved in various open source projects like OpenStack, OPNFV, Kubernetes, Hyperledger

Outline

❏ Motivation❏ Overview of OpenStack Storage Function Chaining❏ Future Plans

Motivation - 1: Flashback on part 1

Motivation - 2: Why do people need to chain things

You know, the chain thing

- Salvatore Tessio

Motivation - 2: Why do people need to chain things

➢ Peter deployed Ceph Block, deduplication and compression services in three VMs. (Dedup and Comp might come from ZFS ZVOL)

➢ Now he wants to have a storage service that offers a compressed and deduped Ceph Block, for his compute service.

➢ Naturally he could just chain the three storage service VMs he’s got together.➢ How does he chain it ?

Outline

❏ Motivation❏ Overview of OpenStack Storage Function Chaining

❏ OpenStack Network Function Chaining Reference❏ OpenStack Storage Function Chaining Design❏ Potential benefits

❏ Future Plans

OpenStack Neutron SFC

Outline



❏ Future Plans

OpenStack Storage Function Chaining Design➢ General Ideas:

○ Utilize as many existing OpenStack Storage Services as possible○ Borrow Ideas from Networking SFC○ Minimal changes to the current modules○ Try Block first

➢ Challenges:○ Cinder attaches physical arrays for the VM, how do we make Cinder attach a VM for another

VM ?○ Cinder does not dynamically add or remove a storage backend○ Networking○ Performance

OpenStack Storage Function Chaining Design➢ Start discussion like a community:

○ http://www.meetup.com/Shanghai-SFV-Meetup/

➢ Initial thoughts discussed during the meetups:○ Use LVM

■ Passed. Although LVM could store the data each time, you still need a way to link them○ Use iSCSI

■ Passed. We will need static config, and no O/M functionality like Cinder involved.○ Divide up separate LUNs on the array for each VM

■ Passed. Again, difficult and no OpenStack module involved to help

http://www.meetup.com/Shanghai-SFV-Meetup/

http://www.meetup.com/Shanghai-SFV-Meetup/

We need a module that could help couple these VMs together

OpenStack Storage Function Chaining Design


Coupler

MappingTable

PolicyConsumer

Coupler Cinder Plugin

Allocator

● Coupler provides a virtual backend service

○ Mapping Table : Provide a mapping for storage service VM

○ Policy Consumer : Analyze policies injected into Coupler

○ Allocator : Allocating fake IQNs so that for each /dev/vdX they know which chain they belong to

○ Cinder Plugin : expose as a “normal” storage backend driver


● Coupler allows VMs to act as an virtualized storage backend, and provide volume mounts to another● Multiple VMs can be chained flexibly to compose a new storage service● Ceph can also be natively hosted inside Openstack. So as other storage backends● Network part is tricky …

Outline



❏ Future Plans

Storage Service Chaining Potential BenefitsDecompose storage systems into

fine-grained functions, virtualized

into VMs.

For one type of function, there will

be VMs providing diverse algorithm

implementation.

The policy chains VMs to provide

customized storage services.

VMs comprise pools. They scale-out

to serve more users.

Users have the freedom to choose

what feature they want.

Storage Service Chaining Potential Benefits (2)

● We could finally enable OpenStack to use a service that OpenStack itself provided, just like k8se finally got something that would make storage backends running as services provided by OpenStack, provided to other OpenStack services, like what k8s already could do.

Storage Service Chaining Potential Benefits

Outline

❏ Motivation❏ Overview of OpenStack Storage Function Chaining❏ Future Plans

Future Plan1. Try to code it out, and fix some of the challenges.

a. Coupler development : https://github.com/openstack/couplerb. New use cases or requirements : https://github.com/OpenSFV/specsc. Write the coupler driver for Cinder

2. File and Object ?3. Standardization opportunities.4. Possible collaboration with Kolla5. Policy module study6. Demo it out.

https://github.com/openstack/coupler

https://github.com/OpenSFV/specs

Thank You!Contact us: accelazh@gmail, [email protected]

storage is not virtualized enough - part 2: let's do service chaining

Technology