docker talk at cloud austin

23
DOCKER @ FLUX7 LABS

Upload: flux7

Post on 27-Jan-2015

106 views

Category:

Technology


1 download

DESCRIPTION

 

TRANSCRIPT

Page 1: Docker talk at Cloud Austin

DOCKER @ FLUX7 LABS

Page 2: Docker talk at Cloud Austin

DOCKER AUSTIN

FOR MORE DISCUSSIONS ON DOCKER!

Page 3: Docker talk at Cloud Austin

Who am I?

● Aater Suleman○ Part-time UT Professor ○ Geek, Architect, Developer, Ops, DevOps …

● Co-founder & CEO Flux7 Labs ○ DevOps Solutions

■ Deployments■ Cost/performance optimized large scale

website (Ruby on rails, node.js, Django) and Hadoop deployments

Page 4: Docker talk at Cloud Austin

VyScale Dev Flow

Docker's impact on performance (whitepaper WIP)

Multi-tenancy

Live process migration using CRIU (criu.org)

Four projects:

Page 5: Docker talk at Cloud Austin

Application: Single Service Provider

Receive Sensor Data

Report Generation

based on data

Report sent to End User

Internet of Things -- Solar Panel Monitoring

XML Data over TCP

Big Data Analytics

Page 6: Docker talk at Cloud Austin

Single Provider System

Provider Span

Location1 Location2Location3

Gateway1 Gateway2

Gateway3sensors

A provider has Mifi routers installed at multiple Locations which collect data from sensors and sends it to a remote TCP server via the internet.

TCP serverPort 6000

Cassandra port 9160

Flask AppPort 80

BrowserUses the Flask app at port 80Internet

Page 7: Docker talk at Cloud Austin

COMPONENTS

1. Cassandra for data persistence which we later use for generating reports for each gateway.

2. A Twisted TCP server listening at PORT 6000, for data ingestion from multiple gateways owned by the provider.

3. A Flask app serving at PORT 80 as the admin panel for setting customizations and viewing reports.

Page 8: Docker talk at Cloud Austin

Customer NCustomer 2 …Customer 1 Each customer can have multiple gateways

commissioned to them.

Remote Twister TCP Server (Non–Blocking I/O)

Cassandra NoSQL data store(High Volume High Velocity Write which scales Linearly across the

cluster )

Power consumption status on website and

mails

*G - Gateway

Web App

Mailer

Page 9: Docker talk at Cloud Austin

SINGLE PROVIDER LAUNCH

For launching the single provider version, the following was done:

1. nohup python tcp_server.py & # For firing up the TCP server.

2. nohup python flask_app.py & # For firing up the admin panel

Both these code bases houses hard-coded Cassandra KEYSPACE

Success!

Page 10: Docker talk at Cloud Austin

Application: Multiple Service Providers

Page 11: Docker talk at Cloud Austin

Provider 1 sends data to port 6001

and accesses flask app at port 8081

Provider 2 sends data to port 6002 and accesses flask app at

port 8082

Flask container-runs flask app at port 80. Exposes port 80 and

published it to port 8081 for provider 1

Flask container-runs flask app at port 80. Exposes

port 80 and published it to port 8082 for provider 2

TCP server container-runs at port 6000.

Exposes port 6000 and published it to port 6001

for provider 1

TCP server container-runs at port 6000. Exposes

port 6000 and published it to port 6002 for provider 2

Cassandra

Internet

Page 12: Docker talk at Cloud Austin

KNEE-JERK APPROACH

Sprinkle Tenant ID everywhere in the code and DB

Time consuming

Expensive

Poor isolation

Security

Maintenance

Rigidity

Page 13: Docker talk at Cloud Austin

An alternate solution is to use Virtual Machine (VM)

Hosts are expensive ($)VMs are expensive (high overhead)

MULTIPLE HOST/VMS

Page 14: Docker talk at Cloud Austin

AND THE SOLUTION

Page 15: Docker talk at Cloud Austin

How: Isolated environments for running multiple instances of the app

WHY DOCKER?

Docker containers provide isolation that is

Fast

Inexpensive

Page 16: Docker talk at Cloud Austin

Create a docker container for the new version of the app

PLAN

Setup environments/dependencies correctly

Start a Cassandra container.

Page 17: Docker talk at Cloud Austin

# start a docker container for consuming gateway data at gateway_portstart_command = 'python software/remote_server.py ' + provider_idremote_server = docker_client.create_container('flux7/labs', # docker imagecommand=start_command, # start command contains the keyspace parameter, keyspace is the provider_idname='remote_server_' + provider_id, # name the container, name is provider_idports=[(6000, 'tcp'),]) # open port for binding, remote_server.py listens at 6000docker_client.start(remote_server, port_bindings={6000: ('0.0.0.0', gateway_port)}, links={'db': 'cassandra'})

AUTOMATION

# start a docker container for serving admin panel at admin_portstart_command = 'python software/flask_app.py ' + provider_idremote_server = docker_client.create_container('flux7/labs', # docker imagecommand=start_command, # start command contains the keyspace parameter, keyspace is the provider_idname='admin_panel_' + provider_id, # name the container, name is provider_idports=[(80, 'tcp'),]) # open port for binding, remote_server.py listens at 6000docker_client.start(remote_server, port_bindings={80: ('0.0.0.0',admin_port)}, links={'db': 'cassandra'})

An automation was the next foreseeable step, and for that we found Docker-py extremely useful. We used something like:

# Yes. We love Python!def start_provider(provider_id, gateway_port, admin_port ):docker_client = docker.Client(base_url='unix://var/run/docker.sock', version='1.6', timeout=100)

Page 18: Docker talk at Cloud Austin

For now, a locally running container serving at PORT 9160 using the command

similar to this:

docker run -d -p 9160:9160 -name db flux7/cassandra

OUR SOLUTION- EXPLAINED

Page 19: Docker talk at Cloud Austin

▪ Create a keyspace ‘provider1’ using pycassaShell.

We fired up our two code bases on two separate containers like this:

OUR SOLUTION- EXPLAINED

docker run -name remote_server_1 -link db:cassandra -

p 6000:6000 flux7/labs python software/remote_server.

py provider1

docker run -name flask_app_1 -link db:cassandra -p

6000:6000 flux7/labs python software/flask_app.py

provider1

Page 20: Docker talk at Cloud Austin

DOCKER ISSUES DISCOVERED

Docker does not support multiple instances of Cassandra running on the

same machine.

Hosting multiple database instances on a single machine can quickly

cause resource shortages

Page 21: Docker talk at Cloud Austin

❑ Followed the traditional solution to make an application multi-tenant

OUR SOLUTION

Code Changes •• To data ingestion server and web server by adding the keyspace parameter to the DB accesses.

Cassandra KEYSPACE /

provider ID •• Passed to each instance of the app on the

command line.

❑ Each provider in the data store gets a separate namespace without making

any changes to the column family schema.

Use of KEYSPACE as the namespace for each provider in the data store

Page 22: Docker talk at Cloud Austin

LESSONS WE LEARNED

1. Docker is an extremely fast and elegant isolation framework: easy to port, cheap to run, easy to orchestrate

2. Multi-tenancy != changing the app to support multiple tenants

3. Docker orchestration frameworks are not at par with Docker today. What we have written is yet another one but for multi-tenancy.

4. Dockerfiles still need work -- we used shell scripts in some places

5. We can run multiple commands/container

Page 23: Docker talk at Cloud Austin

DOCKER AUSTIN

FOR MORE DISCUSSIONS ON DOCKER!