monitoring cloud native apps on pivotal cloud foundry with appdynamics

Monitoring Cloud Native Apps on PCFNima Badiey, Head of Business Development, Pivotal

Copyright © 2015 AppDynamics. All rights reserved. 2

> sudo whoamiLast login: Tuesday Dec 1 14:15:00 on appsphere> Which twitter

@badnima> History

Pivotal – Cloud FoundryVmware – Cloud Foundry (v1)Joyent – Node.js and SmartOS (Open Solaris) Six Apart – TypePad and Moveable TypeFlickr - APIsDeloitte – PPT + XLSSun – Solaris (UNIX) Desktops and ServersBoeing – 777


What?

Pivotal is working with AppDynamics on Cloud Foundry integration. Pivotal Cloud Foundry customers will be able to use AppDynamics to monitor their applications. This solution will be supported by AppDynamics. Pivotal is very interested in working with AppDynamics customers to make sure that

this solution meets their needs.

4

AgileMethods

Tools

12 Factor+ App Frameworks

DevOps Automation

Microservices& Containers

Continuous Delivery

Cloud Agnostic

Becoming Cloud Native

Cloud Native

DevOps ContinuousDelivery

ContainersMicro services


Why is Cloud Native Important?


…high performing organizations ship code 30 times faster (and complete these deployments

8,000 times faster), have 50% fewer failed deployments, and restore service 12 times

faster than their peers.

Time is Money


Easy as…


Extreme ProgrammingScrumKanban

CodebaseDependenciesConfigurationBacking ServicesBuild/Release/RunProcessesPort BindingConcurrencyDisposabilityDev/Prod ParityLogsAdmin Processes

EurekaHystrixTurbineRibbonZuulSpinnaker

How Do We Apply It?

Agile 12 Factor Microservices

Continuous Delivery Cloud DevOps

Vertical vs Horizontal ScaleApp Server ConfigurationBuildpacksExternal Backing ServicesIntegration ContractsAPI Controls


The Innovation Cycle

1. Design

2. Build

3. Configure

5. Deploy

6. Run

1. Provision

2. Authorize

3. Integrate

4. Monitor4. Audit

5. Scale

6. Recover

7. Upgrade Dev Loop

Business

Ops Loop

PIVOTAL CLOUD FOUNDRYCloud Native at Your Service

• 2011 - Founded the OSS project at VMware• 2013 - Formed Pivotal with Cloud Foundry as base platform• 2013 - Pivotal Cloud Foundry 1.0• 2014 - Launched Cloud Foundry Foundation• Today - Pivotal leadership:

– Employs 17 of 20 Open Source Project Leads– 70% of the full-time Open Source Engineers– Drives release process and continuous integration– Main influencer of roadmap, new feature development and project leadership


Cloud Foundry Foundation

PLATINUM

GOLD

SILVER


Cloud Foundry Powers Multiple Clouds


Spring Cloud Spring BootDev

Spring BootFrom 0 to app in < 5 min

Spring CloudDesigned for fragile infrastructure in

partnership with Netflix

Application Framework


Pivotal Cloud Foundry

Dev IT Ops

Self healing

Containermanagement

Integrated logging

Auto scaling

Application Performance Management

Runtime Platform

Self service


Infrastructure Automation

Infrastructure

BOSH

AWS VMware OpenStack

manage services not servers

zero downtime

no maintenancewindows

infra as codemulti cloud support

IT Ops

IT Ops

embedded OS


Cloud Native Framework

Contract: 12 Factor App

Contract: BOSH Release

Contract: Cloud Provider Interface

Culture

Dev

Dev IT Ops

Tools

Spring Cloud Spring Boot

BOSH

AWS VMware OpenStack

Application Framework

Runtime Platform

Infrastructure Automation

InfrastructureIT Ops

IT Ops

Pivotal Cloud Foundry and Pivotal Web Services

CLI and Easy to Use Web Console

How it Works: Just “Push” Your App

Deploy Any App Immediately!

Polyglot Agility

Scale Seamlessly

Vibrant Marketplace and User Created Services

Automate Your Ops

Software Developers Spend Too Much Time NOT Writing Software

0 3.25 6.5 9.75 13 16.25

Administrative tasks

Design and Coding

Brainstorming

Waiting for Test/Build

Managing Environments

Average Hours/Week


Let’s Test This Out


And many more...

Pivotal Cloud Foundry: Worldwide Cross Industry Adoption

APPDYNAMICS INTEGRATIONHow We Built It


PLATFORM

HOST

What Needs to be Measured?

NETWORKING (Connectivity)

VM VMVMVMVMCONTAINER CONTAINER

APPAPP

ROUTES (Traffic) SERVICES

SERVICES

SERVICES

SERVICES

SERVICES


Why Not Build Our Own APM?

• APM is a solved problem• Information vs. Intelligence• Frees us to focus on instrumentation


APM: It’s Good, But… There Are Many Repeatable Steps• RTFM• Create Account• Download Agent(s)• Manage Keys• Instrument App Server• Log-In to dashboard• !%X$@$ App Restart!

Turning on APM is a repeatable pattern (e.g., light switch) so why

not automate the pattern?


HOST

VM

How it Works From Developer’s Perspective

CONTAINER

My App (*.war)

Buildpack

+

=DEA

(Containerized App Code)

Developer’s app is “pushed” to CF

https://blog.pivotal.io/pivotal-cloud-foundry/products/monitoring-java-apps-appdynamics

Ser

vice

Bro

ker

The containerized app is then automatically instrumented and will begin publishing via the APM agent directly to AppDynamics upon completion of staging

Buildpacks are a convenient way of packaging framework and/or runtime support for an app. In the buildpack, we added “detect logic” to identify if AppDynamics was running as a service, and to automatically instrument apps

The Service Broker automates the connectivity between apps and any add-on service such as AppDynamics via GUI, CLI or Scripts


The Key Components

Machine Agent

Service Broker

An AppDynamics Machine Agent is a Java application that can consume system and runtime information from a JMX Endpoint. It can be configured to consume and relay information about runtime statistics and vm instance information of the various Cloud Foundry components, exposed by the Pivotal Ops Metrics Tile, back to the AppDynamics monitoring dashboard. This information does not pertain the application specific performance metrics but more of general runtime state of Cloud Foundry like number of apps running, number of DEA instances, memory, cpu, disk usage of various vms hosting the different subsystems of CF like Cloud Controller, UAA, NATS etc. The Machine Agent would be deployed as an application on Cloud Foundry.

Service Broker + Machine Agent

Tile


The Key Components

Machine Agent

Service Broker

The AppDynamics Service Broker would be a Java Service Broker that provides the endpoint information about the AppDynamics Controller (running on the Cloud) along with License keys or other information relevant to the AppDynamics application agent and would be deployed as a Java Application on Cloud Foundry. The Broker would expose the AppDynamics service on the Marketplace and allow users to directly create service instance and bind it to the application either from the Pivotal App Manager Console or from the command line. The service broker would be a static one in the sense that no new service would be really provisioned and only a static data about the AppDynamics collector required by the AppDynamics agent would be provided as part of the credential when an application binds to the service.


Tile


The Key Components

Machine Agent

Service Broker

The Cloud Administrator would utilize the AppDynamics Pivotal Tile to properly install and manage the Service Broker and Machine Agent on Pivotal Cloud Foundry. The Machine Agent application would be configured with the connectivity information to the AppDynamics SaaS Controller via the configuration settings on the Tile. The Tile would install the AppDynamics Service Broker as an application on Cloud Foundry and register it as a Service Broker while exposing its service plans on the Marketplace. Information about the AppDynamics Controller endpoint and license keys would be provided to the Service Broker at time of install and this data would then be relayed to the AppDynamics Agents when the AppDynamics service is bound to the application. The agent would then communicate with the SaaS Controller to instrument and provide back performance metrics. The Machine agent would relay the various runtime metrics of Cloud Foundry back to its Controller. This would allow users to monitor the application performance as well as the runtime status of various Cloud Foundry components from the AppDynamics Monitoring Dashboard.


Tile


Download AppD Tile from network.pivotal.io Install AppD Tile via PCF Ops Manager


Simple Configuration of the Service Broker


Monitor PCF Infrastructure using Machine Agent Extensions


AppDynamics in Services Marketplace


AppDynamics Service Plans


Binding AppDynamics to an App


App Monitoring Dashboard


Create Custom Dashboards


Monitor Infrastructure


Future Considerations: CF Metrics Firehose

dopplerdoppler

loggregator

doppler trafficcontroller

external

etcd

cf cli

vmvm

source

diego

dea

metron

statdsinjector

cc

nozzle

drainbinder

syslog drain

collector

VM


In Conclusion

Pivotal is working with AppDynamics on Cloud Foundry integration. Pivotal Cloud Foundry customers will be able to

run AppDynamics APM agents/solution within their applications on Pivotal Cloud Foundry, and this solution will be supported by AppDynamics. Pivotal is very interested in

working with AppDynamics customers to make sure that this solution meets their needs.


Free e-book from O’Reillyhttp://bit.ly/cloud-native-book

Available soon from O’Reilly Publishinghttp://shop.oreilly.com/product/0636920038252.do?code=CYBER15


Useful Links

pivotal.io/paas

network.pivotal.io

docs.pivotal.io

cloudfoundry.org

bosh.io

github.com/cloudfoundry


Many Thanks to:

PIVOTAL• Ryan Morgan• Ben Hale• Sabha Parameswaran• Josh McKenty• Scott Frederick

APPDYNAMCIS• Matt Polly• Pranta Das• Mahesh Gandhe• Sri Dharmasanam• Nikhil Katre • Naveen Kondapalli• Dustin Whittle


Q&A

Thank You