data driven infrastructure

Friday, August 23, 13

DataDrivenInfrastructure


?


?

http://git.io/UmUdKAFriday, August 23, 13

http://git.io/Px7kDg

http://git.io/Px7kDg

Let meTell youa story


What is a

Cloud


I don’t have to think about IT

With the Cloud...


aaSPlatformSoftware

InfrastructureFriday, August 23, 13

Where is our cloud?


UtopiaFriday, August 23, 13

James Fryman


Automations Junkie


Are youCrazy?


Goals?Data?What


Machine

Parsable


There is SystemOne


Feedback

PuppetDBProvisioning

Configuring

Destroying

gPanel


Partsof the

System


provisioner


Home GrownFriday, August 23, 13

Home Grown


controller


Lots O’ Models


file { '/etc/facter/facts.d/quagga_manage_service': ensure => present, replace => false,} if $::quagga_manage_service { $service_state[ensure] = running $service_state[enable] = true} else { $service_state[ensure] = undef $service_state[enable] = undef} service { 'quagga': ensure => $service_state[ensure], enable => $service_state[enable],}


orchestrator


ChatOps


Feedbackof the

System


Metrics&Monitoring


begin JSON.parse(RestClient.get(url)).each do |cache| data["#{cache['target']}"] = 0 count = 0 cache["datapoints"].each do |point| unless (point[0].nil?) data["#{cache['target']}"] += point[0] count += 1 end end data["#{cache['target']}"] /= count data["total"] += data["#{cache['target']}"] endrescue ZeroDivisionError => e puts "UNKNOWN metric not in graphite!" exit EXIT_UNKNOWNrescue => e puts e.message exit EXIT_CRITICALend


event


eventlog


eventalertlog


eventalertlog error


Must self-correct


Feedback

PuppetDBProvisioning

Configuring

Destroying

gPanel


Deployable using text files


Modularity


haproxy::proxy { $es_proxy_name: proxy => 'listen', mode => 'http', ip => $::ipaddress_lo, port => '9200', config => { balance => 'roundrobin', }, }


haproxy::proxy::member { $es_proxy_name: hostname => $::ec2_local_ipv4, port => '9200', param => [ 'weight 1', 'maxconn 1000', 'check', ], }


collectd::plugin { [ 'cpu', 'load', 'memory', 'swap', 'irq', 'exec', 'entropy', ]: }

collectd::plugin { [ 'df', 'interface', 'protocols', 'disk', ]: config => true }


Level 4:Templates

Level 5:Data Driven


SelfAuthoritative


def self.collect_resources(resource) query = [ "and", ["=", "type", "File"], ["=", "exported", true], ["=", "tag", "nagios::object::#{resource}"], ["=", ["node", "active"], true] ] self.search('/v2/resources', query.to_json) end


# Seed the initial file with false, and enable # on next run with gh-nagios enable_pager <hostname>

file { '/etc/facter/facts.d/enable_pager.txt': ensure => file, owner => 'root', group => 'root', mode => '0644', content => 'enable_pager=false', replace => false,}


@@file { "/etc/nagios/objects/${type}.d/${filename}.cfg": content => template('nagios/etc/nagios/object.erb'), mode => '0444', group => 'nagios', notify => Service['nagios'], }


Tips


Refactoring


least to most

specificFriday, August 23, 13

frymanet.com

mysqlnginx rails

rubycommon admin

package repos


graduate to

paramsFriday, August 23, 13

class ntp::params { $defaults = { package => { version => ‘latest’, }, config => { servers => [‘pool.ntp.org’], }, }}


externalize


class ntp::params { $defaults = { package => { version => hiera(‘ntp_package_version’), }, config => { servers => hiera(‘ntp_servers’), }, }}


CloudFormation


"Resources": { "RendererServerGroup": { "Type": "AWS::AutoScaling::AutoScalingGroup", "Properties": { "AvailabilityZones": { "Fn::GetAZs": "" }, "LaunchConfigurationName": { "Ref": "LaunchConfig" }, "MinSize": "2", "MaxSize": "16", "Tags": [ { "Key": "Environment", "Value": "Production", "PropagateAtLaunch": "true" }, { "Key": "Role", "Value": "renderer", "PropagateAtLaunch": "true" } ] } },Friday, August 23, 13

Autoloading


# autoloader.ppclass nagios::autoload_helpers { $helpers = get_nagios_helpers()

case $::puppetversion { /^3/: { include $helpers } default: { nagios::autoload_helpers::import_shim { $helpers: } } }}


module Puppet::Parser::Functions newfunction(:get_nagios_helpers, :type => :rvalue, :doc => "Grab all modules that have nagios helpers for import") do

module_path = File.expand_path('..', Puppet::Module.find('nagios',compiler.environment.to_s).path)

helpers = Dir["#{module_path}/**/nagios/helpers.pp"].map do |d| "#{d.split('/')[-4]}::nagios::helpers" done

helpers endend


Modeling


Be Dynamic


<%- if @comment -%>### <%= @comment %><%- end -%><%= @type %> <% if @label %><%= @label %><% end %> {<%- @config.sort.reverse.each do |key,value| -%> <%- if value.class == Array -%> <%- value.each do |element| -%> <%= key %>(<%= element %>); <%- end -%> <%- else -%> <%= key %>(<%= value %>); <%- end -%><%- end -%>};


Fencing Resources


# Seed the initial file with false, and enable # on next run with gh-nagios enable_pager <hostname>

file { '/etc/facter/facts.d/enable_pager.txt': ensure => file, owner => 'root', group => 'root', mode => '0644', content => 'enable_pager=false', replace => false,}


if $::enable_pager { @@file { "/etc/nagios/objects/${type}.d/${filename}.cfg": content => template('nagios/etc/nagios/object.erb'), mode => '0444', group => 'nagios', notify => Service['nagios'], }}


Buy it?


It’s About


Missing?What’s


Language


Predictive Analysis


Coming

Home


Goals?Data?What


SystemsThinking

Recap


There is SystemOne

Machine

ParsableMust be aKnownQuantity

SelfAuthoritative

Must self-correctFriday, August 23, 13

jfryman

[email protected]


mailto:[email protected]

mailto:[email protected]

what’s next?


data driven infrastructure

Technology

end end data

proxy proxy

cache data

message exit exit

true self

pager file

end rescue zerodivisionerror

unknown rescue