tdc 2013 7 dicas para acelerar os testes
DESCRIPTION
Apresentação realizada no TDC 2013, na trilha de testes contendo dicas de como utilizar técnicas e ferramentas atuais para acelerar os testes de software e obter resultados mais efetivos.TRANSCRIPT
![Page 1: TDC 2013 7 Dicas para acelerar os testes](https://reader035.vdocuments.net/reader035/viewer/2022062513/5571c932d8b42a3f5e8b47f0/html5/thumbnails/1.jpg)
© 2011 IBM Corporation
7 Dicas para Acelerar os Testes com Resultados Efetivos12 de Julho de 2013
Felipe Freire – [email protected] Software, Rational
TDC 2013
@pfelipebr
![Page 2: TDC 2013 7 Dicas para acelerar os testes](https://reader035.vdocuments.net/reader035/viewer/2022062513/5571c932d8b42a3f5e8b47f0/html5/thumbnails/2.jpg)
© 2012 IBM Corporation
Enabling Product and Service Innovation | Rational
Sobre a a apresentação – ObjetivoDar armas ao testador para mudar a organização
Compartilhar experiências com testes (dicas)
Experiência IBM em clientes globais
Eventos de testes
STAREAST / STARWEST
BRATESTE
Ajudar a trazer novas idéias para sua empresa
![Page 3: TDC 2013 7 Dicas para acelerar os testes](https://reader035.vdocuments.net/reader035/viewer/2022062513/5571c932d8b42a3f5e8b47f0/html5/thumbnails/3.jpg)
© 2012 IBM Corporation
Enabling Product and Service Innovation | Rational
Agenda
Introdução
Dicas
Dê foco aos testes
Antecipe a execução dos testes
Versione tudo (release)
Cloud – Flexibilize os ambientes de testes
Virtualize os serviços e ambientes de testes
Automatize corretamente
Mobile – testar múltipals configurações
Encerramento
3
![Page 4: TDC 2013 7 Dicas para acelerar os testes](https://reader035.vdocuments.net/reader035/viewer/2022062513/5571c932d8b42a3f5e8b47f0/html5/thumbnails/4.jpg)
© 2012 IBM Corporation
Enabling Product and Service Innovation | Rational
4
Speed Value
of digital content in 2012, up 50% from 2011
New Insights
1.1.Technology factorsTechnology factors1.1.Technology factorsTechnology factors
2. People skills
3. Market factors
4. Macro-economic factors
Factors impacting organizations:Factors impacting organizations:
Technology is the key to success, according to CEOs …
IBM Global CEO Study
1
5. Regulatory concerns
6. Globalization
7. Socio-economic factors
8. Environmental issues
9. Geopolitical factors
![Page 5: TDC 2013 7 Dicas para acelerar os testes](https://reader035.vdocuments.net/reader035/viewer/2022062513/5571c932d8b42a3f5e8b47f0/html5/thumbnails/5.jpg)
© 2012 IBM Corporation
Enabling Product and Service Innovation | Rational
5
Qualidade de Software: Problema de custo em todas as indústrias
Softwares são culpados por mais problemas de negócio do que qualquer outro produto feito pelo homem
A baixa qualidade dos softwares tornou-se um dos tópicos mais caros da história humana
– > $150 bilhões por ano nos EUA.
– > $500 bilhões por ano no mundo.
Projetos cancelados devido a baixa qualidade são > 15% mais custosos do que projetos bem sucedidos de mesmo tipo e tamanho..
Source: Capers Jones, 2011Based on 675 companies, 35 government/military groups, 13,500 projects, 50-75 new projects/month, 24 countries, 15 lawsuites
![Page 6: TDC 2013 7 Dicas para acelerar os testes](https://reader035.vdocuments.net/reader035/viewer/2022062513/5571c932d8b42a3f5e8b47f0/html5/thumbnails/6.jpg)
© 2012 IBM Corporation
Enabling Product and Service Innovation | Rational
6
Desafios de entregar no prazo com qualidade
34% de todos os novos projetos de TI implantam com atraso**
experimenta atrasos na integração, configuração ou teste da aplicação
41%41%experimentam atrasos para identificação de problemas e ajustes em produção
45%45%das aplicações sofrem roll-back por problemas de qualidade
51%51%
tempo médio para entregar uma mudança simples
4-6 Semanas
4-6 Semanas
tempo médio para isolar um defeito
3-4 Semanas
3-4 SemanasGAP GAP
ProduçãoDesenvolvimento Testes
1. Dificuldade em atender às requisições de desenvolvimento e testes para configuração de ambientes – infraestrutura, middleware e aplicações2. Defeitos em aplicações e problemas de desempenho causam 50% dos rollbacks
1. Dificuldade em atender às requisições de desenvolvimento e testes para configuração de ambientes – infraestrutura, middleware e aplicações2. Defeitos em aplicações e problemas de desempenho causam 50% dos rollbacks
![Page 7: TDC 2013 7 Dicas para acelerar os testes](https://reader035.vdocuments.net/reader035/viewer/2022062513/5571c932d8b42a3f5e8b47f0/html5/thumbnails/7.jpg)
© 2012 IBM Corporation
Enabling Product and Service Innovation | Rational
Piores práticas – experiência pessoal
1. Deixar os testes para o final do projeto
1. Dois ciclos de testes são o suficiente
1. Qualquer um pode criar e executar os testes
1. Todos os testes devem ser detalhados e muito bem estruturados
1. Buscar 100% de automação
1. Executar o maior número de testes possíveis
1. Todos os defeitos encontrados devem ser corrigidos
1. Ignorar configurações para os testes
7
http://www.slideshare.net/pfelipe/ibm-rational-piores-prticas-em-testes
![Page 8: TDC 2013 7 Dicas para acelerar os testes](https://reader035.vdocuments.net/reader035/viewer/2022062513/5571c932d8b42a3f5e8b47f0/html5/thumbnails/8.jpg)
© 2012 IBM Corporation
Enabling Product and Service Innovation | Rational
1) Dê foco aos testes
O esforço de testes pode beirar o infinito!
– Testar todas as variáveis, cenários, configurações…
Onde os testes são mais importantes?
– Histórico de outros projetos similares onde deu problema?
– Risco: quaiscaracterísticas do sistema são mais críticas?
Abordagem dos testes
Na média, desenvolvedores profissionais cometem de 15 a 50 erros a cada mil linhas de código criadas.
Code Complete (2nd edition) http://cc2e.com
CoQ = Technical Debt (Agile)
![Page 9: TDC 2013 7 Dicas para acelerar os testes](https://reader035.vdocuments.net/reader035/viewer/2022062513/5571c932d8b42a3f5e8b47f0/html5/thumbnails/9.jpg)
© 2012 IBM Corporation
Enabling Product and Service Innovation | Rational
![Page 10: TDC 2013 7 Dicas para acelerar os testes](https://reader035.vdocuments.net/reader035/viewer/2022062513/5571c932d8b42a3f5e8b47f0/html5/thumbnails/10.jpg)
© 2012 IBM Corporation
Enabling Product and Service Innovation | Rational
![Page 11: TDC 2013 7 Dicas para acelerar os testes](https://reader035.vdocuments.net/reader035/viewer/2022062513/5571c932d8b42a3f5e8b47f0/html5/thumbnails/11.jpg)
© 2012 IBM Corporation
Enabling Product and Service Innovation | Rational
2) Antecipe a execução dos testes
Desenvolvimento iterativo / ágil
– Entregas parciais
– Oportunidade de verificar a qualidade antecipadamente
– Ciclos de testes alinhados com as iterações / sprints
“Boas equipes desenvolvem bons softwares"
![Page 12: TDC 2013 7 Dicas para acelerar os testes](https://reader035.vdocuments.net/reader035/viewer/2022062513/5571c932d8b42a3f5e8b47f0/html5/thumbnails/12.jpg)
© 2012 IBM Corporation
Enabling Product and Service Innovation | Rational
![Page 13: TDC 2013 7 Dicas para acelerar os testes](https://reader035.vdocuments.net/reader035/viewer/2022062513/5571c932d8b42a3f5e8b47f0/html5/thumbnails/13.jpg)
© 2012 IBM Corporation
Enabling Product and Service Innovation | Rational
Testes de Sistema
Requisitos
TesteUnitário
UAT
OAT
Performance
Barreiras da Qualidade
Ambiente Tradicional
Big Bang
× Múltiplos defeitos identificados de uma vez× Maior risco× Mais caro
![Page 14: TDC 2013 7 Dicas para acelerar os testes](https://reader035.vdocuments.net/reader035/viewer/2022062513/5571c932d8b42a3f5e8b47f0/html5/thumbnails/14.jpg)
© 2012 IBM Corporation
Enabling Product and Service Innovation | Rational
Requisitos
TesteUnitário
Performance
UAT
OAT
Testes Integrados
Teste de Integração
Sys Tests
Acelera o teste Reduz custo Diminui o risco
![Page 15: TDC 2013 7 Dicas para acelerar os testes](https://reader035.vdocuments.net/reader035/viewer/2022062513/5571c932d8b42a3f5e8b47f0/html5/thumbnails/15.jpg)
© 2012 IBM Corporation
Enabling Product and Service Innovation | Rational
3) Versione tudo – inclusive a infraestrutura
![Page 16: TDC 2013 7 Dicas para acelerar os testes](https://reader035.vdocuments.net/reader035/viewer/2022062513/5571c932d8b42a3f5e8b47f0/html5/thumbnails/16.jpg)
© 2012 IBM Corporation
Enabling Product and Service Innovation | Rational
Troque o seu servidor por uma nuvem
Exemplo: 300 servidores para as equipes de desenvolvimento
– Cada equipe tem os seus servidores
– Hardware obsoleto
– Sem controle
– Testes e desenvolvimento rodando na mesma máquina
![Page 17: TDC 2013 7 Dicas para acelerar os testes](https://reader035.vdocuments.net/reader035/viewer/2022062513/5571c932d8b42a3f5e8b47f0/html5/thumbnails/17.jpg)
© 2012 IBM Corporation
Enabling Product and Service Innovation | Rational
4) Flexibilize os ambientes de testes
Cloud
DevOps
↪Common Source Control
↪Automated Builds (Build Definitions)
↪Continuous Integration (CI)
↪Automated Delivery
↪Continuous Delivery to Test (CD)
↪Continuous Delivery to Production-like Systems (Infrastructure as Code)
↪Continuous Delivery thru Prod (Cloud based)
![Page 18: TDC 2013 7 Dicas para acelerar os testes](https://reader035.vdocuments.net/reader035/viewer/2022062513/5571c932d8b42a3f5e8b47f0/html5/thumbnails/18.jpg)
© 2012 IBM Corporation
Enabling Product and Service Innovation | Rational
A esteira de entregas
Implantar
Ambiente de execução do sistema
Repositório deControle de fontes
Artefatos fontes
.jsp
.java
.html
chef recipes
.sh
Artefatos entregáveis
Build, Empacota, Testes unitáriosBinários Configuração da plataforma
Biblioteca
18
![Page 19: TDC 2013 7 Dicas para acelerar os testes](https://reader035.vdocuments.net/reader035/viewer/2022062513/5571c932d8b42a3f5e8b47f0/html5/thumbnails/19.jpg)
© 2012 IBM Corporation
Enabling Product and Service Innovation | Rational
E depois que o software é construído?
19
Instruções de Instalação
Red Hat Linux
Nanana na nana na nanana. Na nanana na nana. Nana nanana nananan na nana na nana nanana na na nanana.
Apache Web Server
Nanana na nana na nanana. Na nanana na nana. Nana nanana nananan na nana na nana nanana na na nanana.
CLOUD
deploy.rb
cookbook_file "/home/jazz/.jrubyrc" do
# handle segfaults that occur when using JRuby w/ the IBM JDK
# see: http://jira.codehaus.org/browse/JRUBY-5700
source "jrubyrc"
mode "0644"
end
env_options = {
# ensure that build scripts can easily access Java and Jruby from the normal path
'ANT_HOME' => node[:ant][:ant_home],
'JAVA_HOME' => node[:java][:java_path],
'PATH' => "#{node[:java][:java_path]}/bin:#{node[:jruby][:install_dir]}/jruby/bin:#{ENV["PATH"]}"
}
# install external gem libs
%w{cucumber httpclient rake net-ssh net-scp httpclient jruby-openssl json-jruby rubyzip}.each do |gem_name|
execute "install_#{gem_name}" do
action :run
command "#{node[:jruby][:install_dir]}/jruby/bin/jgem install #{gem_name}"
environment(env_options)
end
deploy.rb
cookbook_file "/home/jazz/.jrubyrc" do
# handle segfaults that occur when using JRuby w/ the IBM JDK
# see: http://jira.codehaus.org/browse/JRUBY-5700
source "jrubyrc"
mode "0644"
end
env_options = {
# ensure that build scripts can easily access Java and Jruby from the normal path
'ANT_HOME' => node[:ant][:ant_home],
'JAVA_HOME' => node[:java][:java_path],
'PATH' => "#{node[:java][:java_path]}/bin:#{node[:jruby][:install_dir]}/jruby/bin:#{ENV["PATH"]}"
}
# install external gem libs
%w{cucumber httpclient rake net-ssh net-scp httpclient jruby-openssl json-jruby rubyzip}.each do |gem_name|
execute "install_#{gem_name}" do
action :run
command "#{node[:jruby][:install_dir]}/jruby/bin/jgem install #{gem_name}"
environment(env_options)
end
![Page 20: TDC 2013 7 Dicas para acelerar os testes](https://reader035.vdocuments.net/reader035/viewer/2022062513/5571c932d8b42a3f5e8b47f0/html5/thumbnails/20.jpg)
© 2012 IBM Corporation
Enabling Product and Service Innovation | Rational
Testing Tradeoffs
CostF
idelity
Traditional Staging Environment
Testing in Cloud-basedContinuousDelivery
Testing in Cloud-basedContinuous Delivery
with Service Simulation Traditional UAT without DevOps
Typical JUnit Tests
Build Verification Tests
![Page 21: TDC 2013 7 Dicas para acelerar os testes](https://reader035.vdocuments.net/reader035/viewer/2022062513/5571c932d8b42a3f5e8b47f0/html5/thumbnails/21.jpg)
© 2012 IBM Corporation
Enabling Product and Service Innovation | Rational
21
DevOps - checklist
• Do you use source control?
• Can you deploy a system in one step?
• Do you deploy your applications daily and verify them?
• Do you have an issue tracking system for operations, linked to a bug database used for development?
• Do you validate platform software against expected KPIs, before deploying your application?
• Do you have well defined delivery pipeline?
• Do you have agreed upon patterns for applications and platforms?
• Can your developers launch, use, and destroy representative environments on demand?
• Do you provide Infrastructure and Platform as a Service for your development teams?
• Do you have automated tests to validate your application function and security?
• Do your new operation engineers understand how to automate system administration?
• Do your operations and development teams collaborate regularly?
*Based on “The Joel Test: 12 Steps to Better Code”http://www.joelonsoftware.com/articles/fog0000000043.html
![Page 22: TDC 2013 7 Dicas para acelerar os testes](https://reader035.vdocuments.net/reader035/viewer/2022062513/5571c932d8b42a3f5e8b47f0/html5/thumbnails/22.jpg)
© 2012 IBM Corporation
Enabling Product and Service Innovation | Rational
Define constrained systems and services for virtualization
Integration OperabilityUser AcceptanceSystem IntegrationSystemUnit Integration OperabilityUser AcceptanceSystem IntegrationSystemUnit
Performance
Developer uses virtual services to unit test code on their workstation – no elaborate environment required
Development and Test teams uses virtual services for early stage advanced testing
The full system or components of the system can be performance tested early in the cycle. Virtual services can be used when components are not available
Later in the cycle, final pre-production versions of services are swapped in as they become available. Final testing is done against real services
5) Virtualização de serviços e aplicaçõesbenefícios para todo o ciclo
![Page 23: TDC 2013 7 Dicas para acelerar os testes](https://reader035.vdocuments.net/reader035/viewer/2022062513/5571c932d8b42a3f5e8b47f0/html5/thumbnails/23.jpg)
© 2012 IBM Corporation
Enabling Product and Service Innovation | Rational
Incremental Integration TestingIncremental Integration Testing
Actual Service/App
Virtual Service/App
Test Virtualization is an enabler for continuous Integration Testing
Services, applications, systems are introduced into the continuous integration cycle in a prioritized, controlled fashion.
Test Virtualization Solution is a key enabler for Continuous Integration Testing
![Page 24: TDC 2013 7 Dicas para acelerar os testes](https://reader035.vdocuments.net/reader035/viewer/2022062513/5571c932d8b42a3f5e8b47f0/html5/thumbnails/24.jpg)
© 2012 IBM Corporation
Enabling Product and Service Innovation | Rational
6) Automatize corretamente
![Page 25: TDC 2013 7 Dicas para acelerar os testes](https://reader035.vdocuments.net/reader035/viewer/2022062513/5571c932d8b42a3f5e8b47f0/html5/thumbnails/25.jpg)
© 2012 IBM Corporation
Enabling Product and Service Innovation | Rational
Há retorno no investimento em automação dos testes?
Testes executados com maior frequencia
Testes levam menos tempo para executar
Leva menos esforço humano para executar os testes
Podemos cobrir mais características do sistema
Podemos executar o equivalente a dias/semanas de testes em poucos minutos / horas
![Page 26: TDC 2013 7 Dicas para acelerar os testes](https://reader035.vdocuments.net/reader035/viewer/2022062513/5571c932d8b42a3f5e8b47f0/html5/thumbnails/26.jpg)
© 2012 IBM Corporation
Enabling Product and Service Innovation | Rational
Ferramentas substituem testadores?
“Podemos reduzir o número de testadores uma vez que utilizarmos ferramentas de
automação?”
– Quem são seus testadores?
• Seres “não-pensantes”, ou
• Investigadores inteligentes?
– Precisamos de mais skills, não menos
– Automação pode liberar o testador para realizar mais designs de testes
e exploração
• E assim achar mais defeitos
Ferramentas não substituem testadores, só suportam eles!
![Page 27: TDC 2013 7 Dicas para acelerar os testes](https://reader035.vdocuments.net/reader035/viewer/2022062513/5571c932d8b42a3f5e8b47f0/html5/thumbnails/27.jpg)
© 2012 IBM Corporation
Enabling Product and Service Innovation | Rational
Gerentes: o que buscar na automação dos testes
Benefícios da automação devem ser crescentes – ex: número de testes, cobertura
Custos devem ser cada vez menores
– Custo de construção para novos testes, manutenção (por teste), análise de falhas
Sinais de problemas na arquitetura do testware – testadores não-técnicos conseguem escrever e executar testes automatizados?
– Me mostre um teste automatizado rodando
– Vamos mudar para outra ferramenta? (observe a reação)
![Page 28: TDC 2013 7 Dicas para acelerar os testes](https://reader035.vdocuments.net/reader035/viewer/2022062513/5571c932d8b42a3f5e8b47f0/html5/thumbnails/28.jpg)
© 2012 IBM Corporation
Enabling Product and Service Innovation | Rational
7) Mobile – testar múltipals configurações
Client Tier Devices
Mobile-specific challenges:Lots of device targetsProvisioning rules and
artifactsCurated app storesDependent upon
backend service versions
![Page 29: TDC 2013 7 Dicas para acelerar os testes](https://reader035.vdocuments.net/reader035/viewer/2022062513/5571c932d8b42a3f5e8b47f0/html5/thumbnails/29.jpg)
© 2012 IBM Corporation
Enabling Product and Service Innovation | Rational
![Page 30: TDC 2013 7 Dicas para acelerar os testes](https://reader035.vdocuments.net/reader035/viewer/2022062513/5571c932d8b42a3f5e8b47f0/html5/thumbnails/30.jpg)
© 2012 IBM Corporation
Enabling Product and Service Innovation | Rational
![Page 31: TDC 2013 7 Dicas para acelerar os testes](https://reader035.vdocuments.net/reader035/viewer/2022062513/5571c932d8b42a3f5e8b47f0/html5/thumbnails/31.jpg)
© 2012 IBM Corporation
Enabling Product and Service Innovation | Rational
![Page 32: TDC 2013 7 Dicas para acelerar os testes](https://reader035.vdocuments.net/reader035/viewer/2022062513/5571c932d8b42a3f5e8b47f0/html5/thumbnails/32.jpg)
© 2012 IBM Corporation
Enabling Product and Service Innovation | Rational
Resumo
1. Dê foco aos testes
2. Antecipe a execução dos testes
3. Versione tudo (release)
4. Cloud – Flexibilize os ambientes de testes
5. Virtualize os serviços e ambientes de testes
6. Automatize corretamente
7. Mobile
![Page 33: TDC 2013 7 Dicas para acelerar os testes](https://reader035.vdocuments.net/reader035/viewer/2022062513/5571c932d8b42a3f5e8b47f0/html5/thumbnails/33.jpg)
© 2012 IBM Corporation
Enabling Product and Service Innovation | Rational
Collaborative Lifecycle Management
Delivery PipelineDelivery Pipeline
OSLC
ConfigurationAutomationConfigurationAutomation
OSLC
Continuous IntegrationContinuous Integration
OSLC
Quality ManagementQuality Management
OSLC
Provisioning AutomationProvisioning Automation
OSLC
Cloud
Pull configurations
Track work
Trigger deployment Deploy
service
Track quality
Track work
Trigger Delivery
Update configurations
Pull changes
Continuous Delivery
Change ManagementChange Management
OSLC
OSLC
Requirement ManagementRequirement Management
Trigger tests
Service Simulation & Testing
Service Simulation & Testing
OSLC
Configure endpoints
Start stubs
Application MonitoringApplication Monitoring
OSLC
Monitor application
Configure agents
Link defect
Track incident
Incident ManagementIncident Management
OSLC
Deployment DesignDeployment Design
OSLC
Generate automation
![Page 34: TDC 2013 7 Dicas para acelerar os testes](https://reader035.vdocuments.net/reader035/viewer/2022062513/5571c932d8b42a3f5e8b47f0/html5/thumbnails/34.jpg)
© 2012 IBM Corporation
Enabling Product and Service Innovation | Rational
Links
Jazz.Net
6 Ways for Enterprises to Adopt DevOps blog – http://ibm.co/xq71xY
Understanding and Adopting DevOps (Series on my Blog) – http://bit.ly/MyDevOps
Blog da Rational Brasil – O Mundo depende de software
– https://www.ibm.com/developerworks/mydeveloperworks/blogs/rationalbrasil/
Dorothy Graham Independent Test Consultant - www.DorothyGraham.co.uk
Felipe Freire (@pfelipebr)[email protected]