alm 4 azure with screenshots
DESCRIPTION
Presentation covering ALM TFS2010 MTM for Azure Cloud Development given at the 3e Windows Azure User Group Meeting , 26 mei 2011TRANSCRIPT
![Page 1: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/1.jpg)
1
26 mei 2011
ALM 4 Azure T F S 2 0 1 0 f o r A z u r e C l o u d D e v e l o p m e n t
![Page 2: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/2.jpg)
• Sogeti | Management Consultant
www.ClemensReijnen.nl
@ClemensReijnen [email protected]
Clemens Reijnen
2
![Page 3: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/3.jpg)
work together __with work packages __on alm artifacts
3
![Page 4: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/4.jpg)
http://msdn.microsoft.com/en-us/library/bb385832.aspx
4
![Page 5: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/5.jpg)
ALM 4 Azure Goals: Collaborative
High Quality Flexible Automated Repeatable Efficient C
lou
d S
yste
ms
5
![Page 6: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/6.jpg)
Scenario 1: Dev only Scenario 2: Dev and Test Scenario 3: Dev, Test and Build, Deploy Scenario 4: Dev, Auto Tests and Build, Deploy Scenario 5: Dev, Auto Tests, Build, Deploy, Acc test and Operations
6
![Page 7: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/7.jpg)
Scenario 1: Dev only Scenario 2: Dev and Test Scenario 3: Dev, Test and Build, Deploy Scenario 4: Dev, Auto Tests and Build, Deploy Scenario 5: Dev, Auto Tests, Build, Deploy, Acc test and Operations
7
![Page 8: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/8.jpg)
Scenario 1: Developer only
Staging Production
Developers implementeren het systeem Local unittesting
TEAM
Source Repository
(TFS) C# (implementatie + unittests)
‘SprintReview’Build Compile Run Unit Tests
1
2
3
8
![Page 9: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/9.jpg)
dry run – local storage and compute
![Page 10: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/10.jpg)
10
test run – cloud storage and local compute
![Page 11: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/11.jpg)
11
debug – Intellitrace deploy
![Page 12: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/12.jpg)
12
![Page 13: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/13.jpg)
13
debug – Get Intellitrace file
![Page 14: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/14.jpg)
14
debug – Find exception
![Page 15: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/15.jpg)
15
debug
![Page 16: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/16.jpg)
16
debug – send to peer
![Page 17: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/17.jpg)
Scenario 1: Developer only
Pro: Easy installation and configuration
Single click deployment from VS2010
Con: No collaboration Easy deployment errors (configuration) What about test and ops
17
![Page 18: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/18.jpg)
Scenario 1: Dev only Scenario 2: Dev and Test Scenario 3: Dev, Test and Build, Deploy Scenario 4: Dev, Auto Tests and Build, Deploy Scenario 5: Dev, Auto Tests, Build, Deploy, Acc test and Operations
18
![Page 19: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/19.jpg)
Scenario 2: Developer with manual tester
Staging Production
Developers implementeren het systeem Local unittesting
Testers specificeren testcases “Dry” run van tests tegen Development Fabric
TEAM
Source Repository
(TFS) C# (implementatie + unittests)
‘SprintReview’Build Compile Run Unit Tests
1
2
3
4
19
![Page 20: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/20.jpg)
![Page 21: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/21.jpg)
Microsoft Test Manager
21
![Page 22: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/22.jpg)
Staging Production
TEAM
Source Repository
(TFS)
1
2
3
4
Functional, system, manual tests executed in compute emulator
with MTM.
Build Verification Tests (unit tests) and other quality gates,
executed in the build.
Platform accepatance tests executed on
Azure staging - tested with MTM.
1 2 4
![Page 23: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/23.jpg)
23
Microsoft Test Manager – Specify Test Cases
![Page 24: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/24.jpg)
24
Link Requirements to Tasks and Test Cases
![Page 25: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/25.jpg)
25
Specify test case
![Page 26: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/26.jpg)
26
Test with or without debugging
![Page 27: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/27.jpg)
27
without Visual Studio
![Page 28: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/28.jpg)
4
2
3
Get the CSX folder and CSFG file 1
1
![Page 29: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/29.jpg)
Challenges with changing URL’s
![Page 30: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/30.jpg)
30
Shared Steps 2 the rescue
![Page 31: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/31.jpg)
31
1 Shared Step : open application
![Page 32: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/32.jpg)
32
1 Shared Step : used by all test cases
![Page 33: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/33.jpg)
33
Fast Forward
![Page 34: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/34.jpg)
Diagnostic Data Adapters
![Page 35: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/35.jpg)
35
Capture information during test execution
![Page 36: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/36.jpg)
Create your own adapter.
![Page 37: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/37.jpg)
37
For example: WAD
![Page 38: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/38.jpg)
38
Or Intellitrace Files
![Page 39: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/39.jpg)
Capabilities • Link test cases to requirements • Fast Forward • Test data Iterations • Test case management • Test configurations
Interesting capabilities 4 Azure • Shared steps ( 4 VIPS Swap )
– common practice: new hosted service only change the name in the DNS…
• Diagnostic adapters ( Rich bug reports ) capturing the logs
Not working capabilities in Azure • Test Impact 4 manual tests • IntelliTrace 4 manual tests (although possible custom adapter)
39
![Page 40: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/40.jpg)
Scenario 2: Developer with manual tester
Pro: Easy installation and configuration
Single click deployment from VS2010 Testers connected, same heartbeat as dev
Proven quality
Con: Easy deployment errors (configuration) Time consuming (deploy and test) Not repeatable (annoyed testers) Testers connected
40
![Page 41: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/41.jpg)
Scenario 1: Dev only Scenario 2: Dev and Test Scenario 3: Dev, Test and Build, Deploy Scenario 4: Dev, Auto Tests and Build, Deploy Scenario 5: Dev, Auto Tests, Build, Deploy, Acc test and Operations
41
![Page 42: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/42.jpg)
Scenario 3: Developer with manual tester and deployment build
Staging Production
Developers implementeren het systeem Local unittesting
Testers specificeren testcases Dry run van tests tegen Development Fabric
TEAM
Source Repository
(TFS) C# (implementatie + unittests)
‘SprintReview’Build Compile Run Unit Tests
Package (CSRun.exe, CSPack,exe, CSManage.exe) Deploy 2 Staging (CMdLets, Powershell)
1
2
3
4
42
![Page 43: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/43.jpg)
Management API CSPack CSRun CSManage PowerShell CmdLets Build Targets
Automatic Deployment
http://scottdensmore.typepad.com/blog/2010/03/azure-deployment-for-your-build-server.html http://blogs.msdn.com/b/tomholl/archive/2011/02/23/using-msbuild-to-deploy-to-multiple-windows-azure-environments.aspx
43
![Page 44: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/44.jpg)
44
Change .ccproj project
![Page 45: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/45.jpg)
45
Use PowerShell script
CSPack used in Target file
![Page 46: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/46.jpg)
Configure build definition
![Page 47: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/47.jpg)
1. Change .ccproj project • New build target which uses the built-in CorePublish target
2. Create PowerShell script • Called by build target and uses Azure CMDLets
3. Edit Build Config • Add the necessary parameters
4. Do something with certificates • Put the Azure certificate on the build server
Deploy during build
47
![Page 48: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/48.jpg)
Pro: Easy installation and configuration
No click deployment from build Repeatable ‘proven’ deployments*
Testers connected, same heartbeat as dev Proven quality
*demo needs some additional environment tweaks for different csfg files
per environment
Con: Time consuming testing Application can contain ‘annoying’ bugs Build workflow knowledge necessary Powershell, ccproj tweaks, target files, certificates
Scenario 3: Developer with manual tester and deployment build
48
![Page 49: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/49.jpg)
Scenario 1: Dev only Scenario 2: Dev and Test Scenario 3: Dev, Test and Build, Deploy Scenario 4: Dev, Auto Tests and Build, Deploy Scenario 5: Dev, Auto Tests, Build, Deploy, Acc test and Operations
49
![Page 50: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/50.jpg)
Scenario 4: Developer with automated regresion tests, manual tests and deployment build
Staging Production
Developers implementeren het systeem Local unittesting
Testers specificeren testcases Dry run van tests tegen Development Fabric Automation van testcases
(moeten gedraaide testcases zijn voor CodedUI) UI automation tests run tegen Development Fabric
en associëren met test plan
TEAM
Source Repository
(TFS) C# (implementatie + unittests) Test cases (automated UI tests)
‘SprintReview’Build Compile Run Unit Tests
Package (CSRun.exe, CSPack,exe, CSManage.exe) Deploy 2 Staging (CMdLets, Powershell)
1
2
3
4
5
50
![Page 51: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/51.jpg)
CodedUI
![Page 52: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/52.jpg)
52
![Page 53: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/53.jpg)
53
Different URL’s
1
2
![Page 54: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/54.jpg)
54
UIMap Editor
![Page 55: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/55.jpg)
55
Execute from MTM
Execute from VS
![Page 56: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/56.jpg)
56
Execute from Build
Execute from Test Agent
![Page 57: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/57.jpg)
57
Automated Test Settings
![Page 58: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/58.jpg)
a) At the Build server? (no … only unit test)
b) In the Development environment? (only dry run for autoamtion)
c) In the Test environment? (sure, functional tests.. You need emulator at the test env)
d) In the Azure environment? Some definitely not all (money)
Where to run the auto test?
58
![Page 59: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/59.jpg)
Pro: No click deployment from build
Repeatable ‘proven’ deployments* Testers connected, same heartbeat as dev
Proven quality Automated BVT on different Environments
Comfortable Acc Testing Done Done
*demo needs some additional environment tweaks for different csfg files
per environment
Con: Build workflow knowledge necessary Powershell, ccproj tweaks, target files, Certificates Test Infrastructure knowledge necessary A balanced thinking of test automation
Scenario 4: Developer with automated regresion tests, manual tests and deployment build
59
![Page 60: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/60.jpg)
Scenario 1: Dev only Scenario 2: Dev and Test Scenario 3: Dev, Test and Build, Deploy Scenario 4: Dev, Auto Tests and Build, Deploy Scenario 5: Dev, Auto Tests, Build, Deploy, Acc test and Operations
60
![Page 61: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/61.jpg)
Staging Production
Developers implement the system Local unit testing
Testers specify test cases Dry run tests cases against Development Fabric Automation of test cases
UI automation tests execute against Development
Fabric and associate with a test case Operations determine instrumentation data DFO, SLA Usages Metrics, payment
‘Release ’Drop Package (same as test env
package) Acc testing met MTM en
Diagnostic Data Collectors
Automated ALM for Azure Cloud development Efficient, repeatable with proven quality fast flexible delivering of useful functionality … Making optimal use of the characteristics of Azure: - Identical environments (Guest OS versioning). - Separate in depended staging production environments. - Easy up and downgrade of applications. - Unambiguous deployment mechanism - rich diagnostic capabilities - Simplified Application Deployment & Management - …
TEAM
BUILD (TFS)
Acceptation
C# (implementation + unittests) Test cases (automated UI tests)
‘SprintReview’Build Compile Run Unit Tests Package (CSRun.exe, CSPack,exe, CSManage.exe) Deploy 2 Staging (CMdLets, Powershell) Run automated Test Suites (MTC.exe) with different
environments (test controller – agents infrastructure) Collect and publish test results “Build Verification Tests” green -> deployment 2 ‘test
'production
1 2
3
4
61
App fabric Application Manager (June)
![Page 62: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/62.jpg)
Scenario 1: Dev only Scenario 2: Dev and Test Scenario 3: Dev, Test and Build, Deploy Scenario 4: Dev, Auto Tests and Build, Deploy Scenario 5: Dev, Auto Tests, Build, Deploy, Acc test and Operations Scenario 6: TFS on Azure Scenario 7: Other Infrastructure on Azure
62
![Page 63: Alm 4 Azure with screenshots](https://reader033.vdocuments.net/reader033/viewer/2022060109/55526cd6b4c905d41d8b5180/html5/thumbnails/63.jpg)
63