practical management of development & qa environments for sharepoint 2013
DESCRIPTION
Speakers: Ognyan Guglev & Radi Atanassov In this session we will share how we maintain our environments for development, quality assurance and demonstration purposes. We've put in a lot of thought into optimising what we do and to deliver a highly-available, performing experience to our delivery teams. For our work we have over 90 farms, so the challenges in maintaining them are not insignificant. Due to advances in the SharePoint platform we believe it is becoming increasingly difficult to maintain SharePoint environments for every project, client or product. We have a strong requirement to be flexible and efficient on hardware and at the same time be able to spawn development environments on demand. Automation here with SCVMM is key to a sustainable work front. We will discuss our goals as a consultancy company, how we deal with licenses, whether we prefer centralised or decentralised team environments, how to automate VM's with Service Center Virtual Machine Manager, how to deal with Microsoft SQL and Active Directory, DNS and IP addresses, what we do to make the developer's time as productive as possible and a whole set of other tips and tricks we put in place. We will also share our Apps development and Office 365 development landscapes. Overall, this session is infrastructure focused, but will be valuable and practical both for administrators and developers, it will cover experiences for both sides of the spectrum.TRANSCRIPT
Radi Atanassov & Ognyan Guglev
OneBit Software
Practical management of development & QA
environments for SharePoint 2013
For Developers
About Us
• SharePoint
2010 MCM
• SharePoint
Server MVP
• OneBit
Software
• Web
Platform
User Group
Radi Atanassow Ognyan Guglev
• SharePoint
2010 Administrator
• 2008 Server
Enterprise
Administrator
• 2010 Enterprise
Exchange
Administrator
• OneBit
Software
Agenda
The needs of Dev and QA teams
Challenges from the perspective of IT
Deployment options
Cloning vs Provisioning
Licensing
PowerShell automation
SCVMM
Performance& Hardware
Tips, Tricks & Recommendations
DEFINE THE BUSINESS NEEDS
AND IT CHALLENGES
SP Developers and QA Teams
They need servers:
SharePoint Server
Office Web Apps Server
Workflow Manager Server
Server management requires knowledge
and time…
QA needs a little bit more:
Different environments
Automated deployment & tests
Defining the strategy
1 per project vs 1 for
many projects
1 for each version of
SharePoint
Developers?
Team Leaders?
QA Team?
IT?
How many servers? Who creates servers?
Single VM
Multiple VM’s
Retention & IT Governance
Too many VM’s could become a key concern
When should you delete/archive the virtual
machines?
Who should delete/archive virtual machines?
Responsibilities of IT
Be responsible for the “high-availability” of the
development & QA teams
Provide overall management of these needs
Creation, retention, networking, storage,
performance, automation
Manage the hardware
Environment Goals
Save developer time!
Provision development environments in under 3 hours
Automating repetitive tasks (installation, configuration, tools, code repositories, shortcuts)
Make the developer experience the same
Avoid data loss (thin VM’s)
Avoid restoring broken environments
The best developer performance
Remote work
Managing Environments
INFRASTRUCTURE
CONSIDERATIONS
Infrastructure Considerations
Windows Server
Active Directory Domain Services
Microsoft SQL Server
SharePoint
Visual Studio & Source Control
Workflow Manager?
Office Web Apps Server?
Client test machine (Office, browsers)
Deployment Options - Desktops
Deployment Options - Centralised
Deployment Options - Mixed
Deployment Options - Shared
Workflow Manager 2013
Office Web Apps Server 2013
Active Directory
Deployment Options - Shared
Workflow Manager 2013
Office Web Apps Server 2013
Active Directory
SQL Server?
Deployment Options - Cloud
Comparisons
IT manages the
server infrastructure
Saves developer/QA
time
Group Policy
1 IT can service
many teams
Server performance
Workstations for
more power
Dev can manage
their VM’s
(performance,
retention, network)
Better for snapshot
if services are not
shared
Centralised Decentralised
Comparisons
Requires lots of
resources
Easier to use, self-
contained
Easier to move
Can snapshot
No central
management
Better performing
dev machines
Harder to move &
has dependencies
Must snapshot the
entire team
Centralised
management
Complete VM (AD, SQL…) Shared Services
In detail – Shared AD
No need to create domain
every time you install a new
SharePoint machine
No need to create accounts
and policies for every
machine
Can access servers by FQDN
Centralized management of
servers
Ease of setting policies, file
shares, software installation,
software updates
Cannot move the server outside the
domain
Not recommended to "play around"
with the domain setting and make
changes
Not recommended to change
passwords to user accounts - it can
impact many machines
Cannot create/revert snapshots as
this will impact the time
synchronization between machines
Developers need to request or add
entries in the DNS server
Cannot clone VMs (required sysprep
& rejoin then fix SP)
PROs CONs
Can move the machine
anywhere as it holds all the roles
Can make development against
the domain and make some
changes without impacting the
other machines
Unique control on accounts and
passwords
Can snapshot and revert
No need to create hosts file
entries to develop against a
demo website with fixed host
header (use DNS!)
Hard to implement FQDN resolution
Hard to implement and deploy policies on all machines
No centralisedmanagement of all servers (out-of-the-box)
Hard to automate software, file shares and other updates
Always have to consider the netbios name when provisioning new domain/forest
PROs CONs
In detail – AD in a complete VM
Workflow Manager
Must be part of the
same Forest
Better VM
performance
IT manages
certificates
Extremely difficult to
install correctly
(dev’s usually don’t
know)
Needs its own VM
Centralised Decentralised
Office Web Apps
Must be part of the
same Forest
Better VM
performance
Requires it’s own VM! Cannot be installed on SharePoint
Cannot be installed on a Domain Controller
Difficult to install correctly (dev’s usually don’t know)
Slows down environment
Centralised Decentralised
Single Workstation
Option 1
VM 1 – AD, SQL, SP
VM 2 – OWA, WFM
Option 2
VM 1 – AD, SP
VM 2 – SQL, OWA, WFM
Key Decisions
DNS, DHCP and other network considerations
Cross-service Integration
Performance
Managing VMs& Policies
Remote access
File shares for Dev Tools
Script & Component Library
Windows & SharePoint Updates
Visual Studio Updates
Do you clone your VM’s or
create many copies from
scratch?
Question
VM – Cloning
Easily copy a VM to
another location with all
the software installed
Fast solution when
another person need to join the project
Hard to manage &
identify copied VMs
which one is the right one
Same hostnames
(SP doesn’t like
renaming)
IP conflicts
Does not work with a centralised domain
PROs CONs
VM – Recreating
You have a clean
machine built from
scratch
No IP conflicts
New domain
New farm
Fresh SQL
Takes more time to
create
You have to install/copy all the tools and updates needed
You have to configure all the services
You have to assign permissions if needed
PROs CONs
Our approach
Automate provisioning & installation
Automate settings
Use templates
Our Recommended Practices
“Thin” workstations – no desktop/my documents Prevents data loss
Improves security
Loopback check
Stop Windows Updates
Add Shortcuts (14, 15 hive)
Windows priority– programs or background services
Limit logs
Minimize components & services
Avoid development& test with the administrator account
Environment PATH variables
Preinstall everything necessary
Stop IntelliTrace & tweak ReSharper
Stop unnecessary services like Antivirus & Defender
Our Recommended Practices
“Design/discuss” the environment together
with the Project Leader
Script Library
Component Library
Common Code
Base Solution Templates
Environment usability
Local administrator access
Ease-out domain policies
Development with the “Administrator” account
should be avoided
PowerShell execution policy
Background Info
Educate about shortcuts, network share, files
Remove unnecessary VMs
“data” inside the VM (desktop files)
Files not committed/checked in
Test/POC/Sample projects
Licensing
Windows Server– 180 days
SharePoint– Unlimited
SQL– Express or 180 days
Visual Studio– 60 days
Others…
Client demo/qa/stg environments
We sometimes host environments for money
Access from the internet
*.contoso.com //if needed
TFS Lab Manager: Automated deployment for
ALM requirements
Hardware recommendations for
VM’s
CPU - Configure a 1-to-1 mapping of virtual processor to logical processors for best performance - Be aware of over commitment
Memory - Ensure enough memory is allocated to each virtual machine - Be aware of NUMA
Disk - Be aware of underlying disk read/write contention between different virtual machines to their virtual hard disks. Use fixed disks, not dynamically expanding. Separate on different disks.
Integration - Ensure that integration components (HV/VM) are installed on the virtual machine
Time Synchronization – Be sure time is synchronized always with Domain Controller
Hardware Costs
CPU– Intel Core I7 4-core 3.4GHz HT
RAM– 16GB DDR3
HDD– 1TB
SSD – 128GB
Monitors
~1500 USD
CPU - E5-2620 6-
core 2.1 GHz HT
RAM – 64GB ECC
HDD – 4x1TB WD
BE RAID10
~2300 USD
Workstation Server
Environment Automation
Define what templates you need based on the projects
Sysprep VM templates
Group Policy Objects – Firewalls, RDP, Local Admins, File Shares, WSUS, Certificates, Homepages etc.
AD Accounts creation
DNS Forwarders
Install and configure SQL
Install and configure SharePoint
Install and configure Visual Studio
Other software and automation tasks
ULS, Fiddler, ILSpy, SP Designer etc.
Automating QA – TFS + Lab Manager
Testing Architecture
VM Template Library SCVMM Environment Standard Environment
VM Template 2VM Template 1
Test Agent Test Agent Test AgentTest AgentVMMAgent
Web Server
VMDatabase
Server
VM
VM Host Web Server Database Server
Test AgentTest Agent
Team Foundation Server
System Center Virtual Machine Manager (SCVMM)
Team Build
Test Controller
TFS Lab Manager
THIS IS HOW WE DO IT
Our choices & solutions
Environment mix:
1 environment per project per person
Team environment with dev VMs for each developer & shared services
Scripted deployment& installation
File share for dev tools, scripts, etc.
Distributed source control
Client demo environments
Separate SCVMM clouds for Dev/Staging and Production Environments
Dev/Staging in an isolated network
NON-DOMAIN MEMBERS
DNS ServerDomain:
onebitsoftware.net
2 way transitive trust with conditional forwarders
Other Production Servers
Domain:onebitsoftware.net Domain:onebitdev.test
Domain A - SharePoint Environment - 1 Tier
Domain B – SharePoint Environment - 3 Tier
\\DevTools
SP – Single ServerAD + SQL + SP + VS
SP-APP SP-WFE
AD+SQL
Workstation 1 Workstation 2 Workstation 3
Reverse Proxy
Http://sp.onebitsoftware.net
RDP or VMPortal
SP – Single ServerAD + SQL + SP + VS
Domain C - SharePoint Environment - 1 Tier
Hyper-V hosts
Other Dev Servers
Forwarders to the ISP DNS ServersGoogle DNS ServersRoot hints
Forwarders to the onebitdev.test DNS
Forwarders to the onebitdev.test DNS
Non-secure and secure dynamic updates
Forwarders to the onebitdev.test DNS
DNS ServerDomain: onebitdev.test
How
we d
o it
• SCVMM – How we use it
• Web Administration– Create ShrarePointDev Environment 1Tier
• VM Templates – Syspreped OSs
• Service Templates
• Review of PowerShell scripts– Variables (User input), Scripts (Functions and help), Email Notifications (show emails)
DEMO: What We Do – SCVMM +
PowerShell