innovations in data center power: software-defined power ... · homogenous, virtualized...

In the field of data center power, software tools play many roles, including data collection, analyzing power quality, switching loads, tracking long-term energy use and compliance, and, coming soon, arbitrage and billing for external power sources. As these tools become more integrated, and more intelligent, their power, and ultimately their adoption, will increase.

The introduction of new terms and technologies, along with the usual marketing spin, has created some confusion in this area. Software-defined power, like the broader software-defined data center, is about creating a layer of abstraction that makes it easier to continuously match power resources with changing data center needs.

Significant cost savings can be achieved by treating power as a virtual resource that is managed through a control plane, and that can be controlled, capped, used, stored, or even sold to meet changing demands, service levels, or policies.

Innovations in data center power: Software-defined power and energy optimizationFOCUS | MARCH 2018By Andy Lawrence, Executive Director, Research, Uptime Institute, and Rhonda Ascierto, Research Director, Data Centers and Critical Infrastructure, 451 Research

EMERGINGTECHNOLOGIES

Across the industry, integrated and responsive software is enabling greater efficiency and reliability, as well as lower cost of ownership. This will eventually be true for data center power infrastructure. If power management and control software can help to guarantee the same level of availability or ‘apparent redundancy’ for mission-critical applications as can be achieved by directly using hard-wired physical equipment, then the argument for software-defined power approaches is clear–lower capex without additional risk, and much greater flexibility in how power is directed, managed, and deployed. While the relative immaturity of many of the component technologies and the complexity of software-defined power (and confusion over various related technologies) will likely slow adoption, as will a commitment to existing investments and cultural factors, software-defined power is an approach that the data center industry will likely embrace eventually. Some jumps in imagination and a willingness to try different approaches will be needed, and for this reason adoption of these newer approaches will probably be seen at the extremes of the market–small, distributed datacenters, and huge, hyperscale plants.

Over time, the thinking about consistently operating data centers at reduced peak loads will change, and the case for software-defined power will be more compelling. If data centers operated at higher levels of utilization (close to their design peak) using software tools, then data centers could, for the work they do, be smaller and cheaper.

INSIDE VIEW: Powerful technology, but economics and architecture will slow adoption

Clarification of termsThe management of power by software is not new. Indeed, it lies at the very root of all IT and of all data center technologies. But holistic, integrated, management-level systems are relatively new, and there is some confusion over the terminology and application. The following descriptions will help explain and perhaps disambiguate some of the terms:

Data center energy management. This is a generic term usually used to describe the monitoring, management, reporting, and forecasting of energy consumption in the data center at different levels. This term is mostly used to describe application areas where more generic energy management tools can be applied (for example, from building energy management).

Data center power management. Data center power management is a term that can have different meanings. It is often used by electrical equipment providers to refer specifically to tools that manage power quality–to track voltage drift, drop offs, harmonics, and surges, for example. These functions are likely to be carried out by an EPMS (electric power management systems) product from suppliers such as Schneider, Eaton, Emerson/Vertiv, and many others. In a software management and DCIM context, data center power management is most likely to be used to describe the monitoring of power use, especially where it is possible to report and analyze data, such as use against capacity or ongoing costs.

Related to this, power analytics is also sometimes used as a category to describe how power use behavior can be modeled against an ideal performance model to identify faults or configuration errors. IT power management refers specifically to the management of IT energy consumption, through the control of sleep states, or using techniques such as power capping (controlling frequencies or voltages).

Software-defined power. Software-defined power, like the broader software-defined data center (SDDC), is about creating a layer of abstraction that makes it easier to continuously match resources with changing needs. The term software-defined power was first applied by the (now defunct) software company Power Assure in mid-2013. It took up the idea of the software-defined data center (itself a term introduced by VMWare in 2012), as way of describing how, at a virtualized level, resources such as compute power, servers, storage, and networking are matched to application service levels. For software-defined power, the resource is the electricity required to power (and cool) all of that equipment. Most recently, the term has been adopted by Virtual Power Systems, which combines a software management system with a distributed in-row Lithium Ion (Li-Ion) battery system, in order to intelligently match power with load. VPS now owns the trademark on the term SDP, although it allows it to be used by others in context.

Innovations in data center power: Software-defined power and energy optimization


Data center energy optimization (DCEO). In 2014, 451 Research introduced its own term, DCEO. DCEO is part of data center service optimization and describes a set of software tools that leverage DCIM data to intelligently manage the supply, distribution, storage, and use of energy in the data center (451 Research, 2014).

DCEO in its full scope extends beyond most definitions of energy management, because it positions energy optimization as a control and integration point. DCEO envisages the development of applications that reach both into the IT itself–to pick up data about applications, virtual machines, and power utilization–and to reach back into the grid, which can be seen both as a source of power and as a customer for power when economics or circumstances require. In this way, it may be possible to participate in demand/response schemes, for example, or use more grid power when it is most expensive. Arguably, DCEO and SDP are similar.

Data center microgrid. Recently, data centers have become increasingly associated with microgrids. Microgrids have been defined by Berkeley Labs as a localized group of electricity sources and loads that are synchronized with the traditional grid (macrogrid). But this local grid can disconnect and function alone from the rest of the grid as conditions dictate. At a simple level, it may be argued that all data centers that can operate without the grid are microgrids.

Microgrids use software and intelligence, but the fundamental role they undertake is electrical source and load management, much of which is embedded in the hardware and associated controls. They must manage the switchover between different sources (such as wind, solar, fuel cell, and grid), transforming and rectifying current, transferring loads, and synchronizing voltages and frequencies.

Most microgrids are designed or deployed for campuses, self-contained industrial developments, or in places (such as small islands) where a small grid is designed in place of a large-scale power grid. But there is some convergence occurring with data center energy management, both inside large data centers, which are in effect power hungry campuses, and also in campuses and smart cities, where the huge and largely stable load of a data center can help to smooth out uneven demand and supply and justify investments in microgrid technology. Companies such as ABB, Schneider Electric, Emerson Network Power, and Eaton are all investing in this area, seeing an opportunity to provide critical equipment and add controls systems to optimize power loads and sources. Along with this, the energy management software is also evolving rapidly.



From monitoring to control

What do all these systems and terms have in common? Is there an overlap between them or are they something different?

Setting aside the real-time power quality and switching tools, in our analysis, all of the systems can do the following: collect energy use data from lots of sources (either in real time or with high frequency), provide centralized management, and produce reports and analytics. Some can do real-time analysis of power quality.

Suppliers will often point out that these tasks are far from trivial, especially as they require aggregation, normalization, and considerable protocol management or conversion to speak to many devices, and possibly integration with on-board server power monitoring. If real-time monitoring is required, the problem becomes yet more complex, because if the data is to produce meaningful reporting, then scalability and synchronization become an issue.

Even so, this level of energy and power monitoring is becoming very common and eventually will be, if not commoditized, then fairly routine and embedded in multiple systems.

Many suppliers who we would put in the DCEO and/or software-defined power category can do, or aspire to do, much more. This may include the ability to:

• Interact or exchange data or power with grid sources

• Exchange dynamic pricing and transact with grid or other power sources

• Reach into servers for real-time system monitoring, including VM and application use.

• Change or control power sources based on policy, thresholds, and more.

• Change, move, or control power loads at data center, rack, server, or application level, based on policy, thresholds and more.

• Integrate power sources, switch sources, normalize voltage, currents, etc. under single management (microgrid functions).



The products, technologies, and methods to achieve all these types of functions and the integration required are far from trivial. These are areas where people use competing or ill- defined terminology and where there is complexity around costs, benefits, and outcomes. The ultimate goal is ambitious: to enable data centers to not only use energy efficiently, but also to allocate and price it according to true costs and demand and to make real-time, and possibly automated, policy decisions about how this is done.

What is software-defined power?Most software-defined power product development to date has focused on IT power management. The value proposition is simple and, seemingly, compelling. Servers, even modern ones, draw power, often a lot, including when they aren’t doing much. Considerable savings can be made, especially at scale, by identifying when servers are doing nothing and putting them into low-power states or turning them off.

If the software can also move and consolidate (usually virtualized) workloads, even bigger savings can be made by moving the loads onto fewer servers and putting the rest to sleep (an idea first promoted by another defunct startup, Cassart, in 2007–but at that time, few companies had homogenous, virtualized infrastructure). Given that the energy use of the IT and the supporting mechanical/electrical infrastructure accounts for as much as 40% of data center operational costs, this type of active power management has always seemed at least worth considering–provided that infrastructure capital savings can be achieved. A variant of this is that some DCIM suppliers enable customers to automatically move workloads to public clouds or colocation data centers, based on predetermined power thresholds (matched to the criticality level of the workload).

Power capping, a technique used by some power management tools, is less straightforward in its use cases. Using frequency or voltage controls, the technique can be used to hold down power consumption at times of high consumption or to prevent breaches of known power capacity limits. It, too, has failed to catch on so far, despite mature products from suppliers such as IBM, HPE and Dell, partly because IT staffs don’t want to buy powerful equipment only to prevent it from working to its full performance.Beyond the rackWhile most focus to date has been on IT power management, software-defined power spreads beyond the IT and the rack. Some suppliers are taking the technology further, using the intelligence for switching and aggregating power in the data center on demand. Vertiv, for example, is working with customers to enable a move from a 2N architecture for uninterruptible power supplies (UPS) to lower N+1 architectures, while still maintaining high levels of redundancy



for workloads that are critical. This is achieved by sharing capacity between power sources under software control.

The primary benefit of a 2N UPS is that it provides a dedicated reserve of power when needed, but at the price of low utilization (50%). Traditional N+1 architectures will offer spare capacity to all the UPSs at lower capex by adding a spare system in case any one unit fails. But this has its limitations–the coverage that one or two spare units provides decreases progressively as the number of UPS systems increases.

Although the risk of more than one unit failing may be small, there will be many situations where the risk is heightened–during maintenance, for example. A software-defined N+1 or N+2 system configuration promises similar capex to N+1 or N+2, but with high power utilization and relatively higher redundancy. In a failure or maintenance situation, transfer switches move the load between the UPSs, exploiting the fact that none will likely be running at capacity. Effectively, they create a pool in a way that is similar to a pool of storage or processing in a software-defined data center configuration. In this example, software monitors the availability of each reserve UPS and coordinates the transfers of loads to them. Because of the way suppliers are implementing the crossover of the transfer switches, it is creating a redundant/Concurrently Maintainable architecture, while having (in most cases) less than 200% UPS coverage (this approach is sometimes called 2N+1).

This type of approach replaces spare/overprovisioning at the infrastructure level with software and switching to manage demand/supply dynamically. It calls for constant balancing/monitoring, and management software that can reliably take action in real time. This is expected in tightly engineered proprietary systems, but becomes more challenging where multiple sources of power and demand need to be balanced. A future direction is for all sources of power to be called upon where needed or load shed in a controlled way if needed.

This takes data centers toward microgrid-type technology. Software controls can be used to incorporate standby power from generators or on-site power-generation sources, or distributed batteries/UPSs. There are many examples of this emerging.

Methode Electronics, for example, works with supplier VPS to enable peak power shaving using intelligent lithium-ion batteries (which, unlike traditional lead-acid batteries, can be discharged and recharged hundreds to thousands of times without significantly impacting the battery capacity or life). A handful of data centers are already supplementing their power with renewable sources–in a software-defined power architecture, smaller increments of power can be critical at key times, especially when combined with new storage technologies (Li-Ion batteries, supercapacitors, etc.).



Beyond the data center walls, the value of software-defined power extends to transactive energy –utility providers, nearby businesses, or aggregators will buy, use, or distribute critical powerat certain times, or under certain conditions. Demand/response schemes, currently used by (we believe) no more than tens of–or perhaps one hundred–data center operators, will become more valuable, more flexible, and less risky as software-defined power architectures are introduced.

Drivers for adoptionThe early adopters of software-defined power/microgrids are most likely to be data centers with high-density racks, particularly those that handle high-performance computing (scientific research, complex graphical rendering, etc.). For these operators, scheduling certain (possibly batch) workloads to run during nonpeak business hours can delay or reduce the need for data center capacity expansion.

However, this technology has big benefits for all operators. The move from 2N electrical architectures to software-defined N+1 is being driven by enterprises (and others) seeking lower costs by driving up utilization (and lowering the capex of backup power equipment). Colocation companies, meanwhile, are able to spread costs and risks, increasing margins or reducing prices/risks for clients.

For enterprises, in particular, being able to segment and prioritize redundancy levels for specific workloads makes lighter physical redundancies more attractive. They can apply software-driven controls more liberally for workloads that are less critical, and with very strict controls on those that are highly critical. The aim is to drive higher levels of utilization, which means facilities can be designed and operated with less overprovisioning (for rare but expected spikes in workload volume), and can instead build smaller data centers and plan to operate closer to peak capacity.

Barriers to adoptionFor many situations, the key to success of software-defined power is prioritizing the right mix of workloads, which requires establishing a hierarchy of criticality–something that most data centers don’t necessarily have the visibility, tools, or processes to do. Data from several tools and systems must be integrated (typically from DCIM, ITSM, and virtual machine management, at a minimum), which often requires facilities and IT departments to work closely together. Interdisciplinary business issues can also arise.

The complexity of software-defined power is also a deterrent. The tolerance to electrical failure is sub-milliseconds, which means that policy engines and software controls (and their correlating hardware) must all have extremely fast–and reliable–response rates. Interoperability between different systems can add considerable complexity.



Suppliers will need to do a better job of hiding and embedding some of these issues, including within control systems, DCIM software and cloud-delivered management offerings that can be easily turned on once a data center is fully instrumented, such as DMaaS.

The return on investment for software-defined power can also be unclear in regions where energy prices are low or where energy regulatory environments are unclear or uneven. Quantifying savings in lowered capex on servers, or on power infrastructure, can also be tricky, since server makers continue to deliver improvements in throughput per kilowatt with each new generation. Over time, we believe these issues will diminish (driven by the ongoing scrutiny of data center costs) and that software-defined power will become a foundation for the next generation of highly efficient, responsive data centers.



innovations in data center power: software-defined power ... · homogenous, virtualized...

Documents