SHOULD I FIRE MY TRADER OR PAY HIM A MILLION?Gary Karz and Wayne H. Wagner

ITG Solutions Network

As quantitative assessment of the equity trading process becomes widely accepted, the natural question arises whether the computed “quality scores” of firms, desks and individual traders should influence trader compensation.

Investment managers agree on the need fair, complete, and relevant tools to evaluate their performance. A Plexus Group survey updated in 2005, indicated that 65% of the firms surveyed considered trade cost analytics in determining trader bonuses, up from less than half two years earlier. While many variables and nuances are involved, the trend toward less subjective compensation systems will continue. This follows larger industry trends around quantifying investment process and performance.

Bonus policies reflect a firm’s philosophy on the role of individuals and teams. The structure of these policies should promote actions and behaviors critical to investment success. Bonus criteria should encourage accountability and a desire for improvement while rewarding successful performance.

Assuredly, trading is an important component of performance. Therefore trading goals and evaluations need to be related to contribution to investment performance. A fair and appropriate bonus system should combine measures of firm and fund performance with quantitative and qualitative measures of trader contribution.

The Plexus database confirms the importance of linking trader compensation to profitability and performance. The Value Added difference between a 25th percentile trader and the 75th percentile trader in a recent quarter (a sample of almost 400 traders) has narrowed to 23 bps. Assuming 100% turnover, the difference in annual portfolio returns between an average and a great trader would be 46 bps. With 200% turnover the difference between an average trader and a great trader approaches one percent, certainly large enough to capture the attention of a chief investment officer.

Let’s step into the mind of the trader. While some traders many be lazy, incompetent or corrupt,1 these are not issues we’re concerned with here, even though such traders are not easy to ferret out. We’re focusing on the human behavior of traders who are doing the best they can under the circumstances. All of their accumulated experience goes into each trade: how do I read the charts and recent activity, who do I trust, how big a trade can the market accommodate, is it better to trade now or to wait, should I use an alternative market, etc. ad infinitum. With this in mind, the competent and conscientious trader chooses a 1 These terms will be clarified later.

1

way to complete the trade that has the highest probability of achieving the best result. Surely, this meets the definition of best execution, whatever the outcome.

But we have a problem. How do we know that the trader is conscientious and competent? How does the trader know? We need some standards by which to judge. It’s not good enough for CIO’s and Compliance Officers to accept the trader’s own statement that he is a good trader. Portfolio managers, who live under the performance gun every day, would have little tolerance for such an assertion.

What makes for a good measure?

An optimal implementation strategy requires coordination of portfolio manager, trader, and broker roles. Thus any incentive policy must [a] align these disparate interests and [b] deal effectively with the fact that the responsibility for good trading performance is shared by manager, trader and broker.

We can identify three characteristics that would be desirable in any measure used to determine trader compensation:

Most importantly, the measure most correlate highly with the primary objective of delivering superior performance to clients.

The measure must not be gamable. A clever trader should not be able to trade in a manner that produces superior scores without satisfying the primary performance objective. Nor should the evaluated person be able to alter or edit trading data records without supervision.

The measure must create a level playing field for all traders measured. Differences in external factors affecting trader performance, such as soft dollar requirements, constraints placed by the portfolio manager, level of discretion available, work load, pairs trading, and many other complicating factors, may influence trading style, flexibility and recorded quality.

We will return to the initial requirements shortly, but let us first point out the implausibility of accounting for “all factors.” Three forces interplay: [1] the inability of the human mind to grasp any more than a few dimensions of complexity; [2] external constraints that affect the outcomes; and [3] the impossibility of categorizing and recording all the information necessary to accurately portray and contextualize every trading situation.

1. The typical trader’s primary lament about quantitative evaluation is that it oversimplifies the complexity of the task at hand. “What I do is more an art than a science.” The trader’s world is assuredly complex and fraught with randomness, uncertainty and downright deceit. Thus the trader cannot

2

understand everything yet must make decisions within a (very) limited timeframe. This situation, which Herbert Simon described as bounded rationality, is one in which trading decisions are not fully thought through and are rational only within limits such of time and cognitive capability. This is a factor to consider, not a criticism. Indeed, we should marvel at traders’ ability to make split-second decisions on incomplete or conflicting information.

2. An institutional equities trader typically sits between the portfolio manager(s) and the market. We can presume that the trader will be free to apply his best skills when trading judgment are unfettered by restrictive trade instructions. Unfortunately, we live in a world where portfolio manager interference and institutional constraints cast a long shadow. Consider soft dollars. When required in moderation, they can be satisfied with “no brainer” trades: moderate share amounts in liquid names while free of market imbalances effects. If the level of soft dollar commitments encroach into more difficult trading situations, they resulting constrictions will impinge on the trader’s option to chose another trading route or venue.

Are traders given complete discretion over trading speed, with the intent to minimize impact, or are the traders instructed to complete the order within a specific period of time? In the latter case the primary question is finding the liquidity and impact is secondary. If the traders are instructed to get trades done regardless of the impact, they should not be penalized for lagging a benchmark that incorporates the results of less encumbered traders. And if the realized returns are strong, the instruction to trade quickly is appropriate since improving portfolio returns should be the primary objective over minimizing trading costs.

This issue is especially relevant to firms with historically strong short-term returns – timely orders. Traders at firms with very timely orders might show costs near or worse than the benchmark, but this may in fact represent superior trading. These traders are fighting a head wind since they must trade quickly to benefit the portfolios. Traders working less timely orders can wait longer and spread the orders out over longer periods to reduce the impact without worrying about the stocks moving away.

3. Finally consider the problem of data acquisition. To illustrate, think about organizing an attempt to evaluate algorithmic trading. At the first level we might evaluate the offerings broker by broker. At a second level, we might identify which algorithm was applied. At a more detailed level, we might wish to record the settings of the parameters that control the algorithm. The more data we record, the more accurate the evaluation and the finer the distinctions that can be made. This data is increasingly being captured, and we shall undoubtedly see a number of interesting stories covering this area in the foreseeable future.

3

However, complete, accurate data recording is an onerous chore. Either this data is recorded automatically – not the common state of affairs – or it must be entered by the trader. Most good traders object to being saddled with this unpleasant task, so much detail is unrecorded

Each of these issues represents a complexity which makes us skeptical of creating a trader score for compensation purposes. In addition, each issue defines an area in which it would be difficult and expensive to derive a complete solution. Accordingly, our recommendation would be that these measures do not weigh heavily in trader evaluation. As a rule of thumb, we would suggest that no more than 10-25% of a traders compensation be based on the numbers. As an alternative, there could be no specific weight attached, but the trading quality reports are used as part of the raw material in deciding trader compensation.

With the exception of proprietary traders trading their own capital for profit, very few traders could be adequately evaluated on only one measure. In addition to trading, traders often participate in portfolio management, client servicing, marketing, compliance and even general management activities. Thus other factors must be considered when designing a trader compensation package. The survey disclosed that supervisor reviews, peer reviews, years of experience, and years with the firm are the most common factors used to determine salary. Firm profitability, supervisor reviews, and firm investment performance are the primary factors for bonuses.

The bottom line is that many firms find it worthwhile to partially base trader compensation on difficulty adjusted trade costs measures if:

1) It represents a relatively small percentage of compensation – not more than 25%, in our opinion.

2) The data on which the measurement is both reasonably clean and complete (including evaluation of trades delayed or never completed.)

3) The traders have the discretion to exercise judgment and make a contribution.

4) Returns from stock picking are not especially strong nor weak, which make it difficult to separate manager performance from trader contribution.

5) Order times and spacing are not unreasonable, such as orders clustered near the end of day or large buy/sell imbalances running counter to market trend.

What specific quantitative benchmarks are firms using?

The choice of a benchmark for use in trader evaluation demands an understanding of what behavior the firm wants to promote. For evaluating the desk and/or individual traders, we have seen firms using the following.

1. Value Added: Aggregate Trade Cost less the Benchmark (in b.p.). Gives credit for incurring costs less than expectations

4

2. Value Added: Aggregate Trade Cost less the Benchmark (in currency). Same as above, but accounts for activity level and growth in client assets.

3. Aggregate Value Added relative to prior period. Focuses on improvement, but each high water mark gets increasingly more difficult to duplicate.

4. Aggregate Value Added relative to peers. Evaluates by comparison to other traders in other firms performing similar tasks.

a. Versus Large/Small and/or Value/Growth sub-universes for the desk.

b. Versus evaluator-supplied trader universes and/or other traders at the firm for individual traders.

These quantitative factors are added to the organizational and reputational factors mentioned in the previous section.

Preventing Gaming

No system of compensation will achieve its primary goal if it is possible for those evaluated to game against the evaluation system rather than pursue the primary goal. Unfortunately in the case of securities transactions, there are many ways to game that are difficult for a naïve evaluation system to detect. Barr Rosenberg said this succinctly in a Berkeley Program in Finance meeting in 1985:

…we have to look at this measure as very beneficial for screening people who don’t know it’s being used to evaluate them…. [A]nyone who knew they were going to be evaluated by this measure would be some combination of dumb, impotent, or corrupt, depending on how they behaved. Dumb if they didn’t figure they could game it, impotent if they were incapable of gaming it and corrupt if they did….A fourth possibility [is] the saintly mode, where even though you know you can game this method you don’t game it. Barr Rosenberg, as recalled and recorded by Ted Aronson.

Some examples of how traders could game the system include: Gaming the VWAP by deferring trades . Traders can simply not trade until the

next day if they are behind the VWAP that day. Thus the bonus committee must review orders not traded and/or calculate timing costs from day to day.

Compensation based on trade costs versus some pre-trade standard potentially have the same problem. When traders know the price has moved against them they may not trade. If traders have to be 100% done by some

5

point it's less of an issue, but uncompleted trades which do not affect the quality of trading measurement are suspect.

Crushing the open . If traders are judged versus the open (or by implementation shortfall and the order is time stamped before the open), they will try to get as much done on the open as possible, perhaps pushing up their strike price in the process.

Banging the close . A very similar problem occurs when traders (or funds, especially index funds) try to eliminate tracking error by using MOC orders, which if the order are illiquid, we will see a jump on the close and then reversion the next day.

This is undoubtedly an incomplete list; creative traders search constantly for ways to find a special edge. Certainly we want clever traders on our trade desk but we want the skills focused on the main task of performance. Thus we recommend that only a minor portion of trader compensation should be tied to quantitative scoring.

Conclusion

Trading is an important contributor to investment performance, and the goals and aspirations of the traders need to be aligned with their contribution to superior performance. Trader base pay should reflect the level of experience and difficulty of the assignments. Well-designed trader bonus systems meld measures of fund performance with quantitative and qualitative measures of trader contribution.

The majority of investment firms partially base trader compensation on calculated assessments of trading quality, and the trend is unlikely to abate. Quantitative tools judiciously applied add a measure of objectivity to the process, allowing firms to identify the good traders and compensate them according to their contribution, while leaving the final determination open to the influence of team judgments and mitigating circumstances.

6