"Human beings can, and still do, send orders from their computers to the matching engines, but this accounts for less than half of all US share trading. The remainder is algorithmic: it results from share-trading computer programs. Some of these programs are used by big institutions such as mutual funds, pension funds and insurance companies, or by brokers acting on their behalf. The drawback of being big is that when you try to buy or sell a large block of shares, the order typically can’t be executed straightaway (if it’s a large order to buy, for example, it will usually exceed the number of sell orders in the matching engine that are close to the current market price), and if traders spot a large order that has been only partly executed they will change their own orders and their price quotes in order to exploit the knowledge. The result is what market participants call ‘slippage’: prices rise as you try to buy, and fall as you try to sell.
In an attempt to get around this problem, big institutions often use ‘execution algorithms’, which take large orders, break them up into smaller slices, and choose the size of those slices and the times at which they send them to the market in such a way as to minimise slippage. For example, ‘volume participation’ algorithms calculate the number of a company’s shares bought and sold in a given period – the previous minute, say – and then send in a slice of the institution’s overall order whose size is proportional to that number, the rationale being that there will be less slippage when markets are busy than when they are quiet. The most common execution algorithm, known as a volume-weighted average price or VWAP algorithm (it’s pronounced ‘veewap’), does its slicing in a slightly different way, using statistical data on the volumes of shares that have traded in the equivalent time periods on previous days. The clock-time periodicities found by Hasbrouck and Saar almost certainly result from the way VWAPs and other execution algorithms chop up time into intervals of fixed length.
‘Electronic market-making’ algorithms replicate what human market makers have always tried to do – continuously post a price at which they will sell a corporation’s shares and a lower price at which they will buy them, in the hope of earning the ‘spread’ between the two prices – but they revise prices as market conditions change far faster than any human being can. Their doing so is almost certainly the main component of the flood of orders and cancellations that follows even minor changes in supply and demand.
‘Statistical arbitrage’ algorithms search for transient disturbances in price patterns from which to profit. For example, the price of a corporation’s shares often seems to fluctuate around a relatively slow-moving average. A big order to buy will cause a short-term increase in price, and a sell order will lead to a temporary fall. Some statistical arbitrage algorithms simply calculate a moving average price; they buy if prices are more than a certain amount below it and sell if they are above it, thus betting on prices reverting to the average. More complicated algorithms search for disturbances in price patterns involving more than one company’s shares.
No one in the markets contests the legitimacy of electronic market making or statistical arbitrage. Far more controversial are algorithms that effectively prey on other algorithms. Some algorithms, for example, can detect the electronic signature of a big VWAP, a process called ‘algo-sniffing’. This can earn its owner substantial sums: if the VWAP is programmed to buy a particular corporation’s shares, the algo-sniffing program will buy those shares faster than the VWAP, then sell them to it at a profit. Algo-sniffing often makes users of VWAPs and other execution algorithms furious: they condemn it as unfair, and there is a growing business in adding ‘anti-gaming’ features to execution algorithms to make it harder to detect and exploit them. However, a New York broker I spoke to last October defended algo-sniffing:
I don’t look at it as in any way evil … I don’t think the guy who’s trying to hide the supply-demand imbalance [by using an execution algorithm] is any better a human being than the person trying to discover the true supply-demand. I don’t know why … someone who runs an algo-sniffing strategy is bad … he’s trying to discover the guy who has a million shares [to sell] and the price then should readjust to the fact that there’s a million shares to buy.
Whatever view one takes on its ethics, algo-sniffing is indisputably legal. More dubious in that respect is a set of strategies that seek deliberately to fool other algorithms. An example is ‘layering’ or ‘spoofing’. A spoofer might, for instance, buy a block of shares and then issue a large number of buy orders for the same shares at prices just fractions below the current market price. Other algorithms and human traders would then see far more orders to buy the shares in question than orders to sell them, and be likely to conclude that their price was going to rise.
Speeds are increasing all the time. In Hasbrouck and Saar’s data, which come from 2007 and 2008, the salient unit of trading time was still the millisecond, but that’s now beginning to seem almost leisurely: time is often now measured in microseconds (millionths of a second). The London Stock Exchange, for example, says that its Turquoise trading platform can now process an order in as little as 124 microseconds. Some market participants are already talking in terms of nanoseconds (billionths of a second), though that’s currently more marketing hype than technological reality.
Because the timescales of trading have changed, the significance of space has also altered. A few years ago, it was common to proclaim the ‘end of geography’ in financial markets, and it’s certainly true that if one is thinking in terms of hour-by-hour or even minute-by-minute market movements, it doesn’t really matter whether a trader is based in London, New York, Tokyo, Singapore or São Paulo. However, that’s not the case in high-frequency trading. Imagine, for example, that your office is in Chicago, the second largest financial centre in the US, and you want to trade on the New York Stock Exchange. You are around 800 miles away from the matching engines in Mahwah, and sending a message that distance, using the fastest fibre-optic route between Chicago and New Jersey that I know of, takes around 16 milliseconds. That’s a huge delay: you might as well be on the moon.
The solution is what’s called ‘colocation’: placing the computer systems on which your algorithms run next to the matching engines in data centres such as Mahwah. Colocation isn’t cheap – a single rack on which to place your server can cost you $10,000 a month, and it has become a big earner for exchanges and other electronic trading venues – but it’s utterly essential to high-frequency trading. Even the precise whereabouts of your computers within data centres is a matter of some sensitivity: you hear tales (possibly apocryphal) of traders gaining entry to centres and trying to have holes drilled in walls so that the route from their server to the matching engine is shorter.
The overall prices of US shares, and of the index futures contracts that are bets on those prices, fell by about 6 per cent in around five minutes, a fall of almost unprecedented rapidity (it’s typical for broad market indices to change by a maximum of between 1 and 2 per cent in an entire day). Overall prices then recovered almost as quickly, but gigantic price fluctuations took place in some individual shares. Shares in the global consultancy Accenture, for example, had been trading at around $40.50, but dropped to a single cent. Sotheby’s, which had been trading at around $34, suddenly jumped to $99,999.99. The market was already nervous that day because of the Eurozone debt crisis (in particular the dire situation of Greece), but no ‘new news’ arrived during the critical 20 minutes that could account for the huge sudden drop and recovery, and nothing had been learned about Accenture to explain its shares losing almost all their value.
The trigger was indeed an algorithm, but not one of the sophisticated ultra-fast high-frequency trading programs. It was a simple ‘volume participation’ algorithm, and while the official investigation does not name the firm that deployed it, market participants seem convinced that it was the Kansas City investment managers Waddell & Reed. The firm’s goal was to protect the value of a large position in the stock market against further declines, and it did this by programming the algorithm to sell 75,000 index future contracts. (These contracts track the S&P 500 stock-market index, and each contract was equivalent to shares worth a total of around $55,000. The seller of index futures makes money if the underlying index falls; the buyer gains if it rises.) The volume participation algorithm calculated the number of index futures contracts that had been traded over the previous minute, sold 9 per cent of that volume, and kept going until the full 75,000 had been sold. The total sell order, worth around $4.1 billion, was unusually large, though not unprecedented: the SEC/CFTC investigators found two efforts in the previous year to sell the same or larger quantities of futures in a single day. But the pace of the sales on 6 May was very fast.
On both those previous occasions, the market had been able to absorb the sales without crashing. In the first few minutes after the volume participation algorithm was launched, at 2.32 p.m. on 6 May, it looked as if the market would be able to do so again. Electronic market-making algorithms bought the futures that the volume participation algorithm was selling, as did index-arbitrage algorithms. (These programs exploit discrepancies between the price of index futures and the price of the underlying shares. A large sell order in the index futures market will often create just such a discrepancy, which can be profited from by buying index futures and selling the underlying shares.) Algorithmic trading was still in the benign zone that it occupies most of the time: electronic market makers and arbitrageurs were ‘providing liquidity’, as market participants put it, making it possible for the volume participation algorithm to do its intended large-scale selling.
However, high-frequency traders usually program their algorithms to be ‘market neutral’, in other words to insulate their trading positions from fluctuations in overall market levels. From around 2.41 p.m., therefore, those algorithms started to sell index futures to counterbalance their purchases, and the electronic index futures market entered a spasm of the kind identified by Hasbrouck and Saar. One algorithm would sell futures to another algorithm, which in its turn would try to sell them again, in a pattern that the SEC/CFTC investigators call ‘hot potato’ trading. In the 14-second period following 2.45 and 13 seconds, more than 27,000 futures contracts were bought and sold by high-frequency algorithms, but their aggregate net purchases amounted to only around 200 contracts. By 2.45 and 27 seconds, the price of index futures had declined by more than 5 per cent from its level four and a half minutes earlier. The market had entered a potentially catastrophic self-feeding downward spiral.
Fortunately, though, the electronic trading platform on which these index futures were being bought and sold – the Chicago Mercantile Exchange’s Globex system – is programmed to detect just such a spiral. Its ‘Stop Logic Functionality’ is designed to interrupt self-feeding crashes and upward price spikes. A ‘stop’ is an order that is triggered automatically when prices reach a preset adverse level. Buyers of index futures, for example, will sometimes try to protect themselves from catastrophic losses by placing stop orders that will sell those futures if their prices fall below a given level. However, these sales can potentially begin a cascade, causing further price falls which in turn trigger further stop orders. The goal of the Stop Logic Functionality is to halt this process by giving human traders time to assess what is happening, step in and pick up bargains.
Pushing the red button on an official market maker’s system, therefore, did not entirely remove the bids to buy and offers to sell, but reduced the bids to the lowest possible price that could be entered into electronic trading systems (one cent), and increased the offers to the maximum possible price ($99,999.99). These ‘stub quotes’ allow market makers to fulfil their formal obligations, while being so hopelessly unattractive that under normal circumstances no one would ever want to take a market maker up on them. In the case of several stocks, however, the evaporation of the market by around 2.45 p.m. was so complete that stub quotes were the only ones left. In consequence, ‘market orders’ (orders simply to buy or to sell at the best available price) were executed against stub quotes, hence Accenture’s price of a cent and Sotheby’s of $99,999.99.
As Steve Wunsch, one of the pioneers of electronic exchanges, put it in another TABB forum discussion, US share trading ‘is now so complex as a system that no one can predict what will happen when something new is added to it, no matter how much vetting is done.’ If Wunsch is correct, there is a risk that attempts to make the system safer – by trying to find mechanisms that would prevent a repetition of last May’s events, for example – may have unforeseen and unintended consequences.
Systems that are both tightly coupled and highly complex, Perrow argues in Normal Accidents (1984), are inherently dangerous. Crudely put, high complexity in a system means that if something goes wrong it takes time to work out what has happened and to act appropriately. Tight coupling means that one doesn’t have that time. Moreover, he suggests, a tightly coupled system needs centralised management, but a highly complex system can’t be managed effectively in a centralised way because we simply don’t understand it well enough; therefore its organisation must be decentralised. Systems that combine tight coupling with high complexity are an organisational contradiction, Perrow argues: they are ‘a kind of Pushmepullyou out of the Doctor Dolittle stories (a beast with heads at both ends that wanted to go in both directions at once)’.