The idea that retail investors are losing out to sophisticated speed traders is an old claim in the debate over HFT, and it’s pretty much been discredited. Speed traders aren’t competing against the ETrade guy, they’re competing with each other to fill the ETrade guy’s order. While Lewis does an admirable job in the book of burrowing into the ridiculously complicated system of how orders get routed, he misses badly by making this assumption.
The trouble with the stock market — with all of the public and private exchanges — was that they were fantastically gameable, and had been gamed: first by clever guys in small shops, and then by prop traders who moved inside the big Wall Street banks. That was the problem, Puz thought. From the point of view of the most sophisticated traders, the stock market wasn’t a mechanism for channeling capital to productive enterprise but a puzzle to be solved. “Investing shouldn’t be about gaming a system,” he says. “It should be about something else.”
The same system that once gave us subprime-mortgage collateralized debt obligations no investor could possibly truly understand now gave us stock-market trades involving fractions of a penny that occurred at unsafe speeds using order types that no investor could possibly truly understand. That is why Brad Katsuyama’s desire to explain things so that others would understand was so seditious. He attacked the newly automated financial system at its core, where the money was made from its incomprehensibility.
Update: For some highly technical information on High Frequency Trading I was pointed to this set of articles from ACM, Association of Computing Machinery.
Through a series of microwave towers, the dish beams market data 734 miles to the Chicago Mercantile Exchange’s computer warehouse in Aurora, Ill., in 4.13 milliseconds, or about 95% of the theoretical speed of light, according to the company.
Fiber-optic cables, which are made up of long strands of glass, carry data at roughly 65% of light speed.
The goal of this article is to introduce the problems on both sides of the wire. Today a big Wall Street trader is more likely to have a Ph.D from Caltech or MIT than an MBA from Harvard or Yale. The reality is that automated trading is the new marketplace, accounting for an estimated 77 percent of the volume of transactions in the U.K. market and 73 percent in the U.S. market. As a community, it’s starting to push the limits of physics. Today it is possible to buy a custom ASIC application- specific integrated circuit to parse market data and send executions in 740 nanoseconds or 0.00074 milliseconds.4 Human reaction time to a visual stimulus is around 190 million nanoseconds.
By 2005, most shops were also modifying kernels and/or running realtime kernels. I left HFT in late 2005 and returned in 2009, only to discover that the world was approaching absurdity: by 2009 we were required to operate well below the one-millisecond barrier, and were looking at tick-to-trade requirements of 250 microseconds. Tick to trade is the time it takes to:
1. Receive a packet at the network interface.
2. Process the packet and run through the business logic of trading.
3. Send a trade packet back out on the network interface.
To do this, we used realtime kernels with bypass drivers (either InfiniBand or via Solarflare’s
As market data enters the switch, the Ethernet frame is parsed serially as bits arrive, allowing partial information to be extracted and matched before the whole frame has been received.
Then, instead of waiting until the end of a potential triggering input packet, pre-emption is used to start sending the overhead part of a response which contains the Ethernet, IP, TCP and FIX headers. This allows completion of an outgoing order almost immediately after the end of the triggering market feed packet.
The overall effect is a dramatic reduction in latency to close to the minimum that is theoretically possible.
Algorithmic trading bots react instantaneously to keywords in news reports and even tweets, which is likely why the market fell so quickly. Stocks started to recover about three minutes later but took seven minutes to return to their earlier levels.
The program placed orders in 25-millisecond bursts involving about 500 stocks, according to Nanex, a market data firm. The algorithm never executed a single trade, and it abruptly ended at about 10:30 a.m. ET Friday.
Translation: The ultimate goal of many of these programs is to gum up the system so it slows down the quote feed to others and allows the computer traders (with their co-located servers at the exchanges) to gain a money-making arbitrage opportunity.
HFT affects all investors to an extent, because stocks are now priced differently than in the past. The market used to consist mostly of investors analyzing cash flows and balance sheets, trying to calculate a company’s fair value. HFTs, on the other hand, react to movements in stock prices alone. That is not necessarily a bad thing, but since HFTs are responsible for two-thirds of the trading volume, we have the strange situation where they can set the price based on what they perceive others’ perceptions to be.