[Patrick notes: Welcome to the first episode of Complex Systems. I have always produced an edited transcript with my podcasts, and frequently take the opportunity to add notes, in this format.]

Some years ago, I built a trading simulation for a startup I was running, Starfighter. It was, at the time, by far the most impressive technical artifact I'd ever made, including a scratch-built fake stock exchange. I recently ran into Ricki Heicklen, who has a business teaching trading, and said we should swap notes on comparative pedagogical game design.

Trading doesn't have a vast body of useful information targeted at people of various skill levels. There is a vast amount of fanfic. Actual practitioners work at a relatively small number of firms and are extremely disincentivized from explaining how the world really works to outsiders. Ricki and I talk about some of the lessons people learn early in their trading careers, how those correspond to market microstructure which impacts everything tied to the financial markets, and the security mindset. There are also some (perhaps surprisingly) portable lessons from trading to many other complex systems.

Sponsor: This podcast is sponsored by Check, the leading payroll infrastructure provider and pioneer of embedded payroll. Check makes it easy for any SaaS platform to build a payroll business, and already powers 60+ popular platforms. Head to checkhq.com/complex and tell them patio11 sent you.

Timestamps:
(00:00) Intro
(00:58) Ricki’s background in trading
(01:07) Teaching trading: pedagogical approaches
(03:47) Challenges in learning trading
(07:08) The importance of adverse selection
(08:02) Crowdfunding and market dynamics
(11:49) Understanding order books
(19:53) Sponsor: Check
(21:07) Liquidity and market microstructure
(27:48) Teaching trading through practical examples
(35:14) Reacting to new market information
(36:47) Understanding order cancellations
(41:29) Order types and other oddities
(56:00) Introduction to arbitrage
(01:05:00) Teaching position sizing and risk management
(01:10:05) Thoughts on quant trading
(01:12:44) Trading simulations and mental models
(01:13:08) Simulated Insider Trading Exercise
(01:20:18) Security Mindset in Trading
(01:24:41) Information Leakage in Trading Firms
(01:38:17) Final Thoughts

Transcript

Patrick McKenzie: Hey everybody, my name is Patrick McKenzie, better known as patio11 on the Internet, and I'm here with Ricki.

Ricki Heicklen: Hi, I'm Ricki Heicklen, not quite yet as well known on the internet, but I'm very excited to be here to talk about teaching trading.

Patrick McKenzie: Yep. So, some people might know I ran a startup back in the day called Starfighter, which had a product called Stockfighter: as the coolest thing I had made to date at that point, I coded up a fake stock exchange. I recently learned that you also created some fake trading interfaces to actually teach trading. (Ours is mostly about assessing programming ability rather than the trading logic itself.)

[Patrick notes: The name Starfighter was an homage to picked by my cofounders Erin Ptacek and Thomas Ptacek to a classic sci-fi movie The Last Starfighter, where an advanced alien civilization recruits generals for an intergalactic war by concealing a work sample test in arcade game cabinets in out-of-the-way locations like southern California. We likewise believed that many talented candidates did not necessarily live in tech hotspots.]

I'd love to just have an open ended conversation on how does one teach trading? What are the technical choices you made on that stock exchange? Maybe compare and contrast with another person who might've been coming at a similarly shaped problem from a very different angle.

So – take it away. Why don't you give people a little bit of professional context on where you're coming at this from?

Ricki Heicklen: Great. So my credentials as it relates to this conversation are that I was a US equities trader at Jane Street Capital for two and a half years. I really enjoyed trading, but way more than that, I really enjoyed teaching trading. The summer internship was my favorite part of the job there by far, and figuring out what makes sense pedagogically for teaching trading, especially in contrast to other disciplines. 

I joined Jane Street straight out of undergrad, so I had had a lot of experience of sitting in lecture classrooms and not as much of more interactive, dynamic teaching, the type that Jane Street did when I arrived there.

Figuring out the pedagogical merits of each, and how to tailor teaching trading toward the audience of people that firms are often hiring straight out of undergrad is a special pet interest of mine. 

Patrick McKenzie: So I presume you're kind of in the demographic center of people that get hired at finance firms, but for the benefit of people who haven't been in that industry, can you just sketch out the general profile that ends up doing the sort of job you did?

Ricki Heicklen: Sure. So I went to Princeton University, an Ivy League school; I studied computer science with a focus on theoretical computer science. I had an internship at Jane Street the summer going into my senior year in college. 

I was then hired to work at Jane Street full time as a quantitative trader and started working there immediately after graduating. This is certainly true of the median employee – went to a fancy college or university, for New York traders, mostly coming from the United States, which is where I was. 

Often it’s people who have experience spending a long time thinking about math problems or puzzles, but not necessarily a lot of life experience under their belt. Certainly not a lot of professional training and often less background than you might expect in economics and finance specifically – rather, a general comfort with concepts around probability, expected value, and math puzzles writ large.

[Patrick notes: Interestingly, this demographic profile has changed over time and varies/varied with respect to which part of finance one found oneself in. A frequent feature of Wall Street biographies in the 1960s through 2000 or so is that upstart investment banks and other businesses recruited much more broadly than Wall Street generally, and that this tended to calcify after the firms had reached substantial success. Similarly, HFT shops used to be known for hiring almost solely for acumen but now have lists of feeder schools which resemble the rest of Wall Street.]

Patrick McKenzie: I remember a person who went to my university came up to me and said that they had an interview later in the day with a finance firm, and they understood that I read the Wall Street Journal a lot – could I please explain to them the difference between a stock and a bond in case that came up? I was very worried, and I'm a little less worried these days. (Still a little worried if one is at that point, but be that as it may.)

The part about competitions is interesting. One of my theories is that there's some selection effect for people who have that competitive mindset and want to play games about these sorts of things, but I generally tend to think that strong performance in games, particularly in competitive environments, probably predicts performance in real life.

At least in the tech industry, we don't back-propagate that into our decisions for advertising at places like the International Math Olympiad. Is finance more rational than we are on this?

Ricki Heicklen: I think it's possible finance is more rational. Now, I don't personally know how it is that software goes about recruiting as well, but I think that part of what quantitative trading firms are often trying to do is they are trying to recruit people who have the raw intelligence, that when combined with training that those firms are capable of providing in-house will turn into good trading skills. 

I think with software development, there is both more opportunity for people to get good at software development external to the places that are hiring for it, and therefore more ease at measuring what somebody's skill in that domain is than there is for trading.

Right now, the state of the world is if you want to learn how to be a good trader, basically your only option is to go to a good trading firm. It is very hard to find good materials for learning how to trade, learning how to have the kind of intuitions and heuristics about a market that a trader ought to develop, in any context outside of a firm.

There are a couple of reasons for this. One is that the firms are incentivized to not spread that information too far and wide, and another is that trading skills are not skills that you can easily pick up from reading a book or from consuming a YouTube lecture series. They're way easier to learn through actually being immersed in the environment and doing it yourself.

That means that you're going to need a really good ratio of teachers to students in order to properly transmit trading knowledge to those students. This is just not going to be that widely available when you're bottlenecked on how many good traders there are, and when those traders will benefit a lot more from full time trading than they will from teaching those skills to a few dozen other people.

Patrick McKenzie: It honestly surprised me when I heard that firms had intensive training practices because my model of the world for this kind of thing is, “you join a desk and sit next to someone who's been doing it for a while,” and essentially have an apprenticeship model, versus a classroom lecture simulation-based model. I'm glad that some investment is happening into simulations.

[Patrick notes: teams are “desks'' in finance, in a fun bit of metonymy-based entomology. The trading used to happen at a particular physical desk, and then the venue for trading became known as the desk even though there might not be a physical location synonymous with it anymore, and then the people trading at the venue started calling themselves the desk.]

Actually, one of the reasons we picked ‘build a fake stock exchange’ as the thing Starfighter would do to measure engineering ability was because it is so hard on the outside to figure out how these things worked. There's kind of a mystique around making piles of money by being smarter than the next person. [Patrick notes: this is part of the appeal of dramatic accounts of finance the Big Short or Margin Call.]

We thought, well, we can make the 5,000th game about dwarves killing orcs, or we can make one of the first games that is actually true to life about trading stocks. We will certainly get some snipes from people who do it for a living, but that isn't exactly the audience.

We’ll have this fun fictional environment. It's a very deep domain now that we can teach to people – learning stuff is fun – and then they can show off their stuff in the other domain that we're really interested in, which was for us programming ability that they probably had before they joined the game. Everybody wins from the discovery of knowledge there. 

(Anyhow, I still cry tears that Starfighter is no longer in business.)

My toy economic model for a high speed trading firm is, (1) we hire very smart people directly out of undergrad, (2) ???, (3) they pull a lot of money out of the economy. Somewhere in the middle there is connecting them to infrastructure that they cannot access if they're back at home at E*Trade; some of that is imparting them this wisdom.

You have taught trading classes at Jane Street before – how do you teach trading?

Ricki Heicklen: Great question.

I think that there is indeed this big black box there that a lot of people don't know about. I was a quant trader at Jane Street for two and a half years, but despite trading being my lifelong passion, making US equities markets slightly more efficient, I left it for teaching because it just pays a lot better. [Patrick notes: I will explicitly flag this as dry wit for the benefit of people far from this industry.]

I ended up in a role where I was teaching trading for a couple of summers to a group of precocious, smart high school students with no background knowledge in trading. I got to really refine a curriculum that was in part inspired by the experience I'd had trading, and in part reactive to what worked well and what didn't work well at teaching trading concepts to a small group of smart but financially illiterate high school students.

[Patrick notes: As an aside, people often assume that “illiterate” is a value judgment on a person’s worth, in part because educational attainment is the primary way one establishes a legitimate claim to status in the American professional managerial class. “Illiterate” is a state that literally everyone is born in even with respect to their native language. It is a concise way to say that someone has not yet expended, or had expended on their behalf, the prodigious amount of effort required to get to literacy. One reason I make this defense of the word is it is critically important to people’s world models that some people are illiterate, both vis written literacy in a particular language and also in every other interesting domain. This includes people who are broadly esteemed in society. And so, when one is designing systems, one needs to think about how they interact with people who occupy a much larger slice of the literacy spectrum than system designers tend to.]

Patrick McKenzie: So what's the first thing you would tell someone? 

Like, “I'm going to impart a bit of information upon you to get you ready for understanding the US equities markets…” – if you only had one sentence, what is that?

Ricki Heicklen: The number one sentence for purposes of trading, in general, is to think about adverse selection. Adverse selection is the concept that, conditional on getting to do a trade with someone, your trade might be worse than you'd previously thought it would be – that the world that you are looking at is one that has lots of different models that will explain different systems, and you can make predictions of what those models would output for numbers. But as soon as you are putting an order into a market, you need to think about the profitability of your trade, if it gets traded with, versus if it doesn't. If it doesn't, it profits zero.

So the fact of somebody else's willingness to trade with you should adjust your model, and therefore you should calculate the profitability of the order that you submit based on limiting yourself to worlds in which those trades do happen, i.e. worlds in which the trade that you want to do is worse than it otherwise would have appeared.

Patrick McKenzie: One way that adverse selection shows up in the capital allocation markets in startup-land, where I spent much of my life, is that there is this notion that okay, we have a professionalized class of venture capitalists; we also have angel investors who might be rich people who have some amount of domain knowledge about the tech industry specifically, because that's how they ended up in the class of rich people; and then we have public market participants who unfortunately can't get access to tech stocks in their first 10 years of life.

[Patrick notes: Because it is uncouth in tech circles to observe that working in tech will fairly reliably make one wealthy, one purpose of saying that one is an angel investor is to say that, by implication, one is an accredited investor. Many people in tech know that accreditation does not mean that one has passed a test or achieved a credential; it means that one is wealthy and/or has stable, high income.]

A thing that has been happening is crowdfunding to fund new startups; won't this allow people to have access to these high flying companies before they go public? [Patrick notes: For example, see Regulation Crowdfunding at the SEC, which kicked off something of a boomlet in crowdfunding platforms in the U.S. targeting $1,000-$5,000 equity investments from hundreds or thousands of not-necessarily-accredited investors.]

But (speaking solely for myself and not as a representative of anyone else in the world) if a company has decided to raise money on a crowdfunding market, it has been passed on by the people who have made it their life's work to find profitable investments in venture capital. And it has also been passed on by rich people in tech who can easily write $250,000 angel checks. [Patrick notes: The typical angel check used to start at about $25,000 but a) dollars are worth less these days and b) some people who write these checks can comfortably swing dozens of $250k investments.] And it is attempting to get $1,000 checks from people in the crowdfunding markets. 

There is a reason why it is rational for them to get $1,000 checks. [Patrick says that reason explicitly: Better investors, who write the larger checks, have passed on the opportunity to invest.] Therefore, I hear people that there is some notion of “equitable access to growth opportunities in the market,” but I don't think that public crowdfunding will actually give the general public opportunities to tap into growth markets on an equal footing with VC because, bluntly, a level playing field is one in which professionals destroy amateurs.

Ricki Heicklen: Yep. I agree fully with all of that. 

I think that the high level advice I give to people when thinking about adverse selection is, “why should you do any trades?” If adverse selection were as powerful a principle as I'm claiming that it is, shouldn't nobody ever trade with one another, especially in zero-sum environments?

I think the answer that I give to that is, you need a story for why – despite the fact that this trade is available to you, i. e. despite the fact that there’s somebody on the other side of the trade who wants to do it with you, and nobody else has taken your side of the trade yet – it is still worthwhile for you to do it.

There are a lot of different explanations for why this might be. One explanation is, “nobody else has had the opportunity to do it yet.” You are actually the first person to get there. That might be true for those early-stage VCs – honestly, I didn't even fully follow all the different players in the ecosystem you just mentioned, I might get some of those details wrong. That might be true for the people who get the first opportunity to invest.

As there are more and more people who had that opportunity and turned it down, the phenomenon of adverse selection should be a larger and larger factor in your weighing against whether you choose to invest. But there could be other reasons.

Patrick McKenzie: Yeah, an interesting difference between the private markets and the public markets is that – to do a gross generalization, and I know you can come up with all the ways that this is not true – everyone gets access to an incoming order at approximately the same time, where in VC-land, the thing that you want most is differentiated deal flow, which means when someone has an idea for their new company, they think of you and pitch you on the opportunity of investing first. At that point, you are essentially a person who has exclusive rights to take this trade or pass on it. 

In the public markets, everyone (almost) simultaneously gets the right to take the trade or pass on it. Then we have dueling robots battling down to the microsecond of who grabs it first.

VCs would also say, “Also, money is not fungible in VC markets because our dollars are better than their dollars” – VCs are mostly lying about this. (Hmm, I might be a VC in the future, in which case I might be lying a little bit less. But broadly, VCs are lying about this.)  [Patrick notes: VC investment is intrinsically bundled with advice and sometimes bundled with introductions to customers/candidates/other investors, market color, and sometimes fairly elaborate services organizations. But, speaking candidly, advice, introductions, market color, and professional services are abundant in the world, and risk capital is relatively rare. Almost all conversations with VCs start because someone wants risk capital.] 

So, I mentioned that it is not the case that “in the real world” – the VC is part of the real world, too, just like the Internet is – in public equities markets and other markets that are traded, bond markets, options, derivatives, etc., it's not actually the case that everyone sees an order at the same time.

Can you talk about the big structural reasons for that, and then the micro-structural reasons? (You know, you might be seeing it some fraction of a millisecond later, which is way too late.)

Ricki Heicklen: Sure. I think one of the high level structural reasons for that is that a lot of trading happens in the kind of order-book form – in which you are approaching an order book seeing orders that people have input into it, into that market – that include bids and offers. 

Bids being to buy a product – I don't know how financially literate the crowd will be on these kinds of structural order book things. [Patrick notes: Again, no judgment implied by this phrasing.]

Patrick McKenzie: Why don't we assume they're pretty smart and if I think we're flying over the head of the average audience member, I'll do a little bit of explanation. If you go way over my head, I'll ask you to explain.

Ricki Heicklen: Okay, great.

One of the reasons for that is that a lot of the times that someone's approaching a market, this is in the context of day trading, in which you have continuous trading of people submitting bids and orders into the order book, and people having the opportunity to come see those orders and decide whether to trade with them.

If an order is still on the order book, it means nobody has decided to trade with it yet, despite potentially having seen it. 

Patrick McKenzie: Can I stop you for a moment to give people definitions of some of these terms, so you can correct my understanding? 

Ricki Heicklen: Absolutely.

Patrick McKenzie: So, if you want to be better informed than almost all financial journalists, you can understand what an order book is. (That fact frustrates me immensely.) An order book is simply an ordered list on each side of the market of the people who want to buy stock and the people who want to sell stock. It's typically ordered by pricing, and then at equal levels of pricing it's ordered by time, so the first person to place an order, price being equal, will get the fill – that means they get the opportunity to trade with the incoming order.

The best price from the perspective of the person who is attempting to trade right now gets the order first. And if the trade that they have expressed interest in doing is bigger than the first person that trades with them, they fill that person's entire order, which is “resting on the book” because it hasn't been filled yet.

Then they trade with the next order in the priority queue, and they keep doing so until their order is entirely satisfied. If their order can't be entirely satisfied, the typical thing that happens is that the remaining part of their order rests on the order book. You can express your opinion when you're doing a trade on whether you want that to happen or not via order types and similar things – that gets a little bit into the weeds. The important thing to know is, an order book is simply a priority list of people who want to buy and people who want to sell. Two different lists.

Ricki Heicklen: That's great. Just to hopefully make it a little bit clearer for listeners who learn best by examples I'll explain what this might look like in an order book for Shmapple stock. 

So let's say a Shmapple stock right now has a bunch of bids and offers. You might have somebody bidding to buy Shmapple stock for $101, somebody else bidding $100, somebody else bidding $99. You might have offers to sell this stock for $105, $106, $107.

The best bid, i.e. the bid that an incoming offer to sell would first trade with (if it desires to do so) would be the $101 bid, the highest bid on that list. Any offer to sell which comes in at $101 or lower should be happy to trade with it. 

Trading in environments that are strictly financial means, “I think you're wrong.” So, the $101 bid will only trade with an offer that directly contradicts it, that thinks that the value is not higher than $101, but instead lower than $101, which would be an offer to sell at a price of $101 or under.

[Patrick notes: “Trading” here means something subtly different than “placing trades.” There exist many participants in the market who will trade not because of a fundamental belief that the price is wrong. For example, a great deal of all net flows into stock markets are price-insensitive households who are saving for retirement. Their trades are not arguments against the current value of Google or the S&P 500 except in a pretty tenuous sense; they’re expressions of faith that over decades the returns to owning those assets will be pretty attractive.] 

In contrast, on the offer side, the best offer is the lowest offer available. Just as if you have two t-shirt stands – one person selling them for $15, and another person selling identical t-shirts for $13 – you, a buyer, who's happy paying up to $20 to buy a t-shirt, would prefer to pay $13 for the identical t-shirt. 

So too, the lowest offer available on the order book is going to be the one somebody who enters the order book to buy for a price higher than that offer would trade with first.

Patrick McKenzie: I have a question for you that often gets people in their first day of trading: If the highest bid, the highest offer to buy something, is currently $100, and I say I will sell it for $95 or more – in most cases, what price do we actually transact at?

Ricki Heicklen: Great question. This is often one of the first questions I ask people on day one in Intro to Trading of the trading bootcamp that I run. 

I think some popular answers include, “it should be the midpoint of the two,” because each of you have happy prices and you'd be willing therefore to split the surplus, split the difference of how much additional money you would make relative to your happy price.

Some people think “it should be the price that the most recent person said” because the other person should certainly be happy with that. But, in fact, the way that this works in standard financial markets, certainly in US equities markets, is that the price that this order will transact at is the price of the order that was already in the box.

So if there was a bid for $100 and somebody comes in with an offer to sell for $95, the trade will happen at a price of $100. Why is this? One mechanical explanation for this is it makes trading happen a lot more efficiently, because if it were not the case, somebody who wants to sell and is happy to sell for prices all the way down to $95 will need to go look at the order book, see what orders exist in it, calculate those, and then sell one share at a price of a hundred dollars and a second share at a price of $99.5 and a third share at a price of $99, and enter each of those orders manually, separately.

If the exchange can execute the behavior that that person would have intelligently desired if they'd taken the time to do so, and cause that to automatically be the transaction that takes place, they'll do a better job attracting people who can just enter simple orders that fully communicate their preferences and trust that the exchange will execute those as desired.

Patrick McKenzie: I think there's a subtlety here that will certainly have implications later when we talk about security, but in most markets, and certainly in the US equities market, the order goes on the book, the exchange will quote you the orders, and orders which are on the book are public information.

You can buy a data product from the exchange (and many other people) that shows all the orders in substantially real time. For you as an individual person, when you go to your brokerage there is some button you can click that will get you that information in either substantially real time or on a 15 minute delay, if e.g. you're looking at some free news site.

[Patrick notes: Selling the data feeds to professionals and to other infrastructure providers, like e.g. brokerages who purchase it to facilitate the investing of their individual customers, is a primary way by which stock exchanges make money. The reason for the 15 minute time delay on free sources like Yahoo News is to protect the value of the far-less-delayed feeds.]

So the fact of the order is public. Who is making the order is not. Why don't we just say, you're trading with Gladys in Kansas, versus Goldman Sachs?

Ricki Heicklen: Great question. One reason is that Goldman Sachs doesn't want this to be the design of the market; they want to be able to capture the flow that would be selected against if other firms want to only trade with, or other people want to only trade, the more naive buyers.

Let me say that the other way around. If I come to an order book and I see some orders being placed by Goldman Sachs and some orders being placed by Gladys from Kansas, then I'm more likely to want to trade with Gladys from Kansas because I think it's less likely that she's on the right side of the trade there and ripping me off.

This is something that Goldman Sachs doesn't want me to do – avoid trades with them specifically. Goldman Sachs will therefore be more attracted to exchanges where they get to hide that information, where that information isn't fully publicly available. 

There's still lots of ways in which you, a trader, can infer from market behavior which trades are more likely to be from Gladys from Kansas.

Patrick McKenzie: Can I interrupt with one here? Because it's poorly understood in many spaces. If you've ever heard the words “payment for order flow,” when you're using a retail retail exchange, such as Robinhood or E*Trade or basically any app that is on your phone, you are not sending orders to a real stock exchange in most cases. 

(You will get a price that's as good or better as the real stock exchanges under the law, and an incredible amount of human effort is spent on making sure that happens, because if that is violated, people get very large fines, much larger than the trade you are doing.) Instead, your brokerage will have made a deal with an internalizer, and the internalizer might be a high frequency trading firm.

[Patrick notes: Regulation NMS says, in part, that trades must happen at a price at or better for the customer than the NBBO: National Best Bid/Offer. Actually calculating the NBBO and routing customers orders is a non-trivial distributed systems problem. There are dozens of stock exchanges in the U.S. and the one with the best price at this instant may not be one whose name you are familiar with. Which is fine: as a non-specialist user of the financial system, you almost certainly instruct your broker “route to best available exchange” and trust that they’ll do their job effectively, much like you virtually never instruct your computer precisely how to move a packet to google.com.]

What the internalizer does is they aggregate volume from the other retail traders and they essentially take the other sides of those trades immediately at a price that is at or better than the best price on the market, and then they do whatever high frequency magic they want to after taking that to market, make it elsewhere, and because they know that the retail trading population – Gladys from Kansas – is not generally informed by the standards of Wall Street professionals, that facing that flow and providing liquidity as a service, they can quote – you're going to have to explain to the audience what this means – they can quote that liquidity at lower prices because there is less of that adverse selection risk. That's why they are able to give a better price to internalized orders than they are on the public markets.

Ricki Heicklen: Yep. That's exactly right. 

Patrick McKenzie: So why does liquidity have a price?

Ricki Heicklen: Liquidity in a market has value because it means you will need to pay the other person less in order to get to do a trade there, because there are lots of people there who are willing to offer you those trades. In particular, one way to think about this is the size of the spreads. The spread is the difference between the best bid – the highest bid – and the best offer – the lowest offer.

Anytime that you want to come into a market and immediately do a trade, you need to cross that spread. So if somebody is willing to buy Shmapple stock for a hundred dollars and someone else is willing to sell it for $101, it's reasonable to start with an approximation that the fair value of Shmapple stock is $100.5.

However, if you want to trade it, you're going to need to pay an additional 50 cents there per share in order to do so. Now let's say the offer, the best offer, was actually $200. You're now going to need to cross a spread that causes you to pay an expected $50 between your best estimate of fair, the midpoint, $150, and either the bid or the offer, depending which direction you want to go with that trade.

How tight the spread is is a decent proxy for liquidity. So is how many dollars trade each day in that product, which is also going to be a pretty clear indication of that, and you are going to have to pay less to have someone take the other side of that trade.

Patrick McKenzie: Conceptually speaking, the thing you are paying for liquidity is “certainty that you get the thing you want immediately.”

And so a thing I often like to say is, “there is a price associated with houses.” People are buying and selling houses all the time, but it will take you many, many, many weeks to get out of buying and selling a house. And since that bid-ask spread is wide, and the transaction will take many weeks to settle, houses are illiquid. (They also have huge fees.)

[Patrick notes: I recently wrote about a particular genre of transactional friction in buying a house: title insurance, where I paid $4,000 to insure against the risk that the person selling the house didn’t actually have the right to sell it. Nobody needs to pay $4,000 when buying the equivalent amount of Google stock to insure against the risk that the person facing them in the trade might not actually own the stock they sold. That this is true is a fascinating and powerful demonstration of how orderly markets are and how low transaction costs can get when markets are well-ordered.]  

So what are you paying for with liquidity? You're paying for, “I'm willing to take less money out of the system, in return for more certainty that I get what I really want out of the trade,” which if you're buying, that means “I want certainty that I get the stock right now at the price I want.”

If I’m selling, I want certainty that I'm canceling my position in this stock and getting at least as much money as I want. 

Why is certainty valuable? Among other things, if you put in a buy order and then go away for the rest of the day, like a retail trader might reasonably do, and then come back the next morning, they might not have the Shmapple stock that they tried to buy, and they would be sad because they had a point of view on Shmapple, and now they have not expressed that point of view, and the price might be very different tomorrow. So you buy liquidity because you like the prices that liquidity represents, and you sell liquidity because selling valuable things to people for money is a great way to make money in a capitalist system.

Ricki Heicklen: That's correct. The other thing going on here is that one of the things that trading firms are selling, one of the value propositions of a trading firm, is that they are taking on some amount of risk by holding positions that aren't necessarily positions they want to be holding, but that they know later on they'll be able to sell.

So part of providing liquidity here is, let's say right now a bunch of people want to buy Shmapple stock and later they're going to want to sell it. That will mean that a quantitative trading firm can therefore sell it at an inflated price now to people who want to buy it for that higher price, and later on will be able to buy it back and cover their position for a lower price when there's more people who want to sell it. 

However, they're taking on risk by holding on to Shmapple stock between those two points in time. They are being compensated for that risk to the tune of the spread between the price that they're buying it for now and the price that they're selling it for later.

Other people don't want to be holding positions in stocks that might move as a result of, say, changing news in the world or other idiosyncratic reasons for volatility in stock prices that have nothing to do with the positions that they actually want to attain. 

So this is one of those basic services that are being provided by the middlemen in these pairs of trades that might happen on Monday and Tuesday when people want to buy versus when people want to sell, let's say, in order to make it possible for others to trade immediately and not take on that risk.

Patrick McKenzie: And there's a fascinating history to how liquidity is provided. Back in the day, it was provided by specialists who, let's say, got to know a particular stock or a particular product, et cetera, and became something of experts in it. But these days, this is largely done by, you call them quantitative trading firms, sometimes people say market makers, but they, to a greater or lesser degree, are quoting markets in very, very many things, and it is highly likely that they don't have a super developed opinion on, say, Shmapple's new marketing campaign. 

This is a ticker that has had some past behavior, for which they have large amounts of evidence of what that correlates to in the world. But they are largely not in the business of assessing Shmapple's business. Other people in the economy are doing that – they are the ones who are buying liquidity. The people who are selling liquidity are in the business of assessing how the market reacts to Shmapple, and getting there perhaps a little bit faster than the next person.

Ricki Heicklen: That's exactly right, Patrick. I think a lot of what's going on here is that the quant trading firms are providing the service of ensuring liquidity.

They are better positioned to do this because they are better capitalized, and because they have access to the technology that allows them to be competitive in this regard, and not just lose out to slower times that might be where adverse selection would come in for other people trying to provide this service.

In particular, they have the ability to have increasingly diversified positions as they enter more and more different stocks, or, beyond that, different asset classes in what positions they put on, such that if they capture real microstructural patterns in what kinds of trades are likely to happen when certain things will be profitable, they might lose out on Shmapple today because somebody else has read Shmapple's earnings reports or white paper and have a better sense of how that company will perform, but in the aggregate, they're likely to do well such that a few cases aren't as likely to cause them to lose out too much.

Now, you might say there are people looking at white papers and earning reports in each and every one of those stocks. So how does this actually net out to be better? Won't you see them lose out on each of those individually? And I think the answer to that is that, by doing these kinds of research studies that look at all of these different stocks at the same time, they're looking for patterns, they’re looking for effects in the market, that in aggregate will beat out whatever the different individual idiosyncrasies across different companies are. 

They're specifically looking for past performance that they expect to be indications of future performance of stocks across the board, even after accounting for the fact that lots of those stocks will also have moves of the kind that quantitative trading firms won't be as much experts on.

Patrick McKenzie: So you said the word “market microstructure” in there and market microstructure is a topic that I'm absolutely fascinated by. I think it’s a topic that is much better illustrated by example than by any sort of theory-crafting, and which is very difficult to actually understand, to touch in the world, without having high speed hookups to exchanges, etc.

[Patrick notes: The best book on this subject, by a fairly substantial margin, is Trading and Exchanges: Market Microstructure for Practitioners. Unlike Michael Lewis’ book on HFT, Flash Boys, T&E coheres with reality. If you know Lewis’ thesis in Flash Boys and want to read a very, very thorough refutation, see Flash Boys: Not So Fast. As an aside, after the excellent Big Short and the should-be-in-fiction-aisle Flash Boys, Michael Lewis went on to write about cryptocurrency. Applying his usual formula, he selected an outsider hero sticking it to the man, one Sam Bankman-Fried. As it happens, both Ricki and I have written reviews of Going Infinite. Spoiler: they rhyme.]

There's a whole technical substrate here that people spend their lives building. How do you teach people like, “I'm a bright student just out of high school.” Market microstructure is important to understand. How do you express that in your lessons?

Ricki Heicklen: I'm glad you asked, because this takes me back to a few high level concepts about how it is that I approach the pedagogy of trading.

In particular, I find that the best way to learn trading is by doing it. On day one, the first class that I have people participating in my trading bootcamp go through is a class where you walk in the door and immediately you start trading. 

What does that look like? I have an order book that I've written out on a board. I've seeded it with a couple orders of my own that have a huge spread between them, and I've written up a contract on that board that will resolve to a specific number. 

I like to keep this as far away from actual knowledge of finance and the economy as possible, so my first contract will be, “What is the sum of the number of siblings that each person in this room has?”

This is a nice market for purposes of illustrating trading concepts, because each person in the room has some amount of private information, i.e. the number of siblings they have, has some rough sense of how many siblings on average somebody in the world might have, and then can whittle that down to, what about people primarily from the united states, what about people from the socioeconomic backgrounds that we assess other people in this classroom to most likely be in.

Patrick McKenzie: Can I jump in there to explain a little bit of background knowledge? What I understand from this notion of a contract is, at the end of the trading session, we're actually going to look, and then regardless of what you paid to get three shares in this contract, you will get three shares times this actual number.

So if it turns out that we have a combination of, say, 68 siblings in this room, and your entry point was like 30 per share, you've done pretty well for yourself there, possibly, But that isn't the end of the story, right? 

Ricki Heicklen: That's exactly right. That is a correct understanding of how it is that these contracts get made and then settled. 

(By the way, I'm using the word “Shmapple” because I find that when I use names of actual products people have a ton of nitpicky reasons like, “Apple stock actually doesn't behave like that” or something.) 

In the case of stock trading you are not going to be in a position where you buy this contract and then later the true value of Shmapple stock is revealed, and anybody holding a long position gets paid that number of dollars and anyone holding a short position has to pay that number of dollars.

We do things differently in this trading bootcamp because it is useful to be able to talk about the discrepancies between the models that we form and what the truth ends up being, and ways that we're surprised by that, because it is useful to.

In my class and in the entire curriculum, we try to use contracts that were resolved to specific numbers both because I want to keep things removed from specific facts or impressions people have about specific financial products in the real world, but also so that we can then settle those to certain values and compare for people what the models they originally had of the world predicted, with what those values ended up being.

Everything is best taught through example in the worlds of trading and the more data you're able to eventually present before people the more opportunities there are for noticing ways that there might be differences. If you're just looking at the value of Shmapple stock in the real market at any point in time, you are going to have a whole bunch of other factors that come into play that shape why it is that a stock price might be a certain thing. Therefore, you're going to have less of a good read on how strong your signal was, how strong your personal information in determining something was, because there are so many other possible sources of variance, sources of noise that are clogging your ability to assess the goodness of your initial trade.

Patrick McKenzie: I also like as a pedagogical approach that this allows people to infer some of the lessons without simply being told some of the lessons.

One lesson that is much easier for people to infer than it is to simply be told it and believe the truth of it, is that the people who have the best model for the world may or may not make the most money from trading. Don't the smartest guys in the room always win? Probably not the case. 

Can you sketch out for people who've never traded before: Why could you be the person with the best estimate of the number of siblings of everyone in the room – because you did a quick head count, you correctly pegged the ethnicity of everyone in the room, you made some assumptions based on your undergrad research in comparative anthropology, and also you were a mental math whiz in in high school – why do you still not make the most money?

Ricki Heicklen: First off, I actually think in markets like these, where there will be a settlement to the correct value, you're more likely to make money by having good models than you will in markets like for various stocks in the US equities markets, where a lot of what you're doing is trying to price things relative to what people in the market will think something's worth than to a model that takes into account e.g. the earnings reports of a company and figures out what the actual value proposition of the product that they create is.

You are much more interested in what the directions that these prices move within the next few minutes or within the next few days will be. This also exists in a smaller form in the markets that I run that might resolve on the order of half an hour from now, where if you can notice what trends will happen in the next 10 minutes or what trends are already emerging, you can profit by buying and then selling or by selling and then buying a contract that doesn't actually accumulate a position that you will get paid proportional to at the end, but instead, does what's called flipping a contract, where by taking both legs of the contract, you can make money on the difference between those two prices.

Patrick McKenzie: Tell us a little more about the structure of that first lesson in learning to trade.

Ricki Heicklen: Absolutely. So when people walk into the classroom we have a contract up on the board: the sum of the number of siblings that each person in this room has.

Everyone now has a little bit of private information – how many siblings they have – and some models of the world that are going to imperfectly capture the number of siblings that we specific people in the room have, but we'll get them to an approximate number that causes trading to happen at the right order of magnitude. 

We go around and play Tighten or Trade, a game in which each person on their turn needs to either tighten the spread by improving the best bid, in that case increasing it, or improving the best offer, decreasing it – or they need to trade with one of the existing orders in the book. 

This is an artificial constraint to ensure that trading happens in an environment that is zero sum and therefore you should be paranoid about approaching if there weren't such constraints forcing you to do trades with one another.

Patrick McKenzie: So the pedagogical reason for putting in that constraint is because in real life you always have the option to do nothing, right?

Ricki Heicklen: Yep, that's exactly right. Now I've found empirically when teaching this class that people don't necessarily have that paranoia on day one, of avoiding trading in zero sum environments, but in order to make sure that it happens, putting this constraint of requiring people to tighten their trade is a way of guaranteeing that trades will occur. 

Then what I do is, while trading is still open, I have each person in the room come up and write the number of siblings that they have on the board one by one so that we are calculating the sum of the number of siblings that we have, combined, in real time. Trading is still open and people are continuing to trade, but they're updating their models as each new number gets written.

And this allows me to say things to people like, “Hey, somebody just wrote a seven up on the board. This presumably updates your value for the true sum of the number of siblings in this room upwards. What is the first thing you want to do?” 

Usually the first thing that people in the room do is go, “whoa,” and then they look at the trades that they've already done and try to figure out how much money they gained or lost as a result of that. For this, I chastise them. I say the absolute first thing you want to do is going to be orienting toward the order book. You want to be staying out and clearing all the orders that you have that might be stale, even if you don't remember whether or not those orders are good, even if you don't remember what side of the order book they're on. 

The first thing that happens is, you have new information that markets are different from what you thought, and the fastest thing for you to do to protect yourself is to be out on the stale orders of yours that are now stale to new information. 

The next thing you want to do is to go in the direction of the new information's indication. And you want to be doing this to approximately the right order of magnitude.

In terms of how much you move the price, if you see a seven, that's a big surprise relative to a one, which might be your expectation of how many siblings someone has, such that if the spread had been one wide or two wide, you should be happy lifting the offer even if you weren't paying attention beforehand to what your model says the exact sum should be.

Patrick McKenzie: And explain the jargon. Lifting the offer means I will buy at a price larger than the cheapest offer to sell right now, and transact with that person. And I think you might have a constraint in this first game just for simplicity that you can only transact with one person at a time?

Ricki Heicklen: That is true – usually I will relax that constraint by the time we start writing the ages on the board.

The flow of this class will usually be that, as people begin trading with Tighten or Trade, there's a whole bunch of things that they don't know about what will happen with their orders. That is okay. The important part is that we are immersing ourselves in trading directly, because it will be a faster way of learning it than by going over all of the rules of trading and all of the things you might want to pay attention to, which certainly can't all be enumerated even just in that first class, and instead figuring out reactively, as we execute these trades, what should happen.

Patrick McKenzie: Can I give an example here of something a bright high school student is going to learn after 30 minutes that eludes financial reporters, members of congress, and financial regulators?

[Patrick notes: Of many, many artifacts I am subtweeting here, see the NYAG report where a New York regulator uses subpoena power on the crypto industry and identifies complex order types as the problem. I was flabbergasted.]

Ricki Heicklen: Mm hmm.

Patrick McKenzie: When you are quoting markets, you will cancel a lot of orders, and that's okay. A thing that surprised me when I wrote my first bots to market-make for Starfighter, which was necessary to give other people things to trade against, was the bots kept canceling ~98 percent of orders.

I know I'm not doing that for bad reasons, because I can see the code in front of me, and I'm just trying to make the stupidest thing that works. Then I read in congressional reports later, “Oh, these high speed traders are doing obviously evil activity by putting orders on the book that they never intend to actually fill, and we measure it and they cancel 95 percent of their orders.” 

[Patrick notes: Let me quote a report to Congress: “ One criticism relates to its generation of so-called phantom liquidity, in which market liquidity that appears to be provided by HFT may be fleeting and transient due to the posting and then almost immediate cancellation of trading orders.”] 

I'm like, that doesn't seem obviously evil to me. That seems like the natural way that this works. And you're giving the explanation why at the speed that we perceive updates to the world or updates to the market, we should be canceling everything and then putting new stuff on.

Ricki Heicklen: Yeah, I think the conservative way to approach things is anytime there's new information and your model shifts, you should be extra paranoid about the orders that you have that are still posted to the book, and you should be rushing to clear those orders, or to say out on your orders so that they're removed from the order book, in particular because of the concern we talked about earlier of adverse selection. 

If you still have orders on the book, let's say those orders are good to the fair value, i.e. you would be happy for them to be traded with, people are not necessarily going to trade with them because they don't want to do bad trades.

But let's say your orders are stale in the direction of being bad. Somebody is going to come in, see that, and trade with it, if they can do that faster than you can clear your order. It is more efficient for you to clear your orders, than for you to recalculate what you think the new fair value is based on having now added in that person's seven siblings, subtracted them out from the number that you multiply by your expectation of the average number of siblings, make sure you've counted how many people have already written, come up with a new number and decide if your order looks good to that number. 

A lot of the thing that I'm trying to convey in the trading curriculum as a whole is that, to be a good trader, you don't necessarily need to be the person to get the exact right number after many minutes of painstaking equations and double checking every single odd constant that gets added to the end of that equation.

You need to be the fastest, you need to be going in the correct direction, and you need to have some sense of approximately how much you think the price of this asset should move, or how much you think the price of this stock should move. Those things need to come first because if you are the first trader, it is possible that you will get a good trade. If you are the 10th trader, it is way less likely because someone of the first nine traders was able to do the overwhelming majority of the good trades and take them out from under you. 

You are looking to maximize in dollar terms to make as many dollars as possible, and in order to do that you need to be fast. You need to be fast because, if you are going slower, it is more likely that your model will have mistakes conditional on getting filled, even though your model now feels like it is so much more well thought out and more likely to be correct, if you weren't then also conditioning on that fill.

Patrick McKenzie: Yep. So, one of these first lessons that you’re teaching people on level one of this game is that adverse selection is a thing and they need to be really, really paranoid about it. 

In Stockfighter, on level one there are noise traders – non-informed traders – that are buying and selling at – we'll call them random numbers, they weren't entirely random but we'll say that they are random numbers – and you are trying to quote a market for them. The primary pedagogic purpose of that was just being able to teach people what a spread is – they're guaranteed to make money here quoting. 

On the next level, we're going to put in robots that are trading in the market that have more than zero information about the true value of this thing, and you need to start quoting your spreads tighter as you get more information from the market.

I think one of the things I love about your game design is that there's almost an explicit uncertainty bucket, where everyone has their own mental model of the world when the day starts, but there is some uncertainty in that mental model, and that bucket gets drained over time as people go up to the board and put up the number of siblings they have. (I have a high score on that one, anyhow.)

Ricki Heicklen: Same. I'm the one with seven siblings, which is part of what motivates this example.

Patrick McKenzie: Yep. So every time someone goes up and puts a number on the board, you learn a little bit more, but you also have an implication that there is less new information that's going to come after me, and that's another lesson that is easier for bright people to pick up on themselves after actually doing it than it is to say, “by the way, every time you get more information, there's less information in the world that you don't already know.”

[Patrick notes: This is a useful intuition to have for people who say “Bayesian reasoning” a lot. We also see it directly in markets: prediction markets for the presidential election will necessarily converge on election night as more districts report totals. In equities, uncertainty about a quarter tends to decrease as the quarter proceeds and facts about the world become known by the firm and external observers, before decreasing sharply–but critically not to zero!–when the firm publishes quarterly audited financials.]

Ricki Heicklen: That's absolutely correct. Another feature of this game is, usually by the point that we have people start writing the number of siblings they have up on the board, we've relaxed the constraints of Tighten or Trade, and people are now allowed to clear their orders from the market. 

In fact, it is prudent for them to do so.

As a result, we want to incentivize people to trade and not just have liquidity entirely dry up in the market when there isn't enough trading. We can do this by adding in naive customer flow: every minute we will flip a coin; if it's heads, we'll lift the best offer, i.e. buy from the best offer. If it's tails, we will sell to the best bid. And this will incentivize people to tighten those spreads because they will be competing with one another for that top position, so that they'll get to do trades with what is obviously explicitly an intentionally naive customer flow, uninformed trades from the coin flip bot.

Patrick McKenzie: Yep. So this brings up a topic which you haven't really addressed yet that is relevant to market microstructure called order types. Professional traders will virtually never express an offer, but naive traders just say, “I like Shmapple, I want to buy this stock. I don't care what the price is.” They will put in something called the market order, and a market order transacts with the best price available on the sell side of the book. A buy order that will try to transact with the best possible price on the sell side of the book, regardless of what is happening in the world, the current size of the spread, etc.; they are saying, “I'm willing to pay the most amount of money for liquidity, I just want my fill.”

Market makers love when people send in market orders because it almost announces, “Please charge me the most for this,” though you're not in most cases allowed to choose whether you interact with market orders or not

Ricki Heicklen: I think your point is correct. Market orders are much more likely to come from naive customers. This is because a market order is strictly dominated by a limit order with a really high price.

There is some number of dollars above which you would not want to buy Shmapple stock. If you specify a limit order with a limit of $200,000, that's strictly better than a market order because any cases where the market order would trade and the limit order wouldn't are cases where you should probably feel really, really sad, in that above $200,000 range, to have traded.

[Patrick notes: Matt Levine reported a recent loss for Interactive Brokers following a large number of retail customers putting limit orders in to buy Berkshire Hathaway stock, which caused them to be filled not at the (clearly bogus) price they wanted, but rather a more realistic price which was nonetheless clearly erroneous. Interactive Brokers took the loss in the interests of facilitating an orderly market.]

Because of that, a market order is more likely to be a naive customer. However, I think where that point is less relevant is that you don't often have access to the information of something being a market order or the ability to select preferential trading with them. There are many structures, including many auction mechanisms, that will prioritize those market orders in terms of who you trade with: a willingness to buy at a higher price than other market participants should allow you to get some kind of priority in a case where those orders are being aggregated and then executed all at the same time, and this should be a reason to make people who are providing to Order Flow, i.e. market makers, in an auction more inclined toward wanting to do so. 

But I do think this fact about market orders is a really useful fact for purposes of making the decision not to send them, and less useful for you as a market maker, because you’re just not able to take advantage of it in nearly as many cases. Does that make sense?

Patrick McKenzie: Yep. All that makes sense.

One of the things that Stockfighter focused on is teaching people that there's many types of order types available that have different physical characteristics when they interact with an order book, which is unaccountably to me described as weird magic that is probably being done by the financial industry to abuse people in various regulatory reports, but it's actually quite straightforward.

Another type, for example, is fill or kill. Can you just explain in a nutshell to people what fill or kill means? 

Ricki Heicklen: Sure. When someone says a fill or kill order, they're basically saying either immediately fill this order for me if there's an order that would take the other side of that trade in the book, or kill this order – don't leave it up. This is as distinguished typically from limit orders, which are orders that if you send them into the market, will stay on the order book and other people can come and transact with them later. 

One reason you might want to do this is because of similar to earlier, the concern about adverse selection, in which if you leave an order up, it will get traded with in cases where it is stale to new information that comes out, or cases where other people see that order and judge it to be bad. 

Fill or kill isn't totally immune to the problem of adverse selection because it's still true that any order it gets to trade with is an indication that somebody else is happy trading with you, even after conditioning on your choice to trade with them and has therefore put up an order that would trade with this. But it is a lot safer in terms of not having the problem of stale orders out on the market that then get traded against.

Patrick McKenzie: Another order type is “all or nothing” where, similar to fill or kill, you want to know the results of this trade immediately. All or nothing adds one additional constraint. For example, if I'm buying 100 shares of Shmapple, and I really like that number 100 – I want to end this order with 100 shares of Shmapple exactly and not, say, 81 shares and then the order is done, like it would be in the case of a fill or kill – so I say “give me all the 100 shares that I want or cancel this order immediately and in no case put it on the book.”

And now in less than two minutes, you have learned more than many financial regulators in the United States! (sighs)

This stuff is far less magical than people believe it to be. It's far less like there is some secular priesthood that exists in New York and Chicago that has secret knowledge available to them. All this is in documents you can read online, and it can literally be explained to, again, bright high schoolers.

And yet people think it's magical. But the implications of it – that can use teams of PhDs working for the entirety of their professional careers to grind out one more basis point.

Ricki Heicklen: Yeah. And I think that the important feature here isn't that this information is theoretically inaccessible or that it is kept secret, but instead that there is nobody out there who both has a really good understanding of this information and is incentivized to take that and teach it to you and in particular to teach it at scale such that a lot of people can learn it, because of the constraints on ways that teaching it through doing it necessarily require a large number of teachers relative to a small number of students in order to convey that information. 

Now the way that we got to the previous topic was saying, sometimes there's naive customer flow. In the case of the coin flip, it is very clearly, explicitly, and intentionally marked as such. Often, the signals that flow is naive might not be as accessible to you, a participant in the market, and a lot of what our curriculum does is gradually make those signals more and more subtle.

It's not, “we are announcing that we're flipping a coin in 30 seconds.” It's, “we are giving you some historical data about the behavior of bots in a market with various different financial products – you need to look at that data and see if you can figure out which bots are behaving intelligently and what they're doing.”

Patrick McKenzie: I love this because we had to build a very similar thing for Stockfighter to fake a history of – we randomly generated companies. And so you might have Amalgamated Dwarves of Moria Mining Incorporated which obviously doesn't correspond to a company in the real world. (Or does it?)

But we have to make a plausible history for Amalgamated Mines of Moria. So how did you make a plausible history for bots trading in whatever contracts you do in your training?

Ricki Heicklen: Great question. The way in which I make a plausible history is by modeling it as closely as possible on a plausible present, and the best way to do that is by actually just having it happen in the markets. While people are building their trading bots, we turn on the market maker bots and the naive customer flow bots and the intelligent, sophisticated hedge fund manager bots.

They can see the behavior those bots have in markets, albeit for a much smaller size than we'll end up turning it up to during the practice round. They will gain that information in real time, including how they're interacting with it and how it responds to their interactions with it. So instead of just pouring over market data that they won't necessarily know how to extrapolate from, they can engage with that information directly.

Patrick McKenzie: I love the way that different people have different approaches to this problem.

I never said this publicly when Stockfighter was running because it would trivialize all of the challenges and then some, but for Stockfighter, Thomas and Erin and I discussed many ways to do plausible random walks about simulated companies – but why do a random walk when history already has them for me?

So I grabbed from Yahoo Finance the history of the S&P 500 closing prices on every day for the last 15 years. Every time there was, say, Dwarves of Moria, I would randomly pick a particular year and a particular company – e.g. for the duration of this this trading session, at a vastly sped up track timescale, the true value of dwarves of Moria is tracking the realized performance of Bank of America, 2007. That let the game feel quite fresh because sometimes Dwarves of Moria is a utility stock and it trades in a not exactly wildly varying range; sometimes it's a stock that is quite cyclical, and sometimes you're riding Bank of America in the middle of the 2008 financial crisis and you get to have a lot of fun. 

Another thing that we did to turn that “platonic true value” of the stock into actual trades by the bots, is some bots got to actually look at the true value – for some of them, we gave them a better approximation of the true value; for some of them we put a wider error range on the look that they got on the true value. And then other bots had a strategy which didn't really depend on the true value. They were just buying or they were selling. Or, in some cases there were bots that really hated each other and just were doing the thing that the other bot least wanted to happen, regardless of what other more informed bots in the market were doing.

Ricki Heicklen: I'll tell you a bit about why we don't do that – why it is that in our trading curriculum we explicitly model the movement in stock prices as random walks for some of the stocks that we're putting on the market instead of using actual historical returns. One reason is because it's simpler and when you're in the early phases of teaching somebody about trading, keeping the API, keeping the parameters of the trading game much simpler will do a better job at inculcating those first order lessons than something that also incorporates a bunch of noise that comes from, you know, the various other things that might be happening to Bank of America stock in the background that they won't be able to anticipate.

It's also simpler for purposes of reviewing and teaching it and saying, “your model doesn't need to be taking into account all of these other complicated things, but instead should be interacting with this product in this way.” We are very explicit up front about what algorithms will determine the prices of these stocks as they move forward.

The other reason for this is because part of what we're trying to teach is about the relationships and in particular the ability to do arbitrage between different markets. One thing we have going is we have three different stocks, call them A, B, and C, that are each determined by a random walk that happens continuously over the two hour trading period. We then have a separate product, ETF, which is worth exactly the sum of stocks A, stocks B, and stocks C, but with different bot behavior happening in that market, such that, as you see the prices move in A, B, and C – and those prices will get moved by sophisticated trading bots that know the true future value of them as determined by just the number output by the random walk function – then you can take that information and go and move the markets in ETF and convert shares of A, B, and C into shares of the ETF and vice versa.

Patrick McKenzie: Do you use a sort of authorized participant model for that, or as just anybody playing can do this?

Ricki Heicklen: Anybody playing can do whatever trades they want to do and can also do whatever creations and redemptions of that ETF that they want to do. Sometimes we'll create enforcement mechanisms like, “you need to have a certain number of shares in order to do this conversion,” or we'll have a small fee to do that conversion or redemption in order to help people incorporate the calculation of that fee.

But the very first order, early on in this bootcamp thing that we do, is as simple as possible and doesn't require your pricing model to also incorporate that fee or think about getting to integer numbers and the ways in which those binary thresholds should then change how it is that you value these things depending on what your exact positions are, or how much free capital you see yourself having.

I cannot emphasize this more: keeping things as simple as possible early on is really valuable. That's why using real life historical returns won't give you the ability to lever up the impactfulness of those early lessons – and getting people to the point where they understand the first order things actually takes a lot more time than you might expect it to.

You mentioned that class one on day one is teaching you a bit about the laws of adverse selection, and I actually want to correct that. I think class one on day one is just teaching you, “what is the API for interacting with an order book?” It's still teaching you some lessons as we go – in particular, teaching you lessons about why is it that an exchange might be designed in this way, or what is it that you should be worried about paying attention to; how much magnitude there is in terms of contract size, but it's also just teaching you, “what does contract size mean when there are two orders out there?” Why is that equivalent to there being one order and then another order at the same price as opposed to meaning, say, “for 35 dollars I'll buy the package of two shares as opposed to two shares each for $35.” Why does that make sense in the context of the Exchange’s API to this order book? 

(Day two is when we start teaching adverse selection.)

Patrick McKenzie: I realized a few seconds ago I said something about authorized participants, and so for folks who don't know how ETFs work: You've probably heard the advice that you should only invest in diversified index funds; an ETF is a legal wrapper, which is essentially a tradable diversified index fund in most cases.

So you have perhaps SPY, which is the S&P 500; there is a price for this, and that price says that you have some weighted basket of stocks that the ETF represents. In this example, it's just a one to one to one basket to make the math maximally easy. And what happens is there's people in capitalism called “authorized participants” who can go to the ETF issuer and say either, “I have brought you a basket, please give me one of your ETFs in exchange for this basket,” or, “I would like to redeem my ETFs or some size of that, for the basket of the underlying.” They do that to arbitrage out the differences in the price of the constituents of the basket and the basket sold as the ETF. 

Thus, you can be reasonably confident when you are trading ETFs that they tend to track the market's best estimate of the net asset value of the things individually without having to think much more about that. As a person wants to just consume the ETF as a product, you, it would be a terrible world if when you, you say, “hey, I just want to buy long term exposure for my pension fund or for my IRA to the growth stocks in the US market,” that you had to also look into the collection of growth stocks and see, are any of these mispriced right now? 

The ETF wrapper and the constant efforts of market makers that are quoting these things and high frequency trading firms means that the prices are always basically correct in real time. That's what we are spending large amounts of money for.

Ricki Heicklen: That's exactly right. A big part of the role of those quant firms is to bring those markets in line with each other.

They're doing arbitrage between those markets in a way that is causing those markets to reflect the truth, such that people who just want to gain exposure to growth stocks across the market don't need to be thinking about each of those individual stocks. 

Patrick McKenzie: Arbitrage I find is often misunderstood by a lot of very smart people who think they understand what it means but might not have a model that actually fits the market. Can you give us examples of what arbitrage looks like in real life?

Ricki Heicklen: Sure thing. 

I think often one of the sources of confusion is people use the word arbitrage to explain a whole bunch of different things that are not in fact raw arbitrage in the form I'm about to describe. Arbitrage is the act of trading two different products that are essentially the same product, or aggregate to become the same product, in a way that causes you to profit risk free.

What does this mean? Let's say you have two stocks, A and B, and there is a separate stock, SUM, that is the sum of A and B – that might be an exchange traded fund, an ETF, that is the basket of A and B combined. If you can buy stock A, buy stock B and sell this ETF (the sum of A and B) and make a profit by doing all of those trades, you've done arbitrage.

You've managed to engage with the market in a way that allows you to end up with no position. You now have no exposure to A, B, or SUM; when there are movements that happen where A increases by a dollar, you profit a dollar off of having a long position in A, but you also lose a dollar off of your short position in SUM.

And as a result, you have not profited or, or lost out. You have not made a profit or loss in the aggregate of the things you're holding, and therefore it is essentially the same as if you were to successfully create that ETF and cancel out your positions, or equivalently, redeem it and cancel out your positions in the stocks.

Patrick McKenzie: And the way that people in finance explain this, because they want to make it sound very complicated and not like a thing that a bright high schooler will understand in the first 30 seconds of that explanation, is they say, “you're delta neutral with respect to A.” There's a bunch of Greek letters you have to learn.

Ricki Heicklen: Yes, that's exactly right.

One of the ways that we try to teach arbitrage specifically is by creating markets that are still based on things in the real world. We have a crosswords contest in which people race doing crosswords that are up on a screen, doing the exact same crossword – we have the fastest time of a member of team green, the fastest time of a member of team orange, the sum of those two fastest times and the difference green minus orange of those two times.

Then after we've concluded trading, I'll ask everybody to take a few minutes to calculate, to come up with a state of the order books that would allow them to do arbitrage. The first mistake that people make here is they say, “Oh, let's say green is trading for 10, Orange is trading for 14 and some is trading for 26. I could buy the first two and sell some.” 

Why is that wrong? It's wrong because trading at isn't one specific number that you could do any transaction at that number. There is a bid and there is an offer. You need to be comparing the offers in green and orange with the bid in the sum – or vice-versa, the bids in green and orange if you want to sell it and the offer in some if you want to be buying it – and figure out are there any sets of trades that you could do that would be profitable, recognizing the need to cross the spreads. You don't just get to transact at whatever the midpoint is in that market or whatever the last price is in that market. Figuring out which sides of the order book you need to be looking at together to see if you have an arbitrage opportunity is the kind of thing that is conceptually, in theory, trivial, but so, so easy to make a mistake on.

While teaching this class I regularly make a sign error – accidentally think that we're supposed to go in one direction and not the other. Working through specific, concrete examples is going to get students way closer to not making those errors, or figuring out where those mistakes will crop up, than just reasoning from first principles about what you want to do.

It also helps them write up in spreadsheets that are reading in the electronic markets we have. What cases cause there to be an arbitrage and what don't? The thing that I then push them to do is not just check if those models make sense from first principles, but, let's change the stock price of green by one value – how does this push everything going on? Let's change the offer here by one value. How does that push everything going on? Let's say there's this settlement and you've taken on these positions. What is your PNL? 

Walking through different examples of how changing the prices that these values either trade at or settle to, and how that changes your profitability, how that changes the payouts of your positions, does a way better job informing people of the directional pushes and the effects that they have than just explaining from first principles why those would be the case.

Patrick McKenzie: I also think that this does a good job of informing these traders that are learning how to trade of something that people on internet message boards don't yet know, which is that many things that your in snap judgment appear arbitrageable are not in fact actually arbitrageable. Even in toy models like this, where it’s very easy to have a correct model of what an arbitrage should look like, a small amount of misunderstanding that the book, the sign, etc. will suggest that you're wrong. Informed people make that mistake all the time. Thankfully, computers, knock on wood, hopefully make it a little less.

Ricki Heicklen: Yep, that's exactly right. And I think figuring out how to properly compare the bids and offers, the prices you could actually transact at, instead of just your sense of approximately what the price is, is one of the things that is going to help prevent people from missing out on food – prevent people from being overconfident in what they initially identify as an arbitrage.

Patrick McKenzie: Just for backstory for people: there is some true arbitrage. E.g. in the United States, at least, shares of Shmapple trade in multiple different venues at the same time, but every share of Shmapple is legally equivalent to each other and they all settle in exactly the same fashion at the end of the day. In principle, though, they can be trading at different prices in different places. One market might be in San Francisco, one might be in Chicago, and there's some latency between them. That’s a strict arbitrage. 

Something which people often describe as arbitrage, but it's just a little more complicated, is, let's say you have a share of Shmapple in the United States.

It's priced in dollars, naturally, because we price everything in the United States in dollars. But people in Japan love their Shmapple as well. It's just as tasty on that side of the Pacific. So there is a product created in Japan to represent a share of Shmapple, but it is priced in yen.

[Patrick notes: The reverse is perhaps more common to the experience of the typical reader. If you’ve ever purchased “Nintendo stock” and don’t speak Japanese, you likely purchased a Nintendo ADR (American Depository Receipt). A share of NTDOY, for example, represents a claim against a bank for 0.25 shares of Nintendo. Real Nintendo shares are issued by a Japanese company in Japan, are sold for yen, and are therefore difficult to buy for most people who grew up with Game Boys. NTDOY makes something which proxies them very accessible in e.g. U.S. discount brokerage accounts, for dollars.]

People often say, well, if the pricing of those two products gets way out of whack, then there should be an arbitrage ability of buying Shmapple in the United States and selling it in Japan – but you've now introduced another relationship which is between the US dollar and the Japanese yen and that makes life so much more difficult. And you've introduced another, another, relationship which is that the infrastructure available to you to move around dollars, and the infrastructure available to move around yen, have funny physical realities that make that much harder than it sounds.

Ricki Heicklen: Yeah, I want to add one more complication, which is anytime you're engaging with multiple financial products, you are adding additional risk about what might happen with those products. Let's say you buy shares in an ETF and then the company that is issuing that ETF goes belly up or mismanaged their portfolio or reported a number that wasn't actually the correct number, but was in fact reporting someone's error somewhere. There's so many opportunities for error at each piece in the system, including the ones that you assume are true or think it would be impossible to ever be violated, that adding complications to your portfolio in the interest of succeeding at arbitrage diminishes the guarantee that you are making risk free money. 

You are taking on additional risks the more products you're interacting with.

So one of the concerns about arbitrage that I teach is this concern about needing to cross spreads in both cases – this is something I sometimes call transaction costs in addition to the kinds of fees you might need to pay for each trade – but another is the risk of just having multiple different positions, both for purposes of your own internal accounting and ways you're more likely to incur spreadsheet errors, for example, and also in terms of the other sources of risk that come from external factors about why you might think you have a certain position, e.g. that your books don't necessarily match some of your counterparty's books for some reason, or those of the exchange you're interacting with, because there are errors all over the system. 

There are times that one set of trades will get busted or canceled retroactively, and other sets of trades that you've done will not – so even though you thought you had a flat position, a flat delta as you mentioned people sometimes refer to it, in fact, you do not, little known to you.

Patrick McKenzie: So there are these risks that we have models for, but they're not exactly the risks that we are taking on when we express a trade. There is infrastructure built up in trading firms to manage those risks, but one of the basic things that one could do as an individual trader is position sizing – not putting all of your eggs in one basket because if that basket breaks, you're going to be in for a rough time.

Even if you think it is strictly advantageous to take the trade offer to me, you might not take the trade in the maximum size available to you because of that position sizing thing. How do you teach people intuitions about position sizing?

Ricki Heicklen: Position sizing is one of the things that I've struggled with teaching the most because I think there's this intuition of, if a trade is good, you should do that trade for the full size available to you until the point that you've moved the market such that it's no longer good. And again, when I'm teaching things to a first approximation, that's pretty reasonable. You want to do good trades. One reason to do similarly sized trades across a lot of different markets is that you will get better data about how good your trading is and how much you're improving.

Another reason is because you will be better diversified to not lose out against noise for purposes of your own portfolio staying positive, separate from your ability to track it for educational purposes or for accounting purposes.

There's also just the fact that you will be less likely to go down to zero and then no longer be able to make money in the markets if you have size that's spread more evenly between different places. 

How is it that I teach sizing?

I'm going to be honest with you: a lot of the fundamental lessons about sizing are ones that we don't get to in the time that we have available to us. 

We're trying to teach fundamentals that are easy to digest. It's not that sizing isn't important – it's extremely important – it's that sizing is just a little bit too complicated to be able to successfully teach how to do successfully, especially cause you have two different questions.

One is a question of the impact that your trades will have and how they'll move the market in terms of optimal sizing if that's the only opportunity available to you, and another is sizing in terms of the relative values of different parts of your portfolio and how you want to keep it steady in light of like different things changing.

Patrick McKenzie: Trading is a very deep domain, just like programming is a very deep domain – no one comes out of CS undergrad and is ready to write every program in the world. That's one reason that after going through training, people go through some assisted structure with working in a firm, both in some sort of mentorship relationship with people, perhaps something that looks a little bit like an apprenticeship, while also having guardrails put around them to prevent e.g. the things about sizing that they don't know yet from blowing up the entirety of their firm, just like we have junior engineers who push code every day in the in software companies across the world who cannot take down the entire internet with a bad code push, there's been automated systems that have been built by people who have been there, done that and gotten the t shirt for it which will test those changes and reject them if it is the kind of thing that would take down, large systems over the entire Internet.

Ricki Heicklen: Just to re-articulate, I think when you ask about how to teach sizing, there's two different important things to be teaching on how to size trades reasonably.

One is for a specific given trade, if you were only optimizing the decisions you were making as they pertain to that trade, that was the only opportunity you had, what is the optimal size to do that trade for purposes of like your expected value of its performance? That's the kind of thing that we'll often teach through having these liquidity-providing bots or having bots that are acting like naive customers in a market that will trade up a stock a certain amount such that you can give people kind of this formal equation, and we try to keep those as simple as possible of how much it'll move the market. Then people can figure out based on how much the market in a few of the stocks moves, how much should the market in the exchange traded fund move and therefore how much should they trade it up in order to get it to that point.

Likewise, in terms of arbitrage between the ETF and the stock, you should figure out which of those legs of the trade will be more constrained, which one has less size available for you to take on, and have that be your cap on the size that you take on in those two markets so that you're not sitting with two trades that would have been good if you could have done them for arbitrary size but now have much more size on one of them than on the other.

That's as it pertains to a specific trade. In terms of the question of sizing as it pertains to sizing your different positions across lots of different markets in ways that are reasonably balanced with each other, that level of sophistication is often beyond the scope of a two day trading bootcamp, or even when I do the longer version, a 10 day trading bootcamp for high school students.

It will be the kind of thing I will touch on insofar as it pedagogical benefits for people to have lots of different positions that allow them to track performance with less noise getting in the way of the performance of their trades, and i'll talk a little bit about why diversification is important for purposes of not having a portfolio that can easily drop to zero and take you out of the pool. 

But the question of what exact equations you want to use to result in sizing between different positions based on how much money is to be made in those different markets, and also on the fact that you want to have some balance between them, tends to be one step more complicated than the things that we end up getting around to covering in this curriculum. I think it's a hard thing to teach and it's important to teach well, and you shouldn't start trading with your own money before having a good understanding of it, but it is it is interesting to me that it is harder to teach than some of the other concepts that I do manage to cover, like adverse selection and movements in price or how much you want to trade toward that price.

Patrick McKenzie: Stepping back for a minute from what you're teaching to people to the general design of a business of teaching to people: presumably you're not teaching high school students such that they can graduate from their two day boot camp and then open up Robinhood and actually trade Shmapple. What are you attempting to leave high school students with?

Ricki Heicklen: Yeah, that's a great question. I'm trying to leave them with two different things. Number one is a sense of, do you like this thing? Do you maybe want to be a quant trader? Would you enjoy thinking about these questions a lot more and going forth and doing them? That's something that inclines me very far toward the educational end of the spectrum because I'm trying to give them a flavor for what quant trading is. I think a lot of people who are very knowledgeable in domains that range from software engineering to math puzzles to history have just no idea of what it is a quant trader does, and in particular, what kinds of thinking skills and tools a quant trader ought to have, so even getting people to the point of understanding why heuristics are so important for trading, why speed is important for trading, and what kinds of things to be paranoid about or to pay attention to in markets, will get them a lot of the way there in terms of just having this general domain knowledge that can inform whether it's something they want to dig into more.

My other goal is to teach them about the kinds of heuristics around trading that can inform them whether or not a specific area or trade is one worth then investing a lot more effort into thinking about. 

Adverse selection is a concept that's trying to teach you to be more paranoid about your trades. Information about naive customer flow is teaching you that despite adverse selection and other attributes of more sophisticated market players doing better than you, it is still worthwhile sometimes to do trades if you can identify the good ones. And, when should you have a story for why it might be good to trade, such that you then invest a lot more time thinking about that trade, and trying to understand whether your story for it is true or not? 

I guess I'll add a third thing which is that, I think that a lot of the features in financial markets and the things that I'm trying to teach toward crop up in lots of different places in life – adverse selection being one of the biggest examples of this – in ways that people are not necessarily paying attention to, but once you've been a certain amount trading-pilled, once you've gone through this curriculum, you'll notice a lot more and be able to incorporate it into your ability to ascertain whether an environment is more cooperative leaning or competitive leaning – whether an environment is high trust or low trust – and how to set up incentives and agreements and contracts so that your environment can either be more safely high trust or more legibly low trust in order to cause people to make choices and do things that will be positive expectancy for them, and ideally, in the world I'm trying to create, positive expectancy for everyone by giving us the tools required to determine what places are low trust, and how to make places high trust so that we can cooperate and not burn the commons on values that might be good for all of us.

Patrick McKenzie: Absolutely agree with that. (And goodness, I hope to never actually be making time sensitive trades in my life – if I am, I'm doing something probably grossly wrong.) But the experience of trading in simulated environments, building a simulated environment for other people to trade in – I still wish Stocktrader worked out as a business – has given me a lot of mental shortcuts and models that apply to other areas in life and cross play in very, very vivid ways.

[Patrick notes: The conversation picks up on a new thread here.]

Patrick McKenzie: So in the final level of Stockfighter, level 6, we have in the game 100 bots running a variety of strategies, and we've built up intuitions for how the various bots work over the course of the previous five levels.

There's one novel bot who is using simulated insider information to trade. So a real insider – someone in the business who probably should not be trading their own stock and definitely should not be using insider information to trade their stock – might be in the accounting department of a company and they can see the quarterly numbers before the market gets released the quarterly numbers on the earnings release date.

Earnings releases will typically move a market in one direction or the other. If you can see the earnings release before the rest of the market sees the earnings release, you will know what direction it will likely move the market in, and likely be able to take trades which will pay off once the actual release happens.

The way we simulated this in the market was we had a world simulation of the true value of the stock and some bots had better approximations and some bots had worse; the insider got the true value of the world simulation, and 30 days before everyone else did. It was able to make not very sophisticated, but obvious trades based on that information.

And the task we gave players was, okay, there are a hundred traders in this market, you have access to the order book, find out who the insider is. How would you go about doing that?

[Patrick notes: Ricki’s answer here was delivered in real-time after thinking for approximately 15 seconds. Less than 100 of 50,000 technologists, many of whom strove for several hours, successfully implemented any of the four ideas she came up with on the spot.]

Ricki Heicklen: Great question, and I love this exercise. It feels to me like a good simulation of what a lot of interacting in real financial markets is like, in that you will have some participants more sophisticated than others, and identifying which ones are which is especially important.

I think the first thing that I would look to is, if I can see the behavior of the different entities with names attached to them – this is not going to be true in many major financial markets in the real world, but might have been in your simulation – I'm going to look at which ones are kind of always moving in the same direction as the one that the earnings report moves the stock in, in advance of it.

I'm going to want to look in particular at ones that do these trades very shortly before the earnings report is released because it is likely that they will want to focus their positions there so that there's less noise, so that they are less susceptible to effects of noise that happen over the course of 30 days.

But I'll also just want to be applying a filter of, do they go in the same direction as the stock ends up moving during the 30 day period prior to the earnings release. 

I'm going to want to look, if there are lots of different financial products available in this market, at the highest-leverage ones to figure out where it is that those traders are putting most of their efforts.

So if you see that buying, say, out of the money call options on companies that then move up a bunch – 

Patrick McKenzie: This is the classic Matt Levine point. If you're going to do insider training, don't do it with very out of the money call options that are expiring this Friday.

Ricki Heicklen: Yep, and nevertheless he keeps gathering more and more examples of people behaving in this way. Of course, he gets to see the examples of the ones who get caught and not the ones who don't get caught, but…

Patrick McKenzie: We as society are being adverse-selected when we see the results of the legal process. We are finding the dumbest crooks.

[Patrick notes: For a deeper discussion of this, see my conversation with Tyler Cowen. We detect a surprisingly small amount of malfeasance in traditional finance and, in narratives of the malfeasance we detect, the perpetrators seem to mostly be nincompoops.]

Ricki Heicklen: Correct, and therefore less likely to figure out from that how to protect ourselves against the more sophisticated ones. However, it is still true that some bots, some individuals, some insider traders will behave in these ways. So looking first at the products that are the highest leverage for making money in this way feels like a very reasonable initial starting point.

And finally, if you have a way of tracking the profits and losses of individual entities, again, if you can track the individual agents’ trades and what their balance is at each point in time, the ones that see the biggest bump following an earnings report and whatever ensuing market volatility takes place on account of that earnings report will likely be the ones that you should be paying more attention to.

A more sophisticated insider trader might make some deliberately bad trades in there in order to throw you off the scent, might go in the opposite direction at certain points in time, but to a first approximation, those are the kinds of signals you're going to want to be filtering on to find the insider trader in this market.

Patrick McKenzie: So I also loved this level and I love your thoughts on it. We had somewhere between 60 and 80 people successfully solved this level and asked each of them for a write up on what their strategy was, and you identified a number of the strategies that people came up with off the top of your head, so congratulations! Again, most of our folks were amateurs at this, and you've actually been a professional, but the first phase of the level was, it's really helpful to be able to see who is making which orders on the order book and the order tape. 

The order tape, by the way, is just a feed as orders get filled that gets broadcast to all participants in the market if they want it.

Our simulated stock exchange did not allow you to see that, if one was using it legitimately. But as we are about to discuss, sometimes you can get illegitimate sources of information, and so we asked people to find a vulnerable internal API. Well, we didn't ask them – we just gave them the opportunity to find the one API endpoint that would disclose participant IDs along with the other details of their order. And some of them found that and then didn't give names, you would just have IDs that were effectively anonymous to you but you could correlate which trades belong to which person and make inferences. 

So many interesting things came out of it. I could talk all day about some of the solutions. One was formatted in latex like a journal article. People put a lot of effort into these things. One of my favorites was simply, “my task here is not to make money anymore. My task is just to catch the insider trader. I will use myself as bait and see who transacts with me the most successfully.” That's the bad guy.

Ricki Heicklen: I love that. I love that in particular because if you can do that with a small amount of money and then successfully identify them and then either follow their trades and trade in the same direction, if you can demonstrate that doing so would still allow you to profit if they hadn't sucked up all of the value there, then you'll have an aggregation of trades that make money in expectation.

Now, that doesn't necessarily mean that's legal to do – you're going to want to check what the regulatory conditions of the market you're in are – but insofar as that abiding by the rules of the game you're presenting to them, I love that idea as a way of approaching it and a way of figuring out what a market's like.

As I often tell my students, the best way to figure out what trades will cause what effects are by doing those trades for a small size and seeing what happens as a result of them. And that will save you so much time of doing so, in particular for things like catching your own mistakes in, let’s say in arbitrage land, where you do what we jokingly call “garbitrage” of going in the exact opposite direction of the arbitrage you intend to perform. 

This is something you will catch better by just trying and failing to do the arbitrage you want, and why you should always turn on your bots with, you know, a one hundredth of the size that you would want them to actually end up trading with.

Patrick McKenzie: This is why for pedagogical games, I love the idea of being able to make a fiction in which things that in the real world are unethical, immoral, illegal, physically impossible – and say, okay, let's develop some intuitions for you in a world where hacking the stock exchange is an option literally available to you. Not plus-EV in the regulated markets in the United States… it apparently worked in crypto for a very long time, but be that as it may. 

You develop intuitions about, okay, in a world where I can hack the stock exchange, “oh man, I would be so much more dangerous.” And then, okay, so what are the things that are not quite hacking the stock exchange, but do leak information between participants – and given that I now know, that if someone is able to sort of read my hand to correlate what I'm doing, what strategies I'm trying to run in the market, with their own trading activity, how to avoid sending those signals such that other people would read the signals in this iterated game.

Ricki Heicklen: Yes. I'll pause for a moment and say one more sentence about your earlier point about hacking: Let's say, you had managed to patch this hack or let's say you've managed to patch this vulnerability and it was indeed the case that people could not associate trades with certain individual IDs, but all they had was the tape with all trades that occurred.

There are still ways that I would try to detect signals in certain trades that told me that they were made by the same person across different times, and that allowed me to detect that person making trades. One simple example of this is if the person always trades for a certain size or at a certain time of day or with some other signature like that. For example, if they are always taking up to a certain size when it would actually be profitable to take more because they are capital constrained in some way. 

Paying attention to those kinds of signals in markets will allow you to have not 100 percent confidence that they're the same ID, but at least increase your probability of the same ID and figure out how to trade profitably in light of that, even though you don't have full certainty the way you would if you managed to find the vulnerability that releases those IDs directly.

Patrick McKenzie: And this is mind blowing for me because, as the person who developed this challenge, and seeing 80 write ups of people solving it, no one was able to successfully solve it without first doing the security research to de-anonymize trades. But playing back what I know about the behavior of that bot (because I wrote it), it was absolutely possible to solve this challenge without actually de-anonymizing the trades. That’s amazing.

Ricki Heicklen: I’m curious. What were those tell signals? Were there things like time of day patterns that happened? Were they a certain size or constrained to be a certain maximum size?

Patrick McKenzie: So it would have been relatively simple things. A number of people picked up on the simple signals after they had de-anonymized all trades. One person won it by the eyeball test. – They de-anonymized trades and then wrote out the graphs of which trades each entity performed over time and put a hundred graphs on their screen. And were able to do things like, okay, just visually, these 15 bots are running the same strategy. None of them are the insider. These are market makers; that isn't the insider. And then here is a person who trades like buy, buy, buy, buy, big sell, buy, buy, buy, buy, big sell, and those are the only trades he makes. He was like, “I'm pretty sure that that would be a successful insider trading strategy. But, even if I didn't, after graphing a hundred of these, one of these things is not like the other.” 

So, you could presumably see, without de-anonymizing anyone, just the signal that the insider thought in advance due to a fictitious capital constraint about how much he would want to buy, and so he could only buy up to that capital constraint, regardless of whether there would be more or less. And then he would do it in even chunks to disguise that. 

The person who, if his capital constraint was he could only buy 250 shares, always buys in lots of 25s so that no one sees he's getting 250 at the same time – the person who always puts in a lot of little 25 orders, but always sells in around lots of 250, that's the bad guy.

Ricki Heicklen: Nice. That's a great point, and I especially like the point there about how useful graphs can be. I think that a mistake that people often make with trading is they will have a giant CSV with tons of data points, and the only thing they'll do with that is calculate summary statistics, like your t-statistic for how good a certain trade would be, or even read through the tape of like what trades happen in a certain time period, but in a way that human brains are less likely to successfully consume than pictures.

Picture books are so much easier to read than massive tomes of just words straightforwardly, and likewise, seeing charts of this data will be a very quick way of causing you to notice patterns that deviate from the other things you see than trying to read through an entire CSV.

Patrick McKenzie: Yep, absolutely agreed.

So, to segue from this concept of adverse selection into the broader security mindset, how does that show up in one's activities as a professional trader?

Ricki Heicklen: Sure. So, when you're a professional trader, there are a lot of people out there who are out to get you. There's a very strong incentive for people to come and hack into your systems and try to attack you.

And there are lots of cybersecurity vulnerabilities that might exist that I'm less interested in talking about today than some of the more interesting dynamics of human vulnerabilities. Humans are very leaky machines.

We give away information to other people all the time that we don't realize we're giving away. You should model any person who has access to certain information as reasonably likely to cause others, especially people paying attention, to find out some of that information, and you should pay attention to what kinds of information are easier to get from that.

I especially like when a firm or company or organization will do some degree of internal red teaming, internally trying to figure out how you would approach extracting information from another person. Obviously that's informed by what you know from being on the inside of a company, but giving a bounty to external people for red teaming successfully or having some kind of designed project that allows people internally to try to extract information is often one of the best ways of finding where your vulnerabilities are.

Patrick McKenzie: So what's an example in real life of a thing that a trader might tip their hand on without intending to do that?

Ricki Heicklen: Let's say you are a trader who has a lot of information about a specific strategy that you execute day after day. And you're at dinner talking to somebody externally, and they're interested in what you're working on.

Well, you know that you can't reveal the specific details of the strategy, and you know that you can't explain any piece of your code. Let's say you're talking to somebody external. That might be a roommate of yours who happens to also work in finance. It might be out at a bar late at night. You will often end up revealing information, even if you are not disclosing any of the details, much less the code of the specific strategy in question, just by revealing what things you care about: what areas of the market, what countries, what asset classes, etc, you are thinking about. You might reveal this just by mentioning something about that asset class. You might even reveal it just by knowing more about that asset class when the other person brings up a few things with you.

If they talk to you about stocks and bonds and options and you reveal that, “oh, I actually don't know much about bonds, that's not my area of expertise, but I know a lot about options,” they now have a bit of data that there's more profit to be made in options in expectation than they had previously thought because your firm has you focusing on options.

If they can get a good sense of how many people are in each different desk in a firm, or what desks even exist in that firm – if you have a desk specifically devoted to trading certain kinds of commodities, that will tip them off to the fact that those commodities have money to be made in them.

Sometimes the only thing that a competitor will need is to know where your focus is in order to be able to then take five of their researchers and say, hey, I think there's money to be made in Brazilian options, or something like that – let’s put some attention into that part of the ecosystem, and now we too will be able to extract that trade.

Patrick McKenzie: I love this thing that there are things which seem very professionally normative and not leaking inside information at all, which allow others to adversarially reconstruct things that very much are proprietary information – like simply asking a question like, Oh, how many people sit with you? Or, how many friends do you have at work? Or, even like, do you feel lonely at work? “No, I've got seven buddies!”

That plus your LinkedIn profile could already be enough to leak market-moving information to other people. 

An example of this in Techland: For doing formal security research often there is a complicated dance around the disclosure of vulnerabilities; some people have the idea of, “don't don't disclose the specific nature of the vulnerability, just disclose what product it's in” and maybe a very high-level description that won't allow someone else to figure it out. But often, the most important thing is simply understanding that there is one particular thing that has one particular bug class in it, and if you know those two bits of information, a researcher, for good or ill, could quickly reconstruct that research. So the industry has largely moved in a direction where the disclosure of vulnerabilities happens after the vulnerability has been patched.

Ricki Heicklen: That's exactly right. And I think that a lot of people in trading firms are especially worried about malicious actors – people who might take their trading strategies deliberately to other firms and provide those other firms with those strategies, and this is absolutely something to be concerned about. It doesn't even take 10 percent of your traders being malicious, it really only takes one or two of your traders being malicious and then leveraging the amount of information they have access to to another firm. 

But, there are also a lot of these cases of accidental information leakage, in which somebody ends up communicating something to someone else at a different firm, inadvertently, whether through saying how many people sit near them on their desk, or what kinds of products they're thinking about, or even, reactively to other people's questions, indicating what they do and don't know, or what assumptions they make.

One of my favorite examples of this is that the financial industry uses a whole bunch of acronyms. This is because acronyms are more efficient communicators and often clearer ways to express something to someone else. It's also just because acronyms in general are useful.

Many of these acronyms are overloaded, and there are multiple things they can mean. If somebody external to your firm uses an acronym with you, and you assume it means a certain thing and react as though it means that thing, and they previously thought it meant something else, all of a sudden they know that you think about that concept a bunch, and that that concept is therefore relevant to your work because there is some money to be made in that realm or that asset class or that category of stocks.

This is just so hard to defend against. 

Patrick McKenzie: So, things I've seen in real life that have commercially significant consequences without giving away anything. Oh, I don't even know if it's possible to not give away anything now. 

Something as simple as a book recommendation in the context of who is doing the recommendation and when they are getting to that book leaks information about e.g. if it is the CTO at a particular firm who is suddenly attempting to bone up on a particular industry and they tweet out what they are reading about, you know, “I'm very into insurance tech right now,” that should move your estimates of whether that firm is institutionally interested in insurance where it wasn't.

And given the contours of that firm you know, if it's unexpected that they're involved in insurance, that is probably useful information to someone somewhere. The classic example of a side channel for this is, planes are very easy to track on the internet for various reasons that get more into aviation than anything else. Some people travel by private planes, and those private planes are registered to them. If a CEO routinely flies to a particular place that only has one interesting business, it is highly likely that there is a deal happening there, and most frequently that deal might be I'm attempting to acquire this interesting business.

And so, people will get an extremely commercially significant bit of information out of something which is one, legally required, two, absolutely anodyne, and three, does not on its surface look like inside information at all, because it isn't inside information, it's open source.

Ricki Heicklen: That's fascinating. I love that example. Another one that might be similarly publicly available is who somebody is following on Twitter.

Patrick McKenzie: Oh goodness, do I have stories!

[Patrick notes: Sorry, Internets. Sometimes, the price of knowledge is having a deserved reputation for being able to keep secrets.]

Ricki Heicklen: And I'd love to hear them all. There are many Twitter accounts that will tweet out stock advice and people working at financial firms will often follow those Twitter accounts – the Twitter accounts they think are more likely to give advice that is a leading indicator of something that will happen in the market.

If you just scroll through somebody's followers, and you see that there are certain financial advisors, or certain funds, or certain individuals that they're following, whether those individuals are people who say things that are relevant for markets, like Donald Trump or Joe Biden, or whether they're people who are giving advice about where to invest, you'll end up with a competitive advantage because now you know those are places to pay attention to.

Patrick McKenzie: I love Twitter as a product, but there are many, many excellent reasons to not use Twitter. One is that your likes are public – that has caused cancellation of various people for liking tweets that have unpopular opinions. That's one obvious threat model. One far less obvious threat model is, what is the CEO of this firm thinking about in their private moments on a day-to-day basis?

And one can very easily back solve from that to what the firm might be engaged in. Or, is there a person at another firm who suddenly picks up four followers in a row from a particular firm? Very plausibly, a conversation has happened. There's just an infinite number of these side channels.

So what can one do about it, aside from not using Twitter and never having friends?

Ricki Heicklen: So, in the case of Twitter, I think there's an easy solution, which is having an alt or an alternative account that you do your business from, but it doesn't solve all the cases, for example, somebody's public profile quickly being followed by a number of people from a certain industry.

I guess you could also not take private jets, but there might be reasons that those are advantageous to you. 

Patrick McKenzie: There's a lot of corporate shell games done where the private jets are now owned less by the family office and more by Nondescriptively-named Aviation Company LLC.

Ricki Heicklen: I think that these are some approaches, but a lot of these are approaches that patch existing vulnerabilities instead of systematically defending you against those things.

I think in terms of systematic defenses, there's some firm behavior that I think is pretty reasonable that has to do with siloing employees, or taking employees and making sure that they only have access to the amount of information that they need local to the work that they're doing, and not access to every single trading strategy, or how much money the firm is making, because the best way to keep someone from leaking information is preventing them from having that information in the first place.

Siloing employees comes with a lot of costs. Trends that you might notice between different trading desks, or phenomena you might notice in the market that turn out to be relevant to somebody else's strategy won't propagate nearly as easily between different parts of the firm. A mistake that somebody is making is not going to propagate as easily between different parts of the firm.

And two strategies that look like they might be doing different things but are actually doing the same trade and duplicating that trade and therefore in aggregate sizing it too largely, putting on too much size in that trade will not be as easy to detect if you don't have a bird's eye view of everything happening, and if you don't have a lot of eyes on everything happening.

But the benefits of siloing for purposes of information leakage are large, especially the larger your firm gets. First of all, you're going to have a stronger defense against malicious actors who leave, join another firm, and give their trading strategies, because they'll only know one or two of the strategies or whatever they were focused on, and not the entire firm's IP.

Second of all, you are improving how safe your firm is in cases where people leak information inadvertently – they're not going to be revealing as much information about what different desks are focused on.

Patrick McKenzie: So just like how good actors – and I think we can reasonably assume that most high-frequency trading firms are creating value for the world by making markets more efficient, and we'll say they're good people, like every corner donut shop is in expectation a good person – just like good actors leak information about themselves, bad actors also leak information about themselves, and so this is one of the things that you can cross apply from trading to much of the rest of life. 

For example, if one is heavily interested in preventing fraud in payments – and oh goodness, was I heavily interested in that one for a while – one of the things you mentioned earlier in the conversation for busting an insider trader is, “do they tend to do their trades at a particular time of day?” because that might suggest a relatively less sophisticated actor who hasn't done all their trades with an electronic system yet who has particular hours in which they wake and go to sleep and is at their computer at during those hours.

There are not an infinite amount of bad actors in the world, although people sometimes pretend that way, and often they tend to cluster geographically and often they have relatively normal sleep schedules by the standards of the areas in which they have clustered. And so in some cases, not exactly giving away secret sauce the, the timestamp at which an account is opened if it is normal for a heavy computer user in one part of the world, at a time which would be 3 AM for 98 percent of the user base, that's sort of Bayesian evidence in the direction of this account might be controlled by a bad actor.

Another one, which is widely used to my personal frustration. You were talking about knowledge of acronyms and how that conveys something about something you know.

There are two broad types of people in the world who know what a SAR means. It's either people who have worked in the compliance department, or it is fraudsters or money launderers and other bad actors.

I happen to know what a SAR means, but I'm not legible as having worked in a compliance department with respect to my banks. And many compliance departments will tell people in mandatory compliance training, if someone is anonymously well informed about details of the banking industry, and they haven't checked that they are a financial professional in their customer questionnaire, they are probably the bad guy.

And so on at least one occasion, I was at the point of getting a SAR filed on me, which by the way means a Suspicious Activity Report – your financial institution thinks you are up to no good and is about to apprise the government of that fact. And by the way, if you get two filed on you, your accounts are probably going to get closed.

I was at the point of getting a SAR filed on me because I had been asked a question by a banker and, and said, Oh, I can see why that is suspicious, but there's no need to file a SAR on this because, actual thing. And that now they had two bits of data on me: One, I was doing a transaction that they felt was outside of the norms, and two, I know what a SAR was, and therefore they should go forward with filing the SAR. 

This is an example of a maladaptive case of a system that might work out well in expectation for the bank, which is why banks continue to do that and why my life continues to generate so much fun, relatable content for Twitter.

Okay, so thanks very much, Ricki, for taking the time to talk with us today. Where can people follow you online?

Ricki Heicklen: You're welcome to follow me on Twitter – my name is Ricki Heicklen, my Twitter handle is “tradegal_”. You can also find me blogging at Bayes Shammai. I've written there recently about things ranging from the FTX collapse to adverse selection, covering a lot of what we talked about today. 

I also recently wrote a review of the book Going Infinite by Michael Lewis in Asterisk Magazine, which discusses FTX's rise and collapse, discussing some of the mistakes that I think he made in writing it.

Patrick McKenzie: Yeah, that is one of my favorite takedowns of the Michael Lewis book which, oh boy, it's frustrating that he had such a great work in The Big Short, and that then his works have been getting progressively farther and farther from reality since then.

Do you have a quick capsule take on Flash Boys and Michael Lewis's understanding of market microstructure?

Ricki Heicklen: I don't. A lot of Jane Street people have strong views on this – I actually didn't read Flash Boys.

Patrick McKenzie: Okay, I have a strong point of view on Flash Boys from the perspective of, “I've never actually worked in that sector of finance, but I had to make a fake stock exchange”: he just doesn't know what he's talking about. “Flash Boys: Not So Fast” is a book length takedown of this that is available on Amazon – it is a much better book than Flash Boys itself is. 

Anyhow, thank you so much for taking the time to chat with us, Ricki. I'm patio11 on most places on the internet and people can presumably find me if you're already listening to this. I hope to see you next time.

Ricki Heicklen: Thank you so much, Patrick. Really appreciate being here.