This week I caught up with Stephen Grugett, one of the co-founders of Manifold Markets, a prediction markets startup. (I am a small angel investor in it, after having been an avid Manifold user and an off-again-on-again prediction markets user for ~20 years.) We discussed putting prices on the future, some of the mechanics and design decisions, and how markets are better for epistemics than many competing approaches.

[Patrick notes: As always, I write notes in the transcript in this format.

For historical context, this conversation was recorded on June 4th, 2024. You might remember those halcyon mists of history, where we were discussing two elderly men vying for the presidency.

On June 4th, Manifold users assigned only a 95% probability that Biden would win the Democratic nomination. I was one of the (many) people who updated negatively on that in the wake of the debate, buying No for the first time on at 77% and adding on several more times in the 60s, 50s, 40s, and 20s. My reason for these transactions was simple: I'm a Japanese salaryman who knows relatively little about American politics but who is familiar with the concept of coordination games conducted publicly via other-than-strictly-true statements. The market, of course, resolved to zero on July 22nd.]

Sponsor: This podcast is sponsored by Check, the leading payroll infrastructure provider and pioneer of embedded payroll. Check makes it easy for any SaaS platform to build a payroll business, and already powers 60+ popular platforms. Head to checkhq.com/complex and tell them patio11 sent you.

Timestamps

(00:00) Intro
(00:26) Understanding prediction markets
(02:15) Manifold's calibration and performance
(06:48) The role of bots in prediction markets
(10:52) User-created markets and community practices
(15:49) Forecasting, and superforecasters
(18:26) Crafting good markets on Manifold
(23:07) Sponsor: Check
(24:20) Evolution of Manifold's loan system
(26:13) Market participation and capital efficiency
(27:57) The LK-99 superconductor markets on Manifold
(28:57) Social aspects of market participation
(32:31) Building Manifold and user growth trajectory
(34:57) Innovative use cases for prediction markets
(38:52) Prediction market vs. traditional market
(42:22) Play money markets with real cash prizes
(46:14) Historical and regulatory context of prediction markets
(49:32) The importance of market optics
(50:44) Wrap

Transcript

Patrick McKenzie: Hi everybody. I'm Patrick McKenzie, better known as patio11 on the internets. I'm here with Stephen Grugett from Manifold.

Stephen Grugett:  Hi Patrick. It's great to be here, thanks for having me on.

Patrick: Thanks much for coming. 

So I've long been an enthusiast in prediction markets and Manifold is the product in prediction markets that really grabbed me, but it's one of these technologies that is vastly under-understood outside of a relatively narrow community of practice.

[Patrick notes: I became prediction pilled by reading Robin Hanson et al on the Marginal Revolution blog in the early 2000s, and was an avid hobby user of Foresight Exchange for several years, ending when I made the curious decision to stake all of my fake Internet points against a strengthening yen. 20-something me apparently believed that, after several correct calls on elections during my online political junkie phase, I should try my hand at currency speculation. I don’t pretend to know what he was thinking.] 

Patrick: Can you just explain for the audience: what is the prediction market?

Stephen: A prediction market is a place where you can bet on the outcome of events. An event can be anything. A typical example would be the upcoming presidential election: you can create a prediction market on whether Trump will win the 2024 US presidential election. 

Typically in a prediction market, people are purchasing shares that pay out if the event occurs, so if I'm betting against Trump, I will purchase “No” shares in the outcome of that event, and each “No” share will pay out one dollar, or a thousand mana, or something similar.

Patrick: (In the event that he is not elected.)

Stephen: Right.

Patrick: There’s this fun property of prediction markets where the the cost of a “No” share and the cost of a “Yes” share in the same event should sum to (should,* asterisks, there's some funness here, which I'm sure we'll get into) but in principle, they sum to a dollar in many implementations, so you can read from the price a probability estimate - not from any individual trader, not any individual authority, but the market's best sense of the truth, given what it knows right now.

[Patrick notes: People who are new to prediction markets really struggle with the concept of Yes and No shares. I think it is to Manifold’s credit that they largely abstract it away via the UI: your position is always a positive number, just labeled with a direction, and should you simultaneously hold a Yes and a No in the same market they are immediately autocancelled for $1 without you needing to be aware of that. Out of tiny UX decisions like this are very compelling products made.]

[Patrick notes: So what are the fun edge cases I was alluding to? OK, you can extend the general concept of binary markets to e.g. betting on the length of my sabbatical by expressing whether you think the current prediction is low or high, which gets quantized via some magic into several binary bets on underlying buckets.]

Stephen: That's right, yeah. This is why people care about prediction markets.

It's not just a spiced-up poll; it really is one of the best ways of aggregating humanity's opinion on something in general, because when money is at stake, you know, people tend to get serious about predicting things. 

Patrick: Sort of like, empirically, if we expect to be actually good at predicting things in, I don't know, journalism, we might have a written record.

Perhaps we could go back and score a written record against the facts as they arrive. I don't know that anyone actually does that for journalism, but I'm assuming for Manifold you look at historical prices and then look at whether the market was well calibrated or not.

You want to share anything about that?

Stephen: Sure. We at Manifold do publicly track our own calibration. I think one surprising thing is how well this system works with just play money and relatively low dollar volumes in general.

[Patrick notes: Manifold also grades each user individually on their calibration. I get an A, which is profoundly motivating to the part of me that will never, ever graduate high school.]

Stephen: Just having the basic market set up plus a few participants can get results that are pretty good. 

One of the typical findings with prediction markets (and for expert aggregation systems in general) is that typically market mechanisms and other systems outperform both individual experts and the general public.

Patrick: They often, I think, outperform consensus of experts as well, if I'm not mistaken – there's some sort of social desirability bias and similar, which makes the sum of a number of smart people in some ways less added value than you would expect given the number of smart people involved.

Stephen: Yes, I think it's very sensitive to the exact way that you're aggregating expert opinion –prediction markets are a way of aggregating expert opinion, and non expert opinion too.

Patrick: You can do a market on, in principle, just about anything. There were markets on, notably, will Russia invade the Ukraine or not – which are obviously sensitive in one sense but important questions for society – and there were many experts that were pretty confident in their opinion of how life would go down, and prediction markets are maybe better-calibrated at getting early answers on those consequential questions.

Stephen: That's right. It's interesting that you bring up the Ukraine invasion market, because I think this is one of Manifold's first predictive successes.

Pretty shortly after we had launched the platform, we had a number of markets on this question of whether Russia would invade – I think we were above 50 percent like a month out from the actual invasion, despite many pundits claiming that this was totally impossible and far-fetched, etc.

[Patrick notes: NYT, Jan 20th 2022: “A group of Russia experts — including Frederick Kagan, who has advised U.S. military leaders in the past — [believe the troop buildup is a mix of bluff and distraction]”. Russia’s Deputy Foreign Minister, Jan 19th: “We will not attack, strike, invade, quote unquote, whatever, Ukraine.”  Manifold, Jan 20th: 49% chance of war. World we live in, February 24th: Russia invades.]

Stephen: It really felt vindicating for us that we were working on the right thing – and a vindication of prediction markets in general, that even when operating at a relatively low scale, they can still produce useful and meaningful results that can be interpreted by everyone in the public.

Patrick: I'm a relatively minor user of Manifold (also a relatively minor angel investor, full disclosure). I mostly use it myself as a fun game to keep me honest about my crypto doomsdaying on Twitter. I remember last year or so, early last year, sometime after the collapse of FTX, there were several markets made on whether CZ, the CEO of Binance, would continue being the CEO of Binance, and would he get into various forms of legal trouble?

The markets were trading very low relative to what I thought the number was. I made a comment on Manifold saying, “This currently has a 10 percent chance that CZ leaves his employment as the CEO of Binance, which radically understates the likelihood that he would leave under some sort of negotiated agreement to protect the company from legal liability. I'm buying this up to 25.” And then go figure, that did indeed happen and I won $30 of fake money and felt like king of the world.

Prediction markets are very good at tapping my, like, someone's being wrong on the internet (in the XKCD form of things) – and also, a little bit of enforced epistemic humility on making sure that you're not the person that is being wrong on the Internet and just sort of browbeating other people because you have more Twitter followers than them, or a position of social esteem or similar.

Stephen: I think that's a great example. It also shows the power of markets to allow individuals to add information into the forecast in a piecemeal fashion. You knew something about this facet of the law that was relevant to this case. Maybe (for the sake of argument) you don't have views on the overall evolution of crypto in general, but if you know something about this particular case, you can come into the market, and add that information in.

This is one of the benefits of a market-like mechanism versus other forecasting aggregation mechanisms. 

Patrick: And either informally or strikingly formally, you can look at differences between the markets and say things like, “Oh, if if this market says that hypothetically CZ has 60 percent chance of going to prison and this market says 20 percent chance that he leaves his job as the CEO, those doesn't really cohere with each other” – even if I don't have a strong point of view on the actual course of future events with respect to CZ, I have a strong point of view that these prices are wrong relative to each other. I can sell one and buy the other and cause them to converge towards what the true relationship of these two independent but tied situations should be.

[Patrick notes: For much, much more on this, see the conversation with Ricki on teaching trading. Sometimes new information in one market should inform prices in a separate one. ]

Stephen: Yeah, that's right. That's another very powerful feature of markets.

Patrick: And I understand that there actually are some bots running on Manifold that do things like the “arbitrage” and other forms of algorithmic training?

Stephen: That's right – I actually operate one myself. I operate Manifold's house bot called Acceleration.

Patrick: Can you tell the audience a little bit about Acceleration? I have some background knowledge, but…

Stephen: There are still many secrets that I will not reveal… but I have talked a little bit about this in public before. The basic strategy is something akin to what Citadel is doing when they're buying retail order flow.

So, the vast majority of traders, when they trade in marketplaces, are not informed and don't really have a good opinion on where the market price is going to be – so the basic strategy is to first, identify who is not bringing new important information into the marketplace, and then secondly, bet against them and return the market price back to where it was before.

[Patrick notes: Acceleration used to give me action, but has been refusing to do so lately. Crafty little bugger.]

Stephen: That's the very basic strategy. There are many, many details that you need to do in order to not lose money – the most important of which is being very sensitive to exploits and people trying to game the patterns that you're looking for, to spoof the credentials that you're using to try and detect if they're good traders or not.

Besides my bot, there are a bunch of different algorithmic strategies that our users are running as well.

I guess the first and most simple type of bot is a simple arbitrage bot: it's looking for markets that are on the exact same topic – if you have a market on whether Trump will be elected or whether Biden will be elected there's a very obvious relationship between those two markets – and if they get out of whack, there are bots that will bet those back into equilibrium.

There are bots that look at external news sources or other bookies to provide information; Manifold in terms of sports betting is quite small compared to other major sports books, so if you copy the prices from other sports books and bring them onto Manifold, that actually turns out to be a really great strategy, so there are people doing that as well. 

In the future, I expect people will add even more sophisticated strategies. I haven't seen this yet, but I suspect in a year or two, there'll be some true, more complicated statistical arbitrage strategies, which look at baskets of different markets that have some sort of statistical relationship, and then buy or short these baskets in order to make sure they reach an equilibrium.

There are time decay strategies: many events are structured in the form of, “will X happen by this date?”  and as the present movement moves closer to that date, then the probability naturally needs to go down over time – you can set up a bot to perform this pretty simple strategy for you.

There are tons of variations of these things. I think we're probably in a very unusual position among other early stage startups in that we have such an extensive botting ecosystem relative to the overall size of our platform just because of the high caliber of people on our platform today, but I think it goes a long way towards making our markets pretty efficient.

Patrick: For those of you in the audience, if you've never coded a trading bot before, one of the things I learned from my experience running Starfighter was that it's surprisingly accessible, particularly when you're not trying to connect to a real world stock exchange and potentially lose several multiples of your net worth by doing a off by one error – a fun intellectual challenge if you ever want to, and there's just a publicly available API that people can grab a key for and start doing these if they want.

Stephen: Yeah, I think it actually is a good introduction to algorithmic trading.

Patrick: Yep, and that is a fun field to be in, even if you never actually personally pursue it professionally.

Also gives you a little bit more understanding of, what do the likes of Jane Street and Citadel and similar actually get paid for? Oh, it turns out making markets is actually rather difficult. 

Anyhow, so we talked about one market on this sort of big headline news, a societal event, and one on a thing of relatively niche interest with respect to people that are involved in the crypto sphere, but in principle, these can go as far down the ladder as long as there's two people with different opinions on something, right?

[Patrick notes: I have been using Manifold to track some of my longstanding Twitter gentlemen’s wagers (example) or put a price on a news article .] 

Stephen: Exactly. And one of the strengths of our platform is that we allow user-created prediction markets. So if there's something that you are very interested in knowing, or that you have a very strong opinion on, you can create a market on this and see which traders you can attract to come and participate in your marketplace, even if it's relatively niche.

One of our users, for instance, is very into sumo wrestling and created a bunch of markets on this (well, at least to me) obscure sumo wrestling tournament in Japan.

Patrick: One of the things that I've done for a number of years, which is an outgrowth of, I suppose, a cultural practice in our weird corner of the internet is, when you have a severe disagreement with someone on the internet, bet on them for it for epistemic hygiene purposes. 

My typical is the $50 Twitter bet with someone. A lot of my bets are crypto-adjacent; someone was saying Vanguard will introduce a ETF which includes Bitcoin, and I said, “I don't think that is likely to happen in the next 10 years. I'll bet you on that.” 

Patrick: Now that Manifold exists, I can reply to that thread with, “Oh, in addition to the bet, if anyone wants action on this, here's a market on this exact question, go.” (That market has moved in my favor since it was posted… so we shall see if I'm right or not, but I continue to think that it's unlikely before 2030.”

It's also interesting just to have a place to record things that one was confident about at one point to look back to in 2030. In some ways, Hacker News comments and Twitter and my blog keep me honest, but this is legible – I like that.

[Patrick notes: The wrongest I’ve ever been on the Internet? Probably back in 2009, when I confidently predicted the failure of Facebook as a business.

Cryptocurrency enthusiasts would probably point to any number of early 2010s comments about cryptocurrency. I have yet to conclude I was wrong about most of them.] 

Stephen: I think one of the biggest strengths of Manifold is that it does create this public ledger of people's predictions. You can actually track how good of a forecaster this particular pundit is. I think it's just a great practice, both in making people a lot more humble – even if you're a very good predictor, you're still going to be wrong a fairly large percentage of the time – and when you're right, it's helpful for you to know how right you are, relative to the rest of the world, and to figure out exactly where your particular strengths are.

Patrick: I think it's also nice in distinguishing truly contrarian opinions. We have something of a meme in the community that contrarian opinions go a little bit too far; there are going to be places where you disagree with other intelligent, well informed people, and it's nice to know, are you actually disagreeing with other intelligent, well informed people? 

Like, there exist markets where I'm an 80 and the market is a 20, and for bankroll/sizing purposes, I will not immediately buy it to 80 all by myself, but it is useful to say, “Oh, on this one, there's actually a great difference in opinion between me and other people who consider themselves interested in and well informed of this topic.”

Then if I consider myself a bold truth teller by betting 60 when the market is at 58, ehhhh – you're contributing information to it, because there is a difference between 58 and 60, but you're not exactly sticking your neck out there.

Stephen: One of the other big benefits of a marketplace is just having a canonical probability on things, the fact that it's this public number that everyone can see that's out there. I think it's a great way to ground conversations and make sure that everyone is on the same page.

Patrick: I think – and I don't want to put words in their mouths, but – I've seen Nate Silver and maybe Matt Levine point out to prediction markets a number of times for, like, “this doesn't seem likely to happen, and the prediction markets have it at 22 comma, but [yada, yada].”

There is this deafening news ecosystem around us, which includes many people who are strikingly good at what they do – Nate Silver and Matt Levine are both strikingly good at what they do – and there are a number of people who have well-paid jobs at institutions that have a lot of institutional capital who… might not have a positive record if their predictions were actually scored.

[Patrick notes: I have a great deal of regard for Kelsey Piper, and the transcript for that episode includes an important explanation for why I do not treat other articles from a three-letter private equity fund as persuasive evidence about the future.] 

Patrick: Being able to sift out, like, “okay, in this new cycle that we find ourselves in, where there are competing inputs regarding a complex situation which is changing on an hour-to-hour or day-by-day basis, where are we really at right now?” Just boiling that down to, “here's a number; you don't have to believe it, you can bet against it if you don't believe it, but, you know, first best approximation, it's a 36 right now.”

Stephen: Yeah, I think it's invaluable. There is this other factor though, that trading ability and forecasting ability are different.

A lot of the top traders on our platform actually are the “superforecasters” that were part of Tetlock’s forecasting group, but we also have a lot of top traders who have nothing to do with this ecosystem and discovered Manifold through some totally orthogonal venue.

Then the flip side of that also is true: there are top forecasters who are very bad traders, and the reason is that they're not very good at position sizing or dealing with market mechanics.

Patrick: I am better at calling winners than I am at making money off of the winners that I call. 

People who've been reading Marginal Revolution and similar probably know who Tetlock is and what Superforecasting is, but can you give the brief version for people on the internet who haven't fallen down that rabbit hole?

Stephen: Sure, so Philip Tetlock is a professor who studies forecasting in an academic context. He was interested in seeing if there were people who are better at forecasting things than other people, in general. It's not obvious that there would be; things in the future are difficult to predict.

Patrick: Maybe it's just everyone is rolling the same dice. 

Stephen: What he found was that, in a controlled university setting, even in this kind of limited environment, there are people who can consistently outperform other people in predicting – including expert matter. Non-experts outperforming experts in their own given field, because they're using these basic forecasting techniques or principles.

What he found is that the best forecasters out there tend to operate in a pretty similar fashion. 

The first thing he noticed is that they tend to be more quantitative. Instead of saying something is “probably going to happen,” “maybe going to happen,” “likely going to happen,” they'll put a number on it.

If the number is, like, 75 percent, that sounds a little vague – maybe you don't really have a good sense of what precisely you mean by that – but once you put a specific number on it, it makes it much easier to grade yourself against your own future predictions, score yourself retroactively to see how you're doing, and it makes it easier to compare yourself with other people.

Your “very likely” could mean 75 percent, but someone else's “very likely” could just mean like 55 percent. Without that extra position, it's very hard to improve.

Patrick: One of the fun facts I collect about the world is that, in some places in the American intelligence community, there is a list of adjectives and the numbers those adjectives should correspond to – partly because, like, when we're writing the President's Daily Brief and we say we think it is “supremely likely” that something relevant to the security of the United States happens, we want that message to get across exactly versus a roll of the dice with whoever is delivering the message on that day.

[Patrick notes: I probably should have used one of the actual reserved words, like “almost certain” (93%+). At risk of stating the obvious: the intelligence community may not be properly calibrated with regards to things it has said were “almost certain” over the years. To put it mildly.] 

Patrick: The other thing is that, a thing that has been known to happen in the course of human events is that people confidently predict something, and the future happens, and people will look at the previous confident prediction and say, “yep, totally nailed it.”

But people who had previously read that prediction might not agree that the future that arrived is the thing described in that document – so numbers are harder to fudge in that fashion.

Stephen: Yeah, I think it's an accountability mechanism. It's very easy to incorporate weasel words into your predictions that make them unfalsifiable in the future. If you give a definite number, then it's much harder.

Patrick: So most markets on Manifold are user-generated content, essentially.

How do you help the community make markets that are good markets – that, when an event happens in the future, people will know (at the time of the event, and also at the time of participation in the market) that that event will satisfy or not satisfy the market conditions?

Stephen: In general, we try to promote better-crafted markets both on a manual level, in terms of our own curation, and on an algorithmic level. We have the concept of a reviewer score, like your internal score on Uber for how good your driver is – we have a similar mechanism where you can rate market creators in terms of how well they crafted the resolution criteria for their markets and how well they resolved it.

That's a factor. The big thing – I think one of the biggest factors – is people will imitate well-crafted markets. I think for first time forecasters it can be surprisingly difficult to create good markets; people are not used to thinking in terms of clear, unambiguous outcomes.

Patrick: I think getting good at writing Manifold markets is a lot like getting good at writing contracts and people are surprised by lawyers going to pedantic amounts of detail. 

Things like, “will Joe Biden be the next president?” If you express that in a sentence, what does that mean, exactly, in edge cases where he won the popular vote, but uh oh, the electoral vote is a different thing? What if he wins the election and then dies prior to inauguration? Does that count or not?

Being rigorous around those edge cases is part of what makes the markets intellectually interesting, and also avoids lots of ruffled feathers during resolution in the close cases.

Stephen: One of the benefits of having user-created markets is that we can see different creators who have different juridical philosophies.

Some creators will try and spell out in painstaking detail all possible edge cases in the market description; others will try to do the act according to the more common meaning within questions. I think there are iterations of both of those approaches that work well and work poorly.

It kind of is a stylistic difference of which you prefer; there are better iterations of each approach. You might think naively that the approach where you spell out all possible edge cases in advance is the best, but there are some downsides with that approach as well.

Patrick: Yeah. Among others, the person who has some budget of their time allocated to placing this bet now has to do, like, real professional work on reading and understanding what is effectively a proto-legal document – where, aspirationally, they might want just, “heads or tails, yes or no, is is the next U.S. president going to be one of Biden or Trump?”

Stephen: Exactly, yeah. There is a real-world cost to enumerating more details. I think the other danger is that, if you say you're going to resolve to the details that you lay out in your market, if there's some contingency you didn't imagine, then that can create even more headaches because people were expecting that it would resolve according to the things that you laid out in advance.

If you don't do that, in some cases it can be better. A lot of this is still a judgment call.

Patrick: An interesting sort of emergent culture I've seen on Manifold is, in cases where there's some judgment call to be made, sometimes there's a discussion in the comments: the person who creates the market is the person who judges the market currently, and sometimes there's a discussion between that creator and stakeholders in the market on, “okay, the facts before us that are currently reported are this. Indicatively, I think this is a yes now. What do people feel?” 

I remember on one of the CZ markets, the trigger was him being imprisoned, and there was a question of fact of, “he's been convicted, but can I confidently push the button on yes, he is imprisoned now?” I piped up and said, “as the largest Yes holder – so I'm the person who most has skin in the game on you should push the Yes button immediately – I feel like, if he is eventually sent to prison, we will be able to say Yes in the future, but if there is some sort of appeal or something, a No holder could have a reasonable case that he wasn't he wasn't imprisoned. 

And because Manifold has this other mechanism for delaying the time value money question, there's no pressing reason to do it immediately today. We can wait until we have definitive proof. 

It was resolved recently when the moderator of the market went to the Bureau of Prison site and like, “okay, he's on the list now at being at this facility, that definitely counts as being in prison,” and I think all stakeholders of the market would be like, “yep, when the prison says you're in prison, you're probably in prison.”

Stephen: Yeah, that's a good case.

Prediction markets around the “bond” rate

Patrick: Can we talk about that then – the time value of money thing? Manifold has this loan concept, and you probably understand it a little bit better than I do. Can you explain why loans are important to the proper working of the system?

Stephen: Ah, so we've recently eliminated loans as a feature from the platform.

Patrick: Oh, okay!

Stephen: This is semi-recent news related to our moving towards real cash prizes. Our previous loan mechanism in many ways is the perfect mechanism in a purely play money environment. 

There are things you can do in play money that are good, that work well, that you cannot do well with things that involve real money or redemptions to things with monetary value.

Stephen: The way our mechanism worked previously was that every day we internally would tally up all of your open positions, we would calculate the present value of all of your open positions in all of the markets, minus your pre-existing loan amount, and then give you some percentage of that back each day so you could reinvest in the marketplace. 

Over time we gradually upped that percentage until I believe it was like 2 percent. 2% daily is a ridiculous percentage when annualized. I think this had many interesting effects on the Manifold ecosystem.

The first thing is that, when you have a loan system in place, it makes it much more profitable and reasonable to bet in markets that don't resolve for a long period of time. This is the main reason why we introduced this to the platform in the first place – because alternatively, without the loan mechanism: In the U.S., the risk-free rate is around 5%. If you're paying 5 percent per year and you're betting on something that resolves in 2030, then you have to have a huge edge over what the current market price is to make that meaningful in present dollar terms.

Patrick: And another thing that I've noticed in prediction markets for many, many years now – Manifold not quite my first rodeo there – is that the price of participation around the extremes, if you're pushing the market either towards zero or to a hundred, it's kind of disproportionate relative to placing bets elsewhere in the spectrum. If something is trading at 95 right now, and you think “Really? That's, like, all but a sure thing, I'd be a 98,” you have to put 98 cents of capital in and then lock it up for a very long time to express that point of view.

Whereas the short side – it's trading at 95, but I really think it's a 50 – it only has to put in five cents of capital to move the market in the other direction. Because of that sort of efficiency in the use of capital, things that probably should be very close to a hundred don't trade at exactly very close to a hundred, and things that should be very close to zero kind of bias their way above zero.

Loans are a way that mitigates the impact of the extreme cost of capital of moving very likely events or very unlikely events towards the poles.

Stephen: Yeah, that's right. Since we've eliminated loans, what we've seen is that – for instance, one of our largest markets is on whether for LK99, (the purported superconductor material) whether the Meissner effect would replicate in a paper this year.

The true probability of this is very close to zero, but it's trading at around 4 percent today because that's what the risk-free rate is. It does introduce this complication in the platform where you have to consider, if it converges to whatever the risk-free rate is, that's a sign the probability is low and you can't get much additional pricing accuracy outside of that.

Patrick: LK99 was a fascinating social event that happened on the internet. It was this well traded market on Manifold, but adjacent to like a Twitter meme cluster essentially of the, “we're so back” versus… what was the meme?

Stephen: “It's so over.”

Patrick: “We're so back, it's so over.” I was on team “we're so back” at 20.

[Patrick notes: I have no particular expertise in materials science..] 

Patrick: A weird thing is that memes are a way that people create meaning in life.

I think I will be quoting people [when I say], “yeah, I was in LK99 at 20 for a number of years, regardless of whether this comes to pass (which looks very unlikely to come to pass right now) simply because it encapsulates a sense of optimism” – and, like, “consequential frontier research is not over yet” is a thing I want to tell people about myself in terms of my beliefs.

That was an interesting thing I learned from Manifest, which is the conference associated with Manifold, last year. It had never been obvious to me, but someone– a Twitter personality–said, “One reason that people participate in markets of all sorts is like the sports team effect. You are inhabiting a role, joining a tribe and announcing your intentions to the world, which might or might not be optimal for making money.”

That is extremely evident in sports betting markets. We used to think, “but in the world of high finance and the U. S. equities market, there's no such thing as a tribe of people getting together to push a ticker for obviously irrational reasons” – then the last couple of years, GameStop happened, and so we have abundant evidence that that is the thing that occasionally happens. 

Anyhow, the notion of market participation as a fashion statement effectively is an interesting one for me.

Stephen: Yeah, for me, I was on the other side of that trade.

One of my largest positions ever is betting against LK99. When the price spiked around a week after the news broke, I was a little bit worried; I called up one of my physicist friends, and he reassured me that it almost certainly wasn't real. 

Patrick: And just by that action, you contribute more knowledge to the market because you are the person who can actually phone a physicist and get expert information here.

[Patrick notes: In the real world, hedge funds organize “expert calls” at industrial scale. But also in the real world, journalists at prestigious finance publications can’t find a working fax number for Mizuho and so assume that Mark Karpelès is entitled to presumed truthfulness when he says that the Magic the Gathering Online eXchange has brought down wire transfers for the second largest bank in Japan for the last several weeks.]

Stephen: Yeah, but there were a bunch of people like me – including a bunch of people with actual firsthand experience in materials science, which I do not have – who were taking the other side of this marketplace. Even with that, I think the price remained a little bit higher for a bit longer than I expected. 

I think it took around a month for the price to converge to the risk-free rate, which is around zero. 

In any case, I think Manifold, throughout the entire first month of the LK99 frenzy, did provide a dampening effect on public discourse. On Twitter superconducting almost seemed like an inevitability if you were to judge it based on the memes, but if you come over to look at our market, probability’s trading somewhere between 10 and 30 percent.

Patrick: I actually used it in preference to Twitter and I'm a – I don't know if addict is quite the right word, I spend an awful lot of time with Twitter open on my phone. 

The discourse was a little bit overwhelming on Twitter, and to make sense of the noise, I would go to Manifold and see, okay, in the last 12 hours has the price moved, and is there a comment by someone who's explaining their new marginal bet or explaining, “oh, it went from 10 to 20 due to this additional paper, this additional video that dropped yada, yada” – versus, like, competing meme warfare on Twitter, which is interesting and a fun sociological behavior, but maybe not the best way to understand the shape of the world.

Stephen: Yeah, I think the act of commenting (typically after you bet, for obvious reasons), can provide a lot of value. Once you already have your position in the marketplace, it can make a lot of sense to talk your book, and in doing so isolate the actual key drivers of the event itself – and if you're very convincing, the market will follow you. 

Patrick: In the culture that is Manifold and prediction markets, it's okay to talk your book because we're ultimately here to get an estimate of the true likelihood of things. In the culture that is traditional markets and media around traditional markets, talking your book is often very frowned upon and certain actors are explicitly forbidden from doing it.

I've used that as an interesting difference. I do have a tendency of explaining my bets before I make them, and so I hope nobody codes up a bot that reads my comments and then immediately assumes that I go through with the intention announced in the comment and sells into the incoming order. 

(That would make money, guys. If anyone wants a quick programming thing, you would successfully burn me enough to change my behavior!)

Let’s see. What has been surprising about building Manifold?

Stephen: Ah! So many things. I think the general trajectory of user growth has been very surprising in both directions.

One of the interesting things about Manifold is that, immediately after we launched, we had a bunch of traction. We've had markets created on our platform every single day since we launched. We were able to get our start basically through the blogger Scott Alexander; we had applied for a grant for the ACX grant competition, which he was hosting, and once we found out that we were going to win the grant, we scrambled to make sure that we had a product that we could launch on the day that it went out.

Once the announcement went out, we immediately had a bunch of his readership (which are largely Bay Area rationalist/rationalist-adjacent people) up on the platform, and it was great. It seeded our community and we've been off to the races. 

I think after that though, I feel like our growth trajectory just has been moved in very, very weird places. We got tremendous user growth from very unlikely sources.

One of the first big communities outside of the rationalists that we got on board was this streamer named Destiny.

Patrick: I still do not know who Destiny is. Sorry, for people who, like me, are among the unenlightened, who is Destiny?

Stephen: I will admit that I actually didn't know who Destiny was before their community signed up en masse to join Manifold either, but apparently some people in their subreddit found our site. 

The first use case that they used for Manifold was not actually predicting things related to Destiny. It was using our prediction markets as meme stocks – they would just create a prediction market and they would title it, like, “Destiny Stock.” Then they would use the market mechanism to bet it up or down in accordance with their sentiment at the time – but with the price not really grounded by anything.

I think that is one of the surprising things about Manifold: how many very weird use cases people can find for our markets. 

Over time, members of the Destiny community did branch out and actually start predicting both on drama within the Destiny community using a more traditional prediction market format, and, you know, other things not related to Destiny as well.

Stephen: I guess the very extreme and varied approaches to markets are one of the things that surprise me. We've seen people incorporate prediction markets into their dating life, predicting whether their friends will start dating or get married. We've seen people use prediction markets basically as a bounty to get people to go and do their chores or clean up things outside. We've seen prediction markets on past events to postdict controversial things where it's hard to source information directly – you can create a prediction market on whether you'll discover that X about the past.

The use cases just are tremendous. It is a very powerful tool.

Patrick: One example of postdiction might be the lab leak controversy with respect to the origins of COVID: you could put a number on what the community currently believes – “it was a lab leak” or not – or, “will there be a point in the future at which we unambiguously declare that the evidence points one way or the other?”

If I'm understanding postdiction correctly?

Stephen: That's right. That’s a great example. Our market is still pretty uncertain on that, by the way – I believe it's trading a little bit above 60 percent in favor of lab leak after Scott Alexander's review of this debate between Peter Miller and Saar something who runs Rootclaim.

The market shifted in the, in the other direction,

Patrick: The market shifted from a higher probability of lab leak to lower? 

Stephen: To lower. 

Patrick: I recall from that debate, the naturally occurring hypothesis was judged to be the winner by the panel of judges versus the lab leak hypothesis side, which was the Rootclaim folks.

What's another example of a market that is currently active, that for whatever (intellectual or et cetera) reason you're interested in at the moment?

Stephen: Since we were just talking about postdiction, one of the other interesting postdiction markets to me was the one on the Nord Stream 2 pipeline bombing. Immediately after this event occurred, media in the U. S. and Europe declared Russia to be the culprit behind this, so a number of our users created markets on who we would eventually find out the culprit to be. 

This has not resolved yet because there's still no definitive evidence on who is responsible.

But since then, I believe like last year or a couple years after the pipeline bombing occurred, Western sources have shifted towards the view that it's a NATO-aligned person, probably Ukraine, who's responsible for this. Our markets initially, I think, were still still against the Russia hypothesis in the beginning, and then after these additional reports came out have pushed that even larger in the other direction. 

Patrick: I like the notion broadly that it kind of functions as a minority report for the things which are institutionally, you know, “here's a narrative and we're going for it” – responding with a “hmm, really though?” 

Then you can kind of avoid the contrasting pole of just being overly cynical about how government and journalism functions and always pushing the No button. If you disbelieve everything you read in the newspaper, you will have a very disordered life. 

[Patrick notes: Zvi has a very good meditation on this, responding to Scott Alexander. Journalism is a game like Dungeons and Dragons is a game. The game has rules. Sometimes the DM throws away the rules. Sometimes the player treats them as suggestions. Sometimes the rules shift over the decades. But if you tell someone “Roll initiative”, they will not throw a football at your head.

Many unsophisticated consumers of journalism believe that one of the rules is “Never, ever lie.”, and you should roll to disbelieve that illusion. Many exceptionally unsophisticated consumers believe “Journalists always lie.”, and you don’t even need to roll to disbelieve that one.]

Patrick: But remembering, say, the COVID experience, there were any number of claims made by well-credentialed people that might not have been totally based in reality.

I don't know if I'm being maximally fair in saying that many institutions don't seem to be super enthusiastic about remembering that they made those claims. It’s nice when controversial claims are made in the moment – even just the fact of a market existing on something, it's like pinning it to the internet and saying, “hey, there's a material dispute here, let's check in in a while.”

Stephen: Yeah, I think it's very helpful. You can't memory-hole things with, with markets. The fact that the media sources completely changed their tune seamlessly overnight and totally forgot about their previous years of reporting – this is a thing that markets can help keep people honest on.

Patrick: Yeah. 

So stepping back to the philosophical question, what is different in a prediction market versus a traditional market? What percentage value does the price of Coca Cola encode?

Stephen: Traditional financial assets typically represent income streams and something – so, a share in Coca Cola classically is a claim on all potential dividends or proceeds from a sale of Coca Cola in the future. 

A prediction market contract is a derivative. If you’re investing in a basket of stocks you expect those to increase over time because the economy is growing and because there's a risk premium associated with holding equities; the same is not true for holding a basket of random contracts on prediction markets since, by nature, they're more zero-sum. 

On Manifold, and for prediction markets in general, typically there's a very natural subsidizer, someone who's willing to pay for information; in this sense participation in markets can be more positive-sum than you would think, because someone is actively injecting capital into the market in order to entice traders.

But outside of that, there just is this fundamental difference of, one thing is tracking income streams.

Patrick: And there's a literal button on Manifold markets that says “Subsidize this market because I care about the outcome more than I care about winning money or losing money.”

Stephen: That's right.

Patrick: For people who might understand the mechanics of the capital market a little less, how does that subsidy translate into “the market generates more information?”

Stephen: Sure. So what you're doing mechanistically when you subsidize is you're taking a bunch of your money and you're adding it to the market's liquidity pool.

The liquidity pool right now is just a set of Yes shares and No shares for traders to trade against. The more shares there are in this pool, the greater the opportunity for traders to make money because they can place a larger bet; if there's only one dollar in the liquidity pool, you can only make up to one dollar if you're right.

If there's a million dollars in the liquidity pool, then that means you can place a much larger bet and also that the price will move against you much less when you're trading in the marketplace.

Patrick: So conceptually speaking, the subsidized market functions as sort of a bounty for information – “I will naturally always take a loss by injecting liquidity here, but perhaps I'm willing to pay up to 500 for a credible argument in the way of either yes or no, and I don't know which argument I'm going to get.”

Someone could say, “Oh, I will snap up the free $500 because the market started 50-50 and this is not a 50 to me, this is an 85” and then maybe their participation will bring in other people who say, “85? Really? That's rich. It's a 66 at best.” 

So then you, the original person injecting liquidity, benefit from that increased interest between  rational profit maximizers attempting to make some money by being better-calibrated than the next person.

Stephen: That's right. And it also can be a much cheaper alternative to the traditional way of gathering feedback, which is paying someone to produce a report. For the cost of doing that, you can actually get a bunch of different people, a bunch of different traders weighing in from a wide variety of perspectives to come in and price your market.

Patrick: Yeah. I don't know what my charge-out rate would be for a week of work these days, but it would probably be pretty expensive, but I view Internet points in a way very differently than I do dollars and have done a lot of work to get $30 US equivalent of internet points on Manifold and other places.

Which brings us to an interesting segue. The market started as play money in this currency called mana, and it has recently transitioned, or is in the process of transitioning, to something with more real money involved. Can you talk about both what that is, and the decision-making process to transition that way?

Stephen: Sure. The model we're moving towards now is “play money markets with real cash prizes.” 

For legal and usability reasons, the model that we're moving towards is mixed. My hope is that we will retain many of the benefits of both play money and real money.

Mechanistically, the way this will work is that we (the house at Manifold) will hand pick a number of markets that we think are relatively unambiguous, have good resolution criteria, and are concerned with interesting or socially important topics, and we will make those prize markets.

Those prize markets will not pay out in mana, which is our play currency, but a different currency called prize points, and the prize points are redeemable for cash. That mechanism is the only way to earn real cash prizes on our platform. We are using U.S. sweepstakes law as the legal mechanism to do this.

Unfortunately, there are like five or six states (I believe) in the US where these types of sweepstakes are prohibited, so this mechanism won't be available universally – but those people living in those states will still be able to access the rest of the platform and engage with it in a play money fashion like they have previously. 

We believe that this setup will let us have the advantages of both the play money world and the real money world. Obviously the ability to earn real cash is important, and typically what you see in most prediction markets or asset markets in general is that there’s like a power law distribution of volume: the top markets tend to get an order of magnitude more trading volume than the next biggest ones. 

So by just figuring out what those ones are and making those prize markets, we can capture most of the real dollar volume that would be going to those, while also still maintaining this ability to have a long tail of fun, casual, and more subjective markets. So even with this change, you're still going to be able to come onto Manifold and create a market on whether your dog will jump in a puddle of mud today, or things that are silly or fun. 

We think that's always been a core part of Manifold and something that we hope to retain indefinitely.

Patrick: It's been interesting seeing in the markets I've participated in, the incredible amount of leverage between the amount of money notionally at stake. So Manifold, well, it's a play currency that you can't cash out. You do have to cash in to get mana in many cases. And so I think notionally a hundred mana was a dollar or something.

Am I quoting that right?

Stephen: Oh, as part of this move to real cash prizes under sweepstakes, we've changed the mana purchase rate. So now one dollar buys a thousand mana. 

Patrick: So, quoting things in dollars – constant dollars, although dollars aren’t constant, whatever – quoting things in dollars for the purpose of legibility to the audience, many of the proposition arguments around, you know, criminal prosecutions in cryptocurrency (a dearly held niche interest of mine) are denominated in the tens or hundreds of dollars between low dozens to high dozens of participants, of people expressing their views on this.

Now, actually in society, the lawyers in these bankruptcy cases are billing $5 to 10 million a month; there's criminal prosecutions, which are quite costly, etc.; and while the societal purpose of these structures is broader than simply, did he do it?, the fact that you can get an in some cases shockingly accurate point of view on the question “did he do it?”, for orders of magnitude less in cost, and then get that ratified by the formal process and the fullness of time, is a kind of amazing feature of this technology. 

(And I think prediction markets are a technology, although they've been scandalously underused.)

One reason prediction markets remain underused: people in positions of authority have tried to do these before, haven't they?

Stephen: Yeah. I think my favorite prediction market usage history fact comes from Robin Hanson. I believe he was telling me that, at the turn of the 20th century, total trading volume on US elections was greater than stock market trading, which is an amazing fact. 

Patrick: Whoa. Wouldn't have guessed that one.

Stephen: The U. S. has a very long history of election betting, that was very active, [and] that was mostly shut down during the progressive era in the 1920s.

Election betting can be seen as a type of prediction market; it's still one of the most important types of markets today by volume. The reason that those are much less prominent in the U. S. than they were previously is mostly just regulation – that these are prohibited by regulatory fiat. In other countries like the UK, the regulatory climate is much more favorable both to political betting and prediction markets in general, so you can still see pretty large liquid markets on the outcome of their elections (and the outcome of our elections as well).

Patrick: I remember when I was getting into prediction markets in my early 20s, attempting to get onto one of the UK sites, and it's exactly as much fun as you can imagine, with (wow, it was hard to move money around the internet in the 2004 range) jumping through hoops, trying to find a combination of credit cards that will allow you to give a payment processor money to put it on the system.

Eventually I gave up and just used a play market. But these days, on the flip side, I mostly just use play markets because even if I was paid money for being right through this mechanism, it's probably less motivational to me. (Interesting things at various parts of the life cycle.)

On the question of “markets have been tried, but found wanting, but maybe not for the ideal reasons”: I don't remember the exact event, but I remember the American government trying to do internal prediction markets on national security relevant things, and here was a hue and cry in Congress, if I remember –  “wait, but that's making money off of people dying.” That sort of moral distaste scuppered it.

Do you remember? 

Stephen: Yeah, I think the specific scenario was terrorism – that a bunch of markets had been set up internally to track the probability of terrorist attacks. This came up in a Senate hearing one time, and the public became outraged after a senator was discussing this; they ended up shutting the program down.

Patrick: Yeah. To be clear, we have markets on terrorism – they're just, like, Delta stock. 

[Patrick notes: Byrne Hobart has made this point many times before. Airline stocks didn’t return to the world of September 10th, 201 for 17 years. If you were quite confident that terrorism was mispriced in early 2001, either because you had made a study of unclassified intelligence reports or because you had recently started flight classes and skipped the day they taught landing, there were a lot of ways to make money.]

Patrick: But sometimes when you give people a narrow viewpoint of a thing, or explain an upcoming technology in a way that is not maximally favorable to it, people will – through their elected representatives and through various democratic or semi-democratic processes, like using pressure in the media – sort of strangle the emerging technology in the bed for maybe less than fully considered reasons.

I think that's one thing that tech founders and the tech industry should be aware of. PR is not just a distraction.

Stephen: Yeah, absolutely. I think a big problem in general is just a general educational issue – people don't really understand the role that markets play and how having good market prices is very helpful to everyone and makes society richer and better off. The other thing, you know, is just optics. 

Many markets which sound bad on a first-glance reading could actually be rewritten – if the question were posed in a slightly different way but having the same content, it could still provide the same level of information. 

For instance, you might create a market on whether Joe Biden will be the Democratic nominee. Consider that market versus a market like, “Will Biden die?” The latter has much worse vibes but probably a large percentage of the former is health effects or some other reason to be incapacitated.

[Patrick notes: As noted previously, this comment has been… substantially enriched by subsequent events in the six weeks between recording and publication.] 

Stephen: But a lot of this is just something for us and for market creators in general to be aware of: if you phrase things in a nice way then the public doesn't mind.

Patrick: I think we should be – generally, in tech – careful about the stories told about our products, both by ourselves and by the broader world. But, you know, don't allow the story to eclipse the fact of the thing. 

I think the fact of the thing with prediction markets is that we care intensely about being right about the shape of reality, and a technological increase that would get us closer to the shape of reality across a broad number of things all at once is just a very fascinating technology, and so I'm glad you folks are working on it so adeptly.

Do you have any closing thoughts that you'd like to leave to the audience, or tell them how to get involved at Manifold?

Stephen: Yeah, well, I would say that one of the reasons to use Prediction Markets is that it's fun. It's actually a very fun activity to test your wits against the marketplace – and that it is genuinely educational. You'll learn more about both specific object level facts in the world, and your ability to deal with risk.

If you want to get started, it's as simple as typing “Manifold Markets” into Google and pressing enter. You can bet against patio11 and you can see how well you do versus him!

Patrick: All of you crypto enthusiasts, step right up and take my money. I'm in almost all of the anti-crypto markets.

[Patrick notes: For full disclosure, I only very rarely invest in public markets in non-index funds, historically at about 25% of my public portfolio (as a concession to my desire to play World of Warcraft with less dragons and better loot). Out of an abundance of scrupulosity, I'll disclose that that allocation is currently long MicroStrategy puts. I'm also short a much smaller amount of Tether (natively in crypto, to satisfy my friends and colleagues who say I absolutely must be a user given my professional interests). If you can’t diagram out how to take the other side of these trades definitely don’t seek to take the either side of these trades.]

Patrick: All right. Well, thanks very much for your time, Stephen, and for the rest of you folks, I will see you around the internet.

Stephen: Alright, thanks so much for having me. This was great.