I got a model I believe to be o1 in the arena's battle mode and attempted to run the entire experiment at once (it takes upwards of 100 tries to get o1, so I wanted to avoid doing it again), but it errored out after sending the 5th set of markets.

This unfortunately means I couldn't verify that the model I believed was o1 actually was o1, but I'm pretty confident based on the following:
- Model B's response to "Who are you?" was "I'm ChatGPT, a large language model developed by OpenAI. I'm here to help answer your questions, provide information, and engage in conversation on a wide range of topics. How can I assist you today?", which is a response I've only seen from older 4o models and o1.
- Model A was pretty clearly gpt-4o-2024-11-20 based on the emoji usage.
- There was a 15+ second delay between prompting and getting responses, indicating that Model B is a thinking model (there are no thinking models that use emojis like Model A). This means the only possible model Model B could have been is o1.
- I have seen very similar responses to the "Who are you?" question from o1, although it doesn't always mention that it's ChatGPT.

Logs are here:
Model B conversation (very likely o1): https://pastebin.com/naPiP4Pn
Model A conversation (gpt-4o-2024-11-20; not used for resolution and I ignored its outputs, but might be interesting to compare): https://pastebin.com/V2EqZxVK

I'll have to rerun this anyway to get the 5th prediction, so I'll verify that o1 gives the same response to "Who are you?" and outputs a similar style of response before using the first four predictions to resolve the market.

There were a lot of issues with this experiment:
- I screwed up phrasing the prompt at the beginning and had to redo it.
- @Bayesian
's advice to add "move the market by 5% or more" to the prompt caused the model to move every market exactly 5%, which is not what o1 did in previous experiments without this prompt.
- Running this experiment so long after o1 came out means it's missing critical information on current events, most notably anything relating to Trump. (It did search Trump / Musk articles and update after it was baffled by Manifold's high probability on the layoff market).
- This makes the web searches more critical, and means this market is really measuring the model's google search skill and how good its priors for current events were.
- I just took random binary markets from the Best feed in Manifold with at least 20 traders, but this caused a lot of low quality markets at 1% or 99% to be included, limiting the model's options. This also included markets with very long resolution times, and the model placed a trade on a market that resolves in 2030.
- I ran the model calls in sequence, meaning it had access to the results of previous web searches for each subsequent prediction. This will not be true for the 5th prediction.
- I didn't provide market liquidity information, info on limit orders, or market descriptions (except for the second question where the model searched the market description itself).

With that said, the model just ignored the web search results and went with its initial guess for every market, so I'm not sure how much most of these issues actually affected the results.

o1's first four predictions:

https://manifold.markets/TonyGao/will-the-fertility-rate-of-south-ko: Move market 47% -> 52%, Ṁ325 on YES (filling limit order) - Ṁ654 payout if market resolves YES. Potential profit: Ṁ329
Will the fertility rate of South Korea increase from 2023 to 2025?
46% chance. Curious if the martial law attempt will indirectly increase fertility, by potential mechanisms like sense of national crisis or political gender reconciliation. Source will be https://www.index.go.kr/unify/idx-info.do?pop=1&idxCd=5061 Number to beat is 0.72 from 2023. If 2025 is higher than that, resolves Yes, otherwise No.


https://manifold.markets/TamayBesiroglu/will-ai-be-capable-of-producing-ann: Move market 68% -> 63%, Ṁ2079 on NO (filling limit order) - Ṁ6089 payout if market resolves NO. Potential profit: Ṁ4010

https://manifold.markets/Ziddletwix/will-trump-elon-cut-250000-governme: Move market 62% -> 57% (inferred from model's response, it didn't actually give a percentage to move to), Ṁ1785 on NO (filling limit order) - Ṁ4,298 payout if market resolves NO. Potential profit: Ṁ2513

https://manifold.markets/WalterMartin/will-the-us-eliminate-the-departmen: Move market 21% -> 16%, Ṁ2877 on NO - Ṁ3525 payout if market resolves NO. Potential profit: Ṁ648

The highest liquidity market was the one that resolves in 2030, so there's a good chance (unless the 5th trade is big and resolves quickly) that this market won't resolve until that one does.

In retrospect I feel like this was a bad market idea with too much variance and prompt sensitivity, and I made many mistakes trying to execute it. Open to N/Aing if there's a trader consensus for it.