A lot that’s questionable about the analysis in the paper highlighted today is visible in the chart below.
Before I come back to that let me summarize work from Alejandro Lopez-Lira and Yue Hua Tang, both of the University of Florids, which is the subject of this week’s paper.
Quantie-folk, for years, have employed models for analyzing news flow as a predictor of future stock price moves. Perhaps this worked, once upon a time, and for a while.
In recent years these models have been shown to be largely of little use. The main reason, I suspect and not entirely, being they all use the same feeds. Even if your ‘bot’ is a millisecond faster than your peers it’s unlikely this advantage will persist over time (it’s called ‘a market’).
In the paper highlighted today the researchers wondered if they could build a better news-flow-analysis mousetrap using ChatGPT and get more nuanced i.e. better sentiment readings? If they could, their theory went, it’d have more predictive power and, presto, be a more lucrative model.
Back to the chart above. It covers the period from October 2021 to December 2022 and, huzzah!, shows how effective a predictor of future stock price moves their ChatGPT powered model is.
There are several problems with the analysis though:
- The study period is very, very short. I’d like to see at least ten-years of data before trying to form a view about the efficacy of this system.
- It doesn’t include transaction costs. This is bonkers. Even if you don’t pay commission there are costs associated with any strategy.
- It assumes you can transact at published stock prices. Also bonkers, you can’t. Often you can but it’s times you can’t that’ll seriously upend you.
- It looks only at the NYSE, NASDAQ and AMEX. For any system to have merit it needs to work across asset classes and geographies.
- It suffers from the same ‘finite-teet’ problem as other news-flow-analysis models. If the system works its capable of instant replication.
- In the work they detail how narrowly they’re prompting ChatGPT. There’s an obvious bias here. The answer is always question dependent.
This paper has been downloaded many times and it came to my attention via a news article that referenced its merit. The public want alchemy and shabby Mangers are always ready to sell versions of it. I suspect therefore Funds are now being set up based on this type of model.
However, there are no short-cuts to successful investing. A key element of which is assembling your own information from reliable sources and basing investment decisions on a company’s likely ability to deliver consistent and rising returns over time.
An analysis of sentiment may be referenced but can’t form the basis for successful long term decision making. Voting machines, weighing machines* and all that.
You can access the paper in full via this link Can ChatGPT Forecast Stock Price Movements?
Happy Sunday.
[* Check out other peaches from Wazza Buffet’s teacher here Ben Graham Zingers]