This article presents an approach to using AI models for solving financial advisory tasks. The authors prepared a specialized financial dataset, fine-tuned the model, and developed a framework for enriching user queries (RAG). The resulting model was used to predict stock trends. Additionally, a trading strategy was tested, where Chinese stocks with an upward trend were purchased. The test yielded a good result: a return of 30.8% and a Sharpe ratio of 1.6. This is significantly better than the baseline CSI300 index (return 1.7%, Sharpe 0.92).
A bit more about the research and results.
One common area where LLM models are applied is financial analysis. This article examines the ability to predict stock trends. To improve the quality of responses from the base versions of the models, two methods are used:
Retrieval Augmented Generation (RAG). The essence of the method is that additional relevant information is added to the user's query. For example, if a user asks: "Predict the rise and fall of Apple Inc in the next month," the method enriches the context by adding Apple's reports, price data, news related to Apple, and so on. This method firstly focuses the model on processing relevant data and secondly provides information that was not available at the time of its training.
Finetuning involves training the base model on financial data (AlphaFin dataset) and a manually created dataset called ChainOfThoughts (CoT). CoT provides examples of reasoning used to prepare responses. The Low-Rank Adaptation (LoRa) method was applied for finetuning.
To implement RAG and fine-tune the model, the authors compiled a specialized dataset: AlphaFin (available here, though the file descriptions are in Chinese). It includes:
Research datasets prepared for training financial models: FPB, FinQA, convFinQA, HeadLine.
StockQA dataset
Financial News
Financial Reports CoT
By gathering all this, the authors developed the StockChain framework, which includes:
Fine-tuned model StockGPT
AlphaFin data
Implementation of the RAG method for enriching user queries
This framework was used for trend prediction. Example query:
"According to ..., please judge the trend of the company and give a clear answer up or down. Input: <reports & stock price>, Output: <Up/down>."
The results of StockChain were compared with those of classical ML methods (which were fed only the timeseries data without additional context), non-specialized LLMs (ChatGLM2, ChatGPT-3.5 turbo), and specialized financial models FinMA and FinGPT.
The result – ChainGPT outperformed these models in both trend prediction metrics (Accuracy, ROUGE) and in the results of the trading strategy, which involved buying stocks with an upward trend prediction each month.
In our opinion, the research demonstrates the correct approach to using LLM models for stock analysis. The only significant concern with the study design is the potential for look-ahead bias. Since the main LLM models were trained on data that includes the test period (2020-2023), the results may appear inflated. However, these results can still be used not as absolute values but for evaluating the differences between models.