We demonstrate that semantic shifts in news language can be causally linked to financial market shocks, with partisan differences influencing predictive power. Text signals prove particularly valuable during unexpected events like COVID-19.

This study introduces a novel suite of historical large language models (LLMs) pre-trained specifically for accounting and finance, utilising a diverse set of major textual resources. The models are unique in that they are year-specific, spanning from 2007 to 2023, effectively eliminating look-ahead bias, a limitation present in other LLMs. Empirical analysis reveals that, in trading, these specialised models outperform much larger models, including the state-of-the-art LLaMA 1, 2, and 3, which are approximately 50 times their size. The findings are further validated through a range of robustness checks, confirming the superior performance of these LLMs.

Demonstrates that compressing LLM embeddings can improve performance on noisy regression tasks like financial prediction by reducing overfitting, suggesting compression acts as a regularization mechanism.

Demonstrates that traditional machine learning methods combining fundamental data with text embeddings outperform current LLMs at credit rating forecasting, highlighting limitations of LLMs for multimodal financial tasks.

A series of point-in-time language models designed to remain uninformed about future information, addressing temporal bias in LLMs for time-series forecasting applications.

Using transformer-based language models to extract features from Reddit posts for COVID-19 forecasting, outperforming traditional epidemiological data in predicting upward trend signals.