Most of neural networks require massive data for model training, which limits their forecast accuracy for short-term solar power forecasting in data-scarce scenarios. To address this limitation, this paper aims to apply the large language models (LLMs), specifically Chronos model, to short-term solar power forecasting in data-scarce scenarios. In particular, a framework is introduced to convert continuous time-series data in solar power forecasting into the discrete data format required by LLMs. Once pre-trained, LLMs can be directly applied to other datasets for short-term solar power forecasting without the need for further fine-tuning. Simulations on four solar datasets show that the LLM significantly outperforms 7 popular statistical models and 2 state-of-the-art neural networks in both probabilistic and deterministic solar power forecasting, with forecast horizons ranging from 30 minutes to 2 hours. This highlights the superiority of LLMs for short-term solar power forecasting in data-scarce scenarios.