The proliferation of IoT devices has expanded the attack surface, making zero-day threat detection an urgent priority. This paper proposes LLMEdgeSec, a framework that leverages large language models (LLMs) for log reasoning to detect and contextualize zero-day exploits in real-time. Unlike conventional anomaly detectors that rely on static signatures or pre-defined rules, LLMEdgeSec applies prompt-based reasoning over system logs, communication metadata, and device telemetry, enabling proactive identification of previously unseen attack vectors. The framework employs a dual-stream architecture: (1) a lightweight transformer encoder on edge nodes for local feature extraction, and (2) a cloud-based LLM interface for contextual reasoning and cross-device threat correlation. A zero-shot calibration method mitigates hallucinations and reduces false positives. Evaluation on TON IoT, CIC-IDS2018, and a real-world smart factory dataset demonstrates F1 scores exceeding 92% on zero-day samples with communication overhead below 8MB per federated round. Results show significant improvements in accuracy, interpretability, and adaptability over state-of-the-art baselines.