Blog

ArabicWeb24: Creating a high quality Arabic Web-only pre-training dataset

LightOn Pulse #1

Innovations and Advancements (2024.07)

Training Mamba Models on AMD MI250/MI250X GPUs with Custom Kernels

In this blogpost we show how we can train a Mamba model interchangeably on both NVIDIA and AMD and we compare both training performance and convergence in both cases. This shows that our training stack is becoming more GPU-agnostic.

Comment transformer les LLMs en Agents Intelligents ?

Développement des Capacités Agentiques pour les LLMs afin d'automatiser les flux de travail et créer des assistants intelligents

Passing the Torch: Training a Mamba Model for Smooth Handover

We present our explorations on training language models based on the new Mamba architecture, which deviates from the traditional Transformer architecture.

LightOn AI Meetup Creating a Large Dataset for Pretraining LLMs

Partenariat LightOn & Orange Business

Orange Business, LightOn et HPE s'associent pour offrir des solutions d'IA générative de confiance

Elevating Enterprise AI: Paradigm and Key Insights

WAICF 2024

Fine-tuning vs. Efficient Fine-tuning: A Business Lens on AI Optimization with LightOn's Solutions