Passing the Torch: Training a Mamba Model for Smooth Handover TL;DR We experiment with the Warmup-Stable-Decay (WSD) learning rate scheduler and a novel positional weighting of the loss for language model pre-training; We find that WSD outperforms the cosine sch... research technical Apr 10, 2024
LightOn AI Meetup Creating a Large Dataset for Pretraining LLMs This summary presents the key takeaways from a video featuring Guilherme Penedo from Hugging Face, discussing various aspects of training large language models (LLMs) and utilizing them effectively. L... Mar 22, 2024
Partnership LightOn & Orange Business Orange Business and LightOn launched a new offer for end-to-end and trusted generative AI projects in France. This strategic partnership between Orange Business, LightOn, and HPE, a leader in hybrid c... Mar 19, 2024
Elevating Enterprise AI: Paradigm and Key Insights The World AI Cannes Festival (WAICF) 2024 offered the ideal setting for LightOn to present Paradigm , a cutting-edge Generation AI (GenAI) software platform tailored for the enterprise sector. This un... Feb 15, 2024
The Magic of Tokens in Generative AI: A Deep Dive Token : It's a term that floats around the realm of Generative AI, often leaving many scratching their heads. Far from the realm of cryptocurrency or reward systems, in the world of Artificial Intelli... Dec 11, 2023
Turning Up the Heat: The Role of Temperature in Generative AI In the culinary world, temperature can be the difference between a perfectly seared steak and a charred piece of meat. Similarly, in the realm of Generative AI, there's a kind of "temperature" that de... Dec 11, 2023
The Powerhouse Behind Artificial Intelligence: Why GPUs Are Essential for Large Language Models Brace yourselves as we journey through the bustling world of Artificial Intelligence and uncover the magic behind one of its key components - the GPU. Once upon a time, in a digital world not so far a... Dec 11, 2023
Fine-tuning vs. Efficient Fine-tuning: A Business Lens on AI Optimization with LightOn's Solutions In today's competitive business landscape, leveraging AI effectively can be a game-changer. But how do you tailor AI models to your unique needs efficiently and cost-effectively? Dive into the realms ... Dec 11, 2023
Unlock the Potential of Prompt Tuning with Paradigm by LightOn Dive into Simplified prompt finetuning with Paradigm Navigating through Artificial Intelligence, prompt management stands out as a key navigator for enhancing the answers of models n, enabling them to... Dec 11, 2023
Navigating Data Privacy and Compliance with LightOn's Innovative Large Language Model Factory In the rapidly evolving digital landscape, data privacy and compliance are paramount, especially when harnessing the power of Large Language Models (LLMs) to drive insights and innovation. LightOn ste... Dec 8, 2023
Introducing Alfred-40B-1023: We are thrilled to unveil Alfred-40B-1023, the latest iteration of our celebrated open-source Language Model. Building on the solid foundation of its predecessor, Alfred-40B-1023 represents a signific... Nov 17, 2023
Docaposte launches its 1st sovereign generative AI solution in partnership with French players LightOn, Aleia, and NumSpot. Docaposte, a key player in digital trust and a subsidiary of La Poste Group, joins forces with LightOn, Aleia, and NumSpot to offer its first sovereign and industrial generative AI solution. Available... Generative AI Oct 24, 2023