algorithms

From the Ball-Proximal (Broximal) Point Method to Efficient Training of LLMs

-

B9 L2 R2325

This talk introduces the Ball-Proximal Point Method, a new foundational algorithm for non-smooth optimization with surprisingly fast convergence, and Gluon, a new theoretical framework that closes the gap between theory and practice for modern LMO-based deep learning optimizers.

From the Ball-Proximal (Broximal) Point Method to Efficient Training of LLMs

-

B9 L2 R2325

This talk introduces the Ball-Proximal Point Method, a new foundational algorithm for non-smooth optimization with surprisingly fast convergence, and Gluon, a new theoretical framework that closes the gap between theory and practice for modern LMO-based deep learning optimizers.

Machine learning models gather momentum

1 min read ·
New methods for training machine learning models are quicker and more accurate than current approaches, previously considered state-of-the-art.

Peter Richtarik

Professor, Computer Science
Professor Richtárik specializes in big data optimization and machine learning. He is known for his work on randomized coordinate descent algorithms, stochastic gradient descent and federated learning.

Konstantin Mishchenko

Ph.D. Student, Computer Science
Konstantin Mishchenko is a Ph.D. candidate at Visual Computing Center (VCC) in King Abdullah University of Science and Technology (KAUST), studying under the supervision of Professor Peter Richtarik in his research group. Education and Early Career Konstantin Mishchenko obtained his bachelor degree in Computer Science and Physics from Moscow Institute of Physics and Technology in 2016. After that in 2017, he received his master degree in Machine Learning MASH from Université Paris Dauphine in Paris, France. Konstantin had many internships during his studies. The most recent was with Amazon in

Elnur Gasanov

Ph.D. Student, Computer Science

Alibek Sailanbayev

Alibek Sailanbayev is a Ph.D. candidate in Applied Mathematics and Computational Science at Visual Computing Center (VCC) in King Abdullah University of Science and Technology (KAUST), studying under the supervision of Professor Peter Richtarik in his research group. Education and Early Career Alibek obtained his bachelor degree Applied Mathematics from Nazarbayev University in Kazakhstan. He joined KAUST in Fall 2017 and received his M.S. degree in Applied Mathematics and Computational Sciences in December 2018. He is now pursuing Ph.D. studies. Alibek took a Summer Internship at Loria/Inria