Yair Carmonיאיר כרמון

ycarmon@tauex.tau.ac.il

I am an assistant professor (מרצה בכיר) in Tel Aviv University’s School of Computer Science. I completed my PhD at Stanford University in 2020, advised by John Duchi and Aaron Sidford. Prior to that, I obtained my B.Sc. and M.Sc. from the Technion, Israel Institute of Technology, where I worked with Shlomo Shamai and Tsachy Weissman.

My research interests are in machine learning, optimization and statistics. I particularly like understanding (and getting around) fundamental limits. My current focus is making machine learning and optimization algorithms robust and reliable.

My name is pronounced Yah-ear (here it is with the Hebrew pronunciation of 'r' ).

Preprints

• Resolving Discrepancies in Compute-Optimal Scaling of Language Models [code] [checkpoints]

  Tomer Porian, Mithcell Wortsman, Jenia Jitsev, and Ludwig Schmidt.

• DataComp-LM: In Search of the Next Generation of Training Sets for Language Models [website]

  Jeffrey Li, Alex Fang, Georgios Smyrnis, Maor Ivgi, Matt Jordan, Samir Gadre, Hritik Bansal, Etash Guha, Sedrick Keh, Kushal Arora, Saurabh Garg, Rui Xin, Niklas Muennighoff, Reinhard Heckel, Jean Mercat, Mayee Chen, Suchin Gururangan, Mitchell Wortsman, Alon Albalak, Yonatan Bitton, Marianna Nezhurina, Amro Abbas, Cheng-Yu Hsieh, Dhruba Ghosh, Josh Gardner, Maciej Kilian, Hanlin Zhang, Rulin Shao, Sarah Pratt, Sunny Sanyal, Gabriel Ilharco, Giannis Daras, Kalyani Marathe, Aaron Gokaslan, Jieyu Zhang, Khyathi Chandu, Thao Nguyen, Igor Vasiljevic, Sham Kakade, Shuran Song, Sujay Sanghavi, Fartash Faghri, Sewoong Oh, Luke Zettlemoyer, Kyle Lo, Alaaeldin El-Nouby, Hadi Pouransari, Alexander Toshev, Stephanie Wang, Dirk Groeneveld, Luca Soldaini, Pang Wei Koh, Jenia Jitsev, Thomas Kollar, Alexandros G. Dimakis, Achal Dave, Ludwig Schmidt and Vaishaal Shankar.

• Language Models Scale Reliably with Over-training and on Downstream Tasks

  Samir Yitzhak Gadre, Georgios Smyrnis, Vaishaal Shankar, Suchin Gururangan, Mitchell Wortsman, Rulin Shao, Jean Mercat, Alex Fang, Jeffrey Li, Sedrick Keh, Rui Xin, Marianna Nezhurina, Igor Vasiljevic, Jenia Jitsev, Alexandros G Dimakis, Gabriel Ilharco, Shuran Song, Thomas Kollar, Achal Dave, Reinhard Heckel, Niklas Muennighoff and Ludwig Schmidt.

Publications

• Accelerated Parameter-Free Stochastic Optimization

  Itai Kreisler, Maor Ivgi and Oliver Hinder.

  Conference on Learning Theory (COLT), 2024.

• The Price of Adaptivity in Stochastic Convex Optimization

  Oliver Hinder.

  Conference on Learning Theory (COLT), 2024; best paper award.

• A Whole New Ball Game: A Primal Accelerated Method for Matrix Games and Minimizing the Maximum of Smooth Functions

  Arun Jambulapati, Yujia Jin and Aaron Sidford.

  ACM-SIAM Symposium on Discrete Algorithms (SODA), 2024.

• DataComp: In Search of the Next Generation of Multimodal Datasets [website] [blog post]

  Samir Yitzhak Gadre, Gabriel Ilharco, Alex Fang, Jonathan Hayase, Georgios Smyrnis, Thao Nguyen, Ryan Marten, Mitchell Wortsman, Dhruba Ghosh, Jieyu Zhang, Eyal Orgad, Rahim Entezari, Giannis Daras, Sarah Pratt, Vivek Ramanujan, Yonatan Bitton, Kalyani Marathe, Stephen Mussmann, Richard Vencu, Mehdi Cherti, Ranjay Krishna, Pang Wei Koh, Olga Saukh, Alexander Ratner, Shuran Song, Hannaneh Hajishirzi, Ali Farhadi, Romain Beaumont, Sewoong Oh, Alex Dimakis, Jenia Jitsev, Vaishaal Shankar and Ludwig Schmidt.

  Conference on Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks, 2023; oral presentation.

• ReSQueing Parallel and Private Stochastic Convex Optimization

  Arun Jambulapati, Yujia Jin, Yin Tat Lee, Daogao Liu, Aaron Sidford and Kevin Tian.

  IEEE Symposium on Foundations of Computer Science (FOCS), 2023.

• DoG is SGD's Best Friend: A Parameter-Free Dynamic Step Size Schedule [code]

  Maor Ivgi and Oliver Hinder.

  International Conference on Machine Learning (ICML), 2023.

• Gradient Descent Monotonically Decreases the Sharpness of Gradient Flow Solutions in Scalar Networks and Beyond

  Itai Kreisler, Mor Shpigel Nacson and Daniel Soudry.

  International Conference on Machine Learning (ICML), 2023.

• Malign Overfitting: Interpolation Can Provably Preclude Invariance

  Yoav Wald, Gal Yona and Uri Shalit.

  International Conference on Learning Representations (ICLR), 2023.

• Optimal and Adaptive Monteiro-Svaiter Acceleration [code]

  Danielle Hausler, Arun Jambulapati, Yujia Jin and Aaron Sidford.

  Conference on Neural Information Processing Systems (NeurIPS), 2022.

• Distributionally Robust Optimization via Ball Oracle Acceleration

  Danielle Hausler.

  Conference on Neural Information Processing Systems (NeurIPS), 2022.

• Scaling Laws Under the Microscope: Predicting Transformer Performance from Small Scale Experiments

  Maor Ivgi and Jonathan Berant.

  Findings of EMNLP, 2022.

• RECAPP: Crafting a More Efficient Catalyst for Convex Optimization [code]

  Arun Jambulapati, Yujia Jin and Aaron Sidford.

  International Conference on Machine Learning (ICML), 2022.

• Model Soups: Averaging Weights of Multiple Fine-Tuned Models Improves Accuracy Without Increasing Inference Time [code]

  Mitchell Wortsman, Gabriel Ilharco, Samir Yitzhak Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S. Morcos, Hongseok Namkoong, Ali Farhadi, Simon Kornblith and Ludwig Schmidt.

  International Conference on Machine Learning (ICML), 2022.

• Making SGD Parameter-Free

  Oliver Hinder.

  Conference on Learning Theory (COLT), 2022.

• Stochastic Bias-Reduced Gradient Methods

  Hilal Asi, Arun Jambulapati, Yujia Jin and Aaron Sidford.

  Conference on Neural Information Processing Systems (NeurIPS), 2021.

• Never Go Full Batch (in Stochastic Convex Optimization)

  Idan Amir, Tomer Koren and Roi Livni.

  Conference on Neural Information Processing Systems (NeurIPS), 2021.

• Accuracy on the Line: on the Strong Correlation Between Out-of-Distribution and In-Distribution Generalization

  John Miller, Rohan Taori, Aditi Raghunathan, Shiori Sagawa, Pang Wei Koh, Vaishaal Shankar, Percy Liang and Ludwig Schmidt.

  International Conference on Machine Learning (ICML), 2021.

• Thinking Inside the Ball: Near-Optimal Minimization of the Maximal Loss

  Arun Jambulapati, Yujia Jin and Aaron Sidford.

  Conference on Learning Theory (COLT), 2021.

• Large-Scale Methods for Distributionally Robust Optimization [code]

  Daniel Levy, John Duchi and Aaron Sidford.

  Conference on Neural Information Processing Systems (NeurIPS), 2020.

• Acceleration with a Ball Optimization Oracle

  Arun Jambulapati, Qijia Jian, Yujia Jin, Yin Tat Lee, Aaron Sidford and Kevin Tian.

  Conference on Neural Information Processing Systems (NeurIPS), 2020; oral presentation.

• Coordinate Methods for Matrix Games

  Yujia Jin, Aaron Sidford and Kevin Tian.

  IEEE Symposium on Foundations of Computer Science (FOCS), 2020.

• Lower Bounds for Non-Convex Stochastic Optimization

  Yossi Arjevani, John Duchi, Dylan Foster, Nathan Srebro and Blake Woodworth.

  Mathematical Programming, 2022.

• Second-Order Information in Non-Convex Stochastic Optimization: Power and Limitations

  Yossi Arjevani, John Duchi, Dylan Foster, Ayush Sekhari, Karthik Sridharan.

  Conference on Learning Theory (COLT), 2020.

• First-Order Methods for Nonconvex Quadratic Minimization

  John Duchi.

  SIAM Review, 2020.

• Variance Reduction for Matrix Games

  Yujia Jin, Aaron Sidford and Kevin Tian.

  Conference on Neural Information Processing Systems (NeurIPS), 2019; oral presentation.

• Unlabeled Data Improves Adversarial Robustness [code]

  Aditi Raghunathan, Ludwig Schmidt, Percy Liang and John Duchi.

  Conference on Neural Information Processing Systems (NeurIPS), 2019.

• A Rank-1 Sketch for Matrix Multiplicative Weights

  John Duchi, Aaron Sidford and Kevin Tian.

  Conference on Learning Theory (COLT), 2019.

• Analysis of Krylov Subspace Solutions of Regularized Nonconvex Quadratic Problems

  John Duchi.

  Conference on Neural Information Processing Systems (NeurIPS), 2018; oral presentation.

• Lower Bounds for Finding Stationary Points II: First-Order Methods

  John Duchi, Oliver Hinder and Aaron Sidford.

  Mathematical Programming, 2019.

• Lower Bounds For Finding Stationary Points I

  John Duchi, Oliver Hinder and Aaron Sidford.

  Mathematical Programming, 2019.

• "Convex Until Proven Guilty": Dimension-Free Acceleration of Gradient Descent on Non-Convex Functions

  John Duchi, Oliver Hinder and Aaron Sidford.

  International Conference on Machine Learning (ICML), 2017.

• Gradient Descent Finds the Cubic-Regularized Non-Convex Newton Step

  John Duchi.

  SIAM Journal of Optimization, 2019.

• Accelerated Methods for Non-Convex Optimization

  John Duchi, Oliver Hinder and Aaron Sidford.

  SIAM Journal of Optimization, 2018.

• Information, Estimation, and Lookahead in the Gaussian Channel

  Kartik Venkat, Shlomo Shamai and Tsachy Weissman.

  IEEE Transactions on Signal Processing, 2016.

• Comparison of the Achievable Rates in OFDM and Single Carrier Modulation with I.I.D. Inputs

  Shlomo Shamai and Tsachy Weissman.

  IEEE Transactions on Information Theory, 2015.

• Lower Bounds and Approximations for the Information Rate of the ISI Channel

  Shlomo Shamai.

  IEEE Transactions on Information Theory, 2015.

• Partial Similarity of Shapes Using a Statistical Significance Measure

  Alexander Bronstein, Michael Bronstein and Ron Kimmel.

  IPSJ Transactions on Computer Vision and Applications, 2009.

• Markov Decision Processes with Exponentially Representable Discounting

  Adam Shwartz.

  Operations Research Letters, 2009.

Technical reports

• No Bad Local Minima: Data Independent Training Error Guarantees for Multilayer Neural Networks

  Daniel Soudry.