This webpage is deprecated. Please go to igormolybog.github.io for the up-to-date webpage
 |
Research Scientist, |
About me
I am a Research Scientist at Meta AI working on the training of Generative Artificial Intelligence. My research interests cover the area of computational techniques for learning from data that are highly scalable with the availability of the compute. More specifically, I am working on optimizers and model architecture scaling schemes that would allow predictable and efficient training of Transformer models containing hundreds of billions of parameters.
In 2022, I graduated with a Ph.D. from the group of Professor Javad Lavaei at the Department of Industrial Engineering and Operations Research at University of California, Berkeley, where I worked on problems of algorithmic analysis and optimal control of complex safety-critical systems, such as power systems, transportation and telecommunication networks, AI recommendation and navigation systems, robotic systems, and others. My research spanned the theory of non-convex and conic optimization, stochastic control, machine learning, and computational and sampling complexity of learning algorithms. I designed data processing algorithms that are robust to noise and highly scalable with the amount of available computational resources.
News
July 2023: Our paper Llama 2: Open Foundation and Fine-Tuned Chat Models landed with a splash, making it to the top of hackernews.
May 2023: I will present the paper A Theory on Adam Instability in Large-Scale Machine Learning at the FAIR <> GenAI Workshop. slides
April 2023: Our paper Over-parametrization via Lifting for Low-rank Matrix Sensing: Conversion of Spurious Solutions to Strict Saddle Points got accepted at the 2023 International Conference on Machine Learning (ICML).
March 2023: New paper A Theory on Adam Instability in Large-Scale Machine Learning is accessible online.
January 2023: New paper Over-parametrization via Lifting for Low-rank Matrix Sensing: Conversion of Spurious Solutions to Strict Saddle Points is accessible online.
October 2022: I am invited to organize a session “Large-scale smooth optimization for Generative AI” at the INFORMS Annual Meeting 2023.
June 2022: I am joining Meta AI (FAIR) as a Research Scientist specializing on large-scale optimization.
May 2022: I defended my thesis and graduated with a Ph.D. in Engineering!
April 2022: My Thesis The complexity of non-convex and conic optimization problems in data science applications is available online.
February 2022: I will give a talk at the Department of Electrical and Computer Engineeting of University of Hawai'i at Manoa. slides
December 2021: I will give a talk “Computation-information complexity trade-off in Tensor PCA” Random Matrices and Random Landscapes seminar at Mathematical Sciences Research Institute, Berkeley, CA. (slides)
November 2021: I will give a talk “Topological complexity of polynomials” at the Control and Optimization seminar, IEOR Department at University of California, Berkeley. (slides)
September 2021: I am organising session “Reaching Global Optimum in Non-Convex Optimization Problems” at the INFORMS Annual Meeting 2021.
July 2021: I will present the paper When Does MAML Objective Have Benign Landscape? on the 2021 IEEE Conference on Control Technology and Applications (CCTA).
May 2021: I will present the paper No spurious solutions in non-convex matrix sensing: Structure compensates for isometry on the 2021 American Control Conference (ACC).
December 2020: Our paper Role of sparsity and structure in the optimization landscape of non-convex matrix sensing got accepted for publication in Mathematical Programming.
September 2020: Our paper Conic Optimization for Quadratic Regression Under Sparse Noise got accepted for publication in the Journal of Machine Learning Research.
June 2020: Our new paper Global convergence of MAML for LQR is accessible online.
August 2019: I gave a talk “Frontiers of Deep Learning: overview of Simon's Institute summer workshops” at the Control and Optimization seminar, IEOR Department at University of California, Berkeley. (slides)
July 2019: The paper Towards Robust and Scalable Power System State Estimation to appear in Proc. 58th IEEE Conference on Decision and Control
May 2019: New paper on non-convex learning: No Spurious Solutions in Non-convex Matrix Sensing: Structure Compensates for Isometry
January 2019: New paper on data analytics: Conic Optimization for Robust Quadratic Regression
December 2018: Our paper On Sampling Complexity of the Semidefinite Affine Rank Feasibility Problem has been designated for oral presentation on Thirty-Third AAAI Conference on Artificial Intelligence
November 2018: I will give a talk “Conic Optimization For Robust Quadratic Regression” at the 57th IEEE Conference on Decision and Control
October 2018: Our paper On Sampling Complexity of the Semidefinite Affine Rank Feasibility Problem was accepted on Thirty-Third AAAI Conference on Artificial Intelligence
September 2018: I successfully passed Doctoral Qualifying Examination.
September 2018: I gave a talk “Geometry of SDP relaxations for rank constrained problems” at the Power Systems Seminar for IEOR Department at University of California, Berkeley.
September 2018: New paper on SDP relaxations of rank-constrained problems: On Sampling Complexity of the Semidefinite Affine Rank Feasibility Problem.
July 2018: Our paper Conic Optimization for Robust Quadratic Regression: Deterministic Bounds and Statistical Analysis to appear in IEEE Conference on Decision and Control, 2018.
July 2018: I will give a talk on “Conic Optimization For Robust State Estimation: Deterministic Bounds And Statistical Analysis” at INFORMS Annual Meeting
May 2018: I successfully passed Ph.D. Preliminary Exam.
April 2018: I gave a talk on “Conic Relaxations for State Estimation under Sparse Noise” at the Power Systems Seminar for IEOR Department at University of California, Berkeley.
March 2018: New paper on robust nonlinear regression for bad data detection: Conic Optimization for Robust Quadratic Regression: Deterministic Bounds and Statistical Analysis.
August 2017: I joined the department of Industrial Engineering and Operations Research at University of California, Berkeley as a M.Sc/PhD Scholar.