research
∙
07/06/2023
What Should Data Science Education Do with Large Language Models?
The rapid advances of large language models (LLMs), such as ChatGPT, are...
share
research
∙
06/09/2023
DP-HyPO: An Adaptive Private Hyperparameter Optimization Framework
Hyperparameter optimization, also known as hyperparameter tuning, is a w...
research
∙
05/28/2023
Reward Collapse in Aligning Large Language Models
The extraordinary capabilities of large language models (LLMs) such as C...
research
∙
05/27/2023
The Implicit Regularization of Dynamical Stability in Stochastic Gradient Descent
In this paper, we study the implicit regularization of stochastic gradie...
research
∙
04/21/2023
The Isotonic Mechanism for Exponential Family Estimation
In 2023, the International Conference on Machine Learning (ICML) require...
research
∙
10/31/2022
A Law of Data Separation in Deep Learning
Multilayer neural networks have achieved superhuman performance in many ...
research
∙
09/29/2022
On Quantum Speedups for Nonconvex Optimization via Quantum Tunneling Walks
Classical algorithms are often not effective for solving nonconvex optim...
research
∙
06/14/2022
A Truthful Owner-Assisted Scoring Mechanism
Alice (owner) has knowledge of the underlying quality of her items measu...
research
∙
06/09/2022
Analytical Composition of Differential Privacy via the Edgeworth Accountant
Many modern machine learning algorithms are composed of simple private a...
research
∙
06/06/2022
FIFA: Making Fairness More Generalizable in Classifiers Trained on Imbalanced Data
Algorithmic fairness plays an important role in machine learning and imp...
research
∙
01/31/2022
Causal Inference Principles for Reasoning about Commonsense Causality
Commonsense causality reasoning (CCR) aims at identifying plausible caus...
research
∙
12/17/2021
Neurashed: A Phenomenological Model for Imitating Deep Learning Training
To advance deep learning methodologies in the next decade, a theoretical...
research
∙
10/27/2021
You Are the Best Reviewer of Your Own Papers: An Owner-Assisted Scoring Mechanism
I consider the setting where reviewers offer very noisy scores for a num...
research
∙
10/11/2021
Imitating Deep Learning Dynamics via Locally Elastic Stochastic Differential Equations
Understanding the training dynamics of deep learning models is perhaps a...
research
∙
10/06/2021
An Unconstrained Layer-Peeled Perspective on Neural Collapse
Neural collapse is a highly symmetric geometric pattern of neural networ...
research
∙
05/28/2021
Weighted Training for Cross-Task Learning
In this paper, we introduce Target-Aware Weighted Training (TAWT), a wei...
research
∙
05/27/2021
Characterizing the SLOPE Trade-off: A Variational Perspective and the Donoho-Tanner Limit
Sorted l1 regularization has been incorporated into many methods for sol...
research
∙
05/18/2021
Oneshot Differentially Private Top-k Selection
Being able to efficiently and accurately select the top-k elements witho...
research
∙
04/05/2021
Rejoinder: Gaussian Differential Privacy
In this rejoinder, we aim to address two broad issues that cover most co...
research
∙
03/15/2021
A Central Limit Theorem for Differentially Private Query Answering
Perhaps the single most important use case for differential privacy is t...
research
∙
01/29/2021
Layer-Peeled Model: Toward Understanding Well-Trained Deep Neural Networks
In this paper, we introduce the Layer-Peeled Model, a nonconvex yet anal...
research
∙
10/27/2020
Toward Better Generalization Bounds with Locally Elastic Stability
Classical approaches in learning theory are often seen to yield very loo...
research
∙
10/22/2020
Label-Aware Neural Tangent Kernel: Toward Better Generalization and Local Elasticity
As a popular approach to modeling the dynamics of training overparametri...
research
∙
10/22/2020
Sharp Bias-variance Tradeoffs of Hard Parameter Sharing in High-dimensional Linear Regression
Hard parameter sharing for multi-task learning is widely used in empiric...
research
∙
10/20/2020
Towards Understanding the Dynamics of the First-Order Adversaries
An acknowledged weakness of neural networks is their vulnerability to ad...
research
∙
07/30/2020
A Power Analysis for Knockoffs with the Lasso Coefficient-Difference Statistic
In a linear model with possibly many predictors, we consider variable se...
research
∙
07/21/2020
The Complete Lasso Tradeoff Diagram
A fundamental problem in the high-dimensional regression is to understan...
research
∙
07/01/2020
The Price of Competition: Effect Size Heterogeneity Matters in High Dimensions
In high-dimensional linear regression, would increasing effect sizes alw...
research
∙
04/15/2020
On Learning Rates and Schrödinger Operators
The learning rate is perhaps the single most important parameter in the ...
research
∙
03/10/2020
Sharp Composition Bounds for Gaussian Differential Privacy via Edgeworth Expansion
Datasets containing sensitive information are often sequentially analyze...
research
∙
11/26/2019
Deep Learning with Gaussian Differential Privacy
Deep learning models are often trained on datasets that contain sensitiv...
research
∙
10/15/2019
The Local Elasticity of Neural Networks
This paper presents a phenomenon in neural networks that we refer to as ...
research
∙
05/07/2019
Gaussian Differential Privacy
Differential privacy has seen remarkable success as a rigorous and pract...
research
∙
02/11/2019
Acceleration via Symplectic Discretization of High-Resolution Differential Equations
We study first-order optimization methods obtained by discretizing ordin...
research
∙
12/21/2018
The FDR-Linking Theorem
This paper introduces the FDR-linking theorem, a novel technique for und...
research
∙
10/21/2018
Understanding the Acceleration Phenomenon via High-Resolution Differential Equations
Gradient-based optimization algorithms can be studied from the perspecti...
research
∙
07/11/2018
Differentially Private False Discovery Rate Control
Differential privacy provides a rigorous framework for privacy-preservin...
research
∙
07/01/2018
