research
∙
07/24/2023
RLCD: Reinforcement Learning from Contrast Distillation for Language Model Alignment
We propose Reinforcement Learning from Contrast Distillation (RLCD), a m...
share
research
∙
07/18/2023
Landscape Surrogate: Learning Decision Losses for Mathematical Optimization Under Partial Information
Recent works in learning-integrated optimization have shown promise in s...
research
∙
06/27/2023
Extending Context Window of Large Language Models via Positional Interpolation
We present Position Interpolation (PI) that extends the context window s...
research
∙
06/24/2023
H_2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models
Large Language Models (LLMs), despite their recent impressive accomplish...
research
∙
05/25/2023
Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer
Transformer architecture has shown impressive performance in multiple re...
research
∙
05/03/2023
Pre-train and Search: Efficient Embedding Table Sharding with Pre-trained Neural Cost Models
Sharding a large machine learning model across multiple devices to balan...
research
∙
04/24/2023
A Cookbook of Self-Supervised Learning
Self-supervised learning, dubbed the dark matter of intelligence, is a p...
research
∙
02/03/2023
Searching Large Neighborhoods for Integer Linear Programs with Contrastive Learning
Integer Linear Programs (ILPs) are powerful tools for modeling and solvi...
research
∙
01/09/2023
Learning to compile smartly for program size reduction
Compiler optimization passes are an important tool for improving program...
research
∙
12/20/2022
DOC: Improving Long Story Coherence With Detailed Outline Control
We propose the Detailed Outline Control (DOC) framework for improving lo...
research
∙
12/15/2022
Local Branching Relaxation Heuristics for Integer Linear Programs
Large Neighborhood Search (LNS) is a popular heuristic algorithm for sol...
research
∙
11/23/2022
EurNet: Efficient Multi-Range Relational Modeling of Spatial Multi-Relational Data
Modeling spatial relationship in the data remains critical across many d...
research
∙
10/22/2022
SurCo: Learning Linear Surrogates For Combinatorial Nonlinear Optimization Problems
Optimization problems with expensive nonlinear cost functions and combin...
research
∙
10/13/2022
Re3: Generating Longer Stories With Recursive Reprompting and Revision
We consider the problem of automatically generating longer stories of ov...
research
∙
08/22/2022
Efficient Planning in a Compact Latent Action Space
While planning-based sequence modelling methods have shown great potenti...
research
∙
08/12/2022
AutoShard: Automated Embedding Table Sharding for Recommender Systems
Embedding learning is an important technique in deep recommendation mode...
research
∙
06/30/2022
Denoised MDPs: Learning World Models Better Than the World Itself
The ability to separate signal from noise, and reason with clean abstrac...
research
∙
06/02/2022
Understanding the Role of Nonlinearity in Training Dynamics of Contrastive Learning
While the empirical success of self-supervised learning (SSL) heavily re...
research
∙
04/01/2022
On the Importance of Asymmetry for Siamese Representation Learning
Many recent self-supervised frameworks for visual representation learnin...
research
∙
02/11/2022
Understanding Curriculum Learning in Policy Optimization for Solving Combinatorial Optimization Problems
Over the recent years, reinforcement learning (RL) has shown impressive ...
research
∙
01/29/2022
Deep Contrastive Learning is Provably (almost) Principal Component Analysis
We show that Contrastive Learning (CL) under a family of loss functions ...
research
∙
12/16/2021
Learning Bounded Context-Free-Grammar via LSTM and the Transformer:Difference and Explanations
Long Short-Term Memory (LSTM) and Transformers are two popular neural ar...
research
∙
11/19/2021
Dynamic Graph Representation Learning via Graph Transformer Networks
Dynamic graph representation learning is an important task with widespre...
research
∙
10/11/2021
Towards Demystifying Representation Learning with Non-contrastive Self-supervision
Non-contrastive methods of self-supervised learning (such as BYOL and Si...
research
∙
10/07/2021
Multi-objective Optimization by Learning Space Partitions
In contrast to single-objective optimization (SOO), multi-objective opti...
research
∙
09/17/2021
CompilerGym: Robust, Performant Compiler Optimization Environments for AI Research
Interest in applying Artificial Intelligence (AI) techniques to compiler...
research
∙
06/29/2021
Latent Execution for Neural Program Synthesis Beyond Domain-Specific Languages
Program synthesis from input-output examples has been a long-standing ch...
research
∙
06/19/2021
Learning Space Partitions for Path Planning
Path planning, the problem of efficiently discovering high-reward trajec...
research
∙
06/18/2021
MADE: Exploration via Maximizing Deviation from Explored Regions
In online reinforcement learning (RL), efficient exploration remains par...
research
∙
02/25/2021
Understanding Robustness in Teacher-Student Setting: A New Perspective
Adversarial examples have appeared as a ubiquitous property of machine l...
research
∙
02/17/2021
Provably Efficient Policy Gradient Methods for Two-Player Zero-Sum Markov Games
Policy gradient methods are widely used in solving two-player zero-sum g...
research
∙
02/12/2021
Understanding self-supervised Learning Dynamics without Contrastive Pairs
Contrastive approaches to self-supervised learning (SSL) learn represent...
research
∙
11/22/2020
FP-NAS: Fast Probabilistic Neural Architecture Search
Differential Neural Architecture Search (NAS) requires all layer choices...
research
∙
10/01/2020
Understanding Self-supervised Learning with Dual Deep Networks
We propose a novel theoretical framework to understand self-supervised l...
research
∙
08/28/2020
Real-world Video Adaptation with Reinforcement Learning
Client-side video players employ adaptive bitrate (ABR) algorithms to op...
research
∙
08/27/2020
Planning in Learned Latent Action Spaces for Generalizable Legged Locomotion
Hierarchical learning has been successful at learning generalizable loco...
research
∙
08/14/2020
Joint Policy Search for Multi-agent Collaboration with Imperfect Information
To learn good joint policies for multi-agent collaboration with imperfec...
research
∙
07/01/2020
Learning Search Space Partition for Black-box Optimization using Monte Carlo Tree Search
High dimensional black-box optimization has broad applications but remai...
research
∙
06/29/2020
Towards Automated Neural Interaction Discovery for Click-Through Rate Prediction
Click-Through Rate (CTR) prediction is one of the most important machine...
research
∙
06/11/2020
Few-shot Neural Architecture Search
To improve the search efficiency for Neural Architecture Search (NAS), O...
research
∙
06/03/2020
FBNetV3: Joint Architecture-Recipe Search using Neural Acquisition Function
Neural Architecture Search (NAS) yields state-of-the-art neural networks...
research
∙
04/12/2020
FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensions
Differentiable Neural Architecture Search (DNAS) has demonstrated great ...
research
∙
09/30/2019
Over-parameterization as a Catalyst for Better Generalization of Deep ReLU network
To analyze deep ReLU network, we adopt a student-teacher setting in whic...
research
∙
09/10/2019
Bayesian Relational Memory for Semantic Visual Navigation
We introduce a new memory architecture, Bayesian Relational Memory (BRM)...
research
∙
06/28/2019
A Neural-based Program Decompiler
Reverse engineering of binary executables is a critical problem in the c...
research
∙
06/17/2019
Sample-Efficient Neural Architecture Search by Learning Action Space
Neural Architecture Search (NAS) has emerged as a promising technique fo...
research
∙
06/06/2019
One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers
The success of lottery ticket initializations (Frankle and Carbin, 2019)...
research
∙
06/06/2019
Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLP
The lottery ticket hypothesis proposes that over-parameterization of dee...
research
∙
06/03/2019
Hierarchical Decision Making by Generating and Following Natural Language Instructions
We explore using latent natural language instructions as an expressive a...
research
∙
05/31/2019
