We introduce Bit-Swap, a scalable and effective lossless data compression
technique based on deep learning. It extends previous work on practical
compression with latent variable models, based on bits-back coding and
asymmetric numeral systems. In our experiments Bit-Swap is able to beat
benchmark compressors on a highly diverse collection of images. We’re releasing
code for the method and optimized models such that people can explore and
advance this line of modern compression ideas. We also release a demo and
a pre-trained model for Bit-Swap image compression and decompression on your
own image. See the end of the post for a talk that covers how bits-back coding
and Bit-Swap works.
Continue
Nicholas Carlini
Aug 13, 2019
It is important whenever designing new technologies to ask “how will this
affect people’s privacy?” This topic is especially important with regard to
machine learning, where machine learning models are often trained on sensitive
user data and then released to the public. For example, in the last few years
we have seen models trained on users’ private emails, text
messages,
and medical records.
This article covers two aspects of our upcoming USENIX Security
paper that investigates to what extent
neural networks memorize rare and unique aspects of their training data.
Specifically, we quantitatively study to what extent following
problem actually occurs in practice:
Continue
To operate successfully in a complex and changing environment, learning agents must be able to acquire new skills quickly.
Humans display remarkable
skill in this area — we can learn to recognize a new object from one example,
adapt to driving a different car in a matter of minutes, and add a new slang
word to our vocabulary after hearing it once. Meta-learning is a promising
approach for enabling such capabilities in machines. In this paradigm, the
agent adapts to a new task from limited data by leveraging a wealth of
experience collected in performing related tasks. For agents that must take
actions and collect their own experience, meta-reinforcement learning (meta-RL)
holds the promise of enabling fast adaptation to new scenarios. Unfortunately,
while the trained policy can adapt quickly to new tasks, the meta-training
process requires large amounts of data from a range of training tasks,
exacerbating the sample inefficiency that plagues RL algorithms. As a result,
existing meta-RL algorithms are largely feasible only in simulated
environments. In this post, we’ll briefly survey the current landscape of
meta-RL and then introduce a new algorithm called PEARL that drastically
improves sample efficiency by orders of magnitude. (Check out the research paper and the code.)
Continue
Daniel Ho, Eric Liang, Richard Liaw
Jun 7, 2019
Effect of Population Based Augmentation applied to images, which differs at different percentages into training.
In this blog post we introduce Population Based Augmentation (PBA), an
algorithm that quickly and efficiently learns a state-of-the-art approach to
augmenting data for neural network training. PBA matches the previous best
result on CIFAR and SVHN but uses one thousand times less
compute, enabling researchers and practitioners to effectively learn
new augmentation policies using a single workstation GPU. You can use PBA
broadly to improve deep learning performance on image recognition tasks.
We discuss the PBA results from our recent paper and then show how
to easily run PBA for yourself on
a new data set in the Tune framework.
Continue
Eugene Vinitsky
Jun 3, 2019
We are in the midst of an unprecedented convergence of two rapidly growing
trends on our roadways: sharply increasing congestion and the deployment of
autonomous vehicles. Year after year, highways get slower and slower: famously,
China’s roadways were paralyzed by a two-week long traffic jam in 2010. At the
same time as congestion worsens, hundreds of thousands of semi-autonomous
vehicles (AVs), which are vehicles with automated distance and lane-keeping
capabilities, are being deployed on highways worldwide. The second trend offers
a perfect opportunity to alleviate the first. The current generation of AVs,
while very far from full autonomy, already hold a multitude of advantages over
human drivers that make them perfectly poised to tackle this congestion. Humans
are imperfect drivers: accelerating when we shouldn’t, braking aggressively,
and make short-sighted decisions, all of which creates and amplifies patterns
of congestion.
Continue
Communicating the goal of a task to another person is easy: we can use language, show them an image of the desired outcome, point them to a how-to video, or use some combination of all of these. On the other hand, specifying a task to a robot for reinforcement learning requires substantial effort. Most prior work that has applied deep reinforcement learning to real robots makes uses of specialized sensors to obtain rewards or studies tasks where the robot’s internal sensors can be used to measure reward. For example, using thermal cameras for tracking fluids, or purpose-built computer vision systems for tracking objects. Since such instrumentation needs to be done for any new task that we may wish to learn, it poses a significant bottleneck to widespread adoption of reinforcement learning for robotics, and precludes the use of these methods directly in open-world environments that lack this instrumentation.
We have developed an end-to-end method that allows robots to learn from a modest number of images that depict successful completion of a task, without any manual reward engineering. The robot initiates learning from this information alone (around 80 images), and occasionally queries a user for additional labels. In these queries, the robot shows the user an image and asks for a label to determine whether that image represents successful completion of the task or not. We require a small number of such queries (around 25-75), and using these queries, the robot is able to learn directly in the real world in 1-4 hours of interaction time, resulting in one of the most efficient real-world image-based robotic RL methods. We have open-sourced our implementation.
Our method allows us to solve a host of real world robotics problems from pixels in an end-to-end fashion without any hand-engineered reward functions.
Continue
Imagine a robot trying to learn how to stack blocks and push objects using
visual inputs from a camera feed. In order to minimize cost and safety
concerns, we want our robot to learn these skills with minimal interaction
time, but efficient learning from complex sensory inputs such as images is
difficult. This work introduces SOLAR, a
new model-based reinforcement learning (RL) method that can learn skills –
including manipulation tasks on a real Sawyer robot arm – directly from
visual inputs with under an hour of interaction. To our knowledge, SOLAR is the
most efficient RL method for solving real world image-based robotics tasks.
Our robot learns to stack a Lego block and push a mug onto a coaster with only
inputs from a camera pointed at the robot. Each task takes an hour or less of
interaction to learn.
Continue
Existing Computer Vision Setting v.s. Real-World Scenario
One day, an ecologist came to us. He wanted to use modern computer vision
techniques to perform automatic animal identification in his wildlife camera
trap image datasets. We were so confident because it sounded just like a basic
image classification problem. However, we failed. The dataset he provided was
extremely long-tailed and open-ended. As usual, when we did not have enough
training data, we asked if it was possible to provide more data for the tail
classes and just ignore the open classes that might appear in the testing
dataset. Unfortunately, collecting more data was not the option. It could take
an extremely long time for these ecologists to take photos of rare and secluded
animals in the wild. For some endangered animals, they even had to wait for
years for one single shot. At the same time, new animal species kept coming in,
and old animal species kept leaving. The total class number was never fixed in
such a dynamic system. Moreover, the identification of rare and new animals has
more conservational values than abundant animals. If we could only do well on
the abundant classes, the method would never be practically usable. We tried
all possible methods we could think of (data augmentation, sampling techniques,
few-shot learning, imbalanced classification, etc.); but none of the existing
methods could handle abundant classes, scarce classes and open classes at the
same time (Fig. 1).
Figure 1: There exists a considerable gap between the existing computer vision
setting and the real-world scenario.
Continue
Figure 1: Our model-based meta reinforcement learning algorithm enables a
legged robot to adapt online in the face of an unexpected system
malfunction (note the broken front right leg).
Humans have the ability to seamlessly adapt to changes in their environments:
adults can learn to walk on crutches in just a few seconds, people can adapt
almost instantaneously to picking up an object that is unexpectedly heavy, and
children who can walk on flat ground can quickly adapt their gait to walk
uphill without having to relearn how to walk. This adaptation is critical for
functioning in the real world.
Continue
Annie Xie
Apr 11, 2019
In many animals, tool-use skills emerge from a combination of observational
learning and experimentation. For example, by watching one another, chimpanzees
can learn how to use twigs to “fish” for insects. Similarly, capuchin
monkeys demonstrate the ability to wield sticks as sweeping tools to pull
food closer to themselves. While one might wonder whether these are just
illustrations of “monkey see, monkey do,” we believe these tool-use abilities
indicate a greater level of intelligence.
Left: A chimpanzee fishing for termites. Right: A gorilla using a stick to
gather herbs. (source)
The question our new work explores is: can we enable robots to use tools in the
same way — through observation and experimentation?
A requisite for performing complex multi-object manipulation tasks, such as
those involved in tool use, is an understanding of physical cause-and-effect
relationships. Therefore, the ability to predict how one object might
interact with another is crucial. Our prior work has investigated how
visual predictive models of cause-and-effect can be learned from unsupervised
robot interaction with the world. After learning such a model, the robot can
plan to accomplish a diverse set of simple tasks, including cloth folding and
object arrangement. However, if we consider the more complex interactions that
occur in tool-use tasks, such as how a broom can sweep dirt into a dustpan,
undirected experimentation isn’t enough.
Hence, taking inspiration from how animals learn, we designed an algorithm that
allows robots to learn tool-use skills through a similar paradigm of imitation
and interaction. In particular, we show that, with a mix of demonstration data
and unsupervised experience, a robot can use novel objects as tools and even
improvise tools in the absence of traditional ones. Further, depending on the
demands of the task, our method demonstrates the ability to decide whether to
use the provided tools. In this post, we will describe how this works.
Continue
