Life Long Learning Manifesto

(a.k.a. “Continual Learning”)

Draft of 10/26/2019 – Jaime Carbonell

Life-long learning (LLL) has been a dream in Artificial Intelligence for decades. This document sets forth a set of desiderata for modern LLL. Unlike past efforts such as CYC and NeLL, which accumulated passive knowledge, a true life-long learner should be able to learn and perform new tasks over time:

  1. without forgetting how to perform useful previously-learned tasks,
  2. at a faster rate than was need to learn earlier related tasks,
  3. with increasing ability to generalize across tasks,
  4. with parsimonious memory requirements as knowledge accumulates,
  5. with selective forgetting or suppression of no-longer accurate knowledge,
  6. with the ability to reflect upon what it can do and what it does not yet know.
Some of the above desiderata map to current areas of research. For instance a known phenomenon in neural nets is known as “catastrophic forgetting;” when trained sequentially to perform a series of tasks performance degrades sharply on earlier tasks as new ones are learned. The areas of transfer learning and multi-task learning address learning rates for new tasks by transferring knowledge from earlier ones to speed learning and to make it possible with when little training data is available for the new related tasks.

This Manifesto has been posted "as is" in tribute to Jaime Carbonell, the long-time directory of the Language Technologies Institute at Carnegie Mellon University, who died following a long illness on February 28, 2020.

Byron Spice (2020-02-28). "LTI Mourns Loss of Founder and Director Jaime Carbonell".

Further discussion of lifelong learning and other issues raised in the manifesto may be found at: