Copy of `MIT - Press Statistics Glossary`

The wordlist doesn't exist anymore, or, the website doesn't exist anymore. On this page you can find a copy of the original information. The information may have been taken offline because it is outdated.

MIT - Press Statistics Glossary Category: Mathematics and statistics Date & country: 13/09/2007, USA Words: 279

Activation
The time-varying value that is the output of a neuron.

Activation Function
A function that translates a neuron's net input to an activation value.

Adaptation
An internal change in a system that mirrors an external event in the system's environment.

Adaptive
Subject to adaptation; can change over time to improve fitness or accuracy.

Affine
An equation that can be written in terms of matrix-vector multiplication and vector addition.

Agent
See Autonomous Agent.

AI
An abbreviation for Artificial Intelligence.

Algorithm
A detailed and unambiguous sequence of instructions that describes how a computation is to proceed and can be implemented as a program.

Algorithmic Complexity
The size of the smallest program that can produce a particular sequence of numbers. Regular patterns have low algorithmic complexity and random sequences have high algorithmic complexity.

Always Cooperate
A Prisoner's Dilemma strategy that cooperates with its opponent under all circumstances (the exact opposite of always defect).

Always Defect
A Prisoner's Dilemma strategy that never cooperates with its opponent under any circumstance (the exact opposite of always cooperate).

Analog
Having a continuous value.

Analytical
Can be symbolically represented in a closed form that does not require any of the complex aspects of a program such as an iterative sum.

Analytical Solution
An exact solution to a problem that can be calculated symbolically by manipulating equations (unlike a numerical solution).

Arms Race
Two or more species experience adaptation to one another in a coevolutionary manner. This often seen in predator-prey systems.

Artificial Intelligence
The science of making computers do interesting things that humans do effortlessly.

Artificial Life
The study of life processes within the confines of a computer.

Associative Memory
Memory that can be referenced by content, as opposed to location. Hopfield networks will act as associative memories when trained with the Hebbian learning rule.

Asynchronous
Describes events that occur independently of each other but on a similar time scale.

Attractor
A characterization of the long-term behavior of a dissipative dynamical system. Over long periods of time, the state space of some dynamical systems will contract toward this region. Attractors may be fixed points, periodic, quasiperiodic, or chaotic. They may also be stable or unstable.

Autonomous Agent
An entity with limited perception of its environment that can process information to calculate an action so as to be goal-seeking on a local scale. A boid is an example of an autonomous agent.

Axiom
A statement that is assumed to be true and can later be used along with theorems to prove other theorems. Also, the starting configuration of an L-System.

Backpropagation
An algorithm for efficiently calculating the error gradient of a neural network, which can then be used as the basis of learning. Backpropagation is equivalent to the delta rule for perceptrons, but can also calculate appropriate weight changes for the hidden layer weights of a multilayer perceptron by generalizing the notion of an error correction term. In the simplest case, backpropagation is a type of steepest descent in the search space of the network weights, and it will usually converge to…

Basin of Attraction
A region of state space in which all included states of a dynamical system ultimately lead into the attractor.

Bias
See threshold.

Bifurcation
The splitting of a single mode of a system's behavior into two new modes. This usually occurs as a function of a continuously varying control parameter. A cascade of bifurcations will usually precede the onset of chaos.

Binary
Written in a form that uses only 0s and 1s. A string of bits.

Bit
The smallest unit of information; the answer to a yes/no question; the outcome of a coin toss; a 0 or a 1.

Boid
An autonomous agent that behaves like a simplified bird but will display flocking patterns in the presence of other boids.

Boolean
Taking only 0/1, true/false, yes/no values.

Bottom-Up
A description that uses the lower-level details to explain higher-level patterns; related to reductionism.

Brown Noise-Brownian Motion
A form of randomness that is the result of cumulatively adding white noise, to yield a random walk pattern.

Bucket Brigade Algorithm
A learning algorithm that is a method for adjusting the strengths of the classifiers of a classifier system. ``Winning'' classifiers pay a portion of their earnings to other classifiers that assisted them in being activated, similar to an economic system.

Byte
Eight bits. In programming, often used to store a single text character.

Cantor Set
A simple fractal set composed of an uncountable infinity of dust-like points, but that also has 0 measure (meaning that the sum width of all points is 0). The Cantor set is constructed by removing the middle third of a unit line segment, and then recursively removing the middle third of any remaining line segments, for an infinite number of steps.

Cellular Automaton (CA)
A discrete dynamical system that is composed of an array of cells, each of which behaves like a finite-state automaton. All interactions are local, with the next state of a cell being a function of the current state of itself and its neighbors. Conway's Game of Life is a CA.

Chaos-Chaotic
Irregular motion of a dynamical system that is deterministic, sensitive to initial conditions, and impossible to predict in the long term with anything less than an infinite and perfect representation of analog values.

Chomsky Hierarchy
Four classes of languages (or computing machines) that have increasing complexity: regular (finite-state automata), context-free (push-down automata), context-sensitive (linear bounded automata), and recursive (Turing machines).

Classifier
A rule that is part of a classifier system and has a condition that must be matched before its message (or action) can be posted (or effected). The strength of a classifier determines the likelihood that it can outbid other classifiers if more than one condition is matched.

Classifier System
An adaptive system similar to a Post production system that contains many ``if ... then'' rules called classifiers. The state of the environment is encoded as a message by a detector and placed on the message list from which the condition portion of the classifiers can be matched. ``Winning'' classifiers can then post their own messages to the message list, ultimately forming a type of computation that may result in a message being translated into an action by an effector. The strengths of the c…

Co-Recursively Enumerable (CO-RE)
The complement of a set that can be recursively enumerated.

Coevolution
Two or more entities experience evolution in response to one another. Due to feedback mechanisms, this often results in a biological arms race.

Combinatorial Optimization
A class of problems in which the number of candidate solutions is combinatorial in size. Each possible solution has an associated cost. The goal is to find the solution with the lowest cost. Because of the vast numbers involved, explicit search an entire search space is not always possible.

Complement
A set composed of all elements that are not members of another set.

Complete
Describes a formal system in which all statements can be proved as being true or false. Most interesting formal systems are not complete, as proved in Gödel's Incompleteness Theorem.

Complex Number
A number that has a real component and an imaginary component and is characterized as a point on a plane (instead of the real number line).

Complex System
A collection of many simple nonlinear units that operate in parallel and interact locally with each other so as to produce emergent behavior.

Complexity
An ill-defined term that means many things to many people. Complex things are neither random nor regular, but hover somewhere in between. Intuitively, complexity is a measure of how interesting something is. Other types of complexity may be well defined; see the index for other references.

Compressible
Having a description that is smaller than itself; not random; possessing regularity.

Computable
Expressible as a yes/no question that can be answered in any case by a computer in finite time.

Computation
The realization of a program in a computer.

Connectivity
The amount of interaction in a system, the structure of the weights in a neural network, or the relative number of edges in a graph.

Conservative System
A dynamical system that preserves the volume of its state space under motion and, therefore, does not display the types of behavior found in dissipative systems.

Consistence
In formal systems, having the property that all statements are either true or false.

Continuous
Taking a real value, i.e., not discrete. Dynamical systems may operate in continuous time or space.

Control
Exerting actions to manipulate a system or environment in a goal-seeking manner.

Convergence
For computers, halting with an answer; for dynamical systems, falling into an attractor; for searches (e.g., backpropagation and genetic algorithms), finding a location that cannot be improved upon; for infinite summations, approaching a definite value.

Conway's Game of Life
A cellular automaton rule set that operates on a two-dimensional grid. Each cell changes its state according to the states of its eight nearest neighbors: dead cells come alive with exactly three live neighbors, and cells die if they have anything but two or three neighbors. The Game of Life can display complex patterns such as gliders, fish, and glider guns, and is also capable of universal computation.

Countable Infinity
Having the same number of objects as the set of natural numbers.

Crossover
A genetic operator that splices information from two or more parents to form a composite offspring that has genetic material from all parents.

Darwinism
The theory of evolution as proposed by Charles Darwin, which combined variation of inheritable traits with natural selection. After the discovery of the physical mechanism of genetics, this was further refined into neo-Darwinism.

Decision Problem
A problem in which all questions take the form ``Is something a member of a particular set?'' and all answers are either ``yes'' or ``no.''

Delta Rule
The perceptron learning rule that specifies that weight changes should be proportional to the product of a weight's input and the error (or delta) term for the perceptron.

Derivative
An expression that characterizes how the output of a function changes as the input is varied. Unlike integrals, derivatives can be calculated in an analytical manner very easily.

Detector
A sensor that translates the state of a classifier's environment into a message that is suitable for posting to the message list of the classifier system.

Determinant
A quantity of a matrix that characterizes the amount of expansion or contraction that the matrix inflicts on a vector when that vector is multiplied by the matrix.

Deterministic
Occurring in a non-random manner such that the next state of a system depends only on prior states of the system or the environment. Perfect knowledge of previous states implies perfect knowledge of the next state.

Diagonal Matrix
A matrix that has 0 entries along all nondiagonal entries, i.e., only the main diagonal may have non-zero values.

Difference Equation
An equation that describes how something changes in discrete time steps. Numerical solutions to integrals are usually realized as difference equations.

Differential Equation
A description of how something continuously changes over time. Some differential equations can have an analytical solution such that all future states can be known without simulation of the time evolution of the system. However, most can have a numerical solution with only limited accuracy.

Differentiation
The act of calculating a derivative; the inverse operation of calculating an integral.

Diffusion Limited Aggregation
A type of stochastic fractal formed by particles floating about in a random manner until they stick to something solid.

Discrete
Taking only non-continuous values, e.g., Boolean or natural numbers.

Dissipative System
A dynamical system that contains internal friction that deforms the structure of its attractor, thus making motion such as fixed points, limit cycles, quasiperiodicity, and chaos possible. Dissipative systems often have internal structure despite being far from equilibrium, like a whirlpool that preserves its basic form despite being in the midst of constant change.

Diverge
For algorithms or computers, to run forever and never halt; for iterative systems (like the equations for the Mandelbrot set), reaching a state such that all future states explode in size.

Dot Product
The inner product of two vectors.

Dynamical System
A system that changes over time according to a set of fixed rules that determine how one state of the system moves to another state.

Dynamics-Dynamical
Pertaining to the change in behavior of a system over time.

Ecology
The study of the relationships and interactions between organisms and environments.

Ecosystem
A biological system consisting of many organisms from different species.

Edge of Chaos
The hypothesis that many natural systems tend toward dynamical behavior that borders static patterns and the chaotic regime.

Effector
The part of a classifier system that can translate messages into actions that can manipulate a system or an environment.

Eigenvalue
The change in length that occurs when the corresponding eigenvector is multiplied by its matrix.

Eigenvector
A unit length vector that retains its direction when multiplied to the matrix that it corresponds to. An (n * n) matrix can have as many as n unique eigenvectors, each of which will have its own eigenvalue.

Embedding
A method of taking a scalar time series and using delayed snapshots of the values at fixed time intervals in the past so that the dynamics of the underlying system can be observed as a function of the previously observed states.

Emergent
Refers to a property of a collection of simple subunits that comes about through the interactions of the subunits and is not a property of any single subunit. For example, the organization of an ant colony is said to ``emerge'' from the interactions of the lower-level behaviors of the ants, and not from any single ant. Usually, the emergent behavior is unanticipated and cannot be directly deduced from the lower-level behaviors. Complex systems are usually emergent.

Entropy
A measure of a system's degree of randomness or disorder.

Environment
If that which is under study is a system, then the rest of the world is the environment.

Equilibrium
A state of a system that, if not subjected to perturbation, will remain unchanged.

Ergodic
The property of a dynamical system such that all regions of a state space are visited with similar frequency and that all regions will be revisited (within a small proximity) if given enough time.

Euclidean
Pertaining to standard geometry, i.e., points, lines, planes, volumes, squares, cubes, triangles, etc.

Euler's Method
The simplest method of obtaining a numerical solution of a differential equation. There are many other numerical techniques that are more accurate; however, an analytical solution (i.e., a closed form of an integral) is always preferred but not always possible.

Evolution
A process operating on populations that involves variation among individuals, traits being inheritable, and a level of fitness for individuals that is a function of the possessed traits. Over relatively long periods of time, the distribution of inheritable traits will tend to reflect the fitness that the traits convey to the individual; thus, evolution acts as a filter that selects fitness-yielding traits over other traits.

Evolutionary Stable Strategy (ESS)
In game theory and biology, a strategy that, when possessed by an entire population, results in an equilibrium such that mutation of the strategy can never result in an improvement for an individual. Always Defect is an ESS, while Always Cooperate is not.

Excitatory
Refers to a neural synapse or weight that is positive such that activity in the source neuron encourages activity in the connected neuron; the opposite of inhibitory.

Experimentation
One process by which scientists attempt to understand nature. A phenomenon is observed and/or manipulated so that changes in the phenomenon's state can be seen. The resulting data can be used to derive new models of a process or to confirm an existing model. Experimentation is the complement of theorization. See also simulation.

Expert System
A special program that resembles a collection of ``if ... then'' rules. The rules usually represent knowledge contained by a domain expert (such as a physician adept at diagnosis) and can be used to simulate how a human expert would perform a task.

Feedback
A loop in information flow or in cause and effect.

Feedback Neural Network
A neural network that has every neuron potentially connected to every other neuron. The activations of all neurons are updated in parallel (synchronous or asynchronous order), unlike a feedforward or recurrent neural network.

Feedforward Neural Network
A neural network that is organized with separate layers of neurons. Connections in such a network are limited to one direction such that the activations of the input neurons are updated first, followed by any hidden layers, and then finished with the outputs.