Copy of `MIT  Press Statistics Glossary`
The wordlist doesn't exist anymore, or, the website doesn't exist anymore. On this page you can find a copy of the original information. The information may have been taken offline because it is outdated.


MIT  Press Statistics Glossary
Category: Mathematics and statistics
Date & country: 13/09/2007, USA Words: 279

ActivationThe timevarying value that is the output of a neuron.
Activation FunctionA function that translates a neuron's net input to an activation value.
AdaptationAn internal change in a system that mirrors an external event in the system's environment.
AdaptiveSubject to adaptation; can change over time to improve fitness or accuracy.
AffineAn equation that can be written in terms of matrixvector multiplication and vector addition.
AgentSee Autonomous Agent.
AIAn abbreviation for Artificial Intelligence.
AlgorithmA detailed and unambiguous sequence of instructions that describes how a computation is to proceed and can be implemented as a program.
Algorithmic ComplexityThe size of the smallest program that can produce a particular sequence of numbers. Regular patterns have low algorithmic complexity and random sequences have high algorithmic complexity.
Always CooperateA Prisoner's Dilemma strategy that cooperates with its opponent under all circumstances (the exact opposite of always defect).
Always DefectA Prisoner's Dilemma strategy that never cooperates with its opponent under any circumstance (the exact opposite of always cooperate).
AnalogHaving a continuous value.
AnalyticalCan be symbolically represented in a closed form that does not require any of the complex aspects of a program such as an iterative sum.
Analytical SolutionAn exact solution to a problem that can be calculated symbolically by manipulating equations (unlike a numerical solution).
Arms RaceTwo or more species experience adaptation to one another in a coevolutionary manner. This often seen in predatorprey systems.
Artificial IntelligenceThe science of making computers do interesting things that humans do effortlessly.
Artificial LifeThe study of life processes within the confines of a computer.
Associative MemoryMemory that can be referenced by content, as opposed to location. Hopfield networks will act as associative memories when trained with the Hebbian learning rule.
AsynchronousDescribes events that occur independently of each other but on a similar time scale.
AttractorA characterization of the longterm behavior of a dissipative dynamical system. Over long periods of time, the state space of some dynamical systems will contract toward this region. Attractors may be fixed points, periodic, quasiperiodic, or chaotic. They may also be stable or unstable.
Autonomous AgentAn entity with limited perception of its environment that can process information to calculate an action so as to be goalseeking on a local scale. A boid is an example of an autonomous agent.
AxiomA statement that is assumed to be true and can later be used along with theorems to prove other theorems. Also, the starting configuration of an LSystem.
BackpropagationAn algorithm for efficiently calculating the error gradient of a neural network, which can then be used as the basis of learning. Backpropagation is equivalent to the delta rule for perceptrons, but can also calculate appropriate weight changes for the hidden layer weights of a multilayer perceptron by generalizing the notion of an error correction term. In the simplest case, backpropagation is a type of steepest descent in the search space of the network weights, and it will usually converge toâ€¦
Basin of AttractionA region of state space in which all included states of a dynamical system ultimately lead into the attractor.
BiasSee threshold.
BifurcationThe splitting of a single mode of a system's behavior into two new modes. This usually occurs as a function of a continuously varying control parameter. A cascade of bifurcations will usually precede the onset of chaos.
BinaryWritten in a form that uses only 0s and 1s. A string of bits.
BitThe smallest unit of information; the answer to a yes/no question; the outcome of a coin toss; a 0 or a 1.
BoidAn autonomous agent that behaves like a simplified bird but will display flocking patterns in the presence of other boids.
BooleanTaking only 0/1, true/false, yes/no values.
BottomUpA description that uses the lowerlevel details to explain higherlevel patterns; related to reductionism.
Brown NoiseBrownian MotionA form of randomness that is the result of cumulatively adding white noise, to yield a random walk pattern.
Bucket Brigade AlgorithmA learning algorithm that is a method for adjusting the strengths of the classifiers of a classifier system. ``Winning'' classifiers pay a portion of their earnings to other classifiers that assisted them in being activated, similar to an economic system.
ByteEight bits. In programming, often used to store a single text character.
Cantor SetA simple fractal set composed of an uncountable infinity of dustlike points, but that also has 0 measure (meaning that the sum width of all points is 0). The Cantor set is constructed by removing the middle third of a unit line segment, and then recursively removing the middle third of any remaining line segments, for an infinite number of steps.
Cellular Automaton (CA)A discrete dynamical system that is composed of an array of cells, each of which behaves like a finitestate automaton. All interactions are local, with the next state of a cell being a function of the current state of itself and its neighbors. Conway's Game of Life is a CA.
ChaosChaoticIrregular motion of a dynamical system that is deterministic, sensitive to initial conditions, and impossible to predict in the long term with anything less than an infinite and perfect representation of analog values.
Chomsky HierarchyFour classes of languages (or computing machines) that have increasing complexity: regular (finitestate automata), contextfree (pushdown automata), contextsensitive (linear bounded automata), and recursive (Turing machines).
ClassifierA rule that is part of a classifier system and has a condition that must be matched before its message (or action) can be posted (or effected). The strength of a classifier determines the likelihood that it can outbid other classifiers if more than one condition is matched.
Classifier SystemAn adaptive system similar to a Post production system that contains many ``if ... then'' rules called classifiers. The state of the environment is encoded as a message by a detector and placed on the message list from which the condition portion of the classifiers can be matched. ``Winning'' classifiers can then post their own messages to the message list, ultimately forming a type of computation that may result in a message being translated into an action by an effector. The strengths of the câ€¦
CoRecursively Enumerable (CORE)The complement of a set that can be recursively enumerated.
CoevolutionTwo or more entities experience evolution in response to one another. Due to feedback mechanisms, this often results in a biological arms race.
Combinatorial OptimizationA class of problems in which the number of candidate solutions is combinatorial in size. Each possible solution has an associated cost. The goal is to find the solution with the lowest cost. Because of the vast numbers involved, explicit search an entire search space is not always possible.
ComplementA set composed of all elements that are not members of another set.
CompleteDescribes a formal system in which all statements can be proved as being true or false. Most interesting formal systems are not complete, as proved in Gödel's Incompleteness Theorem.
Complex NumberA number that has a real component and an imaginary component and is characterized as a point on a plane (instead of the real number line).
Complex SystemA collection of many simple nonlinear units that operate in parallel and interact locally with each other so as to produce emergent behavior.
ComplexityAn illdefined term that means many things to many people. Complex things are neither random nor regular, but hover somewhere in between. Intuitively, complexity is a measure of how interesting something is. Other types of complexity may be well defined; see the index for other references.
CompressibleHaving a description that is smaller than itself; not random; possessing regularity.
ComputableExpressible as a yes/no question that can be answered in any case by a computer in finite time.
ComputationThe realization of a program in a computer.
ConnectivityThe amount of interaction in a system, the structure of the weights in a neural network, or the relative number of edges in a graph.
Conservative SystemA dynamical system that preserves the volume of its state space under motion and, therefore, does not display the types of behavior found in dissipative systems.
ConsistenceIn formal systems, having the property that all statements are either true or false.
ContinuousTaking a real value, i.e., not discrete. Dynamical systems may operate in continuous time or space.
ControlExerting actions to manipulate a system or environment in a goalseeking manner.
ConvergenceFor computers, halting with an answer; for dynamical systems, falling into an attractor; for searches (e.g., backpropagation and genetic algorithms), finding a location that cannot be improved upon; for infinite summations, approaching a definite value.
Conway's Game of LifeA cellular automaton rule set that operates on a twodimensional grid. Each cell changes its state according to the states of its eight nearest neighbors: dead cells come alive with exactly three live neighbors, and cells die if they have anything but two or three neighbors. The Game of Life can display complex patterns such as gliders, fish, and glider guns, and is also capable of universal computation.
Countable InfinityHaving the same number of objects as the set of natural numbers.
CrossoverA genetic operator that splices information from two or more parents to form a composite offspring that has genetic material from all parents.
DarwinismThe theory of evolution as proposed by Charles Darwin, which combined variation of inheritable traits with natural selection. After the discovery of the physical mechanism of genetics, this was further refined into neoDarwinism.
Decision ProblemA problem in which all questions take the form ``Is something a member of a particular set?'' and all answers are either ``yes'' or ``no.''
Delta RuleThe perceptron learning rule that specifies that weight changes should be proportional to the product of a weight's input and the error (or delta) term for the perceptron.
DerivativeAn expression that characterizes how the output of a function changes as the input is varied. Unlike integrals, derivatives can be calculated in an analytical manner very easily.
DetectorA sensor that translates the state of a classifier's environment into a message that is suitable for posting to the message list of the classifier system.
DeterminantA quantity of a matrix that characterizes the amount of expansion or contraction that the matrix inflicts on a vector when that vector is multiplied by the matrix.
DeterministicOccurring in a nonrandom manner such that the next state of a system depends only on prior states of the system or the environment. Perfect knowledge of previous states implies perfect knowledge of the next state.
Diagonal MatrixA matrix that has 0 entries along all nondiagonal entries, i.e., only the main diagonal may have nonzero values.
Difference EquationAn equation that describes how something changes in discrete time steps. Numerical solutions to integrals are usually realized as difference equations.
Differential EquationA description of how something continuously changes over time. Some differential equations can have an analytical solution such that all future states can be known without simulation of the time evolution of the system. However, most can have a numerical solution with only limited accuracy.
DifferentiationThe act of calculating a derivative; the inverse operation of calculating an integral.
Diffusion Limited AggregationA type of stochastic fractal formed by particles floating about in a random manner until they stick to something solid.
DiscreteTaking only noncontinuous values, e.g., Boolean or natural numbers.
Dissipative SystemA dynamical system that contains internal friction that deforms the structure of its attractor, thus making motion such as fixed points, limit cycles, quasiperiodicity, and chaos possible. Dissipative systems often have internal structure despite being far from equilibrium, like a whirlpool that preserves its basic form despite being in the midst of constant change.
DivergeFor algorithms or computers, to run forever and never halt; for iterative systems (like the equations for the Mandelbrot set), reaching a state such that all future states explode in size.
Dot ProductThe inner product of two vectors.
Dynamical SystemA system that changes over time according to a set of fixed rules that determine how one state of the system moves to another state.
DynamicsDynamicalPertaining to the change in behavior of a system over time.
EcologyThe study of the relationships and interactions between organisms and environments.
EcosystemA biological system consisting of many organisms from different species.
Edge of ChaosThe hypothesis that many natural systems tend toward dynamical behavior that borders static patterns and the chaotic regime.
EffectorThe part of a classifier system that can translate messages into actions that can manipulate a system or an environment.
EigenvalueThe change in length that occurs when the corresponding eigenvector is multiplied by its matrix.
EigenvectorA unit length vector that retains its direction when multiplied to the matrix that it corresponds to. An (n * n) matrix can have as many as n unique eigenvectors, each of which will have its own eigenvalue.
EmbeddingA method of taking a scalar time series and using delayed snapshots of the values at fixed time intervals in the past so that the dynamics of the underlying system can be observed as a function of the previously observed states.
EmergentRefers to a property of a collection of simple subunits that comes about through the interactions of the subunits and is not a property of any single subunit. For example, the organization of an ant colony is said to ``emerge'' from the interactions of the lowerlevel behaviors of the ants, and not from any single ant. Usually, the emergent behavior is unanticipated and cannot be directly deduced from the lowerlevel behaviors. Complex systems are usually emergent.
EntropyA measure of a system's degree of randomness or disorder.
EnvironmentIf that which is under study is a system, then the rest of the world is the environment.
EquilibriumA state of a system that, if not subjected to perturbation, will remain unchanged.
ErgodicThe property of a dynamical system such that all regions of a state space are visited with similar frequency and that all regions will be revisited (within a small proximity) if given enough time.
EuclideanPertaining to standard geometry, i.e., points, lines, planes, volumes, squares, cubes, triangles, etc.
Euler's MethodThe simplest method of obtaining a numerical solution of a differential equation. There are many other numerical techniques that are more accurate; however, an analytical solution (i.e., a closed form of an integral) is always preferred but not always possible.
EvolutionA process operating on populations that involves variation among individuals, traits being inheritable, and a level of fitness for individuals that is a function of the possessed traits. Over relatively long periods of time, the distribution of inheritable traits will tend to reflect the fitness that the traits convey to the individual; thus, evolution acts as a filter that selects fitnessyielding traits over other traits.
Evolutionary Stable Strategy (ESS)In game theory and biology, a strategy that, when possessed by an entire population, results in an equilibrium such that mutation of the strategy can never result in an improvement for an individual. Always Defect is an ESS, while Always Cooperate is not.
ExcitatoryRefers to a neural synapse or weight that is positive such that activity in the source neuron encourages activity in the connected neuron; the opposite of inhibitory.
ExperimentationOne process by which scientists attempt to understand nature. A phenomenon is observed and/or manipulated so that changes in the phenomenon's state can be seen. The resulting data can be used to derive new models of a process or to confirm an existing model. Experimentation is the complement of theorization. See also simulation.
Expert SystemA special program that resembles a collection of ``if ... then'' rules. The rules usually represent knowledge contained by a domain expert (such as a physician adept at diagnosis) and can be used to simulate how a human expert would perform a task.
FeedbackA loop in information flow or in cause and effect.
Feedback Neural NetworkA neural network that has every neuron potentially connected to every other neuron. The activations of all neurons are updated in parallel (synchronous or asynchronous order), unlike a feedforward or recurrent neural network.
Feedforward Neural NetworkA neural network that is organized with separate layers of neurons. Connections in such a network are limited to one direction such that the activations of the input neurons are updated first, followed by any hidden layers, and then finished with the outputs.