Copy of `MIT  Press Statistics Glossary`
The wordlist doesn't exist anymore, or, the website doesn't exist anymore. On this page you can find a copy of the original information. The information may have been taken offline because it is outdated.


MIT  Press Statistics Glossary
Category: Mathematics and statistics
Date & country: 13/09/2007, USA Words: 279

Phase TransitionIn physics, a change from one state of matter to another. In dynamical systems theory, a change from one mode of behavior to another.
PlanningIn computer science, and particularly in artificial intelligence, the task of determining a stepwise plan to accomplish a very specific task.
PolynomialA function in which the output is the sum of terms that are the products of constant values and the input raised to some integer power. The polynomial of a polynomial is another polynomial. From a time complexity point of view, polynomials are wellbehaved.
Post Production SystemA model of computation that resembles a collection of ``if ... then'' rules and is capable of universal computation.
PredatorPrey SystemAn ecosystem in which one portion of the population consumes another. With three or more species, simple predatorprey interactions can lead to chaos and biological arms races. See also LotkaVolterra system.
Prime NumberA natural number that can be evenly divided only by itself and 1.
Prisoner's DilemmaA nonzerosum game in which both players have incentive not to cooperate under any circumstances. Thus, the optimal game theory strategy of always defect has the paradoxical property that both players would have a higher payoff if they ignored the advice of game theory.
ProbabilityThe likelihood that a random event will occur.
ProgramAn algorithm that is written in a programming language for execution on a physical computer.
ProofA sequence of statements in which each subsequent statement is derivable from one of the previous statements or from an axiom of a formal system. The final statement of a proof is usually the theorem that one has set out to prove.
QuasiperiodicRefers to a form of motion that is regular but never exactly repeating. Quasiperiodic motion is always composed of multiple but simpler periodic motions. In the general case, for motion that is the sum of simpler periodic motions, if there exists a length of time that evenly divides the frequencies of the underlying motions, then the composite motion will also be periodic; however, if no such length of time exists, then the motion will be quasiperiodic.
Random WalkA walk in one or more dimensions that is dictated by the outcome of a coin toss. The direction of each step of the walk is specified by the coin toss. The resulting random motion is often referred to as Brownian motion.
RandomRandomnessWithout cause; not compressible; obeying the statistics of a fair coin toss.
Rational NumberA number that can be expressed as a fraction.
Real NumberAny number that can be represented with a potentially infinite decimal expansion to the right of the decimal point. Natural, rational, irrational, and incomputable numbers are all real numbers.
Recurrent Neural NetworkA network similar to a feedforward neural network except that there may be connections from an output or hidden layer to the inputs. Recurrent neural networks are capable of universal computation.
RecursiveStrictly speaking, a set or function is recursive if it is computable; however, in the usual sense of the word, a function is said to be recursive if its definition make reference to itself. For example, factorial can be defined as x! = x * (x  1)! with the base case of 1! equal to 1. See also selfreferential.
Recursively Enumerable (RE)A potentially infinite set whose members can be enumerated by a universal computer; however, a universal computer may not be able to determine that something is not a member of a recursively enumerable set. The halting set is recursively enumerable but not recursive.
ReductionismThe idea that nature can be understood by dissection. In other words, knowing the lowestlevel details of how things work (at, say, the level of subatomic physics) reveals how higherlevel phenomena come about. This is a bottomup way of looking at the universe, and is the exact opposite of holism.
Regular ExpressionA definition for a class of strings that can be recognized by a finitestate automaton. An example of a class of strings that is regular would be legal mathematical expressions using only ``+'' and digits. An example that is not regular is the same legal mathematical expressions as before, but with properly nested parentheses.
SaddleA type of surface that is neither a peak nor a valley but still has a 0 gradient. Saddle points are situated such that moving in one direction takes one uphill, while moving in another direction would be downhill. Hence, saddles look like the things that cowboys ride on.
ScalarA single number, as opposed to a multidimensional vector or matrix.
SchemaSchemataA similarity template used to analyze genetic algorithms. By using wildcard characters, a schema defines an entire class of strings that may be found in a population.
Search SpaceA characterization of every possible solution to a problem instance. For a neural network the search space is defined as all possible assignments to the network weights; for a genetic algorithm, it is every conceivable value assignment to the strings in the population.
SearchSearch MethodA method for finding a region of interest in a search space. Usually, the interesting regions correspond to solutions to a specific problem. Hillclimbing, steepest descent (ascent), simulated annealing, and genetic algorithms are all search methods.
SelectionSee natural selection.
SelfOrganizationA spontaneously formed higherlevel pattern of structure or function that is emergent through the interactions of lowerlevel objects.
SelfOrganized Criticality (SOC)A mathematical theory that describes how systems composed of many interacting parts can tune themselves toward dynamical behavior that is critical in the sense that it is neither stable nor unstable but at a region near a phase transition. SOC systems display events in a powerlaw distribution and are never quite at equilibrium. See also edge of chaos and selforganization.
SelfReferentialReferring directly back to oneself through information flow, influence, or cause and effect. See SelfReferential.
SelfSimilarAn object that is structurally recursive in that a part will look like the whole. See also fractal.
SensitivityThe tendency of a system (sometimes chaotic) to change dramatically with only small perturbations.
SetA collection of things, usually numbers. Sets may be infinite in size.
Shadowing LemmaImplies that a numerical simulation of chaos may ``shadow'' a real trajectory of a real chaotic system.
SigmoidalAn ``S'' shaped function that is often used as an activation function in a neural network.
SimulateSimulationExperimentation in the space of theories, or a combination of experimentation and theorization. Some numerical simulations are programs that represent a model for how nature works. Usually, the outcome of a simulation is as much a surprise as the outcome of a natural event, due to the richness and uncertainty of computation.
Simulated AnnealingA partially random method of search and optimization usually used for combinatorial optimization problems. The technique is modeled on how the molecular structure of metals is disordered at high temperatures but very ordered and crystalline at low temperatures. In simulated annealing, a problem instance is reformulated so that it loosely resembles disordered material. Gradually, the temperature is lowered such that the ordered states correspond to good solutions to a problem.
Space ComplexityA function that describes the amount of memory required for a program to run on a computer to perform a particular task. The function is parameterized by the length of the program's input. See also time complexity.
SpaceFillingRefers to a curve that manages to twist and turn in such a way that it actually fills a space or volume. All spacefilling curves are fractal.
Special FunctionIn Lisp or Stutter, a builtin function that may or may not fully evaluate its arguments, such as the if primitive.
StableHaving a basin of attraction that is nonzero in size; an attractor that can withstand some form of perturbation.
Standard DeviationA measure of the spread of a set of data. For a Gaussian distribution, the standard deviation hints at the width of the tails of the distribution function.
State SpaceIn this book, another name for the phase space of a dynamical system. Roughly speaking, if the dynamics of a dynamical system can be described by n values, then the state space is the ndimensional volume that the system moves through. Systems that are continuous in time will form a smooth trajectory through this volume, while discrete systems may jump to different locations on subsequent time steps. In either case, if a system ever returns to a previously visited location in the state space, thâ€¦
StatementIn a formal system, a string of characters that is formed according to welldefined rules such that it is legal for the language that is the formal system. For example, in the formal system of arithmetic, the expression ``5 + 3 * (2  4)'' is a valid and wellformed statement, but ``5 + )3 * * (2 ( 4)'' is not.
Steepest Descent (Ascent)A search method that uses the gradient information of a search space and moves in the opposite direction from the gradient until no further downhill (or uphill) progress can be made. See also hillclimbing.
StochasticSomething that is random.
Strange AttractorAn attractor of a dynamical system that is usually fractal in dimension and is indicative of chaos.
StrategyIn game theory, a policy for playing a game. A strategy is a complete recipe for how a player should act in a game under all circumstances. Some policies may employ randomness, in which case they are referred to as mixed strategies.
StrengthFor a classifier system, a classifier's relative ability to win a bidding match for the right to post its message on the message list.
StringAny sequence of letters, numbers, digits, bits, or symbols.
StutterA silly programming language used in this book that is based on Lisp and is capable of universal computation.
Symmetric MatrixA matrix with the lowerleft half equal to the mirror image of the upperright half.
SynapseThe junction between two neurons in which neural activity is propagated from one neuron to another. See also excitatory, inhibitory, and weight.
SynchronousActing in a lockstep fashion, with each event occurring in a precise order, or in such a way as to eliminate the notion of order entirely.
SystemSomething that can be studied as a whole. Systems may consist of subsystems that are interesting in their own right. Or they may exist in an environment that consists of other similar systems. Systems are generally understood to have an internal state, inputs from an environment, and methods for manipulating the environment or themselves. Since cause and effect can flow in both directions of a system and environment, interesting systems often posses feedback, which is selfreferential in the strâ€¦
TheoremA statement in a formal system that has proof.
TheorizationA process by which scientists attempt to understand nature; it is the complement to experimentation. Theorization is the process of building mathematical models for how things work. Scientists always desire theories that are simpler than the data they explain. See also Occam's Razor and simulation.
Three Body ProblemThe problem of determining the future positions and velocities of three gravitational bodies. The problem was proved unsolvable in the general case by Henri Poincaré, which forshadowed the importance of chaos. Although no analytical solutions are possible in the worst case, a numerical solution is sometimes sufficient for many tasks.
ThresholdA quantity added to (or subtracted from) the weighted sum of inputs into a neuron, which forms the neuron's net input. Intuitively, the net input (or bias) is proportional to the amount that the incoming neural activations must exceed in order for a neuron to fire.
Time ComplexityA function that describes the amount of time required for a program to run on a computer to perform a particular task. The function is parameterized by the length of the program's input. See also space complexity.
Time SeriesA sequence of values generated from a dynamical system over time. Chaotic systems can be analyzed by examining the time series generated by a single portion of the system. See also embedding.
TimeReversibleA property of dynamical systems that can be run unambiguously both forward and backward in time. The Hénon map, Lorenz system, and vant cellular automata are all timereversible, while the logistic map, the MackeyGlass system, and most other cellular automata are not. Timereversible systems are described by functions that are invertible.
TitforTatAn effective strategy for playing the Iterated Prisoner's Dilemma. TitforTat starts by cooperating, and then does whatever its opponent did in the previous round of play.
TopDownA method of examining things that first looks at higherlevel phenomena and then tries to explain lowerlevel patterns in terms of the higherlevel observations. This is the exact opposite of bottomup. See also holism and reductionism.
TransposeAn operation that flips a matrix about the main diagonal.
Turing MachineA model of computation that uses an underlying finitestate automaton but also has an infinite tape to use as memory. Turing machines are capable of universal computation.
Turtle GraphicsA simple language for drawing graphics in which a ``turtle'' is used to make strokes on a plotting device. Typical commands include ``move forward,'' ``draw forward,'' and ``turn left.''
Uncountable InfinityAn order of infinity that is larger than the number of natural numbers. The number of real numbers is uncountably infinite.
Universal ApproximationHaving the ability to approximate any function to an arbitrary degree of accuracy. Neural networks are universal approximators.
Universal ComputationCapable of computing anything that can in principle be computed; being equivalent in computing power to a Turing machine, the lambda calculus, or a Post production system.
Universal ComputerA computer that is capable of universal computation, which means that given a description of any other computer or program and some data, it can perfectly emulate this second computer or program. Strictly speaking, home PCs are not universal computers because they have only a finite amount of memory. However, in practice, this is usually ignored.
UnstableHaving a basin of attraction that is 0 in size; being such that the slightest perturbation will forever change the state of a system. A pencil balanced on its point is unstable.
Value FunctionA builtin function in Lisp or Stutter that evaluates all of its arguments prior to being executed, e.g., car, cdr, and cons.
VantA virtual ant; a type of cellular automaton that vaguely emulates the activity of one or more ants.
VariationGenetic differences among individuals in a population.
VectorA onedimensional array of numbers that can be used to represent a point in a multidimensional space.
WeightIn a neural network, the strength of a synapse (or connection) between two neurons. Weights may be positive (excitatory) or negative (inhibitory). The thresholds of a neuron are also considered weights, since they undergo adaptation by a learning algorithm.
White NoiseNoise that uniformly distributed in the frequency domain; randomness that is uniformly distributed; thus, a white noise process with a range of 0 to 1 would yield a random number in this range with probability equal for all possible values. Brown noise is a result of cumulatively adding white noise.
XORThe exclusiveor function; given two Boolean inputs, the output of XOR is 1 if and only if the two inputs are different; otherwise, the output is 0.
ZeroSum GameIn game theory, a game in which a win for one player results in an equal but opposite loss for the other players.