Neural Network

Front 1

Bayesian decision theory

Back 1

Imagine the classic signal detection problem: two overlapping distributions (noise and signal), and we must discern noise from signal, and the various decisions we must make (hit,miss,flase alarm, correct rejection). However, sometimes making the right decision has to be considered in light of the costs of different types of errors. Ex: high cost of false alarms affects our criterion (we can't afford to make incorrect predictions). Natural loss functions may be 'hard-wired' into the brain. And Bayesian Decision Theory provides tools to model performance as a function of utility.

Front 2

Bayesian optimal cue integration

Back 2

Integrating different perceptual cues. To enable interactions with the environment sensory info has to converge into a coherent and unambiguous multimodal percept of the body and the world. What is the statistically optimal strategy for combining redundant sensory info? We find the "Maximum-Likelihood-Estimation" (MLE). This will be the weighted average of the estimations of the different sensory stimuli with weights w(i).

Front 3

belief propogation

Back 3

From google: involves passing 'mesages' (probabilities) between the nodes of a graphical model that captures the causal structure of the environment. Involves two types of computation: marginalization (summation over local joint distributions) and multiplication of local marginal probabilities. Because computations are local, the algorithm is well suited to neural applications. For our class, we want belief propogation to do MAP estimation.

Front 4

bidirectional processing

Back 4

The visual system shows both feedforward and feedback pathways, making it a system with bideirectional processing. One simplified example of this is top-down processing vs. bottom up processing. Bottom-up processing will tell you how the lower visual areas will respond to a certain stimuli. However, with cognitive information, a percept can change, and this change in perception can be seen even in lower level visual areas. This is because of the feedback pathways from the higher visual areas affecting the lower visual areas.

Front 5

coarse coding

Back 5

A neuron can be 'tuned' to various features or dimensions of an input pattern or stimulus (simple cells: position, orientation, spatial frequency, spatial phase, motion direction and speed, and occularity ). Features can be coarsely sampled (few detectors to span the range_ and the receptive fields can be broad. Broadly tuned cells mean that similar inputs to the cell's preffered input also fire the cell. Coarse coding with broad tuning functions result in overlap of the tuning. Fine coding means that neurons finely or 'densely' sample the feature space, typically with correspondingly narrower tuning functions and receptive fields that are more closely packed. Neurons here are more closerly tuned to the exact feature, and show little or no response to similar features.

Front 6

contingent adaptation

Back 6

from wiki:? aka contingent aftereffect. Illusory percepts that are apparent on a test stimulus after exposure to an induction stimulus for an extended period. For example, black and white gratings appear colored contingent on the orientation of the gratings. An aftereffect requiring a period of induction to produce. Look at a red horizontal grating and a green vertical grating for minutes. A black horizontal grating will look green, and a black vertical grating will look red. Known for lasting over an hour. Occurs via a neural network which uses an anti-hebbian rule for lateral interaction. These neurons serve as decorrelators: these units detect different coincidences.

Front 7

cortical maps

Back 7

There are more than 30 distinct visual areas in the macaque cortex. Earlier areas typically show a spatial topographic representation of visual space (nearby regions of visual space map to nearby regions of cortex). Retinotopic map of V1 is the clearest example. The widespread use of spatial organization in cortex suggests the possibility of a general constraint underlying the layout of features on cortex and the development of neural receptive field organization. V1 doesn't simply represent nearby retinal points at nearby cortical locations. Physiological research has shown that V1 brings together info from the two eyes, along similar orientations as well as location. Neurons with similar orientation preferences and various degrees of relative input from the two eyes are organized into 'hypercolumns' (note: not all species have occular dominance columns and the function of such columns is not understood)

Front 8

directed vs. undirected graph

Back 8

arrows vs no arrows. Arrows represent directionality in a graph, either causal in a bayesian net or information flow in a neural network. No arrows (undirected) means that there's some connection between the two but it's bidirectional (there is not a particular flow of cause or information).

Front 9

EM

Back 9

Expectation Maximization. This is a type of unsupervised learning.

Front 10

encoder network

Back 10

From google:??? one approach to automatically learn features from unlabeled data. An unsupervised learning algorithm that applies backpropagation, setting the targets to be equal to the inputs(?)

Front 11

explaining away

Back 11

One kind of graphical model showing how scene properties influence image measurement. When auxiliary evidence is taken into account, the missing information is explained. The idea here is that one can have a probabilistic structure that gives rise to "competing explanations" for some image data. In general, “explaining away” is a phenomenon that occurs in probabilistic belief networks in which two (or more) variables influence a third variable whose value can be measured (Pearl, 1988). Once measured, it provides evidence to infer the values of the influencing variables. “Explaining away” is also a characteristic of perceptual inferences, for example when there are alternative perceptual groupings

Front 12

features of intermediate complexity

Back 12

Ullman, S., Vidal-Naquet, M., & Sali, E. (2002)..This is discussed in the Ullman et al. (2004) paper. Instead of only representing simple features (wavelets, Gabors, etc.) the visual system might encode features of intermediate complexity. These features would be shared by members of a class, and would be useful for classification tasks

Front 13

Gibbs Sampling

Back 13

Gibbs sampling is a method of reconstructing a distribution when it would be difficult to compute analytically (for example, in high dimensional space).

Front 14

grandmother cell

Back 14

The simple way of understandign grndmother cells is to think of a Cartesian set of orthogonal vectors. Each value within the vector represents one neuron. With a complete set of vectors, you can represent the learned connections/weights. These weights tell you which item the system preditcs. If each of these combinations represents a different item, then you only need to look at a single neuron to see which item the neurons predict. This idea that one cell represents one term or idea is the idea of grandmother cells. This was the idea in the paper that found the 'Jennifer Aniston cell' that only responded to Jennifer Aniston's face. However, people generally don't think that neurons work that way -- instead they think weights are more like the Wilke set of orthogonal vectors where you can't look at a single neuron to see the prediction -- instea you have to look at the pattern of activity across all neurons.

Front 15

K-means

Back 15

This is a way of sorting things deterministically (not probabilistically) and it results in hard assignment (either one group or the other). Doing 2 things: associating each point to a mean, and calculating the means. Make the sum of squared differences as small as possible.You have to know in advance how many cluters there are. Four general steps: 1) randomly choose data points for the sets, 2) calculate the mean of each cluster, 3) calculate how far the points are from the means and 4) repeat 2 & 3 until it converges.

Front 16

K-nearest neighbor classifier

Back 16

A method of classifying, where you assign a new data point to a class based on the nearest already assigned data points. The "k" refers to the number of nearby "neighbor" points that you compare with. This makes it a nonparametric (you don't assume a probability model) but supervised (have labeled examples) type of classification.

Front 17

logistic regression

Back 17

Using a logit function combined with linear regression to predict categorical (in particular 2 categories) of a variable. A simple perceptron is equivalent to logistic regression, in that you want to predict 0 or 1 (no fire or fire) based on continuous input.