Machine Learning

Front 1

Hypothesis space for a decision tree

Back 1

All possible decision trees

Front 2

Hypothesis space for a learner

Back 2

All possible outcomes for the specific learner

Front 3

How do you construct a decision tree?

Back 3

Ask questions about status/target/attributes sequentially

Front 4

What is the definition of entropy?

Back 4

Sum[i](

-p_i*log2(p_i)

)

Wher p_i is the probability of event i

Front 5

What is overfitting?

Back 5

Overfitting is when the learned models are overly specialized for the training samples. This leads to poor generalization

Front 6

Name two reasons for overfitting

Back 6

- Non-representative sample

- Noisy examples

- Too complex model

Front 7

What is Occam's razor?

Back 7

"The simplest solution is often the correct one"

Front 8

How do you prevent overfitting?

Back 8

- Seperate available data into two sets; one for training and one for validation.

Front 9

What is "bagging"?

Back 9

Bootstrap aggregration for decision trees.

Front 10

Which error is produced by bias?

Back 10

The difference between the average (expected) prediction of our model and the correct value.

Front 11

Which error is produced by variance?

Back 11

The variability of a model prediction for a given data point between different realizations of the model.

Front 12

What is linear regression?

Back 12

Linear regression tries to estimate a function f which predict the output of the model

Front 13

What is RANSAC an abbreviation for?

Back 13

RANdom SAmpling Consensus

Front 14

Describe the RANSAC algorithm

Back 14

Using a randomly selected set S, determine which points in S are within a given distance to the model. If the number of point in S which satisfy the distance criteria are greater than some threshold, re-estimate the model using the points in S.

Repeat the above N times and select the largest set S, consensus set, and re-estimate using this S.

Front 15

What is the difference between Ridge Regression and Least Squares?

Back 15

Ridge Regression uses a shrinkage penalty factor.

Front 16

Which features does Ridge Regression include?

Back 16

All features are included when using Ridge Regression

Front 17

What does the acronym 'The Lasso' stand for?

Back 17

Least Absolute Shrinkage and Selection Operator

Front 18

What is a mathematical benefit of using Lasso over Ridge?

Back 18

Some of lasso's coefficients will be exaclty zero

Front 19

What is a discrete value?

Back 19

A discrete value is a value from a predefined set

Front 20

How can we tell that two events are independent of each other?

Back 20

P(A|B) = P(A)

Front 21

Which type of value requires classification, discrete or continous?

Back 21

Discrete-value problems use classification.

Front 22

Which type of value requires regression, discrete or continous?

Back 22

Continous-value problems use regression.

Front 23

On which assumption is the Naive Bayes Classifier based?

Back 23

That all events are independent

Front 24

What is the basic premise of an artificial neuron?

Back 24

Using several inputs, construct a value representing all the inputs, compare against a threshold and return a +/- answer (usually).

Front 25

When does Perceptron Learning converge?

Back 25

Always, if the problem is solvable.

Front 26

Using Perceptron Learning, when does weight change?

Back 26

When the output is wrong.

Front 27

What is another name for Delta Rule?

Back 27

LMS-rule

Front 28

Using the Delta Rule, when do weights change?

Back 28

Always, the separating plane is always nudged a little.

Front 29

When does LMS-rule converge?

Back 29

Only in the mean

Front 30

When does the Delta Rule converge?

Back 30

Only in the mean

Front 31

What is an advantage of using LMS over Perceptron?

Back 31

LMS will find an optimal solution even if the problem can't be fully solved.

Front 32

When using hyperplanes, one faces certain structural risks. What is one counter-measure?

Back 32

The use of margins, which allows for some buffer surrounding the hyperplane.

Front 33

Which problems might occur when scattering low-dimension data into higher dimensions?

Back 33

1. Many free parameters -> bad generalization
2. Extensive computation

Front 34

What is the main purpose of the kernel function when working with SVM's?

Back 34

To transform low-dimensional data into high-dimensional data, but only by using scalar products of the low-dimensional values.

Front 35

Name two common types of kernel functions

Back 35

Polynomial kernels

Radial base kernels

Front 36

What is the point of having multiple layers in Artificial Neurons?

Back 36

A layered network can create arbitrary decision surfaces, i.e. non-linear.

Front 37

Are multi-layered ANN's continuous or discrete in values?

Back 37

The threshold functions output is continuous and the input signal may be of varying character.

Front 38

What is the difference between the output of single layered and multi layer Artificial Neurons?

Back 38

The first has discrete value output, and the second continuous.

Front 39

What is the difference between Decision Forest and Bagging?

Back 39

Decision Forest, also known as Random Forest, is a combination of Bagging and a random feature selection.

Front 40

What are ensemble methods?

Back 40

Ensemble methods combine weak learners to harness their combined strengths

Front 41

What is the general idea behind the ensemble method Boosting?

Back 41

Combine multiple hypotheses

Front 42

When using BackProp in an ANN, what is a hypotheses?

Back 42

A set of weights for all the connections