2023 k Back Propagation Neural Network For the multi layer neural network that you will be implementing in the following | Assignment Collections

k

Back Propagation

Neural Network

For the multi

–

layer neural network that you will be implementing in the following problems, you may use 

either the hyperbolic tangent or the sigmoid for the activation function.  Obviously you will be implementing 

the back propagation method to train the n

etwork.  Your code should include an arbitrary method that allows 

you to code it for any number of input dimensions, hidden layers, neurons and output neurons, (i.e. you need 

it to be able to allow you to change a variable and it changes the number of laye

rs, or the number of neurons 

at a given layer, or the number of dimensions, or the number of output layers…in other words this cannot be 

hard coded

for a specific data set

).   

1.

(25pts) 

Develop a multi

–

layer neural network to solve the 

regression problem: f

(x) = 1/x

, (for only the 

positive values of x)

.  Be sure to create a testing and training set.  

I am 

NOT

providing you with data. 

The 

number of hidden layers and neurons 

is your discretion, however, you must provide an explanation as to why 

the number of l

ayers and nodes you used was a good choice.  You will need to create multiple different 

structures that you train and p

lot this error for the different models to show why you chose the final model 

that you did for this problem.

Report your network configu

ration, and comment on your observations 

regarding the performance of your network as you try to determine the number of hidden layers and hidden 

nodes of your final network, once you determine the correct model

:

a.

(10pts) 

T

rack 

your training and testing history.  

This means that you 

check your training performance 

and your testing performance at multiples of some fixed number of 

iterations

(and over many Epochs 

also)

[thus you will provide a graph of this information]

, implement 

the online learning method.

Be 

sure to label plots appropriately.

(Remember to scale the data in the range that provides the best 

results for your activation function

… normalization is a must)

b.

(5pts) 

What did you observe regarding the value of the learni

ng parameter and how the network 

performed given

: 

(do one of each)  

i.

(2.5pts)

a fixed value, 

ii.

(2.5pts) 

and a time decreasing value 

c.

(2.5pts) 

Choose a couple of points beyond your training set (i.e., if your max training input 

is 

x=10, 

try testing your netw

ork with, say, x=10.5, x=

12

, and x=

30

).  What do you observe regarding the 

networks ability to generalize for data that is beyond its training set (note, you may have to increase 

the v

alue of x 

to a large number 

to get a good idea

to answer this

)

.

d.

(2.5pts)

Briefly comment on 

the extrapolation capability 

compared 

to the interpolation capability

of 

the network.

e.

(5pts) 

Plot the final results

showing the capability of your network to determine the function f(x) = 

1/x versus the function f(x) = 1/x.

2.

(25pts)

Develop a multi

–

layer neural network to

classif

y

the 

IRIS 

data set.  

Use K

–

Fold Cross Validation for 

this problem.

The code you use for thi

s problem will be an extension of the code you created for problem 1.

a.

(12pts) Implement the NN and r

eport your network configuration: number of hidden layers, number 

of nodes per hidden layer, learning rate/learning schedule, encoding of the output

, etc.

b.

(

5pts) 

Plot 

the 

error metric versus the number of training steps (similar to problem 

1

).

c.

(8pts) 

Comment on how well your network learned the data.  Things to think about: did the network 

classify the data well (or not), and why (or why not); how well did it

classify each class independently 

(you might consider contingency

table

s for this

); and what observation do you have regarding number 

of training samples

: was the normalization technique I used adequat