Laboratory Task 3

Genheylou Felisilda - DS4A

Instruction: Perform a forward and backward propagation in python using the inputs from Laboratory Task 2

import numpy as np

# Inputs and target
x = np.array([1, 0, 1])
y = np.array([1])

# Learning rate
lr = 0.001

# Initialize weights and biases (from Task 2)
W_hidden = np.array([[0.2, 0.4, -0.5],   # weights for hidden neuron 1
                     [-0.3, 0.1, 0.2]])  # weights for hidden neuron 2
b_hidden = np.array([-0.4, 0.2])         # biases for hidden layer

W_output = np.array([-0.3, -0.2])        # weights for output neuron
b_output = 0.1                            # bias for output neuron

# ReLU activation function
def relu(z):
    return np.maximum(0, z)

# Derivative of ReLU
def relu_derivative(z):
    return np.where(z > 0, 1, 0)

Forward Propagation

# Hidden layer
z_hidden = np.dot(W_hidden, x) + b_hidden
a_hidden = relu(z_hidden)

# Output layer
z_output = np.dot(W_output, a_hidden) + b_output
a_output = relu(z_output)

# Compute error
error = 0.5 * (y - a_output)**2

print("Forward Propagation Results:")
print("Hidden layer pre-activation:", z_hidden)
print("Hidden layer activation:", a_hidden)
print("Output pre-activation:", z_output)
print("Predicted output:", a_output)
print("Error:", error)

Forward Propagation Results:
Hidden layer pre-activation: [-0.7  0.1]
Hidden layer activation: [0.  0.1]
Output pre-activation: 0.08
Predicted output: 0.08
Error: [0.4232]

Backward Propagation

# Output layer delta
delta_output = (a_output - y) * relu_derivative(z_output)

# Hidden layer delta
delta_hidden = delta_output * W_output * relu_derivative(z_hidden)

# Update output weights and bias
W_output -= lr * delta_output * a_hidden
b_output -= lr * delta_output

# Update hidden weights and biases
W_hidden -= lr * np.outer(delta_hidden, x)
b_hidden -= lr * delta_hidden

print("\nBackward Propagation Updates:")
print("Updated hidden weights:\n", W_hidden)
print("Updated hidden biases:", b_hidden)
print("Updated output weights:", W_output)
print("Updated output bias:", b_output)

Backward Propagation Updates:
Updated hidden weights:
 [[ 0.2       0.4      -0.5     ]
 [-0.300184  0.1       0.199816]]
Updated hidden biases: [-0.4       0.199816]
Updated output weights: [-0.3      -0.199908]
Updated output bias: [0.10092]

The network adjusts its weights and biases gradually to minimize error, illustrating that learning is iterative. The ReLU activation guides which signals influence updates, emphasizing how activation functions affect learning. This demonstrates that even simple networks rely on repeated forward and backward passes to approximate target outputs effectively.