Skip to content

天楚锐齿

人工智能 云计算 大数据 物联网 IT 通信 嵌入式

天楚锐齿

  • 下载
  • 物联网
  • 云计算
  • 大数据
  • 人工智能
  • Linux&Android
  • 网络
  • 通信
  • 嵌入式
  • 杂七杂八

《神经网络与深度学习》中的Python 3.x 代码network2.py

2019-01-22

《神经网络与深度学习》(Michael Nielsen著)中的代码是基于python2.7的,下面为移植到python3下的代码:

network2.py代码:

点击下载:network2

“””network2.py
~~~~~~~~~~~~~~
An improved version of network.py, implementing the stochastic
gradient descent learning algorithm for a feedforward neural network.
Improvements include the addition of the cross-entropy cost function,
regularization, and better initialization of network weights. Note
that I have focused on making the code simple, easily readable, and
easily modifiable. It is not optimized, and omits many desirable
features.
“””
#### Libraries
# Standard library
import json
import random
import sys
# Third-party libraries
import numpy as np
import time
#### Define the quadratic and cross-entropy cost functions
class QuadraticCost(object):
@staticmethod
deffn(a, y):
“””Return the cost associated with an output “a“ and desired output
“y“.
“””
return0.5*np.linalg.norm(a-y)**2
@staticmethod
defdelta(z, a, y):
“””Return the error delta from the output layer.”””
return (a-y) * sigmoid_prime(z)
class CrossEntropyCost(object):
@staticmethod
deffn(a, y):
“””Return the cost associated with an output “a“ and desired output
“y“. Note that np.nan_to_num is used to ensure numerical
stability. In particular, if both “a“ and “y“ have a 1.0
in the same slot, then the expression (1-y)*np.log(1-a)
returns nan. The np.nan_to_num ensures that that is converted
to the correct value (0.0).
“””
return np.sum(np.nan_to_num(-y*np.log(a)-(1-y)*np.log(1-a)))
@staticmethod
defdelta(z, a, y):
“””Return the error delta from the output layer. Note that the
parameter “z“ is not used by the method. It is included in
the method’s parameters in order to make the interface
consistent with the delta method for other cost classes.
“””
return (a-y)
#### Main Network class
class Network(object):
def__init__(self, sizes, cost=CrossEntropyCost):
“””The list “sizes“ contains the number of neurons in the respective
layers of the network. For example, if the list was [2, 3, 1]
then it would be a three-layer network, with the first layer
containing 2 neurons, the second layer 3 neurons, and the
third layer 1 neuron. The biases and weights for the network
are initialized randomly, using
“self.default_weight_initializer“ (see docstring for that
method).
“””
self.num_layers =len(sizes)
self.sizes = sizes
self.default_weight_initializer()
self.cost=cost
defprint_init_bias_weight(self):
print(“Time: “+str(time.strftime(‘%Y-%m-%d %H:%M:%S’,time.localtime(time.time()))))
print(“输入图片像素点数:”+str(self.sizes[0]) +”, 输入神经网络层数:”+str(self.num_layers))
print(“第 1 层为输入层(共 “+str(self.sizes[0])+” 个神经元),权重和偏置都不需要。”)
for i inrange(1, self.num_layers):
print(“第 “+str(i+1)+” 层共有 “+str(self.sizes[i])+” 个神经元,初始权重和偏置如下。”)
print(” “)
layer = 1
print(“训练前的权重和偏置:”)
for lw, lb inzip(self.weights, self.biases):
layer += 1
num_nw = 0
for w, b inzip(lw, lb):
num_nw += 1
print(“第 “+str(layer)+” 层第 “+str(num_nw)+” 个神经元的权重(共”+str(len(w))+”个输入):”)
print(str(w))
print(“第 “+str(layer)+” 层第 “+str(num_nw)+” 个神经元的偏置(共”+str(len(b))+”个输出):”)
print(str(b))
print(” “)
print(” “)
defdefault_weight_initializer(self):
“””Initialize each weight using a Gaussian distribution with mean 0
and standard deviation 1 over the square root of the number of
weights connecting to the same neuron. Initialize the biases
using a Gaussian distribution with mean 0 and standard
deviation 1.
Note that the first layer is assumed to be an input layer, and
by convention we won’t set any biases for those neurons, since
biases are only ever used in computing the outputs from later
layers.
“””
self.biases = [np.random.randn(y, 1) for y inself.sizes[1:]]
self.weights = [np.random.randn(y, x)/np.sqrt(x)
for x, y inzip(self.sizes[:-1], self.sizes[1:])]
deflarge_weight_initializer(self):
“””Initialize the weights using a Gaussian distribution with mean 0
and standard deviation 1. Initialize the biases using a
Gaussian distribution with mean 0 and standard deviation 1.
Note that the first layer is assumed to be an input layer, and
by convention we won’t set any biases for those neurons, since
biases are only ever used in computing the outputs from later
layers.
This weight and bias initializer uses the same approach as in
Chapter 1, and is included for purposes of comparison. It
will usually be better to use the default weight initializer
instead.
“””
self.biases = [np.random.randn(y, 1) for y inself.sizes[1:]]
self.weights = [np.random.randn(y, x)
for x, y inzip(self.sizes[:-1], self.sizes[1:])]
deffeedforward(self, a):
“””Return the output of the network if “a“ is input.”””
for b, w inzip(self.biases, self.weights):
a = sigmoid(np.dot(w, a)+b)
return a
defSGD(self, training_data, epochs, mini_batch_size, eta,
lmbda=0.0,
evaluation_data=None,
monitor_evaluation_cost=False,
monitor_evaluation_accuracy=False,
monitor_training_cost=False,
monitor_training_accuracy=False):
“””Train the neural network using mini-batch stochastic gradient
descent. The “training_data“ is a list of tuples “(x, y)“
representing the training inputs and the desired outputs. The
other non-optional parameters are self-explanatory, as is the
regularization parameter “lmbda“. The method also accepts
“evaluation_data“, usually either the validation or test
data. We can monitor the cost and accuracy on either the
evaluation data or the training data, by setting the
appropriate flags. The method returns a tuple containing four
lists: the (per-epoch) costs on the evaluation data, the
accuracies on the evaluation data, the costs on the training
data, and the accuracies on the training data. All values are
evaluated at the end of each training epoch. So, for example,
if we train for 30 epochs, then the first element of the tuple
will be a 30-element list containing the cost on the
evaluation data at the end of each epoch. Note that the lists
are empty if the corresponding flag is not set.
“””
training_data_list = list(training_data)
evaluation_data_list = list(evaluation_data)
if evaluation_data:
n_data = len(evaluation_data_list)
n = len(training_data_list)
evaluation_cost, evaluation_accuracy = [], []
training_cost, training_accuracy = [], []
print(“=================================================================”)
print(“=================================================================”)
loop = 0
for j inrange(epochs):
loop += 1
print(“Time: “+str(time.strftime(‘%Y-%m-%d %H:%M:%S’,time.localtime(time.time()))))
random.shuffle(training_data_list)
training_data = training_data_list
evaluation_data = evaluation_data_list
mini_batches = [
training_data[k:k+mini_batch_size]
for k inrange(0, n, mini_batch_size)]
for mini_batch in mini_batches:
self.update_mini_batch(
mini_batch, eta, lmbda, len(training_data))
print(“Epoch %s training complete”% j)
if monitor_training_cost:
cost = self.total_cost(training_data, lmbda)
training_cost.append(cost)
print(“Cost on training data: {}”.format(cost))
if monitor_training_accuracy:
accuracy = self.accuracy(training_data, convert=True)
training_accuracy.append(accuracy)
print(“Accuracy on training data: {} / {}”.format(
accuracy, n))
if monitor_evaluation_cost:
cost = self.total_cost(evaluation_data, lmbda, convert=True)
evaluation_cost.append(cost)
print(“Cost on evaluation data: {}”.format(cost))
if monitor_evaluation_accuracy:
accuracy = self.accuracy(evaluation_data)
evaluation_accuracy.append(accuracy)
print(“Accuracy on evaluation data: {} / {}”.format(
self.accuracy(evaluation_data), n_data))
print(“\n”)
print(“=================================================================”)
print(“=================================================================”)
print(” “)
layer = 1
print(“训练后的权重和偏置:”)
for lw, lb inzip(self.weights, self.biases):
layer += 1
num_nw = 0
for w, b inzip(lw, lb):
num_nw += 1
print(“第 “+str(layer)+” 层第 “+str(num_nw)+” 个神经元的权重(共”+str(len(w))+”个输入):”)
print(str(w))
print(“第 “+str(layer)+” 层第 “+str(num_nw)+” 个神经元的偏置(共”+str(len(b))+”个输出):”)
print(str(b))
print(” “)
print(” “)
print(“=================================================================”)
print(“=================================================================”)
print(“每层测试结果的计算公式: (输入1*权重1 + 输入2*权重2 + 输入3*权重3 + …) + 偏置 “)
print(“a1*w11—|”)
print(“a2*w12—|+b1—|”)
print(“a3*w13—| |”)
print(“……—| |-output”)
print(“a1*w21—| |”)
print(“a2*w22—|+b2—|”)
print(“a3*w23—|”)
print(“……—|”)
print(“Time: “+str(time.strftime(‘%Y-%m-%d %H:%M:%S’,time.localtime(time.time()))))
return evaluation_cost, evaluation_accuracy, \
training_cost, training_accuracy
defupdate_mini_batch(self, mini_batch, eta, lmbda, n):
“””Update the network’s weights and biases by applying gradient
descent using backpropagation to a single mini batch. The
“mini_batch“ is a list of tuples “(x, y)“, “eta“ is the
learning rate, “lmbda“ is the regularization parameter, and
“n“ is the total size of the training data set.
“””
nabla_b = [np.zeros(b.shape) for b in self.biases]
nabla_w = [np.zeros(w.shape) for w in self.weights]
for x, y in mini_batch:
delta_nabla_b, delta_nabla_w = self.backprop(x, y)
nabla_b = [nb+dnb for nb, dnb in zip(nabla_b, delta_nabla_b)]
nabla_w = [nw+dnw for nw, dnw in zip(nabla_w, delta_nabla_w)]
self.weights = [(1-eta*(lmbda/n))*w-(eta/len(mini_batch))*nw
for w, nw inzip(self.weights, nabla_w)]
self.biases = [b-(eta/len(mini_batch))*nb
for b, nb inzip(self.biases, nabla_b)]
defbackprop(self, x, y):
“””Return a tuple “(nabla_b, nabla_w)“ representing the
gradient for the cost function C_x. “nabla_b“ and
“nabla_w“ are layer-by-layer lists of numpy arrays, similar
to “self.biases“ and “self.weights“.”””
nabla_b = [np.zeros(b.shape) for b in self.biases]
nabla_w = [np.zeros(w.shape) for w in self.weights]
# feedforward
activation = x
activations = [x] # list to store all the activations, layer by layer
zs = [] # list to store all the z vectors, layer by layer
for b, w inzip(self.biases, self.weights):
z = np.dot(w, activation)+b
zs.append(z)
activation = sigmoid(z)
activations.append(activation)
# backward pass
delta = (self.cost).delta(zs[-1], activations[-1], y)
nabla_b[-1] = delta
nabla_w[-1] = np.dot(delta, activations[-2].transpose())
# Note that the variable l in the loop below is used a little
# differently to the notation in Chapter 2 of the book. Here,
# l = 1 means the last layer of neurons, l = 2 is the
# second-last layer, and so on. It’s a renumbering of the
# scheme in the book, used here to take advantage of the fact
# that Python can use negative indices in lists.
for l inrange(2, self.num_layers):
z = zs[-l]
sp = sigmoid_prime(z)
delta = np.dot(self.weights[-l+1].transpose(), delta) * sp
nabla_b[-l] = delta
nabla_w[-l] = np.dot(delta, activations[-l-1].transpose())
return (nabla_b, nabla_w)
defaccuracy(self, data, convert=False):
“””Return the number of inputs in “data“ for which the neural
network outputs the correct result. The neural network’s
output is assumed to be the index of whichever neuron in the
final layer has the highest activation.
The flag “convert“ should be set to False if the data set is
validation or test data (the usual case), and to True if the
data set is the training data. The need for this flag arises
due to differences in the way the results “y“ are
represented in the different data sets. In particular, it
flags whether we need to convert between the different
representations. It may seem strange to use different
representations for the different data sets. Why not use the
same representation for all three data sets? It’s done for
efficiency reasons — the program usually evaluates the cost
on the training data and the accuracy on other data sets.
These are different types of computations, and using different
representations speeds things up. More details on the
representations can be found in
mnist_loader.load_data_wrapper.
“””
if convert:
results = [(np.argmax(self.feedforward(x)), np.argmax(y))
for (x, y) in data]
else:
results = [(np.argmax(self.feedforward(x)), y)
for (x, y) in data]
returnsum(int(x == y) for (x, y) in results)
deftotal_cost(self, data, lmbda, convert=False):
“””Return the total cost for the data set “data“. The flag
“convert“ should be set to False if the data set is the
training data (the usual case), and to True if the data set is
the validation or test data. See comments on the similar (but
reversed) convention for the “accuracy“ method, above.
“””
data_list = list(data)
cost = 0.0
for x, y in data:
a = self.feedforward(x)
if convert: y = vectorized_result(y)
cost += self.cost.fn(a, y)/len(data_list)
cost += 0.5*(lmbda/len(data_list))*sum(
np.linalg.norm(w)**2 for w in self.weights)
return cost
defsave(self, filename):
“””Save the neural network to the file “filename“.”””
data = {“sizes”: self.sizes,
“weights”: [w.tolist() for w inself.weights],
“biases”: [b.tolist() for b inself.biases],
“cost”: str(self.cost.__name__)}
f = open(filename, “w”)
json.dump(data, f)
f.close()
#### Loading a Network
def load(filename):
“””Load a neural network from the file “filename“. Returns an
instance of Network.
“””
f = open(filename, “r”)
data = json.load(f)
f.close()
cost = getattr(sys.modules[__name__], data[“cost”])
net = Network(data[“sizes”], cost=cost)
net.weights = [np.array(w) for w in data[“weights”]]
net.biases = [np.array(b) for b in data[“biases”]]
return net
#### Miscellaneous functions
def vectorized_result(j):
“””Return a 10-dimensional unit vector with a 1.0 in the j’th position
and zeroes elsewhere. This is used to convert a digit (0…9)
into a corresponding desired output from the neural network.
“””
e = np.zeros((10, 1))
e[j] = 1.0
return e
def sigmoid(z):
“””The sigmoid function.”””
return1.0/(1.0+np.exp(-z))
def sigmoid_prime(z):
“””Derivative of the sigmoid function.”””
return sigmoid(z)*(1-sigmoid(z))
def test():
import mnist_loader
training_data, validation_data, test_data = mnist_loader.load_data_wrapper()
# net = Network([784, 30, 10], cost=QuadraticCost)
net = Network([784, 30, 10], cost=CrossEntropyCost)
# net.large_weight_initializer()
net.default_weight_initializer()
net.print_init_bias_weight()
# net.SGD(training_data, 30, 10, 0.5, evaluation_data=test_data,
net.SGD(training_data, 5, 10, 0.5, evaluation_data=test_data,
monitor_evaluation_cost=True, monitor_evaluation_accuracy=True, monitor_training_cost=True, monitor_training_accuracy=True)
net.save(“network2_save.dat”)
# 如果停止了,要接着再训练
print(“######################################################################################################”)
print(“########################################## 新的一轮测试:##############################################”)
time.sleep(3)
training_data, validation_data, test_data = mnist_loader.load_data_wrapper()
net = load(“network2_save.dat”)
# net.default_weight_initializer() #不要再初始化了
net.print_init_bias_weight()
net.SGD(training_data, 5, 10, 0.5, evaluation_data=test_data,
monitor_evaluation_cost=True, monitor_evaluation_accuracy=True, monitor_training_cost=True, monitor_training_accuracy=True)
net.save(“network2_save.dat”)
if __name__ == ‘__main__’:
test()

 

 

1,176次阅读

Post navigation

前一篇:

《神经网络与深度学习》中的Python 3.x 代码network1.py

后一篇:

《神经网络与深度学习》中的Python 3.x 代码network3.py

发表评论 取消回复

邮箱地址不会被公开。 必填项已用*标注

个人介绍

需要么,有事情这里找联系方式:关于天楚锐齿

=== 美女同欣赏,好酒共品尝 ===

微信扫描二维码赞赏该文章:

扫描二维码分享该文章:

分类目录

  • Linux&Android (79)
  • Uncategorized (1)
  • 下载 (28)
  • 云计算 (37)
  • 人工智能 (8)
  • 大数据 (24)
  • 嵌入式 (34)
  • 杂七杂八 (34)
  • 物联网 (59)
  • 网络 (23)
  • 通信 (21)

文章归档

近期文章

  • 使用Python渲染OpenGL的.obj和.mtl文件
  • 用LVGL图形库绘制二维码
  • Android使用Messenger和SharedMemory实现跨app的海量数据传输
  • CAN信号的c语言解析代码
  • QT qml下DBus的使用例子

近期评论

  • 硕发表在《使用Android的HIDL+AIDL方式编写从HAL层到APP层的程序》
  • maxshu发表在《使用Android的HIDL+AIDL方式编写从HAL层到APP层的程序》
  • Ambition发表在《使用Android的HIDL+AIDL方式编写从HAL层到APP层的程序》
  • Ambition发表在《使用Android的HIDL+AIDL方式编写从HAL层到APP层的程序》
  • maxshu发表在《Android9下用ethernet 的Tether模式来做路由器功能》

阅读量

  • 使用Android的HIDL+AIDL方式编写从HAL层到APP层的程序 - 16,803次阅读
  • 卸载深信服Ingress、SecurityDesktop客户端 - 12,077次阅读
  • 车机技术之Android Automotive - 6,661次阅读
  • 车机技术之车规级Linux-Automotive Grade Linux(AGL) - 5,857次阅读
  • Linux策略路由及iptables mangle、ip rule、ip route关系及一种Network is unreachable错误 - 5,709次阅读
  • 在Android9下用ndk编译vSomeIP和CommonAPI以及使用例子 - 5,657次阅读
  • linux下的unbound DNS服务器设置详解 - 5,600次阅读
  • linux的tee命令导致ssh客户端下的shell卡住不动 - 4,998次阅读
  • 车机技术之360°全景影像(环视)系统 - 4,896次阅读
  • libwebp(处理webp图像)的安装和使用 - 4,749次阅读

功能

  • 文章RSS
  • 评论RSS

联系方式

地址
深圳市科技园

时间
周一至周五:  9:00~12:00,14:00~18:00
周六和周日:10:00~12:00

标签

android AT命令 centos Hadoop hdfs ip ipv6 kickstart linux mapreduce mini6410 modem OAuth openstack os python socket ssh uboot 内核 协议 安装 嵌入式 性能 报表 授权 操作系统 数据 数据库 月报 模型 汽车 测试 深信服 深度学习 源代码 神经网络 统计 编译 网络 脚本 虚拟机 调制解调器 车机 金融
© 2023 天楚锐齿