{"id":981,"date":"2019-01-22T10:46:12","date_gmt":"2019-01-22T02:46:12","guid":{"rendered":"http:\/\/www.max-shu.com\/blog\/?p=981"},"modified":"2019-01-22T10:46:12","modified_gmt":"2019-01-22T02:46:12","slug":"%e3%80%8a%e7%a5%9e%e7%bb%8f%e7%bd%91%e7%bb%9c%e4%b8%8e%e6%b7%b1%e5%ba%a6%e5%ad%a6%e4%b9%a0%e3%80%8b%e4%b8%ad%e7%9a%84python-3-x-%e4%bb%a3%e7%a0%81network1-py","status":"publish","type":"post","link":"http:\/\/www.max-shu.com\/blog\/?p=981","title":{"rendered":"\u300a\u795e\u7ecf\u7f51\u7edc\u4e0e\u6df1\u5ea6\u5b66\u4e60\u300b\u4e2d\u7684Python 3.x \u4ee3\u7801network1.py"},"content":{"rendered":"

\u300a\u795e\u7ecf\u7f51\u7edc\u4e0e\u6df1\u5ea6\u5b66\u4e60\u300b\uff08Michael Nielsen<\/span>\u8457<\/span>\uff09\u4e2d\u7684\u4ee3\u7801\u662f\u57fa\u4e8epython2.7\u7684\uff0c\u4e0b\u9762\u4e3a\u79fb\u690d\u5230python3\u4e0b\u7684\u4ee3\u7801\uff1a<\/p>\n

mnist_loader.py\u4ee3\u7801\uff1a<\/strong><\/span><\/p>\n

\u70b9\u51fb\u4e0b\u8f7d\uff1a<\/strong><\/span>mnist_loader<\/a><\/p>\n

 <\/p>\n

“””
\nmnist_loader
\n~~~~~~~~~~~~<\/p>\n

A library to load the MNIST image data. For details of the data
\nstructures that are returned, see the doc strings for “load_data“
\nand “load_data_wrapper“. In practice, “load_data_wrapper“ is the
\nfunction usually called by our neural network code.
\n“””<\/p>\n

#### Libraries
\n# Standard library
\ntry:
\nimport cPickle as pickle
\nexcept ImportError:
\nimport pickle<\/p>\n

# import cPickle
\nimport gzip<\/p>\n

# Third-party libraries
\nimport numpy as np<\/p>\n

def load_data():
\n“””Return the MNIST data as a tuple containing the training data,
\nthe validation data, and the test data.<\/p>\n

The “training_data“ is returned as a tuple with two entries.
\nThe first entry contains the actual training images. This is a
\nnumpy ndarray with 50,000 entries. Each entry is, in turn, a
\nnumpy ndarray with 784 values, representing the 28 * 28 = 784
\npixels in a single MNIST image.<\/p>\n

The second entry in the “training_data“ tuple is a numpy ndarray
\ncontaining 50,000 entries. Those entries are just the digit
\nvalues (0…9) for the corresponding images contained in the first
\nentry of the tuple.<\/p>\n

The “validation_data“ and “test_data“ are similar, except
\neach contains only 10,000 images.<\/p>\n

This is a nice data format, but for use in neural networks it’s
\nhelpful to modify the format of the “training_data“ a little.
\nThat’s done in the wrapper function “load_data_wrapper()“, see
\nbelow.
\n“””
\nwith gzip.open(‘.\/mnist.pkl.gz’, ‘rb’) as f:
\ntraining_data, validation_data, test_data = pickle.load(f, encoding=’latin1′)
\n#show_mnist_data(training_data, validation_data, test_data)<\/p>\n

#f = gzip.open(‘.\/mnist.pkl.gz’, ‘rb’)
\n#training_data, validation_data, test_data = cPickle.load(f, encoding=”latin1″)
\n#f.close()<\/p>\n

return (training_data, validation_data, test_data)<\/p>\n

def show_mnist_data(training_data, validation_data, test_data):
\nfw = open(“.\/log.txt”, ‘w’)
\nnp.set_printoptions(precision=4, linewidth=800)
\nnum = 0
\nfw.write(“\u4e0b\u9762\u4e3aMNIST\u6240\u6709\u56fe\u7247\u7070\u5ea6\u5316\u4e4b\u540e\u518d\u6570\u5b57\u5316\u62bd\u6837\u4e4b\u540e\u7684\u6570\u636e\u3002\\n”)
\nfw.write(“====== TRAINING DATA Start… ======\\n”)
\nfor x,y in training_data:
\nnum += 1
\nif(num > 10):
\nbreak
\nfw.write(“++++++ “+str(num)+ ” ++++++\\n”)
\nfw.write(“28×28\u7684\u56fe\u7247\u6570\u636e\uff1a\\n”)
\nfor i in x.reshape(28, 28):
\nfw.write(str(i)+”\\n”)
\nfw.write(“==>”+ “\u56fe\u7247\u5bf9\u5e940~9\u517110\u4e2a\u6570\u5b57\u7684\u6743\u91cd\uff1a”+ str(y.tolist())+”\\n”)
\nif num > 10:
\nfw.write(“……….\\n”)
\nfw.write(“====== TRAINING DATA End. ======\\n”)
\nfw.write(” \\n”)
\nfw.write(” \\n”)<\/p>\n

num = 0
\nfw.write(“====== VALIDATION DATA Start… ======\\n”)
\nfor x,y in validation_data:
\nnum += 1
\nif(num > 10):
\nbreak
\nfw.write(“++++++ “+str(num)+ ” ++++++\\n”)
\nfw.write(“28×28\u7684\u56fe\u7247\u6570\u636e\uff1a\\n”)
\nfor i in x.reshape(28, 28):
\nfw.write(str(i)+”\\n”)
\nfw.write(“==>”+ “\u56fe\u7247\u6240\u5bf9\u5e94\u7684\u6570\u5b57\uff1a”+ str(y.tolist())+”\\n”)
\nif num > 10:
\nfw.write(“……….\\n”)
\nfw.write(“====== VALIDATION DATA End. ======\\n”)
\nfw.write(” \\n”)
\nfw.write(” \\n”)<\/p>\n

num = 0
\nfw.write(“====== TEST DATA Start… ======\\n”)
\nfor x,y in test_data:
\nnum += 1
\nif(num > 10):
\nbreak
\nfw.write(“++++++ “+str(num)+ ” ++++++\\n”)
\nfw.write(“28×28\u7684\u56fe\u7247\u6570\u636e\uff1a\\n”)
\nfor i in x.reshape(28, 28):
\nfw.write(str(i)+”\\n”)
\nfw.write(“==>”+ “\u56fe\u7247\u6240\u5bf9\u5e94\u7684\u6570\u5b57\uff1a”+ str(y.tolist())+”\\n”)
\nif num > 10:
\nfw.write(“……….\\n”)
\nfw.write(“====== TEST DATA End. ======\\n”)
\nfw.write(” \\n”)
\nfw.write(” \\n”)
\nfw.close()<\/p>\n

def load_data_wrapper():
\n“””Return a tuple containing “(training_data, validation_data,
\ntest_data)“. Based on “load_data“, but the format is more
\nconvenient for use in our implementation of neural networks.<\/p>\n

In particular, “training_data“ is a list containing 50,000
\n2-tuples “(x, y)“. “x“ is a 784-dimensional numpy.ndarray
\ncontaining the input image. “y“ is a 10-dimensional
\nnumpy.ndarray representing the unit vector corresponding to the
\ncorrect digit for “x“.<\/p>\n

“validation_data“ and “test_data“ are lists containing 10,000
\n2-tuples “(x, y)“. In each case, “x“ is a 784-dimensional
\nnumpy.ndarry containing the input image, and “y“ is the
\ncorresponding classification, i.e., the digit values (integers)
\ncorresponding to “x“.<\/p>\n

Obviously, this means we’re using slightly different formats for
\nthe training data and the validation \/ test data. These formats
\nturn out to be the most convenient for use in our neural network
\ncode.”””
\ntr_d, va_d, te_d = load_data()
\ntraining_inputs = [np.reshape(x, (784, 1)) for x in tr_d[0]]
\ntraining_results = [vectorized_result(y) for y in tr_d[1]]
\ntraining_data = zip(training_inputs, training_results)
\nvalidation_inputs = [np.reshape(x, (784, 1)) for x in va_d[0]]
\nvalidation_data = zip(validation_inputs, va_d[1])
\ntest_inputs = [np.reshape(x, (784, 1)) for x in te_d[0]]
\ntest_data = zip(test_inputs, te_d[1])<\/p>\n

# show_mnist_data(training_data, validation_data, test_data) #\u8f93\u51fa\uff0c\u4f46\u662fzip\u7684iter\u53d8\u91cf\u53ea\u80fd\u7528\u4e00\u6b21\uff0c\u8f93\u51fa\u4e4b\u540ezip\u5c31\u53d8\u7a7a\u4e86
\nreturn (training_data, validation_data, test_data)<\/p>\n

def vectorized_result(j):
\n“””Return a 10-dimensional unit vector with a 1.0 in the jth
\nposition and zeroes elsewhere. This is used to convert a digit
\n(0…9) into a corresponding desired output from the neural
\nnetwork.”””
\ne = np.zeros((10, 1))
\ne[j] = 1.0
\nreturn e<\/p>\n

<\/div>\n
<\/div>\n
network1.py\u4ee3\u7801\uff1a<\/strong><\/span><\/div>\n
\u70b9\u51fb\u4e0b\u8f7d\uff1anetwork1<\/a><\/strong><\/span><\/div>\n
<\/div>\n

#!\/usr\/bin\/python
\n# -*- coding: UTF-8 -*-<\/p>\n

import random
\nimport numpy as np<\/p>\n

def sigmoid(z):
\n“””The simgoid fnction.”””
\nreturn 1.0\/(1.0+np.exp(-z))<\/p>\n

def sigmoid_prime(z):
\n“””Derivative of the sigmoid function.”””
\nreturn sigmoid(z)*(1-sigmoid(z))<\/p>\n

class Network(object):
\ndef __init__(self, sizes):
\nself.num_layers = len(sizes)
\nself.sizes = sizes
\nself.biases = [np.random.randn(y, 1) for y in sizes[1:]]
\nself.weights = [np.random.randn(y, x) for x, y in zip(sizes[:-1], sizes[1:])]<\/p>\n

print(“\u8f93\u5165\u56fe\u7247\u50cf\u7d20\u70b9\u6570\uff1a”+str(self.sizes[0]) +”, \u8f93\u5165\u795e\u7ecf\u7f51\u7edc\u5c42\u6570\uff1a”+str(self.num_layers))
\nprint(“\u7b2c 1 \u5c42\u4e3a\u8f93\u5165\u5c42\uff08\u5171 “+str(self.sizes[0])+” \u4e2a\u795e\u7ecf\u5143\uff09\uff0c\u6743\u91cd\u548c\u504f\u7f6e\u90fd\u4e0d\u9700\u8981\u3002”)
\nfor i in range(1, self.num_layers):
\nprint(“\u7b2c “+str(i+1)+” \u5c42\u5171\u6709 “+str(self.sizes[i])+” \u4e2a\u795e\u7ecf\u5143\uff0c\u521d\u59cb\u6743\u91cd\u548c\u504f\u7f6e\u5982\u4e0b\u3002”)
\nprint(” “)
\nlayer = 1
\nprint(“\u8bad\u7ec3\u524d\u7684\u6743\u91cd\u548c\u504f\u7f6e\uff1a”)
\nfor lw, lb in zip(self.weights, self.biases):
\nlayer += 1
\nnum_nw = 0
\nfor w, b in zip(lw, lb):
\nnum_nw += 1
\nprint(“\u7b2c “+str(layer)+” \u5c42\u7b2c “+str(num_nw)+” \u4e2a\u795e\u7ecf\u5143\u7684\u6743\u91cd(\u5171”+str(len(w))+”\u4e2a\u8f93\u5165)\uff1a”)
\nprint(str(w))
\nprint(“\u7b2c “+str(layer)+” \u5c42\u7b2c “+str(num_nw)+” \u4e2a\u795e\u7ecf\u5143\u7684\u504f\u7f6e(\u5171”+str(len(b))+”\u4e2a\u8f93\u51fa)\uff1a”)
\nprint(str(b))
\nprint(” “)
\nprint(” “)<\/p>\n

def feedforward(self ,a):
\n“””return the output of the network if “a” is input.”””
\nfor b, w in zip(self.biases, self.weights):
\na = sigmoid(np.dot(w, a)+b)
\nreturn a<\/p>\n

def SGD(self, training_data, epochs, mini_batch_size, eta, test_data=None):
\n“””Train the neural network using mini-batch stochastic gradient descent.
\nThe “training_data” is a list of tuples “(x, y)” representing the training inputs and the desired outputs.
\nThe other non-optional parameters are self-explanatory.
\nIf “test_data” is provided then the network will be evaluated against the test data after each epoch, and partial progress printed out.
\nThis is useful for tracking progress, but slows things down substantially.
\n“””
\ntraining_data_list = list(training_data)
\ntest_data_list = list(test_data)
\nif test_data:
\nn_test = len(test_data_list)
\nn = len(training_data_list)
\nprint(“=================================================================”)
\nprint(“=================================================================”)
\nprintStr = “”
\nloop = 0
\nfor j in range(epochs):
\nloop += 1
\nrandom.shuffle(training_data_list)
\ntraining_data = training_data_list
\nmini_batches = [training_data[k:k+mini_batch_size] for k in range(0, n, mini_batch_size)]
\nfor mini_batch in mini_batches:
\nself.update_mini_batch(mini_batch, eta)
\nif test_data:
\ntest_data = test_data_list
\nprintStr += (“Epoch {0}: \u6210\u529f\u8bc6\u522b\uff1a{1} \/ \u603b\u5171\uff1a{2}”.format(j, self.evaluate(test_data), n_test)+”\\n”)
\nprint(“Epoch {0}: \u6210\u529f\u8bc6\u522b\uff1a{1} \/ \u603b\u5171\uff1a{2}”.format(j, self.evaluate(test_data), n_test))
\nelse:
\nprint(“Loop “+str(loop))
\nprintStr += (“Epoch {0} complete”.format(j)+”\\n”)
\nprint(“Epoch {0} complete”.format(j))<\/p>\n

print(“=================================================================”)
\nprint(“=================================================================”)<\/p>\n

print(” “)
\nlayer = 1
\nprint(“\u8bad\u7ec3\u540e\u7684\u6743\u91cd\u548c\u504f\u7f6e\uff1a”)
\nfor lw, lb in zip(self.weights, self.biases):
\nlayer += 1
\nnum_nw = 0
\nfor w, b in zip(lw, lb):
\nnum_nw += 1
\nprint(“\u7b2c “+str(layer)+” \u5c42\u7b2c “+str(num_nw)+” \u4e2a\u795e\u7ecf\u5143\u7684\u6743\u91cd(\u5171”+str(len(w))+”\u4e2a\u8f93\u5165)\uff1a”)
\nprint(str(w))
\nprint(“\u7b2c “+str(layer)+” \u5c42\u7b2c “+str(num_nw)+” \u4e2a\u795e\u7ecf\u5143\u7684\u504f\u7f6e(\u5171”+str(len(b))+”\u4e2a\u8f93\u51fa)\uff1a”)
\nprint(str(b))
\nprint(” “)
\nprint(” “)<\/p>\n

print(“=================================================================”)
\nprint(“=================================================================”)
\nprint(printStr)
\nprint(“\u6bcf\u5c42\u6d4b\u8bd5\u7ed3\u679c\u7684\u8ba1\u7b97\u516c\u5f0f\uff1a (\u8f93\u51651*\u6743\u91cd1 + \u8f93\u51652*\u6743\u91cd2 + \u8f93\u51653*\u6743\u91cd3 + …) + \u504f\u7f6e “)
\nprint(“a1*w11—|”)
\nprint(“a2*w12—|+b1—|”)
\nprint(“a3*w13—| |”)
\nprint(“……—| |-output”)
\nprint(“a1*w21—| |”)
\nprint(“a2*w22—|+b2—|”)
\nprint(“a3*w23—|”)
\nprint(“……—|”)<\/p>\n

def update_mini_batch(self, mini_batch, eta):
\n“””Update the network’s weights and biases by applying gradient descent using backpropagation to a single mini batch.
\nThe “mini_batch” is a list of tuples “(x, y)”, and “eta” is the learning rate.
\n“””
\nnabla_b = [np.zeros(b.shape) for b in self.biases]
\nnabla_w = [np.zeros(w.shape) for w in self.weights]
\nfor x, y in mini_batch:
\ndelta_nabla_b, delta_nabla_w = self.backprop(x, y)
\nnabla_b = [nb + dnb for nb, dnb in zip(nabla_b, delta_nabla_b)]
\nnabla_w = [nw + dnw for nw, dnw in zip(nabla_w, delta_nabla_w)]
\nself.weights = [w – (eta\/len(mini_batch))*nw for w, nw in zip(self.weights, nabla_w)]
\nself.biases = [b – (eta\/len(mini_batch))*nb for b, nb in zip(self.biases, nabla_b)]<\/p>\n

def backprop(self, x, y):
\n“””Return a tuple “(nabla_b, nabla_w)“ representing the gradient for the cost function C_x.
\n“nabla_b“ and “nabla_w“ are layer-by-layer lists of numpy arrays, similar to “self.biases“ and “self.weights“.
\n“””
\nnabla_b = [np.zeros(b.shape) for b in self.biases]
\nnabla_w = [np.zeros(w.shape) for w in self.weights]
\n#feedforward
\nactivation = x
\nactivations = [x] # list to store all the activations, layer by layer
\nzs = [] # list to store all the z vectors, layer by layer
\nfor b, w in zip(self.biases, self.weights):
\nz = np.dot(w, activation) + b
\nzs.append(z)
\nactivation = sigmoid(z)
\nactivations.append(activation)
\n# backward pass
\ndelta = self.cost_derivative(activations[-1], y) * sigmoid_prime(zs[-1])
\nnabla_b[-1] = delta
\nnabla_w[-1] = np.dot(delta, activations[-2].transpose())
\n# Note that the variable l in the loop below is used a little
\n# differently to the notation in Chapter 2 of the book. Here,
\n# l = 1 means the last layer of neurons, l = 2 is the
\n# second-last layer, and so on. It’s a renumbering of the
\n# scheme in the book, used here to take advantage of the fact
\n# that Python can use negative indices in lists.
\nfor l in range(2, self.num_layers):
\nz = zs[-l]
\nsp = sigmoid_prime(z)
\ndelta = np.dot(self.weights[-l+1].transpose(), delta) * sp
\nnabla_b[-l] = delta
\nnabla_w[-l] = np.dot(delta, activations[-l-1].transpose())
\nreturn (nabla_b, nabla_w)<\/p>\n

def evaluate(self, test_data):
\n“””Return the number of test inputs for which the neural network outputs the correct result.
\nNote that the neural network’s output is assumed to be the index of whichever neuron in the final layer has the highest activation.
\n“””
\ntest_results = [(np.argmax(self.feedforward(x)), y) for (x, y) in test_data]
\nreturn sum(int(x == y) for (x, y) in test_results)<\/p>\n

def cost_derivative(self, output_activations, y):
\n“””Return the vector of partial derivatives \\partial C_x \/ \\partial a for the output activations.
\n“””
\nreturn (output_activations – y)<\/p>\n

def test():
\nimport mnist_loader
\ntraining_data, validation_data, test_data = mnist_loader.load_data_wrapper()
\nnet = Network([784, 30, 10])
\nnet.SGD(training_data, 30, 10, 3.0, test_data=test_data)<\/p>\n

if __name__ == ‘__main__’:
\ntest()<\/p>\n

<\/div>\n","protected":false},"excerpt":{"rendered":"

\u300a\u795e\u7ecf\u7f51\u7edc\u4e0e\u6df1\u5ea6\u5b66\u4e60\u300b\uff08Michael Nielsen\u8457\uff09\u4e2d\u7684\u4ee3\u7801\u662f\u57fa\u4e8epython2.7\u7684\uff0c\u4e0b\u9762\u4e3a\u79fb\u690d\u5230py …<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[443],"tags":[726,108,446,447],"class_list":["post-981","post","type-post","status-publish","format-standard","hentry","category-443","tag-network1-py","tag-python","tag-446","tag-447"],"views":2114,"_links":{"self":[{"href":"http:\/\/www.max-shu.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/981","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.max-shu.com\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.max-shu.com\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.max-shu.com\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.max-shu.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=981"}],"version-history":[{"count":2,"href":"http:\/\/www.max-shu.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/981\/revisions"}],"predecessor-version":[{"id":985,"href":"http:\/\/www.max-shu.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/981\/revisions\/985"}],"wp:attachment":[{"href":"http:\/\/www.max-shu.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=981"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.max-shu.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=981"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.max-shu.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=981"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}