卷积神经网络(CNN)的简单介绍及代码实现
卷积神经网络(cnn)的基础介绍见 ，这里主要以代码实现为主。
cnn是一个多层的神经网络，每层由多个二维平面组成，而每个平面由多个独立神经元组成。
以mnist作为数据库，仿照lenet-5和tiny-cnn( ) 设计一个简单的7层cnn结构如下：
输入层input：神经元数量32*32=1024；
c1层：卷积窗大小5*5，输出特征图数量6，卷积窗种类6，输出特征图大小28*28，可训练参数(权值+阈值(偏置))5*5*6+6=150+6，神经元数量28*28*6=4704；
s2层：卷积窗大小2*2，输出下采样图数量6，卷积窗种类6，输出下采样图大小14*14，可训练参数1*6+6=6+6，神经元数量14*14*6=1176；
c3层：卷积窗大小5*5，输出特征图数量16，卷积窗种类6*16=96，输出特征图大小10*10，可训练参数5*5*(6*16)+16=2400+16，神经元数量10*10*16=1600；
s4层：卷积窗大小2*2，输出下采样图数量16，卷积窗种类16，输出下采样图大小5*5，可训练参数1*16+16=16+16，神经元数量5*5*16=400；
c5层：卷积窗大小5*5，输出特征图数量120，卷积窗种类16*120=1920，输出特征图大小1*1，可训练参数5*5*(16*120)+120=48000+120，神经元数量1*1*120=120；
输出层output：卷积窗大小1*1，输出特征图数量10，卷积窗种类120*10=1200，输出特征图大小1*1，可训练参数1*(120*10)+10=1200+10，神经元数量1*1*10=10。
下面对实现执行过程进行描述说明：
1. 从mnist数据库中分别获取训练样本和测试样本数据：
(1)、原有mnist库中图像大小为28*28，这里缩放为32*32，数据值范围为[-1,1]，扩充值均取-1；总共60000个32*32训练样本，10000个32*32测试样本；
(2)、输出层有10个输出节点，在训练阶段，对应位置的节点值设为0.8，其它节点设为-0.8.
2. 初始化权值和阈值(偏置)：权值就是卷积图像，每一个特征图上的神经元共享相同的权值和阈值，特征图的数量等于阈值的个数
(1)、权值采用uniform rand的方法初始化；
(2)、阈值均初始化为0.
3. 前向传播：根据权值和阈值，主要计算每层神经元的值
(1)、输入层：每次输入一个32*32数据。
(2)、c1层：分别用每一个5*5的卷积图像去乘以32*32的图像，获得一个28*28的图像，即对应位置相加再求和，stride长度为1；一共6个5*5的卷积图像，然后对每一个神经元加上一个阈值，最后再通过tanh激活函数对每一神经元进行运算得到最终每一个神经元的结果。
(3)、s2层：对c1中6个28*28的特征图生成6个14*14的下采样图，相邻四个神经元分别进行相加求和，然后乘以一个权值，再求均值即除以4，然后再加上一个阈值，最后再通过tanh激活函数对每一神经元进行运算得到最终每一个神经元的结果。
(4)、c3层：由s2中的6个14*14下采样图生成16个10*10特征图，对于生成的每一个10*10的特征图，是由6个5*5的卷积图像去乘以6个14*14的下采样图，然后对应位置相加求和，然后对每一个神经元加上一个阈值，最后再通过tanh激活函数对每一神经元进行运算得到最终每一个神经元的结果。
(5)、s4层：由c3中16个10*10的特征图生成16个5*5下采样图，相邻四个神经元分别进行相加求和，然后乘以一个权值，再求均值即除以4，然后再加上一个阈值，最后再通过tanh激活函数对每一神经元进行运算得到最终每一个神经元的结果。
(6)、c5层：由s4中16个5*5下采样图生成120个1*1特征图，对于生成的每一个1*1的特征图，是由16个5*5的卷积图像去乘以16个5*5的下采用图，然后相加求和，然后对每一个神经元加上一个阈值，最后再通过tanh激活函数对每一神经元进行运算得到最终每一个神经元的结果。
(7)、输出层：即全连接层，输出层中的每一个神经元均是由c5层中的120个神经元乘以相对应的权值，然后相加求和；然后对每一个神经元加上一个阈值，最后再通过tanh激活函数对每一神经元进行运算得到最终每一个神经元的结果。
4. 反向传播：主要计算每层神经元、权值和阈值的误差，以用来更新权值和阈值
(1)、输出层：计算输出层神经元误差；通过mse损失函数的导数函数和tanh激活函数的导数函数来计算输出层神经元误差。
(2)、c5层：计算c5层神经元误差、输出层权值误差、输出层阈值误差；通过输出层神经元误差乘以输出层权值，求和，结果再乘以c5层神经元的tanh激活函数的导数，获得c5层每一个神经元误差；通过输出层神经元误差乘以c5层神经元获得输出层权值误差；输出层误差即为输出层阈值误差。
(3)、s4层：计算s4层神经元误差、c5层权值误差、c5层阈值误差；通过c5层权值乘以c5层神经元误差，求和，结果再乘以s4层神经元的tanh激活函数的导数，获得s4层每一个神经元误差；通过s4层神经元乘以c5层神经元误差，求和，获得c5层权值误差；c5层神经元误差即为c5层阈值误差。
(4)、c3层：计算c3层神经元误差、s4层权值误差、s4层阈值误差；
(5)、s2层：计算s2层神经元误差、c3层权值误差、c3层阈值误差；
(6)、c1层：计算c1层神经元误差、s2层权值误差、s2层阈值误差；
(7)、输入层：计算c1层权值误差、c1层阈值误差.
代码文件：
cnn.hpp：
#ifndef _cnn_hpp_
#define _cnn_hpp_
#include
#include
namespace ann {
#define width_image_input_cnn 32 //归一化图像宽
#define height_image_input_cnn 32 //归一化图像高
#define width_image_c1_cnn 28
#define height_image_c1_cnn 28
#define width_image_s2_cnn 14
#define height_image_s2_cnn 14
#define width_image_c3_cnn 10
#define height_image_c3_cnn 10
#define width_image_s4_cnn 5
#define height_image_s4_cnn 5
#define width_image_c5_cnn 1
#define height_image_c5_cnn 1
#define width_image_output_cnn 1
#define height_image_output_cnn 1
#define width_kernel_conv_cnn 5 //卷积核大小
#define height_kernel_conv_cnn 5
#define width_kernel_pooling_cnn 2
#define height_kernel_pooling_cnn 2
#define size_pooling_cnn 2
#define num_map_input_cnn 1 //输入层map个数
#define num_map_c1_cnn 6 //c1层map个数
#define num_map_s2_cnn 6 //s2层map个数
#define num_map_c3_cnn 16 //c3层map个数
#define num_map_s4_cnn 16 //s4层map个数
#define num_map_c5_cnn 120 //c5层map个数
#define num_map_output_cnn 10 //输出层map个数
#define num_patterns_train_cnn 60000 //训练模式对数(总数)
#define num_patterns_test_cnn 10000 //测试模式对数(总数)
#define num_epochs_cnn 100 //最大迭代次数
#define accuracy_rate_cnn 0.985 //要求达到的准确率
#define learning_rate_cnn 0.01 //学习率
#define eps_cnn 1e-8
#define len_weight_c1_cnn 150 //c1层权值数，5*5*6*1=150
#define len_bias_c1_cnn 6 //c1层阈值数，6
#define len_weight_s2_cnn 6 //s2层权值数,1*6=6
#define len_bias_s2_cnn 6 //s2层阈值数,6
#define len_weight_c3_cnn 2400 //c3层权值数，5*5*16*6=2400
#define len_bias_c3_cnn 16 //c3层阈值数,16
#define len_weight_s4_cnn 16 //s4层权值数，1*16=16
#define len_bias_s4_cnn 16 //s4层阈值数，16
#define len_weight_c5_cnn 48000 //c5层权值数，5*5*16*120=48000
#define len_bias_c5_cnn 120 //c5层阈值数，120
#define len_weight_output_cnn 1200 //输出层权值数，120*10=1200
#define len_bias_output_cnn 10 //输出层阈值数，10
#define num_neuron_input_cnn 1024 //输入层神经元数，32*32=1024
#define num_neuron_c1_cnn 4704 //c1层神经元数，28*28*6=4704
#define num_neuron_s2_cnn 1176 //s2层神经元数，14*14*6=1176
#define num_neuron_c3_cnn 1600 //c3层神经元数，10*10*16=1600
#define num_neuron_s4_cnn 400 //s4层神经元数，5*5*16=400
#define num_neuron_c5_cnn 120 //c5层神经元数，1*120=120
#define num_neuron_output_cnn 10 //输出层神经元数，1*10=10
class cnn {
public:
cnn();
~cnn();
void init(); //初始化，分配空间
bool train(); //训练
int predict(const unsigned char* data, int width, int height); //预测
bool readmodelfile(const char* name); //读取已训练好的bp model
protected:
typedef std::vector wi_connections;
typedef std::vector wo_connections;
typedef std::vector io_connections;> 16) & 255;
ch4 = (i >> 24) & 255;
return((int)ch1 << 24) + ((int)ch2 << 16) + ((int)ch3 << 8) + ch4;
}
static void readmnistimages(std::string filename, double* data_dst, int num_image)
{
const int width_src_image = 28;
const int height_src_image = 28;
const int x_padding = 2;
const int y_padding = 2;
const double scale_min = -1;
const double scale_max = 1;
std::ifstream file(filename, std::ios::binary);
assert(file.is_open());
int magic_number = 0;
int number_of_images = 0;
int n_rows = 0;
int n_cols = 0;
file.read((char*)&magic_number, sizeof(magic_number));
magic_number = reverseint(magic_number);
file.read((char*)&number_of_images, sizeof(number_of_images));
number_of_images = reverseint(number_of_images);
assert(number_of_images == num_image);
file.read((char*)&n_rows, sizeof(n_rows));
n_rows = reverseint(n_rows);
file.read((char*)&n_cols, sizeof(n_cols));
n_cols = reverseint(n_cols);
assert(n_rows == height_src_image && n_cols == width_src_image);
int size_single_image = width_image_input_cnn * height_image_input_cnn;
for (int i = 0; i < number_of_images; ++i) {
int addr = size_single_image * i;
for (int r = 0; r < n_rows; ++r) {
for (int c = 0; c < n_cols; ++c) {
unsigned char temp = 0;
file.read((char*)&temp, sizeof(temp));
data_dst[addr + width_image_input_cnn * (r + y_padding) + c + x_padding] = (temp / 255.0) * (scale_max - scale_min) + scale_min;
}
}
}
}
static void readmnistlabels(std::string filename, double* data_dst, int num_image)
{
const double scale_max = 0.8;
std::ifstream file(filename, std::ios::binary);
assert(file.is_open());
int magic_number = 0;
int number_of_images = 0;
file.read((char*)&magic_number, sizeof(magic_number));
magic_number = reverseint(magic_number);
file.read((char*)&number_of_images, sizeof(number_of_images));
number_of_images = reverseint(number_of_images);
assert(number_of_images == num_image);
for (int i = 0; i < number_of_images; ++i) {
unsigned char temp = 0;
file.read((char*)&temp, sizeof(temp));
data_dst[i * num_map_output_cnn + temp] = scale_max;
}
}
bool cnn::getsrcdata()
{
assert(data_input_train && data_output_train && data_input_test && data_output_test);
std::string filename_train_images = e:/gitcode/nn_test/data/train-images.idx3-ubyte;
std::string filename_train_labels = e:/gitcode/nn_test/data/train-labels.idx1-ubyte;
readmnistimages(filename_train_images, data_input_train, num_patterns_train_cnn);
readmnistlabels(filename_train_labels, data_output_train, num_patterns_train_cnn);
std::string filename_test_images = e:/gitcode/nn_test/data/t10k-images.idx3-ubyte;
std::string filename_test_labels = e:/gitcode/nn_test/data/t10k-labels.idx1-ubyte;
readmnistimages(filename_test_images, data_input_test, num_patterns_test_cnn);
readmnistlabels(filename_test_labels, data_output_test, num_patterns_test_cnn);
return true;
}
bool cnn::train()
{
out2wi_s2.clear();
out2bias_s2.clear();
out2wi_s4.clear();
out2bias_s4.clear();
in2wo_c3.clear();
weight2io_c3.clear();
bias2out_c3.clear();
in2wo_c1.clear();
weight2io_c1.clear();
bias2out_c1.clear();
calc_out2wi(width_image_c1_cnn, height_image_c1_cnn, width_image_s2_cnn, height_image_s2_cnn, num_map_s2_cnn, out2wi_s2);
calc_out2bias(width_image_s2_cnn, height_image_s2_cnn, num_map_s2_cnn, out2bias_s2);
calc_out2wi(width_image_c3_cnn, height_image_c3_cnn, width_image_s4_cnn, height_image_s4_cnn, num_map_s4_cnn, out2wi_s4);
calc_out2bias(width_image_s4_cnn, height_image_s4_cnn, num_map_s4_cnn, out2bias_s4);
calc_in2wo(width_image_c3_cnn, height_image_c3_cnn, width_image_s4_cnn, height_image_s4_cnn, num_map_c3_cnn, num_map_s4_cnn, in2wo_c3);
calc_weight2io(width_image_c3_cnn, height_image_c3_cnn, width_image_s4_cnn, height_image_s4_cnn, num_map_c3_cnn, num_map_s4_cnn, weight2io_c3);
calc_bias2out(width_image_c3_cnn, height_image_c3_cnn, width_image_s4_cnn, height_image_s4_cnn, num_map_c3_cnn, num_map_s4_cnn, bias2out_c3);
calc_in2wo(width_image_c1_cnn, height_image_c1_cnn, width_image_s2_cnn, height_image_s2_cnn, num_map_c1_cnn, num_map_c3_cnn, in2wo_c1);
calc_weight2io(width_image_c1_cnn, height_image_c1_cnn, width_image_s2_cnn, height_image_s2_cnn, num_map_c1_cnn, num_map_c3_cnn, weight2io_c1);
calc_bias2out(width_image_c1_cnn, height_image_c1_cnn, width_image_s2_cnn, height_image_s2_cnn, num_map_c1_cnn, num_map_c3_cnn, bias2out_c1);
int iter = 0;
for (iter = 0; iter < num_epochs_cnn; iter++) {
std::cout << epoch: << iter + 1;
for (int i = 0; i < num_patterns_train_cnn; i++) {
data_single_image = data_input_train + i * num_neuron_input_cnn;
data_single_label = data_output_train + i * num_neuron_output_cnn;
forward_c1();
forward_s2();
forward_c3();
forward_s4();
forward_c5();
forward_output();
backward_output();
backward_c5();
backward_s4();
backward_c3();
backward_s2();
backward_c1();
backward_input();
updateweights();
}
double accuracyrate = test();
std::cout << , accuray rate: << accuracyrate < accuracy_rate_cnn) {
savemodelfile(e:/gitcode/nn_test/data/cnn.model);
std::cout << generate cnn model << std::endl;
break;
}
}
if (iter == num_epochs_cnn) {
savemodelfile(e:/gitcode/nn_test/data/cnn.model);
std::cout << generate cnn model << std::endl;
}
return true;
}
double cnn::activation_function_tanh(double x)
{
double ep = std::exp(x);
double em = std::exp(-x);
return (ep - em) / (ep + em);
}
double cnn::activation_function_tanh_derivative(double x)
{
return (1.0 - x * x);
}
double cnn::activation_function_identity(double x)
{
return x;
}
double cnn::activation_function_identity_derivative(double x)
{
return 1;
}
double cnn::loss_function_mse(double y, double t)
{
return (y - t) * (y - t) / 2;
}
double cnn::loss_function_mse_derivative(double y, double t)
{
return (y - t);
}
void cnn::loss_function_gradient(const double* y, const double* t, double* dst, int len)
{
for (int i = 0; i < len; i++) {
dst[i] = loss_function_mse_derivative(y[i], t[i]);
}
}
double cnn::dot_product(const double* s1, const double* s2, int len)
{
double result = 0.0;
for (int i = 0; i < len; i++) {
result += s1[i] * s2[i];
}
return result;
}
bool cnn::muladd(const double* src, double c, int len, double* dst)
{
for (int i = 0; i = 0 && x = 0 && y = 0 && channel < depth);
return (height * channel + y) * width + x;
}
void cnn::calc_out2wi(int width_in, int height_in, int width_out, int height_out, int depth_out, std::vector& out2wi)
{
for (int i = 0; i < depth_out; i++) {
int block = width_in * height_in * i;
for (int y = 0; y < height_out; y++) {
for (int x = 0; x < width_out; x++) {
int rows = y * width_kernel_pooling_cnn;
int cols = x * height_kernel_pooling_cnn;
wi_connections wi_connections_;
std::pair pair_;
for (int m = 0; m < width_kernel_pooling_cnn; m++) {
for (int n = 0; n < height_kernel_pooling_cnn; n++) {
pair_.first = i;
pair_.second = (rows + m) * width_in + cols + n + block;
wi_connections_.push_back(pair_);
}
}
out2wi.push_back(wi_connections_);
}
}
}
}
void cnn::calc_out2bias(int width, int height, int depth, std::vector& out2bias)
{
for (int i = 0; i < depth; i++) {
for (int y = 0; y < height; y++) {
for (int x = 0; x < width; x++) {
out2bias.push_back(i);
}
}
}
}
void cnn::calc_in2wo(int width_in, int height_in, int width_out, int height_out, int depth_in, int depth_out, std::vector& in2wo)
{
int len = width_in * height_in * depth_in;
in2wo.resize(len);
for (int c = 0; c < depth_in; c++) {
for (int y = 0; y < height_in; y += height_kernel_pooling_cnn) {
for (int x = 0; x < width_in; x += width_kernel_pooling_cnn) {
int dymax = min(size_pooling_cnn, height_in - y);
int dxmax = min(size_pooling_cnn, width_in - x);
int dstx = x / width_kernel_pooling_cnn;
int dsty = y / height_kernel_pooling_cnn;
for (int dy = 0; dy < dymax; dy++) {
for (int dx = 0; dx < dxmax; dx++) {
int index_in = get_index(x + dx, y + dy, c, width_in, height_in, depth_in);
int index_out = get_index(dstx, dsty, c, width_out, height_out, depth_out);
wo_connections wo_connections_;
std::pair pair_;
pair_.first = c;
pair_.second = index_out;
wo_connections_.push_back(pair_);
in2wo[index_in] = wo_connections_;
}
}
}
}
}
}
void cnn::calc_weight2io(int width_in, int height_in, int width_out, int height_out, int depth_in, int depth_out, std::vector& weight2io)
{
int len = depth_in;
weight2io.resize(len);
for (int c = 0; c < depth_in; c++) {
for (int y = 0; y < height_in; y += height_kernel_pooling_cnn) {
for (int x = 0; x < width_in; x += width_kernel_pooling_cnn) {
int dymax = min(size_pooling_cnn, height_in - y);
int dxmax = min(size_pooling_cnn, width_in - x);
int dstx = x / width_kernel_pooling_cnn;
int dsty = y / height_kernel_pooling_cnn;
for (int dy = 0; dy < dymax; dy++) {
for (int dx = 0; dx < dxmax; dx++) {
int index_in = get_index(x + dx, y + dy, c, width_in, height_in, depth_in);
int index_out = get_index(dstx, dsty, c, width_out, height_out, depth_out);
std::pair pair_;
pair_.first = index_in;
pair_.second = index_out;
weight2io[c].push_back(pair_);
}
}
}
}
}
}
void cnn::calc_bias2out(int width_in, int height_in, int width_out, int height_out, int depth_in, int depth_out, std::vector& bias2out)
{
int len = depth_in;
bias2out.resize(len); max_value) {
max_value = neuron_output[i];
pos = i;
}
}
return pos;
}
bool cnn::readmodelfile(const char* name)
{
file* fp = fopen(name, rb);
if (fp == null) {
return false;
}
int width_image_input =0;
int height_image_input = 0;
int width_image_c1 = 0;
int height_image_c1 = 0;
int width_image_s2 = 0;
int height_image_s2 = 0;
int width_image_c3 = 0;
int height_image_c3 = 0;
int width_image_s4 = 0;
int height_image_s4 = 0;
int width_image_c5 = 0;
int height_image_c5 = 0;
int width_image_output = 0;
int height_image_output = 0;
int width_kernel_conv = 0;
int height_kernel_conv = 0;
int width_kernel_pooling = 0;
int height_kernel_pooling = 0;
int num_map_input = 0;
int num_map_c1 = 0;
int num_map_s2 = 0;
int num_map_c3 = 0;
int num_map_s4 = 0;
int num_map_c5 = 0;
int num_map_output = 0;
int len_weight_c1 = 0;
int len_bias_c1 = 0;
int len_weight_s2 = 0;
int len_bias_s2 = 0;
int len_weight_c3 = 0;
int len_bias_c3 = 0;
int len_weight_s4 = 0;
int len_bias_s4 = 0;
int len_weight_c5 = 0;
int len_bias_c5 = 0;
int len_weight_output = 0;
int len_bias_output = 0;
int num_neuron_input = 0;
int num_neuron_c1 = 0;
int num_neuron_s2 = 0;
int num_neuron_c3 = 0;
int num_neuron_s4 = 0;
int num_neuron_c5 = 0;
int num_neuron_output = 0;
fread(&width_image_input, sizeof(int), 1, fp);
fread(&height_image_input, sizeof(int), 1, fp);
fread(&width_image_c1, sizeof(int), 1, fp);
fread(&height_image_c1, sizeof(int), 1, fp);
fread(&width_image_s2, sizeof(int), 1, fp);
fread(&height_image_s2, sizeof(int), 1, fp);
fread(&width_image_c3, sizeof(int), 1, fp);
fread(&height_image_c3, sizeof(int), 1, fp);
fread(&width_image_s4, sizeof(int), 1, fp);
fread(&height_image_s4, sizeof(int), 1, fp);
fread(&width_image_c5, sizeof(int), 1, fp);
fread(&height_image_c5, sizeof(int), 1, fp);
fread(&width_image_output, sizeof(int), 1, fp);
fread(&height_image_output, sizeof(int), 1, fp);
fread(&width_kernel_conv, sizeof(int), 1, fp);
fread(&height_kernel_conv, sizeof(int), 1, fp);
fread(&width_kernel_pooling, sizeof(int), 1, fp);
fread(&height_kernel_pooling, sizeof(int), 1, fp);
fread(&num_map_input, sizeof(int), 1, fp);
fread(&num_map_c1, sizeof(int), 1, fp);
fread(&num_map_s2, sizeof(int), 1, fp);
fread(&num_map_c3, sizeof(int), 1, fp);
fread(&num_map_s4, sizeof(int), 1, fp);
fread(&num_map_c5, sizeof(int), 1, fp);
fread(&num_map_output, sizeof(int), 1, fp);
fread(&len_weight_c1, sizeof(int), 1, fp);
fread(&len_bias_c1, sizeof(int), 1, fp);
fread(&len_weight_s2, sizeof(int), 1, fp);
fread(&len_bias_s2, sizeof(int), 1, fp);
fread(&len_weight_c3, sizeof(int), 1, fp);
fread(&len_bias_c3, sizeof(int), 1, fp);
fread(&len_weight_s4, sizeof(int), 1, fp);
fread(&len_bias_s4, sizeof(int), 1, fp);
fread(&len_weight_c5, sizeof(int), 1, fp);
fread(&len_bias_c5, sizeof(int), 1, fp);
fread(&len_weight_output, sizeof(int), 1, fp);
fread(&len_bias_output, sizeof(int), 1, fp);
fread(&num_neuron_input, sizeof(int), 1, fp);
fread(&num_neuron_c1, sizeof(int), 1, fp);
fread(&num_neuron_s2, sizeof(int), 1, fp);
fread(&num_neuron_c3, sizeof(int), 1, fp);
fread(&num_neuron_s4, sizeof(int), 1, fp);
fread(&num_neuron_c5, sizeof(int), 1, fp);
fread(&num_neuron_output, sizeof(int), 1, fp);
fread(weight_c1, sizeof(weight_c1), 1, fp);
fread(bias_c1, sizeof(bias_c1), 1, fp);
fread(weight_s2, sizeof(weight_s2), 1, fp);
fread(bias_s2, sizeof(bias_s2), 1, fp);
fread(weight_c3, sizeof(weight_c3), 1, fp);
fread(bias_c3, sizeof(bias_c3), 1, fp);
fread(weight_s4, sizeof(weight_s4), 1, fp);
fread(bias_s4, sizeof(bias_s4), 1, fp);
fread(weight_c5, sizeof(weight_c5), 1, fp);
fread(bias_c5, sizeof(bias_c5), 1, fp);
fread(weight_output, sizeof(weight_output), 1, fp);
fread(bias_output, sizeof(bias_output), 1, fp);
fflush(fp);
fclose(fp);
out2wi_s2.clear();
out2bias_s2.clear();
out2wi_s4.clear();
out2bias_s4.clear();
calc_out2wi(width_image_c1_cnn, height_image_c1_cnn, width_image_s2_cnn, height_image_s2_cnn, num_map_s2_cnn, out2wi_s2);
calc_out2bias(width_image_s2_cnn, height_image_s2_cnn, num_map_s2_cnn, out2bias_s2);
calc_out2wi(width_image_c3_cnn, height_image_c3_cnn, width_image_s4_cnn, height_image_s4_cnn, num_map_s4_cnn, out2wi_s4);
calc_out2bias(width_image_s4_cnn, height_image_s4_cnn, num_map_s4_cnn, out2bias_s4);
return true;
}
bool cnn::savemodelfile(const char* name)
{
file* fp = fopen(name, wb);
if (fp == null) {
return false;
}
int width_image_input = width_image_input_cnn;
int height_image_input = height_image_input_cnn;
int width_image_c1 = width_image_c1_cnn;
int height_image_c1 = height_image_c1_cnn;
int width_image_s2 = width_image_s2_cnn;
int height_image_s2 = height_image_s2_cnn;
int width_image_c3 = width_image_c3_cnn;
int height_image_c3 = height_image_c3_cnn;
int width_image_s4 = width_image_s4_cnn;
int height_image_s4 = height_image_s4_cnn;
int width_image_c5 = width_image_c5_cnn;
int height_image_c5 = height_image_c5_cnn;
int width_image_output = width_image_output_cnn;
int height_image_output = height_image_output_cnn;
int width_kernel_conv = width_kernel_conv_cnn;
int height_kernel_conv = height_kernel_conv_cnn;
int width_kernel_pooling = width_kernel_pooling_cnn;
int height_kernel_pooling = height_kernel_pooling_cnn;
int num_map_input = num_map_input_cnn;
int num_map_c1 = num_map_c1_cnn;
int num_map_s2 = num_map_s2_cnn;
int num_map_c3 = num_map_c3_cnn;
int num_map_s4 = num_map_s4_cnn;
int num_map_c5 = num_map_c5_cnn;
int num_map_output = num_map_output_cnn;
int len_weight_c1 = len_weight_c1_cnn;
int len_bias_c1 = len_bias_c1_cnn;
int len_weight_s2 = len_weight_s2_cnn;
int len_bias_s2 = len_bias_s2_cnn;
int len_weight_c3 = len_weight_c3_cnn;
int len_bias_c3 = len_bias_c3_cnn;
int len_weight_s4 = len_weight_s4_cnn;
int len_bias_s4 = len_bias_s4_cnn;
int len_weight_c5 = len_weight_c5_cnn;
int len_bias_c5 = len_bias_c5_cnn;
int len_weight_output = len_weight_output_cnn;
int len_bias_output = len_bias_output_cnn;
int num_neuron_input = num_neuron_input_cnn;
int num_neuron_c1 = num_neuron_c1_cnn;
int num_neuron_s2 = num_neuron_s2_cnn;
int num_neuron_c3 = num_neuron_c3_cnn;
int num_neuron_s4 = num_neuron_s4_cnn;
int num_neuron_c5 = num_neuron_c5_cnn;
int num_neuron_output = num_neuron_output_cnn;
fwrite(&width_image_input, sizeof(int), 1, fp);
fwrite(&height_image_input, sizeof(int), 1, fp);
fwrite(&width_image_c1, sizeof(int), 1, fp);
fwrite(&height_image_c1, sizeof(int), 1, fp);
fwrite(&width_image_s2, sizeof(int), 1, fp);
fwrite(&height_image_s2, sizeof(int), 1, fp);
fwrite(&width_image_c3, sizeof(int), 1, fp);
fwrite(&height_image_c3, sizeof(int), 1, fp);
fwrite(&width_image_s4, sizeof(int), 1, fp);
fwrite(&height_image_s4, sizeof(int), 1, fp);
fwrite(&width_image_c5, sizeof(int), 1, fp);
fwrite(&height_image_c5, sizeof(int), 1, fp);
fwrite(&width_image_output, sizeof(int), 1, fp);
fwrite(&height_image_output, sizeof(int), 1, fp);
fwrite(&width_kernel_conv, sizeof(int), 1, fp);
fwrite(&height_kernel_conv, sizeof(int), 1, fp);
fwrite(&width_kernel_pooling, sizeof(int), 1, fp);
fwrite(&height_kernel_pooling, sizeof(int), 1, fp);
fwrite(&num_map_input, sizeof(int), 1, fp);
fwrite(&num_map_c1, sizeof(int), 1, fp);
fwrite(&num_map_s2, sizeof(int), 1, fp);
fwrite(&num_map_c3, sizeof(int), 1, fp);
fwrite(&num_map_s4, sizeof(int), 1, fp);
fwrite(&num_map_c5, sizeof(int), 1, fp);
fwrite(&num_map_output, sizeof(int), 1, fp);
fwrite(&len_weight_c1, sizeof(int), 1, fp);
fwrite(&len_bias_c1, sizeof(int), 1, fp);
fwrite(&len_weight_s2, sizeof(int), 1, fp);
fwrite(&len_bias_s2, sizeof(int), 1, fp);
fwrite(&len_weight_c3, sizeof(int), 1, fp);
fwrite(&len_bias_c3, sizeof(int), 1, fp);
fwrite(&len_weight_s4, sizeof(int), 1, fp);
fwrite(&len_bias_s4, sizeof(int), 1, fp);
fwrite(&len_weight_c5, sizeof(int), 1, fp);
fwrite(&len_bias_c5, sizeof(int), 1, fp);
fwrite(&len_weight_output, sizeof(int), 1, fp);
fwrite(&len_bias_output, sizeof(int), 1, fp);
fwrite(&num_neuron_input, sizeof(int), 1, fp);
fwrite(&num_neuron_c1, sizeof(int), 1, fp);
fwrite(&num_neuron_s2, sizeof(int), 1, fp);
fwrite(&num_neuron_c3, sizeof(int), 1, fp);
fwrite(&num_neuron_s4, sizeof(int), 1, fp);
fwrite(&num_neuron_c5, sizeof(int), 1, fp);
fwrite(&num_neuron_output, sizeof(int), 1, fp);
fwrite(weight_c1, sizeof(weight_c1), 1, fp);
fwrite(bias_c1, sizeof(bias_c1), 1, fp);
fwrite(weight_s2, sizeof(weight_s2), 1, fp);
fwrite(bias_s2, sizeof(bias_s2), 1, fp);
fwrite(weight_c3, sizeof(weight_c3), 1, fp);
fwrite(bias_c3, sizeof(bias_c3), 1, fp);
fwrite(weight_s4, sizeof(weight_s4), 1, fp);
fwrite(bias_s4, sizeof(bias_s4), 1, fp);
fwrite(weight_c5, sizeof(weight_c5), 1, fp);
fwrite(bias_c5, sizeof(bias_c5), 1, fp);
fwrite(weight_output, sizeof(weight_output), 1, fp);
fwrite(bias_output, sizeof(bias_output), 1, fp);
fflush(fp);
fclose(fp);
return true;
}
double cnn::test()
{
int count_accuracy = 0;
for (int num = 0; num max_value_t) {
max_value_t = data_single_label[i];
pos_t = i;
}
}
if (pos_y == pos_t) {
++count_accuracy;
}
sleep(1);
}
return (count_accuracy * 1.0 / num_patterns_test_cnn);
}
}
测试代码如下：
int test_cnn_train()
{
ann::cnn cnn1;
cnn1.init();
cnn1.train();
return 0;
}
int test_cnn_predict()
{
ann::cnn cnn2;
bool flag = cnn2.readmodelfile(e:/gitcode/nn_test/data/cnn.model);
if (!flag) {
std::cout << read cnn model error << std::endl;
return -1;
}
int width{ 32 }, height{ 32 };
std::vector target{ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 };
std::string image_path{ e:/gitcode/nn_test/data/images/ };
for (auto i : target) {
std::string str = std::to_string(i);
str += .png;
str = image_path + str;
cv::mat src = cv::imread(str, 0);
if (src.data == nullptr) {
fprintf(stderr, read image error: %s\n, str.c_str());
return -1;
}
cv::mat tmp(src.rows, src.cols, cv_8uc1, cv::scalar::all(255));
cv::subtract(tmp, src, tmp);
cv::resize(tmp, tmp, cv::size(width, height));
auto ret = cnn2.predict(tmp.data, width, height);
fprintf(stdout, the actual digit is: %d, correct digit is: %d\n, ret, i);
}
return 0;
}
通过执行test_cnn_train()函数可生成cnn model文件，执行结果如下：
通过执行test_cnn_predict()函数来测试cnn的准确率，通过画图工具，每个数字生成一张图像，共10幅，如下图：
测试结果如下：
MCU业界将打响低功耗MCU新品之战?
化纤生产线轴承室磨损解决方案
能量回馈电抗器基础知识
基于多速率DA的根升余弦滤波器的FPGA实现
一文洞悉Python必备50种算法
卷积神经网络(CNN)的简单介绍及代码实现
PCB设计的成本考虑因素是什么？
2005年沈阳冬季ARM免费培训班
Linux下误删除KVM虚拟机的数据恢复案例
解决安防监控痛点不是口号，需要行动
最高44亿，阳光、三雄、星宇等5份“成绩单”藏着哪些秘密？
网络（服务器）线路的简单介绍
详解时序约束的基本方法
磷铜阳极介绍
浅谈模拟IC设计师眼中的BJT
如何在 Rockchip Arm 开发板上安装 Docker Tailscale K3s Cilium?
LIN总线工作原理介绍
基于云的芯片设计和验证的强大安全性
一图看懂骁龙888如何重新定义顶级AI体验
电动车转换器坏了的表现
cnn是一个多层的神经网络，每层由多个二维平面组成，而每个平面由多个独立神经元组成。
以mnist作为数据库，仿照lenet-5和tiny-cnn( ) 设计一个简单的7层cnn结构如下：
输入层input：神经元数量32*32=1024；
c1层：卷积窗大小5*5，输出特征图数量6，卷积窗种类6，输出特征图大小28*28，可训练参数(权值+阈值(偏置))5*5*6+6=150+6，神经元数量28*28*6=4704；
s2层：卷积窗大小2*2，输出下采样图数量6，卷积窗种类6，输出下采样图大小14*14，可训练参数1*6+6=6+6，神经元数量14*14*6=1176；
c3层：卷积窗大小5*5，输出特征图数量16，卷积窗种类6*16=96，输出特征图大小10*10，可训练参数5*5*(6*16)+16=2400+16，神经元数量10*10*16=1600；
s4层：卷积窗大小2*2，输出下采样图数量16，卷积窗种类16，输出下采样图大小5*5，可训练参数1*16+16=16+16，神经元数量5*5*16=400；
c5层：卷积窗大小5*5，输出特征图数量120，卷积窗种类16*120=1920，输出特征图大小1*1，可训练参数5*5*(16*120)+120=48000+120，神经元数量1*1*120=120；
输出层output：卷积窗大小1*1，输出特征图数量10，卷积窗种类120*10=1200，输出特征图大小1*1，可训练参数1*(120*10)+10=1200+10，神经元数量1*1*10=10。
下面对实现执行过程进行描述说明：
1. 从mnist数据库中分别获取训练样本和测试样本数据：
(1)、原有mnist库中图像大小为28*28，这里缩放为32*32，数据值范围为[-1,1]，扩充值均取-1；总共60000个32*32训练样本，10000个32*32测试样本；
(2)、输出层有10个输出节点，在训练阶段，对应位置的节点值设为0.8，其它节点设为-0.8.
2. 初始化权值和阈值(偏置)：权值就是卷积图像，每一个特征图上的神经元共享相同的权值和阈值，特征图的数量等于阈值的个数
(1)、权值采用uniform rand的方法初始化；
(2)、阈值均初始化为0.
3. 前向传播：根据权值和阈值，主要计算每层神经元的值
(1)、输入层：每次输入一个32*32数据。
(2)、c1层：分别用每一个5*5的卷积图像去乘以32*32的图像，获得一个28*28的图像，即对应位置相加再求和，stride长度为1；一共6个5*5的卷积图像，然后对每一个神经元加上一个阈值，最后再通过tanh激活函数对每一神经元进行运算得到最终每一个神经元的结果。
(3)、s2层：对c1中6个28*28的特征图生成6个14*14的下采样图，相邻四个神经元分别进行相加求和，然后乘以一个权值，再求均值即除以4，然后再加上一个阈值，最后再通过tanh激活函数对每一神经元进行运算得到最终每一个神经元的结果。
(4)、c3层：由s2中的6个14*14下采样图生成16个10*10特征图，对于生成的每一个10*10的特征图，是由6个5*5的卷积图像去乘以6个14*14的下采样图，然后对应位置相加求和，然后对每一个神经元加上一个阈值，最后再通过tanh激活函数对每一神经元进行运算得到最终每一个神经元的结果。
(5)、s4层：由c3中16个10*10的特征图生成16个5*5下采样图，相邻四个神经元分别进行相加求和，然后乘以一个权值，再求均值即除以4，然后再加上一个阈值，最后再通过tanh激活函数对每一神经元进行运算得到最终每一个神经元的结果。
(6)、c5层：由s4中16个5*5下采样图生成120个1*1特征图，对于生成的每一个1*1的特征图，是由16个5*5的卷积图像去乘以16个5*5的下采用图，然后相加求和，然后对每一个神经元加上一个阈值，最后再通过tanh激活函数对每一神经元进行运算得到最终每一个神经元的结果。
(7)、输出层：即全连接层，输出层中的每一个神经元均是由c5层中的120个神经元乘以相对应的权值，然后相加求和；然后对每一个神经元加上一个阈值，最后再通过tanh激活函数对每一神经元进行运算得到最终每一个神经元的结果。
4. 反向传播：主要计算每层神经元、权值和阈值的误差，以用来更新权值和阈值
(1)、输出层：计算输出层神经元误差；通过mse损失函数的导数函数和tanh激活函数的导数函数来计算输出层神经元误差。
(2)、c5层：计算c5层神经元误差、输出层权值误差、输出层阈值误差；通过输出层神经元误差乘以输出层权值，求和，结果再乘以c5层神经元的tanh激活函数的导数，获得c5层每一个神经元误差；通过输出层神经元误差乘以c5层神经元获得输出层权值误差；输出层误差即为输出层阈值误差。
(3)、s4层：计算s4层神经元误差、c5层权值误差、c5层阈值误差；通过c5层权值乘以c5层神经元误差，求和，结果再乘以s4层神经元的tanh激活函数的导数，获得s4层每一个神经元误差；通过s4层神经元乘以c5层神经元误差，求和，获得c5层权值误差；c5层神经元误差即为c5层阈值误差。
(4)、c3层：计算c3层神经元误差、s4层权值误差、s4层阈值误差；
(5)、s2层：计算s2层神经元误差、c3层权值误差、c3层阈值误差；
(6)、c1层：计算c1层神经元误差、s2层权值误差、s2层阈值误差；
(7)、输入层：计算c1层权值误差、c1层阈值误差.
代码文件：
cnn.hpp：
#ifndef _cnn_hpp_
#define _cnn_hpp_
#include
#include
namespace ann {
#define width_image_input_cnn 32 //归一化图像宽
#define height_image_input_cnn 32 //归一化图像高
#define width_image_c1_cnn 28
#define height_image_c1_cnn 28
#define width_image_s2_cnn 14
#define height_image_s2_cnn 14
#define width_image_c3_cnn 10
#define height_image_c3_cnn 10
#define width_image_s4_cnn 5
#define height_image_s4_cnn 5
#define width_image_c5_cnn 1
#define height_image_c5_cnn 1
#define width_image_output_cnn 1
#define height_image_output_cnn 1
#define width_kernel_conv_cnn 5 //卷积核大小
#define height_kernel_conv_cnn 5
#define width_kernel_pooling_cnn 2
#define height_kernel_pooling_cnn 2
#define size_pooling_cnn 2
#define num_map_input_cnn 1 //输入层map个数
#define num_map_c1_cnn 6 //c1层map个数
#define num_map_s2_cnn 6 //s2层map个数
#define num_map_c3_cnn 16 //c3层map个数
#define num_map_s4_cnn 16 //s4层map个数
#define num_map_c5_cnn 120 //c5层map个数
#define num_map_output_cnn 10 //输出层map个数
#define num_patterns_train_cnn 60000 //训练模式对数(总数)
#define num_patterns_test_cnn 10000 //测试模式对数(总数)
#define num_epochs_cnn 100 //最大迭代次数
#define accuracy_rate_cnn 0.985 //要求达到的准确率
#define learning_rate_cnn 0.01 //学习率
#define eps_cnn 1e-8
#define len_weight_c1_cnn 150 //c1层权值数，5*5*6*1=150
#define len_bias_c1_cnn 6 //c1层阈值数，6
#define len_weight_s2_cnn 6 //s2层权值数,1*6=6
#define len_bias_s2_cnn 6 //s2层阈值数,6
#define len_weight_c3_cnn 2400 //c3层权值数，5*5*16*6=2400
#define len_bias_c3_cnn 16 //c3层阈值数,16
#define len_weight_s4_cnn 16 //s4层权值数，1*16=16
#define len_bias_s4_cnn 16 //s4层阈值数，16
#define len_weight_c5_cnn 48000 //c5层权值数，5*5*16*120=48000
#define len_bias_c5_cnn 120 //c5层阈值数，120
#define len_weight_output_cnn 1200 //输出层权值数，120*10=1200
#define len_bias_output_cnn 10 //输出层阈值数，10
#define num_neuron_input_cnn 1024 //输入层神经元数，32*32=1024
#define num_neuron_c1_cnn 4704 //c1层神经元数，28*28*6=4704
#define num_neuron_s2_cnn 1176 //s2层神经元数，14*14*6=1176
#define num_neuron_c3_cnn 1600 //c3层神经元数，10*10*16=1600
#define num_neuron_s4_cnn 400 //s4层神经元数，5*5*16=400
#define num_neuron_c5_cnn 120 //c5层神经元数，1*120=120
#define num_neuron_output_cnn 10 //输出层神经元数，1*10=10
class cnn {
public:
cnn();
~cnn();
void init(); //初始化，分配空间
bool train(); //训练
int predict(const unsigned char* data, int width, int height); //预测
bool readmodelfile(const char* name); //读取已训练好的bp model
protected:
typedef std::vector wi_connections;
typedef std::vector wo_connections;
typedef std::vector io_connections;> 16) & 255;
ch4 = (i >> 24) & 255;
return((int)ch1 << 24) + ((int)ch2 << 16) + ((int)ch3 << 8) + ch4;
}
static void readmnistimages(std::string filename, double* data_dst, int num_image)
{
const int width_src_image = 28;
const int height_src_image = 28;
const int x_padding = 2;
const int y_padding = 2;
const double scale_min = -1;
const double scale_max = 1;
std::ifstream file(filename, std::ios::binary);
assert(file.is_open());
int magic_number = 0;
int number_of_images = 0;
int n_rows = 0;
int n_cols = 0;
file.read((char*)&magic_number, sizeof(magic_number));
magic_number = reverseint(magic_number);
file.read((char*)&number_of_images, sizeof(number_of_images));
number_of_images = reverseint(number_of_images);
assert(number_of_images == num_image);
file.read((char*)&n_rows, sizeof(n_rows));
n_rows = reverseint(n_rows);
file.read((char*)&n_cols, sizeof(n_cols));
n_cols = reverseint(n_cols);
assert(n_rows == height_src_image && n_cols == width_src_image);
int size_single_image = width_image_input_cnn * height_image_input_cnn;
for (int i = 0; i < number_of_images; ++i) {
int addr = size_single_image * i;
for (int r = 0; r < n_rows; ++r) {
for (int c = 0; c < n_cols; ++c) {
unsigned char temp = 0;
file.read((char*)&temp, sizeof(temp));
data_dst[addr + width_image_input_cnn * (r + y_padding) + c + x_padding] = (temp / 255.0) * (scale_max - scale_min) + scale_min;
}
}
}
}
static void readmnistlabels(std::string filename, double* data_dst, int num_image)
{
const double scale_max = 0.8;
std::ifstream file(filename, std::ios::binary);
assert(file.is_open());
int magic_number = 0;
int number_of_images = 0;
file.read((char*)&magic_number, sizeof(magic_number));
magic_number = reverseint(magic_number);
file.read((char*)&number_of_images, sizeof(number_of_images));
number_of_images = reverseint(number_of_images);
assert(number_of_images == num_image);
for (int i = 0; i < number_of_images; ++i) {
unsigned char temp = 0;
file.read((char*)&temp, sizeof(temp));
data_dst[i * num_map_output_cnn + temp] = scale_max;
}
}
bool cnn::getsrcdata()
{
assert(data_input_train && data_output_train && data_input_test && data_output_test);
std::string filename_train_images = e:/gitcode/nn_test/data/train-images.idx3-ubyte;
std::string filename_train_labels = e:/gitcode/nn_test/data/train-labels.idx1-ubyte;
readmnistimages(filename_train_images, data_input_train, num_patterns_train_cnn);
readmnistlabels(filename_train_labels, data_output_train, num_patterns_train_cnn);
std::string filename_test_images = e:/gitcode/nn_test/data/t10k-images.idx3-ubyte;
std::string filename_test_labels = e:/gitcode/nn_test/data/t10k-labels.idx1-ubyte;
readmnistimages(filename_test_images, data_input_test, num_patterns_test_cnn);
readmnistlabels(filename_test_labels, data_output_test, num_patterns_test_cnn);
return true;
}
bool cnn::train()
{
out2wi_s2.clear();
out2bias_s2.clear();
out2wi_s4.clear();
out2bias_s4.clear();
in2wo_c3.clear();
weight2io_c3.clear();
bias2out_c3.clear();
in2wo_c1.clear();
weight2io_c1.clear();
bias2out_c1.clear();
calc_out2wi(width_image_c1_cnn, height_image_c1_cnn, width_image_s2_cnn, height_image_s2_cnn, num_map_s2_cnn, out2wi_s2);
calc_out2bias(width_image_s2_cnn, height_image_s2_cnn, num_map_s2_cnn, out2bias_s2);
calc_out2wi(width_image_c3_cnn, height_image_c3_cnn, width_image_s4_cnn, height_image_s4_cnn, num_map_s4_cnn, out2wi_s4);
calc_out2bias(width_image_s4_cnn, height_image_s4_cnn, num_map_s4_cnn, out2bias_s4);
calc_in2wo(width_image_c3_cnn, height_image_c3_cnn, width_image_s4_cnn, height_image_s4_cnn, num_map_c3_cnn, num_map_s4_cnn, in2wo_c3);
calc_weight2io(width_image_c3_cnn, height_image_c3_cnn, width_image_s4_cnn, height_image_s4_cnn, num_map_c3_cnn, num_map_s4_cnn, weight2io_c3);
calc_bias2out(width_image_c3_cnn, height_image_c3_cnn, width_image_s4_cnn, height_image_s4_cnn, num_map_c3_cnn, num_map_s4_cnn, bias2out_c3);
calc_in2wo(width_image_c1_cnn, height_image_c1_cnn, width_image_s2_cnn, height_image_s2_cnn, num_map_c1_cnn, num_map_c3_cnn, in2wo_c1);
calc_weight2io(width_image_c1_cnn, height_image_c1_cnn, width_image_s2_cnn, height_image_s2_cnn, num_map_c1_cnn, num_map_c3_cnn, weight2io_c1);
calc_bias2out(width_image_c1_cnn, height_image_c1_cnn, width_image_s2_cnn, height_image_s2_cnn, num_map_c1_cnn, num_map_c3_cnn, bias2out_c1);
int iter = 0;
for (iter = 0; iter < num_epochs_cnn; iter++) {
std::cout << epoch: << iter + 1;
for (int i = 0; i < num_patterns_train_cnn; i++) {
data_single_image = data_input_train + i * num_neuron_input_cnn;
data_single_label = data_output_train + i * num_neuron_output_cnn;
forward_c1();
forward_s2();
forward_c3();
forward_s4();
forward_c5();
forward_output();
backward_output();
backward_c5();
backward_s4();
backward_c3();
backward_s2();
backward_c1();
backward_input();
updateweights();
}
double accuracyrate = test();
std::cout << , accuray rate: << accuracyrate < accuracy_rate_cnn) {
savemodelfile(e:/gitcode/nn_test/data/cnn.model);
std::cout << generate cnn model << std::endl;
break;
}
}
if (iter == num_epochs_cnn) {
savemodelfile(e:/gitcode/nn_test/data/cnn.model);
std::cout << generate cnn model << std::endl;
}
return true;
}
double cnn::activation_function_tanh(double x)
{
double ep = std::exp(x);
double em = std::exp(-x);
return (ep - em) / (ep + em);
}
double cnn::activation_function_tanh_derivative(double x)
{
return (1.0 - x * x);
}
double cnn::activation_function_identity(double x)
{
return x;
}
double cnn::activation_function_identity_derivative(double x)
{
return 1;
}
double cnn::loss_function_mse(double y, double t)
{
return (y - t) * (y - t) / 2;
}
double cnn::loss_function_mse_derivative(double y, double t)
{
return (y - t);
}
void cnn::loss_function_gradient(const double* y, const double* t, double* dst, int len)
{
for (int i = 0; i < len; i++) {
dst[i] = loss_function_mse_derivative(y[i], t[i]);
}
}
double cnn::dot_product(const double* s1, const double* s2, int len)
{
double result = 0.0;
for (int i = 0; i < len; i++) {
result += s1[i] * s2[i];
}
return result;
}
bool cnn::muladd(const double* src, double c, int len, double* dst)
{
for (int i = 0; i = 0 && x = 0 && y = 0 && channel < depth);
return (height * channel + y) * width + x;
}
void cnn::calc_out2wi(int width_in, int height_in, int width_out, int height_out, int depth_out, std::vector& out2wi)
{
for (int i = 0; i < depth_out; i++) {
int block = width_in * height_in * i;
for (int y = 0; y < height_out; y++) {
for (int x = 0; x < width_out; x++) {
int rows = y * width_kernel_pooling_cnn;
int cols = x * height_kernel_pooling_cnn;
wi_connections wi_connections_;
std::pair pair_;
for (int m = 0; m < width_kernel_pooling_cnn; m++) {
for (int n = 0; n < height_kernel_pooling_cnn; n++) {
pair_.first = i;
pair_.second = (rows + m) * width_in + cols + n + block;
wi_connections_.push_back(pair_);
}
}
out2wi.push_back(wi_connections_);
}
}
}
}
void cnn::calc_out2bias(int width, int height, int depth, std::vector& out2bias)
{
for (int i = 0; i < depth; i++) {
for (int y = 0; y < height; y++) {
for (int x = 0; x < width; x++) {
out2bias.push_back(i);
}
}
}
}
void cnn::calc_in2wo(int width_in, int height_in, int width_out, int height_out, int depth_in, int depth_out, std::vector& in2wo)
{
int len = width_in * height_in * depth_in;
in2wo.resize(len);
for (int c = 0; c < depth_in; c++) {
for (int y = 0; y < height_in; y += height_kernel_pooling_cnn) {
for (int x = 0; x < width_in; x += width_kernel_pooling_cnn) {
int dymax = min(size_pooling_cnn, height_in - y);
int dxmax = min(size_pooling_cnn, width_in - x);
int dstx = x / width_kernel_pooling_cnn;
int dsty = y / height_kernel_pooling_cnn;
for (int dy = 0; dy < dymax; dy++) {
for (int dx = 0; dx < dxmax; dx++) {
int index_in = get_index(x + dx, y + dy, c, width_in, height_in, depth_in);
int index_out = get_index(dstx, dsty, c, width_out, height_out, depth_out);
wo_connections wo_connections_;
std::pair pair_;
pair_.first = c;
pair_.second = index_out;
wo_connections_.push_back(pair_);
in2wo[index_in] = wo_connections_;
}
}
}
}
}
}
void cnn::calc_weight2io(int width_in, int height_in, int width_out, int height_out, int depth_in, int depth_out, std::vector& weight2io)
{
int len = depth_in;
weight2io.resize(len);
for (int c = 0; c < depth_in; c++) {
for (int y = 0; y < height_in; y += height_kernel_pooling_cnn) {
for (int x = 0; x < width_in; x += width_kernel_pooling_cnn) {
int dymax = min(size_pooling_cnn, height_in - y);
int dxmax = min(size_pooling_cnn, width_in - x);
int dstx = x / width_kernel_pooling_cnn;
int dsty = y / height_kernel_pooling_cnn;
for (int dy = 0; dy < dymax; dy++) {
for (int dx = 0; dx < dxmax; dx++) {
int index_in = get_index(x + dx, y + dy, c, width_in, height_in, depth_in);
int index_out = get_index(dstx, dsty, c, width_out, height_out, depth_out);
std::pair pair_;
pair_.first = index_in;
pair_.second = index_out;
weight2io[c].push_back(pair_);
}
}
}
}
}
}
void cnn::calc_bias2out(int width_in, int height_in, int width_out, int height_out, int depth_in, int depth_out, std::vector& bias2out)
{
int len = depth_in;
bias2out.resize(len); max_value) {
max_value = neuron_output[i];
pos = i;
}
}
return pos;
}
bool cnn::readmodelfile(const char* name)
{
file* fp = fopen(name, rb);
if (fp == null) {
return false;
}
int width_image_input =0;
int height_image_input = 0;
int width_image_c1 = 0;
int height_image_c1 = 0;
int width_image_s2 = 0;
int height_image_s2 = 0;
int width_image_c3 = 0;
int height_image_c3 = 0;
int width_image_s4 = 0;
int height_image_s4 = 0;
int width_image_c5 = 0;
int height_image_c5 = 0;
int width_image_output = 0;
int height_image_output = 0;
int width_kernel_conv = 0;
int height_kernel_conv = 0;
int width_kernel_pooling = 0;
int height_kernel_pooling = 0;
int num_map_input = 0;
int num_map_c1 = 0;
int num_map_s2 = 0;
int num_map_c3 = 0;
int num_map_s4 = 0;
int num_map_c5 = 0;
int num_map_output = 0;
int len_weight_c1 = 0;
int len_bias_c1 = 0;
int len_weight_s2 = 0;
int len_bias_s2 = 0;
int len_weight_c3 = 0;
int len_bias_c3 = 0;
int len_weight_s4 = 0;
int len_bias_s4 = 0;
int len_weight_c5 = 0;
int len_bias_c5 = 0;
int len_weight_output = 0;
int len_bias_output = 0;
int num_neuron_input = 0;
int num_neuron_c1 = 0;
int num_neuron_s2 = 0;
int num_neuron_c3 = 0;
int num_neuron_s4 = 0;
int num_neuron_c5 = 0;
int num_neuron_output = 0;
fread(&width_image_input, sizeof(int), 1, fp);
fread(&height_image_input, sizeof(int), 1, fp);
fread(&width_image_c1, sizeof(int), 1, fp);
fread(&height_image_c1, sizeof(int), 1, fp);
fread(&width_image_s2, sizeof(int), 1, fp);
fread(&height_image_s2, sizeof(int), 1, fp);
fread(&width_image_c3, sizeof(int), 1, fp);
fread(&height_image_c3, sizeof(int), 1, fp);
fread(&width_image_s4, sizeof(int), 1, fp);
fread(&height_image_s4, sizeof(int), 1, fp);
fread(&width_image_c5, sizeof(int), 1, fp);
fread(&height_image_c5, sizeof(int), 1, fp);
fread(&width_image_output, sizeof(int), 1, fp);
fread(&height_image_output, sizeof(int), 1, fp);
fread(&width_kernel_conv, sizeof(int), 1, fp);
fread(&height_kernel_conv, sizeof(int), 1, fp);
fread(&width_kernel_pooling, sizeof(int), 1, fp);
fread(&height_kernel_pooling, sizeof(int), 1, fp);
fread(&num_map_input, sizeof(int), 1, fp);
fread(&num_map_c1, sizeof(int), 1, fp);
fread(&num_map_s2, sizeof(int), 1, fp);
fread(&num_map_c3, sizeof(int), 1, fp);
fread(&num_map_s4, sizeof(int), 1, fp);
fread(&num_map_c5, sizeof(int), 1, fp);
fread(&num_map_output, sizeof(int), 1, fp);
fread(&len_weight_c1, sizeof(int), 1, fp);
fread(&len_bias_c1, sizeof(int), 1, fp);
fread(&len_weight_s2, sizeof(int), 1, fp);
fread(&len_bias_s2, sizeof(int), 1, fp);
fread(&len_weight_c3, sizeof(int), 1, fp);
fread(&len_bias_c3, sizeof(int), 1, fp);
fread(&len_weight_s4, sizeof(int), 1, fp);
fread(&len_bias_s4, sizeof(int), 1, fp);
fread(&len_weight_c5, sizeof(int), 1, fp);
fread(&len_bias_c5, sizeof(int), 1, fp);
fread(&len_weight_output, sizeof(int), 1, fp);
fread(&len_bias_output, sizeof(int), 1, fp);
fread(&num_neuron_input, sizeof(int), 1, fp);
fread(&num_neuron_c1, sizeof(int), 1, fp);
fread(&num_neuron_s2, sizeof(int), 1, fp);
fread(&num_neuron_c3, sizeof(int), 1, fp);
fread(&num_neuron_s4, sizeof(int), 1, fp);
fread(&num_neuron_c5, sizeof(int), 1, fp);
fread(&num_neuron_output, sizeof(int), 1, fp);
fread(weight_c1, sizeof(weight_c1), 1, fp);
fread(bias_c1, sizeof(bias_c1), 1, fp);
fread(weight_s2, sizeof(weight_s2), 1, fp);
fread(bias_s2, sizeof(bias_s2), 1, fp);
fread(weight_c3, sizeof(weight_c3), 1, fp);
fread(bias_c3, sizeof(bias_c3), 1, fp);
fread(weight_s4, sizeof(weight_s4), 1, fp);
fread(bias_s4, sizeof(bias_s4), 1, fp);
fread(weight_c5, sizeof(weight_c5), 1, fp);
fread(bias_c5, sizeof(bias_c5), 1, fp);
fread(weight_output, sizeof(weight_output), 1, fp);
fread(bias_output, sizeof(bias_output), 1, fp);
fflush(fp);
fclose(fp);
out2wi_s2.clear();
out2bias_s2.clear();
out2wi_s4.clear();
out2bias_s4.clear();
calc_out2wi(width_image_c1_cnn, height_image_c1_cnn, width_image_s2_cnn, height_image_s2_cnn, num_map_s2_cnn, out2wi_s2);
calc_out2bias(width_image_s2_cnn, height_image_s2_cnn, num_map_s2_cnn, out2bias_s2);
calc_out2wi(width_image_c3_cnn, height_image_c3_cnn, width_image_s4_cnn, height_image_s4_cnn, num_map_s4_cnn, out2wi_s4);
calc_out2bias(width_image_s4_cnn, height_image_s4_cnn, num_map_s4_cnn, out2bias_s4);
return true;
}
bool cnn::savemodelfile(const char* name)
{
file* fp = fopen(name, wb);
if (fp == null) {
return false;
}
int width_image_input = width_image_input_cnn;
int height_image_input = height_image_input_cnn;
int width_image_c1 = width_image_c1_cnn;
int height_image_c1 = height_image_c1_cnn;
int width_image_s2 = width_image_s2_cnn;
int height_image_s2 = height_image_s2_cnn;
int width_image_c3 = width_image_c3_cnn;
int height_image_c3 = height_image_c3_cnn;
int width_image_s4 = width_image_s4_cnn;
int height_image_s4 = height_image_s4_cnn;
int width_image_c5 = width_image_c5_cnn;
int height_image_c5 = height_image_c5_cnn;
int width_image_output = width_image_output_cnn;
int height_image_output = height_image_output_cnn;
int width_kernel_conv = width_kernel_conv_cnn;
int height_kernel_conv = height_kernel_conv_cnn;
int width_kernel_pooling = width_kernel_pooling_cnn;
int height_kernel_pooling = height_kernel_pooling_cnn;
int num_map_input = num_map_input_cnn;
int num_map_c1 = num_map_c1_cnn;
int num_map_s2 = num_map_s2_cnn;
int num_map_c3 = num_map_c3_cnn;
int num_map_s4 = num_map_s4_cnn;
int num_map_c5 = num_map_c5_cnn;
int num_map_output = num_map_output_cnn;
int len_weight_c1 = len_weight_c1_cnn;
int len_bias_c1 = len_bias_c1_cnn;
int len_weight_s2 = len_weight_s2_cnn;
int len_bias_s2 = len_bias_s2_cnn;
int len_weight_c3 = len_weight_c3_cnn;
int len_bias_c3 = len_bias_c3_cnn;
int len_weight_s4 = len_weight_s4_cnn;
int len_bias_s4 = len_bias_s4_cnn;
int len_weight_c5 = len_weight_c5_cnn;
int len_bias_c5 = len_bias_c5_cnn;
int len_weight_output = len_weight_output_cnn;
int len_bias_output = len_bias_output_cnn;
int num_neuron_input = num_neuron_input_cnn;
int num_neuron_c1 = num_neuron_c1_cnn;
int num_neuron_s2 = num_neuron_s2_cnn;
int num_neuron_c3 = num_neuron_c3_cnn;
int num_neuron_s4 = num_neuron_s4_cnn;
int num_neuron_c5 = num_neuron_c5_cnn;
int num_neuron_output = num_neuron_output_cnn;
fwrite(&width_image_input, sizeof(int), 1, fp);
fwrite(&height_image_input, sizeof(int), 1, fp);
fwrite(&width_image_c1, sizeof(int), 1, fp);
fwrite(&height_image_c1, sizeof(int), 1, fp);
fwrite(&width_image_s2, sizeof(int), 1, fp);
fwrite(&height_image_s2, sizeof(int), 1, fp);
fwrite(&width_image_c3, sizeof(int), 1, fp);
fwrite(&height_image_c3, sizeof(int), 1, fp);
fwrite(&width_image_s4, sizeof(int), 1, fp);
fwrite(&height_image_s4, sizeof(int), 1, fp);
fwrite(&width_image_c5, sizeof(int), 1, fp);
fwrite(&height_image_c5, sizeof(int), 1, fp);
fwrite(&width_image_output, sizeof(int), 1, fp);
fwrite(&height_image_output, sizeof(int), 1, fp);
fwrite(&width_kernel_conv, sizeof(int), 1, fp);
fwrite(&height_kernel_conv, sizeof(int), 1, fp);
fwrite(&width_kernel_pooling, sizeof(int), 1, fp);
fwrite(&height_kernel_pooling, sizeof(int), 1, fp);
fwrite(&num_map_input, sizeof(int), 1, fp);
fwrite(&num_map_c1, sizeof(int), 1, fp);
fwrite(&num_map_s2, sizeof(int), 1, fp);
fwrite(&num_map_c3, sizeof(int), 1, fp);
fwrite(&num_map_s4, sizeof(int), 1, fp);
fwrite(&num_map_c5, sizeof(int), 1, fp);
fwrite(&num_map_output, sizeof(int), 1, fp);
fwrite(&len_weight_c1, sizeof(int), 1, fp);
fwrite(&len_bias_c1, sizeof(int), 1, fp);
fwrite(&len_weight_s2, sizeof(int), 1, fp);
fwrite(&len_bias_s2, sizeof(int), 1, fp);
fwrite(&len_weight_c3, sizeof(int), 1, fp);
fwrite(&len_bias_c3, sizeof(int), 1, fp);
fwrite(&len_weight_s4, sizeof(int), 1, fp);
fwrite(&len_bias_s4, sizeof(int), 1, fp);
fwrite(&len_weight_c5, sizeof(int), 1, fp);
fwrite(&len_bias_c5, sizeof(int), 1, fp);
fwrite(&len_weight_output, sizeof(int), 1, fp);
fwrite(&len_bias_output, sizeof(int), 1, fp);
fwrite(&num_neuron_input, sizeof(int), 1, fp);
fwrite(&num_neuron_c1, sizeof(int), 1, fp);
fwrite(&num_neuron_s2, sizeof(int), 1, fp);
fwrite(&num_neuron_c3, sizeof(int), 1, fp);
fwrite(&num_neuron_s4, sizeof(int), 1, fp);
fwrite(&num_neuron_c5, sizeof(int), 1, fp);
fwrite(&num_neuron_output, sizeof(int), 1, fp);
fwrite(weight_c1, sizeof(weight_c1), 1, fp);
fwrite(bias_c1, sizeof(bias_c1), 1, fp);
fwrite(weight_s2, sizeof(weight_s2), 1, fp);
fwrite(bias_s2, sizeof(bias_s2), 1, fp);
fwrite(weight_c3, sizeof(weight_c3), 1, fp);
fwrite(bias_c3, sizeof(bias_c3), 1, fp);
fwrite(weight_s4, sizeof(weight_s4), 1, fp);
fwrite(bias_s4, sizeof(bias_s4), 1, fp);
fwrite(weight_c5, sizeof(weight_c5), 1, fp);
fwrite(bias_c5, sizeof(bias_c5), 1, fp);
fwrite(weight_output, sizeof(weight_output), 1, fp);
fwrite(bias_output, sizeof(bias_output), 1, fp);
fflush(fp);
fclose(fp);
return true;
}
double cnn::test()
{
int count_accuracy = 0;
for (int num = 0; num max_value_t) {
max_value_t = data_single_label[i];
pos_t = i;
}
}
if (pos_y == pos_t) {
++count_accuracy;
}
sleep(1);
}
return (count_accuracy * 1.0 / num_patterns_test_cnn);
}
}
测试代码如下：
int test_cnn_train()
{
ann::cnn cnn1;
cnn1.init();
cnn1.train();
return 0;
}
int test_cnn_predict()
{
ann::cnn cnn2;
bool flag = cnn2.readmodelfile(e:/gitcode/nn_test/data/cnn.model);
if (!flag) {
std::cout << read cnn model error << std::endl;
return -1;
}
int width{ 32 }, height{ 32 };
std::vector target{ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 };
std::string image_path{ e:/gitcode/nn_test/data/images/ };
for (auto i : target) {
std::string str = std::to_string(i);
str += .png;
str = image_path + str;
cv::mat src = cv::imread(str, 0);
if (src.data == nullptr) {
fprintf(stderr, read image error: %s\n, str.c_str());
return -1;
}
cv::mat tmp(src.rows, src.cols, cv_8uc1, cv::scalar::all(255));
cv::subtract(tmp, src, tmp);
cv::resize(tmp, tmp, cv::size(width, height));
auto ret = cnn2.predict(tmp.data, width, height);
fprintf(stdout, the actual digit is: %d, correct digit is: %d\n, ret, i);
}
return 0;
}
通过执行test_cnn_train()函数可生成cnn model文件，执行结果如下：
通过执行test_cnn_predict()函数来测试cnn的准确率，通过画图工具，每个数字生成一张图像，共10幅，如下图：
测试结果如下：
MCU业界将打响低功耗MCU新品之战?
化纤生产线轴承室磨损解决方案
能量回馈电抗器基础知识
基于多速率DA的根升余弦滤波器的FPGA实现
一文洞悉Python必备50种算法
卷积神经网络(CNN)的简单介绍及代码实现
PCB设计的成本考虑因素是什么？
2005年沈阳冬季ARM免费培训班
Linux下误删除KVM虚拟机的数据恢复案例
解决安防监控痛点不是口号，需要行动
最高44亿，阳光、三雄、星宇等5份“成绩单”藏着哪些秘密？
网络（服务器）线路的简单介绍
详解时序约束的基本方法
磷铜阳极介绍
浅谈模拟IC设计师眼中的BJT
如何在 Rockchip Arm 开发板上安装 Docker Tailscale K3s Cilium?
LIN总线工作原理介绍
基于云的芯片设计和验证的强大安全性
一图看懂骁龙888如何重新定义顶级AI体验
电动车转换器坏了的表现