环境搭建

pytoch|cuda|Python 版本兼容

【Pytorch、torchvision、CUDA 各个版本对应关系以及安装指令】_torch和cuda对应关系-CSDN博客

一文搞懂 PyTorch、CUDA 与 Python 的版本兼容性问题 - 知乎

【环境搭建】Python、PyTorch与cuda的版本对应表_pytorch cuda版本对应关系-CSDN博客

  • 环境版本选定

    python 3.10|cuda 11.8| torch 2.2.2

  • 安装命令

1
conda install pytorch==2.2.0 torchvision==0.17.0 torchaudio==2.2.0 pytorch-cuda=12.1 -c pytorch -c nvidia
  • 验证torch是否可调用显卡
1
2
3
4
5
6
import torch
torch.cuda.is_available()		# 查看CUDA是否可用
print(torch.cuda.device_count()) # 查看可用的CUDA数量
print(torch.version.cuda) 		# 查看CUDA的版本号

nvcc -V # 查看cuda版本

报错原因解决：

1

numpy2.0修改了底层二进制接口，导致旧版Numpy1.x编译的C/C++扩展模块无法在新版中运行，所以通过回退Numpy版本为1.x

1
pip install "numpy<2"

miniconda安装jupyter

  • miniconda轻量环境下是不自带jupyter工具的，需要我们手动安装
1
2
3
conda install jupyter  # 安装 Jupyter Notebook

conda install nb_conda	# 在 Notebook 中图形化切换环境

TensorBoard

TensorFlow 官方提供的可视化工具，用于展示机器学习模型的训练过程和结构。它能帮助你更好地理解模型的训练过程、调试问题，以及对比不同模型的效果。

Transforms

在深度学习和计算机视觉里，**Transforms（变换）**指的就是对原始数据进行预处理或增强的一系列操作，目的是让模型更容易学习、提升泛化能力、减轻过拟合。

eg：

ToTensor、Compose、Resize、Normalize、…

Dataloader

Dataloader 是 PyTorch 中用于高效加载和处理数据的核心工具，主要用于将数据集划分为小批次（batches），以便在训练神经网络时进行迭代处理。

神经网络

torch网络层级文档地址：torch.nn — PyTorch 2.7 documentation

基本骨架

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
from torch import nn
import torch

# 构建神经网络
class Tudui(nn.Module):
    def __init__(self):
        super().__init__()

    def forward(self, input):
        output = input + 1
        return output
    
tudui = Tudui()
x = torch.tensor(1.0)
output = tudui(x)
print(output)

卷积

2

  • In_channel: 输入通道数

  • kernel_size: 卷积核大小

  • stride: 卷积核步进大小

  • padding: 在输入图像外围填充数值

  • bias：对卷积后的结果加减一个常数

  • Out_channel: 输出通道数（即是卷积核数）

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
import torch
import torch.nn.functional as F
input = torch.tensor([[1, 2, 0, 3, 1],
                      [0, 1, 2, 3, 1],
                      [1, 2, 1, 0, 0],
                      [5, 2, 3, 1, 1],
                      [2, 1, 0, 1, 1]])

kernel = torch.tensor([[1, 2, 1],
                       [0, 1, 0],
                       [2, 1, 0]])

input = torch.reshape(input, (1, 1, 5, 5))
kernel = torch.reshape(kernel, (1, 1, 3, 3))

print(input.shape)
print(kernel.shape)

output = F.conv2d(input, kernel, stride=1, padding=1)
print(output)

卷积层

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
import torchvision
from torch import nn
from torch.utils.data import DataLoader
from torch.nn import Conv2d
import torch
from torch.utils.tensorboard import SummaryWriter

dataset = torchvision.datasets.CIFAR10(root="../datasets", train=False, 
                                       transform=torchvision.transforms.ToTensor(), download=True)

dataloader = DataLoader(dataset, batch_size=64)

class Tudui(nn.Module):
    def __init__(self):
        super(Tudui, self).__init__()
        self.conv1 = Conv2d(in_channels=3, out_channels=6, kernel_size=3, stride=1, padding=0)
    
    def forward(self, input):
        output = self.conv1(input)
        return output

tudui = Tudui()

writer = SummaryWriter('../logs')
step = 0
for data in dataloader:
    imgs, targets = data
    output = tudui(imgs)
    # print(imgs.shape)  # [64, 3, 32, 32]
    # print(output.shape)  # [64, 6, 30, 30]
    writer.add_images('input', imgs, step)
   
    output = torch.reshape(output, (-1, 3, 30, 30))
    writer.add_images('output', output, step)
    step = step + 1

writer.close()

池化层

  • kernel_size: 池化窗口大小
  • cell_mode:影响输出特征图的尺寸和边界数据

3

  • 最大池化作用，保留输入特征的同时减少数据量
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
import torch
from torch import nn
from torch.nn import MaxPool2d

input = torch.tensor([[1, 2, 0, 3, 1],
                      [0, 1, 2, 3, 1],
                      [1, 2, 1, 0, 0],
                      [5, 2, 3, 1, 1],
                      [2, 1, 0, 1, 1]], dtype = torch.float32)

input = torch.reshape(input, (-1, 1, 5, 5))
print(input.shape)

class Tudui(nn.Module):
    def __init__(self):
        super(Tudui, self).__init__()
        self.maxpool = nn.MaxPool2d(kernel_size=3, ceil_mode=True)
    
    def forward(self, input):
        output = self.maxpool(input)
        return output

tudui = Tudui()

output = tudui(input)
print(output.shape)
print(output)

非线性激活

**常见激活函数：**relu 、sigmoid

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
import torch
from torch import nn
from torch.nn import ReLU

input = torch.tensor([[1, -0.5],
                      [-1, 3]])

input = torch.reshape(input, (-1, 1, 2, 2))
print(input.shape)

class Tudui(nn.Module):
    def __init__(self):
        super(Tudui, self).__init__()
        self.relu1 = ReLU()

    def forward(self, input):
        output = self.relu1(input)
        return output

tudui = Tudui()
output = tudui(input)
print(output)

正则化层

线性层

“神经网络线性层”通常指的是全连接层（Fully Connected Layer），在 PyTorch 深度学习框架中通常称为 Linear 层。它的本质是一个仿射变换。
$$
\text{output} = W \cdot x + b
$$

  • x：输入向量（或张量）
  • W：权重矩阵（可训练参数）
  • b：偏置向量（可训练参数）
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
import torch
import torchvision
from torch.utils.data import DataLoader
from torch import nn

dataset = torchvision.datasets.CIFAR10(root="./datasets", train=False,
                                       transform=torchvision.transforms.ToTensor(), download=False)

dataloader = DataLoader(dataset, batch_size=64)

class Tudui(nn.Module):
    def __init__(self):
        super(Tudui, self).__init__()
        self.linear = nn.Linear(196608, 10)
    
    def forward(self, input):
        output = self.linear(input)
        return output

tudui = Tudui()

for data in dataloader:
    img, target = data
    print(img.shape)  # [64, 3, 32, 32]
    input = torch.flatten(img)
    print(input.shape)  # [64, 3, 32, 32] -> [64, 3*32*32]
    output = tudui(input)
    print(output.shape)

sequential

cifar-10 分类模型结构

4

Conv2d 各变量参数计算

5

stride : 1

padding: 2

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
from torch import nn
from torch.nn import Sequential, Conv2d, Linear, Flatten, MaxPool2d
import torch
from torch.utils.tensorboard import SummaryWriter

class Tudui(nn.Module):
    def __init__(self):
        super(Tudui, self).__init__()
        self.model1 = Sequential(
            Conv2d(3, 32, 5, stride=1, padding=2),
            MaxPool2d(2),
            Conv2d(32, 32, 5, stride=1, padding=2),
            MaxPool2d(2),
            Conv2d(32, 64, 5, stride=1, padding=2),
            MaxPool2d(2),
            Flatten(),
            Linear(1024, 64),
            Linear(64, 10),
        )
    
    def forward(self, x):
        x = self.model1(x)
        return x
    

tudui = Tudui()

print(tudui)
input = torch.ones((64, 3, 32, 32))
output = tudui(input)
print(output.shape)  # [64, 10]

writer = SummaryWriter("./logs_seq")
writer.add_graph(tudui, input)
writer.close()

6

损失函数(Loss_Function)

  • 计算实际输出核目标之间的差距
  • 为我们更新输出提供一定的依据（反向传播）
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
import torch
from torch import nn

inputs = torch.tensor([1, 2, 3], dtype=torch.float32)
targets = torch.tensor([1, 2, 5], dtype=torch.float32)

loss = nn.L1Loss(reduction="sum")
result = loss(inputs, targets)
print(result)  

loss_mse = nn.MSELoss()
result_mse = loss_mse(inputs, targets)
print(result_mse)

x = torch.tensor([0.1, 0.2, 0.3])
y = torch.tensor([1])
x = torch.reshape(x, (1, 3))
loss_cross = nn.CrossEntropyLoss()
result_cross = loss_cross(x, y)
print(result_cross)

优化器

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
from torch import nn
from torch.nn import Sequential, Conv2d, Linear, Flatten, MaxPool2d
import torchvision
import torch

dataset = torchvision.datasets.CIFAR10(root="./datasets", train=False,
                                        transform=torchvision.transforms.ToTensor() ,download=False)
dataloader = torch.utils.data.DataLoader(dataset, batch_size=64)

class Tudui(nn.Module):
    def __init__(self):
        super(Tudui, self).__init__()
        self.model1 = Sequential(
            Conv2d(3, 32, 5, stride=1, padding=2),
            MaxPool2d(2),
            Conv2d(32, 32, 5, stride=1, padding=2),
            MaxPool2d(2),
            Conv2d(32, 64, 5, stride=1, padding=2),
            MaxPool2d(2),
            Flatten(),
            Linear(1024, 64),
            Linear(64, 10),
        )
    
    def forward(self, x):
        x = self.model1(x)
        return x

tudui = Tudui()
loss = nn.CrossEntropyLoss()
optim = torch.optim.SGD(tudui.parameters(), lr=0.01)
for epoch in range(20):
    running_loss = 0.0
    for data in dataloader:
        imgs, targets = data
        outputs = tudui(imgs)
        result_loss = loss(outputs, targets)
        optim.zero_grad()
        result_loss.backward()
        optim.step()
        running_loss = running_loss + result_loss
    print(running_loss)

模型保存、加载

1
2
3
4
5
6
7
8
9
10
import torchvision
import torch

vgg16 = torchvision.models.vgg16(pretrained=False)

# 保存方式1, 模型结构 + 模型参数
torch.save(vgg16, "./model/vgg16_model1.pth")

# 保存方式2，模型参数（官方推荐）
torch.save(vgg16.state_dict(), "./model/vgg16_model2.pth")
1
2
3
4
5
6
7
8
9
10
11
import torch
import torchvision

# 方式1保存加载模型
model = torch.load("./model/vgg16_model1.pth")
print(model)

# 方式2保存加载模型
vgg16 = torchvision.models.vgg16(pretrained=False)
vgg16.load_state_dict(torch.load("./model/vgg16_model2.pth"))
print(vgg16)
  • 方式1保存加载的时候需要将网络结构复制到加载处（可以不需要定义）

完整模型训练

利用GPU加速训练

方法一

针对

  • 网络模型
  • 数据（输入、标注）
  • 损失函数

在后面调用,cuda()进行加速处理

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
from doctest import OutputChecker
from scipy import optimize
import torch
import torchvision
from torch.utils.data import DataLoader
from torch import nn
from torch.nn import Conv2d, Linear, ReLU, MaxPool2d, Flatten, Dropout
from torch.utils.tensorboard import SummaryWriter
import time

train_data = torchvision.datasets.CIFAR10(root="./Data", train=True, 
                                          download=False, transform=torchvision.transforms.ToTensor())

test_data = torchvision.datasets.CIFAR10(root="./Data", train=False, 
                                          download=False, transform=torchvision.transforms.ToTensor())

# 长度
train_data_size = len(train_data)
test_data_size = len(test_data)
print("训练集数据的长度为:{}", format(train_data_size))
print("测试集数据的长度为:{}", format(test_data_size))

# 利用DataLoader 加载数据集
train_dataloader = DataLoader(dataset=train_data, batch_size=64)
test_dataloader = DataLoader(dataset=test_data, batch_size=64)

# 搭建神经网络
class Tudui(nn.Module):
    def __init__(self):
        super(Tudui, self).__init__()
        self.model1 = nn.Sequential(
            Conv2d(3, 32, kernel_size=5, stride=1, padding=2),  # 卷积层
            MaxPool2d(kernel_size=2),  # 池化层
            Conv2d(32, 32, kernel_size=5, stride=1, padding=2),  # 卷积层
            MaxPool2d(kernel_size=2),  # 池化层
            Conv2d(32, 64, kernel_size=5, stride=1, padding=2),  # 卷积层
            MaxPool2d(kernel_size=2),  # 池化层
            Flatten(),  # 展平层
            Linear(1024, 64),
            Linear(64, 10)
        )
    
    def forward(self, x):
        x = self.model1(x)
        return x

# 模型测试
# tudui = Tudui()
# input = torch.ones((1, 3, 32, 32), dtype=torch.float32)  # 输入数据的形状为(1, 3, 32, 32)
# Output = tudui(input)  # 前向传播
# print(Output.shape)  # 输出数据的形状为(1, 10)

# 创建网络模型
tudui = Tudui()
if torch.cuda.is_available():
    tuidui = tudui.cuda()  # 将模型移动到GPU上

#  创建损失函数
loss_fn = nn.CrossEntropyLoss()
if torch.cuda.is_available():
    loss_fn = loss_fn.cuda()
# 优化器
learning_rate = 0.01
optimizer = torch.optim.SGD(tudui.parameters(), lr=learning_rate)

# 设置训练网络的一些参数
# 记录训练的轮数
total_train_step = 0
# 记录测试的轮数
total_test_step = 0
# 训练的轮数
epoch = 10

writer = SummaryWriter("./logs_train")
start_time = time.time()
for i in range(epoch):
    print("----------第{}轮训练开始----------".format(i + 1))
    for data in train_dataloader:
        imgs, targets = data
        if torch.cuda.is_available():
            imgs = imgs.cuda()
            targets = targets.cuda()
        output = tudui(imgs)
        loss = loss_fn(output, targets)
        # 优化器优化模型
        optimizer.zero_grad()  # 梯度清零
        loss.backward()
        optimizer.step()
        # 
        total_train_step += 1
        if total_train_step % 100 == 0:
            end_time = time.time()
            print(end_time - start_time)
            start_time = time.time()
            print("训练次数：{}, Loss:{}".format(total_train_step, loss.item()))
            writer.add_scalar("train_loss", loss.item(), total_train_step)
    
    # 测试模型
    total_test_loss = 0
    total_accuracy = 0
    with torch.no_grad():
        for data in test_dataloader:
            imgs, targets = data
            if torch.cuda.is_available():
                imgs = imgs.cuda()
                targets = targets.cuda()
            outputs = tudui(imgs)
            loss = loss_fn(outputs, targets)
            total_test_loss += loss.item()
            # 计算准确率
            accuracy = (outputs.argmax(1) == targets).sum()
            total_accuracy += accuracy.item()
    print("整体测试集上的Loss:{}".format(total_test_loss))
    print("整体测试集上的准确率:{}".format(total_accuracy / test_data_size))
    writer.add_scalar("test_loss", total_test_loss, total_test_step)
    writer.add_scalar("test_accuracy", total_accuracy, total_test_step)
    total_test_step += 1

    # 保存模型
    torch.save(tudui, "./model/tudui_{}.pth".format(i + 1))
    print("模型已保存！")

writer.close()

方法二

1
2
3
4
5
device = torch.device("cuda")

tuidui = tudui.to(device)
loss_fn = loss_fn.to(device)
...

模型推理测试

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
import torch
import torchvision
from PIL import Image
from torch import nn
from torch.nn import Conv2d, Linear, MaxPool2d, Flatten

img_pth = "./imgs/dog.png"
img = Image.open(img_pth)
print(img)

transform = torchvision.transforms.Compose([torchvision.transforms.Resize((32, 32)),
                                             torchvision.transforms.ToTensor()])
image = transform(img)
print(image.shape)  # torch.Size([3, 32, 32])


# 搭建神经网络
class Tudui(nn.Module):
    def __init__(self):
        super(Tudui, self).__init__()
        self.model1 = nn.Sequential(
            Conv2d(3, 32, kernel_size=5, stride=1, padding=2),  # 卷积层
            MaxPool2d(kernel_size=2),  # 池化层
            Conv2d(32, 32, kernel_size=5, stride=1, padding=2),  # 卷积层
            MaxPool2d(kernel_size=2),  # 池化层
            Conv2d(32, 64, kernel_size=5, stride=1, padding=2),  # 卷积层
            MaxPool2d(kernel_size=2),  # 池化层
            Flatten(),  # 展平层
            Linear(1024, 64),
            Linear(64, 10)
        )
    
    def forward(self, x):
        x = self.model1(x)
        return x
    
model = torch.load("./model/tudui_gpu10.pth", map_location=torch.device("cpu"))
print(model)
image = torch.reshape(image, (1, 3, 32, 32))
model.eval()  # 设置模型为评估模式
with torch.no_grad():  # 不计算梯度
    output = model(image)
print(output)
print(output.argmax(dim=1))  # 输出预测结果