免费算力的使用

普通的贫苦学生,实验室有些老师不给配备显卡,又想做深度学习该怎么办呢?这里有两个办法,虽然有效时间只有八九个小时,但是还是可以跑一个小模型了。

colab

这里就需要大家科学上网了。免费版本分配到的显卡T4居多,16G显存也就不讲究什么卡了。

为了介绍colab的使用,这里以训练yolov5为例。

首先你可以从google云端硬盘上传训练数据集,然后新建一个Colaboratory,然后按下面步骤就可以训练了。


import os
from google.colab import drive
drive.mount('/content/drive')
!pwd
!mkdir /content/data
!cp -r /content/drive/MyDrive/AlexeyAB/chef.tar.gz /content/data/
!tar -xf /content/data/chef.tar.gz -C /content/data/
%cd /content/drive/MyDrive/AlexeyAB/yolov5/
!cat data/chef.yaml
!python3 train.py --img 640 --batch 8 --epochs 100 --data data/chef.yaml --cfg models/yolov5l.yaml --weights yolov5l.pt --resume

需要注意的是,如果从云端硬盘读取数据进行训练会非常的慢,因此需要将谷歌云盘的数据拷贝到运行时。

实际测试过程中,本数据集大概20000万张图片,大概跑5个epoch会断开运行时。需要等待下一次连接。

AIstudio

AIstudio是百度开放的算力平台。

这里同样以ppyoloe为例。首先你需要创建一个项目并加载数据集。


创建完成后,选择进入,就进入你选择的环境进行训练。进入环境后你会发现有些依赖没有安装比如pycocotools,需要进行持久化安装。

由于aistudio只能使用pp,因此这里选择PaddleDetection进行目标检测。在

!tar -xf /home/aistudio/data/data173398/chef.tar.gz -C /home/aistudio/data/data173398
!git clone https://ghproxy.com/https://github.com/PaddlePaddle/PaddleDetection.git
%cd /home/aistudio/work/PaddleDetection/
#!python setup.py install
#!pip install pycocotools
!python3 tools/train.py -c /home/aistudio/work/PaddleDetection/configs/chef/ppyoloe_crn_m_300e_coco.yml

两种大抵差不多,好在百度的还算给力,给了v100进行训练,虽然只有16个小时训练,但一般足够。

[10/26 17:10:37] ppdet.utils.checkpoint INFO: Finish loading model weights: /home/aistudio/.cache/paddle/weights/CSPResNetb_m_pretrained.pdparams
[10/26 17:10:39] ppdet.engine INFO: Epoch: [0] [   0/1508] learning_rate: 0.000000 loss: 4.116579 loss_cls: 0.396140 loss_iou: 0.921533 loss_dfl: 2.833213 loss_l1: 6.263835 eta: 1 day, 6:44:08 batch_cost: 2.4458 data_cost: 0.0003 ips: 3.2709 images/s
[10/26 17:16:21] ppdet.engine INFO: Epoch: [0] [ 100/1508] learning_rate: 0.000046 loss: 4.133281 loss_cls: 0.502849 loss_iou: 0.895340 loss_dfl: 2.814341 loss_l1: 5.764224 eta: 1 day, 17:40:25 batch_cost: 3.3323 data_cost: 3.0634 ips: 2.4007 images/s
[10/26 17:21:52] ppdet.engine INFO: Epoch: [0] [ 200/1508] learning_rate: 0.000093 loss: 4.157372 loss_cls: 0.744968 loss_iou: 0.864395 loss_dfl: 2.547033 loss_l1: 5.012901 eta: 1 day, 17:02:31 batch_cost: 3.2369 data_cost: 2.9800 ips: 2.4715 images/s
[10/26 17:27:37] ppdet.engine INFO: Epoch: [0] [ 300/1508] learning_rate: 0.000139 loss: 4.061717 loss_cls: 1.244310 loss_iou: 0.745664 loss_dfl: 2.119184 loss_l1: 4.060899 eta: 1 day, 17:19:47 batch_cost: 3.3718 data_cost: 3.1134 ips: 2.3726 images/s
[10/26 17:33:05] ppdet.engine INFO: Epoch: [0] [ 400/1508] learning_rate: 0.000186 loss: 4.152278 loss_cls: 1.539355 loss_iou: 0.662454 loss_dfl: 1.861851 loss_l1: 2.879002 eta: 1 day, 16:53:12 batch_cost: 3.1978 data_cost: 2.9465 ips: 2.5017 images/s
[10/26 17:38:53] ppdet.engine INFO: Epoch: [0] [ 500/1508] learning_rate: 0.000232 loss: 3.791449 loss_cls: 1.458757 loss_iou: 0.581376 loss_dfl: 1.727980 loss_l1: 1.940926 eta: 1 day, 17:05:24 batch_cost: 3.4013 data_cost: 3.1425 ips: 2.3520 images/s
[10/26 17:44:07] ppdet.engine INFO: Epoch: [0] [ 600/1508] learning_rate: 0.000279 loss: 3.973853 loss_cls: 1.737436 loss_iou: 0.538747 loss_dfl: 1.575755 loss_l1: 1.661677 eta: 1 day, 16:29:10 batch_cost: 3.0581 data_cost: 2.8017 ips: 2.6160 images/s
[10/26 17:49:28] ppdet.engine INFO: Epoch: [0] [ 700/1508] learning_rate: 0.000325 loss: 3.569711 loss_cls: 1.528158 loss_iou: 0.519620 loss_dfl: 1.510346 loss_l1: 1.424136 eta: 1 day, 16:09:07 batch_cost: 3.1270 data_cost: 2.8759 ips: 2.5583 images/s
[10/26 17:55:15] ppdet.engine INFO: Epoch: [0] [ 800/1508] learning_rate: 0.000371 loss: 3.362413 loss_cls: 1.418495 loss_iou: 0.484577 loss_dfl: 1.417350 loss_l1: 1.228386 eta: 1 day, 16:17:41 batch_cost: 3.3965 data_cost: 3.1410 ips: 2.3554 images/s
[10/26 18:00:49] ppdet.engine INFO: Epoch: [0] [ 900/1508] learning_rate: 0.000418 loss: 3.303343 loss_cls: 1.479825 loss_iou: 0.453707 loss_dfl: 1.354239 loss_l1: 1.088576 eta: 1 day, 16:11:14 batch_cost: 3.2520 data_cost: 2.9919 ips: 2.4600 images/s
[10/26 18:06:36] ppdet.engine INFO: Epoch: [0] [1000/1508] learning_rate: 0.000464 loss: 3.147350 loss_cls: 1.375034 loss_iou: 0.436113 loss_dfl: 1.310673 loss_l1: 0.974823 eta: 1 day, 16:15:13 batch_cost: 3.3907 data_cost: 3.1297 ips: 2.3594 images/s
[10/26 18:12:25] ppdet.engine INFO: Epoch: [0] [1100/1508] learning_rate: 0.000511 loss: 3.029524 loss_cls: 1.370945 loss_iou: 0.413161 loss_dfl: 1.230886 loss_l1: 0.876938 eta: 1 day, 16:18:32 batch_cost: 3.4070 data_cost: 3.1497 ips: 2.3481 images/s
[10/26 18:18:00] ppdet.engine INFO: Epoch: [0] [1200/1508] learning_rate: 0.000557 loss: 2.946838 loss_cls: 1.338580 loss_iou: 0.403557 loss_dfl: 1.208012 loss_l1: 0.801417 eta: 1 day, 16:11:21 batch_cost: 3.2597 data_cost: 2.9846 ips: 2.4542 images/s
[10/26 18:23:38] ppdet.engine INFO: Epoch: [0] [1300/1508] learning_rate: 0.000603 loss: 2.878758 loss_cls: 1.319327 loss_iou: 0.387273 loss_dfl: 1.165951 loss_l1: 0.739521 eta: 1 day, 16:06:23 batch_cost: 3.2944 data_cost: 3.0239 ips: 2.4284 images/s
[10/26 18:29:22] ppdet.engine INFO: Epoch: [0] [1400/1508] learning_rate: 0.000650 loss: 2.851048 loss_cls: 1.317960 loss_iou: 0.378277 loss_dfl: 1.132092 loss_l1: 0.722117 eta: 1 day, 16:04:36 batch_cost: 3.3565 data_cost: 3.0872 ips: 2.3834 images/s
[10/26 18:34:58] ppdet.engine INFO: Epoch: [0] [1500/1508] learning_rate: 0.000696 loss: 2.770674 loss_cls: 1.310223 loss_iou: 0.348698 loss_dfl: 1.107126 loss_l1: 0.719412 eta: 1 day, 15:58:16 batch_cost: 3.2736 data_cost: 3.0112 ips: 2.4438 images/s

一个多小时跑一个epoch,还是挺慢的。

需要注意的是百度一周的使用是有限度的,好像是32个小时,谷歌的只要你能连接上就算你有本事。

展开阅读全文

页面更新:2024-03-10

标签:贫苦   显存   云端   限度   持久   显卡   硬盘   小时   环境   数据

1 2 3 4 5

上滑加载更多 ↓
推荐阅读:
友情链接:
更多:

本站资料均由网友自行发布提供,仅用于学习交流。如有版权问题,请与我联系,QQ:4156828  

© CopyRight 2020-2024 All Rights Reserved. Powered By 71396.com 闽ICP备11008920号-4
闽公网安备35020302034903号

Top