video-cnn-feat repo备忘

本篇博文基于xuchaoxi/video-cnn-feat，可以用来视频分帧及CNN&C3D提取frame-level的特征，仅做备忘。

一、 repo路径

./video-cnn-feat-master

二、 虚拟环境

164服务器-python27

三、 安装过程

1. pip requirments.txt

在conda envs python27中使用pip安装mxnet-cuda90等：

安装后的环境为：

$ conda list
# packages in environment at /home/ubuntu/users/zhanghao/anaconda3/envs/python27:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main    defaults
backports                 1.0                        py_2    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
backports.functools_lru_cache 1.6.1                      py_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
backports_abc             0.5                        py_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
blas                      1.0                         mkl    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
ca-certificates           2020.1.1                      0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
certifi                   2019.11.28               py27_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
cffi                      1.13.2           py27h2e261b9_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
chardet                   3.0.4                    pypi_0    pypi
cudatoolkit               9.0                  h13b8566_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
cudnn                     7.6.5                 cuda9.0_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
cycler                    0.10.0                   py27_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
cython                    0.29.14          py27he6710b0_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
dbus                      1.13.14              hb2f20db_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
expat                     2.2.6                he6710b0_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
fontconfig                2.13.0               h9420a91_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
freetype                  2.9.1                h8a8886c_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
functools32               3.2.3.2                  py27_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
futures                   3.3.0                    py27_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
glib                      2.63.1               h5a9c865_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
gst-plugins-base          1.14.0               hbbd80ab_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
gstreamer                 1.14.0               hb453b48_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
icu                       58.2                 he6710b0_3    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
idna                      2.6                      pypi_0    pypi
intel-openmp              2019.4                      243    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
jpeg                      9b                   h024ee3a_2    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
kiwisolver                1.1.0            py27he6710b0_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libedit                   3.1.20181209         hc058e9b_0    defaults
libffi                    3.2.1                hd88cf55_4    defaults
libgcc-ng                 9.1.0                hdf63c60_0    defaults
libgfortran-ng            7.3.0                hdf63c60_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libpng                    1.6.37               hbc83047_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libprotobuf               3.11.4               h8b12597_0    conda-forge
libstdcxx-ng              9.1.0                hdf63c60_0    defaults
libtiff                   4.1.0                h2733197_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libuuid                   1.0.3                h1bed415_2    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libxcb                    1.13                 h1bed415_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libxml2                   2.9.9                hea5a465_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
matplotlib                2.2.3            py27hb69df0a_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
mkl                       2018.0.3                      1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
mkl_fft                   1.0.6            py27h7dd41cf_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
mkl_random                1.0.1            py27h4414c95_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
mxnet-cu90                1.1.0.post0              pypi_0    pypi
nccl                      1.3.5                 cuda9.0_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
ncurses                   6.2                  he6710b0_0    defaults
ninja                     1.9.0            py27hfd86e86_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
numpy                     1.14.6                   pypi_0    pypi
olefile                   0.46                       py_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
opencv-python             3.4.0.12                 pypi_0    pypi
openssl                   1.1.1g               h7b6447c_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pandas                    0.24.2           py27he6710b0_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pcre                      8.43                 he6710b0_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pillow                    6.2.1            py27h34e0f95_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pip                       19.3.1                   py27_0    defaults
protobuf                  3.11.4           py27he1b5a44_0    conda-forge
pycparser                 2.19                       py_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pyparsing                 2.4.7                      py_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pyqt                      5.9.2            py27h05f1152_2    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
python                    2.7.17               h9bab390_0    defaults
python-dateutil           2.8.1                      py_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
python-graphviz           0.8.4                    pypi_0    pypi
python_abi                2.7                    1_cp27mu    conda-forge
pytorch                   0.4.1            py27ha74772b_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pytz                      2019.3                     py_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
qt                        5.9.7                h5867ecd_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
readline                  7.0                  h7b6447c_5    defaults
requests                  2.18.4                   pypi_0    pypi
scipy                     1.1.0            py27hfa4b5c9_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
setuptools                44.0.0                   py27_0    defaults
singledispatch            3.4.0.3                  py27_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
sip                       4.19.8           py27hf484d3e_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
six                       1.13.0                   py27_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
sqlite                    3.31.1               h7b6447c_0    defaults
subprocess32              3.5.4            py27h7b6447c_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
tbb                       2020.0               hfd86e86_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
tbb4py                    2019.8           py27hfd86e86_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
tensorboardx              1.8                        py_0    conda-forge
tk                        8.6.8                hbc83047_0    defaults
torchvision               0.2.1                    py27_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
tornado                   5.1.1            py27h7b6447c_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
urllib3                   1.22                     pypi_0    pypi
wheel                     0.33.6                   py27_0    defaults
xz                        5.2.4                h14c3975_4    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
zlib                      1.2.11               h7b6447c_3    defaults
zstd                      1.3.7                h0b5b093_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main

2. Download resnet-152 model pre-trained on imagenet-11k

cd ./video-cnn-feat-master
bash do_download_resnet152_11k.sh 

$ bash do_download_resnet152_11k.sh 
--2020-09-01 16:58:42--  http://data.mxnet.io/models/imagenet-11k/resnet-152/resnet-152-symbol.json
正在解析主机 data.mxnet.io (data.mxnet.io)... 172.217.27.147
正在连接 data.mxnet.io (data.mxnet.io)|172.217.27.147|:80... 已连接。
已发出 HTTP 请求，正在等待回应... 301 Moved Permanently
位置：http://data.mxnet.io.s3-website-us-west-1.amazonaws.com/models/imagenet-11k/resnet-152/resnet-152-symbol.json [跟随至新的 URL]
--2020-09-01 16:58:43--  http://data.mxnet.io.s3-website-us-west-1.amazonaws.com/models/imagenet-11k/resnet-152/resnet-152-symbol.json
正在解析主机 data.mxnet.io.s3-website-us-west-1.amazonaws.com (data.mxnet.io.s3-website-us-west-1.amazonaws.com)... 52.219.120.235
正在连接 data.mxnet.io.s3-website-us-west-1.amazonaws.com (data.mxnet.io.s3-website-us-west-1.amazonaws.com)|52.219.120.235|:80... 已连接。
已发出 HTTP 请求，正在等待回应... 200 OK
长度： 224147 (219K) [application/json]
正在保存至: “resnet-152-symbol.json”

resnet-152-symbol.json                 100%[=========================================================================>] 218.89K   234KB/s    用时 0.9s  

2020-09-01 16:58:45 (234 KB/s) - 已保存 “resnet-152-symbol.json” [224147/224147])

--2020-09-01 16:58:45--  http://data.mxnet.io/models/imagenet-11k/resnet-152/resnet-152-0000.params
正在解析主机 data.mxnet.io (data.mxnet.io)... 172.217.27.147
正在连接 data.mxnet.io (data.mxnet.io)|172.217.27.147|:80... 已连接。
已发出 HTTP 请求，正在等待回应... 301 Moved Permanently
位置：http://data.mxnet.io.s3-website-us-west-1.amazonaws.com/models/imagenet-11k/resnet-152/resnet-152-0000.params [跟随至新的 URL]
--2020-09-01 16:58:46--  http://data.mxnet.io.s3-website-us-west-1.amazonaws.com/models/imagenet-11k/resnet-152/resnet-152-0000.params
正在解析主机 data.mxnet.io.s3-website-us-west-1.amazonaws.com (data.mxnet.io.s3-website-us-west-1.amazonaws.com)... 52.219.120.235
正在连接 data.mxnet.io.s3-website-us-west-1.amazonaws.com (data.mxnet.io.s3-website-us-west-1.amazonaws.com)|52.219.120.235|:80... 已连接。
已发出 HTTP 请求，正在等待回应... 200 OK
长度： 325133507 (310M) [binary/octet-stream]
正在保存至: “resnet-152-0000.params”

resnet-152-0000.params                  99%[========================================================================> ] 309.85M   939KB/s    剩余 6s    do_download_resnet152_11k.sh: 行 7:  7441 段错误               (核心已转储) wget http://data.mxnet.io/models/imagenet-11k/resnet-152/resnet-152-0000.params

• Error：段错误
• Solution：经研究发现，resnet-152 model的下载跟/home/ubuntu/users/zhanghao/video-cnn-feat-master路径下的common.ini、constant.py和do_download_resnet152_11k.sh三个文件有关。核心转储的错误跟找不到存储路径有关，故将common.ini中的rootpath修改为rootpath=$/home/ubuntu/users/zhanghao/video-cnn-feat-master/VisualSearch

2020-09-01 20:07:11 (90.8 KB/s) - 已保存 “resnet-152-symbol.json” [224147/224147])

--2020-09-01 20:07:11--  http://data.mxnet.io/models/imagenet-11k/resnet-152/resnet-152-0000.params
正在解析主机 data.mxnet.io (data.mxnet.io)... 172.217.27.147
正在连接 data.mxnet.io (data.mxnet.io)|172.217.27.147|:80... 已连接。
已发出 HTTP 请求，正在等待回应... 301 Moved Permanently
位置：http://data.mxnet.io.s3-website-us-west-1.amazonaws.com/models/imagenet-11k/resnet-152/resnet-152-0000.params [跟随至新的 URL]
--2020-09-01 20:07:12--  http://data.mxnet.io.s3-website-us-west-1.amazonaws.com/models/imagenet-11k/resnet-152/resnet-152-0000.params
正在解析主机 data.mxnet.io.s3-website-us-west-1.amazonaws.com (data.mxnet.io.s3-website-us-west-1.amazonaws.com)... 52.219.116.171
正在连接 data.mxnet.io.s3-website-us-west-1.amazonaws.com (data.mxnet.io.s3-website-us-west-1.amazonaws.com)|52.219.116.171|:80... 已连接。
已发出 HTTP 请求，正在等待回应... 200 OK
长度： 325133507 (310M) [binary/octet-stream]
正在保存至: “resnet-152-0000.params”

resnet-152-0000.params                 100%[=========================================================================>] 310.07M  27.2KB/s    用时 1h 47ms

2020-09-01 21:54:26 (49.3 KB/s) - 已保存 “resnet-152-0000.params” [325133507/325133507])

但是下载后的模型并没有意料之中的存储到home/ubuntu/users/zhanghao/video-cnn-feat-master/VisualSearch而是存储在home/ubuntu/users/zhanghao/video-cnn-feat-master/VisualSearch/$/home/ubuntu/users/zhanghao/video-cnn-feat-master/VisualSearch/mxnet_models/imagenet-11k/resnet-152，目前还没搞明白为什么。

四、 Extract frames from videos

1. 处理数据集

Our code assumes the following data organization. We provide the toydata folder as an example.
collection_name

• VideoData

• ImageData

• id.imagepath.txt
  The toydata folder is assumed to be placed at $HOME/VisualSearch/. Video files are stored in the VideoData folder. Frame files are in the ImageData folder.

• Video filenames shall end with .mp4, .avi, .webm, or .gif.

• Frame filenames shall end with .jpg.

2. 视频分帧

须指明collection的名称，即数据集的名称，例如下示的toydata：

collection=toydata
./do_extract_frames.sh $collection

(1) 若按照前面的rootpath，会报错：

# zhanghao @ Lab164 in ~/video-cnn-feat-master/VisualSearch [8:24:34] 
$ collection=toydata
(python27) 
# zhanghao @ Lab164 in ~/video-cnn-feat-master [8:28:58] 
$ make ./do_extract_frames.sh $collection
make: 对“do_extract_frames.sh”无需做任何事。
make: *** 没有规则可制作目标“toydata”。 停止。

(2) 检查do_extract_frames.sh文件

发现，该文件调用了common.ini、generate_videopath.py和video2frames.py三个文件。

(3) 检查commom.ini文件

发现，os.environ[‘HOME‘] ：

(python27) 
# zhanghao @ Lab164 in ~/video-cnn-feat-master [8:34:09] C:1
$ python
Python 2.7.17 |Anaconda, Inc.| (default, Oct 21 2019, 19:04:46) 
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> print[os.environ[‘HOME‘]]
[‘/home/ubuntu/users/zhanghao‘]

os.environ[‘HOME‘] = /home/ubuntu/users/zhanghao，将common,ini中的rootpath修改为：rootpath=$HOME/VisualSearch/。

(4) 检查constant.py程序

发现，当前的ROOTPATH= os.path.join(os.environ[‘HOME‘], ‘VisualSearch‘) = /home/ubuntu/users/zhanghao/video-cnn-feat-master/VisualSearch。（ROOTPATH与rootpath有区别吗？）

(5) 检查generate_videopath.py程序

发现，

 resultfile = os.path.join(rootpath, collection, "id.videopath.txt")

即在路径/home/ubuntu/users/zhanghao/video-cnn-feat-master/VisualSearch/id.videopath.txt 中自动创建id.videopath.txt文件并存储视频名称及其对应路径 ；

 videoFolders = [os.path.join(rootpath, collection, ‘VideoData‘)]

即在路径/home/ubuntu/users/zhanghao/video-cnn-feat-master/VisualSearch/VideoData 中读取视频数据；

 idfile = os.path.join(rootpath, collection, "VideoSets", ‘%s.txt‘ % collection)

即在路径/home/ubuntu/users/zhanghao/video-cnn-feat-master/VisualSearch/toydata/VideoSets/toydata.txt 中自动创建toydata.txt文件夹并存储已分帧视频的名称。

(6) 检查video2frames.py

发现，该文件的输出有id.videopath.txt、ImageData和id.videometa.txt三部分。

• ImageData

output_dir = os.path.join(rootpath, collection, ‘ImageData‘)

路径为：/home/ubuntu/users/zhanghao/video-cnn-feat-master/VisualSearch/toydata/ImageData，用来存储分帧后的图片。

• id.videometa.txt

video_meta_file = os.path.join(rootpath, collection, ‘id.videometa.txt‘)

其中存储的数据shot00001_1 25 128 1280 720对应为video_id, fps, length, width, height。
(7) 运行分帧程序并查看结果

• 以toydata数据集为例,

cd video-cnn-feat-master
bash ./do_extract_frames.sh toydata

• 运行结果

(python27) 
# zhanghao @ Lab164 in ~/video-cnn-feat-master [11:11:34] 
$ bash ./do_extract_frames.sh toydata
/home/ubuntu/users/zhanghao/video-cnn-feat-master/VisualSearch
rootpath= /home/ubuntu/users/zhanghao/video-cnn-feat-master/VisualSearch/
/home/ubuntu/users/zhanghao/video-cnn-feat-master/VisualSearch/toydata/id.videopath.txt
[02 Sep 11:13:11 - generate_videopath.py:line 26] /home/ubuntu/users/zhanghao/video-cnn-feat-master/VisualSearch/toydata/id.videopath.txt exists. quit
/home/ubuntu/users/zhanghao/video-cnn-feat-master/VisualSearch
[02 Sep 11:13:11 - video2frames.py:line 35] extracting frames from video 0 / 1: shot00001_1
[02 Sep 11:13:11 - video2frames.py:line 81] 1 videos -> 10 frames extracted

五、 Extract frame-level CNN features

./do_resnet152-11k.sh $collection
./do_resnext101.sh $collection

以renet152为backbone为例，调用关系为：do_resnet152-11k.sh→do_deep_feat.sh

注意事项

1. 须指明collection所指代的文件夹名称

2. 须在数据集下一级目录按规定格式创建VideoData和ImageData两个文件夹

3. 原始分帧程序的结果为1S/2帧

视频分帧及特征提取repo——video-cnn-feat备忘

原文：https://www.cnblogs.com/rainbowdream/p/13617132.html