【论文复现】PatchCore: 面向全召回率的工业异常检测

本文基于PaddlePaddle复现PatchCore工业异常检测算法，改进特征提取与筛选，用KNN Greedy CoreSet采样构建记忆池，采用新策略算异常得分。在MVTec数据集精度达标，还介绍了相关流程与复现心得。

PatchCore: Towards Total Recall in Industrial Anomaly Detection

1. 简介

本项目基于PaddlePaddle框架复现了PatchCore算法，并在MvTec数据集上进行了实验。        

免费影视、动漫、音乐、游戏、小说资源长期稳定更新！ 👉 点此立即查看 👈

PatchCore对SPADE,PaDiM等一系列基于图像Patch的无监督异常检测算法工作进行了扩展，主要解决了SPADE测试速度太慢的问题，并且在特征提取部分做了一些探索。相比SPADE,PaDiM，PatchCore 仅使用stage2、stage3的特征图进行建模，通过增加窗口大小为3、步长为1、padding为1的平均池化AvgPool2d增大感受野后拼接，使用KNN Greedy CoreSet 采样选取最具代表性的特征点（选择与其他特征点最远的点以实现尽可能平衡的采样，效果类似泊松圆盘）,构建特征向量记忆池，只保留1%~10%的特征数，进而实现高效的特征筛选并用于异常检测。并提出采用了re-weighting策略计算Image-Level的异常得分代替此前的最大值异常得分。 PatchCore 论文: PatchCore: Towards Total Recall in Industrial Anomaly Detection

参考repo:

在此非常感谢openvineo贡献的anomalib项目，提高了本repo复现论文的效率。

本项目为第六届论文复现赛异常检测项目。

2 复现精度

按复现考核标准，使用resnet18 10%特征数 在MVTec AD数据集的测试效果如下表。

Image-Level AUC

Pixel-Level AUC

达到论文复现验收标准.

3 数据集准备

%cd /home/aistudio/data/!tar xvf data116034/mvtec_anomaly_detection.tar.xz >/dev/null

/home/aistudio/data

4 依赖安装

!pip install scikit-image >/dev/null

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.parl 1.4.1 requires pyzmq==18.1.1, but you have pyzmq 22.3.0 which is incompatible.

5 训练

一共有15个类别，这里需要对15个类别分别训练，最后取平均值作为验证指标.(https://github.com/openvinotoolkit/anomalib/blob/development/anomalib/models/stfpm/README.md).

单个类别训练： 以category=carpet为例。

%cd /home/aistudio/Anomaly.Paddle/!python train.py --category carpet --data_path=/home/aistudio/data/ --method=coreset --arch=resnet18 --k=10 --eval

/home/aistudio/Anomaly.PaddleNamespace(arch='resnet18', batch_size=32, category='carpet', cpu=False, crop_size=256, data_path='/home/aistudio/data/', debug=False, einsum=False, eval=True, eval_PRO=False, eval_threthold_step=500, inc=False, k=10, load_projection=None, method='coreset', non_partial_AUC=False, num_workers=0, resize=256, save_model=True, save_model_subfolder=True, save_path='./output/coreset_resnet18_10', save_pic=True, seed=521, test_batch_size=1)W0505 22:47:52.097200   463 device_context.cc:447] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 10.1, Runtime API Version: 10.1W0505 22:47:52.102414   463 device_context.cc:465] device: 0, cuDNN Version: 7.6.100%|██████████████████████████████████| 69183/69183 [00:04<00:00, 14500.91it/s]model resnet18, nParams 2787264Training model 1/1 for carpet| feature extraction | train | carpet |:   0%|            | 0/9 [00:00登录后复制        
对所有类别进行训练：
In [4]
%cd /home/aistudio/Anomaly.Paddle/!python train.py --category all --data_path=/home/aistudio/data/ --method=coreset --arch=resnet18 --k=10 --eval
登录后复制        
/home/aistudio/Anomaly.PaddleNamespace(arch='resnet18', batch_size=32, category='all', cpu=False, crop_size=256, data_path='/home/aistudio/data/', debug=False, einsum=False, eval=True, eval_PRO=False, eval_threthold_step=500, inc=False, k=10, load_projection=None, method='coreset', non_partial_AUC=False, num_workers=0, resize=256, save_model=True, save_model_subfolder=True, save_path='./output/coreset_resnet18_10', save_pic=True, seed=521, test_batch_size=1)W0505 23:08:38.231904  2381 device_context.cc:447] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 10.1, Runtime API Version: 10.1W0505 23:08:38.237079  2381 device_context.cc:465] device: 0, cuDNN Version: 7.6.^C
登录后复制        
6 测试
单个类别测试： 以category=carpet为例。
In [13]
%cd /home/aistudio/Anomaly.Paddle/!python eval.py --data_path=/home/aistudio/data/ --category carpet --method=coreset --arch=resnet18 --k=10
登录后复制        
/home/aistudio/Anomaly.PaddleNamespace(arch='resnet18', batch_size=1, category='carpet', crop_size=256, data_path='/home/aistudio/data/', eval_PRO=False, eval_threthold_step=500, k=10, method='coreset', model_path=None, non_partial_AUC=False, num_workers=0, resize=256, save_path='./output/coreset_resnet18_10', save_pic=True, seed=521, test_batch_size=1)Testing model 1/1 for carpetW0505 23:14:08.773208  3548 device_context.cc:447] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 10.1, Runtime API Version: 10.1W0505 23:14:08.778220  3548 device_context.cc:465] device: 0, cuDNN Version: 7.6.model resnet18, nParams 27872642024-05-05 23:14:18Starting eval model...| feature extraction | test | carpet |:   0%|           | 0/117 [00:00登录后复制        
对所有类别进行测试： 由于已在训练过程中进行测试此处不再重复
In [6]
#%cd /home/aistudio/Anomaly.Paddle/#!python eval.py --data_path=/home/aistudio/data/ --category all --method=coreset --arch=resnet18 --k=10 --eval_PRO
登录后复制    
7 单独预测
In [7]
%cd /home/aistudio/Anomaly.Paddle/!python predict.py /home/aistudio/data/carpet/test/color/000.webp --category carpet --method=coreset --arch=resnet18 --k=10
登录后复制        
/home/aistudio/Anomaly.Paddle/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/__init__.py:107: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working  from collections import MutableMapping/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/rcsetup.py:20: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working  from collections import Iterable, Mapping/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/colors.py:53: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working  from collections import SizedNamespace(arch='resnet18', category='carpet', crop_size=256, k=10, method='coreset', model_path=None, norm=True, picture_path='/home/aistudio/data/carpet/test/color/000.webp', resize=256, save_path='./output/coreset_resnet18_10', save_pic=True, seed=42, threshold=0.4)Testing model for carpetW0505 23:11:38.219069  2823 device_context.cc:447] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 10.1, Runtime API Version: 10.1W0505 23:11:38.224303  2823 device_context.cc:465] device: 0, cuDNN Version: 7.6.model resnet18, nParams 2787264/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/tensor/creation.py:130: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations  if data.dtype == np.object:2024-05-05 23:11:53Starting eval model.../opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/cbook/__init__.py:2349: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working  if isinstance(obj, collections.Iterator):/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/colors.py:101: DeprecationWarning: np.asscalar(a) is deprecated since NumPy v1.16, use a.item() instead  ret = np.asscalar(ex)/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/cbook/__init__.py:2366: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working  return list(data) if isinstance(data, collections.MappingView) else data/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/image.py:425: DeprecationWarning: np.asscalar(a) is deprecated since NumPy v1.16, use a.item() instead  a_min = np.asscalar(a_min.astype(scaled_dtype))/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/image.py:426: DeprecationWarning: np.asscalar(a) is deprecated since NumPy v1.16, use a.item() instead  a_max = np.asscalar(a_max.astype(scaled_dtype))2024-05-05 23:11:55Predict :  Picture /home/aistudio/data/carpet/test/color/000.webp done!Result saved at ./output/coreset_resnet18_10/carpet/carpet_predict.webp
登录后复制        
可以在output/找到如下的类似结果：
8 导出
In [8]
%cd /home/aistudio/Anomaly.Paddle/!python export_model.py --method=coreset --arch=resnet18 --k=10 --model_path=./output/coreset_resnet18_10/carpet.pdparams --save_dir=./output/coreset_resnet18_10
登录后复制        
/home/aistudio/Anomaly.Paddle/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/__init__.py:107: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working  from collections import MutableMapping/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/rcsetup.py:20: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working  from collections import Iterable, Mapping/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/colors.py:53: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working  from collections import SizedNamespace(arch='resnet18', category='leather', img_size=256, k=10, method='coreset', model_path='./output/coreset_resnet18_10/carpet.pdparams', save_dir='./output/coreset_resnet18_10')W0505 23:11:59.439810  2902 device_context.cc:447] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 10.1, Runtime API Version: 10.1W0505 23:11:59.445221  2902 device_context.cc:465] device: 0, cuDNN Version: 7.6.model resnet18, nParams 2787264/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/utils.py:77: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working  return (isinstance(seq, collections.Sequence) and/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpzp85lm0v.py:21The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.  op_type, op_type, EXPRESSION_MAP[method_name]))/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpxdiplums.py:21The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.  op_type, op_type, EXPRESSION_MAP[method_name]))/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmp22pggkiw.py:21The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.  op_type, op_type, EXPRESSION_MAP[method_name]))/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmps33tfdg1.py:21The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.  op_type, op_type, EXPRESSION_MAP[method_name]))/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpht9nggz5.py:21The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.  op_type, op_type, EXPRESSION_MAP[method_name]))/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmp11b99l2s.py:21The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.  op_type, op_type, EXPRESSION_MAP[method_name]))Model is saved in ./output/coreset_resnet18_10.
登录后复制        
9 推理
In [4]
%cd /home/aistudio/Anomaly.Paddle/!python infer.py --model_name='PatchCore' --enable_post_process --use_gpu=True --model_file=output/coreset_resnet18_10/model.pdmodel --input_file=/home/aistudio/data/carpet/test/color/000.webp --params_file=output/coreset_resnet18_10/model.pdiparams --category=carpet  --stats=./output/coreset_resnet18_10/stats --save_path=./output/coreset_resnet18_10
登录后复制        
/home/aistudio/Anomaly.Paddle/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/setuptools/depends.py:2: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses  import imp/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/__init__.py:107: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working  from collections import MutableMapping/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/rcsetup.py:20: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working  from collections import Iterable, Mapping/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/colors.py:53: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working  from collections import SizedInference model(PatchCore)...load train set feature from: ./output/coreset_resnet18_10/statsE0506 19:06:01.010391  1895 analysis_config.cc:91] Please compile with gpu to EnableGpu()--- Running analysis [ir_graph_build_pass]--- Running analysis [ir_graph_clean_pass]--- Running analysis [ir_analysis_pass]--- Running IR pass [simplify_with_basic_ops_pass]--- Running IR pass [layer_norm_fuse_pass]---    Fused 0 subgraphs into layer_norm op.--- Running IR pass [attention_lstm_fuse_pass]--- Running IR pass [seqconv_eltadd_relu_fuse_pass]--- Running IR pass [seqpool_cvm_concat_fuse_pass]--- Running IR pass [mul_lstm_fuse_pass]--- Running IR pass [fc_gru_fuse_pass]---    fused 0 pairs of fc gru patterns--- Running IR pass [mul_gru_fuse_pass]--- Running IR pass [seq_concat_fc_fuse_pass]--- Running IR pass [squeeze2_matmul_fuse_pass]--- Running IR pass [reshape2_matmul_fuse_pass]--- Running IR pass [flatten2_matmul_fuse_pass]--- Running IR pass [map_matmul_v2_to_mul_pass]--- Running IR pass [map_matmul_v2_to_matmul_pass]--- Running IR pass [map_matmul_to_mul_pass]--- Running IR pass [fc_fuse_pass]--- Running IR pass [repeated_fc_relu_fuse_pass]--- Running IR pass [squared_mat_sub_fuse_pass]--- Running IR pass [conv_bn_fuse_pass]I0506 19:06:01.055578  1895 fuse_pass_base.cc:57] ---  detected 15 subgraphs--- Running IR pass [conv_eltwiseadd_bn_fuse_pass]--- Running IR pass [conv_transpose_bn_fuse_pass]--- Running IR pass [conv_transpose_eltwiseadd_bn_fuse_pass]--- Running IR pass [is_test_pass]--- Running IR pass [runtime_context_cache_pass]--- Running analysis [ir_params_sync_among_devices_pass]--- Running analysis [adjust_cudnn_workspace_size_pass]--- Running analysis [inference_op_replace_pass]--- Running analysis [memory_optimize_pass]I0506 19:06:01.060403  1895 memory_optimize_pass.cc:216] Cluster name : conv2d_31.tmp_0  size: 262144I0506 19:06:01.060425  1895 memory_optimize_pass.cc:216] Cluster name : batch_norm_0.tmp_2  size: 4194304I0506 19:06:01.060428  1895 memory_optimize_pass.cc:216] Cluster name : pool2d_0.tmp_0  size: 1048576I0506 19:06:01.060431  1895 memory_optimize_pass.cc:216] Cluster name : relu_0.tmp_0  size: 4194304I0506 19:06:01.060443  1895 memory_optimize_pass.cc:216] Cluster name : conv2d_22.tmp_0  size: 1048576I0506 19:06:01.060452  1895 memory_optimize_pass.cc:216] Cluster name : x  size: 786432--- Running analysis [ir_graph_to_program_pass]I0506 19:06:01.074777  1895 analysis_predictor.cc:714] ======= optimize end =======I0506 19:06:01.075248  1895 naive_executor.cc:98] ---  skip [feed], feed -> xI0506 19:06:01.076575  1895 naive_executor.cc:98] ---  skip [concat_0.tmp_0], fetch -> fetch/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/tensor/creation.py:130: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations  if data.dtype == np.object:Inference modeimage_score:[2.66626549]/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/cbook/__init__.py:2349: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working  if isinstance(obj, collections.Iterator):/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/colors.py:101: DeprecationWarning: np.asscalar(a) is deprecated since NumPy v1.16, use a.item() instead  ret = np.asscalar(ex)/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/cbook/__init__.py:2366: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working  return list(data) if isinstance(data, collections.MappingView) else data/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/image.py:425: DeprecationWarning: np.asscalar(a) is deprecated since NumPy v1.16, use a.item() instead  a_min = np.asscalar(a_min.astype(scaled_dtype))/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/image.py:426: DeprecationWarning: np.asscalar(a) is deprecated since NumPy v1.16, use a.item() instead  a_max = np.asscalar(a_max.astype(scaled_dtype))saved
登录后复制        
输出结果如下：
10 TIPC
因为test_tipc 不支持后处理的可视化环境，tipc中不进行异常图可视化。
首先安装auto_log，需要进行安装，安装方式如下： auto_log的详细介绍参考https://github.com/LDOUBLEV/AutoLog。
git clone https://github.com/LDOUBLEV/AutoLogcd AutoLog/pip3 install -r requirements.txtpython3 setup.py bdist_wheelpip3 install ./dist/auto_log-1.2.0-py3-none-any.whl
登录后复制    In [2]
#%cd /home/aistudio/#!git clone https://github.com/LDOUBLEV/AutoLog%cd /home/aistudio/AutoLog/!pip3 install -r requirements.txt >/dev/null#!python3 setup.py bdist_wheel!pip3 install ./dist/auto_log-1.2.0-py3-none-any.whl >/dev/null
登录后复制        
/home/aistudio/AutoLog
登录后复制        
进行TIPC：
In [11]
%cd /home/aistudio/Anomaly.Paddle/!bash test_tipc/prepare.sh test_tipc/configs/PatchCore/train_infer_python.txt 'lite_train_lite_infer'
登录后复制        
/home/aistudio/Anomaly.PaddleArchive:  MVTec.zip   creating: MVTec/  inflating: MVTec/.DS_Store            creating: MVTec/capsule/  inflating: MVTec/capsule/.DS_Store     creating: MVTec/capsule/test/   creating: MVTec/capsule/test/crack/  inflating: MVTec/capsule/test/crack/001.webp    inflating: MVTec/capsule/test/crack/000.webp    inflating: MVTec/capsule/test/.DS_Store     creating: MVTec/capsule/test/scratch/  inflating: MVTec/capsule/test/scratch/001.webp    inflating: MVTec/capsule/test/scratch/000.webp    inflating: MVTec/capsule/test/scratch/.DS_Store     creating: MVTec/capsule/test/poke/  inflating: MVTec/capsule/test/poke/001.webp    inflating: MVTec/capsule/test/poke/000.webp    inflating: MVTec/capsule/test/poke/.DS_Store     creating: MVTec/capsule/test/good/  inflating: MVTec/capsule/test/good/001.webp    inflating: MVTec/capsule/test/good/000.webp    inflating: MVTec/capsule/test/good/.DS_Store     creating: MVTec/capsule/test/faulty_imprint/  inflating: MVTec/capsule/test/faulty_imprint/001.webp    inflating: MVTec/capsule/test/faulty_imprint/000.webp    inflating: MVTec/capsule/test/faulty_imprint/.DS_Store     creating: MVTec/capsule/test/squeeze/  inflating: MVTec/capsule/test/squeeze/001.webp    inflating: MVTec/capsule/test/squeeze/000.webp    inflating: MVTec/capsule/test/squeeze/.DS_Store     creating: MVTec/capsule/ground_truth/   creating: MVTec/capsule/ground_truth/crack/  inflating: MVTec/capsule/ground_truth/crack/000_mask.webp    inflating: MVTec/capsule/ground_truth/crack/001_mask.webp    inflating: MVTec/capsule/ground_truth/.DS_Store     creating: MVTec/capsule/ground_truth/scratch/  inflating: MVTec/capsule/ground_truth/scratch/000_mask.webp    inflating: MVTec/capsule/ground_truth/scratch/001_mask.webp     creating: MVTec/capsule/ground_truth/poke/  inflating: MVTec/capsule/ground_truth/poke/000_mask.webp    inflating: MVTec/capsule/ground_truth/poke/001_mask.webp     creating: MVTec/capsule/ground_truth/faulty_imprint/  inflating: MVTec/capsule/ground_truth/faulty_imprint/000_mask.webp    inflating: MVTec/capsule/ground_truth/faulty_imprint/001_mask.webp     creating: MVTec/capsule/ground_truth/squeeze/  inflating: MVTec/capsule/ground_truth/squeeze/000_mask.webp    inflating: MVTec/capsule/ground_truth/squeeze/001_mask.webp     creating: MVTec/capsule/train/   creating: MVTec/capsule/train/good/  inflating: MVTec/capsule/train/good/001.webp    inflating: MVTec/capsule/train/good/000.webp
登录后复制        In [12]
%cd /home/aistudio/Anomaly.Paddle/! bash test_tipc/test_train_inference_python.sh test_tipc/configs/PatchCore/train_infer_python.txt 'lite_train_lite_infer'
登录后复制        
/home/aistudio/Anomaly.PaddleNamespace(arch='resnet18', batch_size=32, category='capsule', cpu=False, crop_size=256, data_path='./test_tipc/data/MVTec', debug=False, einsum=False, eval=False, eval_PRO=False, eval_threthold_step=500, inc=False, k=10, load_projection=None, method='coreset', non_partial_AUC=False, num_workers=0, resize=256, save_model=True, save_model_subfolder=False, save_path='./test_tipc/output/PatchCore/', save_pic=False, seed=521, test_batch_size=1)W0505 23:12:50.398147  3193 device_context.cc:447] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 10.1, Runtime API Version: 10.1W0505 23:12:50.403101  3193 device_context.cc:465] device: 0, cuDNN Version: 7.6.model resnet18, nParams 2787264Training model 1/1 for capsule| feature extraction | train | capsule |:   0%|           | 0/1 [00:00 xI0505 23:13:32.247359  3372 naive_executor.cc:98] ---  skip [concat_0.tmp_0], fetch -> fetchW0505 23:13:32.313050  3372 device_context.cc:447] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 10.1, Runtime API Version: 10.1W0505 23:13:32.317571  3372 device_context.cc:465] device: 0, cuDNN Version: 7.6./opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/__init__.py:107: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working  from collections import MutableMapping/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/rcsetup.py:20: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working  from collections import Iterable, Mapping/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/colors.py:53: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working  from collections import Sized/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/tensor/creation.py:130: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations  if data.dtype == np.object:Inference model(PaDiM)...load train set feature from: ./test_tipc/output/PatchCore/stats Run successfully with command - python3.7 infer.py --stats=./test_tipc/output/PatchCore/stats --use_gpu=True --use_tensorrt=False --precision=fp32 --model_file=./test_tipc/output/PatchCore/model.pdmodel --batch_size=1 --input_file=./test_tipc/data/MVTec/capsule/test/crack/001.webp  --params_file=./test_tipc/output/PatchCore/model.pdiparams > ./test_tipc/output/PatchCore/python_infer_gpu_usetrt_False_precision_fp32_batchsize_1.log 2>&1 !  --- Running analysis [ir_graph_build_pass]--- Running analysis [ir_graph_clean_pass]--- Running analysis [ir_analysis_pass]--- Running IR pass [simplify_with_basic_ops_pass]--- Running IR pass [layer_norm_fuse_pass]---    Fused 0 subgraphs into layer_norm op.--- Running IR pass [attention_lstm_fuse_pass]--- Running IR pass [seqconv_eltadd_relu_fuse_pass]--- Running IR pass [seqpool_cvm_concat_fuse_pass]--- Running IR pass [mul_lstm_fuse_pass]--- Running IR pass [fc_gru_fuse_pass]---    fused 0 pairs of fc gru patterns--- Running IR pass [mul_gru_fuse_pass]--- Running IR pass [seq_concat_fc_fuse_pass]--- Running IR pass [squeeze2_matmul_fuse_pass]--- Running IR pass [reshape2_matmul_fuse_pass]--- Running IR pass [flatten2_matmul_fuse_pass]--- Running IR pass [map_matmul_v2_to_mul_pass]--- Running IR pass [map_matmul_v2_to_matmul_pass]--- Running IR pass [map_matmul_to_mul_pass]--- Running IR pass [fc_fuse_pass]--- Running IR pass [repeated_fc_relu_fuse_pass]--- Running IR pass [squared_mat_sub_fuse_pass]--- Running IR pass [conv_bn_fuse_pass]I0505 23:13:44.782842  3435 fuse_pass_base.cc:57] ---  detected 15 subgraphs--- Running IR pass [conv_eltwiseadd_bn_fuse_pass]--- Running IR pass [conv_transpose_bn_fuse_pass]--- Running IR pass [conv_transpose_eltwiseadd_bn_fuse_pass]--- Running IR pass [is_test_pass]--- Running IR pass [runtime_context_cache_pass]--- Running analysis [ir_params_sync_among_devices_pass]--- Running analysis [adjust_cudnn_workspace_size_pass]--- Running analysis [inference_op_replace_pass]--- Running analysis [memory_optimize_pass]I0505 23:13:44.787526  3435 memory_optimize_pass.cc:216] Cluster name : conv2d_31.tmp_0  size: 262144I0505 23:13:44.787545  3435 memory_optimize_pass.cc:216] Cluster name : batch_norm_0.tmp_2  size: 4194304I0505 23:13:44.787549  3435 memory_optimize_pass.cc:216] Cluster name : pool2d_0.tmp_0  size: 1048576I0505 23:13:44.787554  3435 memory_optimize_pass.cc:216] Cluster name : relu_0.tmp_0  size: 4194304I0505 23:13:44.787556  3435 memory_optimize_pass.cc:216] Cluster name : conv2d_22.tmp_0  size: 1048576I0505 23:13:44.787559  3435 memory_optimize_pass.cc:216] Cluster name : x  size: 786432--- Running analysis [ir_graph_to_program_pass]I0505 23:13:44.802613  3435 analysis_predictor.cc:714] ======= optimize end =======I0505 23:13:44.803148  3435 naive_executor.cc:98] ---  skip [feed], feed -> xI0505 23:13:44.804566  3435 naive_executor.cc:98] ---  skip [concat_0.tmp_0], fetch -> fetchW0505 23:13:44.870177  3435 device_context.cc:447] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 10.1, Runtime API Version: 10.1W0505 23:13:44.874680  3435 device_context.cc:465] device: 0, cuDNN Version: 7.6./opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/__init__.py:107: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working  from collections import MutableMapping/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/rcsetup.py:20: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working  from collections import Iterable, Mapping/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/colors.py:53: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working  from collections import Sized/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/tensor/creation.py:130: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations  if data.dtype == np.object:Inference model(PaDiM)...load train set feature from: ./test_tipc/output/PatchCore/stats Run successfully with command - python3.7 infer.py --stats=./test_tipc/output/PatchCore/stats --use_gpu=False --enable_mkldnn=False --cpu_threads=1 --model_file=./test_tipc/output/PatchCore/model.pdmodel --batch_size=1 --input_file=./test_tipc/data/MVTec/capsule/test/crack/001.webp   --precision=fp32 --params_file=./test_tipc/output/PatchCore/model.pdiparams > ./test_tipc/output/PatchCore/python_infer_cpu_usemkldnn_False_threads_1_precision_fp32_batchsize_1.log 2>&1 !  --- Running analysis [ir_graph_build_pass]--- Running analysis [ir_graph_clean_pass]--- Running analysis [ir_analysis_pass]--- Running IR pass [simplify_with_basic_ops_pass]--- Running IR pass [layer_norm_fuse_pass]---    Fused 0 subgraphs into layer_norm op.--- Running IR pass [attention_lstm_fuse_pass]--- Running IR pass [seqconv_eltadd_relu_fuse_pass]--- Running IR pass [seqpool_cvm_concat_fuse_pass]--- Running IR pass [mul_lstm_fuse_pass]--- Running IR pass [fc_gru_fuse_pass]---    fused 0 pairs of fc gru patterns--- Running IR pass [mul_gru_fuse_pass]--- Running IR pass [seq_concat_fc_fuse_pass]--- Running IR pass [squeeze2_matmul_fuse_pass]--- Running IR pass [reshape2_matmul_fuse_pass]--- Running IR pass [flatten2_matmul_fuse_pass]--- Running IR pass [map_matmul_v2_to_mul_pass]--- Running IR pass [map_matmul_v2_to_matmul_pass]--- Running IR pass [map_matmul_to_mul_pass]--- Running IR pass [fc_fuse_pass]--- Running IR pass [repeated_fc_relu_fuse_pass]--- Running IR pass [squared_mat_sub_fuse_pass]--- Running IR pass [conv_bn_fuse_pass]I0505 23:13:57.568568  3490 fuse_pass_base.cc:57] ---  detected 15 subgraphs--- Running IR pass [conv_eltwiseadd_bn_fuse_pass]--- Running IR pass [conv_transpose_bn_fuse_pass]--- Running IR pass [conv_transpose_eltwiseadd_bn_fuse_pass]--- Running IR pass [is_test_pass]--- Running IR pass [runtime_context_cache_pass]--- Running analysis [ir_params_sync_among_devices_pass]--- Running analysis [adjust_cudnn_workspace_size_pass]--- Running analysis [inference_op_replace_pass]--- Running analysis [memory_optimize_pass]I0505 23:13:57.573712  3490 memory_optimize_pass.cc:216] Cluster name : conv2d_31.tmp_0  size: 262144I0505 23:13:57.573740  3490 memory_optimize_pass.cc:216] Cluster name : batch_norm_0.tmp_2  size: 4194304I0505 23:13:57.573742  3490 memory_optimize_pass.cc:216] Cluster name : pool2d_0.tmp_0  size: 1048576I0505 23:13:57.573746  3490 memory_optimize_pass.cc:216] Cluster name : relu_0.tmp_0  size: 4194304I0505 23:13:57.573750  3490 memory_optimize_pass.cc:216] Cluster name : conv2d_22.tmp_0  size: 1048576I0505 23:13:57.573752  3490 memory_optimize_pass.cc:216] Cluster name : x  size: 786432--- Running analysis [ir_graph_to_program_pass]I0505 23:13:57.589735  3490 analysis_predictor.cc:714] ======= optimize end =======I0505 23:13:57.590353  3490 naive_executor.cc:98] ---  skip [feed], feed -> xI0505 23:13:57.591871  3490 naive_executor.cc:98] ---  skip [concat_0.tmp_0], fetch -> fetchW0505 23:13:57.659296  3490 device_context.cc:447] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 10.1, Runtime API Version: 10.1W0505 23:13:57.663693  3490 device_context.cc:465] device: 0, cuDNN Version: 7.6./opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/__init__.py:107: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working  from collections import MutableMapping/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/rcsetup.py:20: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working  from collections import Iterable, Mapping/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/colors.py:53: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working  from collections import Sized/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/tensor/creation.py:130: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations  if data.dtype == np.object:Inference model(PaDiM)...load train set feature from: ./test_tipc/output/PatchCore/stats Run successfully with command - python3.7 infer.py --stats=./test_tipc/output/PatchCore/stats --use_gpu=False --enable_mkldnn=False --cpu_threads=2 --model_file=./test_tipc/output/PatchCore/model.pdmodel --batch_size=1 --input_file=./test_tipc/data/MVTec/capsule/test/crack/001.webp   --precision=fp32 --params_file=./test_tipc/output/PatchCore/model.pdiparams > ./test_tipc/output/PatchCore/python_infer_cpu_usemkldnn_False_threads_2_precision_fp32_batchsize_1.log 2>&1 !
登录后复制        
11 复现心得
cdist运算
利用 broadcast 和 norm 实现 cdist,缺点是目前的 broadcast 机制会成倍消耗显存 使用条件判断适应2d/3d data，说起来没有atleast_nd这种函数还是不太方便
def cdist(X, Y, p=2.0):    dim = max(len(X.shape), len(Y.shape))    if dim==3:        if len(Y.shape)==2:            Y = Y.unsqueeze(0)        elif len(Y.shape)==1:            Y = Y.unsqueeze(0).unsqueeze(0)        else:            assert len(Y.shape)==3            assert Y.shape[0]=X.shape[0]    #B, P, C = X.shape    #1|B, R, C = Y.shape    D = paddle.linalg.norm(X[:, :, None, :]-Y[None, :, :, :], p, axis=-1)    return D
登录后复制        
减少显存占用的写法：
    def cdist(X, Y, p=2.0):    #2d P, C = X.shape| R, C = Y.shape -> P,R    P, C = X.shape    R, C = Y.shape     #3d B, P, C = X.shape|1, R, C = Y.shape -> B, P,R    #D = paddle.linalg.norm(X[:, None, :]-Y[None, :, :], axis=-1)    D = paddle.zeros((P, R))    for i in range(P):        D[i,:] = paddle.linalg.norm(X[i, None, :]-Y, axis=-1)        #D[i,:] = (X[i, None, :]-Y).square().sum(-1).sqrt_()    return D
登录后复制        
希望paddle Hackathon 能够早点补上原生实现(https://github.com/PaddlePaddle/community/blob/master/rfcs/APIs/20240316_api_design_for_cdist.md)
Avgpool2D 默认表现与pytorch不一致
paddle.nn.AvgPool2D 默认exclusive=True, 与pytorch对应的参数为exclusive=False