AI创造营——AnimeGAN视频动漫化一键生成

本文介绍基于PaddleHub实现一键视频动漫化的方法。先升级安装PaddleHub，设置GPU环境，再通过代码将视频提取为图片，选用animegan_v2系列模型（如宫崎骏、新海诚风格）将图片转为动漫风格并合成视频，最后添加原始音频，完成视频动漫化转换。

PaddleHub一键视频动漫化

引子

作为日本动漫的爱好者，特别是对宫崎骏、新海诚的童鞋们有福了。

免费影视、动漫、音乐、游戏、小说资源长期稳定更新！ 👉 点此立即查看 👈

本项目基于 PaddleHub 的动漫风格的转换模型，可以一键将视频转动漫哦！——>{日系风Vlog就在这}。

PaddleHub已将定制打造的街景动漫化模型animegan_v2_hayao_99、animegan_v2_shinkai_53、animegan_v2_paprika_98等多个优质模型开源。可以自行切换转换风格！

效果展示图

动漫化结果

一、安装PaddleHub

#由于PaddleHub升级比较快，建议大家直接升级到最新版本的PaddleHub!pip install --upgrade paddlehub -i https://pypi.tuna.tsinghua.edu.cn/simple

Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simpleRequirement already up-to-date: paddlehub in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (2.0.4)Requirement already satisfied, skipping upgrade: packaging in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (20.9)Requirement already satisfied, skipping upgrade: pyzmq in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (18.1.1)Requirement already satisfied, skipping upgrade: colorama in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (0.4.4)Requirement already satisfied, skipping upgrade: filelock in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (3.0.12)Requirement already satisfied, skipping upgrade: gitpython in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (3.1.14)Requirement already satisfied, skipping upgrade: numpy in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (1.16.4)Requirement already satisfied, skipping upgrade: matplotlib in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (2.2.3)Requirement already satisfied, skipping upgrade: tqdm in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (4.36.1)Requirement already satisfied, skipping upgrade: gunicorn>=19.10.0; sys_platform != "win32" in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (20.0.4)Requirement already satisfied, skipping upgrade: pyyaml in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (5.1.2)Requirement already satisfied, skipping upgrade: Pillow in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (7.1.2)Requirement already satisfied, skipping upgrade: visualdl>=2.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (2.1.1)Requirement already satisfied, skipping upgrade: paddlenlp>=2.0.0rc5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (2.0.0rc7)Requirement already satisfied, skipping upgrade: opencv-python in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (4.1.1.26)Requirement already satisfied, skipping upgrade: colorlog in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (4.1.0)Requirement already satisfied, skipping upgrade: rarfile in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (3.1)Requirement already satisfied, skipping upgrade: easydict in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (1.9)Requirement already satisfied, skipping upgrade: flask>=1.1.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (1.1.1)Requirement already satisfied, skipping upgrade: pyparsing>=2.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from packaging->paddlehub) (2.4.2)Requirement already satisfied, skipping upgrade: gitdb<5,>=4.0.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from gitpython->paddlehub) (4.0.5)Requirement already satisfied, skipping upgrade: cycler>=0.10 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->paddlehub) (0.10.0)Requirement already satisfied, skipping upgrade: pytz in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->paddlehub) (2019.3)Requirement already satisfied, skipping upgrade: python-dateutil>=2.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->paddlehub) (2.8.0)Requirement already satisfied, skipping upgrade: six>=1.10 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->paddlehub) (1.15.0)Requirement already satisfied, skipping upgrade: kiwisolver>=1.0.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->paddlehub) (1.1.0)Requirement already satisfied, skipping upgrade: setuptools>=3.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from gunicorn>=19.10.0; sys_platform != "win32"->paddlehub) (41.4.0)Requirement already satisfied, skipping upgrade: pre-commit in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl>=2.0.0->paddlehub) (1.21.0)Requirement already satisfied, skipping upgrade: Flask-Babel>=1.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl>=2.0.0->paddlehub) (1.0.0)Requirement already satisfied, skipping upgrade: requests in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl>=2.0.0->paddlehub) (2.22.0)Requirement already satisfied, skipping upgrade: bce-python-sdk in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl>=2.0.0->paddlehub) (0.8.53)Requirement already satisfied, skipping upgrade: shellcheck-py in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl>=2.0.0->paddlehub) (0.7.1.1)Requirement already satisfied, skipping upgrade: flake8>=3.7.9 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl>=2.0.0->paddlehub) (3.8.2)Requirement already satisfied, skipping upgrade: protobuf>=3.11.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl>=2.0.0->paddlehub) (3.14.0)Requirement already satisfied, skipping upgrade: jieba in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp>=2.0.0rc5->paddlehub) (0.42.1)Requirement already satisfied, skipping upgrade: seqeval in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp>=2.0.0rc5->paddlehub) (1.2.2)Requirement already satisfied, skipping upgrade: h5py in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp>=2.0.0rc5->paddlehub) (2.9.0)Requirement already satisfied, skipping upgrade: Werkzeug>=0.15 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.0->paddlehub) (0.16.0)Requirement already satisfied, skipping upgrade: click>=5.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.0->paddlehub) (7.0)Requirement already satisfied, skipping upgrade: Jinja2>=2.10.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.0->paddlehub) (2.10.1)Requirement already satisfied, skipping upgrade: itsdangerous>=0.24 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.0->paddlehub) (1.1.0)Requirement already satisfied, skipping upgrade: smmap<4,>=3.0.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from gitdb<5,>=4.0.1->gitpython->paddlehub) (3.0.5)Requirement already satisfied, skipping upgrade: virtualenv>=15.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl>=2.0.0->paddlehub) (16.7.9)Requirement already satisfied, skipping upgrade: identify>=1.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl>=2.0.0->paddlehub) (1.4.10)Requirement already satisfied, skipping upgrade: cfgv>=2.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl>=2.0.0->paddlehub) (2.0.1)Requirement already satisfied, skipping upgrade: importlib-metadata; python_version < "3.8" in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl>=2.0.0->paddlehub) (0.23)Requirement already satisfied, skipping upgrade: aspy.yaml in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl>=2.0.0->paddlehub) (1.3.0)Requirement already satisfied, skipping upgrade: toml in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl>=2.0.0->paddlehub) (0.10.0)Requirement already satisfied, skipping upgrade: nodeenv>=0.11.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl>=2.0.0->paddlehub) (1.3.4)Requirement already satisfied, skipping upgrade: Babel>=2.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from Flask-Babel>=1.0.0->visualdl>=2.0.0->paddlehub) (2.8.0)Requirement already satisfied, skipping upgrade: idna<2.9,>=2.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->visualdl>=2.0.0->paddlehub) (2.8)Requirement already satisfied, skipping upgrade: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->visualdl>=2.0.0->paddlehub) (1.25.6)Requirement already satisfied, skipping upgrade: chardet<3.1.0,>=3.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->visualdl>=2.0.0->paddlehub) (3.0.4)Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->visualdl>=2.0.0->paddlehub) (2019.9.11)Requirement already satisfied, skipping upgrade: pycryptodome>=3.8.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from bce-python-sdk->visualdl>=2.0.0->paddlehub) (3.9.9)Requirement already satisfied, skipping upgrade: future>=0.6.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from bce-python-sdk->visualdl>=2.0.0->paddlehub) (0.18.0)Requirement already satisfied, skipping upgrade: pyflakes<2.3.0,>=2.2.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8>=3.7.9->visualdl>=2.0.0->paddlehub) (2.2.0)Requirement already satisfied, skipping upgrade: pycodestyle<2.7.0,>=2.6.0a1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8>=3.7.9->visualdl>=2.0.0->paddlehub) (2.6.0)Requirement already satisfied, skipping upgrade: mccabe<0.7.0,>=0.6.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8>=3.7.9->visualdl>=2.0.0->paddlehub) (0.6.1)Requirement already satisfied, skipping upgrade: scikit-learn>=0.21.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from seqeval->paddlenlp>=2.0.0rc5->paddlehub) (0.22.1)Requirement already satisfied, skipping upgrade: MarkupSafe>=0.23 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from Jinja2>=2.10.1->flask>=1.1.0->paddlehub) (1.1.1)Requirement already satisfied, skipping upgrade: zipp>=0.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from importlib-metadata; python_version < "3.8"->pre-commit->visualdl>=2.0.0->paddlehub) (0.6.0)Requirement already satisfied, skipping upgrade: joblib>=0.11 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-learn>=0.21.3->seqeval->paddlenlp>=2.0.0rc5->paddlehub) (0.14.1)Requirement already satisfied, skipping upgrade: scipy>=0.17.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-learn>=0.21.3->seqeval->paddlenlp>=2.0.0rc5->paddlehub) (1.3.0)Requirement already satisfied, skipping upgrade: more-itertools in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from zipp>=0.5->importlib-metadata; python_version < "3.8"->pre-commit->visualdl>=2.0.0->paddlehub) (7.2.0)

# 该模型需要gpu跑，所以需要设置下gpu环境%env CUDA_VISIBLE_DEVICES=0%matplotlib inline

env: CUDA_VISIBLE_DEVICES=0

import cv2from PIL import Imageimport numpy as npimport osimport paddlehub as hubfrom moviepy.editor import *from tqdm import tqdm# Config# 原始视频地址original_video_path = './1.mp4'# 提取视频图像的存放地址original_video_img_path = './original_video_img/'# 合成视频存放地址img2video_path = './2.mp4'# 添加声音后的视频最终输出地址output_video_path = './3.mp4'# 从视频提取图片def video2img(video_path, out_path):    cap = cv2.VideoCapture(video_path)    i=1    while True:        ret, frame = cap.read()        if frame is None:            break        else:            cv2.imwrite(out_path + str(i) + ".webp", frame)            i+=1    return# 把图片转动漫并合成视频def ani2video(img_path, org_video_path, out_path, model):    # 获取图片总数    file_list = os.listdir(img_path)    img_num = len(file_list)    # 查看原始视频的参数    cap = cv2.VideoCapture(org_video_path)    ret, frame = cap.read()    # 任选一张图片查看高度和宽度    result = model.style_transfer(images=[cv2.imread(os.path.join(img_path,file_list[0]))])    height = result[0].shape[0]    width = result[0].shape[1]    fps = cap.get(cv2.CAP_PROP_FPS)  # 返回视频的fps--帧率    # 把参数用到我们要创建的视频上    video = cv2.VideoWriter(out_path, cv2.VideoWriter_fourcc('m', 'p', '4', 'v'), fps, (width, height))  # 创建视频流对象    """    参数1 即将保存的文件路径    参数2 VideoWriter_fourcc为视频编解码器 cv2.VideoWriter_fourcc('m', 'p', '4', 'v') 文件名后缀为.mp4    参数3 为帧播放速率    参数4 (width,height)为视频帧大小    """    for i in tqdm(range(img_num)):        f_name = str(i + 1) + '.webp'        item = os.path.join(img_path, f_name)        result = model.style_transfer(images=[cv2.imread(item)]) # 转换动漫风格        video.write(result[0])  # 把图片写进视频    video.release()  # 释放# 从原始视频上提取声音合成到新生成的视频上def sound2video(org_video_path, new_video_path, out_video_path):    # 读取原始视频    video_o = VideoFileClip(org_video_path)    # 获取原始视频的音频部分    audio_o = video_o.audio    # 读取新生成视频    video_clip = VideoFileClip(new_video_path)    # 指向新生成视频的音频部分    video_clip2 = video_clip.set_audio(audio_o)    # 修改音频部分并输出最终视频    video_clip2.write_videofile(out_video_path)

二、从视频提取图像

# 第一步：视频->图像if not os.path.exists(original_video_img_path):    os.mkdir(original_video_img_path)video2img(video_path=original_video_path, out_path=original_video_img_path)

三、图像转换为动漫风格并合成新的视频

# 第二步：转换为动漫效果并合成视频# 根据自己喜好选择风格：## 今敏:'animegan_v2_paprika_98'## 新海诚:'animegan_v2_shinkai_53'## 宫崎骏:'animegan_v2_hayao_99'model = hub.Module(name='animegan_v2_hayao_99', use_gpu=True) ani2video(img_path=original_video_img_path, org_video_path=original_video_path, out_path=img2video_path, model=model)

2024-03-11 09:46:23,753 - INFO - Lock 139930877856336 acquired on /home/aistudio/.paddlehub/tmp/animegan_v2_hayao_99[INFO 2024-03-11 09:46:23,753 filelock.py:274] Lock 139930877856336 acquired on /home/aistudio/.paddlehub/tmp/animegan_v2_hayao_99

Download https://bj.bcebos.com/paddlehub/paddlehub_dev/animegan_v2_hayao_99.tar.gz[##################################################] 100.00%Decompress /home/aistudio/.paddlehub/tmp/tmp3vmh2rme/animegan_v2_hayao_99.tar.gz[##################################################] 100.00%

[2024-03-11 09:46:24,520] [    INFO] - Successfully installed animegan_v2_hayao_99-1.0.12024-03-11 09:46:24,532 - INFO - Lock 139930877856336 released on /home/aistudio/.paddlehub/tmp/animegan_v2_hayao_99[INFO 2024-03-11 09:46:24,532 filelock.py:318] Lock 139930877856336 released on /home/aistudio/.paddlehub/tmp/animegan_v2_hayao_99[2024-03-11 09:46:24,533] [ WARNING] - The _initialize method in HubModule will soon be deprecated, you can use the __init__() to handle the initialization of the object100%|██████████| 2402/2402 [07:05<00:00,  5.65it/s]

四、添加原始音频

# 第三步：加上原始音频if not os.path.exists(output_video_path):    sound2video(org_video_path=original_video_path, new_video_path=img2video_path, out_video_path=output_video_path)else:    print('最终视频已存在，请查看输出路径')

Moviepy - Building video ./3.mp4.MoviePy - Writing audio in 3TEMP_MPY_wvf_snd.mp3

MoviePy - Done.Moviepy - Writing video ./3.mp4

t: 100%|█████████▉| 2398/2403 [00:34<00:00, 90.09it/s, now=None]2024-03-11 09:54:14,198 - WARNING - /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/moviepy/video/io/ffmpeg_reader.py:130: UserWarning: Warning: in file ./2.mp4, 1769472 bytes wanted but 0 bytes read,at frame 2402/2403, at time 80.15/80.15 sec. Using the last valid frame instead.  UserWarning)[WARNING 2024-03-11 09:54:14,198 warnings.py:110] /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/moviepy/video/io/ffmpeg_reader.py:130: UserWarning: Warning: in file ./2.mp4, 1769472 bytes wanted but 0 bytes read,at frame 2402/2403, at time 80.15/80.15 sec. Using the last valid frame instead.  UserWarning)

Moviepy - Done !Moviepy - video ready ./3.mp4