知识图谱与多模态融合在药物预测中的应用研究

# 知识图谱与多模态学习在药物预测中的革命性应用：完整教程 本教程将带您深入了解如何将**知识图谱（Knowledge Graph, KG）** 与**多模态学习（Multimodal Learning, MM）** 结合，应用于药物相互作用预测（Drug-Drug Interaction, DDI）。通过一个名为KG4MM（Knowledge Graph for MultiModal）的完整实现，您将学会如何利用图神经网络（GNN）整合分子图像、文本描述和结构化关系，构建一个既准确又具有可解释性的预测系统。 --- ## 目录 1. [引言：为什么需要KG4MM？](#引言为什么需要kg4mm) 2. [方法论：知识图谱如何引导多模态学习](#方法论知识图谱如何引导多模态学习) 3. [实现概览：系统架构与核心组件](#实现概览系统架构与核心组件) 4. [详细实现步骤](#详细实现步骤) - 4.1 环境准备与库安装 - 4.2 数据准备：加载DrugBank样本 - 4.3 分子结构转换：从InChI到SMILES - 4.4 知识图谱构建 - 4.5 多模态数据处理：图像与文本 - 4.6 编码器开发：视觉与文本特征提取 - 4.7 模型整合：KG引导的多模态预测器 - 4.8 知识提取：获取相关子图 - 4.9 批处理数据整理与数据集准备 - 4.10 模型训练 - 4.11 推理与解释 5. [结果：真实案例演示](#结果真实案例演示) 6. [常见问题与解答](#常见问题与解答) 7. [小提示：优化建议](#小提示优化建议) 8. [总结与展望](#总结与展望) --- ## 引言：为什么需要KG4MM？ 知识图谱已经成为表示不同实体间如何相互关联的重要工具。它将信息编码为节点（Node）和边（Edge），直观地展示实体之间的关联。例如，药物与蛋白质的“结合”关系、药物与疾病的“治疗”关系等。 KG4MM（Knowledge Graph for MultiModal）在此基础上进一步发展，它利用知识图谱来指导从图像和文本中学习的过程。在KG4MM中，图谱扮演着一张“地图”的角色，明确标示出在训练过程中需要重点关注的每种数据类型的部分。这种引导有助于系统将注意力集中在图像中最相关的特征和文本中最具信息量的词语上。 在药物相互作用预测领域，KG4MM提供了明显的优势： - **统一整合**：图结构在一个统一框架下整合了药物的分子图像和文本描述。 - **提高准确性**：同时捕捉化学结构和药理背景信息。 - **可解释性**：知识图谱创建了一条从输入到输出的透明路径，使得理解模型为何得出特定预测变得更容易。 本教程将逐步讲解如何构建知识图谱、整合分子和文本信息，并通过具体示例展示KG4MM如何在实际药物相互作用任务中提高预测准确性和可解释性。 --- ## 方法论：知识图谱如何引导多模态学习 KG4MM方法将知识图谱置于整个流程的核心。图谱指导每种数据类型的处理和理解方式。 在药物相互作用示例中，图谱中的每个药物节点关联两种信息： 1. **分子图像**：源自其SMILES分子式。 2. **文本描述**：包含其类别、官能团及其他关键细节。 KG4MM的独特之处在于它利用图神经网络（GNN）来连接图结构和多模态数据。GNN根据药物在图中的位置，确定其图像的哪些部分以及描述中的哪些词语最值得关注。图中的边——显示药物如何与蛋白质、疾病及其他药物相关联——帮助网络确定哪些视觉和文本特征最具重要性。 > **小提示**：知识图谱不仅仅是提供额外背景信息，它更主动地引导模型关注最具信息价值的数据元素。 KG4MM的优势在于能够结合模式识别神经网络与明确的关系图谱。GNN在处理关联数据方面具有显著优势，因此模型能够基于现有的药物相互作用和生化特性知识进行构建。这种受引导的学习不仅提高了预测准确性，还通过突出显示影响每次预测的具体图谱连接，产生了清晰、可解释的结果。 --- ## 实现概览：系统架构与核心组件 该系统围绕一个核心知识图谱构建，该图谱捕获了药物、蛋白质和疾病之间的有向关系，例如： - 药物**binds_to**（结合到）蛋白质 - 药物**inhibits**（抑制）靶点 - 药物**treats**（治疗）疾病 处理的每一步都依赖于这个结构化的医学知识图谱。 **数据准备**：系统将每个药物节点与两种表示形式关联： - **分子图像**：使用RDKit从SMILES分子式生成。 - **文本描述**：概括药物类别、官能团及其他相关细节。 **图谱建模**：使用图卷积网络（GCN）从每个节点的位置及其在图中的连接中学习，创建编码实体间如何相互关联的嵌入。 **多模态编码**：分别使用ResNet处理分子图像、BERT模型转换文本描述。 **融合与预测**：一个图注意力网络（GAT）融合图嵌入与视觉和文本特征，注意力机制利用图结构对来自各模态的最重要特征进行加权，最终预测模块确定两种药物是否会相互作用。同时，注意力权重揭示了哪些图谱连接、图像区域或文本元素对决策贡献最大。 --- ## 详细实现步骤 ### 4.1 环境准备与库安装 首先，确保安装所有必要的深度学习、图处理和化学信息学软件包。 ```python # install necessary packages !pip install torch torchvision transformers networkx spacy rdflib rdkit pillow scikit-learn matplotlib seaborn torch-geometric # pip did not work !apt-get install openbabel !pip install openbabel-wheel ``` ```python # import libraries import torch import torch.nn as nn from torch.utils.data import Dataset, DataLoader import torchvision.models as models import torchvision.transforms as transforms from transformers import BertModel, BertTokenizer import networkx as nx import numpy as np import matplotlib.pyplot as plt import pandas as pd import json import os from rdkit import Chem from rdkit.Chem import Draw from PIL import Image import io import base64 from openbabel import openbabel from torch_geometric.data import Data import torch_geometric.nn as geom_nn ``` > **小提示**：如果您在安装`openbabel`时遇到问题，可以尝试使用`conda install -c openbabel openbabel`代替。另外，确保您的Python版本为3.7+。 --- ### 4.2 数据准备：加载DrugBank样本 创建一个目录用于存放药物图像，然后从公共仓库下载一个简化的DrugBank样本。 ```python # create directory for data storage !mkdir -p data/drug_images # download DrugBank sample data (simplified version for demonstration) !wget -q -O data/drugbank_sample.tsv https://raw.githubusercontent.com/dhimmel/drugbank/gh-pages/data/drugbank-slim.tsv # load DrugBank data drug_df = pd.read_csv('data/drugbank_sample.tsv', sep='\t') ``` 该文件包含每种药物的唯一标识符、名称、用于分子结构的InChI字符串，以及类别和组等描述性元数据。这个结构化数据集为后续步骤中生成视觉和文本表示奠定了基础。 --- ### 4.3 分子结构转换：从InChI到SMILES 分子结构以InChI格式提供，需要转换为SMILES格式（Simplified Molecular Input Line Entry System），以便RDKit能够处理并生成图像。 ```python # create a SMILES column by converting InChI to SMILES def inchi_to_smiles_openbabel(inchi_str): try: # create Open Babel OBMol object from InChI obConversion = openbabel.OBConversion() obConversion.SetInAndOutFormats("inchi", "smiles") mol = openbabel.OBMol() if obConversion.ReadString(mol, inchi_str): return obConversion.WriteString(mol).strip() else: return None except Exception as e: print(f"Error converting InChI to SMILES: {inchi_str}. Error: {e}") return None # apply the conversion to each InChI in the dataframe drug_df['smiles'] = drug_df['inchi'].apply(inchi_to_smiles_openbabel) ``` > **常见问题**：转换失败怎么办？ > 答：某些InChI字符串可能格式不正确或包含特殊字符，导致OpenBabel无法解析。建议检查原始数据或使用其他化学信息学库（如`rdkit.Chem.MolFromInchi`）作为备选方案。 --- ### 4.4 知识图谱构建 系统构建了一个有向医学知识图谱，用于捕获药物、蛋白质和疾病之间的关系。每个节点代表一种实体，每条边编码一种相互作用（如`binds_to`、`inhibits`、`treats`）。 ```python # initialize a medical knowledge graph medical_kg = nx.DiGraph() # extract drug entities from DrugBank (limit to 50 drugs for demo) drug_entities = drug_df['name'].dropna().unique().tolist()[:50] # create drug nodes for drug in drug_entities: medical_kg.add_node(drug, type='drug') # add biomedical entities (proteins, targets, diseases) protein_entities = ["Cytochrome P450", "Albumin", "P-glycoprotein", "GABA Receptor", "Serotonin Receptor", "Beta-Adrenergic Receptor", "ACE", "HMGCR"] disease_entities = ["Hypertension", "Diabetes", "Depression", "Epilepsy", "Asthma", "Rheumatoid Arthritis", "Parkinson's Disease"] for protein in protein_entities: medical_kg.add_node(protein, type='protein') for disease in disease_entities: medical_kg.add_node(disease, type='disease') # add relationships (based on common drug mechanisms and interactions) drug_protein_relations = [ ("Warfarin", "binds_to", "Albumin"), ("Atorvastatin", "inhibits", "HMGCR"), ("Diazepam", "modulates", "GABA Receptor"), ("Fluoxetine", "inhibits", "Serotonin Receptor"), ("Phenytoin", "induces", "Cytochrome P450"), ("Metoprolol", "blocks", "Beta-Adrenergic Receptor"), ("Lisinopril", "inhibits", "ACE"), ("Rifampin", "induces", "P-glycoprotein"), ("Carbamazepine", "induces", "Cytochrome P450"), ("Verapamil", "inhibits", "P-glycoprotein") ] drug_disease_relations = [ ("Lisinopril", "treats", "Hypertension"), ("Metformin", "treats", "Diabetes"), ("Fluoxetine", "treats", "Depression"), ("Phenytoin", "treats", "Epilepsy"), ("Albuterol", "treats", "Asthma"), ("Methotrexate", "treats", "Rheumatoid Arthritis"), ("Levodopa", "treats", "Parkinson's Disease") ] # known drug-drug interactions (based on actual medical knowledge) drug_drug_interactions = [ ("Goserelin", "interacts_with", "Desmopressin", "increases_anticoagulant_effect"), ("Goserelin", "interacts_with", "Cetrorelix", "increases_bleeding_risk"), ("Cyclosporine", "interacts_with", "Felypressin", "decreases_efficacy"), ("Octreotide", "interacts_with", "Cyanocobalamin", "increases_hypoglycemia_risk"), ("Tetrahydrofolic acid", "interacts_with", "L-Histidine", "increases_statin_concentration"), ("S-Adenosylmethionine", "interacts_with", "Pyruvic acid", "decreases_efficacy"), ("L-Phenylalanine", "interacts_with", "Biotin", "increases_sedation"), ("Choline", "interacts_with", "L-Lysine", "decreases_efficacy") ] # add all relationships to the knowledge graph for s, r, o in drug_protein_relations: if s in medical_kg and o in medical_kg: medical_kg.add_edge(s, o, relation=r) for s, r, o in drug_disease_relations: if s in medical_kg and o in medical_kg: medical_kg.add_edge(s, o, relation=r) for s, r, o, mechanism in drug_drug_interactions: if s in medical_kg and o in medical_kg: medical_kg.add_edge(s, o, relation=r, mechanism=mechanism) ``` 该图谱充当了一个结构化的关系信息来源，模型在处理图像和文本特征的同时会利用这些信息。通过明确地表示领域知识，图谱增强了预测准确性以及解释两种药物为何可能相互作用的能力。 --- ### 4.5 多模态数据处理：图像与文本 每种药物由三种互补的数据类型表示。 **步骤1：生成分子图像** ```python # function to generate molecular structure images using RDKit def generate_molecule_image(smiles_string, size=(224, 224)): try: mol = Chem.MolFromSmiles(smiles_string) if mol: img = Draw.MolToImage(mol, size=size) return img else: return None except: return None ``` **步骤2：构建文本描述** ```python # function to create text description for drugs combining various information def create_drug_description(row): description = f"Drug name: {row['name']}. " if pd.notna(row.get('category')): description += f"Category: {row['category']}. " if pd.notna(row.get('groups')): description += f"Groups: {row['groups']}. " if pd.notna(row.get('description')): description += f"Description: {row['description']}" return description ``` **步骤3：图谱嵌入** 将NetworkX图转换为PyG图，用于现代图神经网络处理。 ```python # convert NetworkX graph to PyG graph for modern graph neural network processing def convert_nx_to_pyg(nx_graph): node_to_idx = {node: i for i, node in enumerate(nx_graph.nodes())} src_nodes = [] dst_nodes = [] edge_types = [] edge_type_to_idx = {} for u, v, data in nx_graph.edges(data=True): relation = data.get('relation', 'unknown') if relation not in edge_type_to_idx: edge_type_to_idx[relation] = len(edge_type_to_idx) src_nodes.append(node_to_idx[u]) dst_nodes.append(node_to_idx[v]) edge_types.append(edge_type_to_idx[relation]) edge_index = torch.tensor([src_nodes, dst_nodes], dtype=torch.long) edge_type = torch.tensor(edge_types, dtype=torch.long) node_types = [] for node in nx_graph.nodes(): node_type = nx_graph.nodes[node].get('type', 'unknown') node_types.append(node_type) unique_node_types = sorted(set(node_types)) node_type_to_idx = {nt: i for i, nt in enumerate(unique_node_types)} node_type_features = torch.zeros(len(node_types), len(unique_node_types)) for i, nt in enumerate(node_types): node_type_features[i, node_type_to_idx[nt]] = 1.0 g = Data( edge_index=edge_index, edge_type=edge_type, x=node_type_features ) idx_to_node = {idx: node for node, idx in node_to_idx.items()} idx_to_edge_type = {idx: edge_type for edge_type, idx in edge_type_to_idx.items()} return g, node_to_idx, idx_to_node, edge_type_to_idx, idx_to_edge_type pyg_graph, node_to_idx, idx_to_node, edge_type_to_idx, idx_to_edge_type = convert_nx_to_pyg(medical_kg) ``` **步骤4：处理并保存所有药物数据** ```python # process drug data to create multi-modal representations drug_data = [] for idx, row in drug_df.iterrows(): if row['name'] in drug_entities and pd.notna(row.get('smiles')): img = generate_molecule_image(row['smiles']) if img: img_path = f"data/drug_images/{row['drugbank_id']}.png" img.sa ve(img_path) description = create_drug_description(row) drug_data.append({ 'id': row['drugbank_id'], 'name': row['name'], 'smiles': row['smiles'], 'description': description, 'image_path': img_path }) drug_data_df = pd.DataFrame(drug_data) ``` --- ### 4.6 编码器开发：视觉与文本特征提取 `MultimodalNodeEncoder`创建了一个单一编码器，将每个节点的分子图像及其文本摘要转换为兼容的特征向量。 ```python # processes visual and textual features for nodes class MultimodalNodeEncoder(nn.Module): def __init__(self, output_dim=128): super(MultimodalNodeEncoder, self).__init__() # image encoder (ResNet) resnet = models.resnet18(pretrained=True) self.image_encoder = nn.Sequential(*list(resnet.children())[:-1]) self.image_projection = nn.Linear(512, output_dim) # text encoder (BERT) self.tokenizer = BertTokenizer.from_pretrained('bert-base-uncased') self.text_encoder = BertModel.from_pretrained('bert-base-uncased') self.text_projection = nn.Linear(768, output_dim) def forward(self, image, text): # image encoding img_features = self.image_encoder(image).squeeze(-1).squeeze(-1) img_features = self.image_projection(img_features) # text encoding encoded_input = self.tokenizer(text, padding=True, truncation=True, return_tensors="pt", max_length=128) input_ids = encoded_input['input_ids'].to(image.device) attention_mask = encoded_input['attention_mask'].to(image.device) text_outputs = self.text_encoder(input_ids=input_ids, attention_mask=attention_mask) text_features = text_outputs.last_hidden_state[:, 0, :] text_features = self.text_projection(text_features) return img_features, text_features ``` > **小提示**：`output_dim=128`是可调整的超参数。增大该值可以增加模型容量，但也可能带来过拟合风险。建议根据数据集大小进行调优。 --- ### 4.7 模型整合：KG引导的多模态预测器 `KGGuidedMultimodalModel`融合每个节点的视觉、文本和类型嵌入，并在知识图谱的指导下预测药物-药物相互作用。 ```python # define KG-guided MultimodalModel class KGGuidedMultimodalModel(nn.Module): def __init__(self, pyg_graph, num_node_types, num_edge_types, node_to_idx, idx_to_node, hidden_dim=128): super(KGGuidedMultimodalModel, self).__init__() self.pyg_graph = pyg_graph self.node_to_idx = node_to_idx self.idx_to_node = idx_to_node self.hidden_dim = hidden_dim self.multimodal_encoder = MultimodalNodeEncoder(output_dim=hidden_dim) self.node_type_embedding = nn.Embedding(num_node_types, hidden_dim) self.gnn_layers = nn.ModuleList([ geom_nn.GCNConv(hidden_dim, hidden_dim), geom_nn.GCNConv(hidden_dim, hidden_dim), ]) self.gat_layer = geom_nn.GATConv(hidden_dim, hidden_dim // 4, heads=4) self.relation_prediction = nn.Sequential( nn.Linear(hidden_dim * 4, hidden_dim * 2), nn.ReLU(), nn.Dropout(0.3), nn.Linear(hidden_dim * 2, hidden_dim), nn.ReLU(), nn.Dropout(0.3), nn.Linear(hidden_dim, 1) ) def get_node_representation(self, node_name, image=None, text=None): if node_name not in self.node_to_idx: return torch.zeros(self.hidden_dim, device=self.pyg_graph.edge_index.device) node_idx = self.node_to_idx[node_name] node_type_feat = self.pyg_graph.x[node_idx] node_type_embedding = self.node_type_embedding(torch.argmax(node_type_feat)) if image is not None and text is not None: img_feat, text_feat = self.multimodal_encoder(image, text) img_feat = img_feat.squeeze(0) text_feat = text_feat.squeeze(0) attention_weights = torch.softmax( torch.matmul( torch.stack([img_feat, text_feat, node_type_embedding]), node_type_embedding ), dim=0 ) combined_feat = ( attention_weights[0] * img_feat + attention_weights[1] * text_feat + attention_weights[2] * node_type_embedding ) return combined_feat else: return node_type_embedding def forward(self, drug1_image, drug1_text, drug1_name, drug2_image, drug2_text, drug2_name): device = self.pyg_graph.edge_index.device x = torch.zeros((self.pyg_graph.x.size(0), self.hidden_dim), device=device) for i, node_name in enumerate([drug1_name, drug2_name]): if node_name in self.node_to_idx: node_idx = self.node_to_idx[node_name] if i == 0: x[node_idx] = self.get_node_representation(node_name, drug1_image, drug1_text) else: x[node_idx] = self.get_node_representation(node_name, drug2_image, drug2_text) edge_index = self.pyg_graph.edge_index for layer in self.gnn_layers: x = layer(x, edge_index) x = torch.relu(x) x = self.gat_layer(x, edge_index) drug1_idx = self.node_to_idx.get(drug1_name, 0) drug2_idx = self.node_to_idx.get(drug2_name, 0) drug1_repr = x[drug1_idx] drug2_repr = x[drug2_idx] concat_repr = torch.cat([ drug1_repr, drug2_repr, drug1_repr * drug2_repr, torch.abs(drug1_repr - drug2_repr) ], dim=0) interaction_prob = torch.sigmoid(self.relation_prediction(concat_repr.unsqueeze(0)).squeeze()) return interaction_prob ``` 模型的核心思想：让图谱的拓扑决定多模态信号如何融合，生成的预测既准确又可直接追溯到潜在的网络结构。 --- ### 4.8 知识提取：获取相关子图 `retrieve_knowledge_subgraph`函数构建一个焦点子图，包含直接边、共享节点以及简单路径，用于解释预测结果。 ```python # function to retrieve knowledge subgraph relevant to a drug pair def retrieve_knowledge_subgraph(graph, drug1, drug2, max_path_length=3): relevant_knowledge = { 'direct_interaction': None, 'common_targets': [], 'paths': [] } # check for direct interaction if graph.has_edge(drug1, drug2): edge_data = graph.get_edge_data(drug1, drug2) relevant_knowledge['direct_interaction'] = edge_data # find common targets (proteins, diseases) drug1_neighbors = set(graph.neighbors(drug1)) if drug1 in graph else set() drug2_neighbors = set(graph.neighbors(drug2)) if drug2 in graph else set() common_neighbors = drug1_neighbors.intersection(drug2_neighbors) for common_node in common_neighbors: node_type = graph.nodes[common_node].get('type', '') if node_type == 'protein' or node_type == 'disease': relevant_knowledge['common_targets'].append(common_node) # find paths between drugs (up to max_path_length) try: paths = list(nx.all_simple_paths(graph, drug1, drug2, cutoff=max_path_length)) relevant_knowledge['paths'] = paths except (nx.NetworkXError, nx.NodeNotFound): pass return relevant_knowledge ``` --- ### 4.9 批处理数据整理与数据集准备 #### 自定义批处理函数 确保批次中不含无效数据。 ```python # custom collate function to handle None values def custom_collate_fn(batch): batch = [item for item in batch if item is not None] if len(batch) == 0: return { 'drug1_img': torch.tensor([]), 'drug1_text': [], 'drug1_name': [], 'drug2_img': torch.tensor([]), 'drug2_text': [], 'drug2_name': [], 'label': torch.tensor([]) } drug1_imgs = torch.stack([item['drug1_img'] for item in batch]) drug1_texts = [item['drug1_text'] for item in batch] drug1_names = [item['drug1_name'] for item in batch] drug2_imgs = torch.stack([item['drug2_img'] for item in batch]) drug2_texts = [item['drug2_text'] for item in batch] drug2_names = [item['drug2_name'] for item in batch] labels = torch.stack([item['label'] for item in batch]) return { 'drug1_img': drug1_imgs, 'drug1_text': drug1_texts, 'drug1_name': drug1_names, 'drug2_img': drug2_imgs, 'drug2_text': drug2_texts, 'drug2_name': drug2_names, 'label': labels } ``` #### 数据集类 结合所有真实相互作用和一组匹配的随机非相互作用对。 ```python # define dataset for DDI prediction class DDIDataset(Dataset): def __init__(self, drug_data_df, drug_drug_interactions, medical_kg, node_to_idx, transform=None): self.drug_data = drug_data_df self.drug_name_to_idx = {row['name']: i for i, row in drug_data_df.iterrows()} self.node_to_idx = node_to_idx self.transform = transform or transforms.Compose([ transforms.Resize((224, 224)), transforms.ToTensor(), transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]) ]) self.pairs = [] drug_names = list(self.drug_name_to_idx.keys()) # positive samples (known interactions) for interaction in drug_drug_interactions: drug1, _, drug2, _ = interaction if drug1 in drug_names and drug2 in drug_names: self.pairs.append((drug1, drug2, 1)) positive_pairs = set((d1, d2) for d1, d2, _ in self.pairs) # generate some negative samples np.random.seed(42) neg_count = 0 max_neg = len(self.pairs) while neg_count < max_neg: i, j = np.random.choice(len(drug_names), 2, replace=False) drug1, drug2 = drug_names[i], drug_names[j] if (drug1, drug2) not in positive_pairs and (drug2, drug1) not in positive_pairs: self.pairs.append((drug1, drug2, 0)) neg_count += 1 def __len__(self): return len(self.pairs) def __getitem__(self, idx): try: drug1_name, drug2_name, label = self.pairs[idx] # get drug1 data drug1_idx = self.drug_name_to_idx[drug1_name] drug1_data = self.drug_data.iloc[drug1_idx] try: drug1_img = Image.open(drug1_data['image_path']).convert('RGB') drug1_img = self.transform(drug1_img) except Exception as e: print(f"Error loading drug1 image for {drug1_name}: {str(e)}") return None drug1_text = drug1_data['description'] # get drug2 data drug2_idx = self.drug_name_to_idx[drug2_name] drug2_data = self.drug_data.iloc[drug2_idx] try: drug2_img = Image.open(drug2_data['image_path']).convert('RGB') drug2_img = self.transform(drug2_img) except Exception as e: print(f"Error loading drug2 image for {drug2_name}: {str(e)}") return None drug2_text = drug2_data['description'] return { 'drug1_img': drug1_img, 'drug1_text': drug1_text, 'drug1_name': drug1_name, 'drug2_img': drug2_img, 'drug2_text': drug2_text, 'drug2_name': drug2_name, 'label': torch.tensor(label, dtype=torch.float32) } except Exception as e: print(f"Error in __getitem__ for index {idx}: {str(e)}") return None ``` --- ### 4.10 模型训练 训练函数包含训练阶段和验证阶段，使用二元交叉熵损失和Adam优化器。 ```python # training function def train_kg4mm_model(model, train_loader, val_loader, epochs=5): device = torch.device('cuda' if torch.cuda.is_a vailable() else 'cpu') model = model.to(device) model.pyg_graph = model.pyg_graph.to(device) criterion = nn.BCELoss() optimizer = torch.optim.Adam(model.parameters(), lr=0.0001) for epoch in range(epochs): # training phase model.train() train_loss = 0 train_correct = 0 batch_count = 0 for batch in train_loader: if len(batch['drug1_img']) == 0: print("Skipping empty batch") continue batch_count += 1 try: drug1_img = batch['drug1_img'].to(device) drug1_text = batch['drug1_text'] drug1_name = batch['drug1_name'] drug2_img = batch['drug2_img'].to(device) drug2_text = batch['drug2_text'] drug2_name = batch['drug2_name'] labels = batch['label'].to(device) batch_size = len(drug1_name) outputs = torch.zeros(batch_size, 1, device=device) for i in range(batch_size): output = model( drug1_img[i].unsqueeze(0), [drug1_text[i]], drug1_name[i], drug2_img[i].unsqueeze(0), [drug2_text[i]], drug2_name[i] ) outputs[i] = output loss = criterion(outputs, labels.unsqueeze(1)) optimizer.zero_grad() loss.backward() optimizer.step() train_loss += loss.item() predictions = (outputs >= 0.5).float() train_correct += (predictions == labels.unsqueeze(1)).sum().item() print(f"Batch {batch_count}: Loss: {loss.item():.4f}") except Exception as e: print(f"Error processing batch {batch_count}: {str(e)}") import traceback traceback.print_exc() continue a vg_train_loss = train_loss / max(1, batch_count) train_acc = train_correct / max(1, batch_count * batch['drug1_img'].size(0)) print(f'Epoch {epoch+1}/{epochs}, Train Loss: {a vg_train_loss:.4f}, Train Acc: {train_acc:.4f}') # validation phase model.eval() val_loss = 0 val_correct = 0 val_batch_count = 0 with torch.no_grad(): for batch in val_loader: if len(batch['drug1_img']) == 0: continue val_batch_count += 1 try: drug1_img = batch['drug1_img'].to(device) drug1_text = batch['drug1_text'] drug1_name = batch['drug1_name'] drug2_img = batch['drug2_img'].to(device) drug2_text = batch['drug2_text'] drug2_name = batch['drug2_name'] labels = batch['label'].to(device) batch_size = len(drug1_name) outputs = torch.zeros(batch_size, 1, device=device) for i in range(batch_size): output = model( drug1_img[i].unsqueeze(0), [drug1_text[i]], drug1_name[i], drug2_img[i].unsqueeze(0), [drug2_text[i]], drug2_name[i] ) outputs[i] = output loss = criterion(outputs, labels.unsqueeze(1)) val_loss += loss.item() predictions = (outputs >= 0.5).float() val_correct += (predictions == labels.unsqueeze(1)).sum().item() except Exception as e: print(f"Error processing validation batch {val_batch_count}: {str(e)}") continue a vg_val_loss = val_loss / max(1, val_batch_count) val_acc = val_correct / max(1, val_batch_count * 4) # Assuming batch_size=4 print(f'Epoch {epoch+1}/{epochs}, Val Loss: {a vg_val_loss:.4f}, Val Acc: {val_acc:.4f}') return model ``` **启动训练**：初始化数据集、数据加载器、模型，然后执行训练。 ```python # initialize dataset and model ddi_dataset = DDIDataset(drug_data_df, drug_drug_interactions, medical_kg, node_to_idx) train_size = int(0.8 * len(ddi_dataset)) val_size = len(ddi_dataset) - train_size train_dataset, val_dataset = torch.utils.data.random_split(ddi_dataset, [train_size, val_size]) train_loader = DataLoader(train_dataset, batch_size=4, shuffle=True, collate_fn=custom_collate_fn) val_loader = DataLoader(val_dataset, batch_size=4, shuffle=False, collate_fn=custom_collate_fn) num_node_types = pyg_graph.x.shape[1] num_edge_types = len(edge_type_to_idx) model = KGGuidedMultimodalModel(pyg_graph, num_node_types, num_edge_types, node_to_idx, idx_to_node) trained_model = train_kg4mm_model(model, train_loader, val_loader, epochs=5) ``` > **小提示**：由于示例数据集较小（仅50种药物和少量相互作用），训练轮数可设为5-10。在实际应用中，建议使用完整的DrugBank数据集并训练更多轮次（如50-100），同时采用学习率衰减和早停策略。 --- ### 4.11 推理与解释 **预测函数**：加载药物数据后，通过模型计算相互作用概率，同时获取知识子图。 ```python def predict_interaction(model, drug1_name, drug2_name, drug_data_df, medical_kg): device = torch.device('cuda' if torch.cuda.is_a vailable() else 'cpu') model = model.to(device) model.eval() drug1_idx = drug_data_df[drug_data_df['name'] == drug1_name].index[0] drug2_idx = drug_data_df[drug_data_df['name'] == drug2_name].index[0] drug1_data = drug_data_df.iloc[drug1_idx] drug2_data = drug_data_df.iloc[drug2_idx] transform = transforms.Compose([ transforms.Resize((224, 224)), transforms.ToTensor(), transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]) ]) drug1_img = Image.open(drug1_data['image_path']).convert('RGB') drug1_img = transform(drug1_img).unsqueeze(0).to(device) drug1_text = [drug1_data['description']] drug2_img = Image.open(drug2_data['image_path']).convert('RGB') drug2_img = transform(drug2_img).unsqueeze(0).to(device) drug2_text = [drug2_data['description']] knowledge = retrieve_knowledge_subgraph(medical_kg, drug1_name, drug2_name) with torch.no_grad(): interaction_prob = model( drug1_img, drug1_text, drug1_name, drug2_img, drug2_text, drug2_name ) return interaction_prob.item(), knowledge ``` **解释函数**：将概率转换为风险等级，并结合知识图谱结构生成自然语言解释。 ```python def explain_interaction_prediction(drug1_name, drug2_name, probability, knowledge): explanation = f"KG-guided multimodal analysis for interaction between {drug1_name} and {drug2_name}:\n\n" if probability > 0.8: risk_level = "High" elif probability > 0.5: risk_level = "Moderate" else: risk_level = "Low" explanation += f"Interaction Risk Level: {risk_level} (Probability: {probability:.2f})\n\n" explanation += "Knowledge Graph Analysis:\n" if knowledge['direct_interaction']: mechanism = knowledge['direct_interaction'].get('mechanism', 'unknown mechanism') explanation += f"✓ Direct Connection: The knowledge graph contains a documented interaction between these drugs with {mechanism}.\n\n" if knowledge['common_targets']: explanation += "✓ Common Target Nodes: These drugs connect to shared entities in the knowledge graph:\n" for target in knowledge['common_targets']: explanation += f" - {target}\n" explanation += " This graph structure suggests potential interaction through common binding sites or pathways.\n\n" if knowledge['paths'] and len(knowledge['paths']) > 0: explanation += "✓ Knowledge Graph Pathways: The model identified these connecting paths in the graph:\n" for i, path in enumerate(knowledge['paths'][:3]): path_str = " → ".join(path) explanation += f" - Path {i+1}: {path_str}\n" explanation += " These graph structures guided the multimodal feature integration for prediction.\n\n" explanation += "Multimodal Integration Process:\n" explanation += " - Knowledge graph structure determined which drug properties were most relevant\n" explanation += " - Graph neural networks analyzed the local neighborhood of both drug nodes\n" explanation += " - Node position in the graph guided the weighting of visual and textual features\n\n" if probability > 0.5: explanation += "Clinical Recommendations (based on graph analysis):\n" explanation += " - Consider alternative medications not connected in similar graph patterns\n" explanation += " - If co-administration is necessary, monitor for interaction effects\n" explanation += " - Review other drugs connected to the same nodes for potential complications\n" else: explanation += "Clinical Recommendations (based on graph analysis):\n" explanation += " - Standard monitoring advised\n" explanation += " - The knowledge graph structure suggests minimal interaction concerns\n" return explanation ``` --- ## 结果：真实案例演示 我们以药物对 **"Goserelin"** 和 **"Desmopressin"** 为例，演示完整的预测与解释流程。 ```python # example usage drug_pair = ("Goserelin", "Desmopressin") prob, knowledge = predict_interaction(trained_model, drug_pair[0], drug_pair[1], drug_data_df, medical_kg) print(f"Predicted interaction probability between {drug_pair[0]} and {drug_pair[1]}: {prob:.4f}") print("\nKnowledge Graph Structure Analysis:") print(f"Direct connection: {knowledge['direct_interaction']}") print(f"Common target nodes: {knowledge['common_targets']}") print(f"Graph paths connecting drugs:") for path in knowledge['paths']: print(f" {' -> '.join(path)}") # visualize the subgraph for these drugs to show the KG-guided approach plt.figure(figsize=(12, 8)) subgraph_nodes = set([drug_pair[0], drug_pair[1]]) for path in knowledge['paths']: subgraph_nodes.update(path) neighbors_to_add = set() for node in subgraph_nodes: if node in medical_kg: neighbors_to_add.update(list(medical_kg.neighbors(node))[:3]) subgraph_nodes.update(neighbors_to_add) subgraph = medical_kg.subgraph(subgraph_nodes) node_colors = [] for node in subgraph.nodes(): if node == drug_pair[0] or node == drug_pair[1]: node_colors.append('lightcoral') elif subgraph.nodes[node].get('type') == 'protein': node_colors.append('lightblue') elif subgraph.nodes[node].get('type') == 'disease': node_colors.append('lightgreen') else: node_colors.append('lightgray') pos = nx.spring_layout(subgraph, seed=42) nx.draw(subgraph, pos, with_labels=True, node_color=node_colors, node_size=2000, arrows=True, arrowsize=20) edge_labels = {(s, o): subgraph[s][o]['relation'] for s, o in subgraph.edges()} nx.draw_networkx_edge_labels(subgraph, pos, edge_labels=edge_labels) plt.title(f"Knowledge Graph Structure Guiding {drug_pair[0]} and {drug_pair[1]} Interaction Analysis") plt.sa vefig('kg_guided_interaction_analysis.png') plt.show() # show explanation explanation = explain_interaction_prediction(drug_pair[0], drug_pair[1], prob, knowledge) print(explanation) ``` **输出结果**： ``` Predicted interaction probability between Goserelin and Desmopressin: 0.54 Knowledge Graph Structure Analysis: Direct connection: {'relation': 'interacts_with', 'mechanism': 'increases_anticoagulant_effect'} Common target nodes: [] Graph paths connecting drugs: Goserelin -> Desmopressin ``` 在戈舍瑞林 (Goserelin) 和去氨加压素 (Desmopressin) 上测试时，模型返回了 **0.54** 的概率，将其归类为 **中等风险** 对。知识图谱揭示了两种药物之间存在一个直接的“相互作用”（interacts_with）关系，具体机制为“增加抗凝作用”（increases_anticoagulant_effect）。没有共享的蛋白质或疾病连接，因此模型主要关注了该机制。 子图可视化中，两种药物以红色突出显示，单条有向边突出显示，清晰展示了是哪种关系驱动了预测。  --- ## 常见问题与解答 **问题1：为什么需要将InChI转换为SMILES？** 答：RDKit对SMILES格式支持更好，生成的分子图像更可靠。虽然InChI也是标准格式，但RDKit的`MolFromSmiles`函数比`MolFromInchi`更稳定。 **问题2：知识图谱中的节点和边是否可以扩展？** 答：完全可以。您可以根据实际数据添加更多类型的节点（如基因、副作用、适应症）和边（如`causes`、`metabolized_by`）。图谱越丰富，模型的可解释性和准确性可能越高。 **问题3：训练时出现“Skipping empty batch”怎么办？** 答：这通常是因为批次中的所有样本都返回了`None`（例如图像加载失败）。请检查数据路径和图像文件是否存在，或者增加数据集大小以减少空批次概率。 **问题4：模型在真实大规模数据上表现如何？** 答：本教程为演示目的使用了简化数据集。在实际场景中（如完整DrugBank），建议使用更深的GNN层（如3-4层GCN）、更大的隐藏维度（256-512），并采用早停和学习率调度。使用GPU（CUDA）可以显著加速训练。 **问题5：可解释性部分是否可靠？** 答：注意力权重和知识图谱路径提供了透明的理由，但需要注意：模型可能学到虚假关联。建议结合临床专家验证，并不要完全依赖自动生成的风险评估。 --- ## 小提示：优化建议 - **数据增强**：对分子图像应用随机旋转、裁剪等增强，可以提高模型的泛化能力。 - **负样本采样**：除了随机采样，还可以使用“硬负样本”（即与正样本结构相似但无相互作用的药物对）来训练更鲁棒的模型。 - **多模态融合策略**：除了本文中的注意力加权，还可以尝试跨模态Transformer或协同注意力机制。 - **超参数调优**：使用`Optuna`或`Ray Tune`自动搜索最佳学习率、隐藏维度、GNN层数等。 - **模型部署**：训练完成后，可以使用`torch.jit.script`将模型导出为TorchScript，便于在生产环境中部署。 --- ## 总结与展望 本教程详细介绍了KG4MM——一种将知识图谱与多模态学习相结合的药物相互作用预测方法。通过将知识图谱置于工作流程的核心，模型能够更好地融合分子图像和文本描述，效果优于单一来源的方法。每个预测都由清晰的图谱证据支持——直接边、共享靶点和连接路径——这使得结果与真实的生物关系关联起来。 KG4MM不仅提供了更强的预测能力，还内置了可解释性，这在新药发现、个性化医疗和药物安全监测等领域具有重要价值。未来，可以进一步探索动态知识图谱（随时间演化的关系）、引入更丰富的多模态数据（如蛋白质3D结构、基因表达谱），以及联合其他预测任务（如药物副作用预测）来构建更全面的药物AI系统。 通过本教程的完整实现，您已经掌握了从零构建一个知识图谱引导的多模态药物相互作用预测模型的核心技术。祝您在药物AI的探索之旅中取得成功！