u012492756的博客

原创 RL--Policy-based methods--PPO

policy gradient 的 off-policy 版本，有 2 个 policy，behavior policy 用来采样，target policy 是训练最终要得到的结果。PPO全称Proximal Policy Optimizationproximal 为近端的，表示要让 behavior policy 和 target policy 差别不太大。虽然采样和训练的2 个 policy，但是需要他们差别较小。

原创 RL--Policy-based methods--Policy Gradient

通过 gradient ascent 让 policy 下 trajectory 的期望 return 变得尽可能的大。

原创 大模型--意图对齐--Instruction Tuning

原创 大模型--高效微调--Prefix Tuning

给每个任务初始化一些virtual token 的 embedding 作为 prefix 添加到序列中，训练时只优化 virtual token embedding只在输入时把 task 的 token embedding 加入输入序列的 embedding。

原创 大模型-高效微调-LoRA

freeze the pre-trained model weights, inject trainable low-rank matrices into each layer, 从而只用训练很少的参数，就可以达到全参数微调一样的效果。

原创 大模型-计算优化-DeepSpeed

DeepSpeed 是什么，原理的简要介绍

原创 GNN-CS224W: 19 GNNs for Science

Pre-Training GNNspre-train要考虑的内容pre-train之前可以先设计好希望训练完的embedding满足什么要求，例如同类型的要接近，不同类型的要距离远，有相同属性的要接近，推理的要求等等pre-train taskself-supervised 和superviseembedding的level例如node level，graph level，link levelTasksAttribute Masking [Node-level] [self

原创 GNN-CS224W: 17 Scaling Up GNNs

Graphs in model applicationapplicationsRecommender systems:Amazon, YouTube 等要预测用户对item(商品、视频)是否感兴趣 (link prediction)还要预测用户、item的类型(node classification)数据量级很大Social networksFacebook, Twitter, Instagram等要做 Friend recommendation (link level task)Us

原创 GNN-CS224W: 16 Position-aware and Identity-aware GNNs and Robustness of GNN

The GNNs so far are not perfectwhat should a perfect GNN do ?A perfect GNN should build an injective function between neighborhood structure (regardless of hops) and node embeddings.根据以前讲的知识，a perfect GNN 应该满足2点：If two nodes have the same neighborhood

原创 GNN-CS224W:13 Community Structure in Networks

connection between edge strength and edge structureedge strength在social network中可以用两个人之间联系的频率来得到，联系的频率越高，则strength越大，否则越小edge structure表示这个edge是处于一个cluster内部，还是处于不同cluster之间edge overlap用edge overlap来表示edge structureOij=∣(N(i)∩N(j))−{i,j}∣∣(N(i)∪N(j)

原创 GNN-CS224W:12 Frequent Subgraph Mining with GNNs

Subgraphs and motifs本节讲的是如何定义subgraph和motif，以及如何用subgraph和motif 来代表graph的信息。NotionsSubgraph DefinitionNode-induced subgraph ( induced subgraph ): Take subset of the nodes and all edges induced by the nodes，all edges指的是edge的两个节点都在node subset里的edgeEdge-

原创 GNN-CS224W: 11 Reasoning over Knowledge Graphs

QueryOne-hop Queries转化为KG completion taskKG completion 实现对候选tail做分类，即(head,relation,tail)(head, relation, tail)(head,relation,tail)是否存在问题，怎么确定候选tail？难道所有的都要试一遍？Path Queriesn hop query 可以表示为q=(va,(r1,…,rn))q=(v_a, (r_1, \dots, r_n))q=(va​,(r1​,…,rn​

原创 GNN-CS224W:10 Knowledge Graph Embeddings

Heterogeneous Graph有不止一种edge、node的graph就是Heterogeneous GraphHeterogeneous Graph可以用4元组来表示：G=(V,E,R,T)G=(V,E,R,T)G=(V,E,R,T)其中vi∈Vv_i \in Vvi​∈V表示node，node 的type可以通过T(vi)T(v_i)T(vi​)得到edge用3元组表示(vi,r,vj)∈E(v_i, r, v_j) \in E(vi​,r,vj​)∈E，关系r∈Rr \in Rr∈R

原创 GNN-CS224W: 9 Theory of Graph Neural Networks

What is the expressive power(ability to distinguish different graph structures) of these GNN models?graph包括2部分：1. node feature；2. graph structure接下来先讨论graph structure，假设node feature都一样（是否会讨论node featue 不知道，接着看）接下来讨论的问题是：How well can a GNN distinguish d

原创 GNN-CS224W: 8 Training GNN

Graph AugmentationRaw input graph ≠ computational graph之前提到的computational graph是完全根据node之间关系确定的，但是不一定非得这样做，可以通过其他方法获得computational graph为什么不用原始的computational graph？因为原始的computational graph存在各种问题：The input graph lacks featuresGraph structure problem

原创 GNN-CS224W: 6-7 Graph Neural Networks

GNN概述GNN定义了一种可以批量encode graph 类型数据的方法的深度方法可以利用local graph structure，也可以利用node featurecomputational graph如下图所示：node 的local structure 定义了它的computational graph每个节点都计算相同的depthnode的输入就是feature，link体现在node 的computational graph上同一个node在不同的层会有不同的embedd

原创 GNN-CS224W: 5 Label Propagation for Node Classification

Task全文都在解决这一个Task: Given a network with labels on some nodes, how do we assign labels to all other nodes in the network?本文涉及的是Probabilistic framework，不涉及深度学习。用到的信息Classification label of a node vvv in network may depend onFeatures of node vvvLabels

原创 GNN-CS224W: 4 Link Analysis: PageRank

treat the web as a graphpage rank的目的是给作为node的网页排序We will cover the following Link Analysis approaches to compute the importance of nodes in a graph:PageRankPersonalized PageRank (PPR)Random Walk with Restarts3种方法是类似的，是逐渐改进的。目的是计算node的importanceIde

原创 GNN-CS224W: 3 Node Embeddings

graph representation learningAutomaticly learning efficient task-independent feature for machine learning with graphs，从而避免了特征工程why embedding？Similarity of embeddings between nodes indicates their similarity in the network.Encode network informationTa

原创 GNN-CS224W: 1-2 Introduction； Traditional Methods for machine learning in Graphs

node-level, link-level, graph-level tasks and features