today idea

#1 why ? virtual node?

In lecture

lecture ; cs224w section 10 — graph augmentation and training.

In paper

one word ; ‘master node’

which is connected to every input node in graph with special edge type. The master node serves as a global scratch space that each node both reads from and writes to in every step of message passing. This allows information to travel long distances during the propagation phase

paper ; Neural Message Passing for Quantum Chemistry

In code

# declare the virtual node embedding 
self.virtualnode_embedding = torch.nn.Embedding(1, emb_dim)
### List of MLPs to transform virtual node at every layer        self.mlp_virtualnode_list = torch.nn.ModuleList()
for layer in range(num_layer - 1):            self.mlp_virtualnode_list.append(torch.nn.Sequential(
torch.nn.Linear(emb_dim, 2*emb_dim),\ #linear
 torch.nn.BatchNorm1d(2*emb_dim),\ #normalization
 torch.nn.ReLU(), \ #activation                                                   torch.nn.Linear(2*emb_dim, emb_dim),\ #linear
 torch.nn.BatchNorm1d(emb_dim),\ #normalization
 torch.nn.ReLU()) # activation)

GitHub ; ogb example

i didnt’ understand how virtual node connect others and what is that format ?

→ just connected all in the batched graph.

### update the virtual nodes
if layer < self.num_layer - 1:
### add message from graph nodes to virtual nodes                virtualnode_embedding_temp = 
global_add_pool(h_list[layer], batch) + virtualnode_embedding                ### transform virtual nodes using MLP

yeah, global_add_pool makes them all in one from batched of message passing. so, we consider how they are proper propagation in batched graph.

best case ; sparse graph batched

worst case ; dense graph batched

solution is the distribution of the each batch.