Thi Linh Hoang

Review of Graph Transformer

2023-11-30T00:00:00-08:00

In last few years, Graph Transformers models become a promising direction for graph learning model due to the successful of Transformers in other tasks like NLP or CV. This note reviews about the recent works adapt Transformer for graph data for more a scaleable, expressive tool include improving positional encoding, attention mechanism, etc.

You can check my note at notion.

Note in Probability basis

2023-08-24T00:00:00-07:00

Some of the concepts of probability I think we will use frequently.

Expectations.

Some useful properties:

Linearity. $\mathbb{E}[\sum_i c_jg_j(X)] = \sum_j c_j \mathbb{E}[g_j(X)]$, where $c_j$’s are not random variables.
Independence. If $X, Y$ are independent random variables $P(X \in A, Y \in B) = P(X \in A) P(X\in B)$ for all $A, B$, then $\mathbb{E}[X, Y] = \mathbb{E}[X] \mathbb{E}[Y]$.
Iterated Expectation. $\mathbb{E}[Y] = \mathbb{E}[\mathbb{E}[Y |X] = \int{\mathbb{E}[y |X = x]p(x)dx}$

Variance. The variance of a random variale is: $Var[X] = \mathbb{E}[(X - \mathbb{E}[X])^2 = \mathbb{E}[X^2] - {\mathbb{E}[X]}^2$

Some useful properties:

Independence. If $X_1, …, X_n$ are independent, then $Var[\sum_i a_iX_i] = \sum_i a_i^2 Var[X_i]$.
Law of total variance. $Var[y] = Var[\mathbb{E}[Y|X]] + \mathbb{E}[Var[Y|X]]$.

Markov’s inequality. If $X$ is a non-negative random variable, then for any $\epsilon > 0$: $P[X \geq \epsilon] \leq \frac{\mathbb{E}[X]}{\epsilon}$

Snails climb up the well

2023-08-17T00:00:00-07:00

A snail climbs a well $x$ feet during the day, and during night, it slips $y$ feet. If the well is $h$ feet deep, how long will it take for the snail to climb?

Solution:

Let $k$ denotes the number of days for the snail climb up the well, $k \in \mathbb{N}$.

If $n \leq x$, $k = 1$.

If $n > x$,

If $ x \leq y$, $k = \infty$,
If $ x > y$ : k is the minimum value satisfies this inequality:

\[(k -1)(x-y) + x \geq h,\] \[\implies k - 1 \geq \frac{h - x}{x - y},\] \[\implies k \geq \frac{h - x}{x - y} + 1,\] \[\implies k = \lceil \frac{h - x}{x - y} + 1 \rceil\]