I read the paper Dissecting Recall of Factual Associations in Auto-Regressive Language Models by Geva et al., and I highly recommend it. If you’re curious about how large language models process in...

In the last post of this series, we tried the instruction-tuned IT model. In this post, we focus on the raw PT model. The following is Table 15 from the Gemma 3 technical report. It compares the p...

In the first post of this series, we examined the specifications of the Gemma 3 model. In this post, we will actually run it and get an intuitive understanding of the inference process. I will assu...

Recently, I’ve become quite fascinated by small LLMs such as Gemma 3 and the idea of running them locally. I tried out Ollama, which builds on top of llama.cpp but wraps it in an elegant, user-frie...

I have attempted to learn the Transformer multiple times. When the now-classical paper Attention Is All You Need first came out in 2017, I was attracted by its title. That was about 5 years after ...

While organizing old files, I realized I had lost my master’s thesis. After some effort, I finally found it online, but unfortunately, it is not very accessible. Upon reading it a bit, I realized m...

I wrote a simple paper to demonstrate that Carnot’s theorem is consistent with solar sails during my undergraduate studies at the Nanjing Institute of Meteorology in 2003-2004. It was originally pu...

I feel the core of human society is trust and disguise, and all social phenomena can be reduced to them.

The two terms “market economy” and “planned economy” can’t be more misleading. As the opposite of planned economy, the market economy sounds like the type of economy without planning, thus messy an...

Innovation means being different. It requires courage to be the minority. Why does one need courage to be a minority? That is because human nature has the tendency to sacrifice the minority for the...