After a decade-long pause, I resumed my hobby of building robots this summer holiday. A decade ago, I was playing with Lego NXT and Arduino. This time, I am going pro with STM32 (e.g., following an...

Just before my summer holiday in China, I became interested in the visual neural systems of insects and other species, and how they compare to each other. Along this line of investigation, I asked ...

I read the paper Dissecting Recall of Factual Associations in Auto-Regressive Language Models by Geva et al., and I highly recommend it. If you’re curious about how large language models process in...

In the last post of this series, we tried the instruction-tuned IT model. In this post, we focus on the raw PT model. The following is Table 15 from the Gemma 3 technical report. It compares the p...

In the first post of this series, we examined the specifications of the Gemma 3 model. In this post, we will actually run it and get an intuitive understanding of the inference process. I will assu...

Recently, I’ve become quite fascinated by small LLMs such as Gemma 3 and the idea of running them locally. I tried out Ollama, which builds on top of llama.cpp but wraps it in an elegant, user-frie...

I have attempted to learn the Transformer multiple times. When the now-classical paper Attention Is All You Need first came out in 2017, I was attracted by its title. That was about 5 years after ...

While organizing old files, I realized I had lost my master’s thesis. After some effort, I finally found it online, but unfortunately, it is not very accessible. Upon reading it a bit, I realized m...

I wrote a simple paper to demonstrate that Carnot’s theorem is consistent with solar sails during my undergraduate studies at the Nanjing Institute of Meteorology in 2003-2004. It was originally pu...

I feel the core of human society is trust and disguise, and all social phenomena can be reduced to them.