[News Figures] Researcher Who Wrote Key Paper on Generative AI Explains Why They Founded a Startup in Tokyo

Leon Jones, Author of 2017 'Transformer' Paper
"Google, Side Effects of Large Corporations... Bureaucracy Felt"
Aiming to Build Clusters with Small Models Rather Than Large AI Models

A former Google AI researcher who contributed to the foundational paper behind generative artificial intelligence (AI), including ChatGPT, has established an AI startup in Tokyo, Japan. Feeling limited by bureaucracy and other constraints at the large corporation Google, he announced that his new company will focus on creating small-scale AI models.

According to CNBC and other sources on the 17th (local time), Leon Jones, a former Google researcher, founded the generative AI company "Sakana AI" together with David Ha, who was previously the head of research at the image generation AI company Stability AI and a former Google colleague.

Researcher Leon Jones during his tenure at Google. [Image source=X (Twitter)]

Jones’s entrepreneurial move is attracting attention because he was one of the eight authors of the 2017 paper "Attention Is All You Need," written by the Google deep learning AI research team. This paper first introduced the neural network model "Transformer," which is central to generative AI, and is regarded as the starting point of generative AI. The term GPT itself stands for "Generative Pre-trained Transformer," highlighting its basis on the Transformer architecture.

Notably, he was the last of the paper’s authors to remain at Google. Other authors had left before Jones, mostly founding startups, and some joined OpenAI, the developer of ChatGPT.

In an interview with CNBC, Jones explained his reason for leaving, saying, "(Google) had the downsides of a large corporation," and "I felt there was so much bureaucracy that it seemed like I couldn’t do anything." He pointed out that the more he developed AI software at Google, the more he questioned why it didn’t work and who was responsible, and that these questions hindered research. He added, "I was spending my time every day trying to obtain resources and data."

Jones joined Google in 2012 via YouTube and, after more than ten years at Google, established a startup in Tokyo. At this company, which he runs with a co-founder he worked with at Google’s Tokyo office, they plan to develop small systems first rather than the large AI systems that big corporations focus on. The company name "Sakana (さかな)," which means "fish" in Japanese, symbolizes their approach to technology development?like small fish gathering to form a large group that moves consistently according to simple rules.

The two founders believe that the large AI models that major corporations are focusing on are like bridges or buildings?structures that cannot be changed?and thus have limitations. They see small-scale models forming swarms as being more sensitive to change and adaptable. Jones pointed out, "The entire company (Google) is focused on one technology," and said that because this is a limited system, innovation inevitably becomes difficult.

Regarding why they established their startup in Tokyo, the two founders explained that compared to North America, Tokyo offers a competitive market for researchers to develop generative AI software, along with high-quality technological infrastructure and a highly educated workforce. They added, "Training data and machines adjusted to work well in non-Western societies and cultures will be the catalyst for the next technological innovation."

This content was produced with the assistance of AI translation services.