Foro3D delves into the past with Talkie-1930, a 13-billion parameter language model trained exclusively on texts from before 1930. This vintage LLM, with no access to the internet or modern information, offers a detailed window into the mindset, culture, and daily life of the early 20th century, mimicking the style of authors like Dickens or Conan Doyle.
Architecture and training: coal dust and steam 🚂
Talkie-1930 was built on a transformer architecture, but its training was limited to a corpus of books, newspapers, and magazines published before 1930. With no access to Wikipedia or social media, its knowledge stops at the Great Depression. The model has been fine-tuned to replicate the linguistic turns of the era, such as the use of formal vocabulary and complex sentence structures. The developers filtered out any references after 1930, including terms like internet or smartphone, to ensure its historical purity.
Ask it about the flu and it will talk to you about sweaters 🧣
If you ask it about climate change, Talkie-1930 will respond with a treatise on the benefits of coal and the need for more factory chimneys. Asking it for a pizza recipe will get you a detailed explanation on how to prepare a meat and potato stew, served with tea. It is the perfect assistant for those who long for a world without WiFi, where the biggest technological concern was the phonograph getting stuck.