Not just startups and companies, but national governments too have begun creating their own AI models and sharing them with the world.
Poland’s government has uploaded several AI models on Hugging Face. These models specialize in Polish and other Slavic and Baltic languages. The models are for anyone to use, and focus on open language tasks for Polish public administration.
“The Polish Ministry of Digital Affairs drives the growth of digital innovation and AI in Poland. Like an architect of the digital future, it identifies the needs of researchers, businesses, and public administration, addressing them on three key fronts: access to technology, legal frameworks, and the spread of knowledge,” the organization’s description says.
The models released by the Polish Ministry of Digital Affairs are called PLLuM. “PLLuM is a family of large language models (LLMs) specialized in Polish and other Slavic/Baltic languages, with additional English data incorporated for broader generalization. Developed through an extensive collaboration with various data providers, PLLuM models are built on high-quality text corpora and refined through instruction tuning, preference learning, and advanced alignment techniques. These models are intended to generate contextually coherent text, offer assistance in various tasks (e.g., question answering, summarization), and serve as a foundation for specialized applications such as domain-specific intelligent assistants,” the model card says.
The model creators have used 150 billion Polish language tokens in the creation of these models, and additional tokens from Slavic, Baltic, and English languages. They also curated the largest Polish collection of manually created “organic instructions” of around 40k prompt-response pairs, including 3.5k multi-turn dialogs. “This human-authored instruction set is based on an extensive typology of human-model interactions and it covers a range of subtle aspects of supervised fine-tuning (SFT) that might be overlooked with automated approaches (including large scale distillation of ‘strong LLMs’). It was also designed to mitigate negative linguistic transfer from non-Polish textual data used in the pre-training phase,” the model card says. The model generated state-of-the-art results on custom benchmarks for Polish-language tasks.
Polish and Slavic languages are very different from English, and English models likely didn’t perform particularly well on them. Also, these countries have small populations, so AI model companies would not have necessarily prioritized building models for these languages. But the Polish authorities seem to have taken matters into their own hands, and using custom Polish data and instruction sets, created what they say is a SOTA model for their use-cases. The AI revolution is well and truly underway, and some governments seem to be responding to the changing times.