Large Language Models (LLMs) are increasingly adopted in public organisations to advance Digital Government Transformation (DGT). However, their deployment in non-English contexts raises critical concerns about digital sovereignty and the legitimacy of AI within local cultural, political, and bureaucratic environments. This study investigates the initial design and use of SVEA, a Swedish-language AI assistant developed under AI Sweden’s national initiative for regional governments and municipalities. Using an in-depth case study approach, we collected data through stakeholder interviews, meetings, surveys, and document analysis. The findings reveal that initial adoption has been slow, although users regarded the assistant as a legitimate alternative to commercial AI tools. Three key challenges emerged during development: (1) technical-contextual mismatches, including file size limitations and incomplete Retrieval-Augmented Generation (RAG) capabilities; (2) legitimacy-versus-capability trade-offs, wherein a secure, localised design enhanced legitimacy but lagged behind commercial models in performance; and (3) limited organisational readiness, affected by privacy concerns, learning curves, and evolving governance frameworks. This study contributes to e-government research by demonstrating how contextual factors—such as language, culture, and bureaucratic norms—shape both the design and perceived legitimacy of AI tools in public administration. It also offers practical strategies for localising LLMs, enhancing digital sovereignty, and strengthening organisational readiness amid global AI adoption.