{"id":1121,"date":"2026-11-06T18:16:15","date_gmt":"2026-11-06T18:16:15","guid":{"rendered":"https:\/\/athelguard.com\/?p=1121"},"modified":"2026-11-06T18:16:15","modified_gmt":"2026-11-06T18:16:15","slug":"size-local-ai-for-the-working-context","status":"publish","type":"post","link":"https:\/\/athelguard.com\/?p=1121","title":{"rendered":"Size Local AI for the Working Context"},"content":{"rendered":"\n

Microsoft\u2019s 5 November 2026 guide<\/a> for running Qwen3.8-27B through Foundry Local offers a useful lesson in hardware planning. Microsoft reports an approximately 18 GB model with 4-bit quantization, with context windows of up to 64K tokens on NVIDIA GPUs with 24 GB of VRAM and up to 262K tokens on NVIDIA GPUs with 32 GB or more.<\/p>\n

Those are Microsoft\u2019s configuration claims, not Athelguard measurements. They make the working context an important part of the purchasing question: how much repository material, instruction history and tool output must remain available during a useful coding session?<\/p>\n\n

\n
\n

Qualify the whole workload<\/p>\n

\n

1 \u00b7 LOAD THE MODEL<\/p>\n

Chosen weights + runtime configuration<\/p>\n<\/div>\n

\u2193<\/p>\n

\n

2 \u00b7 COMPLETE A SESSION<\/p>\n

Repository context + history + tool output<\/p>\n<\/div>\n

\u2193<\/p>\n

\n

3 \u00b7 REPEAT UNDER LOAD<\/p>\n

Concurrent sessions + normal applications<\/p>\n<\/div>\n<\/div>\n

Original Athelguard qualitative test sequence, not a memory allocation chart or benchmark. Informed by Microsoft\u2019s Foundry Local configuration guidance<\/a>, checked 6 November 2026.<\/figcaption>\n<\/figure>\n\n

Budget for the session you need<\/h2>\n

The diagram turns a specification into a practical acceptance test. Loading the model is the first checkpoint. A complete session should also finish with the context and responsiveness the team needs.<\/p>\n

Before choosing a standard workstation configuration, ask the supplier or internal team to demonstrate:<\/p>\n