Size Local AI for the Working Context

Microsoft’s 5 November 2026 guide for running Qwen3.8-27B through Foundry Local offers a useful lesson in hardware planning. Microsoft reports an approximately 18 GB model with 4-bit quantization, with context windows of up to 64K tokens on NVIDIA GPUs with 24 GB of VRAM and up to 262K tokens on NVIDIA GPUs with 32 GB or more.

Those are Microsoft’s configuration claims, not Athelguard measurements. They make the working context an important part of the purchasing question: how much repository material, instruction history and tool output must remain available during a useful coding session?

Budget for the session you need

The diagram turns a specification into a practical acceptance test. Loading the model is the first checkpoint. A complete session should also finish with the context and responsiveness the team needs.

Before choosing a standard workstation configuration, ask the supplier or internal team to demonstrate:

• Ordinary and demanding tasks. Use a representative repository and retain the instructions, search results and tool output the workflow actually needs. Record any context trimming required to finish.
• Shared capacity. Repeat with the intended number of simultaneous sessions while normal development applications are running. Record failures, task completion time and whether the resulting changes pass the team’s acceptance checks.
• The operating boundary. Identify which tasks require a different machine or a cloud fallback, who maintains the runtime, and what those exceptions add to the operating cost.

These checks create a useful comparison between configurations: cost per acceptable completed task, with the workload and fallback policy specified. A larger advertised context ceiling alone provides little evidence about that outcome.

There is also a deployment detail to budget for. The guide uses Foundry Local CLI 0.13.0 or later. The 0.13.0 release, dated 2 November, is a public preview and requires separate CUDA and cuDNN installation for CUDA execution.

For an Independent Technical Assessment, request a reproducible workload record alongside the hardware quotation. It should show which tasks fit, which complete acceptably and which depend on an exception. That is a more useful basis for planning a rollout.