On-prem LLM deployment: Mac Mini vs GPU server for 32B models.

The single most common on-prem deployment failure is not a model choice, not a prompt-engineering gap, and not a data-residency oversight — it is putting the wrong hardware tier under a 32B model and then watching the whole project collapse under its own latency. A team that expected a responsive assistant gets a box that stalls for four seconds before the first token. The pilot is dead before the demo ends.

Deploying a 32B-class model on-prem is not exotic — Qwen 3 32B Q4_K_M quantisation fits comfortably in 22 GB, well inside what a Mac Mini M4 Pro can address. But "fits" and "performs" are different questions. The hardware tier determines whether you get a single-user copilot, a team productivity tool, or a production inference server. Getting that decision right before the purchase order matters, because none of these three tiers are easily upgraded in place.

Below I compare the three tiers OwnAI ships across: the Mac Mini M4 Pro 64 GB, the RTX 4090 workstation build, and the L40S 48 GB server. All benchmarks reference Qwen 3 32B (dense, not the 30B-A3B MoE variant — the MoE numbers you see elsewhere are 5–10× faster and not applicable here) at 8K context, served via llama.cpp or vLLM depending on tier. Hardware prices are India street, May 2026, inclusive of GST.

The three tiers

Benchmark figures for Qwen 3 32B are engineering estimates extrapolated from Qwen 2.5 32B community data (MacRumors, ContaCollective, Hugging Face discussions) and GPU benchmark suites (XiongjieDai llama.cpp sweep, NVIDIA NIM official L40S data). No peer-reviewed, same-hardware, same-model, same-quant published benchmark for Qwen 3 32B dense existed as of May 2026. Treat single-user tok/s as ±20% indicative. Run vLLM benchmark_serving.py against your own hardware before signing a purchase order.

Mac Mini M4 Pro: when 1–5 users is enough

The Mac Mini M4 Pro 14-core / 64 GB configuration is the most underestimated inference node in the Indian enterprise market. Apple's unified memory architecture means the CPU and the GPU share the same 64 GB pool at 273 GB/s — there is no PCIe bottleneck moving weights from system RAM into a separate VRAM buffer. Qwen 3 32B at Q4_K_M quantisation lands at roughly 22 GB, giving you comfortable headroom for the KV cache at 8K context without memory pressure.

Real-world single-user throughput on analogous Qwen 2.5 32B Q4 hardware sits around 11–12 tok/s on the M4 Pro 20-core variant (higher memory bandwidth than the 14-core). With Qwen 3 32B dense, community numbers extrapolate to 14–18 tok/s decode. That is a comfortable reading pace — users processing long GMP SOP drafts or pharmacovigilance summaries do not notice the difference between 14 tok/s and 40 tok/s in daily use.

The Mac Mini breaks down when concurrency increases. At two simultaneous users the model is already bandwidth-saturated and per-user throughput halves. At four simultaneous users you are below 5 tok/s — perceptibly slow. The practical ceiling for genuine multi-user production use is two to three concurrent sessions, which maps to a team of four to six light-to-medium users with staggered usage patterns, or one to two engineers doing heavy, sustained generation.

The power story is compelling: 50 W typical, ₹3,940/year electricity at India's blended commercial tariff of ₹9/kWh. It fits under a desk, makes no noise, requires no UPS above a small consumer unit, and runs macOS — meaning the box also handles administrative work when the LLM is idle. For a pharma QA lead running GMP doc generation or a regulatory affairs team drafting SOPs, this is a serious production tool inside its concurrency envelope.

OwnAI position: we ship the Mac Mini configuration as the standard executive / R&D pilot tier. It is the right hardware for a proof-of-value that must be stood up quickly in a non-server-room environment. If the pilot succeeds and concurrency grows past three users, the upgrade path is to the L40S — the 4090 workstation is rarely the right intermediate step (more on that below).

RTX 4090 Workstation: the awkward middle

The RTX 4090 is a tempting hardware choice. At ₹4.2L for a complete workstation build, it is 2× cheaper than an L40S server and delivers 35–45 tok/s single-user on Qwen 3 32B Q4 — roughly 2.5× faster than the Mac Mini. With vLLM and continuous batching, eight concurrent users can sustain around 15 tok/s each. For a 5–15 user team, these are real numbers.

The problem is not the GPU. The problem is the GPU in a production context. The RTX 4090 is a gaming card. NVIDIA designed it to run at 450 W for hours-long gaming sessions in a well-ventilated consumer tower with a 120mm fan two centimetres from the heatsink. It was not designed to run at sustained 80–90% utilisation in a dusty server room AHU environment at 35°C ambient, 24 hours a day, seven days a week.

The failure modes are predictable. Thermal throttling begins at 83°C — in a warm Indian server room without dedicated AC, the card will spend a meaningful fraction of its time throttled, and the tok/s numbers you measured in a clean office do not survive the production environment. MTBF data for gaming cards in 24/7 industrial use is not published because the cards are not rated for it. NVIDIA's own warranty for the 4090 does not cover datacenter use — the warranty is void the moment the card is installed in a rack or declared as server infrastructure.

There is also an availability reality: NVIDIA officially discontinued the RTX 4090 in late 2024. Indian street pricing in May 2026 reflects an EOL premium — the GPU alone costs ₹2.2–2.55L, with customs delays and distributor markups pushing street prices to ₹2.8–3.4L for some SKUs. The natural replacement at this price point is the RTX 5090 (32 GB GDDR7, faster on all metrics, ₹3.5–4.4L for the GPU), but the 5090 carries the same "gaming card in a server room" caveat.

OwnAI position: we will ship a 4090 workstation configuration if the customer explicitly requests it and accepts a written caveat that the warranty does not cover datacenter environments and that thermal management is the customer's responsibility. We provide a specification sheet that includes the required room AC setpoint (24–26°C), UPS rating (3 kVA online minimum), and cleaning schedule. We do not default-recommend the 4090 for production. If the customer's environment is a clean office with dedicated AC and a UPS, the 4090 is a cost-effective middle tier. If the environment is a shared server room or a warm AHU space, we redirect to the L40S.

L40S Server: production-grade

The NVIDIA L40S 48 GB is OwnAI's default production recommendation for teams of 15 or more. At 60+ tok/s single-user and 30+ users above 15 tok/s with vLLM FP8 serving, it covers the realistic concurrency envelope of most Indian enterprise teams at the 32B class.

The 48 GB GDDR6 pool changes the model-serving equation. Qwen 3 32B at Q4_K_M uses ~22 GB, leaving 26 GB for KV cache — enough to serve 32K-token contexts without constraint. The Mac Mini and the 4090 both require context-capping to protect memory headroom; the L40S does not. For regulatory use cases — GMP deviation investigations, RBI credit-memo drafting, long-form SOP generation — the ability to pass a full 20K-word document through a single context window without chunking is a qualitative difference, not just a throughput number.

The L40S is an Ada Lovelace architecture card with hardware FP8 tensor cores (733 TFLOPS FP8). At high concurrency with vLLM FP8 serving, the L40S's prefill throughput advantage over the older Ampere A6000 widens significantly — the A6000 has no hardware FP8 and emulates it in software. The practical effect: the L40S handles burst prefill (a user submitting a long document for processing) without a visible pause; the A6000 does not.

Power draw is 350 W under load — less than the 4090 workstation at full utilisation. This is a server-class card in a server-class chassis. It ships with a 3–5 year warranty from major channel partners (Netweb Technologies, Locuz, Dell India, HPE India). The warranty survives datacenter deployment. The card is on every major vendor's recommended hardware list.

The India pricing range is wide: ₹5.75L (card-only, Lampo Computers Bangalore) to ₹8L through the PNY 3-year warranty channel, up to ₹15L when bundled in a Dell/HPE workstation chassis with server-grade support. Reyatech obtains three written quotes before any purchase — the spread is real, not a typo.

Above the L40S: a single A100 80 GB (₹15L+ used market in India) or H100 (₹22L+) is the right hardware for 70B-class models or multi-tenant deployments above 50 users. We do not currently quote H100 deployments as a standard configuration — the procurement complexity and the minimum support infrastructure required exceed what we can responsibly deliver as a single-team vendor today. If a customer needs H100 capacity, we refer them to Netweb or Yotta Shakti for the hardware layer and offer our fine-tuning and deployment tooling on top.

What hardware sales reps won't tell you

Every hardware vendor's benchmark suite reports single-user peak decode speed. That number is the least useful number for production planning. What matters is sustained throughput under concurrent load, and the relationship between the two is not linear.

The concurrency math is worth doing explicitly. A typical enterprise user running a 32B LLM for document generation or compliance drafting generates roughly 50,000 tokens per day. Spread across 22 working days and an 8-hour working day:

Average throughput = (50,000 tokens/day) / (22 days/month × 8 hr/day × 3,600 s/hr)
                   ≈ 0.078 tok/s per user (time-averaged)

That sounds negligible. The catch is that real usage is not uniformly distributed — it is bursty. The user is idle for 30 minutes, then submits a 4,000-token document and waits for a 2,000-token analysis. During that burst, the user's instantaneous demand is 30 tok/s. The peak-to-average ratio in typical enterprise usage is 300–500×.

This burst behaviour is why the Mac Mini's 14–18 tok/s sustained translates to a practical ceiling of 2–3 simultaneous users, not the "18 users at 1 tok/s average" a naive calculation would suggest. At the moment any two users trigger a generation simultaneously, the box is saturated and both users experience degraded throughput. The right mental model is: plan for simultaneous bursts, not time-averaged load.

Conversely, a team of 20 users does not need 20 × 30 tok/s = 600 tok/s of sustained throughput. Empirically, in a team of 20 knowledge workers, the 95th-percentile simultaneous burst is 5–8 users at any given moment. That is the number that determines your hardware tier, and it sits comfortably within an L40S envelope (30+ concurrent above 15 tok/s).

The 4090's 35–45 tok/s single-user spec looks like it should cover 10–12 users at 3–4 tok/s average. In production, with vLLM continuous batching, it handles 8 concurrent users acceptably — but only if those 8 users are not all in the middle of a burst simultaneously. In a 15-engineer team that is not guaranteed. The L40S provides the headroom to absorb simultaneous bursts without noticeable queuing.

Bottom line.

The decision tree is simple once the concurrency question is answered honestly:

1. 1–5 users, pilot or R&D context, no 24/7 requirement: Mac Mini M4 Pro 64 GB. Deploy today, ₹2.4L all-in, inference-only. If the pilot succeeds and team size grows, upgrade to L40S — do not try to intermediate with a 4090 unless the environment is a clean office with dedicated cooling.
2. 5–15 users, clean office environment, customer explicitly requests lower upfront cost: RTX 4090 workstation with written warranty-void caveat. Ensure room AC at 24–26°C setpoint, 3 kVA UPS, quarterly GPU thermal-compound service. Budget for a 3-year hardware refresh because the card will not be warrantied for server use.
3. 15–50 users, production, server room or DC environment: L40S 48 GB. This is OwnAI's default recommendation. It is the only tier with a vendor-supported warranty for datacenter deployment, the only tier that serves 32K-token contexts without context-capping, and the only tier where concurrent-user headroom is not a planning constraint at typical Indian enterprise team sizes.
4. 50+ users or 70B-class models: Multi-GPU (2× L40S or 2× RTX 5090) or A100 80 GB. Contact us — this is a custom scoping engagement.

If you want to run the numbers for your own team size and usage pattern, the OwnAI ROI calculator at /pricing#roi lets you plug in user count, daily token volume, and hardware tier and see the 3-year TCO including electricity, AMC, and hardware refresh. The calculator also shows the break-even point against cloud API for your specific workload — which is almost always the more important number than the hardware cost itself.

The related pages that provide context for this hardware decision: how OwnAI's deployment process works end-to-end (fine-tune in cloud, ship adapter to customer hardware, day-two support), and the pricing page for the three hardware tiers as bundled configurations. Book a discovery call to size the right tier for your concurrency.