Turn model selection into a filled, repeatable evaluation sheet.
You will shortlist models by task, license, format, context, hardware fit, and measured behavior, then record a repeatable decision.
Before you begin
Define one task, collect five non-sensitive test examples, and inventory RAM, VRAM, storage, operating system, and runtime support.
Start with the task
A coding assistant, a document summarizer, and an image model have different requirements. Write a short evaluation set containing real, non-sensitive examples and a clear success condition. Model leaderboards can narrow the field, but your own repeatable tasks are a better final test.
Check the model card
Review the intended use, license, supported languages, context length, architecture, training notes, and known limitations. Confirm whether commercial use is allowed if relevant. “Open weights” does not always mean unrestricted use, and a community conversion may have different packaging from the original release.
Fit the hardware
Parameter count, numerical precision, quantization, context length, and runtime overhead all affect memory. A quantized model uses fewer bytes per parameter, but quality and speed can change. Leave headroom for the operating system, application, context cache, and other processes; a model that barely loads may still be impractical.
Evaluate safely
Compare two or three candidates with the same prompts and settings. Record correctness, instruction following, latency, memory use, and failure behavior. Do not evaluate only on a polished demo. Keep the model version and quantization in your notes so results are reproducible.
Read names and variants carefully
A family may have several parameter sizes, instruct and base variants, quantizations, context settings, and community conversions. An instruct model is the usual start for chat; a base model may need specialized prompting or training. Confirm that your runtime supports the exact format and architecture. Record the full identifier, version or digest, quantization, and source so another person can reproduce the test.
Estimate fit without promises
File size is only a lower-bound clue. Runtime overhead and context cache need more memory, and GPU offload may split work between VRAM and RAM. Leave room for the operating system. A CPU-only laptop may run a small quantized model slowly; a supported GPU may accelerate one that fits; a high-memory system may permit larger weights or context. These are illustrative scenarios, not verified compatibility claims.
Create an evaluation sheet
Use the same prompt, source, context, and settings for each candidate. Score correctness against known answers, instruction following, unsupported claims, response time, memory use, and ambiguous-input behavior. Include one misleading input. Keep observations and task priorities visible rather than manufacturing one scientific-looking score. A smaller model that reliably completes a bounded task may be better than a larger model that is difficult to operate.
Licenses, privacy, and maintenance
Check model license and runtime or service terms. “Open weights” does not automatically permit every commercial use or redistribution. Verify the source and conversion. Plan updates deliberately because a new model can change output and invalidate tests. Keep a known-good model until the replacement passes the same evaluation. For cloud models, also review retention, training use, regions, quotas, and current prices.
Model size, context, and task quality
More parameters can improve some capabilities but also increase memory, storage, and compute needs. A longer advertised context can support larger documents, yet effective use may vary and the cache consumes memory. Quantization can make a model practical on smaller hardware while affecting output differently by task. Treat these as tradeoffs to test together. Do not infer that the largest model your computer can barely load is the best model for a fast, repetitive, narrow workflow.
Troubleshoot a disappointing candidate
First verify that you downloaded the intended instruct variant and that the interface uses the correct chat template. Reduce the test to a clear example and remove irrelevant context. Compare with another quantization or model only while holding prompts and settings constant. If a model fails factual tasks, retrieval from approved sources may help; if it cannot follow the required structure, choose another candidate. Preserve failed results because they define limitations and prevent repeating an unsuitable selection later.
Use catalogs without outsourcing model selection
Decision: Discover candidates in catalogs, then decide from model cards, licenses, formats, hardware fit, and a fixed evaluation set.
Relevant options: Ollama library packages models; LM Studio supports model discovery; Hugging Face model cards documents publisher metadata. Catalog presence is not certification.
Worked example: Run two fitting instruct models through five fixed tasks covering format, grounded summary, refusal, and domain use.
Failure mode: Changing prompts, quantization, context, or templates makes the comparison uninterpretable.
Validation: Record every variable, inspect licenses, score prewritten criteria, and retain failures.
Useful next step: Create the evaluation set before downloading the second candidate.
Official sources checked 2026-10-24: Ollama library · LM Studio · Hugging Face model cards. Product availability, support, terms, pricing, and licenses can change. These are vendor or project sources; no independent benchmark is claimed.
Original filled evaluation sheet
| Candidate | Recorded state | Result |
|---|---|---|
| qwen2.5:1.5b | 986 MB; Ollama 0.35.1 | Bounded prompt tested; CPU completed, GPU hit output cap |
| qwen3:4b | 2.5 GB; present locally | NOT_TESTED; no ranking |
Use the evaluation template and define expected answers before testing.
Authoritative references
Reference scope checked 2026-10-27. These sources support the concepts and risk controls; product-specific behavior still requires the nearby vendor documentation and a dated test.
Your next step
Add a candidate only after fixing the prompt, answer key, context, temperature, output limit, and acceptance rule.
Editorial provenance
Nyfir Studios is the publisher and editorially responsible organization. AI tools assisted with drafting and structured review for this candidate; factual claims, test labels, links, and retained evidence require human review before deployment. No named employee, credential, or unrecorded hands-on result is implied.