How to Evaluate an AI Development Partner

Why AI vendor evaluation needs a different framework

A traditional software vendor evaluation asks about process maturity, security controls, and delivery track record — all still relevant here. What it doesn't ask is whether the vendor understands the specific architectural tradeoffs an LLM-based system forces: when to ground answers in retrieved documents versus when to change model behavior through training, how to measure whether the system is actually correct rather than merely fluent, and how to keep proprietary or regulated data from leaking into a third-party model provider's training pipeline. A vendor can hold every general software certification and still lack any of this — AI engineering is a distinct discipline from general application development, and the evaluation has to test for it directly.

Ask whether they correctly diagnose RAG versus fine-tuning

Describe your actual use case and ask the vendor to explain, unprompted, whether they'd approach it with retrieval-augmented generation or fine-tuning, and why. RAG fits problems involving facts that change or need source attribution; fine-tuning fits problems requiring a specific output style, format, or specialized behavior the model doesn't naturally produce. A vendor that proposes fine-tuning for a fact-lookup problem — or RAG for a pure style-transfer task — hasn't actually diagnosed your requirement, they've defaulted to whichever technique they're more comfortable building.

The strongest vendors will also raise the option most buyers don't consider: a hybrid approach combining a fine-tuned model with RAG grounding, appropriate when you need both a specific behavioral style and access to current, citable facts. See our full RAG vs fine-tuning comparison for the complete decision framework you can use to check their reasoning against.

Ask exactly where your data lives, at every stage

Every AI system built on your data creates several places that data can end up: the vector database storing document embeddings, the logs of prompts and model outputs, and potentially the training data of a third-party model provider if usage terms aren't scoped carefully. Ask specifically whether embeddings and retrieval indexes are hosted in your own cloud environment (VPC) or the vendor's shared multi-tenant infrastructure, whether prompts and outputs are logged anywhere outside your control and for how long, and — critically — whether the model provider's terms of service permit your data to be used for training their base model.

For any engagement touching proprietary or regulated data, the answer should be that everything stays inside a boundary you control. A vendor unable to answer these questions precisely, or that hasn't considered them, is not equipped to handle sensitive data regardless of how capable their engineering otherwise appears.

Ask for a real number on hallucination rate or accuracy

Any vendor can claim their AI systems are "reliable" or "accurate" — this is unfalsifiable marketing language unless backed by a number. Ask directly: on your most recent production AI deployment, what was the measured accuracy or hallucination rate against a defined evaluation set, and how was that evaluation set constructed? A vendor with a genuine evaluation methodology will have this number immediately available, along with a description of how they built the test set and how they distinguish a model error from a retrieval error. A vendor that responds with only qualitative reassurance — "we test thoroughly," "our system is highly accurate" — without a specific figure has not actually built the measurement infrastructure that production AI systems require.

Check whether they can connect AI to your actual systems

Most enterprise AI value comes from connecting a model to internal systems — your CRM, ticketing system, or proprietary database — not from the model in isolation. Ask how the vendor handles this integration layer: whether they build custom, one-off integrations per tool, or use a standardized approach like Model Context Protocol that scales across multiple tools and, if you later switch model providers, doesn't require rebuilding every integration from scratch. Also ask how tool access is scoped — whether the AI agent's permissions on your internal systems are least-privilege by default, or whether it has broad access that's harder to audit and constrain.

Verify actual ML engineering depth, not just API integration skill

Calling a hosted LLM API and building a RAG pipeline are valuable, real skills, but they're meaningfully different from training or fine-tuning a model, designing an evaluation framework, or debugging why a retrieval system returns irrelevant results at scale. Ask what fraction of the specific team assigned to your project has actually trained or fine-tuned a model, versus only integrated with a third-party API. For projects that only need RAG and integration work, pure API skill may be entirely sufficient — the mismatch to watch for is a vendor claiming deep AI expertise while the actual team has never done anything beyond prompt engineering and API calls.

Scope a proof of concept before committing full budget

AI projects fail more often from data or approach mismatches than from engineering execution problems — a technique that works well in a demo can fail on your specific, messier real data. Insist on a time-boxed proof of concept, typically 3-6 weeks and 10-15% of expected total budget, tested against a representative sample of your actual data with a defined, measurable success threshold agreed before work starts. A vendor resistant to this structure, or pushing directly to a full production build without validation, is asking you to bet the full budget on an unverified assumption. See how to scope an AI project before committing budget for the complete framework.

Ask about model provider lock-in and portability

A system built around a single model provider's specific API is exposed to that provider's pricing changes, rate limits, and any future deprecation of the exact model version the system depends on. Ask whether the vendor's architecture abstracts the model layer — meaning the prompt structure, retrieval logic, and integration layer would survive a switch to a different provider with limited rework — or whether provider-specific behavior is baked in throughout the system in ways that would require a substantial rebuild to change.

This isn't a demand that every system be provider-agnostic from day one — a startup validating a concept may reasonably accept some lock-in for speed. It matters more for systems expected to run in production for years, where a provider's pricing model, rate limits, or context window changes years from now could otherwise force an expensive, unplanned rebuild. Ask the vendor directly what would be involved in switching providers a year after launch, and listen for whether they've actually thought about this or are answering the question for the first time in front of you.

Ask how they handle model drift after launch

Unlike traditional software, which behaves the same way indefinitely until the code changes, an AI system's behavior can shift over time even without a code change — the underlying model provider may update the model version behind an API, the distribution of real user queries may drift from what the system was originally tuned against, or the documents in a RAG knowledge base may grow stale. A vendor with real production experience will have a concrete answer for how they monitor this: tracking accuracy or hallucination rate over time against a held-out evaluation set, alerting on retrieval quality degradation, and a defined process for when and how the knowledge base or fine-tuning data gets refreshed.

Ask specifically whether post-launch monitoring and maintenance is included in the engagement or billed separately, since AI systems generally require more ongoing attention than a typical CRUD application — a vendor that treats launch as the finish line, with no plan for what happens in month four when the model provider silently updates behavior, is underestimating what production AI actually requires.

Match evaluation depth to what's actually at stake

Not every AI use case carries the same risk if it's wrong, and evaluation rigor should scale accordingly. An internal tool summarizing meeting notes for employee convenience tolerates occasional errors with low consequence — thorough vendor vetting still matters, but the evaluation bar for hallucination rate and auditability can be lower. A customer-facing system making claims about pricing, eligibility, or medical information, or an agentic system that can take real actions on your systems (placing orders, modifying records, sending communications), carries materially higher stakes, and the vendor's evaluation methodology, guardrails against the system acting outside its intended scope, and human-review checkpoints before consequential actions deserve correspondingly more scrutiny. Ask the vendor how their proposed evaluation and safeguards would change between a low-stakes internal tool and a high-stakes customer-facing or agentic system — a vendor with real experience will describe a different, more rigorous process for the latter without being prompted to explain why.

Red flags specific to AI vendors

• Claiming zero hallucination — every LLM-based system has failure modes; a vendor unwilling to discuss theirs hasn't shipped anything to production
• No evaluation methodology — inability to produce an accuracy or hallucination-rate figure from a past deployment
• Defaulting to one architecture regardless of the problem — always proposing fine-tuning, or always proposing RAG, without diagnosing which fits
• Vague answers about data residency — can't specify where embeddings, logs, or prompts are stored
• No production deployments over six months old — only demos and proofs of concept, nothing that's survived real usage and drift
• Pushing straight to full build — skipping a scoped proof of concept against your real data

Ask how they defend against prompt injection and adversarial inputs

A prompt injection attack occurs when a user (or content retrieved by the system, such as a document in a RAG pipeline) contains instructions designed to override the system's intended behavior — tricking the model into ignoring its original instructions, revealing internal system prompts, or taking actions outside its intended scope. This is a genuinely different threat category from traditional web application security, and a vendor without a specific answer here is likely unaware of the risk class entirely, not just declining to discuss it.

Ask concretely: how do they handle user input that attempts to override system instructions, how do they sanitize or validate content retrieved into a RAG pipeline (documents, web pages, or database records the system reads that could themselves contain injected instructions), and what happens if an agentic system with tool access receives an instruction attempting to make it take an unauthorized action. A vendor with real production experience will describe specific mitigations — input/output filtering, tool-permission scoping, instruction-hierarchy enforcement — rather than treating the question as hypothetical or irrelevant to their work.

Confirm the team behind their case studies is the team on your project

AI engineering talent is scarce and unevenly distributed even within a single vendor organization — a firm's strongest ML engineer may have built the impressive case study you were shown, while a much less experienced team is actually staffed on new client work. This gap is more consequential in AI than in general software development, since the difference between an engineer who has actually trained and evaluated models and one who has only called an API shows up directly in system quality. Ask by name who built the specific case study project that impressed you, and confirm whether that same person — or someone with comparably deep experience — is the one who would actually be assigned to your engagement, not a substitute with a thinner AI background brought in to free up the senior engineer for other work.

Understand the cost structure before comparing quotes

AI engagements carry a cost layer standard software projects don't: variable compute cost for model inference, embedding generation, and vector database hosting, which scales with usage rather than being fixed at delivery. Ask the vendor to separate the one-time engineering fee from the projected ongoing compute cost at your expected usage volume — a low engineering quote paired with an unexamined compute cost can end up more expensive over a year than a higher engineering quote with efficient, well-architected inference. This is also where RAG versus fine-tuning matters for cost, not just architecture: RAG's per-query retrieval and generation cost scales with usage in a way that's straightforward to estimate, while fine-tuning has a larger upfront training cost but potentially lower per-query inference cost depending on model size chosen.

Additional requirements for regulated industries

Healthcare and fintech AI deployments need everything above, plus explicit confirmation that retrieval indexes and model context stay within your compliance boundary rather than a shared multi-tenant AI service, and a clear answer on how the vendor prevents PHI or financial data from being sent to a third-party model provider in a way that violates your Business Associate Agreement or data-handling requirements. Ask specifically whether they've deployed AI systems under HIPAA or PCI-DSS scope before, not just general enterprise AI experience — the audit logging, access control, and data-minimization requirements in regulated AI deployments are meaningfully stricter than a standard commercial AI project.

In-house versus partner: know which question you're actually answering

Before evaluating specific AI vendors, confirm you've made the right call on partner versus in-house team in the first place — the criteria differ meaningfully depending on whether AI is a core, permanent capability for your business or a supporting feature. See should we build AI in-house or hire a partner for the complete decision framework, including the hybrid model most companies scaling past their first AI feature actually use.

AI-specific contract terms beyond standard software agreements

Standard software contract terms around IP ownership and liability don't map cleanly onto AI engagements, which introduce questions a generic contract template rarely addresses. If the vendor fine-tunes a model on your data, confirm explicitly who owns the resulting fine-tuned model weights — some vendors retain rights to reuse techniques or even weights across clients unless the contract specifies otherwise, which can matter significantly if a fine-tuned model represents genuine competitive differentiation for your business. Confirm your training and evaluation data isn't retained or reused by the vendor for other clients' projects once the engagement ends, and get explicit contract language on liability allocation for incorrect AI-generated outputs — particularly for any customer-facing system, where an AI-generated error could carry real business or legal consequences depending on your industry.

AI vendor scoring rubric

If you only have time for three questions

Full vendor evaluation takes weeks, but a compressed timeline still benefits from asking the three highest-signal questions from this framework. First, ask for a specific accuracy or hallucination-rate figure from a past production deployment — an unfalsifiable "we're accurate" answer versus a specific number with a described evaluation methodology separates vendors faster than almost anything else. Second, ask exactly where your data lives at every stage, since this alone rules out vendors unprepared for regulated or proprietary data regardless of their technical skill. Third, ask who specifically would be assigned to your project and confirm that person has real model training or evaluation experience, not just API integration — this single question surfaces the gap between a vendor's marketed AI expertise and their actual bench strength faster than any case study review.

What Code Ninety does

Code Ninety diagnoses RAG versus fine-tuning based on the client's actual problem rather than defaulting to one approach, keeps retrieval indexes and embeddings inside the client's own VPC for regulated data, and scopes every AI engagement with a defined proof-of-concept phase and an explicit, agreed success threshold before committing to full production build costs. Code Ninety builds MCP-based integrations for connecting AI agents to internal enterprise systems under the same least-privilege access controls applied to every other integration under SOC 2 Type II. Code Ninety is an AI development company in Pakistan shipping production systems, not pilots. See the AI automation case study for a production deployment of these patterns.

Frequently asked questions

Related reading

• → Retrieval-augmented generation
• → Model Context Protocol
• → How to choose a software development company