A case study gets risky when the writing outruns the evidence.
That risk matters most when you are the buyer. A polished case study can make an agency look proven before you have learned what was commissioned, what reached production, what changed for the client, or which sentences are simply the agency's interpretation of its own work.
You do not need to audit every footnote. You do need to notice when the proof changes category halfway through the story.
The claim record in the image above shows the basic distinction. An approved flow and a release commit can prove that seven steps became four. They cannot prove that activation increased. That second claim needs behavior data tied to the released flow.
Here are three common tells that a case study is asking you to believe more than its evidence supports.
The production fact becomes a business outcome
The agency shows a real artifact: a redesigned checkout, a new identity, a faster site, a campaign launch, or a simplified onboarding flow. Then the writing jumps from what shipped to what the business achieved.
What the evidence proves: The shipped onboarding flow was reduced from seven steps to four.
What the case study claims: The new flow increased activation.
What is missing: Activation data, a comparison period, the population measured, and enough context to connect the change to the work.
This does not mean the design was ineffective. It means the visible evidence proves production, not performance.
Watch for verbs that quietly make this leap: increased, drove, transformed, accelerated, unlocked, lifted, and converted. Each implies an observed change. Screenshots, deliverable lists, launch dates, and repository records do not establish that change on their own.
A careful agency will either provide the result with context or keep the sentence inside the production boundary: the flow was shortened, the page was launched, the system was delivered, or the campaign was produced. That restraint is useful evidence too. It shows the team knows the difference between making something and measuring what happened next.
The internal decision becomes a client quote
Case studies often contain strong strategic language: “The brand needed to feel established, not exclusive,” or “The new message gave customers a clearer reason to choose the product.” The sentence may be smart and the decision may be visible in the work. But whose conclusion is it?
What the evidence proves: The agency documented a positioning decision and applied it across the identity and website.
What the case study implies: The client agreed that this was the business problem and confirmed that the work solved it.
What is missing: An attributed, approved client statement or another source showing that the conclusion belongs to the client.
Look closely at quotation marks, pull quotes, and phrases such as “the team realized,” “stakeholders agreed,” or “the client saw.” A real client quote should have a real speaker, role, and approval. If the client must remain anonymous, the agency should still be able to explain the relationship and how the statement was approved without exposing confidential details.
An agency is allowed to have a point of view. It should label that point of view as its own. Strategy becomes suspect when internal rationale borrows the authority of client testimony.
The concept is framed like commissioned work
Self-initiated and fictional projects can be excellent evidence of taste, range, and execution. They become misleading when the page lets you infer a client relationship that never existed.
What the evidence proves: The agency created a brand concept, prototype, or coded demonstration.
What the case study implies: A named company hired the agency, approved the direction, launched the work, or achieved the stated result.
What is missing: A commissioned relationship, client approval, production adoption, or real-world result.
The tell is often not a direct false statement. It is the omission around the first impression. A company name sits beneath “Selected work.” A polished mockup appears beside “for.” The disclosure arrives at the bottom, after the page has already borrowed the credibility of a client engagement.
The right disclosure should appear early and plainly: self-initiated concept, fictional brand, unsolicited redesign, internal prototype, or speculative exploration. Then judge the work for what it can honestly prove. A concept can demonstrate the agency's craft and decisions. It cannot demonstrate client collaboration, stakeholder management, market adoption, or business impact.
Questions to take into the pitch meeting
You do not need to turn the meeting into a cross-examination. Pick the questions that matter for the kind of risk you are trying to reduce.
- Was this commissioned by the client, and which parts of the work did your team own? This separates a real engagement from a concept and clarifies whether the agency led the work or contributed one piece.
- What actually reached production? Ask which screens, messages, assets, or systems launched, and which remained recommendations or prototypes.
- Which result can you connect directly to the work? A credible answer names the metric, period, comparison, and limits of attribution.
- What evidence sits behind that number or claim? You may not be able to see confidential data, but the agency should be able to describe the source and who approved its use.
- Is that the client's conclusion or your team's interpretation? This is especially useful when a strategic insight sounds like customer or stakeholder feedback.
- What did the client approve you to say publicly? The answer reveals how the agency handles consent, confidentiality, and credit.
- What did this case study leave out? Strong teams can name a constraint, an unmeasured result, an unshipped recommendation, or a part of the engagement they did not own.
- What would you do differently for our situation? This tests whether the agency can transfer judgment without pretending that one past project is a universal formula.
Listen for answers that keep fact, interpretation, and outcome separate. “We shipped it” is not the same as “customers used it.” “We recommended it” is not the same as “the client approved it.” “We designed it” is not the same as “we were hired to design it.”
The boundary is part of the proof
The strongest case study is not the one with the most confident language. It is the one that lets you see where confidence comes from.
Good evidence can be modest: an approved deliverable, a production URL, a dated test, a client statement, or a clearly labeled concept with inspectable craft. What matters is that the story does not promote one kind of evidence into another.
When an agency names what is real, what is inferred, what was commissioned, what shipped, and what remains unmeasured, the boundary itself becomes evidence of how the team will handle your work. You are not only evaluating the project on the page. You are evaluating the agency's judgment when the story is commercially useful.
