Compare two texts with score thresholds your team can use

A content editor sees a similarity score of 78%. One person calls it plagiarism. Another approves the draft because the wording looks different. A third reruns the comparison in another tool and gets 61%.

All three reactions miss the operating issue: a percentage has meaning only when the team knows which method produced it, what was removed before comparison, how long the texts were, and which decision rule applies.

A similarity score is evidence for a review. It is not a plagiarism verdict.

That distinction matters for content teams working on SEO, Answer Engine Optimization, editorial quality, legal review, and AI-assisted production. Near-duplicate pages can carry conflicting product claims, split internal attention, and leave teams unsure which URL should be maintained. A semantic match can also reveal two pages that answer the same buyer question with different wording. An exact-match tool may miss that case entirely.

This guide gives you a practical way to compare two texts, set score thresholds, deal with paraphrases and short passages, protect unpublished material, and record decisions that can still be audited after publication.

What a text similarity comparison actually measures

A text similarity checker compares two passages and returns a measure of how much they resemble each other based on a defined method. The method may compare characters, words, word sequences, sentence structure, or semantic meaning.

The core principles are straightforward:

• Lexical overlap measures shared characters, tokens, or phrases. It is useful for finding copied wording and lightly edited duplicates.
• Semantic similarity measures whether two passages express related meanings, even when they use different words or sentence order.
• Normalization changes the input before scoring. Lowercasing, removing punctuation, stripping references, stemming words, or deleting boilerplate can materially change the result.
• Text length affects confidence. Five shared words in a ten-word snippet mean something different from five shared words in a 1,000-word article.

Mini-example:

Text A: “The platform tracks brand mentions in AI answers.”
Text B: “The software monitors when AI responses mention your brand.”

The sentences have limited exact phrase overlap but strong semantic similarity. A token-overlap method may return a moderate score. A suitable embedding model may return a high score.

Similarity also differs from duplicate-content handling. Google explains that when it finds duplicate or very similar pages, it selects a representative canonical URL, while site owners can signal their preferred version. Its guidance on consolidating duplicate URLs is useful for the publishing side of this decision. A similarity score can help you find candidates for review, but it does not tell Google which page to index.

Plagiarism is a separate judgment involving attribution, permission, authorship, and context. A properly quoted paragraph may have 100% lexical similarity and no plagiarism problem. A close paraphrase without attribution may have modest lexical overlap and still require academic, editorial, or legal review.

Why one percentage produces conflicting decisions

Most comparison pages put two input fields above the fold, return a percentage, and leave the user to invent the policy. That design makes the result look more certain than it is.

Consider a score of 0.78. Its meaning changes under each of these conditions:

• A character matcher found the same words with punctuation changes.
• A five-word n-gram matcher found long shared phrases.
• A Jaccard calculation found many shared tokens but ignored word frequency.
• A cosine calculation compared term-frequency vectors.
• An embedding model found similar meaning despite different wording.
• The tool converted a raw value into a percentage using an undisclosed formula.

The content may also contain a product disclaimer, navigation copy, legal text, references, or a standard author biography. Those repeated blocks can raise lexical scores even when the main arguments differ.

Short text is especially unstable. If two product headings share four specialist terms, the score may look high because there is little other text. That could be expected terminology rather than copied writing. Conversely, a long article can contain one copied paragraph while receiving a low document-level score because the copied section is diluted by the rest of the article.

The practical response is to store the method with the result. “78% similar” is incomplete. “0.78 cosine similarity using embedding model version X, after stripping navigation and references, on two passages of 642 and 711 words” is reviewable.

How text similarity is calculated

There is no single universal formula for how text similarity is calculated. Most systems follow a process like this:

1. Accept and decode the inputs. The system extracts text from pasted content or supported files. File conversion errors, hidden text, and broken character encoding should be caught here.
2. Normalize according to a saved profile. The profile may lowercase text, standardize spaces, normalize Unicode characters, or remove selected boilerplate. The original text must remain available to the reviewer.
3. Split the text into comparison units. Units may be characters, words, sentences, or n-grams. A five-word n-gram is often called a five-word shingle.
4. Calculate one or more raw measures. Common measures include Jaccard overlap, edit distance, sequence matching, term-vector cosine similarity, and embedding cosine similarity.
5. Apply length and section rules. The system may compare whole documents, sliding windows, or corresponding sections. This prevents one copied passage from disappearing inside a long-document average.
6. Return the score with method details. A useful result includes matched passages, input lengths, normalization settings, model version, threshold version, and any warnings.

Token and n-gram overlap

Token overlap asks how many words the texts share. N-gram overlap asks how many sequences of adjacent words they share. Longer n-grams are better at finding copied phrasing, but they are easier to break through minor edits.

The Jaccard coefficient is:

shared items / all unique items across both texts

If Text A contains the unique tokens {brand, ai, answer, track} and Text B contains {brand, ai, response, monitor}, the intersection has two items and the union has six. The Jaccard score is 2/6, or about 0.33.

Stanford’s Introduction to Information Retrieval explains the Jaccard coefficient and the vector space model used for cosine comparisons. These are useful method references because they show what the formulas calculate without pretending that one threshold fits every editorial case.

Cosine similarity for term vectors

A term-vector method records word frequency. Cosine similarity measures the angle between two vectors. Texts with similar term patterns point in similar directions, even if one text is longer.

This can work well for documents that discuss the same concepts with overlapping vocabulary. It can also overrate two unrelated pages in a specialist category because both contain the same necessary terms.

Embedding-based semantic similarity

An embedding model converts text into numerical vectors intended to represent meaning. Cosine similarity then compares those vectors.

This method can identify paraphrases such as “cut editorial review time” and “reduce the minutes editors spend checking drafts.” It may also group texts that discuss the same subject but make different claims. That false positive matters in product, legal, and medical content.

Model choice matters for translation. A multilingual embedding model may place equivalent sentences from two languages close together. A monolingual model may not. The result must record the model and language pair.

A worked score decomposition

Take these two passages:

Text A: “Content teams should record the method, score, threshold version, and reviewer decision.”

Text B: “Editors need an audit record containing the comparison approach, result, policy version, and final human judgment.”

A mock composite result might look like this:

Normalization profile: editorial-v3
Text lengths: 14 tokens / 16 tokens
Token Jaccard score: 0.58, weight 35%
Five-word sequence score: 0.31, weight 25%
Embedding cosine score: 0.89, weight 40%
Weighted composite score: 0.64
Decision band: human review
Warnings: short passage; specialist terms present

The composite calculation is approximately:

(0.58 × 0.35) + (0.31 × 0.25) + (0.89 × 0.40) = 0.64

Those weights are illustrative. They are not a market benchmark. Your policy should state why each component exists and test the result against examples your reviewers have already labeled.

The decomposition explains why the score is moderate. Meaning is closely aligned, while exact phrase and sequence overlap are lower. A bare “64%” would conceal that distinction.

Score thresholds your content team can start with

Thresholds should trigger actions, not labels such as “safe” or “plagiarized.” The following bands are a starting policy for internal editorial triage. Calibrate them with your own corpus before automating publication decisions.

For normalized five-word shingle overlap

• 0.70 to 1.00: High lexical overlap. Block automatic publication, inspect matched passages, and decide whether to merge, quote, cite, rewrite, or preserve approved standard language.
• 0.40 to 0.69: Moderate overlap. Route to an editor when shared sequences occur in the main body. Boilerplate-only matches can follow an approved exception.
• 0.00 to 0.39: Low document-level overlap. Check section-level results before approval, especially for long documents.

For embedding cosine similarity

• 0.90 to 1.00: High semantic proximity. Review whether both texts answer the same intent, repeat the same claim, or differ in a way that the score misses.
• 0.80 to 0.89: Related meaning. Review for search-intent overlap, paraphrase risk, and conflicting facts.
• Below 0.80: Lower semantic proximity under this starting policy. A low whole-document score does not clear individual passages.

Embedding systems differ, so these numbers cannot be transferred blindly from one model to another. A model upgrade creates a new threshold version and requires another validation run.

Additional rules by text length

• Under 50 words: Do not auto-approve or auto-reject. Show the matched terms and require a person to review the context.
• 50 to 299 words: Use passage-level output and a stricter review rule. One shared paragraph may form most of the text.
• 300 words and above: Compare the whole document and overlapping windows. A useful window might contain 150 to 300 words, depending on content structure.
• Large files: Split on headings first, then use overlapping windows inside long sections. Record both the maximum window score and the document score.

Three annotated outcomes

High overlap example:

A: “Paid plans include a 14-day trial.”
B: “The paid plans include a 14-day trial.”

Expected interpretation: Very high lexical and semantic similarity. This may be approved product wording rather than a problem. Confirm that both pages should carry the same current claim and assign one source of truth.

Moderate overlap example:

A: “Measure how often your brand appears in AI-generated buying recommendations.”
B: “Track the frequency with which AI assistants recommend your company to buyers.”

Expected interpretation: Moderate lexical overlap and high semantic similarity. Review the pages for intent duplication, claim consistency, and whether one page should own the topic.

Low overlap with a hidden issue:

Two 1,500-word articles receive a document score of 0.24, but one 120-word window scores 0.93 on five-word shingles.

Expected interpretation: The document-level result is low, yet the matched passage needs review. This is why teams should store maximum passage scores rather than one average.

Similarity is a conditional calculation, not a plagiarism threshold

A team policy should answer four separate questions:

1. How much wording overlaps?
2. How close is the meaning?
3. Is the overlap expected and permitted?
4. What publishing action follows?

A quotation, legal disclaimer, and product specification can create acceptable exact matches. An unattributed translated paragraph can avoid lexical matching while preserving the original work. A semantic model may find it, provided the model supports both languages, but the final judgment still needs context.

For SEO and Answer Engine Optimization, the useful question is often broader than plagiarism: do two pages answer the same buyer query, and do they give AI systems a consistent, citable answer?

AEO concerns how a brand becomes present, accurately represented, and cited in answers generated by AI systems. Text comparison helps content governance by finding repeated answers, conflicting definitions, and pages that compete for the same editorial ownership. It cannot tell you whether an AI assistant actually mentions the brand. That requires observation through AI visibility tracking.

Put every decision in a log

A score policy becomes useful when a future editor can reconstruct the decision. Record at least:

• Comparison ID
• Source document IDs or approved file names
• Input word or token length for each text
• Language detected or supplied
• Normalization profile and version
• Comparison method and model version
• Whole-document score
• Highest passage-level score
• Threshold policy version
• Matched passages shown to the reviewer
• Reviewer decision and reason code
• Follow-up action, such as merge, cite, rewrite, approve, or legal review
• Retention expiry date
• Timestamp and reviewer identity

Suppose a page is published after receiving an embedding score of 0.78. Six months later, a stakeholder asks why it was allowed. The log should show that policy version 2.1 treated 0.78 as below semantic escalation, the five-word shingle score was 0.12, no passage exceeded 0.40, and an editor approved the topic distinction. Without that record, the team is arguing from memory.

Decision codes keep reporting useful. Examples include APPROVED_STANDARD_COPY, MERGE_WITH_CANONICAL, CITATION_REQUIRED, PARAPHRASE_REVIEW, FALSE_POSITIVE_JARGON, and ESCALATE_LEGAL. Free-text notes can supplement these codes but should not replace them.

Privacy rules belong next to the input field

Unpublished launch copy, legal drafts, customer stories, sales enablement, and board material should not be pasted into an unknown service. The privacy review should happen before a team adopts a comparison tool or API.

Ask the provider these questions:

• Is input stored after the comparison finishes?
• Is submitted text used for model training or product improvement?
• What is the default retention period?
• Can an administrator set retention to zero or a fixed number of days?
• Which subprocessors receive the text?
• In which regions is data processed and stored?
• Is data encrypted in transit and at rest?
• Can users delete a comparison and its stored files?
• Are logs separated from raw content?
• What happens to backups after deletion?
• Can sensitive fields be redacted before processing?

A sound operating preset for routine editorial work is short retention for raw text, longer retention for score metadata, and a documented deletion date. Sensitive legal or unreleased product material may require a private deployment, a provider approved by security, or a local comparison method.

Do not write “we never store your text” near an input box unless the full processing path supports that statement. Temporary queues, error logs, backups, and subprocessors count. Plain privacy language earns more trust than a broad promise that cannot survive a security review.

A practical workflow for one-off, regular, and bulk comparisons

Quick: compare two texts online

For a one-off check:

1. Remove page furniture that has no bearing on the decision, such as navigation, cookie text, and repeated footer copy.
2. Keep quotations and standard disclaimers, but mark them so the reviewer can classify expected overlap.
3. Run one lexical method and one semantic method.
4. Inspect matched passages before acting on the overall score.
5. Save the method, threshold version, decision, and deletion date.

This process takes longer than accepting a single percentage, but it prevents routine false alarms.

Regular: an editor or SEO team checklist

Use one saved normalization profile for each content class. Blog posts, product pages, policy documents, and academic papers should not share a policy by default.

A practical pre-flight routine is:

1. Convert Unicode characters to a standard form.
2. Trim leading and trailing spaces.
3. Replace repeated whitespace with one space.
4. Lowercase a comparison copy, while preserving the original.
5. Remove approved navigation and footer blocks.
6. Keep headings and sentence boundaries.
7. Mark quotations, citations, code, and legal boilerplate.
8. Record both original and normalized lengths.

Do not remove stopwords automatically without testing. Words such as “not,” “without,” and “except” can reverse meaning. Stemming can also merge terms that editors need to distinguish.

For duplicate-content decisions, attach the comparison to the content brief or editorial ticket. If the page proceeds, assign one owner for shared product facts and a review date. If the pages address the same intent, consider merging them and using the correct canonical and redirect rules after technical review.

Scale: text similarity API and bulk text comparison

A batch process should compare candidate pairs rather than every document against every other document without a retrieval stage. Start by finding likely candidates through title, topic, keyword, or embedding retrieval. Run the more expensive passage comparison on that shortlist.

A pseudo-request might look like this:

{
  "document_a": {
    "id": "page-1842",
    "language": "en",
    "text": "..."
  },
  "document_b": {
    "id": "page-2091",
    "language": "en",
    "text": "..."
  },
  "normalization_profile": "product-pages-v4",
  "methods": ["jaccard_5gram", "embedding_cosine"],
  "window_words": 200,
  "window_overlap_words": 50,
  "retention_days": 7,
  "threshold_policy": "editorial-2026-02"
}

The response should return separate method scores, top matching windows, warnings, and version identifiers. Avoid an API design that returns only similarity: 78.

For file handling, extract DOCX, PDF, HTML, and plain text into a common internal format. Flag scanned PDFs that need optical character recognition. Preserve heading boundaries and page references so reviewers can locate matches. Reject password-protected or corrupted files with a clear error instead of calculating against partial text.

At this stage, access control matters. Editors may need score metadata without permission to open legal drafts. Keep raw content access narrower than report access.

The brand AI visibility floor example

Picture a weekly content and revenue review. The marketing team has two live pages:

• A product page says the platform “tracks brand mentions and citations in AI answers.”
• A category guide says the platform “measures brand visibility across AI-generated recommendations,” but it lists an older set of supported systems.

A lexical comparison returns 0.42 because the sentences differ. An embedding comparison returns 0.91 because both pages answer nearly the same question. Passage review finds the conflicting support claim.

The team should not treat 0.91 as a plagiarism accusation. It is an ownership signal. One page needs to become the approved source for that product fact. The other should quote, summarize, or link to the maintained source according to its purpose.

Next, the team checks actual prompts that buyers use, such as “Which platforms track brand citations in AI answers?” and “How can a CMO measure visibility in ChatGPT-style recommendations?” This is where a content comparison workflow meets the broader Targetlytics operating model. Similarity analysis governs what the company publishes. AI visibility measurement checks what answer engines say in practice.

An AE or SDR supporting this review should ask discovery questions tied to business use rather than asking only whether the prospect wants a checker:

• Which content types create the most duplicate-review work today?
• Who decides whether two pages should be merged, rewritten, cited, or left alone?
• Which claims create risk when they differ across product, sales, and support content?
• How often does an editor rerun a check because the first score lacks method details?
• Do you need exact-copy detection, semantic intent overlap, or both?
• Which documents may contain customer data or unreleased product details?
• What retention period has security approved?
• Where should an escalation appear: the CMS, a ticket, Slack, or a legal queue?
• How do you currently connect content changes to AI mentions, citations, and influenced pipeline?

These questions expose pipeline leakage in the workflow. A generic checker may satisfy an individual writer while leaving stakeholders without an audit record, security policy, or agreed publishing action.

Where this approach works best, and where it is less effective

Works best for

SEO and content teams

Use it to find overlapping articles, repeated product descriptions, competing intent, and sections that need one maintained source. After a moderate or high score, inspect passages and decide whether to merge, cite, rewrite, or preserve approved copy.

Editors and publishers

Use it for pre-publication triage, syndication checks, version comparisons, and boilerplate management. Escalate unattributed close paraphrases or disputed ownership to the correct editorial or legal process.

Students and academics

Use it to spot close wording between a draft and known source material. Use a full plagiarism service when the task requires comparison against a broad external corpus, citation databases, institutional records, or unpublished submissions.

Legal and compliance teams

Use it to find changed clauses, repeated disclosures, or semantically similar claims across approved material. Keep human review mandatory because one changed word can alter an obligation.

Developers and content operations teams

Use a text similarity API for CMS checks, migration audits, policy drift, and batch comparisons. Require versioned methods, role-based access, retention controls, and passage-level evidence.

Less effective for

This approach is weaker when the source corpus is unknown, the text is extremely short, authorship is disputed, the language pair is unsupported, or the decision depends on legal context. It also performs poorly when teams demand one threshold across product copy, contracts, essays, support articles, and translated pages.

It fails in these contexts because resemblance alone cannot establish provenance, permission, intent, or legal responsibility.

Six mistakes that distort the score

1. Comparing raw pages with all boilerplate included

Cookie notices, menus, disclaimers, and footers can create false positives.

Do this check: run the comparison once with approved boilerplate marked and once without it. If the decision band changes, report both scores and classify the repeated block.

2. Removing too much during normalization

Aggressive stopword removal or stemming can erase negation and product distinctions. “Does store customer text” and “does not store customer text” must never collapse into the same operational meaning.

Do this check: inspect the normalized text beside the original before approving the profile.

3. Trusting a whole-document average

A copied section can hide inside a long article and produce a low overall score.

Do this check: calculate overlapping windows and store the highest passage score with the document score.

4. Treating specialist vocabulary as proof of copying

Two cybersecurity pages will share terms such as encryption, access control, identity provider, and audit log. That overlap may be unavoidable.

Do this check: compare phrase sequences and sentence structure, then ask whether the shared terms have reasonable substitutes.

5. Using a semantic score as a factual-equivalence test

Two sentences can discuss the same subject while contradicting each other. Embeddings may still place them close together.

Do this check: route high semantic matches involving prices, retention, legal terms, integrations, and product support to claim-level review.

6. Changing the model without changing the policy version

Embedding updates can move scores even when the texts stay fixed. Old decisions then become hard to reproduce.

Do this check: rerun a labeled validation set after every method, model, or normalization change.

A useful check: if reviewers cannot explain why a score crossed the threshold, method-level reporting and policy versioning are probably not happening.

Measure the time returned to the team

Do not borrow a generic productivity claim for this workflow. Measure the baseline inside your own editorial operation.

Suppose the team handles 40 comparisons per week. Before a policy exists, each routine case takes 14 minutes because an editor reruns tools, asks what the percentage means, and records the outcome manually. After the team adds fixed normalization, method-level scores, and decision bands, routine triage takes four minutes.

The conditional calculation is:

40 comparisons × 10 minutes removed = 400 editorial minutes per week

That is six hours and 40 minutes returned to other work. It is an operating estimate, not a promised market result.

Measure at least four weeks before and after the change. Separate routine cases from escalations, because legal and authorship disputes should not be rushed. Useful measures include median triage time, percentage sent to human review, percentage of reviewer decisions later reversed, and number of results missing required log fields.

Revenue operations leaders should also inspect where delay matters. If a product launch waits two days because stakeholders are debating an unexplained percentage, the cost is tied to launch flow and sales readiness. If the comparison concerns an old blog post with little traffic or buyer relevance, the same delay has less commercial weight.

Questions marketing managers ask in practice

Does a high similarity score mean plagiarism?

No. It means the selected method found strong resemblance. Review attribution, permission, quotations, standard wording, and provenance before making a plagiarism judgment.

What percentage of text similarity is considered high?

There is no universal percentage. As a starting triage policy, five-word shingle overlap of 0.70 or more and embedding cosine similarity of 0.90 or more can trigger review, but each method and content class needs validation.

How accurate is a similarity checker between two texts?

Accuracy depends on the labeled decision you expect it to reproduce. Test the system against examples that your editors have classified, then report false positives and false negatives by content type, method, language, and length.

Can paraphrasing bypass detection?

Paraphrasing can reduce character, token, and n-gram overlap. A semantic model may still find similar meaning, though it can miss subtle paraphrases or overrate texts that merely share a topic.

Can translated content be compared?

Yes, if the method supports the language pair. Use a multilingual model, record its version, and require bilingual human review for attribution or legal decisions.

Why do two tools return different scores?

They may use different tokenization, normalization, n-gram lengths, embeddings, window sizes, and score mappings. Compare method details before comparing percentages.

How should we compare very short snippets?

Show matched words and phrases, avoid automatic decisions, and ask a reviewer to consider necessary terminology. Scores under 50 words are too sensitive to a few shared tokens for a general rule.

How long can the text be?

That depends on the interface and model limits. For long documents, split by headings and use overlapping windows, then retain the maximum passage score and the whole-document score.

Is my text stored?

The provider should state retention, training use, subprocessors, deletion, processing regions, and backup handling before submission. For confidential drafts, use only a security-approved service or a local method.

Should we remove references and quotations before comparing?

Keep them in the original and mark them as special sections. You may calculate a second score without them, but the decision log should preserve both results and explain the exclusion.

When should we use a full plagiarism service?

Use one when you need comparison against a large external corpus, academic databases, prior submissions, or web sources. Two-text comparison answers a narrower question because both inputs are already known.

Can we automate publishing based on the score?

Automate routing before you automate approval. Low-risk, well-tested content classes may later support automatic actions, but claims about price, security, retention, regulation, and product support should retain human review.

Which score should go into a dashboard?

Store separate lexical, semantic, and maximum-window scores. If executives need one status, display the decision band and link it to the underlying method details rather than averaging away the evidence.

How do we connect similarity checks to AI search visibility?

Use comparison results to clean up repeated or conflicting owned content. Then measure prompts, brand mentions, citations, and competitor presence separately. Content similarity cannot prove that an answer engine used or recommended a page.

A 2026 operating model for AI-assisted content review

AI-assisted writing has made text production faster, while editorial ownership remains a human operating question. Teams can now produce several semantically similar drafts without copying exact sentences. Lexical checks alone will miss much of that overlap.

The better 2026 workflow combines retrieval, passage comparison, claim checks, and human decisions. A system first finds likely related pages. It then reports lexical and semantic evidence. Claim-sensitive sections go to named owners. The final decision is written to a log and tied to a retention policy.

AI can also explain why two passages were grouped, but generated explanations should remain secondary evidence. The method scores, matched text, source IDs, and policy version are the audit record. A fluent explanation can still be wrong.

For brand teams, the next step is connecting internal content governance with external answer behavior. A company may clean up duplicate pages and still be absent from AI recommendations because third-party sources, weak category association, or missing answer coverage remain. Targetlytics addresses that separate measurement problem through competitor, citation, and visibility data. The similarity workflow keeps owned claims orderly; the visibility workflow tests whether the market’s AI interfaces repeat them.

The decision to make now

A text comparison process will help your team find copied wording, close paraphrases, repeated intent, and conflicting claims. It will not establish plagiarism on its own, replace legal judgment, or tell you whether an AI system recommends your brand.

Start with two methods, length-aware thresholds, passage-level review, and a decision log. Treat every threshold as a versioned internal policy. Measure the minutes removed from routine triage and the mistakes caught after review. If nobody can audit a 0.78 result after publication, the process is incomplete.

If your wider concern is whether duplicated or inconsistent content is affecting how AI systems describe your company, run a free Targetlytics AI visibility audit and book a call with the team. You can also start for free; paid plans include a 14-day trial.