How the SimplyScan Engine Delivers Consistent Results

Quick answer: The SimplyScan Engine ensures consistent scores by separating deterministic detection from AI explanation. Rule-based analyzers compute the score with fixed weights, ensuring the same site always gets the same result, while AI provides reasoning and fix prompts that never fluctuate the headline number.

By Daniel A · Kraftwire Software

The SimplyScan Engine delivers consistent results by separating deterministic detection from AI-driven explanation. While large language models (LLMs) are used for reasoning and remediation advice, the core security score is calculated by fixed, rule-based analyzers. This ensures that the same website receives the same score across multiple scans, providing a stable baseline for security monitoring and trust badges.

Run the same scan twice and you expect the same answer. For a long time, security tools that lean entirely on a large language model could not promise that. Ask an LLM to grade a page today and it might say 82; ask it again an hour later, with nothing changed, and it might say 74. That is fine for brainstorming. It is not fine for a report you paid for, a verified security badge you display publicly, or a monitor that is supposed to tell you when something actually got worse.

This is the single most common complaint about LLM-only security tooling. We agreed. So we built how SimplyScan produces a score around a principle we call the SimplyScan Engine.

What Does the SimplyScan Engine Do?

The deterministic engine detects issues and sets your score. The AI explains, triages, and catches the things rules cannot. The AI never touches the number.

That ordering is deliberate, and it is also where the industry has landed. The current best practice is not "AI instead of rules" or "rules instead of AI" · it is a hybrid where a deterministic engine does the detection and the model does the reasoning on top. Academic work on hybrid static-analysis-plus-LLM pipelines reports large accuracy gains and steep false-positive reductions from exactly this split.

Without a deterministic engine, these critical findings could fluctuate based on the model's "mood," leading to a false sense of security or unnecessary panic.

Why Does an LLM Alone Drift?

A language model generates text by sampling. Fixing a random seed helps, but providers document cases where identical requests still diverge.

We do pin those settings because it reduces the wobble, but we refuse to depend on it for the final grade. Anything that decides your score has to be something we control completely. This is especially important for vibe-coded apps built with tools like Lovable or Bolt.new, where rapid iterations mean you need a scanner that can distinguish between a real code change and model variance.

What Does the Engine Measure Deterministically?

Most of a SimplyScan report comes from deterministic analyzers. SEO, Answer Engine Optimization (AEO), accessibility, GDPR compliance, domain health, and email authentication are all computed by pure code that returns the same result every time. The rebuild extended that same discipline to the two dimensions that used to be the model's opinion · security and speed.

For security, the engine looks for the things that are genuinely rule-detectable:

• Leaked credentials · provider API keys, tokens, private keys, and database connection strings, using the same regex-plus-entropy approach proven by open-source secret scanners. Matches are masked before they leave the scanner to fix exposed API keys without creating new risks.
• Live database exposure · the engine makes real unauthenticated calls to detect tables anyone can read, particularly focusing on Supabase RLS misconfigurations.
• Response headers · Content-Security-Policy (CSP), HSTS, and clickjacking protection, with severities calibrated to real-world impact. You can test these individually using our security headers tool.
• Exposed sensitive files · checking for .env files, .git directories, and other exposed files.
• Outdated dependencies · known-vulnerable versions of common front-end libraries matched against a curated CVE database.

The engine also grades threat exposure separately · whether a WAF sits in front of the app or whether server banners leak version numbers. These are the controls that stop automated scanning before they land.

For speed, the engine uses signals that are stable by design · real-user field data from the Chrome UX Report (CrUX), which is a 28-day rolling average, plus static checks on render-blocking resources. We deliberately do not put a lab performance score on the critical path · those vary by roughly five points run to run even on an unchanged page (DebugBear).

Where Does the AI Still Earn Its Place?

Rules are precise but literal. They will never notice that an authentication check is backwards or that an endpoint lets one user read another user's records. That kind of reasoning is where a model genuinely shines, and it is the reason SimplyScan understands AI-built apps better than a generic scanner.

So the AI stays · it reads your code, writes the plain-English explanation, generates the ready-to-paste fix prompt, and surfaces the semantic issues that no regex can see. Those insights are clearly marked as analysis. If the wording shifts slightly between runs, nothing breaks · because that wording was never allowed to move your score.

Answer Engine Optimization (AEO) and AI Visibility

A unique part of the SimplyScan engine is how it handles Answer Engine Optimization (AEO). As AI search engines like Perplexity and ChatGPT Search become primary traffic drivers, your site needs to be readable by LLM crawlers.

AEO is the process of optimizing content so that AI models can accurately synthesize and cite your website in their answers. Unlike traditional SEO, which focuses on keywords and backlinks, AEO focuses on structured data, semantic clarity, and "LLM-friendliness." The SimplyScan engine checks for:

• Semantic HTML structure.
• Presence of a robots.txt that allows AI crawlers.
• Schema.org markup that helps models understand entity relationships.
• High-quality, factual content that matches the "vibe" of the query.

What Does This Mean for You?

• Rescans are trustworthy · Fix an issue, scan again, and the number moves by exactly what you changed · not by model noise.
• Your trust badge is stable · It reflects the state of your site, not the mood of a model.
• Pro Monitoring alerts mean something · A drop in our uptime monitoring or security score is a real regression worth investigating.
• Free and Pro agree · The same site gets the same security score on both · Pro simply unlocks the fixes and deep AI analysis.

We keep the exact rule set, scoring weights, and model configuration private · that is the SimplyScan Engine. But the philosophy is not a secret · deterministic where it counts, intelligent where it helps, and honest about which is which.

Ready to see it? Run a free scan and check the score twice.

FAQ

Why do AI security scanners give different scores each time?

Large language models generate text by sampling, so even at temperature zero the output is only mostly repeatable. Providers document identical requests that still diverge. An LLM asked to grade the same unchanged page can say 82 one hour and 74 the next. That is why score-setting should be handled by deterministic code rather than a model.

Does SimplyScan use AI in its scans?

Yes, but only for reasoning, never scoring. The AI reads your code, writes plain-English explanations, generates ready-to-paste fix prompts, and surfaces semantic issues no regex can see, like a backwards authentication check. The score itself is computed by deterministic analyzers with fixed weights, so AI wording changes between runs can never move your number.

Is a rule-based scanner better than an AI scanner?

Neither alone is sufficient for modern apps. Current industry best practice is a hybrid: a deterministic engine does the detection and scoring, and a language model handles triage, explanation, and semantic findings on top. Research on static-analysis-plus-LLM pipelines reports large accuracy gains and steep false-positive reductions from exactly this split.

What is Answer Engine Optimization (AEO)?

AEO is the practice of optimizing your website to be easily crawled, understood, and cited by AI search engines like ChatGPT, Claude, and Perplexity. It involves using structured data, clear semantic headers, and fast-loading pages. SimplyScan includes an AEO check in every scan to ensure your vibe-coded app is visible to AI.

Do free and Pro SimplyScan scans give the same security score?

Yes. The same site receives the same security score on both plans because the deterministic engine computes it identically. Pro does not refine the number; it unlocks the ready-to-paste fixes, the deep AI code analysis, and repository scanning. That means a free rescan is a trustworthy way to verify a fix actually landed.

Can a scanner detect leaked API keys without exposing them?

Yes. SimplyScan detects provider API keys, tokens, private keys, and database connection strings with regex-plus-entropy matching. Every match is masked before it leaves the scanner, so the report confirms the leak exists without ever echoing the secret itself back to you, keeping your credentials safe during the audit.