Assessing suppliers: what they declare versus what you can observe

Almost every third-party risk programme is built on a single source: what the supplier answers in a questionnaire. It is a legitimate and necessary source, but it has a property worth remembering: it measures the supplier’s perception of itself, filtered through whoever filled in the form and the moment they did it.

The other half of the assessment is the observable: what you can check on your own, without depending on their cooperation, and which does not expire every twelve months. A programme with only the first half is not assessing risk, it is filing declarations.

What is useful about declarations, and how to make them more so

The questionnaire is not useless. It is the only way to learn about internal decisions that leave no external trace: how they manage privileged access, whether their continuity plan has been tested, what they subcontract. The problem is not the instrument, it is how it gets used.

Three changes improve it considerably:

Ask for the evidence, not the status. “Do you have vulnerability management?” is always answered yes. “State the date of your most recent vulnerability assessment report and the percentage of assets covered” forces someone to look at a real number, and an evasive answer is itself informative.

Ask for the scope of the certification, not the certificate. An ISO 27001 certificate is a two-page document with a scope annex. The annex is what matters. It is common for the certificate to cover corporate headquarters and not the platform that will actually serve you.

Fix the cut-off date for every answer. A questionnaire with no reference date mixes data from three different years. The question should always be “as at today” or “in the last twelve months”, and the answer should say which.

What can be observed without asking permission

There is a set of signals anyone can gather from outside, with two virtues: they do not depend on the supplier, and they can be refreshed continuously.

• Exposed surface. Which domains and subdomains they publish, which services answer, which technologies are identifiable in the responses. An administration panel reachable from the internet says more about operational maturity than twenty questionnaire answers.
• TLS and email hygiene. Expired or misconfigured certificates, missing SPF, DKIM or DMARC, absent security headers. Individually these are weak signals, but collectively they are very consistent: teams that neglect this tend to neglect more.
• Public incident and disclosure history. Whether they publish a coordinated disclosure policy, whether they respond to reports, how they communicated their last incident. A supplier that publishes honest post-mortems is a far better signal than one with no visible history.
• Observable patching cadence. When a vulnerability affects a technology identifiable on their surface, how long does the vulnerable version take to disappear? That is an indicator of real process, not declared process.
• Subcontracting structure. Where they host, which CDN they use, which identity providers. Concentration matters: if you and your supplier both depend on the same third party, your diversification is fictional.

None of these signals proves on its own that a supplier is insecure. Their value is in the contrast: when the declared and the observed disagree, you have a specific question to ask, and specific questions are what produce useful conversations.

The contrast is the product, not either source alone

An example of how this looks in practice. The questionnaire says critical patches are applied within fifteen days. External observation shows a web server version with a vulnerability published three months ago, still present.

There are three possible explanations and all of them are informative: the banner is wrong and the real version is patched — which tells you they neither suppress banners nor consider it relevant; the asset is not in their inventory — which reveals a coverage problem bigger than the patch itself; or the fifteen-day commitment is not being met. Any of the three belongs in the file.

That is the real work of an assessment: not scoring, but locating the discrepancies and documenting what was done about each one.

Segment by criticality, because not everyone deserves the same

Applying the same process to all hundred and eighty suppliers in the register guarantees it will be done badly for all of them. Criticality should determine depth, and it should derive from three questions:

1. Which function does it support? If it supports a critical function — and under DORA that classification already exists — the supplier inherits that criticality.
2. What data does it reach? Distinguish those processing personal or confidential data from those that never see it.
3. How reversible is it? A supplier replaceable in two weeks and one that would require an eighteen-month migration are not the same risk even if they do the same job.

That yields three or four tiers. The top tier warrants a full assessment, contractual review and continuous monitoring. The bottom tier needs an up-to-date register and an automated external check. The mistake is not distinguishing at all.

What the regulator expects you to be able to show

Both NIS2 and DORA turned this into an explicit obligation. Article 21(3) of NIS2 requires taking into account the vulnerabilities specific to each direct supplier and the quality of their practices and secure development procedures. DORA goes further: Article 28(4) requires assessment before the contract is signed, and Article 28(3) an up-to-date register of information plus an annual report to the authority.

In both cases the evidence required is the same: for each supplier, the date of the assessment, who performed it, what was reviewed and what was decided. A folder of completed questionnaires answers none of those four questions.

The minimum viable version

If you are starting from scratch with little time, this order gives the most return: a complete supplier register with the function each one supports; classification into three criticality tiers; for the top tier, an evidence-backed questionnaire plus an external surface check; and for all of them, a visible date of last review. With that you can answer an auditor, and you refine from there.

The part that does not scale by hand is keeping the observed side current and linked to the declared side, supplier by supplier. That is exactly the gap the third-party risk module in Violet Security covers — though the criticality judgement, which supplier matters and why, has to come from your organisation.