We Read an MFA Prompt as Evidence of a Stolen Password

Microsoft sign-in logs carry an error code on every failed authentication. Those codes are one of the more useful things in the dataset, and one of the easier things to misread.

We misread one, and the way we misread it is worth describing because the reasoning that led there was not obviously wrong.

The Reasoning

Among the failure codes are several indicating that authentication proceeded past the password stage and failed at multi-factor authentication. The user, or whoever was authenticating, supplied a correct password and was then challenged.

The security implication looked clear. If the password was accepted, the password is known. The sign-in did not complete, but that is a detail. The credential is compromised and should be reset regardless.

We built that into a rule. Any occurrence of those codes was graded critical, with narrative explaining that the password was known to a third party.

What Went Wrong

The rule fired on ordinary staff signing in normally.

Those codes do not exclusively mean that an attacker got the password right. They also appear when a legitimate user enters their own correct password and then does not complete the second factor, which happens constantly. The phone is in another room. The prompt is dismissed by accident. The notification arrives after the session times out. The user is mid-enrolment and has not finished setting the method up.

From the log, a member of staff fumbling their own authenticator prompt and an attacker who has guessed a password and been stopped by MFA look identical, because at the protocol level they are identical. The password was accepted and the second factor was not satisfied. Nothing in the event distinguishes intent.

During an active MFA rollout the rule was firing on nearly every enrolled user.

The Correction

The code alone carries no verdict. It carries a verdict only in combination with context that the code does not contain.

The context that mattered in our case was the client. A legitimate user authenticates through a modern client that can present an MFA challenge. Legacy authentication protocols cannot present a challenge at all, which is why attackers prefer them.

So the corrected rule requires both. The password stage was passed, and the attempt arrived over a protocol associated with credential attacks. A staff member's own MFA prompt through a modern client no longer qualifies.

Other context that would serve the same purpose:

• Source. An address the account has never authenticated from.

• Pattern. The same code across many addresses in a short period.

• Sequence. The code arriving amid a run of password failures rather than in
• isolation.

The General Point

An error code describes a protocol outcome. It does not describe an intention, and it cannot, because the protocol does not know.

This applies well beyond authentication. A file was deleted. A message was sent under another identity. A permission was granted. Every one of those is a fact about a system state, and none is a fact about whether anybody did anything wrong. The determination lives in the surrounding context: who, from where, how often, and whether it fits what that account has done before.

Detections built on a single field are fast to write and produce cases nobody can action, because the case contains a fact rather than a judgement.

How We Found It

Not by review. By backtesting against real activity and reading every row.

The logic was defensible on paper and would have survived a code review, because the misunderstanding was in the premise rather than the implementation. The only thing that surfaced it was running the rule against a tenant where people were enrolling in MFA, and noticing that the accounts it flagged were the ones doing what we had asked them to do.