Raytheon Technologies : A bot built to stop spearphishing

The email comes from a colleague in another department who needs your help.

Hi there Gretchen. You probably don't remember me, but we met at the company picnic last fall. I'm in accounting and was told to contact you about getting access to documents in your department. As a first step, could you register yourself with our internal secure file sharing system by clicking on a link that I will send you. Thanks, - Caroline

Wanting to be helpful, you might go ahead and click the link to help your colleague. That would be a mistake. The message is an example of a sophisticated cyberattack called social engineering.

These sophisticated attacks take steps to manipulate users into making security mistakes or giving away sensitive information. First, the attacker investigates the intended victim to gather necessary information, such as where she or he works and where there might be potential points of entry and weak security protocols. Then, the attacker attempts to gain the victim's trust, perhaps by referencing a shared experience or saying they've met before, as in the message above.

Raytheon BBN is working on a solution for social engineering attacks, called SIENNA - Strategies for Investigating and Eliciting Information from Nuanced Attackers - that combats these attacks while simultaneously trying to gather information about the attacker's identity.

SIENNA is Raytheon BBN's contribution to the DARPA Active Social Engineering Defense, or ASED, program, which is looking for ways to automatically identify, disrupt and investigate social engineering attacks. SIENNA uses a conversational AI framework to power chatbots, AI agents that engage with bad actors to waste the attackers' time by generating relevant, human-like content. The bot has a large pool of content that it can generate to avoid repetition and will continue engaging for as long as the attacker stays responsive.

James Ryan, Ph.D., principal investigator on the SIENNA program at Raytheon BBN, said, 'Our collaborators on the ASED program have trained machine-learning models that recognize traits of social engineering attacks, including aspects of the message language and the provenance of the sender's email domain.'

SIENNA bots masquerade as the intended victim to 'converse' with the attacker, wasting as much of their time as possible and gathering clues about their identity that can be turned over to an ISP or law enforcement. The conversation might go like this:

Sienna Bot: Hi Caroline... Nice to meet you. I'm pretty busy these days but will see what I can do to help you. OK but first actually, are you based here in the Richmond, Virginia, office or somewhere else? Regards -Gretchen

Attacker: I'm based in Richmond, yes! I work remote now, which is probably why you haven't seen me around. Now, were you able to sign up at the link I sent? Let me know if you didn't get it, and I will re-send it.

In response, the bot pretends to have an email problem and learns that the attacker claims to live in Richmond, Virginia. This may turn out to be just be part of the attacker's 'persona,' but it also might be useful data for an investigation. The bot can also infer other data, such as time zone, country and zip code, all while delaying the attacker form moving on to the next attack.

Sienna Bot: Hmm. I keep getting these alerts from my email client that your messages are coming from a compromised device. It's actually really difficult to bypass them. Can you try switching to a different device or email client?

Attacker: I'm sending this from my phone. Is that any better?

Sienna Bot: No, I'm pretty sure it's on your end. That message got flagged too, FYI. Try something else?

'The bot's language understanding is relatively simple and informal, so to avoid revealing that to the attacker and possibly raising suspicion, it tries to dominate the conversation by giving the attacker tasks to do,' said James Ryan, Ph.D., principal investigator on the SIENNA program at Raytheon BBN. 'These 'quests' are created using design knowledge from the community of 'scambaiters,' who take on phishers and scammers by posing as potential victims to waste scammers' time and resources.'

Attacker: I ran a full antivirus check, and it didn't find anything. Are you sure the problem's not on your end?

Sienna Bot: Oh, that fixed it! OK, great. On another topic, how long have you lived here in Virginia? I just came out here for this job, trying to decide if I want to stay here long term.

'The bot deliberately keeps changing the subject to avoid revealing its limitations, and to use any information it learns about the attacker to keep its reactions plausible,' said Brad Tousley, president at Raytheon BBN.

Just as the attacker tries to instill trust in the target, the bot tries to gain the trust of the attacker to keep them engaged. The bot uses the fact that the attacker is also supposedly located in Virginia to keep the conversation going.

Attacker: I've been here a while and I really love it.

Sienna Bot: Nice! What part of Richmond do you live in?

Attacker: I'm over on the west side, what about you?

Sienna Bot: OK. Relatedly, can you do me a favor? Are you using Telegram on your mobile yet? It's a secure messaging app, and I like to run any business stuff through there to avoid privacy issues with email. If you have a username on there, or wouldn't mind installing and registering, let me know. I'd feel better about this whole thing if we could keep the conversation going on Telegram.

At this point, SIENNA's machine learning models estimate that the attacker trusts the bot enough that it can risk asking for more time-consuming or revealing information.

'The bot cannot actually communicate via Telegram, but it can try to get the attacker to waste time setting up an account and trying to connect,' Ryan said.

Attacker: I don't have a Telegram account. I really don't see a problem sticking with email, is that okay?

Sienna Bot: I acknowledge that this is a weird request, and you deserve an explanation. The short version is I lost a lot of money a few years back from someone snooping on my email, so I try to do as much as possible through secure channels now. I promise the Telegram app is easy to use! Let me know your username once you've registered an account. Appreciate it!

Attacker: OK, I signed up for Telegram. My name is Caroline1979 on there. How do I find you?

SIENNA has now captured a Telegram profile name associated with the attacker, another potentially valuable piece of information. The bot makes up a fake but plausible Telegram name to reciprocate.

Sienna Bot: I really appreciate it. So, let's get to it: my account on Telegram is GretchenJones31. I should have sent you an invite. Let me know if you don't see it, otherwise let's continue the conversation over there!

Attacker: Gretchen, I'm not seeing anyone by that name on Telegram. Is there some special thing I need to do to add you?

Sienna Bot: That's funny, I thought the invite went through. I've tried resending it. You may need to wait an hour or so for it to come through. Please accept it and message me there!

(an hour later) Attacker: I'm still not seeing your invite.

'The conversation can continue this way for hours or days,' Ryan said. 'If the bot's machine learning models detect signs of frustration in the attacker's messages, it will abandon the Telegram request and switch to a different quest to keep the attacker engaged, waste more time, and possibly, gather more identity clues.'