Attack Simulation Training in Microsoft 365 allows you to create test attack emails (such as phishing attempts) and send them to your end-users. The users will then be unknowingly tested on their preparedness for such an attack.
In this post, we will cover how to set-up Attack Simulation Training in Microsoft 365 and what that experience is like for the-end user.
What license is required to use Attack Simulation Training?
Firstly, to access the Attack Simulation Training blade in Microsoft 365 our tenant needs to contain the appropriate licenses. them licenses are to be assigned to any user that will benefit from this service.
The minimum license required to use the feature is Defender for Office 365 Plan 2. Defender for Office 365 Plan 2 can be purchased as a standalone license or it is also included in Office 365 E5, Office 365 A5 and Microsoft 365 E5.
What permissions do I need to use Attack Simulation Training?
The minimum level of permissions you require in Microsoft 365 to create and launch attack simulations is the Attack Simulation Administrators role. Other roles include:
- Global Administrator: If you need to access just the Attack Simulation Training feature, do not assign yourself this role. This will give you access to the whole Microsoft 365 tenant
- Security Administrator: This role will provide you access to all security features, not just attack simulation training.
- Attack Simulation Administrators: This will allow you to create and manage all aspects of attack simulation campaigns.
- Attack Payload Author: You can create attack payloads that an administrator can initiate.
Before you create an Attack Simulation
You need to ensure auditing is enabled for your organisation! without this, you will not be able to view the simulation reports and may not get any valid data from your tests. Luckily, this is easy to do.
- Start by opening the Microsoft Purview compliance portal (previously the Compliance Centre). https://compliance.microsoft.com/
- Select Audit from the left-hand menu. You will see a large blue box at the top of the screen that says; Start recording user and admin activity. Click on this and select yes to the prompt.
It may take up to 60 minutes for the changes to take effect in your organisation.
How to create and run an Attack Simulation
Let us create our first attack simulation right from the Microsoft 365 Defender portal.
- Login to the Microsoft 365 Defender portal at https://security.microsoft.com/
2. From the left-hand menu, select Attack simulation training.
3. Select Simulations.
4. Click Launch a simulation.
5. Choose the technique you wish you use in your simulation. You will be presented with 6 different techniques:
- Credential Harvest: Traditional phishing test to encourage users to click a link and enter their login credentials
- Malware Attachment: The email will contain a malicious attachment which your users will be encouraged to open.
- Link in attachment: A mix of the above, where users will receive a non-malicious attachment, but it will contain a malicious link inside.
- Link to malware: Instead of the malicious attachment being attached to the email, the email will contain a link to the attachment in an attempt to evade malware scanning.
- Drive-by URL: The email will contain a link, when if clicked, the webpage will attempt to launch some malicious code.
- OAuth Consent graph: The malicious actor has created a malicious application in Azure. The email will attempt to trick the user to consent for the application to access their data via a link embedded in the email.
6. Choose a meaningful name for your simulation and enter a description. For example:
- Name: Finance Dept – Credential harvesting campaign
- Description: Credential harvesting campaign for the finance department. Created by James on 08/06/2022.
This way anyone else who comes across the simulation will clearly know its purpose, when it was created and by who.
7. Select a payload that is relevant to your target users. The idea here is to use a campaign that will most likely match an email they would receive often. In my scenario, I will select the Document Share Payload and click Next.
If there is no payload relevant to your target users, you can select Create a Payload to design your own.
To also help you decide on a payload you can use the Send a test feature. Here is what I received when performing the test.
8. Choose your target users. You can either choose to send the simulation to all users in your organisation or specific users and groups.
9. On the Assign Training page you can choose to require users to complete training if they get caught out by one of the emails. Optionally you can also choose ‘none’ if no training is required or refer to a custom URL if you use a 3rd party training platform like Knowbe4.
If you do decide to enable training, ensure you select a Due Date. I will leave mine at “30 days after the simulation ends”. But take note, the due date will not enforce your users to take the training, it will be down to you to manage and review.
10. The landing page is the first thing a user see’s when they get caught out by the training. I am going to select use Microsoft default Landing page as it is easy. However, you also have options to design your own landing page or refer to a custom URL.
You will also be able to use your own custom Logo to embed into the page and importantly, you will be able to add payload indicators to the email to help users identify what they did wrong.
Use the preview panel to review the landing page:
11. You should choose to send end-user notifications. I recommend you use the Microsoft default options.
When you select the Microsoft default notification you will see that there are 3 notification types selected. The first is the notification the user will receive when they report a phishing attempt, when they do so, they will receive positive reinforcement to confirm they made the correct decision. The other 2 notifications below are for the training. They will be sent to the user when training is available and to remind them if they have not yet completed the online training.
For delivery preferences, I recommend you set the positive reinforcement notification to Deliver after the campaign ends, otherwise, they may be inclined to tell others that any suspicious emails are tests, making the results of the simulation inaccurate.
For the training reminder, you should set to the most frequent setting of Twice a week. It should be critical that training is conducted immediately if the user has failed the test. They may also be at breach of their cyber security policy without having done the training.
12. You can decide on the Launch Details page to start the simulation immediately or at a specific time.
You should configure the number of days you wish the simulation to end after, based on the nature of your business. This is to ensure all applicable users encounter the simulation emails in some form and provides them enough time to act, or not to act!
I recommend enabling region aware timezone delivery.
13. lastly, review your simulation settings and click Submit.
The user experience
It is important to know what the users may experience during this campaign, so I have launched it upon myself to demonstrate.
Once the simulation begins, each target user will receive an email like the below:
And once opened, it may look something like this:
In the event the user clicks the malicious ‘Open in Teams’ link, they will be directed to a look-a-link Microsoft 365 sign-in page. In the unfortunate event they try to log in with their credentials, they will be redirected to the following page.
Shortly after being alerted from the above notice, the user will receive another email in their inbox with a link to the online training.
Reviewing your attack simulation campaign
Now you have created your campaign you can review how it is going by selecting it from the simulation tab.
This is what you will see.
It is important to know that the simulation impact will be immediately updated if a user is compromised during the simulation test.
My advice for running a Microsoft Attack Simulation
Do not run a single simulation for your whole company. If your employees are grouped into ‘pods’ or ‘rooms’ you should only send one type of simulation email to one person in that room. You may not get accurate results if one user alerts the rest of their immediate team.
Send simulation emails at appropriate times. If there are certain times when staff after away from their desk, schedule the emails to arrive just before they get back to their desk, that way they will be at the top of their inbox and not missed.
Make sure the simulation email is relevant. This one should be easy to understand, finance emails to the finance team, shipping emails to the warehouse team, password reset emails those always forgetting their emails.
Make sure the simulation email is relevant. Part 2. This is the most important point to make. Something that has greatly helped me is to review our support helpdesk and take advantage of some of our users pain points to carefully craft custom simulation campaigns.
Your goal is to trick your staff as the real attack will not hold back!
Summary
Thank you for taking the time to read my post! Running attack simulations is an important step to help bolster your organisation’s security posture. It will help identify if more comprehensive user training is required or may help you justify additional spending on your cyber security.
but how end user will report the email is phishing, is there any “add-in” needs to be added?
Indeed! check out this: https://learn.microsoft.com/en-us/microsoft-365/security/office-365-security/submissions-users-report-message-add-in-configure?view=o365-worldwide
Hi David
Re; Attack Simulation Training
Really good article. Have come across an issue where users have reported the email as phishing and then as a result been directed to do some training ? We attempted some phishing tests recently and some users were saying they reported this as phishing, but then got directed to do some training – which was not the desired result ?
Hi Tony,
Thank you! On the Training assignment page of setting up your simulation, have you set ‘All users’?
Check out the screenshot here: https://learn.microsoft.com/en-us/microsoft-365/security/office-365-security/attack-simulation-training-simulations?view=o365-worldwide#training-assignment
Let me know!