Security Awareness Training: A Design Thinking Project

During the last semester (Spring 2024) of my master’s degree (Master of Engineering in Cybersecurity) at the University of Maryland, College Park, I joined the Innovation Fellowship cohort at the Academy for Innovation and Entrepreneurship. There was an option to take it without credits, and I wanted to explore the application of design thinking in designing security awareness training — so I joined the fellowship.

A special thanks to Mira, Anat, Apoorva, Tami, Jenny, Janya and the entire Academy for Innovation and Entrepreneurship team for their valuable insights and for supporting me throughout the semester.

Why I chose this problem?

In 2018, when I was venturing into security, I read an eye-opening report that said more than 70% of data breaches happen because of human error. I was in school finishing my undergrad, and I had a hard time understanding why and how human errors happen. The most obvious thought that came to me was — why not train humans to reduce the error? I was puzzled about this question for a while, and then I joined my first full-time job as a software engineer. I took my first mandatory training and knew what was not working for me. Fast forward to two and a half years, I joined my first security role. I was in charge of creating and enforcing security training. I was on the opposite side of the table. I started to understand why training a human is hard. However, the statistic remained the same, and so does the question — why can’t we train humans effectively to reduce human errors leading to a cybersecurity incident? I had been thinking about this problem for some time, and the traditional thought process was not working. I have heard about design thinking principles but never used them in any project. I got this opportunity and decided to investigate the problem myself using design thinking principles.

Design Thinking Project Studio

Design Thinking Project Studio was the official name of the class. There were seven learning community sessions throughout the semester scheduled every two weeks. During these sessions, the cohort meets, discusses/ learns about design thinking principles, and collaborates with each other’s projects.

The seven learning sessions were -

LC#1 — (dis)orientation

LC#2 — Noticing

LC#3 — Sensemaking

LC#4 — Ideating

LC#5 — Emerging

LC#6 — Experimenting

LC#7 — Wrap up

I will take you through my learnings and the specific work I did for my problem statement each session.

LC#1 — (dis)orientation

It was my first session of the semester. I met other members of the cohort. Before the session, I worked on project scoping and framing the questions.

Scoping — defining the problem statement I am trying to solve

Questions to ask for scoping -

• What are you hoping to attempt?
• Why are you attempting it?

Idea: I want to attempt designing an engaging security awareness training with high learning outcomes for an individual without taking external help during the evaluation. I am doing this because 74% of data breaches resulted from human errors (Verizon’s Data Breach Investigation Report 2023) and a lack of security awareness. Most of them happen because of a lack of poor misconfigurations, credentials theft through phishing, vishing, OSINT etc. — all are human errors. I believe that by engaging in training that enhances awareness, we can reduce the number of breaches happening due to human error.

Once I defined the idea, it was time to scope the project. Out of all the resources that were provided to us for scoping, I liked the “5Ws” method to start with. In this method, I attempted to answer five questions about my project to scope my project understanding and the work I would be doing this semester. Below are the 5W’s I explored for my problem -

Problem 1: Employee engagement with the existing training

What’s the problem or question I am interested in?

I have noticed that a lot of people in companies do not like completing mandatory security training as they feel like this is an added task to their day-to-day work and complete it just for the sake of completing it without learning/gaining knowledge wholeheartedly. The problem is the current training structure- boring/non-engaging slides with questions for evaluation at the end. What adds more pressure is a quiz at the end to evaluate the understanding of the employee.

Who’s affected most by this problem?

The employees who take the training are most affected.

Where does this problem occur?

It happens in all the departments/ all the verticals in the entire hierarchy — from top management to interns.

When does this problem occur?

Typically the training is offered when an employee joins the organisation, and it is repeated annually after that. Initially, when the employee joins, they can focus on training, although still not engaging, but they have time and can complete it with focus. The real problem arises during the annual training when the employee is loaded with work/ deadlines/ meetings, and on top of that, they have to carve out time for mandatory training. Since there is an assessment at the end of the training, employees would look for shortcuts to finish this training.

Why does this problem occur? Why is the problem important?

It happens because employees are not taking the training for the sake of understanding but to tick-mark another checkbox in the compliance list. It is important because this would not lead to any learning/ understanding and will not decrease the human errors responsible for data breaches.

During the session, we learned about team effectiveness as we all would be working in a team collaborating on each other’s projects. One exercise we did was to partner up with each other and tell the idea to another person in a minute, then change the partner, tell your previous partner’s idea to your new partner and so on until everyone got to know about each other’s problem statement. Then, we were asked to frame the project scope in the given template. My initial project statement was -

I’d like to explore ways to help employees of an organisation to learn security policies, guidelines, best practices effectively with mindfulness in order to be able to identify security issues during their day-to-day business work without specifically stressing about security or focusing too much on it.

Once we scoped the statement, we partnered up again and discussed with our partner about the problem. The initial discussion resulted in the following points from my partner -

• Educate about how phishers do what they do
• celebrate success
• Real-time training
• Employee Relevance (Buy-in)

At the end of the first session, the task was to refine the scope statement.

LC#2 — Noticing

This was our second learning community session. Before the second session, I created my project workspace on Miro, made a project timeline and did stakeholder mapping -

During this session, we learned about noticing — the art of learning people’s needs with empathy.

The basic understanding was to go with the beginner’s mindset keeping the following principles in mind -

• Do not judge
• Question everything
• Truly curious
• Inquisitive
• Find patterns
• Active listening

These principles helped me a lot when I was interviewing stakeholders later in the semester. A couple of activities that we did during the session were — shadowing (observing a team member working on a task and asking questions), journey mapping (mapping the emotional journey(positive/ negative emotions) of your team member throughout a project) and card sorting (sorting animal picture cards in a team and making correlations) emphasizing the pointers above.

After the session, I explored various ways to “notice” what’s happening in the design problem. Below are a couple of ideas -

I found interviewing the stakeholders the most efficient given that everyone I was interviewing was a full-time employee and a quick chat was all I could get.

LC#3 Sensemaking

During this session, we learned about sensemaking — making sense of the raw interview data.

Before I move on to the sensemaking of data, let me explain my interviewing approach.

I interviewed a couple of folks with varied tech knowledge, varied experience, varied tech knowledge and working in different domains but all of them were required to complete a security awareness training in their respective organizations.

I asked the stakeholders about four specific parts of training -

• Content
• Experience
• Assessment
• Incentive

Below is the summary of my findings -

Once we had the interview data, we were asked to infer — ‘why we think the observation happened’ based on our observations and interview data. I took one inference from three parts of training — content, assessment and experience as shown below -

LC#4 Ideating

During the last session, we were tasked to come up with 1–3 ‘How Might We’s before this session. I came up with a couple and during the session, we partner up with other folks and came up with some of the How Might We’s as shown below -

It was an ideation session. Even though I figured out key inferences last session, during this session, we looked at the problem statement from a different perspective, for example, relating it to some other real-world example, putting constraints on the solution etc. It not only helped us refresh our understanding of the problem but also sparked interesting conversations like biomimicry, training compared to getting vaccines, the Duolingo way of learning etc. Below are some the screenshots from the ideating session:

Putting a time constraint on the solution

Relating the problem to a familiar situation

Considering emotional appeal

Some other brainstorming ideas included making the training a competition, adding humour and animation, training in pairs testing each other, podcasting, making it in a puzzle format etc. as shown below:

LC#5 — Emerging

This session was about reflecting on the work we have done so far and the learnings we had about the problem space, the people within the problem space and ourselves.

I was a little skeptical about my problem during this session. I got a chance to look at the training format for my university employees, and my mind began to like it, partly because I started thinking from a security administration point of view. The problems which the stakeholders identified during the interviews were still there, but I fell into the rabbit hole of thinking about how straightforward it was to implement this security awareness program, and it covers all the necessary information. It took me an hour-long discussion with the facilitator to come out of this rabbit hole.

One interesting analogy that came up during this discussion was — this is similar to a flight safety demonstration. Flight safety demonstration is mandatory for airlines, and how many times do we pay attention to it? It encouraged me to draw parallels between my problem statement and the flight safety demonstration.

LC#6 — Experimenting

Before this session, I came up with some ideas to prototype. I came up with these -

• A GenAI tool
• trained with company policies, procedures, guidelines
• trained with employee day-to-day tasks/responsibilities for employee profiling
• trained with threat intel feed and real-world examples
• the idea for this prototype is to create individualized training with feedback-based learning and conversational training with a chatbot. The chatbot would create storylines/conversations based on real-world examples most suitable to an employee’s responsibilities. Assessment can be taken in two ways. One during the conversation itself with the chatbot. The other one, a couple of weeks after training.
• A virtual universe with each employee as a character in the universe and protecting themselves in the cyber world.
• the idea behind this prototype is to gamify the experience. However, I did not consider the practicality and user experience for it when I was noting down prototype ideas.

During the session, I learned about prototyping, specifically ‘testing a prototype’. I had a discussion with my facilitator on two prototype ideas that are listed above. For the virtual universe, we were stuck on this question — would an employee opt-in and be willing to spend time and energy to learn about the game and participate?

For the GenAI tool, the question was — would it make the training enjoyable by making it relevant to an employee through storytelling embedded with emotions so that training would not feel like just another task to an employee?

After brainstorming, I decided to build a prototype for the GenAI tool idea.

LC#7 — Wrap up

It was time to build a prototype and test it. Since, I did not have enough time to build and train a GenAI model chatbot, I came up with a storyline with a couple of possible scenarios and decided to test it out with one of the stakeholders.

I met with a stakeholder to test the prototype. I took them through the flowchart, asking them their response as if it was happening to them in real life. I asked them about their feedback. The feedback was -

They liked -

• Requirement to engage with the chatbot
• Problem-solving elements
• Individualizes the process
• Open-ended conversations

They wished -

• Actual real-life examples would be much more effective in learning

The wondered -

• If there is a way to describe similar real-life experiences back to the chatbot, to learn more and validate our learning and actions so far

This was the last session, and it was the time to reflect and synthesise our learning. We used storytelling to frame the narrative about the work we did this entire semester. We used a couple of ways to frame the storylines. Below is the one I created using the technique — “What we learned” arc -

Some other ways I tried to frame my story while working on this project -

Reflection and Learning

Some happy faces at the end of this session. This is team — ‘Teach an IF to Phish’:

When I joined the fellowship, I was scared of being wrong. I was so much in love with the problem statement that I was afraid of failing to come up with a solution. But during the last session, I felt like — who cares if I am wrong? I learned a lot of things about myself, worked on my fears, tried to be more open about my vulnerabilities and enjoyed the journey — all thanks to awesome facilitators — Mira, Anat and members of this cohort.

Some of the learnings -

• Leave aside your preconceived notions and biases.
• Be more empathetic.
• Be open-minded and curious.
• Listen without judgment.
• I know a lot of stuff, but do I?
• It’s okay to be wrong. It’s okay to pivot. It’s okay!
• And enjoy the process!