Observations of supervisors and an actuarial research student on the qualitative research process

5.1.1 Alex

The reason for choosing a research topic on actuarial education was mainly driven by my perceptions regarding part I of my undergraduate education. When doing part I, and completing those initial eight technical exams, I always wondered what actuaries do in real life. At that time, I felt that I would be less prepared for my further study and future career if I did not have a better sense of the practicality of the profession. Thus, I was incentivised to review the current actuarial education system as it would also give me insights about the overall purpose of all three parts, and how my future study might demand different skills to what I had learned up to this point in time.

In addition, my supervisors suggested that doing the research from an educator's perspective could be interesting, because reviews of professional education were seldom done exclusively from that point of view. This got my attention because as a recent actuarial graduate, I might be able to share some useful suggestions and thoughts based on my own 3 years of study (Part I) and my concurrent study of Part II. So, after some to-ing and fro-ing of ideas with supervisors, my research topic refined down to focus on actuarial educators’ viewpoints about the education system itself.

5.1.2 Supervisors

We are both, what might be termed, ‘keen educators’, who see teaching and education as a major attraction of our University role, rather than a hindrance to other academic priorities such as research. Our collective contribution to education includes policy and operational involvement in professional actuarial education beyond university study, a leadership (Deputy Dean) position in College-level education, and long-term experience as a programme convenor. From this experience, we are acutely aware that actuarial education in its entirety (both inside and outside the University setting) has been subjected, quite rightly, to a range of recent critiques (e.g. see Shepherd, 2005, 2009; Butt et al., 2014, 2016). Such critique has led to some calling for changes to the way education is delivered (e.g. see Shepherd, 2010; Chu et al., 2011). Such calls do not generally seek a decrease in the technical skills required to be an actuary, but rather to focus on developing such skills in the context of real-life applications, including greater use of peer review, group work, and open-ended rather than closed problems.

Given that the recent critiques look at the education system from the perspective of students, we considered that insights from the perspective of educators would be a complementary and constructive addition to this discussion – particularly as research into professional education is relatively scarce (Butt et al., 2014). Pursuing an investigation into education, which has professional implications, also seemed to be a responsible direction to direct our own research efforts (and aligns with Lodhia's (2017) call for research to be meaningful). This is because education is highly significant for any profession, in that it helps define what that profession is. For example, Daykin (2005) highlights the importance to a profession, of maintaining the quality of education of prospective members. It is also good practice to seek ways to improve any service over time, education included.

Alex was also the first research student that either of us had supervised, to use an entirely qualitative research methodology despite having a quantitatively-focussed undergraduate degree. Hence, it was also of interest to us to see how he would deal with far more open and ill-defined issues in the form of competing ideas and opinions, through a qualitative lens, a challenge which Alex realised would be different to what he had previously encountered. This might also then provide a repository of lessons from Alex, to any future student who faces similar challenges in undertaking qualitative research.

5.2.1 Alex

I had not previously conducted research that dealt directly with human subjects, so I first had to get approval from our University's human ethics research committee. Before doing so, I attended a 2-h information and training session, which was useful as it made me think about important issues of preserving confidentiality of participants, and framing research in a way to keep the risk of accidental identification of specific participants to a minimum. These were things I had not previously needed to think about within my studies.

The subsequent process of getting research approval from the human ethics office went more poorly than I was anticipating, for various reasons. Firstly, I did not source and use the most up-to-date template for the overall application, which was simply an oversight of not looking hard enough at the relevant material on-line. Secondly and later in the process, my application was delayed, for reasons such as not providing the protocol number, not following the required format, missing certain information, not explaining things well, and so on. I realised that after three occasions of the human ethics office returning the overall written application to me, that I was not portraying a particularly professional approach, which I look back on with some disappointment. Although much of the ethics application was discussed in general with my supervisors, the first couple of times I sent it in for approval without getting their comments on the detail.

My shortfalls and frustrations in this process taught me an invaluable lesson – the importance of getting approval and feedback before taking actions, where possible. Ironically, doing so approach mimics much of what we are taught in part II actuarial subjects (which I was studying concurrently with the research), whereby appointed actuaries consider and/or approve any changes to matters of pricing, reserving, capital requirements and reinsurance arrangements prior to those things occurring. The analogy to my research is that I now see that although the approval process might take time and effort, it is actually meant to empower you and your research. It helps you consider multiple factors and perspectives, allowing you to proceed with confidence that others with more experience have peer-reviewed and aligned the research approach with more appropriate standards than what I started with.

In hindsight, I appreciated the rigour applied by the human ethics office. Their feedback highlighted that I was doing a serious research project, which needed to abide with established guidance.

5.2.2 Supervisors

In terms of the ethics process, we gave some thought to two possible supervisory approaches. Firstly, letting Alex drive the process as far as he was comfortable, or alternatively, taking a more nurturing, parental type approach. Given Alex's demonstrated high achievement and the high levels of energy he was bringing to the research, we favoured letting him do as much of it himself, so as to allow opportunity to demonstrate initiative, energy and proactivity with what was an unfamiliar task – with us just providing some guidance on the process.

The decision between those two positions was also made to some extent for us, with Alex drafting and submitting ethics application(s) before getting detailed supervisory feedback. With the benefit of hindsight, whilst our admiration of his enthusiasm remains undiminished, the second series of feedback from the human ethics committee, which pointed out omissions and inadequacies in his proposal, could have prompted us to step in and exercise our role more deliberately. This would have had the twin benefits of ensuring Alex did not get more frustrated and demonstrating to the human ethics committee that the proposal was being taken seriously and professionally.

Perhaps the best learning for us as supervisors is that whilst it is certainly good (and ideal) for research students to have freedom and opportunity for initiative within their research, it is also of much benefit to ensure a process exists for feedback at key stages. On this note, it is encouraging to see Alex's own reflections on this matter and also to see him relating it to his other courses, with benefits in seeking advice and not hurriedly proceeding beyond what you are currently equipped for. Getting such feedback can and should be an important tool to assist planning and provide improvement.

5.3.1 Alex

Finding out people's perceptions lends itself naturally to a qualitative research method, for which one tool is interviewed. I also considered a survey, which can be considered as qualitative where questions are open-ended.

I preferred an interview approach over a survey approach, for many reasons. Firstly, the number of lecturers in actuarial studies and who are actuaries themselves is quite small in Australia (approximately 30 in mid-2016). Hence, a small sample may not lend itself to a survey method where it has a goal to draw out meaningful statistical analysis. Secondly, although a survey can be completed more conveniently in many cases, it lacks the opportunity to ask respondents about their thoughts beyond the set questions. In comparison, an interview is a set time commitment of 45–60 min, but the interviewer can communicate more strategically with the interviewee. For example, when a general question like ‘how do you feel about Part I education?’ is asked, the respondent usually provides a few thoughts. Each of these might each lead to extra ideas and insights from the interviewees, with the interviewer then having to make a trade-off, by reducing time allocated to other questions, to allow for this new exploration. As I anticipated that participants would likely have a variety of opinions about the education system which would be difficult to exhaustively cover within a survey format, an interview method was preferred.

Nevertheless, to encourage participation, when contacting possible participants I offered a choice of how they could participate – via a verbal interview, or a written questionnaire. The use of focus groups was also briefly considered but logistical issues involving the amount of time required, the coinciding of available time, geographical separation, and costs of providing an experienced focus group facilitator, meant that this was not considered further.

5.3.2 Supervisors

Alex quickly assessed the various approaches open to him of interviews, surveys, and focus groups. He saw that interviews are a convenient way to explore the perceptions of others (as Stake (1995) points out), and can be used to keep responses relevant to the intent of the research goals (as Ospina (2004) points out). The choice of interview method is also supported by the fact that various other studies in the broader realm of financial services are similarly based on an interview approach (e.g. FSA, 2009; Foster and Warren, 2016).

At this stage of the process, Alex also began to talk with us about other key issues involved with conducting interviews, such as being aware of one's own biases, which can be mitigated to some extent using open-ended questions. This gives interviewees some discretion to focus on what they consider important (Mathers et al., 2002; Ospina, 2004). Alex discusses this in the next section.

We also considered that interviews were an appropriate approach, and aligned with the ideal of having research goals drive the methodology (Abernethy et al., 1999; Flyvberg, 2006; Broadbent and Unerman, 2011; Kaczynski et al., 2014; Lodhia, 2017). What was particularly interesting was to observe and help direct Alex, as a highly achieving ‘quant’, to manage and reflect on the research process, which this article helps him articulate further.

5.4.1 Alex

After getting ethics approval to conduct interviews, I contacted possible participants from a list of thirty actuarial lecturers around Australia (of which 25 were Fellows of the Actuaries Institute). It might sound straightforward, but sending out 30 individual email invitations and then keeping on top of responses did mean I had to keep careful documentation, with some of those contacted answering promptly, and others not so promptly. Keeping a separate and dedicated folder in my email account helped have a clear and easily accessible history of communication with all contacts.

Thirteen of those contacted agreed to participate. Five agreed to a face-to-face interview, seven agreed to a phone interview, and one agreed to do a written interview in the form of a questionnaire. The twelve nonwritten interviews each ranged from 45 min to 1 h.

I felt that face-to-face interviews were far more effective and manageable than phone interviews, for two main reasons. Firstly, there was real benefit with the presence of body language and facial expressions within face-to-face interviews. For example, if a respondent was not interested in the topic, they might raise their eyebrows and start with an expression similar to, ‘well…’. If this occurred, I could adjust the discussion, to get the most out of the interview. Secondly, the performance of phone interviews was highly dependent on the quality of the phone connection. One interview had a problem as I could hardly hear the interviewee's voice. Thankfully, that interview was recorded and hence I could still capture 90 percent of the conversation, through repeated listenings. This taught me another lesson in that, if the quality of the call is bad, I need to be bold enough to politely explain and restart the call. However, this was not the easiest thing to do as I realised it could cause some frustration for interviewees whose participation was, of course, voluntary.

What did not work so well was the written interview/questionnaire. This participant did not have time for a phone interview, and hence I sent them a written list of my indicative interview questions. Perhaps due to their own time limitations, the responses were brief with only ‘yes’ or ‘no’ given to most questions. I then realised that questionnaires should be constructed quite differently to questions used as prompts for interviews, and again should have checked more deliberately for guidance on this from my supervisors before conducting something different to what I was used to.

I found that an important skill for interviews is writing key points during the interview. Writing down key words and thoughts as ‘live’ prompts to link with my intended topics was helpful because it might help the interviewee understand future questions more quickly, and give them a sense of familiarity and flow to the overall interview. Writing key points down is possible rather than trying to capture everything the interviewees have said, as I also used an electronic recording device for later partial or full interview transcription.

It was also useful to take a semi-structured approach to interviews. I first asked relatively general questions, to understand which kind of topics they have already thought of and what topics interested them. Then, according to their interest, I could ask them more detailed and specific questions. Although before each interview I had a list of points that I intended to discuss, they were best seen as a guide to help me decide the amount of time and focus allocated to different questions and topics. This is a good way to conduct interviews and is one definite advantage over surveys.

Time management was another issue. Usually, I spent 15–20 min on the first part of each interview, which involved asking general questions. However, two enthusiastic interviewees spent almost 40 min on this first part – which meant I later ran out of time to cover all specific questions that I wanted to ask. Communication is clearly an art which needs practice, as sometimes it is hard to politely change the direction of a conversation. In subsequent interviews, I prioritised later questions and deliberately spent less time on the first section of the interview.

I also learned to deal with pauses in conversations, which sometimes involved thanking the interviewee for their comment when there was a noticeable pause. I noticed two types of conversational pauses - one long, and one short. A short break of usually 2–3 s was used to take a deliberate pause for breath or recall things from memory. However, a longer break of 5-s or more was used to think of some extra points that required something more than instantaneous replies, and/or involved something that they were unfamiliar with or hadn't considered in any depth before. If time allowed, it was then interesting to hear extra ideas, but if time was short, I preferred to concentrate on opinions that interviewees had spent time considering as I had sent interviewees a copy of the indicative interview questions prior to each interview.

After each interview, I would promptly write a summary of the interview and send this back to the interviewee. This was useful in checking and validating the correctness of my understanding of the interviewee's opinions and comments.

5.4.2 Supervisors

We observed that Alex's confidence grew over time. He seemed to find it particularly useful to write summaries of each interview promptly after each one, to benefit both his own learnings as well as having talking points to share with us in later meetings. It was very useful for him to electronically record each interview, despite this leading to another time-consuming step of transcribing and/or repeated listening to several interviews. He recognised that this approach of recording, re-listening, and getting participants to check his interpretation and recollection was a useful mitigation to his own potential biases and misinterpretations (see, for example, Marshall and Rossman, 2010).

Alex took the approach to first ask general questions about strengths and weaknesses of the actuarial education system, to give interviewees the opportunity to focus on those aspects that were of most interest to them. This is line with others who have researched via semi-structured interview techniques (e.g. Foster and Warren, 2016; Sahi et al., 2013; and see also Abernethy et al., 1999), and also suited what Alex was researching. Then, asking about specific aspects such as consistency between syllabus and assessments, gave interviewees additional freedom to express their comments on those and wider aspects in as much detail as they had given thought to. We also prepared Alex a little through practice interviews with each of us.

Alex did not discuss in depth with us many of his thoughts about the conduct or process of interviews themselves, but did discuss findings and insights he had from interviews. From such discussions, it was clear that he was picking up on the major education issues that we expected him to find. It was also clear that he was confident enough to interpret or view findings in lieu of his own thoughts about the education system. This made for interesting discussions about his own views, which changed over the course of the research – a good sign of real engagement with the research process.

5.5.1 Alex

Having recorded each interview electronically, I was able to review all interview content at any time. Originally, I planned to write an extensive summary of key insights from each interiew, but slowly recognised the benefit of listening to the whole recording again. Therefore, I chose to transcribe as much of the relevant conversation(s) as possible, so that I could easily cite this dialogue in the future.

The use of Microsoft Excel helped tabulate and categorise discussions from different interviews. This categorisation of discussions was a challenge, as I had to rely on my own subjective definitions and interpretation, which was initially quite unsettling. For example, a comment could be that ‘it would be good to have universities provide professional exams in Part III, but universities just don't have the capacity to do that’. In this case, should I categorise this as supporting involvement of Universities in Part III education, or as a shortfall of university-based education in professional services? Such categorisation was subjective; therefore, also was the building up of a framework for reporting the key themes that emerged. Nevertheless, with the benefit of a having recorded dialogue of each interview, I could read each statement multiple times, to classify comments into similar themes as best I could.

Another challenge was how to draw useful conclusions, or to make suggestions to improve the education system, as there was a variety of opinions provided by interviewees. Usually, my recommendation followed the majority of my interviewees, such as assisting students with getting work experience or internships. However, I also supported minority opinions if they were aligned with what constitutes an ‘ideal’ education system based on my earlier literature search. One such example was my suggestion to make greater use of practical examples in the classroom.

I was also cognisant that I was assuming to a degree, that the opinions of thirteen respondents had some representativeness of all actuarial educators in Australia. Given the relatively small number of actuarial educators anyway, and that there was a likely selection effect to some degree with participants being those who have given some thought to the topic, I was reasonably comfortable that this assumption was not wildly unreasonable.

One key learning that I would pass to students who will do similar research in the future is to try to copy much of the interview dialogue down. There may not be a need to fully transcribe all conversations, but it is very useful to write down whole sentences of key discussions. Having such dialogue available means you can review them as many times as required, to make the soundest conclusion possible, based on readily available evidence in the form of quotations. Although this may take time initially, it saves time later when compared to going back and re-listening to original recordings that you may not immediately recall or know which part of an extended conversation was the most useful.

5.5.2 Supervisors

Alex came up with a framework to assess whether the current actuarial education system is robust and effective, in supporting the professional drivers and expectations of the actuarial profession. In doing so, Alex based insights on two things. Firstly, by weighing up and assessing often contrary opinions from interviewees. He conducted enough interviewees to pick up on the major pros and cons of certain parts of the education system and was able to think on the various opinions offered and then come up with his own picture of how things work and could be improved. This often meant siding with those views he had heard most frequently, but this was not exclusively the case. Secondly, by assessing insights in light of the background research he had done on what constitutes an ‘ideal’ education system. This meant that sometimes the views he heard less about were the ones he agreed with, if he judged that these views were aligned with what had made sense to him from his own background research and literature review.

In terms of process, his organisation and construction of argument improved as his thoughts consolidated over time. This offers some encouragement to others who may embark on similar research, as the early days can be obscure, confusing and unsatisfying, as nothing seems particularly concrete nor able to lend itself to any particular framework for understanding what can be complex questions. As for Alex though, as one's own understanding of the ‘context’ grows over time, clarity out of complexity is possible.

We imagine that Alex would now be surprised about how much clearer his thoughts are on actuarial education, as compared to what they were at the commencement of his research.

5.6.1 Alex

I found it very natural and useful to cite extended, word-for-word dialogues from interviewees when writing up my thesis. This was useful for several reasons – one being that because I found it a challenge to make a judgement about both appropriate classifications of comments, and of the ‘rights’ and ‘wrongs’ of certain views, then letting readers make a judgement themselves whether they agree with me or not was best served by providing as much context for comments as possible – which most easily came from the words of participants themselves. Word-for-word citations provided excellent ‘contextual’ evidence in this regard. It was also useful to demonstrate links between differing points made, as we may like to categorise ‘points’ as separate things in some deductive fashion, but of course this is not necessarily how people work or think in reality. I also found that reading participants’ words makes for easier reading in other papers, so wanted to replicate that in my thesis.

Another key thing I learned is that, when you are writing an academic research report, you are best to treat your own views as those derived from the research itself and to take care with assuming too much. I was an actuarial student writing about actuarial education, which naturally means that I have my own opinions about the education system. As such, I started writing with expressions such as ‘I think that….’, and looked for evidence in relation to my thinking. However, my supervisors quickly pointed out that it is more convincing and informative to have insights, conclusions and recommendations starting from the research (background literature and interviews) itself. Although within discussion and conclusions I can certainly add in some of my own perceptions, they are a supplement to, rather than a basis for, research findings. Research findings need to be found in research, not oneself, otherwise it is not really research. This made sense to me, but I was surprised by how I could have constructed a case based on my views as a starting point rather than keeping a level of objectiveness and unbiasedness as much as possible.

5.6.2 Supervisors

It is challenge with interviews to distil insights down to a small number of themes or key points. In both spoken and written dialogue with us, Alex was very clear what views he was heading towards and why he was heading that way. As indicated above, he even changed his mind on some key issues (such as the appropriate timing within the overall education programme, of introducing ‘soft’ skills such as communication, versus ‘hard’ skills such as actuarial mathematics), which attested to us that his own thinking was changing not only in line with what he was hearing, but also in what he was processing and formulating as he was writing things down.

Alex's approach to present raw data in the form of interview quotations, to increase the assurance that what he understood was a reasonable deduction from what was said in interviews, aligned with good practice in qualitative research (see, for example, Shank, 2002; Eisenhardt and Graebner, 2007). As supervisors, it was encouraging to see him appreciate the practice used by others and to critically assess this practice in the light of his particular research topic.

From previous students’ and our own experiences, we know the importance of avoiding procrastination when writing up a thesis – the longer it is left, the harder it gets, and the sooner you start, the more you realise that if you have worked hard, you should find plenty to write about and you will wish you started sooner. Alex did well here – he did not delay the writing up too long, had substantial drafts of the thesis ready in time for feedback to be given in an iterative fashion, was comfortable in presenting partially written sections for interim feedback along the way, he dealt with such feedback directly and openly, and yet still retained his prerogative to write what he wanted to write. This made for an enjoyable process of writing and review.

5.7.1 Alex

In summary, my learning process in this research thesis was very positive. This related to both what I found out about my research topic, and to the research process itself.

In terms of the research topic, I can see now that from an educator's point of view, it is not straightforward to build up an education system that is suitable for every single student, as each student has different characteristics and learning styles. There are many factors to consider, such as trade-offs between teaching a strong theoretical and technical base, with that of using a more applied approach to help students be more adaptable to real-life practice in the future. There are practical realities to consider such as limited resources (time, staff, money), rapidly developing technology, and adherence to professional syllabus requirements, all of which are important factors, and which suggest educational challenges that I had not thought of before.

Despite my enjoyment of, and interest in the research, I do not consider that I could provide a focussed and insightful recommendation for improvement, to take the education system from its current state to an ‘ideal’ status. What I consider surprising about this, is that if at the commencement of the research I thought I would not have a firm conclusion at the end, then I would have been quite disheartened – but in reality, this outcome has not bothered me too much. In doing so, I have taken on board two key realities of actuarial work – that meaningful problems can be difficult to solve (and even understand), and that iterative steps towards something better may be more appropriate than immediate large changes. This reflects that what constitutes ‘ideal’ in education does not mean perfect solutions are available. If I was to follow up the thesis with additional research, I would increase my sample size, and extend to actuaries who are currently working in the industry. As one point of advocacy I made was to bring greater practicality into university actuarial education, it would be interesting to hear how employers in the field feel about this, and whether they would support such goals (e.g. by providing more internships and supervision on applied projects).

I have learned a lot from this research, and various skills have improved, including writing, reading, and communication skills. Slightly more nuanced skills include the ability to weigh up and summarise different opinions, and advocating and arguing for different ideas. I have since started an industry graduate role as an actuarial analyst, and see that the skills learned stand me in good stead for the future.

5.7.2 Supervisors

Alex made a useful and novel contribution to an issue of some importance to a professional body. It was encouraging as supervisors, to see a high-achieving quantitative graduate extend his repertoire of skills into qualitative and essential more relational-based spheres, and to enjoy the process of doing so.

Alex achieved a good grade for his thesis, which contributed to gaining a first-class honours degree in what is a very challenging and selective discipline area. He also managed to have a summary of his thesis and findings published in an industry-relevant, peer-reviewed journal. Alex has thought about and suggested some natural extensions to his thesis, has a genuine interest in academic research at some stage in his career, and most definitely has a range of familiarity with and awareness of the benefits of research methodology that sits outside his own realm of having a predominantly quantitative focus.