It was the summer of 2010, during the first year of my PhD, when I was being examined by the dean of the Design School in her office, in order to progress to the second year. I was one of the newest students, trying to explain my research to the most senior academic in the department. I presented my preliminary research, including a pilot study performed to understand user behaviours and energy use. I had performed a user study with 4 participants and post-interaction interviews. She asked me how I analysed the interview data. I said I put the main ideas on a Word document and counted how many people said what. She then turned to a bookshelf behind her and said, going through her books, that I would benefit from… this, then pulled a book titled “Qualitative Data Analysis with Nvivo”, by Pat Bazeley.

From that moment onwards, the QSR International NVivo software had been my only tool to perform qualitative data analysis. Academic institutions usually have licenses available to all students and staff, making NVivo very popular among researchers. In more than one decade I’ve seen several versions being released, a Mac version developed, and the software improved with new features. But I’m also too familiar with its problems, like crashes (earlier versions), files not open by different versions of the software, the need to export a file from Windows to open on Mac, and the fact that we can’t “Save as…”, only “Save a copy” after you save the current changes, losing the original file.

New entrants

But recently, when I was working for the West Midlands Fire Service, I had to analyse data from workshops and interviews, and we didn’t have access to an NVivo license. So I started looking for alternatives, and the site alternativeto.net showed dozens of options. From the ones I found suitable, compatible with Mac, and allowed a few days of free trial, Dovetailapp.com seemed adequate. Condens.io also seemed able to do the job, but I had to choose one, so decided to subscribe to Dovetail.

Visuals

Right from the start I was impressed by the visuals. Using something other than that old look of a desktop application was a relief. I enjoyed the colour-coded groups of tags, and the adaptive / fluid layout to make use of my large iMac display. But there are some limitations. You can’t change the width of the group box. That may not be a problem for most, but if you like to create long, descriptive tags like myself, these names are all truncated in the group view. You can argue that once you hover over the tag, the full name will appear on the tooltip text after one or two seconds.

When organising your tags, this limitation becomes perhaps annoying, but certainly time consuming, if you have hundreds of tags. That’s because you want to know exactly what tags you’re moving between groups or merging. After some searching I found that there’s a way to show full names, if you display them as lists. When viewing your tag board, click on the […] to view the menu with the ‘view as list’ toggle.

Transcription

You can upload videos directly to Dovetail, and the transcription happens approximately 10 times faster than the duration of the media file. I had 1-hour interviews, which were transcribed in 5 minutes. For comparison, Microsoft Stream can also transcribe videos, but when you upload a recording, it takes longer than the duration of the video itself to process the transcription. NVivo can also transcribe your files, but I never tried this because it’s part of an extra package.

Another impressive feature is voice identification. After uploading an interview, Dovetail detects who’s speaking and assigns that to ‘speaker 1’, ‘speaker 2’ etc. Then you just have to assign that to a participant or yourself, and all occurrences will be then identified. One day I uploaded one video and Dovetail identified 3 participants. I thought there must be something wrong as there was only me and the participant, a firefighter working from the fire station. But when I got to that part of the transcription I noticed that it was the PA system that announced something in the background, and Dovetail clearly identified that as another person speaking.

Tagging

The activity of tagging interview contents is at the core of qualitative data analysis. Both NVivo and Dovetail systems help us do it manually, with pros and cons for both. Once you select a bit of text in Dovetail you can start typing to tag or to see the existing tags that contain what you typed. With NVivo you have to type exactly the first letters of your tags, but it’s easier to reuse recent tags as you can see the previous few tags used on the dropdown menu.

Tag hierarchy

One small limitation with Dovetail is that you can only classify your tags in 3 levels maximum. You have boards, groups and then the individual tags. During the user research for the West Midlands Fire Service I had a board for Suggestions, then a few groups listing all suggestions from users by touchpoint with the system. Inside these groups I had the tags themselves, from which I could access the videos and transcriptions. If I wanted to separate specific tags into a sub-group, there’s no way to do it. I’d have to create another group in the main board.

Having 3 levels for the coding hierarchy is probably enough for most researchers. But sometimes you don’t know your code structure until you tagged your content. Specially with grounded research or discovery phases where you don’t have a code framework to start with. With most of my qualitative research, I enjoyed this freedom, to be able to code freely and find the code hierarchy later. Also, I tend to code every statement from users into very unique and specific tags. Then I group, merge and classify them with time, in an iterative process. I move themes and create the code structure as I go along, until I have robust themes to represent the data.

With NVivo we can have unlimited levels for the coding structure. During one of my research with automated vehicles I had 4 levels and I could create more if needed. Participants were reporting on their experience after being driven by an autonomous pod. One high-level group contained tags where users were talking about how to improve trust and acceptance, and one of the sub-groups of tags were mentioning that we need to improve familiarity with the vehicles. One sub-sub-group contained the concept that the technology is not quite there yet, with the mentions that we need time to build trust, participants want to wait until the technology proves itself safe, and so on.

Online features

I was impressed by the online features I was able to use with Dovetail. You can invite your colleagues to view your analysis and results in nice looking pages. The insights pages can contain photos, quotes and video excerpts of interviews where they can view the evidence and thus add robustness to your findings.

Data protection

We have to consider extra precautions in relation to data protection to make sure it’s in line with the general data protection regulation law (GDPR). Both QSR International and Dovetail are Australian companies, meaning that data will likely be stored and processed outside the EU. Since GDPR came into practice, lots of people were concerned about data leaving the EU.

However, a blog post by Tom Tobim from Twillio defends that “there is a lot of misunderstanding and misinformation on this topic”. He says that data can in fact leave the EU as long as “this data still enjoys the same level of protection it gets under GDPR”. The service handling the data must be subject to a contractual obligation to follow GDPR data protection principles.

I checked the QSR website and they say that all NVivo products are GDPR compliant. It makes sense as it’s a desktop application and data is stored in a database created in your machine. Even if you’re using the online-based transcription engine, they say that “NVivo Transcription data is encrypted both in transit and at rest and only the account owner has access to and control over their data. QSR uses Microsoft Azure cloud services hosted in the EU. This is fully GDPR compliant”.

I checked Dovetail web pages and they say that “our customers can lawfully transfer personal data to our Services outside of the European Economic Area by relying on the Standard Contractual Clauses. Our DPA also contains specific provisions to assist customers in their compliance with the GDPR”. This sounded a bit vague, so I contacted their legal and security departments, who reiterated this, saying that “[w]e have a commitment to GDPR and have designed our services in line with the rules and guidelines set out in the GDPR. We also have a Data Processing Agreement available that customers can sign to ensure our data processing obligations and the relationship between processors and controllers under the GDPR. You can review and sign this agreement at https://dovetailapp.com/help/data-processing-agreement“. These safeguards sounded robust to me. Nevertheless, it’s good practice to check your employer’s internal policies and consult the legal department before committing to use an online tool hosted overseas.

Price

At the time of writing, NVivo was £881, when you buy the basic software license. So you can use it indefinitely for a single user. If you want to add features like transcription, it’s charged at £15 per hour, or £352 per year for unlimited transcriptions.

That’s where I think Dovetail is disrupting the market. I paid $15 per month to use all features including transcription of video files. That was an absolute no brainer for me, especially the non-commitment subscription – I cancelled the credit card subscription after a couple of months of use, once the project finished.

The bottom line

I’m happy that there are new entrants disrupting the traditional, desktop-based software market for qualitative data analysis. Since I had developed a love-hate relationship with NVivo, I was keen to experiment alternatives, and I’m confident that Dovetail is capable of replacing my old tool, at least for performing the main tasks that I personally require. And because it’s a web-based software and probably easier to develop and update, I believe there’s new features planned on their roadmap that could eliminate the current limitations with time.