Methodology in Web Science

I’ve been brewing a bit of a rant about methodology and WebSci. Here we go:

Experiences and subjectivity

Experiences are important, and the study of web-mediated experiences is an important part of Web Science. For example, consider questions about how people use the mobile or social web, or evaluating web design processes. Such areas are, of course, deeply subjective…

This picture could show a happy memory, an arduous trek or just a helpful example of subjectivity...

My doctorate was very much in this area. Coming in as a Computer Scientist, with knowledge of quantitative but not qualitative methods, I found it pretty tough going for a while — I’m happy to say that I got there in the end.

“How did you get there in the end, Clare?”

I’m so pleased you asked. Mixed methods, of course!

Mixed methods

Traditionally, the phrase ‘mixed methods’ refers to the use of both qualitative and quantitative approaches, and I certainly did that. I also used ‘mixed methods’ in the sense of running studies in the lab and in the field. I found both types of ‘mixing’ (changing the type of the data, changing the context of study) to be very valuable: I was better able to ‘trangulate’ my results, corroborating what I already had and being able to investigate intriguing questions from alternative perspectives.

Methodology

So: mixed methods open up richer insights. It’s essential, however, to understand methodology as well as method — that is, how methods are used together. For example, statistical analysis and qualitative coding can help corroborate findings; expert reviews help you ascertain deeper insight into prior results; case studies let you build on lab data while answering ‘how’ and ‘why’ questions.

Mixed methods isn’t as hard as it sounds! It enables multiple perspectives in on data, so you can corroborate results and further investigate the facets of interest. However, it’s essential to understand methodology as well as methods.

Where’s the rant, Clare?

First, two concerns about methods in WebSci:

  1. There is a perceived ‘pressure’ in some fields to seek quantitative results, and to assume the presence of some stats means the work is rigorous. We should be careful, here.
  2. Are we doing stats right? (There’s a debate about this in HCI: see this CACM article.)

The ‘rant’:

I came into the Web Science arena as a Computer Scientist, equipped with maybe half of the ‘WebSci and methods’ equation. I am not an outlier: most people coming into WebSci from a traditional discipline will face similar barriers.

It’s really important that we as a community talk about education: if Web Science is the overlap of multiple disciplines to study society and the web, what tools are we using? WebSci courses are being launched all over the place at the moment — which is wonderful! — but are students being equipped with suitable methods?

Further reading

On mixed methods:
The pressure for quantitative results
Rigour in quantitative and qualitative methods
Mixed methods in WebSci
Advertisements

4 responses to this post.

  1. This is what I want from Web Science education: understanding what methods are meant to be used for what purposes — what kinds of questions you should approach from which angles.

    It’s why I hate the butterfly diagram — showing “overlaps” between fields doesn’t show what you should do with those together (or what you can’t or shouldn’t).

    “It’s essential … to understand methodology… how methods are used together. For example, statistical analysis and qualtative coding can help corroborate data; expert reviews help you ascertain deeper insight into prior results; case studies let you build on lab data while answering ‘how’ and ‘why’ questions.”

    Reply

  2. […] later). We also got onto mixed methods and methodology — for my part, I re-ranted my methodology and methods rant. My main point: mixed methods yield richer and more certain results, but we need to understand […]

    Reply

  3. Posted by jrduboc on February 25, 2012 at 18:09

    Reblogged this on Dub0c.

    Reply

  4. […] always been interested in how we go about research (hence past posts on methodology and mixed methods, and introspection about the make-up of the Web Science community). For some time I’ve been […]

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: