RES 5040: Teacher as Researcher (spring 09)

The data available from Google Analytics about both the Historical Photos Collection and the library websites overall covers June 1, 2008 to present. Since this is less than one year, parallel comparisons of one year to another are not possible, and other comparisons of different months are of limited validity. For example, comparing September 2008 to January 2009 is problematic because the university was in session unequal amounts of time during those months. Different parts of the semester may also see different use patterns as well. Images are still being uploaded to the digital library, and ASU’s library websites in general is developing as well. They are evolving entities--a consideration that must be kept in mind when evaluating them at different points in time. Interpretation of the large amounts of data available through Google Analytics must therefore be carefully done to avoid misconceptions. It must also employ a determination to explore and uncover everything that the numbers can reveal--there seem to be possibilities in this set of data.

I chose 11 reports to run initially of www.library.appstate.edu (overall ASU library sites) and http://contentdm.library.appstate.edu (the Historical Photographs Collection) after being granted access to those domains by the Belk Library staff. The 11 reports are:
Dashboard
Visits for all visitors
Pageviews for all visitors
Average pageviews for all visitors
Bounce rate
Average time on site
% of new visits (new vs. returning)
visitors overview
map overlay of the world
traffic sources overview
content overview

The reports include about 115 pages of data to analyze. Broad comparisons can be made between the sites on a number of key indicators. To facilitate this, I set up two binders, one for the Historical Photographs Collection and one for the Library and Information Commons, with reports organized the same way in each for an easy side by side comparison. As I began to look at the data it did become apparent that there were differences in the patterns of use.

One area of interest to me is geography, and I began to focus on the map overlay of the world. The library gets internet visitors from 187 countries and the Historical Photographs collection gets visitors from 78. This is a large but understandable difference due to the discrepancy in volume of traffic--1,273,837 for the library and 10,462 for historical photographs during window of observation (6/1/08-3/11/09). The number of visitors from each country is given in order. All this was fascinating to me, but I wanted more--the percentages of visitors that came from each country, which was not given in the report. Common sense told me that countries with 1 or 2 visitors would have almost meaninglessly small percentages given the large number of total visitors, especially for the library as a whole. I decided to start by finding the percentage of visitors from the United States, the bulk of the total. By my calculations, 94.7% of the HPC visitors were from the USA, and 98.1% of overall library visitors was from the USA. It seemed like a small, but significant difference.

What did it mean? Upon reflection, the results did not seem right. The Historical Photographs Collection is a local history collection of the college. I would expect it to have more domestic visitors. The university as a whole has international programs that would lead me to believe that it would have a higher percentage of foreign visitors than the HPC. After thinking about this I began to suspect that the CONTENTdm reports involve more than the HPC, and this appears to be the case.

While initially frustrated that the reports run and time spend will not yield meaningful data analysis, I am happy that I was able to evaluate the data carefully enough to detect a problem that would have nullified the validity of the web metric component of my research. While I am not a natural with technology, the key component of research is critical thinking.

Web metric comparison of the HPC with the ASU library as a whole is not feasible given what I have discovered about the domains I was given access to. I may be able support usability tasks/interviews with web metric information about specific pages that come up in the research, but the primary source of data will have to be the tasks and interviews. Qualitative research 1, quantitative research 0.