Head in the Clouds

Many months ago a piece was posted here on Survey Tools, along with a classification of such tools. At that time it was noted that we had not reviewed Qualtrics in its entirety. In the aftermath of that post, I was given full access by the folks at Qualtrics for testing purposes and as a result the University subsequently purchased a campus-wide license for the product. What has transpired since its adoption here has bordered on transformational in the impact it has had on our community!

To date we have 132 registered users, who have created over 200 surveys that have generated over 3600 responses. While these numbers alone are impressive, they certainly do not suggest the need to use the word transformational in its impact. No, it’s only in reviewing the details behind those numbers that one might glean what the true impact has been. One of the more interesting uses and examples of its impact came in the classroom.

A member of the School of Business Faculty who offers a marketing methods class incorporated Qualtrics into his curriculum. While the events may not be in the exactly right order, here’s a brief summary of what transpired. NASCAR visited our campus in the fall. In the aftermath of this event, the professor discussed with NASCAR the possibility of his class conducting a study of NASCAR’s influence on our community, which NASCAR considered and accepted. Students in the class were divided into teams, given a charge as to the nature of the project, taught how to use the tool and then asked to develop and complete their study. The information was collected and then each team developed a report which was subsequently provided to NASCAR for consideration. Ultimately, one of the team’s reports was selected by NASCAR in terms of how valuable the information would be in potentially marketing NASCAR to a younger and culturally diverse audience. The end result for the students enrolled in this class is that they received their instruction on methods, were able to apply these lessons in a practical way, and netted a finished project that was considered by a real world entity. In addition, for those on the winning team, it provided a potential entry on to their resumes!

Add to this story those of faculty and graduate students using the tool for their thesis research, course evaluations, policy and procedure questions, and collaborative research projects, as well as administrative and academic departments applying the tool to evaluate the satisfaction and determine the needs of their clientele and it becomes easier to suggest that the arrival and the sum total use of Qualtrics on our campus is providing a meaningful framework through which transformational change can and is occurring on our campus.

There were two topics that deserved writing about, so I flipped a coin and CommunityWalk was pronounced the winner. What is CommunityWalk? It is part of a new breed of GeoMapping tools that have been designed for novice through sophisticated user.

A wizard takes you through extablishing the parameters of the site, which makes an already simple process, simpler yet. You indicate where you want to build a map, who can see it, and give it a title.

CommunityWalk Wizard

At this point you are presented a work area that that includes the map and tools to use in building your map. Using the Add Marker or Bulk Load tool you can start pinpointing locations you want to highlight. There are four ways to add a marker – search, address, coordinates or by simply pointing and clicking on the site you want to define. In defining the site, the ability exists to classify it, as well as to provide a description and/or picture.

CommunityWalk Workspace

CommunityWalk provides several ways to export your effort to web sites or you can export to your blog as demonstrated below. In addition, their forum page has a post on how to use with an iPhone.

CommunityWalk Map – Virginia State University

CommunityWalk provides both a free version, which includes ads, and a variety of premium packages starting at 4.95 per month.

While still in exploratory mode, this tool and others like it should present another opportunity to present the campus to prospects and others in a different and interesting way, which means you’ll see more on the topic in the future!

In the last post I mentioned that VSU has been gearing up for their annual IT Strategic Plan update. The process of collecting internal input for this activity has been designed to use three collection modes – facilitated sessions, individual meetings and surveys. For this latter grouping, and as expected by far the largest grouping, a set of questions that had been previously used, as well as some new questions are going to be used. When I arrived here, there was no Enterprise wide tool and the tool that was selected on a seat basis had not yet been procured. As a result, I have had the opportunity to use two, view a demo of yet another and research still others over the past few weeks in preparing the survey. What I learned from this exercise is that there are three classifications of survey tools that are available for use. While we might argue on what to call them, for the purpose of this posting we will call them Simple Form, Complex Form and Enterprise Form.

Simple Form Introduction

Within the Simple Form classification, I would include SharePoint Surveys. Based on the version of SharePoint that we are using at VSU, SharePoint Surveys are designed to be used in the most rudimentary of survey activities. That is, it seems like it would be ideal for use in a survey with a limited number of questions and to a well defined population. The following provides my review of several elements of SharePoint.

Sections: There was no functionality to permit sectioning with the survey. Nor was it evident that you could provide an opening introduction to the survey in SharePoint.

Presentation Templates: Based on what I could see, there was no tool provided during the survey development process to customize the look and feel of the survey.

Question Variety: The variety and form of questions that can be presented in the survey is limited to nine types and within some of the types there are presentation options. For instance, the Choice Type permits display in either in Drop Down or List form. In spite of this limited portfolio of options, designing questions can at times be confusing and periodically generates unexpected results. A sample of this is provided in Example 1 and 2, where the creation of Yes/No question with a default of Yes, results in an untitled box that is checked on the survey. Example 1: Creating a Yes/No Question in SharePoint. SharePoint does allow you to require a response, as well as to pre-fill a response. It is possible to reposition questions, but it is a rather cumbersome process in that you are provided a list of all questions and to move a question you need to accurately view what position number you want associated with the question.

Example 1: Creating a Question in SharePoint

SharePoint Survey Question Development

Example 2: Presentation of the Yes/No Question in the SharePoint Survey

Yes/No Response Box in SharePoint

Skip Logic: There was no ability to insert skip logic dependent upon response included in this version.

Data Analysis: The graphical presentation of results is also limited (Example 3), but can be downloaded into a spreadsheet for more detailed analysis. On front page you are only able to see the number of respondents.

Example 3: Summary of SharePoint Survey Question Response

SharePoint Analysis Report

Workflow: For all intents and purposes, there is no workflow and limited survey distribution available. The survey is posted within a folder in SharePoint, you can then cut and paste the address into an email, etc to inform the community of the survey. I also didn’t see a redirect capability that would allow you to forward the respondent to a thank you page once the survey is completed.

Other Similar Software: As of this moment, I have found no other package that provides as limited of functionality as this package.

Complex Form Introduction

Included among these vendors is Survey Monkey. These packages are often times offered solely on a per seat basis, although this is not universally true. In addition, they provide a more robust catalog of services, albeit not everything that would be desirable for constructing comprehensive surveys on a regular basis. The following provides my review of several elements of Survey Monkey.

Sections: There is the ability to develop the survey in sections and have an introduction to some or all of the sections.

Presentation Templates: Survey Monkey offers the ability to use pre-developed templates and provides for some limited template customization. For instance, I was able to load a University graphic to display at the top of the survey as well as make adjustments to the fonts and background colors.

Question Variety: Survey Monkey offers 15 types of questions for inclusion in the survey. Depending upon the question type, you have several options available. It is also possible to randomize response presentation, add a comment field for the question, require responses and pre-fill response. The flow is relatively simple and it is possible to reposition questions as needed in a reasonability easy fashion.

Skip Logic: There is a limited amount of Skip Logic that you can introduce through the survey design process. Specifically you can have two branches off of a question, but the branch must take you to a section and not a specific question. Survey Monkey suggests that you pose the skip logic question at the end of a section. This really didn’t work for me, so I developed sections that only had the skip logic question. Since you are able to reuse the Section Title, it was easy to follow-up with a detail section for people who responded accordingly. If you move sections around, as I did, you need to go back and review whether or not you need to re-point the skip logic. All in all, I found the functionality to be only serviceable.

Data Analysis: The data analysis capabilities found in Survey Monkey are only modestly more robust than that found in SharePoint. As with SharePoint, you can download responses to a spreadsheet. However, Survey Monkey does provide a little more flexibility in the manner in which data is formatted for downloading. The graphical presentation of survey results are definitely better than found in SharePoint, with additional information concerning how many people responded to a particular question (Example 4). Although I haven’t used it in any significant fashion, there is also limited Crosstab functionality.

Example 5: Summary of Survey Monkey Survey Question Response

Survey Monkey Analysis Report

Workflow: Survey Monkey provides three ways to promote the survey. The first is using Survey Monkey’s email function. Through this function you can either upload addresses manually or by using an address book that is created in Survey Monkey. There are significant benefits to using this function, as it will track respondents and have a means for allowing you to send a reminder message. The tool for creating the email message is rather rudimentary, but it works. The second method is to generate a link that can then be embedded in an email or on a web page. The final method provided, pop-ups on web pages, is the niftiest of the tools for what it does and what it takes to achieve it. After entering the some basic information, Survey Monkey generates a single line of code that you embed in your web page for the pop-up to appear. There is also the ability to permit respondent’s to save and return to a survey prior to submitting, but if I understand the methodology they use in determining the respondent, I’m not sure it would work where students are often times using a public lab computer.

Other Similar Software: There are a number of services that are similar to Survey Monkey. One that I visited is Surveygizmo. Each of these two services offers a free version, but each uses a very different method in limiting the free versions functionality.

Enterprise Form Introduction

Unfortunately, I have only had one demo and a limited free account of one Enterprise Form tool – Qualtrics. A single seat license for this product is about double that of Survey Monkey, however for institutions of higher education they offer an Enterprise license at about ten times the seat cost, depending up student enrollment. This realm of products is quantum leaps ahead of what is offered in the other forms, which in my mind would open up a wide array of potential uses throughout the Academic and Administrative Community’s within the University. Here are my observations of this product based on my limited insights.

Sections: Within the Qualtrics the term used is Blocks. Blocks permit several different options, including organizing your survey, creating branches, embedding text and/or URLs, and End of Survey. The branching logic appears to be pretty robust, especially when matched against the other Forms.

Presentation Templates: Qualtrics provides a number of templates that you can draw from. I’m fairly certain that the person who provided the demo stated that you could brand the survey, but I couldn’t see this functionality in the free version.

Question Variety: This is one of the areas where tools provided in the Enterprise Form blows the doors off the other Forms! While it was impossible for me to isolate the number of question types that were offered, I’m guessing that it is the fifties. Among the offerings that impressed me the most (although it is something that would not be used that frequently) are Hot Spots and Heat Maps. I could very easily see this being used in a web redesign project or as a tool for students in a Marketing Class. Another function that would be an asset in the design of the survey is a Page Timer function that is listed amongst the Question Types. Depending upon the way the survey is organized, this function could be used during testing of the survey in assessing the complexity of questions and thereby assist you in increasing the likelihood of someone completing a response once it is publically posted.

Skip Logic: When using Survey Monkey, it felt like I have to organize the survey around how I wanted to use Skip Logic. This certainly does not appear to be true in the Qualtrics solution. It appears, although it was not tested, that you are able to place the logic anywhere in the survey using the Branch Logic and better yet, you are not limited to two conditions. If I had used this tool, I would have combined my three survey instruments into one, using the Branch Logic to direct respondents according to their status, as well as based on responses to specific questions. Just a not however, that with this flexibility also comes a little more complexity.

Data Analysis: This is another area in which the Qualtrics solution excels, as it provides many of the bells and whistles one might desire when reviewing or reporting out on results. Beyond the bland reports that the others deliver, the Qualtrics solution offers an additional 24 or so different graphical outputs of the results that can then be downloaded to Word, Excel, PowerPoint or into a PDF. There is also the capability to post the results with a link that can be embedded into a web page.

Workflow: Qualtrics provides the tools to create a survey link, popup survey, embed a survey, or provide a web site feedback link. As the survey unfolds, you can view the number of respondents, send email messages to those that you wish to have participate in the survey, as well as to those that have not responded or a thank you note to those that have. The tool provided for constructing the messages is rather easy to use and provides the functions necessary to embed graphics, links, change formatting etc.

Other Similar Software: I haven’t really looked at other packages seriously, but Survey Methods appears to fall within the same classification.

Conclusion

While other tools may be appropriate given specific scenarios, there is no question that an Enterprise Form package makes the most sense both economically and in terms of functionality for institutions that are currently supporting multiple survey tool licenses.

Post Note

Other survey vendors not reviewed include: Digivey, StatPac for Windows, Creative Research Systems, QuestionPro, Vovici, Zoomerang.com, Checkbox Survey Solutions, Snap Survey, LimeSurvey (Open Source), Key Survey, PULSEWARE, and eSurveysPro.com

It has been some time, as usual, since the last posting. Of course, as usual, reasons abound for this lapse in postings. This iteration of my insights is dedicated to bringing you up to date on some of these reasons and how this might help create bountiful postings in the future.

Let this story commence in January, right after the holiday breaks wrapped up for yet another year. It was a Monday, when my phone rang and on the other end was Virginia State University’s CIO offering me a position as the Director of Strategic Management Services at the University! The discussion went on to conclude with a start date of January 25^th! So in a matter of a few short weeks the family and I had to find a place to live, pack, move and to the degree possible unpack. Needless to say, this was an exciting and challenging period that we met with unwavering fervor.

Any thought that there would be some time to breathe in the aftermath of this move, was abruptly dispelled as work awaited my attention! I won’t go into great detail here what has transpired since January, as this detail will hopefully provide sufficient fodder for upcoming postings. However, I would be remiss in not at least providing a description of the position and mentioning some of the high level activities.

So what is the realm (and only partial) of my responsibilities? As the Director of Strategic Management Services I report to the Chief Information Officer (CIO) and work collaboratively with departments of the University to provide the CIO with information regarding the direction that the University should take regarding technology initiatives. Among the roles played in this position include the development of comprehensive information technology strategic plans, monitoring project activities, investigating and analyzing operational issues, monitoring trends and advances in the industry, and making recommendations to improve and advance the use of technology within the University.

Fair enough! Then what is it I’ve been involved with since my arrival?

Not mentioned in the description, yet something that has absorbed some of my time is policies and procedures. In particular, five IT policies have been edited/developed, proposed, approved and posted.

There have been three other major involvements that absorb significant portions of my day. First has been the development and release of an RFP for Information Technology Outsourcing. At present, we just finished up a voluminous response to the questions posed by vendors related to this RFP and are just a few short weeks from being buried in responses. Second has been the gearing up of the institution for an implementation of an Enterprise Portal, which was given to us through a grant from the CampusEAI Consortium. And the final involvement has been ramping up the University’s annual IT Strategic Planning process.

Outside of these activities, I have also been pondering what it is that can be done with my primary site, AHigherVision.net. Most of the code was written by me and has been accentuated over time with third party applets. Unfortunately, I have found that my time for this site was mostly spent on fixing broken pieces as opposed to adding new content. This led me to revisit the question of whether there was a readymade framework that could be adopted for the site. The usual contenders were Joomla and Mambo, both Open Source Content Management Solutions, as well as several PHP based tools. I was about to give up again, when I decided to pose a functionality question on the Mambo Forum page. The attention received there in working through what was needed for the site and how to achieve it was nothing less amazing! As a result, I am in the early stages of moving and adding content to what will become the future home of AHigherVision.net. This is going to take time, but I believe it will add to the usability of the site, but more importantly will contribute vastly to the content that is being provided.

Well, you now have some glimpse as to where I’ve been over the past few months and at least some inkling as to where I’m headed.

Just a  short blurb today. In the world where news and information venues are proliferating at an incredible rate, it’s often difficult (especially in higher education) to find a dependable site for learning about what is happening among the major players in the ERP space or in some of the other technology niches. So when I learned a while back about  Edu1World.org , I took a peek. It was definately worth it!

Every week since then, I visit at least once to catch up on what is going on in the industry, as well as to read periodic contributions on other topics from Liz Dietz, Vicki Tambellini, and others. In recent months, they have expanded the site to include premium content. While I haven’t availed myself  of that content, it hasn’t diminished their value to me in keeping current on the ever changing landscape of higher education technology and the organizations that support them.

As all of this suggests, if your are interested in the topic of higher education technology, this surely should be one of your stops on the www express!