2006 User Survey Results

Survey Results

Many thanks to the 256 users who responded to this year's User Survey. This represents a response rate of about 13 percent of the active NERSC users. The respondents represent all six DOE Science Offices and a variety of home institutions: see Respondent Demographics.

The survey responses provide feedback about every aspect of NERSC's operation, help us judge the quality of our services, give DOE information on how well NERSC is doing, and point us to areas we can improve. The survey results are listed below.

You can see the 2006 User Survey text, in which users rated us on a 7-point satisfaction scale. Some areas were also rated on a 3-point importance scale or a 3-point usefulness scale.

The average satisfaction scores from this year's survey ranged from a high of 6.7 (very satisfied) to a low of 4.9 (somewhat satisfied). Across 111 questions, users chose the Very Satisfied rating 4,985 times, and the Very Dissatisfied rating only 51 times. The scores for all questions averaged 6.1, and the average score for overall satisfaction with NERSC was 6.3. See All Satisfaction Ratings.

For questions that spanned the 2006 through 2003 surveys, the change in rating was tested for significance (using the t test at the 90% confidence level). Significant increases in satisfaction are shown in blue; significant decreases in satisfaction are shown in red.

Areas with the highest user satisfaction include the HPSS mass storage system, account and consulting services, DaVinci C/C++ compilers, Jacquard uptime, network performance within the NERSC center, and Bassi Fortran compilers.

7=Very satisfied, 6=Mostly satisfied, 5=Somewhat satisfied, 4=Neutral, 3=Somewhat dissatisfied, 2=Mostly dissatisfied, 1=Very dissatisfied

Areas with the lowest user satisfaction include Seaborg batch wait times; PDSF disk storage, interactive services and performance tools; Bassi and Seaborg visualization software; and analytics facilities.

7=Very satisfied, 6=Mostly satisfied, 5=Somewhat satisfied, 4=Neutral, 3=Somewhat dissatisfied, 2=Mostly dissatisfied, 1=Very dissatisfied

The largest increases in satisfaction over last year's survey are for the Jacquard linux cluster; Seaborg batch wait times and queue structure; NERSC's available computing hardware; and the NERSC Information Management (NIM) system.

7=Very satisfied, 6=Mostly satisfied, 5=Somewhat satisfied, 4=Neutral, 3=Somewhat dissatisfied, 2=Mostly dissatisfied, 1=Very dissatisfied

The largest decreases in satisfaction over last year's survey are shown below.

7=Very satisfied, 6=Mostly satisfied, 5=Somewhat satisfied, 4=Neutral, 3=Somewhat dissatisfied, 2=Mostly dissatisfied, 1=Very dissatisfied

Survey Results Lead to Changes at NERSC

Every year we institute changes based on the previous year survey. In 2006 NERSC took a number of actions in response to suggestions from the 2005 user survey.

1. 2005 user survey: On the 2005 survey 24 users asked us to improve queue turnaround times. Seaborg wait time had the lowest satisfaction rating on the survey, with an average score of 3.95 (out of 7).

   NERSC response: In 2006, NERSC and DOE adjusted the duty cycle of NERSC systems to better balance throughput (reduced queue wait times) and overall utilization, and also agreed not to pre-allocate systems that are not yet in production. This approach has paid off: on the 2006 survey only 5 users commented on poor turnaround times, and the average satisfaction score for Seaborg wait times increased by almost one point.

2. 2005 user survey: On the 2005 survey three Jacquard ratings were among the lowest seven ratings.

   NERSC response: In 2006 NERSC staff worked hard to improve Jacquard's computing infrastructure:

   • We implemented the Maui scheduler in order to manage the queues more effectively.
   • The system was greatly stabilized by reducing the system memory clock speed from 400 MHz to 333 MHz (more nodes were added to Jacquard to compensate for the reduced clock sped).
   • We worked with Linux Networx and its third party vendors to improve MVAPICH.
   • We worked with Mellanox to debug and fix several problems with the Infiniband drivers and firmware on the Infiniband switches that were preventing successful runs of large-concurrency jobs.

   On the 2006 survey four Jacqaurd ratings were significantly higher: those for up time, wait time, and queue structure, as well as overall satisfaction with Jacquard.

3. 2005 user survey: On the 2005 survey four users mentioned that moving data between machines was an inhibitor to doing visualization.

   NERSC response: In early 2006 the NERSC Global Filesystem was deployed to address this issue. It is a large, shared filesystem that can be accessed from all the computational systems at NERSC.

   Moving files between machines did not come up as an issue on the 2006 survey, and users were mostly satisfied with NGF reliability and performance.

4. 2005 user survey: On the 2005 survey 17 users requested more hardware resources.

   NERSC response: In addition to deploying the Bassi POWER5 system in early 2006, NERSC has announced plans to deploy a 19,344 processor Cray XT4 system in 2007. User satisfaction with available computing hardware at NERSC increased by 0.3 points on the 2006 survey, and only ten users requested additional computing resources in the Comments about NERSC section.