The 2008 Best Firms To Work For ranking process

Although each year the evaluation questions are reviewed and updated, the evaluation process does not change significantly. ZweigWhite announced in February 2008 that applications were being accepted and posted the application form (known as the Corporate Survey) online. Firms responded to the questions and returned the application. Next, the firms launched an Employee Survey to their staff. Firms were told to achieve a minimum 20-percent response rate for the anonymous survey.

Next our staff evaluated the Corporate and Employee Surveys, which make up the two equally-weighted components of the grading process, for all firms, and determined a raw score for each component. (The method used to determine the raw scores is described below.) We used the standard deviation and mean to distribute the firms’ scores for each component, which gave more weight to firms that scored well outside of the average. We used the sum of the distributed scores for both components to rank the firms.

The top three firms made it to the final round. The judging panel will re-rank these firms based on their best judgment. We will tally the judges’ results to establish the ranking order of the top three firms.

Determining raw scores
The following background information may be more in depth than you need, as it describes the method we used to determine the raw scores for each component. However, you may be interested in the criteria/method used to determine which firms made it to the final round and the ranks of other firms in the competition.

Corporate Survey—Our staff evaluated the Corporate Survey (CS) for all competing firms. Most questions on the Corporate Survey were evaluated, using one of two methods. The first type compared a firm’s response with the median response of the contestant pool. If a firm met or exceeded the median, then a point(s) was awarded. For example, the median voluntary turnover rate for all the firms that entered the ranking was 9 percent. Therefore, if a firm had a 9 percent or lower turnover rate, it earned points; firms with a turnover rate higher than 9 percent did not earn any points. Questions deemed to quantify more important aspects about firms were worth more points than those questions indicating less important information about firms. For some particularly important questions, bonus points were awarded if a firm met or exceeded the third quartile (75th percentile) of all firms.

The second method simply awarded a point(s) if a benefit was offered or if the affirmative response to a question was given. For example, if a firm offers a flexible medical spending account to staff, it earned points; likewise, if a firm said it has a nondiscriminatory culture regarding age, gender, race, and religion, it earned points. More important attributes were worth more points; for example, hosting a company picnic annually wasn’t worth as many points as beating the average voluntary turnover rate.

Questions were arranged by categories, including culture, benefits, performance/ recognition, compensation, professional development, recruiting & retention, and general.

Employee Survey—All of the non-narrative Employee Survey (ES) questions were graded (other than the demographic questions) using a method that compared, for each question, a firm’s average response to the average response of the contestant pool. For example, employees were asked to respond on a scale of 1 to 6, where 1 is the most undesirable response and 6 is the most desirable response. The average response of all firms was determined for each question and the average response for each question for each firm was calculated. The firm’s average response to the question was compared to the average response of all firms; if a firm met or exceeded the average response, it earned a point for the question. Additionally, bonus points were provided for firms that met or exceeded the 75th percentile on certain questions, which are the most telling/important of those asked in the employee survey.

Questions were arranged by categories, including culture, benefits, performance/ recognition, compensation, professional development, recruiting & retention, and general.

For each firm, the total points earned for each category were normalized so that each category had a total of 10 points. This step helped to account for the fact that there were many more graded questions in some categories than others and that some categories’ questions weren’t necessarily as important as others with fewer questions. Next a unique weighting factor was applied to each category’s normalized point total. The weighting factors are based on data collected from the 2006 rankings. At the beginning of the employee survey, we asked many questions about what was important to the employees of civil and structural engineering firms so that we could better determine how to weight types of questions in the future, rather than going by assumptions. For example, we learned that employees believe that a firm’s culture is more important than its professional development programs. More than 3,500 employees’ responses were incorporated into these findings. The sum of the weighted total points for each category was determined; this was the total raw score for the ES component of the overall score.

Finally the mean and standard deviation were used in combination with the CS scores to rank the firms. This puts the employee survey and the corporate survey on a level playing field for determining the final ranks. (Otherwise the corporate survey may seem worth more than the employee survey, or vice versa, if the point ranges aren’t similar.)