Employers that use hiring tests must ensure that the tests are job-related and consistent with business necessity, as required under the Equal Employment Opportunity Commission's Uniform Guidelines on Employee Selection Procedures (UGESP). The regulations apply to all employers that use employment tests, and encompass virtually any assessment device or methodology used in making employment-related decisions.
The UGESP state that:
- “Employers should administer tests and other selection procedures without regard to race, color, national origin, sex, religion, age (40 or older), or disability.”
- “Employers should ensure that employment tests and other selection procedures are properly validated for the positions and purposes for which they are used. The test or selection procedure must be job-related and its results appropriate for the employer’s purpose. While a test vendor’s documentation supporting the validity of a test may be helpful, the employer is still responsible for ensuring that its tests are valid under UGESP.”
- “To ensure that a test or selection procedure remains predictive of success in a job, employers should keep abreast of changes in job requirements and should update the test specifications or selection procedures accordingly.”
- “Employers should ensure that tests and selection procedures are not adopted casually by managers who know little about these processes. A test or selection procedure can be an effective management tool, but no test or selection procedure should be implemented without an understanding of its effectiveness and limitations for the organization, its appropriateness for a specific job, and whether it can be appropriately administered and scored.”
As with the use of all valid and reliable psychometric measures, the employer has a number of basic responsibilities. We recommend you:
- Ensure that Mental Toughness is actually needed for the job/role
- Provide other evidence of Mental Toughness through the use of structured interviews, references, group exercises, other questionnaires, or other sources. NEVER use MTQ48 scores on their own.
- Set up an equal opportunities monitoring system. Information on gender, ethnic origin, and age of candidates should be collected and stored. At regular intervals the data should be returned to the publisher, AQR LTD., or analyzed locally to ensure that unfair gender or ethnic biasing is not occurring. Advice on how to do this can be obtained from AQR.
- The MTQ48 has been shown to predict performance in stressful situations. However, it is the end user’s responsibility to carry out local validity studies to show that performance in your particular organization is related to MTQ48 scores. Advice on how to do this can be obtained by AQR or you can conduct local validation studies with a qualified IO Psychologist or testing expert.
In addition to UGESP regulations, when evaluating personality assessment tools, we suggest starting with these questions:
- What is the assessment designed to measure, and how will that benefit the organization?
- Is the assessment reliable and accurate?
- Does it effectively predict important workplace behaviors that drive metrics affecting sales, customer satisfaction and turnover?
- How easy is the assessment to take?
- Does it come in multiple languages?
- How easy is it to interpret results?
For answers to these questions and evidence to satisfy UGESP regulations, please contact us for a copy of our EEOC's UGESP document.
Reliability is the most important test for a psychometric measure. If a test is not reliable, it won’t work. A reliable measure is one that you can complete today and again in the future and get the same or very similar results.
Assuming nothing significant has happened to you in the meantime and there is no reason that you should have changed, you should get, within reason, the same score on the second occasion as you did on the first. If you do get a different response, then the test is either faulty in some way or it is picking up something that is changing but for which it was not designed. That is, it’s unreliable. If it is reliable, users can rely on the information generated to help them understand better the individual with whom they are working.
The British Psychological Society and the US Department of Labor both provide guidance as to what is an acceptable score for a measure to be regarded as a good measure. That score is 0.70 or greater. Obviously the higher the score, the more reliable is the measure.
Initial scale reliability of the Mental Toughness Questionnaire 48 (MTQ48)
The MTQ48 has an overall test-retest coefficient of .90, with the internal consistency of the subscales presented in the table below:
All subscales reached the minimum acceptable level (0.70) recommended by Kline (Kline, 1999) when investigating the reliability of psychological constructs. This supports the homogeneity of each subscale and the MTQ48 as a whole.
Initial Construct Validation of the MTQ48
To investigate the convergent validity of the MTQ48, Pearson’s correlations were calculated for the Total scores for MTQ48 and the eight sub-scales of the PREVUE personality scale. A sample of 205 within the development sample completed both the MTQ48 and the PREVUE personality scale.
* - significant at the .05 level
** - significant at the .01 level
As expected there were a number of significant and predicted relationships. The aspects where these relationships were most strongly highlighted were first, in the PREVUE scale dimension of “Excitable-Relaxed” where a correlation of .478 was found. This coupled with the significant correlation (.377) on the “Restless-Poised” scale highlights an important aspect of mental toughness, that being the ability to control anxiety and nervous tension.
The other significant correlations center on the concepts of confidence and the ability to work with others (“Submissive - Assertive and Self-sufficient - Group Oriented).
The three remaining non-relating sub-scales were “Co-operative - Competitive”, “Reactive- Organized” and “Innovative - Conventional”. It can be logically concluded that an approximately equivalent measure of each of these dimensions is required in order to function effective as an individual, therefore it was no surprise to see these sub-scales not significantly correlate with mental toughness.