|
| |
Work Keys® Research
Cardiopulmonary & Neurodiagnostic Sciences
Prepared by: Joseph S. DiPietro, Ph.D., RRT
January 2004
Part One---Post Hoc Study of the Journeyman of the Fellow Craft (Class 2004)
This post hoc study uses Work Keys® evaluation of reading, mathematics writing,
and listening skills to assess and compare learner performances at mastery
level. Since Work Keys® will be administered after self assessment examinations,
this will provide us with a look at comparing performances on these self
assessment examinations as well as basic methods used by the learners to problem
solve.
Numerical Results
Table 1-1. Work Keys summation
Work Keys Focus Group Mean High Score and Frequency Low Score and Frequency
Reading 5.93 7 (w/5 of 15) 3 (w/3 of 15)
Mathematics 5.84 7 (w/1 of 15)
6 (w/4 of 15) 4 (w/4 of 15)
Writing 3.85 5 (w/5 of 15) 3 (w/5 of 15)
Listening 3.28 5 (w/1 of 15) 3(w/8 of 15)*
*The largest percentage of the entire group of 15 scored 3 in Work Keys
Listening category
Table 1-2. Entry-Level Self Assessment Examination summation
Group Cut Score Mean Score Pass 1st Attempt Pass 2nd Attempt Pass >2 attempts
Class 2004 75 88 4 (of 15) 6 (of 15) 5 (of 15)
Work Keys
Composite** ≥17 ≥16 ≥14
WK Listening ≥4 ≥4 ≥3
**Work Keys Composite was calculated by adding the four category scores of each
learner, the maximum score for all four categories together is 24
Table 1-3. Class 2004 Pearson Correlations*** between Work Keys Categories and
Self Assessment Pass and RTH 101, 111, & 131, and BIO 141
Class 2004 SASE RTH 101 RTH 111 RTH 131 BIO 141
0.70 0.73 0.72 0.74 0.75
*** p < 0.0005
Discussion and Conclusions
While Work Keys has been used as a predictor or a fitness type of evaluation
between learners and career areas, this post hoc study was performed on learners
about to graduate as advanced practitioners in cardiopulmonary sciences (Tables
1, 2 and 3). The two areas of strength were reading and mathematics.
Interestingly, the group was incredibly weak in two critical areas: listening
and writing, which lowered the group mean Work Keys Composite.
The following table, showing Pearson correlations between the Work Keys Writing
and Listening and Problem Solving Skill Sets is of import:
Table 1- 4. Writing, Listening, and Pearson Correlations with Problem Solving
Work Keys Category RTH 101 RTH 111
Listening 0.86 0.85
Writing 0.85 0.87
p < 0.005
Learner recognition of inability to make connections (the author’s recognition
as well), is supported, at least in part, by weak writing and listening skills.
But, with these learners in the latter portion of the program (Journeyman level,
nearing Craft Mastery), it is apparent that repetition of foundation
(apprenticeship) skills is not being accomplished, and this is vindicated in the
section of the Entry-Level Self Assessment Examination Section Patient
Evaluation and Modification of Therapeutics. The mean score for the Class of
2004 for this section (as a percentage correct) was 56%. While this parallels
previous class groups, where the mean score for the last five years in this
category is 54.4%, the learners may well have held these skill sets, a lack of
practice of these skill sets prevents the maintenance and growth of these
abilities. This area of the testing is, in part, reflective of weak listening
skills.
CONCLUSION(S)
(1) The Program design in its entirety is NOT requiring learners to practice
problem solving or thinking, at least not at the level required by the advanced
practitioner.
(2) Level I trauma and critical care is absolutely lacking in our program; this
is where evaluation and synthesis (craft mastery skill sets) take place;
required of our Advanced Practitioner matrix…
(3) While task skills are part of a whole picture of learner performance, task
skills can no longer be the focus, either by learners or clinical adjunct
faculties.
(4) While formative and summative testing is of multiple choice in each
respective design, writing requirements must be strengthened as an addition to
formative and summative testing requirements.
Part Two---Entering Freshman---Evaluative comparison with ASSET/COMPASS
Class 2005
This evaluative study for the entering freshman class (Class 2005) will give us
a numerical evaluation of reading, writing, and mathematics skills. We will
compare these performances with ASSET/COMPASS seeking correlations (if any).
Numerical Results
Table 2-1. Work Keys and ASSET Testing Comparison/Pearson Correlation; Entering
Apprentice GPA
Work Keys Composite Mean Score ASSET Composite
Mean Score Pearson Correlation Entering Class GPA Mean Score
17 138 0.88 3.65
p < 0.005
With a Pearson correlation of +0.88 between the Work Keys Composite and ASSET
Composite, Work Keys demonstrates its use as an evaluative/fitness tool prior to
entry into Cardiopulmonary Sciences. The other health professions need to follow
up to determine if Work Keys may be applicable for specific programmatic use.
Work Keys, offers a “degree of fitness” based upon category scores and their
respective relationships to career areas, where ASSET does not provide this
fitness scoring.
Part Three---Learner strengths/weaknesses relating to academic performance
problem solving---Entered Apprentices
Class 2005
Work Keys® scores was compared with freshman numerical performance in RTH 101,
111, and 131. This will be known as Apprenticeship Assessment.
Table 3-1. Correlation Data between Work Keys and RTH 101, 111 & 131.
Work Keys Composite Mean Score 101
Mean Score Correlation
For 101 111
Mean Score Correlation
For 111 131
Mean
Score Correlation
For 131
17 94.2 0.88 93.9 0.89 91.8 0.85
p < 0.005, for all correlation data
To be successful in one of the RTH Entered Apprentice Courses, these performance
data are compared with the Work Keys Composite Value.
Table 3-2. Course mean group grades versus Work Keys Composite Group Mean
Class 2005, Entered Apprentices
Course Grade
(Class 2005, Apprentices) Work Keys Composite Group Mean/Mean Grade Score
Correlation
“A” (94 or above) 18/97.4 0.82
“B” (87-93) 16.6/91.6 0.81
“C” (80-86) 15/82.5 0.815
“D” (<80) (Only 1 “D” grade recorded 14/78 Insufficient data for correlation
“F” (<70) no “F” grades recorded
p < 0.005
Within the confines of the entered apprentice RTH courses (RTH 101, 111, and
131), the Work Keys Composites paralleled grade performance, but the author
would hesitate to use Work Keys as a predictor for course performance without
another group of data from another entering class. However, this portion was
expanded to accomplish a post hoc evaluation of the Journeymen of the Class of
2004.
Table 3-3. Course mean group grades versus Work Keys Composite Group Mean
Class 2004, Journeymen
Course Grade
(Class 2004, Journeymen) Work Keys Composite Group Mean/Mean Grade Score
Correlation
“A” (94 or above) 17.4/97.6 0.83
“B” (87-93) 16.4/92.3 0.82
“C” (80-86) 14.6/84.5 0.815
“D” (<80) (Only 3 “D” grade recorded 14.3/78 0.81
“F” (<70) no “F” grades recorded
p < 0.005
The spread between Work Keys Composites was closer with this Journeyman class(a
mean spread of 1.033 points), whereas the entered apprentice mean spread was
1.333.
Part Four---Student Right and Left Brain Learning Phenomena
Class 2005
Since Work Keys® evaluates reading, writing and mathematics performance, we will
use Work Keys® scores to assist in our developing a mentoring program based upon
how each learner learns, i.e., left or right brain learner. Strong mathematics
performance indicates left brain learning, i.e., logical thinking. Reading and
writing will provide a role since logical (left brain readers) seek progression,
and right brain readers have a tendency to skip details but are reasonably
strong at envisioning the outcome(s).
The Numerical Data
Table 4-1. Left and Right Brain Learning, Work Keys Category Mean Scores and
ASSET Means for the Groups
Left or Right Brain (17 learners) Work Keys math
Max 7
Min 3 Work Keys
Listening
Max 5
Min 1 Work Keys
Reading
Max 7
Min 3 Work Keys Writing
Max 5
Min 2 ASSET
Mean Scores
Predominant
Left (5) 5 3.6 5.8 3.6 139
Predominant
Right (9) 4.66 3.13 5 2.89 130
Both L & R (3) 5.66 3.66 5.67 3.33 143
p < 0.005
Work Keys math and reading for information were strong presentations overall:
lowest score for right brain learners in mathematics, was expected, but still a
strong score.
Listening and writing were the weakest of all categories, both of which parallel
problem solving and critical thinking (evaluation and synthesis which parallels
craft mastery functions). Similar results were noted in the sophomore
(journeyman learners).
ASSET Composites showed strongest in combined left and right brain learners, as
well as left brain learners. Right brain learners faired weaker in the ASSET
Composite.
For our freshman (entered apprentices) we used this information to
(1) L & R Brain “Combi” learners and L Brain learners used as mentors for
mathematics, listening and writing.
(2) R Brain learners used as mentors for global perspectives and global
outcomes.
Course performances were necessarily mixed with regards to left and right brain
learning.
Table 4-2. Left and Right Brain Performance in Apprentice RTH and BIO 141
Courses
Left, Right or “Combi” Brain Predominance RTH 101 RTH 111 RTH 131 BIO 141
Left 93.9 94 96 89
Right 93 94 88 78
“Combi” R & L Brain 97.3 96.2 98 91
Right brain pre-dominants performed weakest in RTH 131 and BIO 141, both of
which focus on details---right brain predominance is more globally oriented,
perhaps one of the precursors to weaker performance in detail courses. However,
RTH 111, while holding a focus on cardiopulmonary anatomy and physiology,
conceptual understanding of physiology is much more critical, perhaps why all
learners perform strongly in this course. Further, RTH 101, an Integrated
Sciences course with foci on mathematics, chemistry, physics and physiologic
chemistry, one might conclude this course would fall into details---rather,
concepts and the understanding of the application of the concepts is crucial to
the “practice” of the craft, the right brain learners did quite well, but it
also implies that the left brain learners hold right brain tendencies stronger
than evaluations demonstrated. Right brain learners are not strong learners in
recall learning strategies, unless “queues” to connections are made which can
include visual, auditory, or mnemonic queues.
Left brain pre-dominants performed well at all course levels, as did left and
right brain “combi’s”.
Part Five---Correlation(s) between Work Keys® and BIO 141
Class 2005
Not necessarily for prediction, this study section reviews relationships between
learner performance on Work Keys® and performance in BIO 141. Previous research
holds that learners in cardiopulmonary sciences have difficulty ‘holding on to’
and ‘being able to use’ foundation anatomy and physiology later in the learners’
program when right and left brain connectional learning and performance is
critical in the clinical environment.
Table 4-2 shows how learners performed (based upon left and right brain learning
phenomena) in the professional core apprentice courses and BIO 141. A discussion
of this phenomena occurred in the previous section, and will not be re-addressed
here.
However, of interest are the comparisons of learners who did not achieve the cut
score in BIO 141 (a grade of “C” or better), those who did make the BIO 141 cut,
the respective Work Keys Composites and right, left or “combi” brain
predominance, and ASSET Composite.
Table 5-1. R & L Brain Predominance, Work Keys, ASSET and BIO 141 & Entering
GPA’s (Entered Apprentices, Class 2005)
Group
Reviewed R, L or Combi Brain Predominance Work Keys Mean Composite ASSET Mean
Composite BIO 141 Letter Grade
Entering Grade Point Average
Those completing BIO 141 w/ “C” or better (10 Students) L and “Combi” brain
(6 students)
18.16
143
≥ “B” grade
3.88
R Brain
(4 students)
15.68
130 All had “B” grades, save one with a “C”
3.28
Those non-completers of BIO 141 w/ < “C” (6 students) L brain
2 student
17.6
139
“D”
3.90
R brain
4 students
15.68
130
“D”
3.26
Sixty-six percent of those not achieving the cut of “C” or better in BIO 141
were right brain learners; these learners held a Work Keys Composite Score of
15.5, and an ASSET Composite of 130. Thirty-four percent of the non-completers
were left brain learners and held a Work Keys Composite of 17 and an ASSET
Composite of 139; both composites were well within the range of those who
passed.
Of the ten (10) learners passing BIO 141 passed with a grade of “B” or better,
60% were L and “Combi” brain learners with a Work Keys Composite Mean of 18.6
and an ASSET Composite Mean of 143. Forty percent of those passing (4 learners)
were right brain learners and held a Work Keys Composite Mean of 15.68, and an
ASSET Composite Mean of 130. All of these right brain learners had a “B” grade
save one with a “C”.
Preliminary Closure and Conclusions
Work Keys has been widely used as a predictive/performance/career focus tool for
learners. This author has not seen Work Keys applied for learners in the Health
Technologies curricula, and perhaps, this is somewhat of a journey in ‘new
territories’.
The research presented is in no way meant to offer a raison d’etre for learner
performance in cardiopulmonary sciences professional core, or BIO 141. It is,
however, a cautionary tale, if you will, because the data does indicate the
following and the subsequent caveats:
(1) Left brain and “combi” brain learners performed well overall, and in courses
where recall is a focus. Logical progressions and detail are their strengths.
Caveat #1: the left brain learner has more difficulty with “concepts” unless
queues of connection are readily available. Caveat #2: “combi” brain learners
handle “concepts” seemingly better than their left brain contemporaries,
probably because these learners have ‘stronger’ right brain involvement. Caveat
#3: linear thinking of these individuals must be enhanced by right brain
(non-linear model) practicing to promote non-linear thinking.
(2) For right brain learners, logical progressions are not strengths, at least
typically. Emotional and non-linear in thought processes, these may provide us
with a possible cause for right brain learner non-achievement in courses where
recall and linear thinking is the status quo. Caveat #4: Linear thinking
development, course preparation, and emotional control must be developed in
these learners.
(3) While not discussed within this research, focus, commitment, and desire are
crucial for any learner’s success, or failure, if one of these salient
characteristics is missing. Caveat #5: A learner lacking either focus,
commitment, or desire, will not be successful, regardless of left, right, or
“combi” brain predominance, entering GPA, ASSET Composites, or Work Keys
Composites.
(4) Caveat #6: Studying is not necessarily preparation.
(5) Work Keys provides an avenue of faculty understanding of exactly where
learners stands with respect to “fitness” in what the learner holds as strengths
and weaknesses. Even though Work Keys offers a projection of performance, it
draws these conclusions based upon present performance---ASSET or COMPASS does
not presently offer this “fitness”.
(6) All Pearson correlations presented in this study, comparing Work Keys with
performance in professional core courses and BIO 141 exceeded +0.70, with p
values less than 0.005, and in some cases less than 0.0005. Caveat #7: Work Keys
alone is not a panacea for screening learners for entry, and should not be used
as a stand alone method of screening for entry into the health technologies
professional cores. It (Work Keys) has provided a wealth of learner information
previously not available---and has also provided a new venue to establish new
ways of approaching learners to show them how to (1) prepare in improved ways;
(2) to work more effectively in teams and with their mentors; (3) use their
gifts (strengths) more effectively to allay fears of non-performance, and remedy
weaknesses; and (4) how to think differently, by using new tools in new ways.
Copyright © 2004
|
