- Recall reliability and validity
- Reliability – Consistency or stability of measure
- Scientific measures are rarely perfect measures of concepts
True Score – Score without error in measure
Random Error – Other factors that might have influenced measure that do not relate to what we’re trying to measure
∴ Actual score = true score + random error
e.g., Reliable IQ should be around 100, unreliable measure fluctuates and varies and reflects error
- Error scores do show variability
- Less error → better reflection of the score
Compute Reliability Coefficient
Test-Retest Reliability – the extent to which scores at two different points in time covariate
- Reliability Coefficient (ideal .80)
↑ – ↑ reliability
↓ – ↓ reliability - Practice effect may be confound in abilities test; solution: longer interval or Alternate Forms Reliability
Alternate Forms Reliability – Two different but equivalent versions of the same measure correlate with each other
- Another problem: some conceptual variables are not expected to be consistent over time
- Participant variable traits are not expected to change over time
- Other variables are expected to change over time – states
Internal Consistency – The assessment of reliability using responses at one point in time
- Most tests have multiple items, but could just correlate individual items
- Extent to which scores on items of scale correlate with each other
- Split-Half Reliability – All items are divided in half, total scores on one half correlated with total scores on other half
e.g., Correlate Part A with Part B scores, with odd/even number division - Cronbach’s Alpha – Correlation of each item with every other item correlated, average of correlation is computed
- Item-Total Correlations – Each item on test correlated with total item on test (without item score on it), average of correlations computed
- If single item doesn’t correlate with every other one, this helps catch scoring error or bad item
- Inter-Rater Reliability – Behavioural measures common practice to have multiple sources of rating and coding of behaviour
- Correlate RA A with all other RAs
- Longer tests are more reliable
e.g., Exam with 2 questions: if you don’t know 1 question, you fail- More measured variables combined together more reliable test will be
- Many measures + average of measurements reduces random error
0 Responses to “Measurement Concepts”