Data Analysis

Data Analysis

For each instrument, five CFA models were fitted (null, one general trait only, uncorrelated methods only, correlated traits only, and traits-and-methods). Convergent validity, discriminant validity, and the presence and effects of method variance were evaluated following the procedures outlined by Widaman (1985) and Bagozzi and Yi (1990). All confirmatory factor analyses were performed using the EQS/Windows statistical software package (Bentler, 1995). For a full explanation of the goodness-of-fit indices used in this study, see Bollen (1989) and Bentler (pp. 92–94). Data analyses were performed using the Satorra and Bentler procedure to correct for non-normal data (Bentler, p. 47).

The data analyses refer to various models derived from Widaman's taxonomy. These models are depicted in Figures 1 to 5. Due to space limitations, we show only the CSE in the figures (see Appendix).

• 
  

  
  Figure 1: Null model, CSE (Model 1);

  
  


• 
  

  
  Figure 2: One general trait, CSE (Model 2);

  
  


• 
  

  
  Figure 3: Methods only, CSE (Model 3);

  
  


• 
  

  
  Figure 4: Traits only, CSE (Model 4);

  
  


• 
  

  
  Figure 5: Multitrait-multimethod, CSE (Model 5).

  
  

Convergent Validity

Convergent validity was assessed in two ways. First, model 1 (null) was compared with model 4 (correlated traits) to see if the addition of traits significantly improved the fit of the model (Bagozzi & Yi, 1990, p. 554). The χ² difference tests between model 1 and model 4 for both instruments are statistically significant (see Table 3): χ² difference _EUCSI = 6802.7, 33 df, p < .001 and χ² difference _CSE = 6512.9, 46 df, p < .001).

Table 3: Goodness of Fit Indices for the CFA Models.

End User Computing Satisfaction Instrument
�	X2	(df)	X2/df	NFI	NNFI	CFI	GFI	AGFI	RMSR
Null Model	10160.2	276	36.8	�	�	�	�	�	�
Model 2	4192.9	253	16.6	.305	.249	.243	.218	.073	.249
Model 3	1904.6	253	07.5	.721	.715	.683	.467	.368	.099
Model 4	3357.5	243	13.8	.484	.425	.402	.275	.105	.234
Model 5	510.9	219	02.3	.929	.936	.944	.868	.819	.062

Computer Self-Efficacy Instrument
�	X2	(df)	X2/df	NFI	NNFI	CFI	GFI	AGFI	RMSR
Null Model	14233.8	780	18.3	�	�	�	�	�	�
Model 2	8554.9	740	11.6	.399	.388	.419	.250	.169	.170
Model 3	3591.1	736	04.9	.748	.775	.788	.650	.610	.058
Model 4	7720.9	734	10.5	.458	.448	.481	.259	.173	.170
Model 5	2066.4	694	02.9	.855	.885	.898	.780	.741	.037
Note: Model 2—One general trait only
Model 3—Uncorrelated methods only
Model 4—Correlated traits only
Model 5—Traits and methods

Second, because convergent validity results when a trait loading on a measure is statistically significant, the trait-item loadings of model 4 were examined (Bagozzi & Yi, 1990). For each item, a statistically significant trait-item loading indicates that the trait explains a statistically significant amount of variance in the item (Bagozzi & Yi; Widaman, 1985).

For the EUCSI, 15 of the trait-item loadings in model 4 are statistically significant at p < .001; C1 (self) and E1 (self) are statistically significant at p < .05; C2 (self), C3 (self), C4 (self), A2 (self), and T2 (self) are statistically significant at p < .01; and E2 (self) and T1 (self) are not statistically significant (p > .05). For the CSE, all trait-item loadings in model 4 are statistically significant (p < .001), with the exception of B8 (self), which is statistically significant (p < .05). Space limitations do not permit the trait-item loadings to be shown.

Discriminant Validity

Discriminant validity was assessed for both instruments in two ways. First, model 2 (one general trait) was compared with model 4 (correlated traits) to determine if the addition of distinct traits significantly improved the fit of the model (Schmitt & Stults, 1986; Widaman, 1985). Model 2 constrains all intertrait correlations to 1.00 while model 4 allows the intertrait correlations to be estimated. A perfect correlation between two latent variables means that it is impossible to empirically discriminate or distinguish between the variables. A significantly better fit of model 4 over model 2 suggests that the intertrait correlations are not equal to 1.00. The χ² difference tests between model 2 and model 4 for both instruments are statistically significant (see Table 3): χ² difference = 835.4, 10 df, p < .001 and χ² difference _CSE = 834, 6 df, p < .001.

Second, the correlations among traits and their standard errors were examined. Discriminant validity among traits is attained when an intertrait correlation is less than 1.00 by an amount greater than twice the standard error (Bagozzi & Yi, 1990). Six of the 10 intertrait correlations in the EUCSI demonstrate discriminant validity. The exceptions are content-format, content-timeliness, accuracy-format, and format-timeliness (see Table 4). Five of the six intertrait correlations in the CSE exhibit discriminant validity, with the exception of file-advanced (see Table 4).

Table 4: Correlations Among Traits and Standard Errors.

End User Computing Satisfaction Instrument (Model 3--Correlated traits)
�	Accuracy	Content	Format	Ease of Use
Accuracy	�	�	�	�
Content	.920 (.019)	�	�	�
Format	.991 (.015)	.993 (.017)	�	�
Ease of Use	.826 (.034)	.770 (.032)	.840 (.029)	�
Timeliness	.938 (.022)	.996 (.015)	.995 (.023)	.780 (.035)

Computer Self-Efficacy Instrument (Model 3--Correlated traits)
�	Basic Skill	File Skill	Advanced Skill
Basic Skill	�	�	�
File Skill	.904 (.015)	�	�
Advanced Skill	.863 (.018)	.990 (.009)	�
Mainframe Skill	.541 (.039)	.701 (.030)	.685 (.031)

Method Variance

The presence and effects of method variance were examined in five ways. First, the overall fit of model 5 (traits and methods) was examined. For the EUCSI, the goodness-of-fit indices (see Table 3) show that model 5 demonstrates an excellent fit to the data, with the exceptions that the χ² value is statistically significant and the GFI is below 0.9. Thirteen of the 24 trait-item loadings are statistically significant at p < .001. C1, C2, C3, C4, F2, T1, and T2 (peer) and A1, A2, E1, and E2 (self) are non-significant, p > .05. All method-item loadings are statistically significant (p < .001).

For the CSE, the goodness-of-fit indices (see Table 3) show that model 5 demonstrates an adequate fit to the data, with the exceptions that the χ² value is statistically significant, the GFI is less than 0.9, and the AGFI is less than 0.8. All trait-item and method-item loadings are statistically significant (p < .001).

Further, the square of the trait-item and method-item loadings is the proportion of the item variance (R²) accounted for by the trait and by the method. The R² value can be used to estimate the reliability of each item, with R² values above 0.50 suggesting acceptable reliability (Bollen, 1989). However, Bagozzi and Yi (1990) used a more relaxed criterion for reliability, stating that if a loading is greater than twice the value of its standard error, then it is statistically significant, and the variance explained by the trait for that item is statistically significant.

For the EUCSI, the statistically significant trait-item and method-item loadings demonstrate reliability using Bagozzi and Yi's (1990) criterion. No trait-item loadings, but all method-item loadings, demonstrate reliability using Bollen's (1989) criterion.

For the CSE, all trait-item and method-item loadings demonstrate reliability using Bagozzi and Yi's (1990) criterion. Using Bollen's (1989) criterion, two trait-item loadings and 18 method-item loadings demonstrate reliability.

Second, model 3 (uncorrelated methods) was compared with model 1 (null) to determine if the addition of the two method factors significantly improved the fit of the model. The χ² difference tests between model 3 and model 1 for both instruments are statistically significant (see Table 3): χ² difference _EUCSI = 8255.6, 23 df, p < .001 and χ² difference _CSE = 10642.7, 44 df, p < .001.

Third, model 4 (correlated traits) was compared to model 5 (traits and methods) to determine if the addition of the method factors significantly improved the fit of the model (Widaman, 1985). The χ² difference tests between model 4 and model 5 for both instruments are statistically significant (see Table 3): χ² difference _EUCSI = 2846.6, 24 df, p < .001 and χ² difference _CSE = 5654.5, 40 df, p < .001.

Fourth, the specific effects of the method factor were assessed by examining the statistical significance of the individual method-item loadings in model 3 (Bagozzi & Yi, 1990). All individual method-item loadings in model 3 for both instruments are statistically significant at p < .001.

Fifth, CFA enables the variance of the model to be partitioned between the variance attributable to trait, to method, and to error. A comparison of the relative amounts of variance explained suggests the relative importance of method variance in the EUCSI and CSE data sets. The five traits of the EUCSI account for 6.8% of the variance and the two methods explain 68.3% of the variance, with the remaining 24.9% of the variance attributable to error. The four traits of the CSE account for 22.7% of the variance and the two methods explain 41.1% of the variance, with the remaining 36.2% of the variance attributable to error.