What is Factor Analysis?

Factor analysis is an independence technique whose primary purpose is to define the underlying structure among the variables in the analysis or reduced the bulkiness of variables with highly intercorrelated factors. Obviously, variables play a key role in any multivariate analysis. Whether in terms of forecast in regression, prediction or in discriminant analysis for classification of materials, or using any other technique, the set of variables upon which relationship are form hence variables are the building block of relationship. To employ multivariate analysis variables are used by their nature, and number. In univariate analysis only single variable are used , however in multivariate variables are in tens, hundreds or even thousand. Therefore description of such phenomenon become difficult with such voluminous variables, then the questions is how such a situation can be described. This large data set need to be pruned to few ones. It is clear that if only few variables were used they may be distinct and different hence description is easy, but as the variables are added overlap is likely to be observed among the variables.

Factor analysis was introduce in multivariate analysis to play a unique role of by providing the tools for analyzing the structure of interrelationship among large number of variables by defining a set of another variables that are highly intercorrelated which are known as factors, or Composite variable. These groups of variables which are by definition highly intercorrelated are assumed to represent dimensions within the data. If the interest of the analysis is only on reducing the number of variables then dimensions can guide in creating new composite measures. However, if there is a conceptual basis for understanding the relationship between variables , then dimensions may actually have meaning for what they collectively represent. But these dimension if no conceptually understanding basis for relationship in the data dimensions cannot adequately described. Factor analysis can achieve two purpose either confirmatory or exploratory perspective. In exploratory its measure strength lies on searching for structure among a sets of variables or as for data reduction. In exploratory, the researcher take only what the data give as output from the analysis, as such have not set any priori constraint on the estimation of the component (factors)or number of component to be extracted. On the other hand the researcher might have a preconceived thought or idea on the actual structure of the data, based on theoretical support or prior research for the existence of this structure (groups) hence in this situation the analysis is basically trying to use confirmatory approach , thereby using the analysis to assess the degree to which the data meet the expected structure. If factor analysis is for exploratory then it is termed as exploratory factor analysis but if prior structure were known to exist then factor analysis is now termed as confirmatory factor analysis or principal component analysis. Therefore factor analysis is divided into either exploratory or confirmatory approach.

Stages in Factor Analysis Stage 1 Defining a research Question from the

In factor analysis we represent the variables y1, y2, . . . , yp as linear combinations of a few random variables f1, f2, . . . , fm (m < p) called factors. The factors are underlying constructs or latent variables that generate the ys. Like the original variables, the factors vary from individual to individual; but unlike the variables, the factors cannot be measured or observed. The existence of these hypothetical variables is therefore open to question. If the original variables y1, y2, . . . , yp are at least moderately correlated, the basic dimensionality of the system is less than p. The goal of factor analysis is to reduce the redundancy among the variables by using a smaller number of factors. Suppose the pattern of the high and low correlations in the correlation matrix is such that the variables in a particular subset have high correlations among themselves but low correlations with all the other variables. Then there may be a single underlying factor that gave rise to the variables in the subset. If the other variables can be similarly grouped into subsets with a like pattern of correlations, then a few factors can represent these groups of variables. In this case the pattern in the correlation matrix corresponds directly to the factors. For example, suppose the correlation matrix has the form. Then variables 1 and 2 correspond to a factor, and variables 3, 4, and 5 correspond to another factor. In some cases where the correlation matrix does not have such a simple pattern, factor analysis will still partition the variables into clusters. Factor analysis is related to principal component analysis in that both seek a simpler structure in a set of variables but they differ in many respects (see Section 13.8). For example, two differences in basic approach are as follows: 1. Principal components are defined as linear combinations of the original variables. In factor analysis, the original variables are expressed as linear combinations of the factors. 2. In principal component analysis, we explain a large part of the total variance of the variables,_ i sii . In factor analysis, we seek to account for the covariance or correlations among the variables. In practice, there are some data sets for which the factor analysis model does not provide a satisfactory fit. Thus, factor analysis remains somewhat subjective in many applications, and it is considered controversial by some statisticians. Sometimes a few easily interpretable factors emerge, but for other data sets, neither the number of factors nor the interpretation is clear