\documentclass{article} %\input{/usr/local/texlive/texmf-local/textPreamble} \usepackage[longnamesfirst,round,semicolon]{natbib} \usepackage[colorlinks=true,citecolor=blue, linkcolor=blue, urlcolor=blue]{hyperref} \usepackage{indentfirst} \usepackage{pdfpages} \usepackage{soul} \usepackage{floatrow} \usepackage{pdflscape} \usepackage{longtable,ltcaption} \usepackage{geometry} \usepackage{afterpage} \usepackage{setspace} \usepackage{etoolbox} \AtBeginEnvironment{longtable}{\singlespace} \usepackage{array} \usepackage{colortbl, xcolor} \usepackage{hanging} \usepackage{morefloats} \usepackage[justification=centering]{caption} \usepackage[nottoc]{tocbibind} \newcolumntype{R}[1]{>{\raggedleft\let\newline\\\arraybackslash\hspace{0pt}}p{#1}} \usepackage{rotating} \usepackage[normalem]{ulem} \usepackage[hang,flushmargin]{footmisc} \setlength{\topmargin}{-15mm} \setlength{\oddsidemargin}{-2mm} \setlength{\textwidth}{165mm} \setlength{\textheight}{220mm} \setlength\tabcolsep{3pt} \newenvironment{blockquote}{ \par \medskip \leftskip=4em\rightskip=2em \noindent\ignorespaces}{ \par\medskip} \newcommand\blfootnote[1]{ \begingroup \renewcommand\thefootnote{}\footnote{#1} \addtocounter{footnote}{-1} \endgroup} \newcommand{\pkg}[1]{{\normalfont\fontseries{b}\selectfont #1}} \let\proglang=\textit \let\code=\texttt \renewcommand{\title}[1]{\begin{center}{\bf \Large #1}\end{center}} \newcommand{\affiliations}{\footnotesize} \newcommand{\keywords}{\paragraph{Keywords:}} \makeatletter \def\@xfootnote[#1]{% \protected@xdef\@thefnmark{#1}% \@footnotemark\@footnotetext} \makeatother \begin{document} \pagenumbering{gobble} \vspace*{\fill} \begin{center} \begin{figure} \ffigbox{\includegraphics[width=.75\textwidth]{figures/SPI27Allfacets135unlabeled.pdf}}{\caption*{}\label{fig:spi27unlabeled}} \end{figure} \vskip -1.0cm \Large\textbf{The SAPA Personality Inventory:\\[0pt]An empirically-derived, hierarchically-organized\\[0pt]self-report personality assessment model}\par \vspace{10mm} \large David M. Condon\par \vspace{10mm} \small First posted: November 5, 2016\par \small Draft dated: \today{}\par \vspace{10mm} \small\textbf{(Life's Rich Pageant)}\par \end{center} \vspace*{\fill} \clearpage \pagenumbering{roman} \section*{Preface} \addcontentsline{toc}{section}{Preface} \centerline{``Do the best you can until you know better. Then when you know better, do better.''} \begin{flushright} \emph{--- Maya Angelou} \end{flushright} This project has its roots in a realization that occurred to me at the beginning of graduate school: self-report personality assessment kinda sucks. Not profound, I know, but hard to deny. The most widely-used measure --- the MBTI\textsuperscript{\textregistered} \citep{MBTI:1998} --- is not at all supported by the scientific community. The MMPI\textsuperscript{\textregistered} \citep{Butcher:2001tq}, which is widely used in clinical settings, has been around for several decades but does not correspond much with current scientific theories of personality structure in non-clinical populations. Even the NEO-PI-R\texttrademark \citep{McCrae:2010}, which has been demonstrated to have broad ecological validity across hundreds of scientific studies, still has some serious issues. First, it's proprietary and the copyright status of the test limits its utility for big data applications --- I can't afford to administer it online to tens of thousands of people every year. Even if I could, I wouldn't because I can't re-use the items or revise the structure of the test. For that matter, I can't even find many details about the methods used to create it. Where did the items come from? Did the authors write them or were they lifted from other measures? Why were they written that way? What items, if any, didn't make the cut and why? Were the items written before the model was identified or was the model empirically-derived? None of this is clear\footnote{I haven't been able to find publicly-available documentation.} and I don't view that as sufficiently scientific. Since the beginning of grad school, I have changed my mind a little as I have learned more about the available options. Several good alternatives exist in the public-domain and a few of these have become widely-used.\footnote{Virtually all of these are related in one way or another to the work of Lew Goldberg and his many collaborators, whose contributions to personality assessment have been inspirational to me over the last several years.} Scientists from many other fields have begun to include these personality assessments in their own research because they increasingly acknowledge the influence of personality on a diverse range of outcomes. These developments suggest to me that there is a growing belief outside of our field that personality can be reasonably well-measured by existing public-domain tools. Yet, I remain concerned because I don't see how we can improve upon current measurement models without explicit documentation of the methods used to create them. Over the last twenty-five years, several new set of scales have been introduced to the field even though the underlying theory of assessment hasn't really changed (the theory being that there are five or maybe six parsimonious dimensions of personality that are worth measuring). Most of these have been foisted into the literature whole cloth, with little discussion of the effect of the many methodological decisions made along the way. This doesn't feel like iterative science to me because each new set of measures uses different \textit{and often freshly-written} sets of items, without explanation or documentation.\footnote{Again, Goldberg's work is a notable exception. His exemplary papers on Big Five factor structure \citep{Goldberg:92, Goldberg:90} included several tables with item-level data and instructions for contacting the author (by mail!) for more information.} \citet{Goldberg:99a} implored us to do better 17 years ago when he released the International Personality Item Pool, but few have tried \citep{Ashton:04, DeYoung:07, Soto:2016jr}. So, I have tried to do better --- if only a little --- on two fronts. First, by ``empirically'' (rather than rationally) identifying hierarchically organized personality scales using existing public-domain personality items. This involved administration of about 700 of the most commonly-used IPIP items to 3 big samples (total N $=$ 126,884) using synthetic aperture sampling procedures. This was not quite as exhaustive as we need, as a field, but it is a substantial improvement upon prior attempts. I am optimistic that these same methods (of sampling and analyses) will someday enable the collection of data on many thousands of items in several languages with millions of participants and I expect that the results of such a project will obviate those reported here. I would embrace this as a milestone of iterative science. Second, I have attempted to encourage iteration by being fully transparent. I documented all steps in the development of these scales and described the rationale/criteria for each decision. Given the time required (not to mention the length of this document), it's possible that I have gone too far. We'll see. The power of this transparency will be realized if other personality researchers contribute to the improvement of this measurement model by making fully transparent revisions, by sharing empirical evidence that relates personality \textit{items} to one another and the outcomes they predict. I know this sounds daunting but the challenges will decrease over time as the protocols for sharing data and methods continue to improve. And the rewards will warrant the effort --- we won't have to start from scratch anymore. As a field, we will iterate more rapidly towards an assessment model that doesn't suck at all.\footnote{Maybe even one that isn't boring \citep{Baumeister:2016}.} \subsection*{Technical comments on reproducibility and formatting} This document is structured more like a monograph than a standard manuscript or a supplementary materials file. It does not adhere to the traditional APA format for research articles and, while it may be referenced heavily by one or more related manuscripts, it is intended to stand independently on its own. It is also a reproducible research document; it includes all of the statistical code I used during the project, from start to finish. While I recognize that this may be distracting to readers who are not trained in psychometrics or who are unfamiliar with the software I used to run this code (\texttt{R}), I have decided that this is an acceptable trade-off if it increases reproducibility and encourages iteration. If you would like to reproduce this work, begin by downloading the accompanying data files at \url{http://dx.doi.org/10.7910/DVN/SD7SVE} (the Exploratory sample), \url{http://dx.doi.org/10.7910/DVN/GU70EV} (the Replication sample), \url{http://dx.doi.org/10.7910/DVN/TZJGAT} (the Confirmatory sample), and \url{http://dx.doi.org/10.7910/DVN/Y72Z1J} (the Supplementary Materials). More information about the data files is given as needed in each of the relevant sections of this document. Once the data have been downloaded, it should be possible to reproduce the published results by using the \texttt{R} code in the ``.rnw'' version of this document (this includes several analyses which have been suppressed in the compiled PDF version for the sake of readability). The code can either be executed manually in \texttt{R} or by compiling the file with \href{http://yihui.name/knitr/demo/manual/}{Knitr} \citep{Xie:2015vc} or \href{http://www.stat.uni-muenchen.de/~leisch/Sweave/Sweave-manual.pdf}{Sweave} \citep{Sweave}. The current PDF version was compiled using Knitr in \href{https://www.rstudio.com/}{RStudio} \citep{Gandrud:2015ux} on \today{}. It mainly uses \Sexpr{R.version$version.string} \citep{R} and \Sexpr{packageDescription("psych")$Package} version \Sexpr{packageDescription("psych")$Version} \citep{Rpsych}. Several other \texttt{R} packages were used as needed for specific tasks; the names and version numbers are listed in Table \ref{tab:Rpacks}. \begin{table}[htbp]\footnotesize \caption{\texttt{R} packages used and version numbers} \begin{center} \begin{tabular} {l r} \hline Package & Version \cr \hline \Sexpr{packageDescription("fields")$Package} \citep{Douglas-Nychka:2015aa} & \Sexpr{packageDescription("fields")$Version} \cr \Sexpr{packageDescription("ggplot2")$Package} \citep{Wickham:2009aa} & \Sexpr{packageDescription("ggplot2")$Version} \cr \Sexpr{packageDescription("grid")$Package} \citep{R-Core-Team:2016aa} & \Sexpr{packageDescription("grid")$Version} \cr \Sexpr{packageDescription("koRpus")$Package} \citep{R:koRpus} & \Sexpr{packageDescription("koRpus")$Version} \cr \Sexpr{packageDescription("lavaan")$Package} \citep{Rosseel:2012aa} & \Sexpr{packageDescription("lavaan")$Version} \cr \Sexpr{packageDescription("maps")$Package} \citep{Richard-A.-Becker:2016aa} & \Sexpr{packageDescription("maps")$Version} \cr \Sexpr{packageDescription("mgcv")$Package} \citep{Wood:2011} & \Sexpr{packageDescription("mgcv")$Version} \cr \Sexpr{packageDescription("multicon")$Package} \citep{Sherman:2015aa} & \Sexpr{packageDescription("multicon")$Version} \cr \Sexpr{packageDescription("parallel")$Package} \citep{R-Core-Team:2016aa} & \Sexpr{packageDescription("parallel")$Version} \cr \Sexpr{packageDescription("plyr")$Package} \citep{Wickham:2011aa} & \Sexpr{packageDescription("plyr")$Version} \cr \Sexpr{packageDescription("psych")$Package} \citep{Rpsych} & \Sexpr{packageDescription("psych")$Version} \cr \Sexpr{packageDescription("qgraph")$Package} \citep{Epskamp:2012aa} & \Sexpr{packageDescription("qgraph")$Version} \cr \Sexpr{packageDescription("RColorBrewer")$Package} \citep{Neuwirth:2014aa} & \Sexpr{packageDescription("RColorBrewer")$Version} \cr \Sexpr{packageDescription("rworldmap")$Package} \citep{South:2011aa} & \Sexpr{packageDescription("rworldmap")$Version} \cr \Sexpr{packageDescription("semPlot")$Package} \citep{Epskamp:2014aa} & \Sexpr{packageDescription("semPlot")$Version} \cr \Sexpr{packageDescription("semTools")$Package} \citep{Contributors:2016aa} & \Sexpr{packageDescription("semTools")$Version} \cr \Sexpr{packageDescription("spam")$Package} \citep{Gerber:2015} & \Sexpr{packageDescription("spam")$Version} \cr \Sexpr{packageDescription("tidyr")$Package} \citep{Wickham:2016aa} & \Sexpr{packageDescription("tidyr")$Version} \cr \Sexpr{packageDescription("xtable")$Package} \citep{Dahl:2016aa} & \Sexpr{packageDescription("xtable")$Version} \cr \Sexpr{packageDescription("zipcode")$Package} \citep{Breen:2012aa} & \Sexpr{packageDescription("zipcode")$Version} \cr \hline \end{tabular} \end{center} \label{tab:Rpacks} \end{table} Please note that it is computationally intensive to reproduce the PDF completely using current technology (in 2016); it may require several hours to run, depending on your environment. An easier alternative might be to simply pull the code out of the rnw file and run it manually in any \texttt{R} interface. If you are patient, you can reproduce the file completely by following these steps: \begin{enumerate} \vspace{-2mm} \item Create a folder on your hard drive where you want to save everything. This should be done on a machine with plenty of working memory. \item Change the following lines of code in the rnw version of this file to match the location of the folder just created on your own computer. The lines which need to be changed are around line 373 of the rnw file and currently read as follows: \-\hspace{6mm}filepathdata $<-$ ``$\sim$/Desktop/data/" \vspace{-2mm} \-\hspace{6mm}filepathfig $<-$ ``$\sim$/Desktop/figures/" I have created separate folders for the data input/output files and for the figures. I did this because this script will generate a lot of files; I prefer to keep the data files separate from the figures. But, this is optional --- you could set the `filepathdata' and `filepathfig' lines to point to the same folder if preferred or use a more complicated structure. Also note that the `/' will need to go in the opposite direction (`\textbackslash ') for those using a PC-based operating system. \item Put the data files (downloaded from Dataverse) into the folders just created. Anything that ends with the extension `.RData' should be placed in the filepathdata folder, everything else goes into the filepathfig folder. You will notice that I have added some additional structure in a few places. If you want to match my structure (recommended), create 2 empty folders inside the folder at the location of filepathdata and name them `factorTable' and `items'; create 2 empty folders in the filepathfig location called `IRT' and `norms'; and create a 3rd folder in filepathfig called `appendices' --- in it, put all of the .png files (these png files have names like `factorTable5\_X.png' and `factorTable27\_X.png') and the PDF versions of the assessment forms (these PDF files have names like `SPI\_135\_27\_\&\_5\_v1\_1') from the Supplementary Materials dataset on Dataverse. \item Open the rnw version of this file in RStudio. Make sure you have updated versions of the \texttt{R} packages listed in Table \ref{tab:Rpacks}. \item Confirm or change your settings in RStudio to ensure that the knitr functionality is being used. Do this by clicking RStudio/Preferences/Sweave, then select `knitr' from the dropdown menu for `Weave Rnw files using:' and `pdfLaTeX' for `Typeset LaTeX into PDF using:'. \end{enumerate} One should then be able to recreate the PDF by simply clicking `Compile PDF' and waiting a few hours. Unfortunately, there are two caveats. First, several figures are generated using external data files that need to be downloaded directly from other sources. While it is all freely available (mainly from the U.S. Census via \href{http://www.census.gov/data/developers/about.html}{API}), researchers are required to register and format the data for themselves. I am not allowed to share the data directly and this means that some of the code chunks cannot be directly reproduced. The Rnw file will still compile because I have provided a reasonably seamless work-around: the Supplementary Materials file includes pre-processed versions of these figures. You can recreate them on your own of course --- I have included the code to do this --- but it will require registration and some foraging around on the U.S. Census website. I have turned off the evaluation of the code for the relevant sections and the necessary images will be pulled in as if they were generated with the code I have given. Note that this external data only serves to contextualize the characteristics of the samples and is not central to the aims of this project. Second caveat: I have also included two other pre-processed data files in the Supplementary Materials. One of these is the output of analyses I ran on the computing cluster to get bootstrapped confidence interval estimates. Again, the code needed to reproduce these analyses is included for those who choose to use it but it will take a very long time to run on a stand-alone machine using modern software at the time of this writing (4 to 6 days). The second pre-processed data file contains item-level qualitative data. More information about these results (and the underlying data) can be found in a separate manuscript. See Condon, Graham, Weston \& Mroczek (in prep). %%%%%%%%%%%%%%%%%% %Add Breaking Line \begin{center} \linethickness{0.5mm} \line(1,0){100} \end{center} %%%%%%%%%%%%%%%%%% I also feel compelled to acknowledge the regrettable length of this document. It is an unfortunate consequence of my efforts to be transparent, but I suspect that many readers will wonder whether all of the tables and figures are really necessary. I have three responses. First, I hope that this document might serve as more than just a detailed journal of my analyses. In fact, there are several passages of text that delve deeply into theoretical issues of personality structure, and the statistical code is totally optional for those who are not interested (I don't think it's necessary to understand any of the major points). I've avoided psychometric jargon whenever possible and tried to provide simple and informal explanations when it was unavoidable. Second, I have tried to organize the text into relatively independent and digestable sections. No one is going to read this in one sitting, so I've split the text into many short sections and made liberal use of cross-referencing links. The majority of the document is made up of appendices --- figures and tables that generally provide descriptive information on the factor analyses output and the derived scales (these include assessment forms and instructions, item response modeling parameters, and population norms) --- and these are only included for the sake of reference. And finally, I hope that the length and topical breadth of this document will make it a resource for those hoping to explore new topics in personality structure. I have highlighted several areas that warrant further research and I suspect many readers will identify weaknesses, gaps and opportunities that haven't occurred to me. I have often been asked over the last few years whether there is anything left to do in the field of personality structure. I think we have our work cut out for us. \begin{flushright} \emph{--- David M. Condon (October 31, 2016)} \end{flushright} \begin{scriptsize} <>= # In this first step we are simply getting things ready by installing and loading necessary packages and functions. These are hidden in the pdf output. rm(list = ls()) suppressPackageStartupMessages(library(psych)) # if you don't have, get it; if not loaded, load it suppressPackageStartupMessages(library(parallel)) suppressPackageStartupMessages(library(plyr)) suppressPackageStartupMessages(library(multicon)) @ \end{scriptsize} \clearpage \tableofcontents \clearpage \pagenumbering{arabic} \begin{center} \large\textbf{The SAPA Personality Inventory:\\[0pt]An empirically-derived, hierarchically-organized\\[0pt]self-report personality assessment model}\par \vspace{3mm} \large David M. Condon\footnote[$\star$]{Preparation of this file was funded in part by grant SMA-1419324 from the National Science Foundation and grant R01-AG018436 from the National Institutes of Health (NIA). Correspondence should be directed to David M. Condon, Assistant Professor, Department of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, \href{mailto:david-condon@northwestern.edu}{david-condon@northwestern.edu}. Please use the following citation information: Condon, D.M. (2017). The SAPA Personality Inventory: An empirically-derived, hierarchically-organized self-report personality assessment model.} \end{center} \section{Introduction} \label{sec:intro} There already exists a surfeit of options for those in search of self-report measures which provide well-validated scores across slightly distinct variations of the Big Five dimensions. Each of these has its own raison d'\^{e}tre and loyal advocates. An incomplete list of the most prominent include the unipolar Big Five factor markers \citep{Goldberg:92}, the NEO-PI-R\texttrademark \citep{Costa:92a, McCrae:2010}, the Abridged Big-Five Dimensional Circumplex \citep[AB5C,][]{Hofstee:92}, the Big Five Inventory \citep[BFI,][]{John:1999, Soto:2016jr}, the Five- and Ten-Item Personality Inventories \citep{Gosling:03b}, and the Mini-IPIP scales \citep{Donnellan:06}. Some of these are available in both the original and slightly altered public-domain formats comprised of items from the International Personality Item Pool (e.g., the ``IPIP'' items corresponding to the Big Five factor markers, the NEO-PI-R\texttrademark, and the AB5C). If the list is expanded to include those measures which are often interpreted in five factor terminology (such as the Hogan Personality Inventory [\citealp{Hogan:92}]) or include the additional dimension of Honesty/Humility (e.g., the IPIP-HEXACO [\citealp{Lee:04}] and the 48-item Questionnaire Big Six scales [\citealp[QB6,][]{Thalmayer:11}]) or are derivative of the five dimensions (e.g., the Big Five Aspect Scales [\citealp[BFAS,][]{DeYoung:07}]) or have been translated into other languages \citep[see][]{Ashton:04, Saucier:09a, DeRaad:10}, the seemingly subtle distinctions between measures borders on esoterica. The flourishing growth of measures illustrates the breadth of support for the (approximately) five factor structure and this consensus has precipitated broader recognition of the utility of personality for predicting important life outcomes \citep{Barrick:91, Hogan:96, John:1999, Paunonen:03, Roberts:07}. Research on the utility of the Big Five scales is based upon the administration of these measures to several million participants over the last two decades \citep{Obschonka:13, Soto:10}; over the same time frame, the nine measures described in the prior paragraph have been cited nearly 20,000 times. In short, the Big Five created a flourishing market for personality assessment and many tools have been developed over the last 25 years to meet the growing demand. Yet, there are many reasons to believe that personality assessments could be better. Here are some of the most prominent: \begin{enumerate} \item \textbf{Limitations of the Big Five as a predictive model.} The Big Five were derived from efforts to identify the most parmonious number of factors in trait-descriptive adjectives. It was not the goal of these efforts to be highly predictive, but rather to balance the demands of explained variance and data reduction. In other words, prediction was not the top priority when the Big Five were identified. This issue is exacerbated by the fact that many Big Five (and Big Six) scales have been designed with a focus on internal consistency rather than predictive validity (this is particularly true for shorter scales). This means that most measures do a good job of assessing relatively narrow operationalizations of the five dimensions and ignore large swaths of the individual differences landscape.\footnote{There are notable exceptions, especially among the longer forms of personality measures (e.g., the NEO-PI-R\texttrademark, the HEXACO and their public-domain counterparts), but these tend to be scales that were explicitly designed to include more nuance within the five or six factors. In other words, the most predictive measures are those that deviate from the Big Five structure by allowing each of the factors to contain some internal structure. Note that it's not accurate to state that these measures presume 24 or 30 factor structures per se (for the HEXACO and NEO respectively) because these facets are explicitly nested within the factors. By analogy, it's the difference between 5 different six-packs of beer and 30 different beers.} As a consequence, the Five Factor measures tend to have narrow predictive validity. \item \textbf{Psychometric advances in measurement.} Over the last 30 years, the use of computer adaptive testing (aka tailored testing) has become common in most testing environments. This technique allows for more brief and efficient testing without loss of precision by matching participant responses to the ``difficulty'' of subsequent items. An application to personality, for example, could be that individuals who strongly endorse an item about sociability (``I enjoy small gatherings'') would subsequently be administered a sociability item that is more difficult (``I enjoy large parties''). The use of computer adaptive tests (CATs) in personality assessment would seem to be a great pairing because it reduces the mean number of items administered to each participant without sacrificing precision. But, there is a hitch. The psychometrics behind CAT assessment presume that all of the items in a test (or scale) are interchangeable measures of the same dimension despite differing levels of difficulty; the items as a set need to be highly ``unidimensional.''\footnote{In psychometric terms, omegas of 0.7 are acceptable, 0.8 is good.} Most Big Five measures don't come close to meeting this criteria. The few that do use narrow operationalizations of the Big Five constructs. More powerful applications would combine CAT-based techniques with the assessment of many narrow personality dimensions. This approach would also offer an important secondary benefit: it would allow for improved evaluations of differential item functioning and structural equivalence across groups (e.g., based on gender, age, SES, health status). \item \textbf{Rationally-derived tools are subject to bias.} It's not entirely accurate to describe measures of personality as being either empirically or rationally derived. They all involve at least a little theory and are rooted in data of one form or another. That said, there is an increasing emphasis on the use of assessments and decision making tools that are more heavily informed by data --- and the bigger the data, the better. On average, rationally-derived models are less predictive, less informative, and less powerful than those which are based exclusively on data. Examples of rationally-derived tools in personality assessment are the facet level scales which are neatly nested within higher level factors. They are rationally influenced in that the organization and scoring of those scales is based on very strict assumptions about the hierarchical structure, even though these assumptions are known to be violated.\footnote{\href{https://en.wikipedia.org/wiki/Spherical_cow}{Consider a spherical cow...}} The developers of these hierarchical measures have favored theory over empiricism, presumably for the sake of simplicity. The risks are that these scales fail to capture the most informative constructs and/or measure them using sub-optimal (inefficient) operationalizations. This doesn't mean they are invalid, but rather that they can \textit{only be less or equally valid} (almost always less, by definition) than empirically-derived scales of the same length. \item \textbf{The data used for most empirically-derived measures is limited and homogenous.} The argument that rationally-derived measures are less useful than those with empirical support depends upon the quality of the underlying data. Quality is, in turn, largely dependent upon the relationship between the representativeness of the data used to develop the scales and the population being assessed. All things being equal, larger samples are more representative than smaller ones (because the influence of the idiosyncratic qualities of each individual participant is reduced) and closer matching of exogenous variables (like demographics) is better than less. There are several widely-used measures of personality that were derived based on large samples. Unfortunately, they're all based on the \textit{same} large sample --- the Eugene-Springfield Community Sample \citep{Goldberg:99a}. This is not necessarily problematic by itself\footnote{In fact, it might be better to use the same sample for direct comparison across measures.} but there is considerable cause for concern about the representativeness of this sample relative to the global population (98.4\% are white homeowners from Oregon). \item \textbf{Scale developers have abandoned the use of trait descriptive adjectives.} The majority of Big Five measures are at least one step removed from the factor-analytic procedures used to evaluate the multidimensional structure of the trait lexicon. This is readily apparent from the format of the measures. All but a few --- notably, the Big Five unipolar and bipolar factor markers \citep{Goldberg:92} and the rationally sorted factors of the Adjective Check List \citep{John:1999} --- use phrases or sentences (``phrased items'') instead of single word adjectives and type-nouns because they are more clear, precise, and contextualized. But, this change in format means that the most widely-used scales are not empirically derived from factor analyses of the finite trait lexicon, nor are they empirically derived from factor analyses of the phrased item universe. Instead, scale developers have assumed that the multidimensional space of phrased items is structurally similar to that of the well-studied space of the trait lexicon. Unfortunately, there is a lot of evidence to suggest that this assumption of structural similarity may not be tenable \citep{Ashton:04, DeRaad:10, Digman:86, Digman:90, Eysenck:91, Eysenck:94, Hendriks:99, Hendriks:03, Lee:05, Pettersson:10, Yarkoni:13}.\footnote{More precisely, I would argue that it's very unlikely that the structure of these multidimensional spaces is identical and that the severity of the problem is related to the extent of the difference.} The point here is that it would be better (to an unknown extent) if the scales being used were based on the structure of the items being used. \item \textbf{The Big Five passes the sniff test but not the yawn test.} It's only been a little while since ``we had no personalities'' \citep{Goldberg:93}. The Big Five was instrumental in helping us to demonstrate that personality has existed all along, in large part by simplifying the morass of personality constructs that had previously been proposed. Now, it seems that the Big Five is maligned for being a little too simple. It has been my experience in discussions with students and colleagues from other disciplines of psychology that there is a perceived misalignment between the five factor structure and the rich diversity of individual differences that can be readily observed on a daily basis. The fact that this perception reflects a limited understanding of the Big Five model does not mean it should be ignored. To the contrary, this problem is now being publicly broadcasted by scholars with considerable influence \citep{Baumeister:2016}. It certainly does not make sense to abandon the best scientific paradigm in favor of one with more panache, but it seems worth evaluating whether more nuanced approaches can address this misalignment. Nuanced approaches that are also compatible with the Big Five and provide incremental validity (even if at the expense of parsimony) would seem to have particularly strong potential. \end{enumerate} In sum, it seems there are many ways in which the assessment of personality might be improved, even though the practical motivation for developing new measures is less clear. Maybe the current options are good enough. It's possible that attempts to empirically-derive new scales will produce measures that are fully redundant with existing Big Five scales. A more likely outcome is that preliminary evidence will emerge for a hierarchical structure that allows for personality assessment across multiple levels (previously referred to as factors, aspects, facets, and nuances). Whatever the outcome in practical terms, there will at least be some benefit to the collection of data from a large sample using a substantial portion of the items in the International Personality Item Pool. So, that is what I've aimed to do --- evaluate the structure of as many items as possible based on administration to as many participants as possible. Here's how things will go: we'll \href{https://youtu.be/rePNg6MmdEQ}{begin} with a description of the process for selecting the items to be administered throughout this research project (Section \ref{sec:IPIPselection}) before describing the characteristics of the samples to which the items were administered (Section \ref{sec:samples}). Then we'll conduct a wide range of sensitivity analyses (Section \ref{sec:sensitivity}), with a specific focus on the stability of correlations within and across all of the samples (Section \ref{sec:stability}). In the next section, we'll conduct exploratory factor analyses (Section \ref{sec:structural}) and then consider the extent to which these analyses replicate (Section \ref{sec:replication}). Next, I will account for item-level qualitative data on clarity, salience, and readability (Section \ref{sec:itemqualitative}). Then, the evidence for replicable structure and item-level qualitative characteristics are used to develop scales at two hierarchical levels in the remainder of Section \ref{sec:scales}. I evaluate the fit of these scales in an independent sample using confirmatory factor analysis procedures in Section \ref{sec:confirmatory}, describe the procedures for naming the scales in Section \ref{sec:naming}, and describe their psychometric properties in Section \ref{sec:psychodescribe}. At last, I conclude with an overview of topics where additional research is needed (Section \ref{sec:future}). \section{Description of the items administered} \label{sec:IPIPselection} \subsection{Comments on previous approaches to item selection} The structure of personality based upon the presumption of the ``Lexical Hypothesis'' \citep{Galton:1884, Goldberg:81a, Goldberg:93} is empirically well-established. It has been argued however that the lexical approach is not well-suited to the development of a hierarchical model of personality \citep{Costa:95} because (1) some important traits are under-represented in the lexicon and (2) trait terms in the lexicon reflect inconsistent breadth. The latter point is perceived to give broader terms an advantage because they account for a disproportionate percentage of the covariance. \citet{Costa:95} claim that these limitations motivated their use of a ``top-down'' theoretical approach to the development of the NEO-PI-R\texttrademark \citep{Costa:92a, McCrae:2010}, a seminally important, proprietary, hierarchical measure of self-report personality. Yet, these limitations do not preclude the possibility of an empirical ``bottom-up'' approach to hierarchical scale development --- they merely suggest that such an approach should not be based on trait-descriptive adjectives and nouns. This is not much of a problem either because most personality measures have long since deviated from assessment by adjective checklist. In fact, the advantages of using so-called ``phrased'' items instead of single word adjectives and type-nouns are widely acknowledged \citep{Briggs:92, Gosling:03b, Goldberg:99a, Hendriks:99, John:1999}. These include the observation that slightly longer phrases result in more clear, precise, and consistent interpretations of items by laypersons, particularly when it comes to low frequency descriptors. Phrased items allow for improved contextualization of trait descriptions and are unencumbered by the requirement of being classified as a recognizable dictionary entry. For example, the degree to which respondents describe themselves as being ``querulous'' is not confounded by familiarity with that specific word. It is far better to ask them if they complain a lot under various circumstances. Unsurprisingly, \citet{Costa:95} were dismissive of previous attempts at bottom-up scale creation using phrased items (e.g., the California Personality Inventory [\citealp[CPI,][]{Gough:75}] and the Minnesota Multiphasic Personality Inventory\textsuperscript{\textregistered} [\citealp[MMPI\textsuperscript{\textregistered},][]{Butcher:2001tq}]\footnote{The original MMPI\textsuperscript{\textregistered} having been developed in the 1930s and 1940s on a scale-by-scale basis using data from patients with a wide range of clinical disorders \citep{Hathaway:1940id} and the CPI having been derived using similar techniques and even many of the MMPI\textsuperscript{\textregistered} items \citep{GrothMarnat:2009}.}). Their dismissal was explicitly driven by the fact that these approaches omitted one or more of the prominent Big Five constructs. Given the breadth of empirical data supporting the Big Five, this seems (post-hoc) like a very good argument though it lacks empirical rigor from an a priori perspective. Of course, this dismissal is also consistent with their personal motivations to bolster the case for their top-down approach.\footnote{Note that their approach has passed muster for several decades now, so I don't mean to imply that their dismissal was inappropriate --- I'm only pointing out that it may have been premature and, perhaps coincidentally, self-serving.} Several other hierarchical measures have been put forth since the NEO-PI-R\texttrademark was released though the extent to which each has taken a truly bottom-up approach is variable and/or unclear. Soto \& John have developed a hierarchical version of the Big Five Inventory that is explicitly tied to the NEO-PI-R\texttrademark facets. The initial version of this model \citep{Soto:09} organized the original 44 items of the BFI \citep{Soto:09} into 10 facets. The second version of this model \citep[BFI-2,][]{Soto:2016jr} has 60 non-IPIP items (several of these are new) designed to tap three theoretically-derived facets for each of the Big Five (with a particular focus on the facets represented in the original BFI). The six factor HEXACO model \citep{Ashton:04, Lee:04} has been designed to include four facet-level scales for each factor though these seem to have been developed using a top-down approach. The Big-Five Aspect Scales \citep{DeYoung:07} approach is difficult to characterize; it is theory driven by virtue of being derivative of the IPIP Big Five Factor Markers \citep{Goldberg:99a} though also based on well-specified scale development procedures. \subsection{Issues of item sampling} \label{sec:itemsamplingissues} An empirical approach to the development of hierarchical personality scales would require a broad yet representative item set; that is, one which samples from the universe of potential items in a manner that is neither strictly uniform (this would ignore the evidence against the equivalent salience of personality constructs) nor intentionally cluster-based (this would increase the likelihood of overlooking constructs that are not highly salient but are still predictive of some important outcome). For the purpose of scale development, representativeness of item sampling would constitute the inclusion of items which represent important features of personality in proportions consistent with their identification across individuals \citep[p. 28,][]{Goldberg:92}. Unfortunately, it is impossible to identify an item set that is so optimally representative given that the full universe of potential items is undefined. A practical solution to this dilemma is provided by the International Personality Item Pool \citep[IPIP,][]{Goldberg:99a, Goldberg:06}. While I am not aware of any prior research exploring the representativeness of the IPIP items to the hypothetical universe of all possible phrased personality items (it is hard to imagine how one would begin to evaluate this), it is currently the best available pool of items for personality scale development --- its utility stems not only from the large number of items but also from the growing number of public-domain personality measures which have been derived based on joint administration of the IPIP items and various proprietary measures \citep{Goldberg:99a}. Since its inception, more than 250 personality scales have been ``created'' by use of this procedure \citep{ipip}. The IPIP items have their origin in a set of approximately 900 items constructed by a team of researchers (Hofstee, De Raad, and Hendriks) at the University of Groningen in the 1990s \citep{Goldberg:99a, Goldberg:09, Hendriks:97, Hendriks:99, Hendriks:03}. The formatting procedures for the items is summarized as follows: \begin{quote} Explicit guidelines (Hofstee, 1991) were followed to ensure creating items for an instrument that can be used for a broad range of educational levels, avoids discrimination of certain people or groups of people, and elicits ratings as objectively as possible. These guidelines address the items' format (phrase items in the third person singular and in observable terms) and wording (avoid idiom, difficult words and expressions, suggestive formulations, etc). \citep[p. 350]{Hendriks:03}. \end{quote} \noindent The first-person English translation of these Groningen items, along with another 500 items generated by Goldberg, served as the initial basis for the IPIP item set. The total item count now stands at 3,329 and each of these have been administered to various (overlapping) subsets of the Eugene-Springfield Community Sample \citep[for more details, see][]{Goldberg:99a, Goldberg:06, Goldberg:09, Goldberg:2016vg}. We briefly considered developing a set of personality scales based on administration of \textit{all} of the IPIP items to very big, representative(ish) online samples. There are two drawbacks to this approach. The first is fully practical. It would be untenable to administer all of the items to online samples that are large enough to generate reasonably replicable (i.e., stable) correlation matrices. While this was done for the Eugene-Springfield Community Sample \citep{Goldberg:99a}, it took many years and required considerable funding from the National Institute of Aging. One alternative would be to hire a panel survey company to collect data from a sample of participants meeting targeted parameters, but this would be relatively expensive (this research project was not grant funded). Despite the expense, the chief difficulty to this approach would stem from the time and burden placed on the participants... it's not trivial to respond to 3,300 items in a reasonably short period of time. The participants would have to be highly compensated and it would require data collection over many brief sessions in order to ensure that the participants could maintain their focus. It would also necessitate very careful screening of participants. If each participant answered every item (i.e., provided ``full information'') and a relatively small sample was used (say, 500 people), then the responses for each individual participant would be very influential. Even in this case, there are many demographics variables to consider when identifying a diverse sample (e.g., gender, age, race/ethnicity, education level, geography) and there is a good chance that several of the resulting ``bins'' would be sparsely populated (and the subsequent scores would be equivalently idiosyncratic to those individuals). Many of these obstacles can be addressed by giving a random subset of the items to each participant as is done with the Synthetic Aperture Personality Assessment \citep[SAPA,][]{Revelle:2016chap, Condon:14, Wilt:11b}. The most important benefit to the SAPA method is that it's (kind of) free.\footnote{Full disclosure: it's only free if you ignore the opportunity costs incurred by one or more graduate students who are paid very low stipends to write the code and manage the data collection.} Since each participant is given only a manageable subset of the items (say, 85), the burden felt by each is relatively small. If participants are given some other motivation to complete the items (this was achieved in these samples by giving participants feedback on their responses), they may not need to be paid at all (or perhaps they could be paid just a little as is frequently done with MTurk-like crowd-sourcing). A bit of math is required to figure out how many more participants would be needed to offset the reduction in responses per participant when all IPIP items are administered. If each participant is administered approximately 85 items (\textit{n}) instead of 3,329, the probability of any one item being administered to a participant (\textit{$p_i$}) is \textit{n}/\textit{P} (85/3,329 = \Sexpr{prettyNum(round(85/3329,2), big.mark=",", scientific=F)}). If the target is a mean of 500 pairwise administrations (for items i and j), then the number of participants needed would be 500/(\textit{$p_i$} x \textit{$p_j$}), where \textit{$p_i$} $\approx$ \textit{$p_j$}. This is approximately \Sexpr{prettyNum(round(500/(85/3329*85/3329),0), big.mark=",", scientific=F)} participants for a sample with reasonably good stability in terms of pairwise item correlations (see Section \ref{sec:stability} for more discussion of stability). So, while this method of random item sampling substantially reduces the participant burden and resource burden relative to ``full information'' data collection, it also dramatically increases the required sample size. Given the historical rate at which we had been able to recruit participants (e.g., our web traffic), it would have taken many years to collect enough data without driving up costs. This obstacle caused me to reconsider whether it was necessary to collect data on all of the IPIP items, especially given that most of them have not shown high correlations with existing personality measures. In fact, I quickly realized that there is considerable overlap among many of the IPIP measures that were derived based on the strength of correlations between IPIP items and extant proprietary measures following administration to the ESCS sample (using the methods described in \citet{Goldberg:99a} and \citet{Goldberg:06}). This does not mean that IPIP items with low-magnitude correlations to other measures are not useful. They may be quite predictive for one or a few narrowly-defined criterion variables, but such items are less likely to be needed for a generalizable set of individual differences measures. This issue relates back to the initial question about identifying a broad but representative pool of items from which to sample. I reasoned that an efficient solution would be to administer the IPIP items in stages. In the first stage, I would administer the largest possible subset while balancing the competing desires for broad coverage of personality constructs and a high number of mean pairwise administrations of the items. All of the work described here encompasses this first stage.\footnote{In the next stage(s), I intend to administer additional IPIP items with overlap to all or some key subset of the items from this stage. After many years of data collection, we may eventually be able to evaluate the structure of all items together. No further discussion of this plan will be described here but please contact me for more information about ongoing data collection and future plans.} Close examination of the many IPIP-based measures suggested that only about 20\% of the item pool was needed for good coverage of many personality constructs. In fact, only 617 unique items were needed to account for all of the items in many widely-used sets of measures and a majority of the items in many more. Table \ref{tab:IPIPscales} shows the 168 IPIP-based scales which are 100\% covered and an additional 235 IPIP-based scales which are roughly 60\% to 85\% covered by this item set. Note that it is difficult to describe the degree and extent of partial coverage. I have focused on the degree of coverage for the multi-construct self-report sets of personality scales listed on the \href{http://ipip.ori.org/newMultipleconstructs.htm}{IPIP website}. This is not an exhaustive list; many other scales (especially single-construct scales) are known to be partially or fully covered. \begin{table}[htbp]\footnotesize \caption{Sets of personality measures covered by the 696 item sample} \begin{center} \begin{tabular} {l r r r r} \hline Set of Measures & Number & Total & Items & Percent \cr (IPIP-based correlates) & of Scales & Items & Covered & Covered \cr \hline \cr \multicolumn{5}{l}{\underline{Personality measures fully-covered by the 696 item sample}}\cr \cr Costa \& McCrae's \citeyearpar{Costa:92a} NEO-PI-R\texttrademark Domains (300 items) & 5 & 100 & 100 & 100\% \cr Costa \& McCrae's \citeyearpar{Costa:92a} NEO-PI-R\texttrademark Facets & 30 & 300 & 300 & 100\% \cr DeYoung, Quilty, \& Peterson's \citeyearpar{DeYoung:07} Big-Five Aspects & 10 & 100 & 100 & 100\% \cr DeYoung, Quilty, \& Peterson's \citeyearpar{DeYoung:07} Big-Five Aspects (Domains) & 5 & 100 & 100 & 100\% \cr DeYoung's \citeyearpar{DeYoung:10a} Plasticity \& Stability scales & 2 & 40 & 40 & 100\% \cr Donnellan, Oswald, Baird \& Lucas' \citeyearpar{Donnellan:06} Big-Five Factor Markers (``Mini-IPIP'') & 5 & 20 & 20 & 100\% \cr Eysenck's \citeyearpar{Eysenck:85} Personality Questionnaire - Revised & 3 & 79 & 79 & 100\% \cr Goldberg's \citeyearpar{Goldberg:92} Big-Five Factor Markers (100 items) & 5 & 100 & 100 & 100\% \cr Goldberg's \citeyearpar{Goldberg:92} Big-Five Factor Markers (50 items) & 5 & 50 & 50 & 100\% \cr Johnson's \citeyearpar{Johnson:14} IPIP NEO-PI-R\texttrademark Domains & 5 & 120 & 120 & 100\% \cr Johnson's \citeyearpar{Johnson:14} IPIP NEO-PI-R\texttrademark Facets & 30 & 120 & 120 & 100\% \cr Lee \& Ashton's \citeyearpar{Lee:04} HEXACO-PI Domains & 6 & 240 & 240 & 100\% \cr Lee \& Ashton's \citeyearpar{Lee:04} HEXACO-PI Facets & 24 & 240 & 240 & 100\% \cr Maples et al.'s \citeyearpar{Maples:2014hr} IPIP NEO-PI-R\texttrademark Domains & 5 & 120 & 120 & 100\% \cr Maples et al.'s \citeyearpar{Maples:2014hr} IPIP NEO-PI-R\texttrademark Facets & 30 & 120 & 120 & 100\% \cr Tellegen's \citeyearpar{Tellegen:2003ue} Multidimensional Personality Questionnaire & 12 & 127 & 127 & 100\% \cr Thalmayer, Saucier, \& Eigenhuis' \citeyearpar{Thalmayer:11} Questionnaire Big-Six (24 items) & 6 & 24 & 24 & 100\% \cr Thalmayer, Saucier, \& Eigenhuis' \citeyearpar{Thalmayer:11} Questionnaire Big-Six (36 items) & 6 & 36 & 36 & 100\% \cr Thalmayer, Saucier, \& Eigenhuis' \citeyearpar{Thalmayer:11} Questionnaire Big-Six (48 items) & 6 & \underline{48} & 48 & 100\% \cr & & 2,084 & & \cr \cr \multicolumn{2}{r}{Total unique items (this set):} & 696 & 696 & \cr \cr \multicolumn{5}{l}{\underline{Personality measures partially-covered by the 696 item sample}}\cr \cr Hogan \& Hogan's \citeyearpar{Hogan:2007tg} HPI High Level Scales & 13 & 130 & 111 & 85\% \cr Saucier's \citeyearpar{Saucier:1997uw} 7-factor scales & 7 & 70 & 54 & 77\% \cr Jackson's \citeyearpar{Jackson:2004tx} Jackson Personality Inventory & 15 & 145 & 111 & 77\% \cr Hogan \& Hogan's \citeyearpar{Hogan:2007tg} HPI Homogenous Item Clusters & 44 & 260 & 179 & 69\% \cr Cattell's \citeyearpar{Cattell:2002wh} 16 Personality Factor Questionnaire & 16 & 163 & 109 & 67\% \cr Cloninger et al.'s \citeyearpar{Cloninger:1994to} Temperament and Character Inventory & 30 & 289 & 189 & 65\% \cr Gough's \citeyearpar{Gough:1996wm} California Psychological Inventory & 33 & 332 & 201 & 61\% \cr Markey \& Markey's \citeyearpar{Markey:2009bc} IPIP Interpersonal Circumplex Scales & 8 & 32 & 19 & 59\% \cr Jackson, Paunonen, \& Tremblay's \citeyearpar{Jackson:2000tj} 6-Factor Personality Questionnaire & 24 & 233 & 137 & 59\% \cr Hofstee, de Raad \& Goldberg's \citeyearpar{Hofstee:92} AB5C facets & 45 & \underline{486} & 278 & 57\% \cr & & 2,140 & & \cr \cr \multicolumn{2}{r}{Total unique items (this set):} & 879 & 429 & \cr \cr \multicolumn{2}{r}{Total unique items (both sets):} & 1,146 & 696 & \cr \hline \end{tabular} \end{center} \label{tab:IPIPscales} \end{table} The identification of these specific 617 items resulted from the desire to include all of the items from a broad and balanced group of measures that were theoretically diverse and frequently referenced in the personality literature, with the caveat that all of the measures needed to be in the public-domain given our intention for online assessment to large samples (and a very limited research budget). These are the scales that were targeted: \begin{itemize} \vspace{-2mm} \item the 100 IPIP items corresponding to the Big Five factor markers \citep[][]{Goldberg:99a} \vspace{-2mm} \item the IPIP-NEO \citep[300 items,][]{Goldberg:99a} \vspace{-2mm} \item the IPIP-HEXACO inventory \citep[240 items,][]{Ashton:07b} \vspace{-2mm} \item the Big Five Aspect Scales \citep[100 items,][]{DeYoung:07} \vspace{-2mm} \item the Questionnaire Big Six scales \citep[48 items,][]{Thalmayer:11} \vspace{-2mm} \item the IPIP-Multidimensional Personality Questionnaire \citep[127 items,][]{Tellegen:08} \vspace{-2mm} \item the Plasticity/Stability scales \citep[40 items,][]{DeYoung:10a} \end{itemize} The sum total of items in these measures is 955, including 338 duplicates and 617 unique items. Of these, 403 items are in only one set of scales, 130 items are included in two sets of scales, 54 items are in three, 20 items in four, and 10 items are in five of the seven sets of IPIP-based scales. We felt that these 617 items provided a balanced set of measures based on the Big Five, Big Six and alternative frameworks, but I also felt that this balance could be improved upon by inclusion of an item set reflecting Eysenck's 3 factor structure \citep{Eysenck:85}. To address this need, I generated a set of items that match the IPIP item format but are based on the 79 items of the Eysenck Personality Questionnaire - Revised \citep{Eysenck:85}; note that the 21 items from the ``lie'' scale were intentionally omitted. This brought the total count of unique items to 696. \subsection{Conclusions about representativeness of the item sample} This 696-item set is not meant to be exhaustive but rather sufficiently balanced and representative for the purpose of developing hierarchically organized sets of scales. It contains public-domain versions of most of the items in self-report assessments of normal-range personality that have been widely-used in academic research over the last 30-40 years.\footnote{Note that these items do not explicitly include those in the Big Five Inventory \citep[BFI, 44 items,][]{John:1999} or the Ten Item Personality Inventory \citep[10 items,][]{Gosling:03b}, both of which are widely-used. Those measures are formatted differently than the IPIP items. The content of these 54 items is covered (with highly similar wording) by items in the 696. Concurrent administration of those items with this set is a priority for future research, but I do not expect that inclusion of those items would extend the conceptual breadth of this 696-item subset of the IPIP.} It seems reasonable to claim that these 696 items cover \textit{most} of the constructs that are prominently recognized in the literature without over-representing \textit{many} constructs of narrow relevance. In any case, it does not need to be optimal. The next challenge (and source of joy!) will come from determining how this pool of items should be refined and expanded in order to improve the predictive utility of the resultant measures for the many applications where individual differences are relevant. If these items can be used to identify a hierarchically organized set of scales which is likely to replicate, these scales might be modified in the future using: (1) factor extension procedures to incorporate items which are not in the current set but correlate highly with factors or facets at one or more levels of the hierarchy; and/or (2) CFA/ICLUST/EFA (replication) procedures to evaluate the extent to which items outside of these 696 warrant the specification of additional facets. The most practical benefit of using this reduced item set (instead of all IPIP items) is that it makes data collection far more feasible in terms of the number of participants needed to collect adequately-sized samples. Since the total item count is squared in the denominator of the sample size calculation, only 1/25ths as many participants are needed if we only use 1/5th as many items. With only 696 items, the number of participants needed to achieve stability was conservatively estimated at \Sexpr{prettyNum(round(500/(84/696*84/696),0), big.mark=",", scientific=F)} participants. The following section describes three samples that were administered these items between December 3, 2013 and February 7, 2017. \section{Description of the samples} \label{sec:samples} The goal of this section is to describe the samples collected and then compare them to the U.S. population using data collected by the U.S. Census Bureau. These descriptions are preceded with a brief discussion of the procedures used for data collection. \subsection{Comments on the data collection procedures} \label{sec:SAPA} Even after reducing the number of IPIP items to be administered (from 3,329 to 696), I still had to deal with the challenge of administering many more items than any one person would be willing to complete. With the exception of the Eugene-Springfield Community Sample \citep{Goldberg:99a}, I'm not aware of any other samples of similar size and breadth in the personality literature. In fact, in the ``big data'' era, breadth of assessment has become an increasing salient concern. Data collection over the internet has made it considerably easier to increase the number of participants in a sample \citep{Buhrmester:11, Obschonka:13, Johnson:14} but it has not changed the fact that individual participants are only willing and able to respond to a limited number of items. Synthetic Aperture Personality Assessment (``SAPA'') techniques represent a methodological innovation for dealing with this obstacle \citep{Revelle:2016chap, Condon:14, Wilt:11b}. Named by analogy to the use of similar techniques in radio and optical astronomy, SAPA essentially makes use of modern sampling procedures. Rather than combining signals from the same source using different telescopes as is done in astronomy, the structure of personality can be studied by combining the responses of many people across more items than any one person is willing to answer. Instead of observing celestial objects beyond the visible range, we can observe the relations between personality constructs which would not otherwise be visible given practical assessment constraints. This can be done by sweeping the assessment ``telescope'' across a wide range of constructs or by focusing for short periods of time on high-priority topics. This procedure is not without precedent. Lord \citeyearpar{Lord:55} has previously described theoretical procedures for the sampling of items (rather than participants) in the context of testing and similar sampling techniques have long been used by the United States Census Bureau \citep{Navarro:93}, the Programme for International Student Assessment \citep[``\href{https://en.wikipedia.org/wiki/Fall_on_Me}{PISA}''][]{Anderson:07, PISA:09}, the German Socioeconomic Panel \citep{HaiskenDeNew:05}, and the National Assessment of Educational Progress \citep{Mislevy:92}. Thanks in large part to the insight and inspiration of Bill Revelle, these techniques have been refined over the last several years for use with public-domain items administered via the internet (\urlstyle{same}\url{sapa-project.org}). In addition to the improved geographical diversity which is common to many internet samples, SAPA procedures allow for evaluation of the structural characteristics of personality items across measures (e.g., various IPIP measures administered together) and domains (e.g., vocational, cognitive ability [\citealp{Condon:14}]), and a broad range of criterion variables such as health behaviors and occupations. Three samples were collected for this project using these SAPA procedures. In the following three sections (\ref{sec:expDesc} to \ref{sec:confDesc}), each of these samples are imported and described. Prior to being imported, the data have been cleaned a bit to: (1) remove scores from participants who tell us that they've previously taken the survey; (2) remove participants who took the survey more than once using the same browser; and (2) drop participants who claim to be under 14 or over 90 years of age. All of the personality items in these samples were administered with the same six response options (``Very Inaccurate'', ``Moderately Inaccurate'', ``Slightly Inaccurate'', ``Slightly Accurate'', ``Moderately Accurate'', ``Very Accurate''). \subsection{Description of the Exploratory sample} \label{sec:expDesc} Here, descriptive statistics (and other characteristics) are shown for the demographics, psychographics and raw data for the first sample, collected between December 3, 2013 and July 26, 2014. This sample is denoted below as the \textbf{Exploratory} sample. These descriptive statistics include the number of participants and information about gender, country of origin, age, marital status, race/ethnicity, educational attainment level and employment status. First, I load the dataset and drop the irrelevant variables. \begin{scriptsize} <>= filepathdata <- "~/Desktop/data/" #filepathdata <- "/Users/.../SPI/analyticFiles/data/" filepathfig <- "~/Desktop/figures/" #filepathfig <- "/Users/.../SPI/analyticFiles/figures/" # load the exploratory dataset exactly as downloaded from Dataverse load(paste(filepathdata, "sapaTempData696items08dec2013thru26jul2014.RData", sep = "")) expSample <- subset(sapaTempData696items08dec2013thru26jul2014, select = -c(RID, occPrestige, occIncomeEst, p1edu, p1occPrestige, p1occIncomeEst, p2edu, p2occPrestige, p2occIncomeEst)) @ \end{scriptsize} Now, transform each of the descriptive statistics into a distinct object in \texttt{R}. These objects will later be used for additional analyses or the creation of tables and figures. \begin{scriptsize} <>= # nE is total Ps in the exploratory sample nE <- dim(expSample)[1] nEtext <- prettyNum(nE, big.mark=",", scientific=F) nEtext # percent female femalePercE <- prettyNum(100 * round((table(expSample$gender)/sum(table(expSample$gender)))["female"], 3), drop0trailing = FALSE) femalePercE # percent U.S. USpercE <- 100 * round((table(expSample$country)/sum(table(expSample$country)))["USA"], 3) USpercE # total U.S. UScountEtext <- prettyNum((table(expSample$country))["USA"], big.mark=",", scientific=F) UScountEtext # number of countries with P>0 countryTable <- sort(table(expSample$country),decreasing=TRUE) countryCountE <- length(countryTable) countryCountE # number of countries with P>49 countryCount50E <- subset(countryTable, countryTable > 49) length(countryCount50E) @ <>= expSampleAge <- describe(expSample[,"age"]) expSampleAge MeanAgeE <- expSampleAge[3] SDAgeE <- expSampleAge[4] MedianAgeE <- expSampleAge[5] MinAgeE <- expSampleAge[8] MaxAgeE <- expSampleAge[9] marstatusE <- data.frame(table(expSample$marstatus)) colnames(marstatusE) <- c("Marital Status", "Participants") ethnicE <- data.frame(table(expSample$ethnic)) colnames(ethnicE) <- c("Ethnicity/Race", "Participants") educationE <- data.frame(table(expSample$education)) colnames(educationE) <- c("Educational Level", "Participants") jobstatusE <- data.frame(table(expSample$jobstatus)) colnames(jobstatusE) <- c("Job Status", "Participants") # plot age by gender pdf(paste(filepathfig, "ageGenderE.pdf", sep = ""), onefile = TRUE) with(expSample, bi.bars(as.numeric(expSample$age), as.numeric(expSample$gender), ylab="Age", main = "")) dev.off() @ \end{scriptsize} \begin{figure} \ffigbox{\includegraphics[width=.75\textwidth]{figures/ageGenderE.pdf}}{\caption{Age by Males (Blue) and Females (Red) in the Exploratory Sample}\label{fig:ageGenderE}} \end{figure} \begin{scriptsize} \begin{table}[htbp]\caption{Marital Status in the Exploratory Sample} \begin{center} \begin{tabular} {l r r } \hline Marital Status & Participants \cr \hline Never Married & \Sexpr{prettyNum(marstatusE[1,2], big.mark=",", scientific=F)} \cr Married & \Sexpr{prettyNum(marstatusE[2,2], big.mark=",", scientific=F)} \cr Domestic Partnership & \Sexpr{prettyNum(marstatusE[3,2], big.mark=",", scientific=F)} \cr Widowed \& Remarried & \Sexpr{prettyNum(marstatusE[7,2], big.mark=",", scientific=F)} \cr Divorced \& Single & \Sexpr{prettyNum(marstatusE[4,2], big.mark=",", scientific=F)} \cr Divorced \& Remarried & \Sexpr{prettyNum(marstatusE[5,2], big.mark=",", scientific=F)} \cr Widowed \& Single & \Sexpr{prettyNum(marstatusE[6,2], big.mark=",", scientific=F)} \cr \hline \end{tabular} \end{center} \label{tab:marstatusE} \end{table} \end{scriptsize} \begin{scriptsize} \begin{table}[htbp]\caption{Race/Ethnicity Among U.S. Participants in the Exploratory Sample} \begin{center} \begin{tabular} {l r r } \hline Ethnicity & Participants \cr \hline White & \Sexpr{prettyNum(ethnicE[13,2], big.mark=",", scientific=F)} \cr African American & \Sexpr{prettyNum(ethnicE[1,2], big.mark=",", scientific=F)} \cr Two Or More Ethnicities & \Sexpr{prettyNum(ethnicE[18,2], big.mark=",", scientific=F)} \cr Mexican & \Sexpr{prettyNum(ethnicE[9,2], big.mark=",", scientific=F)} \cr Other Hispanic & \Sexpr{prettyNum(ethnicE[8,2], big.mark=",", scientific=F)} \cr Chinese & \Sexpr{prettyNum(ethnicE[2,2], big.mark=",", scientific=F)} \cr Other Asian & \Sexpr{prettyNum(ethnicE[7,2], big.mark=",", scientific=F)} \cr Indian & \Sexpr{prettyNum(ethnicE[6,2], big.mark=",", scientific=F)} \cr Puerto Rican & \Sexpr{prettyNum(ethnicE[10,2], big.mark=",", scientific=F)} \cr Filipino & \Sexpr{prettyNum(ethnicE[5,2], big.mark=",", scientific=F)} \cr Native American & \Sexpr{prettyNum(ethnicE[11,2], big.mark=",", scientific=F)} \cr Korean & \Sexpr{prettyNum(ethnicE[4,2], big.mark=",", scientific=F)} \cr Cuban & \Sexpr{prettyNum(ethnicE[16,2], big.mark=",", scientific=F)} \cr Japanese & \Sexpr{prettyNum(ethnicE[3,2], big.mark=",", scientific=F)} \cr Other Pacific Islander & \Sexpr{prettyNum(ethnicE[12,2], big.mark=",", scientific=F)} \cr Native Hawaiian & \Sexpr{prettyNum(ethnicE[17,2], big.mark=",", scientific=F)} \cr Alaskan Native & \Sexpr{prettyNum(ethnicE[15,2], big.mark=",", scientific=F)} \cr Other & \Sexpr{prettyNum(ethnicE[14,2], big.mark=",", scientific=F)} \cr \hline \end{tabular} \end{center} \label{tab:ethnicE} \end{table} \end{scriptsize} \begin{scriptsize} \begin{table}[htdp]\caption{Educational Attainment Level in the Exploratory Sample} \begin{center} \begin{tabular} {l r r } \hline Educational Level & Participants \cr \hline Less than 12 years & \Sexpr{prettyNum(educationE[1,2], big.mark=",", scientific=F)} \cr High school graduate & \Sexpr{prettyNum(educationE[2,2], big.mark=",", scientific=F)} \cr Currently in college/university & \Sexpr{prettyNum(educationE[3,2], big.mark=",", scientific=F)} \cr Some college/university, but did not graduate & \Sexpr{prettyNum(educationE[4,2], big.mark=",", scientific=F)} \cr College/university degree & \Sexpr{prettyNum(educationE[5,2], big.mark=",", scientific=F)} \cr Currently in graduate or professional school & \Sexpr{prettyNum(educationE[6,2], big.mark=",", scientific=F)} \cr Graduate or professional school degree & \Sexpr{prettyNum(educationE[7,2], big.mark=",", scientific=F)} \cr \hline \end{tabular} \end{center} \label{tab:educationE} \end{table} \end{scriptsize} \begin{scriptsize} \begin{table}[htdp]\caption{Employment Status in the Exploratory Sample} \begin{center} \begin{tabular} {l r r } \hline Employment Status & Participants \cr \hline Employed & \Sexpr{prettyNum(jobstatusE[5,2], big.mark=",", scientific=F)} \cr Currently a student & \Sexpr{prettyNum(jobstatusE[1,2], big.mark=",", scientific=F)} \cr Not employed & \Sexpr{prettyNum(jobstatusE[2,2], big.mark=",", scientific=F)} \cr Not employed, seeking work & \Sexpr{prettyNum(jobstatusE[3,2], big.mark=",", scientific=F)} \cr Homemaker & \Sexpr{prettyNum(jobstatusE[4,2], big.mark=",", scientific=F)} \cr Retired & \Sexpr{prettyNum(jobstatusE[6,2], big.mark=",", scientific=F)} \cr \hline \end{tabular} \end{center} \label{tab:jobE} \end{table} \end{scriptsize} In the next code chunk, I create some convenient objects for later analyses and then estimate the missingness. Note that the variance (\textit{sd}) in the pairwise administration of items is high. This is caused by considerable skew in the number of responses given per participant, as can be noted by the large difference between the median and mean values. Participants (in all three samples) were strongly encouraged to complete at least 84 items. Most skipped a couple, some completed many more (up to a maximum of 311). \begin{scriptsize} <>= # Make some convenient objects # Isolate the names of the demographic variables only DemoItems <- colnames(expSample[,c((which(colnames(expSample)=="gender")): (which(colnames(expSample)=="jobstatus")))]) # Isolate the names of the data variables only. DataItems <- colnames(expSample[,c(((which(colnames(expSample)=="jobstatus"))+1): (ncol(expSample)))]) # Get values for estimating the missingness ResponseCountsByParticipantAllItemsE <- apply(expSample, 1, function(x) sum(!is.na(x))) PercResponseCountsByParticipantAllItemsE <- ResponseCountsByParticipantAllItemsE/ncol(expSample) MeanPercResponseCountsByParticipantAllItemsE <- round(mean(PercResponseCountsByParticipantAllItemsE*100), 1) # What percentage of the 696 items were responded to by each participant, on average? MeanPercResponseCountsByParticipantAllItemsE MedianPercResponseCountsByParticipantAllItemsE <- round(median(PercResponseCountsByParticipantAllItemsE*100), 1) # What percentage of the 696 items were responded to by each participant, median? MedianPercResponseCountsByParticipantAllItemsE SDPercResponseCountsByParticipantAllItemsE <- round(sd(PercResponseCountsByParticipantAllItemsE*100), 1) # And the SD SDPercResponseCountsByParticipantAllItemsE ResponseCountsByParticipantTempItemsE <- apply(expSample[,DataItems], 1, function(x) sum(!is.na(x))) MeanResponseCountsByParticipantTempItemsE <- round(mean(ResponseCountsByParticipantTempItemsE), 1) MeanResponseCountsByParticipantTempItemsE <- prettyNum(MeanResponseCountsByParticipantTempItemsE, big.mark=",", scientific=F) # The mean number of items responded to by each participant MeanResponseCountsByParticipantTempItemsE MedianResponseCountsByParticipantTempItemsE <- round(median(ResponseCountsByParticipantTempItemsE), 1) MedianResponseCountsByParticipantTempItemsE <- prettyNum(MedianResponseCountsByParticipantTempItemsE, big.mark=",", scientific=F) # The median number of items responded to by each participant MedianResponseCountsByParticipantTempItemsE SDResponseCountsByParticipantTempItemsE <- round(sd(ResponseCountsByParticipantTempItemsE), 1) SDResponseCountsByParticipantTempItemsE <- prettyNum(SDResponseCountsByParticipantTempItemsE, big.mark=",", scientific=F) # The SD of the number of items responded to by each participant SDResponseCountsByParticipantTempItemsE AdminsMeanE <- mean(sapply(expSample[,DataItems], function(y) sum(length(which(!is.na(y)))))) AdminsMeanE <- prettyNum(round(AdminsMeanE, 0), big.mark=",", scientific=F) # The mean number of administrations per item AdminsMeanE AdminsMedianE <- median(sapply(expSample[,DataItems], function(y) sum(length(which(!is.na(y)))))) AdminsMedianE <- prettyNum(round(AdminsMedianE, 0), big.mark=",", scientific=F) # The median number of administrations per item AdminsMedianE AdminsSDE <- sd(sapply(expSample[,DataItems], function(y) sum(length(which(!is.na(y)))))) AdminsSDE <- prettyNum(round(AdminsSDE,0), big.mark=",", scientific=F) # The SD of the number of administrations per item AdminsSDE pwiseAdmins1E <- pairwiseDescribe(expSample[,DataItems]) pwiseAdminsE <- prettyNum(pwiseAdmins1E, big.mark=",", scientific=F) pwiseAdminsMeanE <- round(pwiseAdmins1E$mean, 0) pwiseAdminsMeanE <- prettyNum(pwiseAdminsMeanE, big.mark=",", scientific=F) # The mean number of pairwise administrations for every possible pairing of items pwiseAdminsMeanE pwiseAdminsMedianE <- round(pwiseAdmins1E$median, 0) pwiseAdminsMedianE <- prettyNum(pwiseAdminsMedianE, big.mark=",", scientific=F) # The median number of pairwise administrations for every possible pairing of items pwiseAdminsMedianE pwiseAdminsSDE <- round(pwiseAdmins1E$sd, 0) pwiseAdminsSDE <- prettyNum(pwiseAdminsSDE, big.mark=",", scientific=F) # The SD of the number of pairwise administrations for every possible pairing of items pwiseAdminsSDE @ \end{scriptsize} \subsection{Description of the Replication sample} \label{sec:repDesc} Here, I conduct the same descriptive analyses for the \textbf{Replication} sample as those shown for the Exploratory sample. The Replication sample was collected between July 26, 2014 and December 22, 2015. Note that the statistical code for describing missingness is not shown in the PDF version --- the information is presented Table \ref{tab:missingness}. \begin{scriptsize} <>= load(paste(filepathdata, "sapaTempData696items26jul2014thru22dec2015.RData", sep = "")) repSample <- subset(sapaTempData696items26jul2014thru22dec2015, select = -c(RID, occPrestige, occIncomeEst, p1edu, p1occPrestige, p1occIncomeEst, p2edu, p2occPrestige, p2occIncomeEst)) # nR is total Ps in the replication sample nR <- dim(repSample)[1] nRtext <- prettyNum(nR, big.mark=",", scientific=F) nRtext # percent female femalePercR <- 100 * round((table(repSample$gender)/sum(table(repSample$gender)))["female"], 3) femalePercR # percent U.S. USpercR <- 100 * round((table(repSample$country)/sum(table(repSample$country)))["USA"], 3) USpercR # total U.S. UScountRtext <- prettyNum((table(repSample$country))["USA"], big.mark=",", scientific=F) UScountRtext # number of countries with P>0 countryTable <- sort(table(repSample$country),decreasing=TRUE) countryCountR <- length(countryTable) countryCountR # number of countries with P>49 countryCount50R <- subset(countryTable, countryTable > 49) length(countryCount50R) @ <>= repSampleAge <- describe(repSample[,"age"]) repSampleAge MeanAgeR <- repSampleAge[3] SDAgeR <- repSampleAge[4] MedianAgeR <- repSampleAge[5] MinAgeR <- repSampleAge[8] MaxAgeR <- repSampleAge[9] marstatusR <- data.frame(table(repSample$marstatus)) colnames(marstatusR) <- c("Marital Status", "Participants") ethnicR <- data.frame(table(repSample$ethnic)) colnames(ethnicR) <- c("Ethnicity/Race", "Participants") educationR <- data.frame(table(repSample$education)) colnames(educationR) <- c("Educational Level", "Participants") jobstatusR <- data.frame(table(repSample$jobstatus)) colnames(jobstatusR) <- c("Job Status", "Participants") # plot age by gender pdf(paste(filepathfig, "ageGenderR.pdf", sep = ""), onefile = TRUE) with(repSample, bi.bars(as.numeric(repSample$age), as.numeric(repSample$gender), ylab="Age", main = "")) dev.off() @ \end{scriptsize} \begin{figure} \ffigbox{\includegraphics[width=.75\textwidth]{figures/ageGenderR.pdf}}{\caption{Age by Males (Blue) and Females (Red) in the Replication Sample}\label{fig:ageGenderR}} \end{figure} \begin{scriptsize} \begin{table}[htdp]\caption{Marital Status in the Replication Sample} \begin{center} \begin{tabular} {l r r } \hline Marital Status & Participants \cr \hline Never Married & \Sexpr{prettyNum(marstatusR[1,2], big.mark=",", scientific=F)} \cr Married & \Sexpr{prettyNum(marstatusR[2,2], big.mark=",", scientific=F)} \cr Domestic Partnership & \Sexpr{prettyNum(marstatusR[3,2], big.mark=",", scientific=F)} \cr Widowed \& Remarried & \Sexpr{prettyNum(marstatusR[7,2], big.mark=",", scientific=F)} \cr Divorced \& Single & \Sexpr{prettyNum(marstatusR[4,2], big.mark=",", scientific=F)} \cr Divorced \& Remarried & \Sexpr{prettyNum(marstatusR[5,2], big.mark=",", scientific=F)} \cr Widowed \& Single & \Sexpr{prettyNum(marstatusR[6,2], big.mark=",", scientific=F)} \cr \hline \end{tabular} \end{center} \label{tab:marstatusR} \end{table} \end{scriptsize} \begin{scriptsize} \begin{table}[htdp]\caption{Race/Ethnicity Among U.S. Participants in the Replication Sample} \begin{center} \begin{tabular} {l r r } \hline Ethnicity & Participants \cr \hline White & \Sexpr{prettyNum(ethnicR[13,2], big.mark=",", scientific=F)} \cr African American & \Sexpr{prettyNum(ethnicR[1,2], big.mark=",", scientific=F)} \cr Two Or More Ethnicities & \Sexpr{prettyNum(ethnicR[18,2], big.mark=",", scientific=F)} \cr Mexican & \Sexpr{prettyNum(ethnicR[9,2], big.mark=",", scientific=F)} \cr Other Hispanic & \Sexpr{prettyNum(ethnicR[8,2], big.mark=",", scientific=F)} \cr Puerto Rican & \Sexpr{prettyNum(ethnicR[10,2], big.mark=",", scientific=F)} \cr Chinese & \Sexpr{prettyNum(ethnicR[2,2], big.mark=",", scientific=F)} \cr Other Asian & \Sexpr{prettyNum(ethnicR[7,2], big.mark=",", scientific=F)} \cr Filipino & \Sexpr{prettyNum(ethnicR[5,2], big.mark=",", scientific=F)} \cr Native American & \Sexpr{prettyNum(ethnicR[11,2], big.mark=",", scientific=F)} \cr Indian & \Sexpr{prettyNum(ethnicR[6,2], big.mark=",", scientific=F)} \cr Korean & \Sexpr{prettyNum(ethnicR[4,2], big.mark=",", scientific=F)} \cr Cuban & \Sexpr{prettyNum(ethnicR[16,2], big.mark=",", scientific=F)} \cr Japanese & \Sexpr{prettyNum(ethnicR[3,2], big.mark=",", scientific=F)} \cr Other Pacific Islander & \Sexpr{prettyNum(ethnicR[12,2], big.mark=",", scientific=F)} \cr Alaskan Native & \Sexpr{prettyNum(ethnicR[15,2], big.mark=",", scientific=F)} \cr Native Hawaiian & \Sexpr{prettyNum(ethnicR[17,2], big.mark=",", scientific=F)} \cr Other & \Sexpr{prettyNum(ethnicR[14,2], big.mark=",", scientific=F)} \cr \hline \end{tabular} \end{center} \label{tab:ethnicR} \end{table} \end{scriptsize} \begin{scriptsize} \begin{table}[htdp]\caption{Educational Attainment Level in the Replication Sample} \begin{center} \begin{tabular} {l r r } \hline Educational Level & Participants \cr \hline Less than 12 years & \Sexpr{prettyNum(educationR[1,2], big.mark=",", scientific=F)} \cr High school graduate & \Sexpr{prettyNum(educationR[2,2], big.mark=",", scientific=F)} \cr Currently in college/university & \Sexpr{prettyNum(educationR[3,2], big.mark=",", scientific=F)} \cr Some college/university, but did not graduate & \Sexpr{prettyNum(educationR[4,2], big.mark=",", scientific=F)} \cr College/university degree & \Sexpr{prettyNum(educationR[5,2], big.mark=",", scientific=F)} \cr Currently in graduate or professional school & \Sexpr{prettyNum(educationR[6,2], big.mark=",", scientific=F)} \cr Graduate or professional school degree & \Sexpr{prettyNum(educationR[7,2], big.mark=",", scientific=F)} \cr \hline \end{tabular} \end{center} \label{tab:educationR} \end{table} \end{scriptsize} \begin{scriptsize} \begin{table}[htdp]\caption{Employment Status in the Replication Sample} \begin{center} \begin{tabular} {l r r } \hline Employment Status & Participants \cr \hline Employed & \Sexpr{prettyNum(jobstatusR[5,2], big.mark=",", scientific=F)} \cr Currently a student & \Sexpr{prettyNum(jobstatusR[1,2], big.mark=",", scientific=F)} \cr Not employed & \Sexpr{prettyNum(jobstatusR[2,2], big.mark=",", scientific=F)} \cr Not employed, seeking work & \Sexpr{prettyNum(jobstatusR[3,2], big.mark=",", scientific=F)} \cr Homemaker & \Sexpr{prettyNum(jobstatusR[4,2], big.mark=",", scientific=F)} \cr Retired & \Sexpr{prettyNum(jobstatusR[6,2], big.mark=",", scientific=F)} \cr \hline \end{tabular} \end{center} \label{tab:jobR} \end{table} \end{scriptsize} \begin{scriptsize} <>= # Get values for estimating the missingness ResponseCountsByParticipantAllItemsR <- apply(repSample, 1, function(x) sum(!is.na(x))) PercResponseCountsByParticipantAllItemsR <- ResponseCountsByParticipantAllItemsR/ncol(repSample) MeanPercResponseCountsByParticipantAllItemsR <- round(mean(PercResponseCountsByParticipantAllItemsR*100), 1) # What percentage of the 696 items were responded to by each participant, on average? MeanPercResponseCountsByParticipantAllItemsR MedianPercResponseCountsByParticipantAllItemsR <- round(median(PercResponseCountsByParticipantAllItemsR*100), 1) # What percentage of the 696 items were responded to by each participant, median? MedianPercResponseCountsByParticipantAllItemsR SDPercResponseCountsByParticipantAllItemsR <- round(sd(PercResponseCountsByParticipantAllItemsR*100), 1) # And the SD SDPercResponseCountsByParticipantAllItemsR ResponseCountsByParticipantTempItemsR <- apply(repSample[,DataItems], 1, function(x) sum(!is.na(x))) MeanResponseCountsByParticipantTempItemsR <- round(mean(ResponseCountsByParticipantTempItemsR), 1) MeanResponseCountsByParticipantTempItemsR <- prettyNum(MeanResponseCountsByParticipantTempItemsR, big.mark=",", scientific=F) # The mean number of items responded to by each participant MeanResponseCountsByParticipantTempItemsR MedianResponseCountsByParticipantTempItemsR <- round(median(ResponseCountsByParticipantTempItemsR), 1) MedianResponseCountsByParticipantTempItemsR <- prettyNum(MedianResponseCountsByParticipantTempItemsR, big.mark=",", scientific=F) # The median number of items responded to by each participant MedianResponseCountsByParticipantTempItemsR SDResponseCountsByParticipantTempItemsR <- round(sd(ResponseCountsByParticipantTempItemsR), 1) SDResponseCountsByParticipantTempItemsR <- prettyNum(SDResponseCountsByParticipantTempItemsR, big.mark=",", scientific=F) # The SD of the number of items responded to by each participant SDResponseCountsByParticipantTempItemsR AdminsMeanR <- mean(sapply(repSample[,DataItems], function(y) sum(length(which(!is.na(y)))))) AdminsMeanR <- prettyNum(round(AdminsMeanR, 0), big.mark=",", scientific=F) # The mean number of administrations per item AdminsMeanR AdminsMedianR <- median(sapply(repSample[,DataItems], function(y) sum(length(which(!is.na(y)))))) AdminsMedianR <- prettyNum(round(AdminsMedianR, 0), big.mark=",", scientific=F) # The median number of administrations per item AdminsMedianR AdminsSDR <- sd(sapply(repSample[,DataItems], function(y) sum(length(which(!is.na(y)))))) AdminsSDR <- prettyNum(round(AdminsSDR,1), big.mark=",", scientific=F) # The SD of the number of administrations per item AdminsSDR pwiseAdmins1R <- pairwiseDescribe(repSample[,DataItems]) pwiseAdminsR <- prettyNum(pwiseAdmins1R, big.mark=",", scientific=F) pwiseAdminsMeanR <- round(pwiseAdmins1R$mean, 0) pwiseAdminsMeanR <- prettyNum(pwiseAdminsMeanR, big.mark=",", scientific=F) # The mean number of pairwise administrations for every possible pairing of items pwiseAdminsMeanR pwiseAdminsMedianR <- round(pwiseAdmins1R$median, 0) pwiseAdminsMedianR <- prettyNum(pwiseAdminsMedianR, big.mark=",", scientific=F) # The median number of pairwise administrations for every possible pairing of items pwiseAdminsMedianR pwiseAdminsSDR <- round(pwiseAdmins1R$sd, 0) pwiseAdminsSDR <- prettyNum(pwiseAdminsSDR, big.mark=",", scientific=F) # The SD of the number of pairwise administrations for every possible pairing of items pwiseAdminsSDR @ \end{scriptsize} \subsection{Description of the Confirmatory sample} \label{sec:confDesc} Here, I provide the same descriptive analyses for the \textbf{Confirmatory} sample as those shown for the Exploratory and Replication samples. The Confirmatory sample was collected between December 22, 2015 and February 7, 2017. \begin{scriptsize} <>= load(paste(filepathdata, "sapaTempData696items22dec2015thru07feb2017.RData", sep = "")) confSample <- subset(sapaTempData696items22dec2015thru07feb2017, select = -c(RID, occPrestige, occIncomeEst, p1edu, p1occPrestige, p1occIncomeEst, p2edu, p2occPrestige, p2occIncomeEst)) # nC is total Ps in the confirmatory sample nC <- dim(confSample)[1] nCtext <- prettyNum(nC, big.mark=",", scientific=F) nCtext # percent female femalePercC <- 100 * round((table(confSample$gender)/sum(table(confSample$gender)))["female"], 3) femalePercC # percent U.S. USpercC <- 100 * round((table(confSample$country)/sum(table(confSample$country)))["USA"], 3) USpercC # total U.S. UScountCtext <- prettyNum((table(confSample$country))["USA"], big.mark=",", scientific=F) UScountCtext # number of countries with P>0 countryTable <- sort(table(confSample$country),decreasing=TRUE) countryCountC <- length(countryTable) countryCountC # number of countries with P>49 countryCount50C <- subset(countryTable, countryTable > 49) length(countryCount50C) @ <>= confSampleAge <- describe(confSample[,"age"]) confSampleAge MeanAgeC <- confSampleAge[3] SDAgeC <- confSampleAge[4] MedianAgeC <- confSampleAge[5] MinAgeC <- confSampleAge[8] MaxAgeC <- confSampleAge[9] marstatusC <- data.frame(table(confSample$marstatus)) colnames(marstatusC) <- c("Marital Status", "Participants") ethnicC <- data.frame(table(confSample$ethnic)) colnames(ethnicC) <- c("Ethnicity/Race", "Participants") educationC <- data.frame(table(confSample$education)) colnames(educationC) <- c("Educational Level", "Participants") jobstatusC <- data.frame(table(confSample$jobstatus)) colnames(jobstatusC) <- c("Job Status", "Participants") # plot age by gender pdf(paste(filepathfig, "ageGenderC.pdf", sep = ""), onefile = TRUE) with(confSample, bi.bars(as.numeric(confSample$age), as.numeric(confSample$gender), ylab="Age", main = "")) dev.off() @ \end{scriptsize} \begin{figure} \ffigbox{\includegraphics[width=.75\textwidth]{figures/ageGenderC.pdf}}{\caption{Age by Males (Blue) and Females (Red) in the Confirmatory Sample}\label{fig:ageGenderC}} \end{figure} \begin{scriptsize} \begin{table}[htdp]\caption{Marital Status in the Confirmatory Sample} \begin{center} \begin{tabular} {l r r } \hline Marital Status & Participants \cr \hline Never Married & \Sexpr{prettyNum(marstatusC[1,2], big.mark=",", scientific=F)} \cr Married & \Sexpr{prettyNum(marstatusC[2,2], big.mark=",", scientific=F)} \cr Domestic Partnership & \Sexpr{prettyNum(marstatusC[3,2], big.mark=",", scientific=F)} \cr Widowed \& Remarried & \Sexpr{prettyNum(marstatusC[7,2], big.mark=",", scientific=F)} \cr Divorced \& Single & \Sexpr{prettyNum(marstatusC[4,2], big.mark=",", scientific=F)} \cr Divorced \& Remarried & \Sexpr{prettyNum(marstatusC[5,2], big.mark=",", scientific=F)} \cr Widowed \& Single & \Sexpr{prettyNum(marstatusC[6,2], big.mark=",", scientific=F)} \cr \hline \end{tabular} \end{center} \label{tab:marstatusC} \end{table} \end{scriptsize} \begin{scriptsize} \begin{table}[htdp]\caption{Race/Ethnicity Among U.S. Participants in the Confirmatory Sample} \begin{center} \begin{tabular} {l r r } \hline Ethnicity & Participants \cr \hline White & \Sexpr{prettyNum(ethnicC[13,2], big.mark=",", scientific=F)} \cr African American & \Sexpr{prettyNum(ethnicC[1,2], big.mark=",", scientific=F)} \cr Two Or More Ethnicities & \Sexpr{prettyNum(ethnicC[18,2], big.mark=",", scientific=F)} \cr Mexican & \Sexpr{prettyNum(ethnicC[9,2], big.mark=",", scientific=F)} \cr Other Hispanic & \Sexpr{prettyNum(ethnicC[8,2], big.mark=",", scientific=F)} \cr Puerto Rican & \Sexpr{prettyNum(ethnicC[10,2], big.mark=",", scientific=F)} \cr Chinese & \Sexpr{prettyNum(ethnicC[2,2], big.mark=",", scientific=F)} \cr Other Asian & \Sexpr{prettyNum(ethnicC[7,2], big.mark=",", scientific=F)} \cr Filipino & \Sexpr{prettyNum(ethnicC[5,2], big.mark=",", scientific=F)} \cr Native American & \Sexpr{prettyNum(ethnicC[11,2], big.mark=",", scientific=F)} \cr Indian & \Sexpr{prettyNum(ethnicC[6,2], big.mark=",", scientific=F)} \cr Korean & \Sexpr{prettyNum(ethnicC[4,2], big.mark=",", scientific=F)} \cr Cuban & \Sexpr{prettyNum(ethnicC[16,2], big.mark=",", scientific=F)} \cr Japanese & \Sexpr{prettyNum(ethnicC[3,2], big.mark=",", scientific=F)} \cr Other Pacific Islander & \Sexpr{prettyNum(ethnicC[12,2], big.mark=",", scientific=F)} \cr Alaskan Native & \Sexpr{prettyNum(ethnicC[15,2], big.mark=",", scientific=F)} \cr Native Hawaiian & \Sexpr{prettyNum(ethnicC[17,2], big.mark=",", scientific=F)} \cr Other & \Sexpr{prettyNum(ethnicC[14,2], big.mark=",", scientific=F)} \cr \hline \end{tabular} \end{center} \label{tab:ethnicC} \end{table} \end{scriptsize} \begin{scriptsize} \begin{table}[htdp]\caption{Educational Attainment Level in the Confirmatory Sample} \begin{center} \begin{tabular} {l r r } \hline Educational Level & Participants \cr \hline Less than 12 years & \Sexpr{prettyNum(educationC[1,2], big.mark=",", scientific=F)} \cr High school graduate & \Sexpr{prettyNum(educationC[2,2], big.mark=",", scientific=F)} \cr Currently in college/university & \Sexpr{prettyNum(educationC[3,2], big.mark=",", scientific=F)} \cr Some college/university, but did not graduate & \Sexpr{prettyNum(educationC[4,2], big.mark=",", scientific=F)} \cr College/university degree & \Sexpr{prettyNum(educationC[5,2], big.mark=",", scientific=F)} \cr Currently in graduate or professional school & \Sexpr{prettyNum(educationC[6,2], big.mark=",", scientific=F)} \cr Graduate or professional school degree & \Sexpr{prettyNum(educationC[7,2], big.mark=",", scientific=F)} \cr \hline \end{tabular} \end{center} \label{tab:educationC} \end{table} \end{scriptsize} \begin{scriptsize} \begin{table}[htdp]\caption{Employment Status in the Confirmatory Sample} \begin{center} \begin{tabular} {l r r } \hline Employment Status & Participants \cr \hline Employed & \Sexpr{prettyNum(jobstatusC[5,2], big.mark=",", scientific=F)} \cr Currently a student & \Sexpr{prettyNum(jobstatusC[1,2], big.mark=",", scientific=F)} \cr Not employed & \Sexpr{prettyNum(jobstatusC[2,2], big.mark=",", scientific=F)} \cr Not employed, seeking work & \Sexpr{prettyNum(jobstatusC[3,2], big.mark=",", scientific=F)} \cr Homemaker & \Sexpr{prettyNum(jobstatusC[4,2], big.mark=",", scientific=F)} \cr Retired & \Sexpr{prettyNum(jobstatusC[6,2], big.mark=",", scientific=F)} \cr \hline \end{tabular} \end{center} \label{tab:jobC} \end{table} \end{scriptsize} \begin{scriptsize} <>= # Get values for estimating the missingness ResponseCountsByParticipantAllItemsC <- apply(confSample, 1, function(x) sum(!is.na(x))) PercResponseCountsByParticipantAllItemsC <- ResponseCountsByParticipantAllItemsC/ncol(confSample) MeanPercResponseCountsByParticipantAllItemsC <- round(mean(PercResponseCountsByParticipantAllItemsC*100), 1) # What percentage of the 696 items were responded to by each participant, on average? MeanPercResponseCountsByParticipantAllItemsC MedianPercResponseCountsByParticipantAllItemsC <- round(median(PercResponseCountsByParticipantAllItemsC*100), 1) # What percentage of the 696 items were responded to by each participant, median? MedianPercResponseCountsByParticipantAllItemsC SDPercResponseCountsByParticipantAllItemsC <- round(sd(PercResponseCountsByParticipantAllItemsC*100), 1) # And the SD SDPercResponseCountsByParticipantAllItemsC ResponseCountsByParticipantTempItemsC <- apply(confSample[,DataItems], 1, function(x) sum(!is.na(x))) MeanResponseCountsByParticipantTempItemsC <- round(mean(ResponseCountsByParticipantTempItemsC), 1) MeanResponseCountsByParticipantTempItemsC <- prettyNum(MeanResponseCountsByParticipantTempItemsC, big.mark=",", scientific=F) # The mean number of items responded to by each participant MeanResponseCountsByParticipantTempItemsC MedianResponseCountsByParticipantTempItemsC <- round(median(ResponseCountsByParticipantTempItemsC), 1) MedianResponseCountsByParticipantTempItemsC <- prettyNum(MedianResponseCountsByParticipantTempItemsC, big.mark=",", scientific=F) # The median number of items responded to by each participant MedianResponseCountsByParticipantTempItemsC SDResponseCountsByParticipantTempItemsC <- round(sd(ResponseCountsByParticipantTempItemsC), 1) SDResponseCountsByParticipantTempItemsC <- prettyNum(SDResponseCountsByParticipantTempItemsC, big.mark=",", scientific=F) # The SD of the number of items responded to by each participant SDResponseCountsByParticipantTempItemsC AdminsMeanC <- mean(sapply(confSample[,DataItems], function(y) sum(length(which(!is.na(y)))))) AdminsMeanC <- prettyNum(round(AdminsMeanC, 0), big.mark=",", scientific=F) # The mean number of administrations per item AdminsMeanC AdminsMedianC <- median(sapply(confSample[,DataItems], function(y) sum(length(which(!is.na(y)))))) AdminsMedianC <- prettyNum(round(AdminsMedianC, 0), big.mark=",", scientific=F) # The median number of administrations per item AdminsMedianC AdminsSDC <- sd(sapply(confSample[,DataItems], function(y) sum(length(which(!is.na(y)))))) AdminsSDC <- prettyNum(round(AdminsSDC,1), big.mark=",", scientific=F) # The SD of the number of administrations per item AdminsSDC pwiseAdmins1C <- pairwiseDescribe(confSample[,DataItems]) pwiseAdminsC <- prettyNum(pwiseAdmins1C, big.mark=",", scientific=F) pwiseAdminsMeanC <- round(pwiseAdmins1C$mean, 0) pwiseAdminsMeanC <- prettyNum(pwiseAdminsMeanC, big.mark=",", scientific=F) # The mean number of pairwise administrations for every possible pairing of items pwiseAdminsMeanC pwiseAdminsMedianC <- round(pwiseAdmins1C$median, 0) pwiseAdminsMedianC <- prettyNum(pwiseAdminsMedianC, big.mark=",", scientific=F) # The median number of pairwise administrations for every possible pairing of items pwiseAdminsMedianC pwiseAdminsSDC <- round(pwiseAdmins1C$sd, 0) pwiseAdminsSDC <- prettyNum(pwiseAdminsSDC, big.mark=",", scientific=F) # The SD of the number of pairwise administrations for every possible pairing of items pwiseAdminsSDC @ \end{scriptsize} \begin{scriptsize} <>= save(list=ls(), file=paste(filepathdata, "descriptivesData.Rdata", sep = "")) @ \end{scriptsize} \begin{scriptsize} \begin{table}[htdp]\caption{Sample sizes and densities} \begin{center} \begin{tabular} {l r r r} \hline & \multicolumn{3}{c}{Sample} \cr & Exploratory & Replication & Confirmatory \cr \hline Total number of respondents & \Sexpr{nEtext} & \Sexpr{nRtext} & \Sexpr{nCtext} \cr & & & \cr Mean percent of responses to all items & \Sexpr{MeanPercResponseCountsByParticipantAllItemsE} & \Sexpr{MeanPercResponseCountsByParticipantAllItemsR} & \Sexpr{MeanPercResponseCountsByParticipantAllItemsC} \cr Median percent of responses to all items & \Sexpr{MedianPercResponseCountsByParticipantAllItemsE} & \Sexpr{MedianPercResponseCountsByParticipantAllItemsR} & \Sexpr{MedianPercResponseCountsByParticipantAllItemsC} \cr Standard deviation percent of responses to all items & \Sexpr{SDPercResponseCountsByParticipantAllItemsE} & \Sexpr{SDPercResponseCountsByParticipantAllItemsR} & \Sexpr{SDPercResponseCountsByParticipantAllItemsC} \cr & & & \cr Mean number of responses per participant & \Sexpr{MeanResponseCountsByParticipantTempItemsE} & \Sexpr{MeanResponseCountsByParticipantTempItemsR} & \Sexpr{MeanResponseCountsByParticipantTempItemsC}\cr Median number of responses per participant & \Sexpr{MedianResponseCountsByParticipantTempItemsE} & \Sexpr{MedianResponseCountsByParticipantTempItemsR} & \Sexpr{MedianResponseCountsByParticipantTempItemsC} \cr Standard deviation number of responses per participant & \Sexpr{SDResponseCountsByParticipantTempItemsE} & \Sexpr{SDResponseCountsByParticipantTempItemsR} & \Sexpr{SDResponseCountsByParticipantTempItemsC} \cr & & & \cr Mean number of responses per item & \Sexpr{AdminsMeanE} & \Sexpr{AdminsMeanR} & \Sexpr{AdminsMeanC} \cr Median number of responses per item & \Sexpr{AdminsMedianE} & \Sexpr{AdminsMedianR} & \Sexpr{AdminsMedianC} \cr Standard deviation number of responses per item & \Sexpr{AdminsSDE} & \Sexpr{AdminsSDR} & \Sexpr{AdminsSDC} \cr & & & \cr Mean pairwise responses for all possible pairings & \Sexpr{pwiseAdminsMeanE} & \Sexpr{pwiseAdminsMeanR} & \Sexpr{pwiseAdminsMeanC} \cr Median pairwise responses for all possible pairings & \Sexpr{pwiseAdminsMedianE} & \Sexpr{pwiseAdminsMedianR} & \Sexpr{pwiseAdminsMedianC} \cr Standard deviation pairwise responses for all possible pairings & \Sexpr{pwiseAdminsSDE} & \Sexpr{pwiseAdminsSDR} & \Sexpr{pwiseAdminsSDC} \cr \hline \end{tabular} \end{center} \label{tab:missingness} \end{table} \end{scriptsize} \subsection{Comparison of the SAPA samples to U.S. Census data} Before comparing the SAPA demographics to the U.S. Census Bureau data, I need to address differences in the way the demographics variables are coded. For race/ethnicity information \citep{censusRace}, marital status \citep{censusMarstatus} and employment status \citep{censusJobstatus}, the census data contain fewer categories than the SAPA data. In the following code chunk, I collapse the SAPA data to fewer categories for the relevant variables in order to allow comparisons across the data sets. For educational attainment level \citep{censusEdustatus}, it is only necessary to rename the SAPA categories to match the census. Note that I have only shown this step for the Exploratory sample as an example. \begin{scriptsize} <>= suppressPackageStartupMessages(library(tidyr)) suppressPackageStartupMessages(library(ggplot2)) suppressPackageStartupMessages(library(RColorBrewer)) # Collapse the exploratory sample for comparison to the US Census # ethnicity expSample$ethnicCensus <- mapvalues(expSample$ethnic, from = c("Chinese", "Japanese", "Korean", "Filipino", "Indian", "OtherAsian"), to = rep("Asian", 6)) expSample$ethnicCensus <- mapvalues(expSample$ethnicCensus, from = c("OtherHisp", "MexAmer", "Prican", "CubanAmer"), to = rep("Hispanic", 4)) expSample$ethnicCensus <- mapvalues(expSample$ethnicCensus, from = c("NativeAmer", "OtherPacIs", "AlaNative", "NativeHaw"), to = rep("Native", 4)) expSample$ethnicCensus <- mapvalues(expSample$ethnicCensus, from = c("Other", "TwoOrMore"), to = rep("TwoOrOther", 2)) # marital status expSample$marstatusCensus <- mapvalues(expSample$marstatus, from = c("WidowSingle", "WidowRemar"), to = rep("Widowed", 2)) expSample$marstatusCensus <- mapvalues(expSample$marstatusCensus, from = c("DivRemar", "DivSingle"), to = rep("Divorced", 2)) expSample$marstatusCensus <- mapvalues(expSample$marstatusCensus, from = c("neverMarried"), to = c("NeverMarried")) expSample$marstatusCensus <- mapvalues(expSample$marstatusCensus, from = c("married"), to = c("Married")) expSample$marstatusCensus <- mapvalues(expSample$marstatusCensus, from = c("domesticPrtnr", "Married"), to = rep("MarriedOrDomesticPrtnr", 2)) # educational attainment level expSample$educationCensus <- mapvalues(expSample$education, from = c("less12yrs"), to = c("LtHSG")) expSample$educationCensus <- mapvalues(expSample$educationCensus, from = c("HSgrad"), to = c("HSG")) expSample$educationCensus <- mapvalues(expSample$educationCensus, from = c("SomeCollege", "CurrentInUniv"), to = rep("SCorAssD", 2)) expSample$educationCensus <- mapvalues(expSample$educationCensus, from = c("CollegeDegree", "InGradOrProSchool"), to = rep("BD", 2)) expSample$educationCensus <- mapvalues(expSample$educationCensus, from = c("GradOrProDegree"), to = c("AdvD")) # employment status expSample$jobstatusCensus <- mapvalues(expSample$jobstatus, from = c("student", "Retired", "Homemaker", "notEmployed"), to = rep("NotCLF", 4)) expSample$jobstatusCensus <- mapvalues(expSample$jobstatusCensus, from = c("notEmployedSeekingWork"), to = c("UnemployedCLF")) expSample$jobstatusCensus <- mapvalues(expSample$jobstatusCensus, from = c("Employed"), to = c("EmployedCLF")) # subset SAPA data for US only expSampleUS <- subset(expSample, country == "USA") # Break down the sample by demo variable in age categories expSampleUS$agecatRace <- cut(expSampleUS$age, c(0, 5, 9, 14, 19, 24, 29, 34, 39, 44, 49, 54, 59, 64, 69, 74, 79, 84, 90), include.lowest=TRUE) expSampleUS$agecatEduAndMar <- cut(expSampleUS$age, c(15, 17, 24, 29, 34, 39, 44, 49, 54, 59, 64, 69, 74, 79, 84, 90), include.lowest=TRUE) expSampleUS$agecatEmploy <- cut(expSampleUS$age, c(16, 17, 24, 29, 34, 39, 44, 49, 54, 59, 64, 90), include.lowest=TRUE) # Crosstabs age and demo variables and include totals expEmployTableSAPA <- addmargins(table(expSampleUS$agecatEmploy, expSampleUS$jobstatusCensus), 1) expRaceTableSAPA <- addmargins(table(expSampleUS$agecatRace, expSampleUS$ethnicCensus), 1) expMarStatusTableSAPA <- addmargins(table(expSampleUS$agecatEduAndMar, expSampleUS$marstatusCensus), 1) expEducationTableSAPA <- addmargins(table(expSampleUS$agecatEduAndMar, expSampleUS$educationCensus), 1) @ \end{scriptsize} \begin{scriptsize} <>= # Collapse the replication sample for comparison to the US Census # ethnicity repSample$ethnicCensus <- mapvalues(repSample$ethnic, from = c("Chinese", "Japanese", "Korean", "Filipino", "Indian", "OtherAsian"), to = rep("Asian", 6)) repSample$ethnicCensus <- mapvalues(repSample$ethnicCensus, from = c("OtherHisp", "MexAmer", "Prican", "CubanAmer"), to = rep("Hispanic", 4)) repSample$ethnicCensus <- mapvalues(repSample$ethnicCensus, from = c("NativeAmer", "OtherPacIs", "AlaNative", "NativeHaw"), to = rep("Native", 4)) repSample$ethnicCensus <- mapvalues(repSample$ethnicCensus, from = c("Other", "TwoOrMore"), to = rep("TwoOrOther", 2)) # marital status repSample$marstatusCensus <- mapvalues(repSample$marstatus, from = c("WidowSingle", "WidowRemar"), to = rep("Widowed", 2)) repSample$marstatusCensus <- mapvalues(repSample$marstatusCensus, from = c("DivRemar", "DivSingle"), to = rep("Divorced", 2)) repSample$marstatusCensus <- mapvalues(repSample$marstatusCensus, from = c("neverMarried"), to = c("NeverMarried")) repSample$marstatusCensus <- mapvalues(repSample$marstatusCensus, from = c("married"), to = c("Married")) repSample$marstatusCensus <- mapvalues(repSample$marstatusCensus, from = c("domesticPrtnr", "Married"), to = rep("MarriedOrDomesticPrtnr", 2)) # educational attainment level repSample$educationCensus <- mapvalues(repSample$education, from = c("less12yrs"), to = c("LtHSG")) repSample$educationCensus <- mapvalues(repSample$educationCensus, from = c("HSgrad"), to = c("HSG")) repSample$educationCensus <- mapvalues(repSample$educationCensus, from = c("SomeCollege", "CurrentInUniv"), to = rep("SCorAssD", 2)) repSample$educationCensus <- mapvalues(repSample$educationCensus, from = c("CollegeDegree", "InGradOrProSchool"), to = rep("BD", 2)) repSample$educationCensus <- mapvalues(repSample$educationCensus, from = c("GradOrProDegree"), to = c("AdvD")) # employment status repSample$jobstatusCensus <- mapvalues(repSample$jobstatus, from = c("student", "Retired", "Homemaker", "notEmployed"), to = rep("NotCLF", 4)) repSample$jobstatusCensus <- mapvalues(repSample$jobstatusCensus, from = c("notEmployedSeekingWork"), to = c("UnemployedCLF")) repSample$jobstatusCensus <- mapvalues(repSample$jobstatusCensus, from = c("Employed"), to = c("EmployedCLF")) # subset SAPA data for US only repSampleUS <- subset(repSample, country == "USA") # Break down the sample by demo variable in age categories repSampleUS$agecatRace <- cut(repSampleUS$age, c(0, 5, 9, 14, 19, 24, 29, 34, 39, 44, 49, 54, 59, 64, 69, 74, 79, 84, 90), include.lowest=TRUE) repSampleUS$agecatEduAndMar <- cut(repSampleUS$age, c(15, 17, 24, 29, 34, 39, 44, 49, 54, 59, 64, 69, 74, 79, 84, 90), include.lowest=TRUE) repSampleUS$agecatEmploy <- cut(repSampleUS$age, c(16, 17, 24, 29, 34, 39, 44, 49, 54, 59, 64, 90), include.lowest=TRUE) # Crosstabs age and demo variables and include totals repEmployTableSAPA <- addmargins(table(repSampleUS$agecatEmploy, repSampleUS$jobstatusCensus), 1) repRaceTableSAPA <- addmargins(table(repSampleUS$agecatRace, repSampleUS$ethnicCensus), 1) repMarStatusTableSAPA <- addmargins(table(repSampleUS$agecatEduAndMar, repSampleUS$marstatusCensus), 1) repEducationTableSAPA <- addmargins(table(repSampleUS$agecatEduAndMar, repSampleUS$educationCensus), 1) # Collapse the confirmatory sample for comparison to the US Census # ethnicity confSample$ethnicCensus <- mapvalues(confSample$ethnic, from = c("Chinese", "Japanese", "Korean", "Filipino", "Indian", "OtherAsian"), to = rep("Asian", 6)) confSample$ethnicCensus <- mapvalues(confSample$ethnicCensus, from = c("OtherHisp", "MexAmer", "Prican", "CubanAmer"), to = rep("Hispanic", 4)) confSample$ethnicCensus <- mapvalues(confSample$ethnicCensus, from = c("NativeAmer", "OtherPacIs", "AlaNative", "NativeHaw"), to = rep("Native", 4)) confSample$ethnicCensus <- mapvalues(confSample$ethnicCensus, from = c("Other", "TwoOrMore"), to = rep("TwoOrOther", 2)) # marital status confSample$marstatusCensus <- mapvalues(confSample$marstatus, from = c("WidowSingle", "WidowRemar"), to = rep("Widowed", 2)) confSample$marstatusCensus <- mapvalues(confSample$marstatusCensus, from = c("DivRemar", "DivSingle"), to = rep("Divorced", 2)) confSample$marstatusCensus <- mapvalues(confSample$marstatusCensus, from = c("neverMarried"), to = c("NeverMarried")) confSample$marstatusCensus <- mapvalues(confSample$marstatusCensus, from = c("married"), to = c("Married")) confSample$marstatusCensus <- mapvalues(confSample$marstatusCensus, from = c("domesticPrtnr", "Married"), to = rep("MarriedOrDomesticPrtnr", 2)) # educational attainment level confSample$educationCensus <- mapvalues(confSample$education, from = c("less12yrs"), to = c("LtHSG")) confSample$educationCensus <- mapvalues(confSample$educationCensus, from = c("HSgrad"), to = c("HSG")) confSample$educationCensus <- mapvalues(confSample$educationCensus, from = c("SomeCollege", "CurrentInUniv"), to = rep("SCorAssD", 2)) confSample$educationCensus <- mapvalues(confSample$educationCensus, from = c("CollegeDegree", "InGradOrProSchool"), to = rep("BD", 2)) confSample$educationCensus <- mapvalues(confSample$educationCensus, from = c("GradOrProDegree"), to = c("AdvD")) # employment status confSample$jobstatusCensus <- mapvalues(confSample$jobstatus, from = c("student", "Retired", "Homemaker", "notEmployed"), to = rep("NotCLF", 4)) confSample$jobstatusCensus <- mapvalues(confSample$jobstatusCensus, from = c("notEmployedSeekingWork"), to = c("UnemployedCLF")) confSample$jobstatusCensus <- mapvalues(confSample$jobstatusCensus, from = c("Employed"), to = c("EmployedCLF")) # subset SAPA data for US only confSampleUS <- subset(confSample, country == "USA") # Break down the sample by demo variable in age categories confSampleUS$agecatRace <- cut(confSampleUS$age, c(0, 5, 9, 14, 19, 24, 29, 34, 39, 44, 49, 54, 59, 64, 69, 74, 79, 84, 90), include.lowest=TRUE) confSampleUS$agecatEduAndMar <- cut(confSampleUS$age, c(15, 17, 24, 29, 34, 39, 44, 49, 54, 59, 64, 69, 74, 79, 84, 90), include.lowest=TRUE) confSampleUS$agecatEmploy <- cut(confSampleUS$age, c(16, 17, 24, 29, 34, 39, 44, 49, 54, 59, 64, 90), include.lowest=TRUE) # Crosstabs age and demo variables and include totals confEmployTableSAPA <- addmargins(table(confSampleUS$agecatEmploy, confSampleUS$jobstatusCensus), 1) confRaceTableSAPA <- addmargins(table(confSampleUS$agecatRace, confSampleUS$ethnicCensus), 1) confMarStatusTableSAPA <- addmargins(table(confSampleUS$agecatEduAndMar, confSampleUS$marstatusCensus), 1) confEducationTableSAPA <- addmargins(table(confSampleUS$agecatEduAndMar, confSampleUS$educationCensus), 1) @ \end{scriptsize} Now it is possible to make tables to help evaluate the differences between the SAPA samples and the U.S. population. One challenge is that the census data breaks each of the variables down by age category in 2 to 5 year increments. I have made tables of the comparison for each category that retain all the info; these can be viewed in the rnw version of the file (see the objects named `employmenttablePCT', `RacetablePCT', `maritaltablePCT', and `educationtablePCT'). Figures \ref{fig:maritalstatusfig} to \ref{fig:jobstatusfig} collapse and reduce the age categories for the sake of visualization but more specific comparisons for these categories are possible. I show the code for employment status as an example in the PDF; see the rnw file for all the remaining variables. To fully reproduce these tables and figures, it is necessary to download the U.S. Census data directly from the relevant U.S. Census Bureau API (\urlstyle{same}\url{www.census.gov/developers/}).\footnote{I have provided pre-processed versions of the figures in the Supplementary Materials file at \url{http://dx.doi.org/10.7910/DVN/YF6STI}.} \begin{scriptsize} <>= # keep eval=FALSE # load the census data load(paste(filepathdata, "Census2012tables.Rdata", sep = "")) # Join the Census and SAPA data employmenttable <- data.frame(as.numeric(as.matrix(EmploymentBoth[,c("NotCLF")])), as.numeric(as.matrix(EmploymentBoth[,"UnemployedCLF"])), as.numeric(as.matrix(EmploymentBoth[,c("EmployedCLF")])), as.data.frame.matrix(expEmployTableSAPA), as.data.frame.matrix(repEmployTableSAPA), as.data.frame.matrix(confEmployTableSAPA)) rownames(employmenttable) <- rownames(EmploymentBothPCT) colnames(employmenttable) <- c("CensusnotCLF", "CensusunempCLF", "CensusempCLF", "SAPAexpnotCLF", "SAPAexpunempCLF", "SAPAexpempCLF", "SAPArepnotCLF", "SAPArepunempCLF", "SAPArepempCLF", "SAPAconfnotCLF", "SAPAconfunempCLF", "SAPAconfempCLF") # Percentages of crosstab tables with Census and SAPA data employmenttablePCT <- cbind(round(prop.table(as.matrix(employmenttable[,1:3]), 1), 3), round(prop.table(as.matrix(employmenttable[,4:6]), 1), 3)) @ \end{scriptsize} \begin{scriptsize} <>= # keep eval=FALSE # Do the same conversions for race, mar status and education racetable <- data.frame(as.numeric(as.matrix(RaceBoth[,c("AfrAmer")])), as.numeric(as.matrix(RaceBoth[,c("Asian")])), as.numeric(as.matrix(RaceBoth[,"Hispanic"])), as.numeric(as.matrix(RaceBoth[,c("Native")])), as.numeric(as.matrix(RaceBoth[,c("White")])), as.numeric(as.matrix(RaceBoth[,c("TwoMore")])), as.data.frame.matrix(expRaceTableSAPA), as.data.frame.matrix(repRaceTableSAPA), as.data.frame.matrix(confRaceTableSAPA)) rownames(racetable) <- rownames(RaceBothPCT) colnames(racetable) <- c("censusAfrAmer", "censusAsian", "censusHispanic", "censusNative", "censusWhite", "censusTwoOther", "SAPAexpAfrAmer", "SAPAexpAsian", "SAPAexpHispanic", "SAPAexpNative", "SAPAexpWhite", "SAPAexpTwoOther", "SAPArepAfrAmer", "SAPArepAsian", "SAPArepHispanic", "SAPArepNative", "SAPArepWhite", "SAPArepTwoOther", "SAPAconfAfrAmer", "SAPAconfAsian", "SAPAconfHispanic", "SAPAconfNative", "SAPAconfWhite", "SAPAconfTwoOther") maritaltable <- data.frame(as.numeric(as.matrix(MaritalBoth[,c("NeverMarried")])), as.numeric(as.matrix(MaritalBoth[,c("Married")])), as.numeric(as.matrix(MaritalBoth[,c("DivSep")])), as.numeric(as.matrix(MaritalBoth[,c("Widowed")])), as.data.frame.matrix(expMarStatusTableSAPA), as.data.frame.matrix(repMarStatusTableSAPA), as.data.frame.matrix(confMarStatusTableSAPA)) rownames(maritaltable) <- rownames(MaritalBothPCT) colnames(maritaltable) <- c("censusNvrMarried", "censusMarried", "censusDivSep", "censusWidowed", "SAPAexpNvrMarried", "SAPAexpMarried", "SAPAexpDivSep", "SAPAexpWidowed", "SAPArepNvrMarried", "SAPArepMarried", "SAPArepDivSep", "SAPArepWidowed", "SAPAconfNvrMarried", "SAPAconfMarried", "SAPAconfDivSep", "SAPAconfWidowed") educationtable <- data.frame(as.numeric(as.matrix(EducationBoth[,c("LtHSG")])), as.numeric(as.matrix(EducationBoth[,c("HSG")])), as.numeric(as.matrix(EducationBoth[,c("SCorAssD")])), as.numeric(as.matrix(EducationBoth[,c("BD")])), as.numeric(as.matrix(EducationBoth[,c("AdvD")])), as.data.frame.matrix(expEducationTableSAPA), as.data.frame.matrix(repEducationTableSAPA), as.data.frame.matrix(confEducationTableSAPA)) rownames(educationtable) <- rownames(EducationBothPCT) colnames(educationtable) <- c("censusLtHSG", "censusHSG", "censusSCorAssD", "censusBD", "censusAdvD", "SAPAexpLtHSG", "SAPAexpHSG", "SAPAexpSCorAssD", "SAPAexpBD", "SAPAexpAdvD", "SAPArepLtHSG", "SAPArepHSG", "SAPArepSCorAssD", "SAPArepBD", "SAPArepAdvD", "SAPAconfLtHSG", "SAPAconfHSG", "SAPAconfSCorAssD", "SAPAconfBD", "SAPAconfAdvD") racetablePCT <- cbind(round(prop.table(as.matrix(racetable[,1:6]), 1), 3), round(prop.table(as.matrix(racetable[,7:12]), 1), 3)) maritaltablePCT <- cbind(round(prop.table(as.matrix(maritaltable[,1:4]), 1), 3), round(prop.table(as.matrix(maritaltable[,5:8]), 1), 3)) educationtablePCT <- cbind(round(prop.table(as.matrix(educationtable[,1:5]), 1), 3), round(prop.table(as.matrix(educationtable[,6:10]), 1), 3)) @ \end{scriptsize} Now we have what we need to generate the figures. I continue by showing the code only for employment status (as an example). \begin{scriptsize} <>= # keep eval=FALSE # Collapse further for figures employmenttable2 <- rbind(colSums(employmenttable[1:2,]), colSums(employmenttable[3:4,]), colSums(employmenttable[5:6,]), colSums(employmenttable[7:8,]), colSums(employmenttable[9:10,]), employmenttable[11,]) rownames(employmenttable2) <- c("16 to 24 years", "25 to 34 years", "35 to 44 years", "45 to 54 years", "55 to 64 years", "65 years and over") # Percentages for figures employmenttablePCT2 <- cbind(round(prop.table(as.matrix(employmenttable2[,1:3]), 1), 3), round(prop.table(as.matrix(employmenttable2[,4:6]), 1), 3), round(prop.table(as.matrix(employmenttable2[,7:9]), 1), 3), round(prop.table(as.matrix(employmenttable2[,10:12]), 1), 3)) colnames(employmenttablePCT2) <- gsub("Census","U.S. Census_",colnames(employmenttablePCT2)) colnames(employmenttablePCT2) <- gsub("SAPAexp","Exploratory_",colnames(employmenttablePCT2)) colnames(employmenttablePCT2) <- gsub("SAPArep","Replication_",colnames(employmenttablePCT2)) colnames(employmenttablePCT2) <- gsub("SAPAconf","Confirmatory_",colnames(employmenttablePCT2)) employmenttablePCT2 <- as.data.frame(employmenttablePCT2) employmenttablePCT2$age <- rownames(employmenttablePCT2) employmenttablePCT2 <- gather(employmenttablePCT2, key="key", value="value",1:(ncol(employmenttablePCT2)-1)) employmenttablePCT2 <- separate(employmenttablePCT2, col = key, into=c("dataset","status"),sep="_") employmenttablePCT2$dataset <- factor(employmenttablePCT2$dataset, levels=c("U.S. Census", "Exploratory", "Replication", "Confirmatory")) employmenttablePCT2$status <- factor(employmenttablePCT2$status, levels=c("notCLF","unempCLF","empCLF"), labels=c("Not Civilian Labor Force","Unemployed in Civilian Labor Force","Employed in Civilian Labor Force")) ggplot(employmenttablePCT2, aes(x=dataset, y=value, fill=status)) + geom_bar(stat="identity", position="fill")+ scale_x_discrete("")+ scale_y_continuous("Percent")+ scale_fill_brewer("Employment Status", palette="Set1") + facet_wrap(~age)+ theme_bw(base_size=15) ggsave(filename = paste(filepathfig, "jobstatusfig.pdf", sep = ""), scale = 1, width = 14, height = 8, units = c("in"), dpi = 300) @ \end{scriptsize} \begin{scriptsize} <>= # keep eval=FALSE racetable2 <- rbind(colSums(racetable[4:5,]), colSums(racetable[6:7,]), colSums(racetable[8:9,]), colSums(racetable[10:11,]), colSums(racetable[12:13,]), colSums(racetable[14:15,])) rownames(racetable2) <- c("15 to 24 years", "25 to 34 years", "35 to 44 years", "45 to 54 years", "55 to 64 years", "65 years and over") maritaltable2 <- rbind(colSums(maritaltable[1:2,]), colSums(maritaltable[3:4,]), colSums(maritaltable[5:6,]), colSums(maritaltable[7:8,]), colSums(maritaltable[9:10,]), colSums(maritaltable[11:12,])) rownames(maritaltable2) <- c("15 to 24 years", "25 to 34 years", "35 to 44 years", "45 to 54 years", "55 to 64 years", "65 years and over") educationtable2 <- rbind(colSums(educationtable[1:2,]), colSums(educationtable[3:4,]), colSums(educationtable[5:6,]), colSums(educationtable[7:8,]), colSums(educationtable[9:10,]), colSums(educationtable[11:12,])) rownames(educationtable2) <- c("15 to 24 years", "25 to 34 years", "35 to 44 years", "45 to 54 years", "55 to 64 years", "65 years and over") racetablePCT2 <- cbind(round(prop.table(as.matrix(racetable2[,1:6]), 1), 3), round(prop.table(as.matrix(racetable2[,7:12]), 1), 3), round(prop.table(as.matrix(racetable2[,13:18]), 1), 3), round(prop.table(as.matrix(racetable2[,19:24]), 1), 3)) maritaltablePCT2 <- cbind(round(prop.table(as.matrix(maritaltable2[,1:4]), 1), 3), round(prop.table(as.matrix(maritaltable2[,5:8]), 1), 3), round(prop.table(as.matrix(maritaltable2[,9:12]), 1), 3), round(prop.table(as.matrix(maritaltable2[,13:16]), 1), 3)) educationtablePCT2 <- cbind(round(prop.table(as.matrix(educationtable2[,1:5]), 1), 3), round(prop.table(as.matrix(educationtable2[,6:10]), 1), 3), round(prop.table(as.matrix(educationtable2[,11:15]), 1), 3), round(prop.table(as.matrix(educationtable2[,16:20]), 1), 3)) colnames(racetablePCT2) <- gsub("census","U.S. Census_",colnames(racetablePCT2)) colnames(racetablePCT2) <- gsub("SAPAexp","Exploratory_",colnames(racetablePCT2)) colnames(racetablePCT2) <- gsub("SAPArep","Replication_",colnames(racetablePCT2)) colnames(racetablePCT2) <- gsub("SAPAconf","Confirmatory_",colnames(racetablePCT2)) colnames(maritaltablePCT2) <- gsub("census","U.S. Census_",colnames(maritaltablePCT2)) colnames(maritaltablePCT2) <- gsub("SAPAexp","Exploratory_",colnames(maritaltablePCT2)) colnames(maritaltablePCT2) <- gsub("SAPArep","Replication_",colnames(maritaltablePCT2)) colnames(maritaltablePCT2) <- gsub("SAPAconf","Confirmatory_",colnames(maritaltablePCT2)) colnames(educationtablePCT2) <- gsub("census","U.S. Census_",colnames(educationtablePCT2)) colnames(educationtablePCT2) <- gsub("SAPAexp","Exploratory_",colnames(educationtablePCT2)) colnames(educationtablePCT2) <- gsub("SAPArep","Replication_",colnames(educationtablePCT2)) colnames(educationtablePCT2) <- gsub("SAPAconf","Confirmatory_",colnames(educationtablePCT2)) racetablePCT2 <- as.data.frame(racetablePCT2) racetablePCT2$age <- rownames(racetablePCT2) racetablePCT2 <- gather(racetablePCT2, key="key", value="value",1:(ncol(racetablePCT2)-1)) racetablePCT2 <- separate(racetablePCT2, col = key, into=c("dataset","status"),sep="_") racetablePCT2$dataset <- factor(racetablePCT2$dataset, levels=c("U.S. Census", "Exploratory", "Replication", "Confirmatory")) racetablePCT2$status <- factor(racetablePCT2$status, levels=c("AfrAmer","Asian","Hispanic","Native","White", "TwoOther"), labels=c("African American","Asian","Hispanic", "Native American", "White", "Two Or More/Other")) ggplot(racetablePCT2, aes(x=dataset, y=value, fill=status)) + geom_bar(stat="identity", position="fill")+ scale_x_discrete("")+ scale_y_continuous("Percent")+ scale_fill_manual("Race/Ethnicity", values = rev(brewer.pal(6,"Set1"))) + facet_wrap(~age)+ theme_bw(base_size=15) ggsave(filename = paste(filepathfig, "racefig.pdf", sep = ""), scale = 1, width = 14, height = 8, units = c("in"), dpi = 300) educationtablePCT2 <- as.data.frame(educationtablePCT2) educationtablePCT2$age <- rownames(educationtablePCT2) educationtablePCT2 <- gather(educationtablePCT2, key="key", value="value",1:(ncol(educationtablePCT2)-1)) educationtablePCT2 <- separate(educationtablePCT2, col = key, into=c("dataset","status"),sep="_") educationtablePCT2$dataset <- factor(educationtablePCT2$dataset, levels=c("U.S. Census", "Exploratory", "Replication", "Confirmatory")) educationtablePCT2$status <- factor(educationtablePCT2$status, levels=c("LtHSG","HSG","SCorAssD","BD", "AdvD"), labels=c("Less Than High School Grad","High School Grad","Some College or Assc Degree","Bachelor's Degree", "Advanced Degree")) ggplot(educationtablePCT2, aes(x=dataset, y=value, fill=status)) + geom_bar(stat="identity", position="fill")+ scale_x_discrete("")+ scale_y_continuous("Percent")+ scale_fill_brewer("Education Level", palette="Set1") + facet_wrap(~age)+ theme_bw(base_size=15) ggsave(filename = paste(filepathfig, "edulevelfig.pdf", sep = ""), scale = 1, width = 14, height = 8, units = c("in"), dpi = 300) maritaltablePCT2 <- as.data.frame(maritaltablePCT2) maritaltablePCT2$age <- rownames(maritaltablePCT2) maritaltablePCT2 <- gather(maritaltablePCT2, key="key", value="value",1:(ncol(maritaltablePCT2)-1)) maritaltablePCT2 <- separate(maritaltablePCT2, col = key, into=c("dataset","status"),sep="_") maritaltablePCT2$dataset <- factor(maritaltablePCT2$dataset, levels=c("U.S. Census", "Exploratory", "Replication", "Confirmatory")) maritaltablePCT2$status <- factor(maritaltablePCT2$status, levels=c("DivSep","Married","NvrMarried","Widowed"), labels=c("Divorced/Separated","Married","Never Married","Widowed")) ggplot(maritaltablePCT2, aes(x=dataset, y=value, fill=status)) + geom_bar(stat="identity", position="fill")+ scale_x_discrete("")+ scale_y_continuous("Percent")+ scale_fill_brewer("Marital Status", palette="Set1") + facet_wrap(~age)+ theme_bw(base_size=15) ggsave(filename = paste(filepathfig, "maritalstatusfig.pdf", sep = ""), scale = 1, width = 14, height = 8, units = c("in"), dpi = 300) @ \end{scriptsize} \begin{figure} \ffigbox{\includegraphics[width=.95\textwidth]{figures/maritalstatusfig.pdf}}{\caption{Distribution of marital status across ages in the U.S. Census and the Exploratory, Replication, and Confirmatory samples}\label{fig:maritalstatusfig}} \end{figure} \begin{figure} \ffigbox{\includegraphics[width=.95\textwidth]{figures/edulevelfig.pdf}}{\caption{Distribution of educational attainment level across ages in the U.S. Census and the Exploratory, Replication, and Confirmatory samples}\label{fig:edulevelfig}} \end{figure} \begin{figure} \ffigbox{\includegraphics[width=.95\textwidth]{figures/jobstatusfig.pdf}}{\caption{Distribution of employment status across ages in the U.S. Census and the Exploratory, Replication, and Confirmatory samples. Civilian labor force excludes anyone who is retired, a student, a homemaker, in jail, in an institution, or not seeking work.}\label{fig:jobstatusfig}} \end{figure} \begin{figure} \ffigbox{\includegraphics[width=.95\textwidth]{figures/racefig.pdf}}{\caption{Distribution of race/ethnicity across ages in the U.S. Census and the Exploratory, Replication, and Confirmatory samples.}\label{fig:racefig}} \end{figure} We also want to know the correlation of the distribution of participants in the samples with the distribution of participants by ZIP code in the U.S. Census. So, the following section of code derives correlations between the SAPA data and the U.S. Census based on the 5-digit ZIP codes and the 3-digit ZIP codes. The rationale for looking at the 3-digit ZIP codes is based on the reality that the modal number of participants for most of the ZIP codes is 1; this severely attenuates the correlation by flattening the variability. Variability in the distribution is much higher for the 3-digit ZIP code regions. Note that I have to first convert ZIP codes (maintained by the U.S. Postal Service) to Zip Code Tabulation Area (ZCTAs, maintained by the U.S. Census Bureau) \citep{ZIP2ZCTA} by importing a cross-walk table.\footnote{This is a pre-processed external data file, included in the Supplementary Materials file at \url{http://dx.doi.org/10.7910/DVN/YF6STI}.} This is necessary because population statistics are only available from the U.S. Census for ZCTAs. There is also code in this section for plotting a U.S. map based on Participants by Zip Code Tabulation Area and then a world map based on Participants by Country. The code is hidden in the pdf version because it is highly technical and not easy to read. \begin{scriptsize} <>= load(paste(filepathdata, "ZCTA.crosswalk.final.Rdata", sep = "")) ZCTA.crosswalk.final <- subset(ZCTA.crosswalk.final, select = c(ZCTA, zip, Pop)) expSampleZCTA <- merge(expSample, ZCTA.crosswalk.final, by= "zip", all.x = TRUE) zipData <- subset(expSampleZCTA, select = c(zip, ZCTA, Pop)) censusZCTApops <- data.frame(ZCTA.crosswalk.final$Pop, ZCTA.crosswalk.final$ZCTA, as.numeric(substr(100000+ZCTA.crosswalk.final$ZCTA,1,4))) colnames(censusZCTApops) <- c("PsCensus", "ZCTA", "ZCTA3") # Here we make a df with # of SAPA Ps by 5 digit ZCTA expSampleZCTAs <- zipData[,"ZCTA"] tableZCTAs <- table(expSampleZCTAs) tableZCTAsNames <- names(tableZCTAs) tableZCTAsData <- as.numeric(as.vector(tableZCTAs)) expSampleZCTApops <- data.frame(tableZCTAsData, tableZCTAsNames) colnames(expSampleZCTApops) <- c("PsSAPA", "ZCTA") # Now we need a df with # of people (census) by 5 digit ZCTA unique <- censusZCTApops[!duplicated(censusZCTApops[,"ZCTA"]),] censusZCTA5pops <- unique[,c("PsCensus", "ZCTA")] zip5 <- as.matrix(merge(censusZCTA5pops, expSampleZCTApops, by = "ZCTA")) PopVsSample5digitCorExpSample <- cor(zip5[,"PsCensus"], zip5[,"PsSAPA"], use="pairwise") # So, we have 5 digit ZCTA codes for this many U.S. participants: sum(zip5[,"PsSAPA"]) # from this many 5 digit ZCTA codes dim(zip5)[1] # And here is the correlation of the distribution of Ps to the distribution of people in those ZCTAs in the census. PopVsSample5digitCorExpSample # Here we make a df with # of SAPA Ps by 3 digit ZCTA expSampleZCTAs3 <- as.numeric(substr(100000+expSampleZCTAs,1,4)) tableZCTAs3 <- table(expSampleZCTAs3) tableZCTAsNames3 <- names(tableZCTAs3) tableZCTAsData3 <- as.numeric(as.vector(tableZCTAs3)) expSampleZCTApops3 <- data.frame(tableZCTAsData3, tableZCTAsNames3) colnames(expSampleZCTApops3) <- c("PsSAPA3", "ZCTA3") # Now we need a df with # of people (census) by 3 digit ZCTA censusZCTApops3 <- subset(censusZCTApops, select = c(PsCensus, ZCTA3)) censusZCTApops3collapsed <- ddply(censusZCTApops3, "ZCTA3", numcolwise(sum)) zip3 <- as.matrix(merge(censusZCTApops3collapsed, expSampleZCTApops3, by = "ZCTA3")) PopVsSample3digitCorExpSample <- cor(zip3[,"PsCensus"], zip3[,"PsSAPA3"], use="pairwise") # So, we have 3 digit ZCTA codes for this many U.S. participants: sum(zip3[,"PsSAPA3"]) # from this many 3 digit ZCTA codes dim(zip3)[1] # And here is the correlation of the distribution of Ps to the distribution of people in those 3 digit ZCTAs in the census. PopVsSample3digitCorExpSample @ <>= # keep eval=FALSE # One of the map drawing packages in this section interferes # with the psych::describe function and it requires ZCTA data from # the US Census so keep this chunk set to FALSE # This code plots a US map based on Zip Code Tabulation Areas. # USmap, zipCode, ZipMap, ZCTAmap zip <- zipData[,"zip"] ZCTA <- zipData[,"ZCTA"] ZCTAn <- 1 - is.na(ZCTA) zipn <- 1 - is.na(zip) zipframe <- data.frame(zip, ZCTA, ZCTAn, zipn) suppressPackageStartupMessages(library(zipcode)) suppressPackageStartupMessages(library(mgcv)) suppressPackageStartupMessages(library(maps)) gridsize = 600 # default is 300 png(file=paste(filepathfig, "USmapE.png", sep = ""), width=800, height=580) suppressWarnings(sapaZip(data = zipframe, gridsize = 600, DV = DV, database = database, average = average, regions = ".", size = 11, miss = 0.05, w = NULL, ncols = NULL, main = "", mapdatabase = mapdatabase)) dev.off() # Now make a world map # countryMap worldMap suppressPackageStartupMessages(library(rworldmap, quietly=TRUE, warn.conflicts=FALSE)) suppressPackageStartupMessages(library(RColorBrewer, quietly=TRUE, warn.conflicts=FALSE)) # First, pull in the country coding key. load(paste(filepathdata, "countrykey.Rdata", sep = "")) countrydata <- table(expSample[, c("country")]) countrydata <- as.data.frame(countrydata) colnames(countrydata) <- c("ISO3", "Participants") # Participants <- as.vector(countrydata) # make a list of numbers for all countries # countrycodes <- c(1:length(countrydata)) # cdtemp <- data.frame(countrycodes, Participants) sapa.country <- merge(countrykey, countrydata, by="ISO3", all.x=TRUE) # Fix the plotting of Svalbard as it is the only visible land mass in the ISO3-based map for which we are not collecting data. # This is bc its part of Norway. So we add a line and give it the same participant count as Norway. sapa.country[236,2] <- sapa.country[158,2] # To keep the plots from being totally weird, use the log xx <- sort(sapa.country[,"Participants"], decreasing=TRUE) five <- xx[5] sapa.country <- scrub(sapa.country, where="Participants", isvalue=c(xx[1], xx[2], xx[3], xx[4]), newvalue=c(xx[5])) sapa.country <- scrub(sapa.country, where="Participants", isvalue=c(xx[5]), newvalue=xx[6]) logParticipants <- log(sapa.country[,"Participants"]) sapa.country <- data.frame(sapa.country, logParticipants) # creates the matrix to be plotted SAPAmap <- joinCountryData2Map(sapa.country, joinCode="ISO3", nameJoinColumn="ISO3") mapWithData <- getMap(projection="none", resolution="low") OfficialISO3codes <- mapWithData@data[["ISO3"]] codesInSAPA <- sapa.country$ISO3 matchPosnsInLookup <- match(as.character(codesInSAPA), as.character(OfficialISO3codes)) failedCodes <- codesInSAPA[is.na(matchPosnsInLookup)] failedCodes # these codes exist in SAPA data but are not plotted matchPosnsInUserData <- match(as.character(OfficialISO3codes), as.character(codesInSAPA)) codesMissingFromSAPA <- as.character(OfficialISO3codes[is.na(matchPosnsInUserData)]) codesMissingFromSAPA # these are the codes not being collected in SAPA # sets the mapping parameters #par(mai=c(0,0,0.2,0), xaxs="i", yaxs="i") # opens the plot window #mypalette<-brewer.pal(7,"YlGnBu") #quantilesplot <- mapCountryData(SAPAmap, nameColumnToPlot = "Participants", catMethod = "quantiles", #colourPalette=mypalette, missingCountryCol = 'white', addLegend = FALSE) #do.call(addMapLegend, c(quantilesplot, legendLabels="all", horizontal=TRUE, legendWidth = 0.5, legendShrink=.5, legendMar = 30)) # I think the following type is better, but perhaps not for the color-blind #It requires changing the function for adding a legend addMapLegend <- function (colourVector = "", cutVector = "", legendLabels = "limits", labelFontSize = 1, legendWidth = 1.2, legendShrink = 0.9, legendMar = 3, horizontal = TRUE, legendArgs = NULL, tcl = -0.5, mgp = c(3, 1, 0), sigFigs = 4, digits = 3, legendIntervals = "page", plottedData = "", catMethod = "pretty", colourPalette = "heat") { require(fields) if (length(colourVector) == length(cutVector)) colourVector <- colourVector[-length(colourVector)] colourBarBreaks = as.numeric(cutVector) tidyPlotBreaks <- signif(colourBarBreaks, sigFigs) zlim <- range(colourBarBreaks, na.rm = TRUE) if (legendIntervals == "page") { colourBarBreaks <- colourBarBreaks[1] + (colourBarBreaks[length(colourBarBreaks)] - colourBarBreaks[1]) * seq(from = 0, to = 1, length.out = length(colourBarBreaks)) } if (legendLabels == "limits") { limitsIndex = c(1, length(colourBarBreaks)) axis.args = list(at = colourBarBreaks[limitsIndex], cex.axis = labelFontSize, mgp = mgp, tcl = tcl, labels = prettyNum(tidyPlotBreaks[limitsIndex], digits = digits, format = "G")) } else if (legendLabels == "none") { if (horizontal) axis.args = list(xaxt = "n") else axis.args = list(yaxt = "n") } else if (legendLabels == "all") { axis.args = list(at = colourBarBreaks[c(1,11,21,31,41,51,61,71,81)], cex.axis = labelFontSize, mgp = mgp, tcl = tcl, labels = prettyNum(tidyPlotBreaks[c(1,11,21,31,41,51,61,71,81)], digits = digits, format = "G")) } image.plot(zlim = zlim, legend.only = TRUE, horizontal = horizontal, legend.args = legendArgs, legend.mar = legendMar, col = colourVector, breaks = colourBarBreaks, axis.args = axis.args, legend.width = legendWidth, legend.shrink = legendShrink) } # Now plot it with the super-awesome-ish legend png(filename = paste(filepathfig, "sapamapE.png", sep = ""), width = 600, height = 600, units = "px", res = NA) par(mai=c(0,0,0.2,0), xaxs="i", yaxs="i") # opens the plot window mypalette<-brewer.pal(9,"YlGnBu") categoricalplot <- mapCountryData(SAPAmap, nameColumnToPlot = "Participants", catMethod = "categorical", colourPalette=mypalette, borderCol = 'gray90', missingCountryCol = 'light grey', addLegend = FALSE, mapTitle='') len <- length(categoricalplot$colourVector) categoricalplot$colourVector <- categoricalplot$colourVector[1:(len-2)] suppressPackageStartupMessages(library(fields, quietly=TRUE, warn.conflicts=FALSE)) suppressPackageStartupMessages(library(spam, quietly=TRUE, warn.conflicts=FALSE)) suppressPackageStartupMessages(library(grid, quietly=TRUE, warn.conflicts=FALSE)) do.call(addMapLegend, c(categoricalplot, legendLabels="none", horizontal=TRUE, legendWidth = 0.5, legendShrink=.8, legendMar = 33, labelFontSize=.8)) # 'legendMar' will control the location of the legend - you have to mess with it. Higher values move it up. Try smaller value if can't see it at all. Also, consider changing the size of the quartz window under 'R/Preferences/Quartz' - legendMar = 38 for 10x10; 32 for 7x7; # legendLabels options are "none", "all" and "limits" dev.off() suppressPackageStartupMessages(library(psych)) # these are some other colors you might want to consider. # mypalette<-brewer.pal(7,"Blues") # mypalette <- c("#b5d0ff", "#96bdff", "#5794ff", "#3b83ff", "#1f71ff", "#035fff", "#1303ff") # mypalette <- c("#a6c6ff", "#96bdff", "#78a9ff", "#5794ff", "#3b83ff", "#1f71ff", "#035fff") # Note that in this file, we've scaled down the highest levels (about the 5 most populated countries) in order to keep the coloring from fading out for most countries. @ \end{scriptsize} These analyses are repeated for the Replication and Confirmatory samples (hidden in the PDF version). Results for the ZIP code correlations are shown in Table \ref{tab:zips}. Figures \ref{fig:USmapE}, \ref{fig:USmapR}, and \ref{fig:USmapC} show the distribution of U.S. participants by ZIP code. Figures \ref{fig:SmapE}, \ref{fig:SmapR}, and \ref{fig:SmapC} show the distribution of all participants by country. \begin{scriptsize} <>= load(paste(filepathdata, "ZCTA.crosswalk.final.Rdata", sep = "")) ZCTA.crosswalk.final <- subset(ZCTA.crosswalk.final, select = c(ZCTA, zip, Pop)) repSampleZCTA <- merge(repSample, ZCTA.crosswalk.final, by= "zip", all.x = TRUE) zipData <- subset(repSampleZCTA, select = c(zip, ZCTA, Pop)) censusZCTApops <- data.frame(ZCTA.crosswalk.final$Pop, ZCTA.crosswalk.final$ZCTA, as.numeric(substr(100000+ZCTA.crosswalk.final$ZCTA,1,4))) colnames(censusZCTApops) <- c("PsCensus", "ZCTA", "ZCTA3") # Here we make a df with # of SAPA Ps by 5 digit ZCTA repSampleZCTAs <- zipData[,"ZCTA"] tableZCTAs <- table(repSampleZCTAs) tableZCTAsNames <- names(tableZCTAs) tableZCTAsData <- as.numeric(as.vector(tableZCTAs)) repSampleZCTApops <- data.frame(tableZCTAsData, tableZCTAsNames) colnames(repSampleZCTApops) <- c("PsSAPA", "ZCTA") # Now we need a df with # of people (census) by 5 digit ZCTA unique <- censusZCTApops[!duplicated(censusZCTApops[,"ZCTA"]),] censusZCTA5pops <- unique[,c("PsCensus", "ZCTA")] zip5 <- as.matrix(merge(censusZCTA5pops, repSampleZCTApops, by = "ZCTA")) PopVsSample5digitCorRepSample <- cor(zip5[,"PsCensus"], zip5[,"PsSAPA"], use="pairwise") # So, we have 5 digit ZCTA codes for this many U.S. participants: sum(zip5[,"PsSAPA"]) # from this many 5 digit ZCTA codes dim(zip5)[1] # And here is the correlation of the distribution of Ps to the distribution of people in those ZCTAs in the census. PopVsSample5digitCorRepSample # Here we make a df with # of SAPA Ps by 3 digit ZCTA repSampleZCTAs3 <- as.numeric(substr(100000+repSampleZCTAs,1,4)) tableZCTAs3 <- table(repSampleZCTAs3) tableZCTAsNames3 <- names(tableZCTAs3) tableZCTAsData3 <- as.numeric(as.vector(tableZCTAs3)) repSampleZCTApops3 <- data.frame(tableZCTAsData3, tableZCTAsNames3) colnames(repSampleZCTApops3) <- c("PsSAPA3", "ZCTA3") # Now we need a df with # of people (census) by 3 digit ZCTA censusZCTApops3 <- subset(censusZCTApops, select = c(PsCensus, ZCTA3)) censusZCTApops3collapsed <- ddply(censusZCTApops3, "ZCTA3", numcolwise(sum)) zip3 <- as.matrix(merge(censusZCTApops3collapsed, repSampleZCTApops3, by = "ZCTA3")) PopVsSample3digitCorRepSample <- cor(zip3[,"PsCensus"], zip3[,"PsSAPA3"], use="pairwise") # So, we have 3 digit ZCTA codes for this many U.S. participants: sum(zip3[,"PsSAPA3"]) # from this many 3 digit ZCTA codes dim(zip3)[1] # And here is the correlation of the distribution of Ps to the distribution of people in those 3 digit ZCTAs in the census. PopVsSample3digitCorRepSample @ <>= # keep eval=FALSE # One of the map drawing packages in this section interferes # with the psych::describe function and it requires ZCTA data from # the US Census so keep this chunk set to FALSE # This code plots a US map based on Zip Code Tabulation Areas. # USmap, zipCode, ZipMap, ZCTAmap zip <- zipData[,"zip"] ZCTA <- zipData[,"ZCTA"] ZCTAn <- 1 - is.na(ZCTA) zipn <- 1 - is.na(zip) zipframe <- data.frame(zip, ZCTA, ZCTAn, zipn) suppressPackageStartupMessages(library(zipcode, quietly=TRUE, warn.conflicts=FALSE)) suppressPackageStartupMessages(library(mgcv, quietly=TRUE, warn.conflicts=FALSE)) suppressPackageStartupMessages(library(maps, quietly=TRUE, warn.conflicts=FALSE)) gridsize = 600 # default is 300 png(file=paste(filepathfig, "USmapR.png", sep = ""), width=800, height=580) suppressWarnings(sapaZip(data = zipframe, gridsize = 600, DV = DV, database = database, average = average, regions = ".", size = 11, miss = 0.05, w = NULL, ncols = NULL, main = "", mapdatabase = mapdatabase)) dev.off() # Now make a world map # countryMap worldMap suppressPackageStartupMessages(library(rworldmap, quietly=TRUE, warn.conflicts=FALSE)) suppressPackageStartupMessages(library(RColorBrewer, quietly=TRUE, warn.conflicts=FALSE)) # First, pull in the country coding key. load(paste(filepathdata, "countrykey.Rdata", sep = "")) countrydata <- table(repSample[, c("country")]) countrydata <- as.data.frame(countrydata) colnames(countrydata) <- c("ISO3", "Participants") # Participants <- as.vector(countrydata) # make a list of numbers for all countries # countrycodes <- c(1:length(countrydata)) # cdtemp <- data.frame(countrycodes, Participants) sapa.country <- merge(countrykey, countrydata, by="ISO3", all.x=TRUE) # Fix the plotting of Svalbard as it is the only visible land mass in the ISO3-based map for which we are not collecting data. This is bc its part of Norway. So we add a line and give it the same participant count as Norway. sapa.country[236,2] <- sapa.country[158,2] # To keep the plots from being totally weird, use the log xx <- sort(sapa.country[,"Participants"], decreasing=TRUE) five <- xx[5] sapa.country <- scrub(sapa.country, where="Participants", isvalue=c(xx[1], xx[2], xx[3], xx[4]), newvalue=c(xx[5])) sapa.country <- scrub(sapa.country, where="Participants", isvalue=c(xx[5]), newvalue=xx[6]) logParticipants <- log(sapa.country[,"Participants"]) sapa.country <- data.frame(sapa.country, logParticipants) # creates the matrix to be plotted SAPAmap <- joinCountryData2Map(sapa.country, joinCode="ISO3", nameJoinColumn="ISO3") mapWithData <- getMap(projection="none", resolution="low") OfficialISO3codes <- mapWithData@data[["ISO3"]] codesInSAPA <- sapa.country$ISO3 matchPosnsInLookup <- match(as.character(codesInSAPA), as.character(OfficialISO3codes)) failedCodes <- codesInSAPA[is.na(matchPosnsInLookup)] failedCodes # these codes exist in SAPA data but are not plotted matchPosnsInUserData <- match(as.character(OfficialISO3codes), as.character(codesInSAPA)) codesMissingFromSAPA <- as.character(OfficialISO3codes[is.na(matchPosnsInUserData)]) codesMissingFromSAPA # these are the codes not being collected in SAPA # sets the mapping parameters #par(mai=c(0,0,0.2,0), xaxs="i", yaxs="i") # opens the plot window #mypalette<-brewer.pal(7,"YlGnBu") #quantilesplot <- mapCountryData(SAPAmap, nameColumnToPlot = "Participants", catMethod = "quantiles", #colourPalette=mypalette, missingCountryCol = 'white', addLegend = FALSE) #do.call(addMapLegend, c(quantilesplot, legendLabels="all", horizontal=TRUE, legendWidth = 0.5, legendShrink=.5, legendMar = 30)) # I think the following type is better, but perhaps not for the color-blind #It requires changing the function for adding a legend addMapLegend <- function (colourVector = "", cutVector = "", legendLabels = "limits", labelFontSize = 1, legendWidth = 1.2, legendShrink = 0.9, legendMar = 3, horizontal = TRUE, legendArgs = NULL, tcl = -0.5, mgp = c(3, 1, 0), sigFigs = 4, digits = 3, legendIntervals = "page", plottedData = "", catMethod = "pretty", colourPalette = "heat") { require(fields) if (length(colourVector) == length(cutVector)) colourVector <- colourVector[-length(colourVector)] colourBarBreaks = as.numeric(cutVector) tidyPlotBreaks <- signif(colourBarBreaks, sigFigs) zlim <- range(colourBarBreaks, na.rm = TRUE) if (legendIntervals == "page") { colourBarBreaks <- colourBarBreaks[1] + (colourBarBreaks[length(colourBarBreaks)] - colourBarBreaks[1]) * seq(from = 0, to = 1, length.out = length(colourBarBreaks)) } if (legendLabels == "limits") { limitsIndex = c(1, length(colourBarBreaks)) axis.args = list(at = colourBarBreaks[limitsIndex], cex.axis = labelFontSize, mgp = mgp, tcl = tcl, labels = prettyNum(tidyPlotBreaks[limitsIndex], digits = digits, format = "G")) } else if (legendLabels == "none") { if (horizontal) axis.args = list(xaxt = "n") else axis.args = list(yaxt = "n") } else if (legendLabels == "all") { axis.args = list(at = colourBarBreaks[c(1,11,21,31,41,51,61,71,81)], cex.axis = labelFontSize, mgp = mgp, tcl = tcl, labels = prettyNum(tidyPlotBreaks[c(1,11,21,31,41,51,61,71,81)], digits = digits, format = "G")) } image.plot(zlim = zlim, legend.only = TRUE, horizontal = horizontal, legend.args = legendArgs, legend.mar = legendMar, col = colourVector, breaks = colourBarBreaks, axis.args = axis.args, legend.width = legendWidth, legend.shrink = legendShrink) } # Now plot it with the super-awesome-ish legend png(filename = paste(filepathfig, "sapamapR.png", sep = ""), width = 600, height = 600, units = "px", res = NA) par(mai=c(0,0,0.2,0), xaxs="i", yaxs="i") # opens the plot window mypalette<-brewer.pal(9,"YlGnBu") categoricalplot <- mapCountryData(SAPAmap, nameColumnToPlot = "Participants", catMethod = "categorical", colourPalette=mypalette, borderCol = 'gray90', missingCountryCol = 'light grey', addLegend = FALSE, mapTitle='') len <- length(categoricalplot$colourVector) categoricalplot$colourVector <- categoricalplot$colourVector[1:(len-2)] suppressPackageStartupMessages(library(fields, quietly=TRUE, warn.conflicts=FALSE)) suppressPackageStartupMessages(library(spam, quietly=TRUE, warn.conflicts=FALSE)) suppressPackageStartupMessages(library(grid, quietly=TRUE, warn.conflicts=FALSE)) do.call(addMapLegend, c(categoricalplot, legendLabels="none", horizontal=TRUE, legendWidth = 0.5, legendShrink=.8, legendMar = 33, labelFontSize=.8)) # 'legendMar' will control the location of the legend - you have to mess with it. Higher values move it up. Try smaller value if can't see it at all. Also, consider changing the size of the quartz window under 'R/Preferences/Quartz' - legendMar = 38 for 10x10; 32 for 7x7; # legendLabels options are "none", "all" and "limits" dev.off() # these are some other colors you might want to consider. # mypalette<-brewer.pal(7,"Blues") # mypalette <- c("#b5d0ff", "#96bdff", "#5794ff", "#3b83ff", "#1f71ff", "#035fff", "#1303ff") # mypalette <- c("#a6c6ff", "#96bdff", "#78a9ff", "#5794ff", "#3b83ff", "#1f71ff", "#035fff") # Note that in this file, we've scaled down the highest levels (about the 5 most populated countries) in order to keep the coloring from fading out for most countries. @ <>= load(paste(filepathdata, "ZCTA.crosswalk.final.Rdata", sep = "")) ZCTA.crosswalk.final <- subset(ZCTA.crosswalk.final, select = c(ZCTA, zip, Pop)) confSampleZCTA <- merge(confSample, ZCTA.crosswalk.final, by= "zip", all.x = TRUE) zipData <- subset(confSampleZCTA, select = c(zip, ZCTA, Pop)) censusZCTApops <- data.frame(ZCTA.crosswalk.final$Pop, ZCTA.crosswalk.final$ZCTA, as.numeric(substr(100000+ZCTA.crosswalk.final$ZCTA,1,4))) colnames(censusZCTApops) <- c("PsCensus", "ZCTA", "ZCTA3") # Here we make a df with # of SAPA Ps by 5 digit ZCTA confSampleZCTAs <- zipData[,"ZCTA"] tableZCTAs <- table(confSampleZCTAs) tableZCTAsNames <- names(tableZCTAs) tableZCTAsData <- as.numeric(as.vector(tableZCTAs)) confSampleZCTApops <- data.frame(tableZCTAsData, tableZCTAsNames) colnames(confSampleZCTApops) <- c("PsSAPA", "ZCTA") # Now we need a df with # of people (census) by 5 digit ZCTA unique <- censusZCTApops[!duplicated(censusZCTApops[,"ZCTA"]),] censusZCTA5pops <- unique[,c("PsCensus", "ZCTA")] zip5 <- as.matrix(merge(censusZCTA5pops, confSampleZCTApops, by = "ZCTA")) PopVsSample5digitCorConfSample <- cor(zip5[,"PsCensus"], zip5[,"PsSAPA"], use="pairwise") # So, we have 5 digit ZCTA codes for this many U.S. participants: sum(zip5[,"PsSAPA"]) # from this many 5 digit ZCTA codes dim(zip5)[1] # And here is the correlation of the distribution of Ps to the distribution of people in those ZCTAs in the census. PopVsSample5digitCorConfSample # Here we make a df with # of SAPA Ps by 3 digit ZCTA confSampleZCTAs3 <- as.numeric(substr(100000+confSampleZCTAs,1,4)) tableZCTAs3 <- table(confSampleZCTAs3) tableZCTAsNames3 <- names(tableZCTAs3) tableZCTAsData3 <- as.numeric(as.vector(tableZCTAs3)) confSampleZCTApops3 <- data.frame(tableZCTAsData3, tableZCTAsNames3) colnames(confSampleZCTApops3) <- c("PsSAPA3", "ZCTA3") # Now we need a df with # of people (census) by 3 digit ZCTA censusZCTApops3 <- subset(censusZCTApops, select = c(PsCensus, ZCTA3)) censusZCTApops3collapsed <- ddply(censusZCTApops3, "ZCTA3", numcolwise(sum)) zip3 <- as.matrix(merge(censusZCTApops3collapsed, confSampleZCTApops3, by = "ZCTA3")) PopVsSample3digitCorConfSample <- cor(zip3[,"PsCensus"], zip3[,"PsSAPA3"], use="pairwise") # So, we have 3 digit ZCTA codes for this many U.S. participants: sum(zip3[,"PsSAPA3"]) # from this many 3 digit ZCTA codes dim(zip3)[1] # And here is the correlation of the distribution of Ps to the distribution of people in those 3 digit ZCTAs in the census. PopVsSample3digitCorConfSample @ <>= # keep eval=FALSE # One of the map drawing packages in this section interferes # with the psych::describe function and it requires ZCTA data from # the US Census so keep this chunk set to FALSE # This code plots a US map based on Zip Code Tabulation Areas. # USmap, zipCode, ZipMap, ZCTAmap zip <- zipData[,"zip"] ZCTA <- zipData[,"ZCTA"] ZCTAn <- 1 - is.na(ZCTA) zipn <- 1 - is.na(zip) zipframe <- data.frame(zip, ZCTA, ZCTAn, zipn) suppressPackageStartupMessages(library(zipcode, quietly=TRUE, warn.conflicts=FALSE)) suppressPackageStartupMessages(library(mgcv, quietly=TRUE, warn.conflicts=FALSE)) suppressPackageStartupMessages(library(maps, quietly=TRUE, warn.conflicts=FALSE)) gridsize = 600 # default is 300 png(file=paste(filepathfig, "USmapC.png", sep = ""), width=800, height=580) suppressWarnings(sapaZip(data = zipframe, gridsize = 600, DV = DV, database = database, average = average, regions = ".", size = 11, miss = 0.05, w = NULL, ncols = NULL, main = "", mapdatabase = mapdatabase)) dev.off() # Now make a world map # countryMap worldMap suppressPackageStartupMessages(library(rworldmap, quietly=TRUE, warn.conflicts=FALSE)) suppressPackageStartupMessages(library(RColorBrewer, quietly=TRUE, warn.conflicts=FALSE)) # First, pull in the country coding key. load(paste(filepathdata, "countrykey.Rdata", sep = "")) countrydata <- table(confSample[, c("country")]) countrydata <- as.data.frame(countrydata) colnames(countrydata) <- c("ISO3", "Participants") # Participants <- as.vector(countrydata) # make a list of numbers for all countries # countrycodes <- c(1:length(countrydata)) # cdtemp <- data.frame(countrycodes, Participants) sapa.country <- merge(countrykey, countrydata, by="ISO3", all.x=TRUE) # Fix the plotting of Svalbard as it is the only visible land mass in the ISO3-based map for which we are not collecting data. This is bc its part of Norway. So we add a line and give it the same participant count as Norway. sapa.country[236,2] <- sapa.country[158,2] # To keep the plots from being totally weird, use the log xx <- sort(sapa.country[,"Participants"], decreasing=TRUE) five <- xx[5] sapa.country <- scrub(sapa.country, where="Participants", isvalue=c(xx[1], xx[2], xx[3], xx[4]), newvalue=c(xx[5])) sapa.country <- scrub(sapa.country, where="Participants", isvalue=c(xx[5]), newvalue=xx[6]) logParticipants <- log(sapa.country[,"Participants"]) sapa.country <- data.frame(sapa.country, logParticipants) # creates the matrix to be plotted SAPAmap <- joinCountryData2Map(sapa.country, joinCode="ISO3", nameJoinColumn="ISO3") mapWithData <- getMap(projection="none", resolution="low") OfficialISO3codes <- mapWithData@data[["ISO3"]] codesInSAPA <- sapa.country$ISO3 matchPosnsInLookup <- match(as.character(codesInSAPA), as.character(OfficialISO3codes)) failedCodes <- codesInSAPA[is.na(matchPosnsInLookup)] failedCodes # these codes exist in SAPA data but are not plotted matchPosnsInUserData <- match(as.character(OfficialISO3codes), as.character(codesInSAPA)) codesMissingFromSAPA <- as.character(OfficialISO3codes[is.na(matchPosnsInUserData)]) codesMissingFromSAPA # these are the codes not being collected in SAPA # sets the mapping parameters #par(mai=c(0,0,0.2,0), xaxs="i", yaxs="i") # opens the plot window #mypalette<-brewer.pal(7,"YlGnBu") #quantilesplot <- mapCountryData(SAPAmap, nameColumnToPlot = "Participants", catMethod = "quantiles", #colourPalette=mypalette, missingCountryCol = 'white', addLegend = FALSE) #do.call(addMapLegend, c(quantilesplot, legendLabels="all", horizontal=TRUE, legendWidth = 0.5, legendShrink=.5, legendMar = 30)) # I think the following type is better, but perhaps not for the color-blind #It requires changing the function for adding a legend addMapLegend <- function (colourVector = "", cutVector = "", legendLabels = "limits", labelFontSize = 1, legendWidth = 1.2, legendShrink = 0.9, legendMar = 3, horizontal = TRUE, legendArgs = NULL, tcl = -0.5, mgp = c(3, 1, 0), sigFigs = 4, digits = 3, legendIntervals = "page", plottedData = "", catMethod = "pretty", colourPalette = "heat") { require(fields) if (length(colourVector) == length(cutVector)) colourVector <- colourVector[-length(colourVector)] colourBarBreaks = as.numeric(cutVector) tidyPlotBreaks <- signif(colourBarBreaks, sigFigs) zlim <- range(colourBarBreaks, na.rm = TRUE) if (legendIntervals == "page") { colourBarBreaks <- colourBarBreaks[1] + (colourBarBreaks[length(colourBarBreaks)] - colourBarBreaks[1]) * seq(from = 0, to = 1, length.out = length(colourBarBreaks)) } if (legendLabels == "limits") { limitsIndex = c(1, length(colourBarBreaks)) axis.args = list(at = colourBarBreaks[limitsIndex], cex.axis = labelFontSize, mgp = mgp, tcl = tcl, labels = prettyNum(tidyPlotBreaks[limitsIndex], digits = digits, format = "G")) } else if (legendLabels == "none") { if (horizontal) axis.args = list(xaxt = "n") else axis.args = list(yaxt = "n") } else if (legendLabels == "all") { axis.args = list(at = colourBarBreaks[c(1,11,21,31,41,51,61,71,81)], cex.axis = labelFontSize, mgp = mgp, tcl = tcl, labels = prettyNum(tidyPlotBreaks[c(1,11,21,31,41,51,61,71,81)], digits = digits, format = "G")) } image.plot(zlim = zlim, legend.only = TRUE, horizontal = horizontal, legend.args = legendArgs, legend.mar = legendMar, col = colourVector, breaks = colourBarBreaks, axis.args = axis.args, legend.width = legendWidth, legend.shrink = legendShrink) } # Now plot it with the super-awesome-ish legend png(filename = paste(filepathfig, "sapamapC.png", sep = ""), width = 600, height = 600, units = "px", res = NA) par(mai=c(0,0,0.2,0), xaxs="i", yaxs="i") # opens the plot window mypalette<-brewer.pal(9,"YlGnBu") categoricalplot <- mapCountryData(SAPAmap, nameColumnToPlot = "Participants", catMethod = "categorical", colourPalette=mypalette, borderCol = 'gray90', missingCountryCol = 'light grey', addLegend = FALSE, mapTitle='') len <- length(categoricalplot$colourVector) categoricalplot$colourVector <- categoricalplot$colourVector[1:(len-2)] suppressPackageStartupMessages(library(fields, quietly=TRUE, warn.conflicts=FALSE)) suppressPackageStartupMessages(library(spam, quietly=TRUE, warn.conflicts=FALSE)) suppressPackageStartupMessages(library(grid, quietly=TRUE, warn.conflicts=FALSE)) do.call(addMapLegend, c(categoricalplot, legendLabels="none", horizontal=TRUE, legendWidth = 0.5, legendShrink=.8, legendMar = 33, labelFontSize=.8)) # 'legendMar' will control the location of the legend - you have to mess with it. Higher values move it up. Try smaller value if can't see it at all. Also, consider changing the size of the quartz window under 'R/Preferences/Quartz' - legendMar = 38 for 10x10; 32 for 7x7; # legendLabels options are "none", "all" and "limits" dev.off() # these are some other colors you might want to consider. # mypalette<-brewer.pal(7,"Blues") # mypalette <- c("#b5d0ff", "#96bdff", "#5794ff", "#3b83ff", "#1f71ff", "#035fff", "#1303ff") # mypalette <- c("#a6c6ff", "#96bdff", "#78a9ff", "#5794ff", "#3b83ff", "#1f71ff", "#035fff") # Note that in this file, we've scaled down the highest levels (about the 5 most populated countries) in order to keep the coloring from fading out for most countries. @ \end{scriptsize} \begin{scriptsize} \begin{table}[htdp]\caption{Correlations of geographic distribution between the SAPA samples and the U.S. Census sample based on ZIP codes} \begin{center} \begin{tabular} {l c c c} \hline & \multicolumn{3}{c}{Sample} \cr & Exploratory & Replication & Confirmatory \cr \hline 5 digit ZIP codes & \Sexpr{round(PopVsSample5digitCorExpSample, 2)} & \Sexpr{round(PopVsSample5digitCorRepSample, 2)} & \Sexpr{round(PopVsSample5digitCorConfSample, 2)} \cr 3 digit ZIP codes & \Sexpr{round(PopVsSample3digitCorExpSample, 2)} & \Sexpr{round(PopVsSample3digitCorRepSample, 2)} & \Sexpr{round(PopVsSample3digitCorConfSample, 2)} \cr \hline \end{tabular} \end{center} \label{tab:zips} \end{table} \end{scriptsize} \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/USmapE.png}}{\caption{Exploratory Sample by U.S. Zip Code Tabulation Area}\label{fig:USmapE}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/sapamapE.png}}{\caption{Exploratory Sample by Country}\label{fig:SmapE}} \end{floatrow} \end{figure} \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/USmapR.png}}{\caption{Replication Sample by U.S. Zip Code Tabulation Area}\label{fig:USmapR}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/sapamapR.png}}{\caption{Replication Sample by Country}\label{fig:SmapR}} \end{floatrow} \end{figure} \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/USmapC.png}}{\caption{Confirmatory Sample by U.S. Zip Code Tabulation Area}\label{fig:USmapC}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/sapamapC.png}}{\caption{Confirmatory Sample by Country}\label{fig:SmapC}} \end{floatrow} \end{figure} \subsection{Conclusions about representativeness of the participant samples} Generalizability, like replicability, is an important consideration when attempting to develop psychological scales that are useful for anything other than idiosyncratic research. Unfortunately, most attempts to demonstrate representativeness when sampling randomly from a broad population are likely to fail and this introduces uncertainty about generalizability which can only be resolved empirically. Still, this issue is so fundamentally important that it seems worth trying to contextualize the representativeness of these samples before moving on to the analyses. These data are certainly not representative of humans as a species. Such a claim would be unjustifiable without considerably more knowledge than is currently available regarding the biological, environmental, and social/cultural factors that affect our thoughts, feelings, and behaviors (though a claim like this would not be unprecedented in personality research [\citealp{McCrae:2005jz}]). Even if we someday achieve broad understanding of these factors, it would be prudent to acknowledge that most members of our species have already lived and died without leaving behind any historical record of their personalities. With few exceptions, we are limited to speculation regarding the manner and circumstances in which these lives played out across the long stretches of our evolutionary past. It would be an untestable claim to assert that any current model fits past populations. At best, we might consider the reference population to be humans who are now living. These data are not representative of them either. It might be marginally plausible to assert such a claim if one were to implement a data collection methodology of far broader scope (and far bigger budget) than the online data collection methods used here. It would be particularly important, for example, to sample from the population of humans who never use the internet because they do not have access to it or because they are not interested in it or because they are unable (e.g., illiterate or impaired). I continue along these lines because I want to be measured in my claims regarding generalizability: \begin{itemize} \item The data are not representative of all individuals who frequent the internet because the website from which the data were collected is only available in English. \vspace{-3mm} \item The data are not (necessarily) representative of English-speaking individuals who frequent the internet for many reasons including the fact that individuals in this population were not equally/randomly likely to be made aware of the site. \vspace{-3mm} \item Even if individuals in this population were made aware of the site in an equivalent or fully-randomized manner, the data would not necessarily be representative because there would remain a sampling bias due to individual differences in the motivation to visit the site and participate (i.e., start the survey). \vspace{-3mm} \item Even within the population of now-living, English-speaking, internet users who would be willing to participate, the data may be biased by the fact that participants were allowed to respond to as many items as they wished (between 16 and 311) and the number of responses provided per participant likely reflects underlying individual differences which \textit{may} be meaningful to evaluations of personality structure. \end{itemize} Despite the many ways in which these data are or are not necessarily representative, I still have some confidence that these samples are adequate for the design of personality scales with potential predictive utility. In part, this confidence is grounded in an assumption that these qualifiers do not exclude \textit{everyone} from the relevant population. To the contrary, I believe that it's a relatively large population of individuals who surfed the internet on occasion between 2014 and 2016 (when most of the data were collected) and who were proficient at reading English (or who used a browser that could translate English to their native language) and who would've been willing to complete a 20 minute personality survey in exchange for feedback about their personality. More practically, I believe these data are sufficiently representative because they have been collected from a more diverse sample of participants than has historically been used in personality research. By way of comparison, the Eugene-Springfield Community Sample --- the large, gold-standard sample that has been widely used in the development and validation of more than 400 personality scales \citep{ipip} --- is notably less diverse than the samples used here. The demographic profile of ESCS participants who completed at least 2 of the 4 surveys administered between 1993 and 2003 \citep{Goldberg:2008vg} is shown in Table \ref{tab:ESCS}. \textbf{98.4\% of the ESCS sample was White and 97.6\% was born in the United States.} \clearpage \begin{scriptsize} \begin{table}[h!] \caption{Demographic Characteristics of the Eugene-Springfield Community Sample} \begin{center} \begin{tabular} {l l r } \hline \multicolumn{2}{l}{Demographic Variable} & Percent \cr \hline & & \cr \multicolumn{2}{l}{White (N=757)} & 98.4\% \cr & & \cr \multicolumn{2}{l}{Female (N=757)} & 56.9\% \cr & & \cr \multicolumn{2}{l}{Age in 2005 (N=757):} & \cr & 30-41 years old & 3.3\% \cr & 42-51 years old & 16.1\% \cr & 52-61 years old & 34.7\% \cr & 62-71 years old & 21.1\% \cr & 72-81 years old & 16.6\% \cr & 82 and over & 8.1\% \cr & & \cr \multicolumn{2}{l}{Marital status in 1999 (N=763):} & \cr & Married & 76.6\% \cr & Divorced & 8.3\% \cr & Widowed & 8.2\% \cr & & \cr \multicolumn{2}{l}{Employment status (N=763):} & \cr & Working for pay full-time & 44\% \cr & Retired & 32\% \cr & Working for pay part-time & 16\% \cr & Homemaker & 9\% \cr & & \cr \multicolumn{2}{l}{Educational attainment level in 1993 (N=763):} & \cr & Not graduated from high school & 1.2\% \cr & High school graduate & 8.9\% \cr & Vocational/technical schooling & 5.9\% \cr & Some college & 28.0\% \cr & College graduate & 20.7\% \cr & Some post-college education & 11.3\% \cr & Post-college degree & 23.9\% \cr & & \cr \multicolumn{2}{l}{Region of Birth in the U.S. (N=763):} & \cr & Pacific Northwest & 40.3\% \cr & Midwest & 20.3\% \cr & California & 15.8\% \cr & Northeast & 9.4\% \cr & Great Plains & 5.5\% \cr & Southeast & 3.9\% \cr & Southwest & 2.4\% \cr & Foreign country & 2.4\% \cr \hline \multicolumn{3}{l}{See \citet{Goldberg:2016vg} for more information.} \cr \end{tabular} \end{center} \label{tab:ESCS} \end{table} \end{scriptsize} \clearpage To be clear, the ESCS has been --- and continues to be --- a tremendous asset to personality researchers. Collection of these data by mail required methodological foresight, persistence and consistent organization on behalf of Lew Goldberg and colleagues at the Oregon Research Institute in addition to selfless participation from hundreds of homeowners in Eugene and Springfield, many of whom responded to several thousand survey items over the course of a decade. The utility of this dataset is hard to overstate. I say this because I don't want my criticism of the representativeness to detract in any way from the importance of their efforts. That said, survey-based research methods have evolved considerably since ESCS data collection began in 1993 and they continue to evolve rapidly. I have taken advantage of these developments to collect larger and more representative samples than have typically been used for this type of research. A couple of other research groups have collected even larger samples than these and it seems obvious that these will soon be superceded by even better samples in the near future. With sufficient funding and coordinated collaboration (perhaps something like Goldberg's suggestion of a personality ``collaboratory'' [\citealp{Goldberg:99a}]), I think it will soon be feasible to collect data that does not require quite so many qualifiers --- datasets that include millions of participants and thousands of items (including those for other domains like interests and values and abilities and goals). In the meantime, I've done my best on a very (very) limited budget; these samples contain hundreds of items and tens of thousands of participants. I believe that these are more likely to generalize than most other samples reported in the literature to date on the basis of diversity, but this is an empirical question for future research. \section{Sensitivity analyses} \label{sec:sensitivity} In this section, I consider the extent to which our results are likely to depend on various analytic options. It should not be presumed that the results of these sensitivity analyses will generalize beyond these data. While citations are always appreciated, demonstration of partial or complete insensitivity of these results to a specific analytic option should not be used as justification for the omission of sensitivity analyses in other projects; it should be expected that the effect of some analytic options will (sometimes) be stronger in data sets with smaller sample sizes. \subsection{Correlation matrices: the effects of ipsatization} \label{sec:ipsatize} Our goal here is to show how ipsatized data differs from that which is not ipsatized. I start by getting the Pearson correlation matrices based on the raw data for both samples. Note that the first step here is to drop all variables except those which I'll be using in the structural analyses (the 696 personality items). \begin{scriptsize} <>= rm(list=setdiff(ls(), c("filepathdata", "filepathfig"))) load(paste(filepathdata, "descriptivesData.RData", sep = "")) rm(list=setdiff(ls(), c("confSample", "DataItems", "DemoItems", "expSample", "filepathdata", "filepathfig", "ItemInfo696", "pwiseAdmins1C", "pwiseAdmins1E", "pwiseAdmins1R", "repSample"))) # Trim down the dictionary a bit ItemInfo696 <- subset(ItemInfo696, select = c(Item, IPIP100, BFAS, HEXACO, QB6, MPQ, IPIPneo, EPQr, PS)) # Need to fix the text of one item. # For some reason, a corruption keeps getting introduced # to produce: "Am self\xd0destructive." ItemInfo696["q_3691",1] <- "Am self-destructive." # for now, focus only on the personality items expSampleTemp696 <- subset(expSample, select = DataItems) # show some of the raw data for example head(expSampleTemp696[,1:8]) psych::describe(expSampleTemp696)[1:8,] # standardize the rows and show some suppressPackageStartupMessages(library(multicon)) expSampleTemp696ip <- ipsatize(expSampleTemp696) # confirm that the sum of the row means (rounded to the 5th digit) is zero isTRUE(sum(round(rowMeans(expSampleTemp696ip, na.rm=TRUE),5), na.rm=TRUE)==0) head(expSampleTemp696ip[,1:8]) psych::describe(expSampleTemp696ip)[1:8,] # this is distinguished from standardizing the columns expSampleTemp696std <- scale(expSampleTemp696, center = TRUE, scale = TRUE) # confirm that the sum of the col means (rounded to the 5th digit) is zero isTRUE(sum(round(colMeans(expSampleTemp696std, na.rm=TRUE),5), na.rm=TRUE)==0) head(expSampleTemp696std[,1:8]) psych::describe(expSampleTemp696std)[1:8,] # if we only standardize the columns, it should have no effect on the correlations # bc it's just a linear transformation of the raw data. # demonstrate this expSampleCor696raw <- cor(expSampleTemp696, use="pairwise") expSampleCor696std <- cor(expSampleTemp696std, use="pairwise") cordiff1 <- expSampleCor696raw-expSampleCor696std # confirm that they are equivalent to the 10th decimal isTRUE(round(sum(cordiff1), 10)==0) # this means that standardizing by rows AND columns produces correlations # that are equivalent to those standardized only by rows (ipsatized) # this approach is occasionally recommended in the literature - don't bother # all we need to compare is normative (raw) and ipsatized # compare the correlations based on raw vs. ipsatized data expSampleCor696ip <- cor(expSampleTemp696ip, use="pairwise") cordiff2 <- expSampleCor696raw-expSampleCor696ip # how different are they? diag(cordiff2) <- NA # some of the 'max' values here are quite large! descordiff2 <- psych::describe(cordiff2)[1:8,c(1:4,8:10,13)] descordiff2 @ \end{scriptsize} At first glance, the max and min values for the differences in correlations between raw and ipsatized data seem non-trivial. Further analyses (see below) indicate that the differences are normally distributed with a mean that is close to zero --- but not quite. Keep in mind that we are comparing a lot of correlations (\Sexpr{prettyNum(((696*696)-696)/2, big.mark = ",")}). \begin{scriptsize} <>= # how many differences are greater than |0.2| in magnitude? sum(abs(cordiff2) > .20, na.rm=TRUE) # greater than |0.15| in magnitude? sum(abs(cordiff2) > .15, na.rm=TRUE) # what's the mean difference? cordiff2plot <- cordiff2 mean(cordiff2plot, na.rm=TRUE) # make a frequency plot of the differences cordiff2plot[upper.tri(cordiff2plot, diag=TRUE)] <- NA pdf(paste(filepathfig, "rawVipCorsE.pdf", sep = ""), onefile = TRUE) plot(density(cordiff2plot, na.rm=TRUE), main="", ylab="Frequency", xlab="Differences in correlations", xlim=c(-.2,.2)) dev.off() @ <>= # make a plot for the replication differences too. # focus only on the personality items repSampleTemp696 <- subset(repSample, select = DataItems) repSampleCor696raw <- cor(repSampleTemp696, use="pairwise") # standardize the rows and show some repSampleTemp696ip <- ipsatize(repSampleTemp696) # confirm that the sum of the row means (rounded to the 5th digit) is zero isTRUE(sum(round(rowMeans(repSampleTemp696ip, na.rm=TRUE),5), na.rm=TRUE)==0) head(repSampleTemp696ip[,1:7]) psych::describe(repSampleTemp696ip)[1:7,] # compare the correlations based on raw vs. ipsatized data repSampleCor696ip <- cor(repSampleTemp696ip, use="pairwise") cordiff3 <- repSampleCor696raw-repSampleCor696ip # how different are they? diag(cordiff3) <- NA descordiff3 <- psych::describe(cordiff3)[c(1:5,8:10),c(1:5, 8:10)] # how many differences are greater than |0.2| in magnitude? sum(abs(cordiff3) > .20, na.rm=TRUE) # greater than |0.15| in magnitude? sum(abs(cordiff3) > .15, na.rm=TRUE) # what's the mean difference? cordiff3plot <- cordiff3 mean(cordiff3plot, na.rm=TRUE) # make a frequency plot of the differences cordiff3plot[upper.tri(cordiff3plot, diag=TRUE)] <- NA pdf(paste(filepathfig, "rawVipCorsR.pdf", sep = ""), onefile = TRUE) plot(density(cordiff3plot, na.rm=TRUE), main="", ylab="Frequency", xlab="Differences in correlations", xlim=c(-.2,.2)) dev.off() # and a plot for the confirmatory differences # focus only on the personality items confSampleTemp696 <- subset(confSample, select = DataItems) confSampleCor696raw <- cor(confSampleTemp696, use="pairwise") # standardize the rows and show some confSampleTemp696ip <- ipsatize(confSampleTemp696) # confirm that the sum of the row means (rounded to the 5th digit) is zero isTRUE(sum(round(rowMeans(confSampleTemp696ip, na.rm=TRUE),5), na.rm=TRUE)==0) head(confSampleTemp696ip[,1:7]) psych::describe(confSampleTemp696ip)[1:7,] # compare the correlations based on raw vs. ipsatized data confSampleCor696ip <- cor(confSampleTemp696ip, use="pairwise") cordiff4 <- confSampleCor696raw-confSampleCor696ip # how different are they? diag(cordiff4) <- NA descordiff4 <- psych::describe(cordiff4)[c(1:5,8:10),c(1:5, 8:10)] # how many differences are greater than |0.2| in magnitude? sum(abs(cordiff4) > .20, na.rm=TRUE) # greater than |0.15| in magnitude? sum(abs(cordiff4) > .15, na.rm=TRUE) # what's the mean difference? cordiff4plot <- cordiff3 mean(cordiff4plot, na.rm=TRUE) # make a frequency plot of the differences cordiff4plot[upper.tri(cordiff4plot, diag=TRUE)] <- NA pdf(paste(filepathfig, "rawVipCorsC.pdf", sep = ""), onefile = TRUE) plot(density(cordiff4plot, na.rm=TRUE), main="", ylab="Frequency", xlab="Differences in correlations", xlim=c(-.2,.2)) dev.off() @ \end{scriptsize} \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/rawVipCorsE.pdf}}{\caption{Exploratory sample raw minus ipsatized correlations}\label{fig:rawVipE}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/rawVipCorsR.pdf}}{\caption{Replication sample raw minus ipsatized correlations}\label{fig:rawVipR}} \end{floatrow} \end{figure} \begin{figure} \begin{center} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/rawVipCorsC.pdf}}{\caption{Confirmatory sample raw minus ipsatized correlations}\label{fig:rawVipC}} \end{floatrow} \end{center} \end{figure} As can be seen in Figures \ref{fig:rawVipE} through \ref{fig:rawVipC}, the primary difference between the matrices is that the raw correlations are slightly more positive than the ipsatized (i.e., they are shifted by about .03). This phenomenon is not only well-documented but also has led to some confusion about the merits of ipsatization \citep{TenBerge:1999bg, Saucier:2002wx}. It seems that this shift is to be expected in cases where the correlations are generally positive --- as is the case when there are an unbalanced number of positively keyed items. At this point, the question of positively or negatively keyed items does not mean much (because we have not identified the factors), but it does mean that there is a general factor in the data which is related to directionality. Some have suggested that this is an indication of acquiescence, others as a general factor of personality (I believe this is a confused interpretation [see \citealp{Revelle:2013hs} for more discussion]). The question here is what to do about it --- some claim that I should remove this within-person variance because I'm attempting to identify a model of individual differences for which only between-person variance is relevant. Others feel that removing this variance is a mistake because the extent to which this variance may be substantive can vary depending on the items/constructs being measured and, in any case, this may itself be an important source of individual differences. In other words, we don't always know what is causing the within-person variance; the phenomenon that precipitates the need for ipsatizing is some combination of response style, acquiescence and social desirability (and maybe other things) and some of these related but inequivalent issues might be relevant to differences between individuals. Agreeableness for example may be a trait that intrinsically reflects the tendency for naysaying. Similarly, extraverts may have more extreme response styles. Reducing the variance through ipsatization might, in theory, alter the structure of the correlations in a way that reduces these effects, but its not clear whether this is justifiable or even a good idea. Our conclusion is that ipsatizing does not seem to have much effect on the correlations except to make them slightly less positive (by about .03). It is difficult to precisely evaluate the uniformity of this effect, but the plots suggest that the differences are approximately randomly distributed (if anything, skewed towards zero). In other words, there is no more than a trivial effect on the stability of these correlation matrices. I will use the normative, non-ipsatized data for all subsequent analyes. There is need for more research in this area; interested readers are directed to \citet{Saucier:2002wx}, \citet{TenBerge:1999bg}, and \citet{Hofstee:98} for more information. \subsection{Correlation matrices: Pearson vs polychoric} \label{sec:pearsonpoly} The second step in our sensitivity analyses is to consider the impact of using the Pearson method for correlations versus polychoric correlations. While polychorics may be theoretically preferred, the extent to which they differ from the Pearson method is unclear and, based on prior experience, there is a practical reason for using Pearson correlations --- it is much, much faster to calculate the Pearson correlations for data sets of this size. Of course, I would not be comparing these two options if I had not been convinced that it is theoretically acceptable to use the Pearson method with only 6 response categories. See \citet{Rhemtulla:2012jv} for excellent treatment of this topic. Empirical evaluation requires us to get the polychoric correlation matrices for both samples in order to compare them against the Pearson. While I'm at it, I will also estimate the confidence intervals of the correlations using bootstrapping procedures (see `cor.ci()' for more information). The code given below calculates the polychoric correlations, then gets the confidence intervals for the polychoric correlations, and then gets the confidence intervals for the Pearson correlations for additional comparison. It should be noted that it takes a lot of processing time to get the polychorics and confidence intervals. In fact, I can only run these analyses on the computing cluster hosted by my institution. The code for these analyses is given in the rnw file (not shown in the PDF) but it is not evaluated in normal runs of the knitr file. The following code chunk shows a comparison of the two correlation matrices (Pearson vs polychoric) for the Exploratory sample as an example. \begin{scriptsize} <>= # keep eval=FALSE # This section of code is very computationally intensive. # I run this on the computing cluster and have made the # output available in the shared data file at: # http://dx.doi.org/10.7910/DVN/YF6STI suppressPackageStartupMessages(library(parallel)) # load(paste(filepathdata, "descriptivesData.Rdata", sep = "")) # when using the off-site computing cluster, the following load command was needed: load("/sscc/.../batch/descriptivesData.Rdata") rm(list=setdiff(ls(), c("DataItems", "expSample"))) gc() expSampleTemp696 <- subset(expSample, select = DataItems) rm(expSample, DataItems) gc() expSampleCorCI <- cor.ci(expSampleTemp696, p=.05, overlap=FALSE, plot = FALSE, poly = FALSE) rm(expSampleTemp696) gc() # save(list=ls(), file=paste(filepathdata, "expSampledata1.rdata", sep = "")) save(list=ls(), file="/sscc/.../batch/expSampledata1.rdata") rm(list=ls()) gc() # load(paste(filepathdata, "descriptivesData.Rdata", sep = "")) # when using the off-site computing cluster, the following load command was needed: load("/sscc/.../batch/descriptivesData.Rdata") rm(list=setdiff(ls(), c("DataItems", "repSample"))) gc() repSampleTemp696 <- subset(repSample, select = DataItems) rm(repSample, DataItems) gc() repSampleCorCI <- cor.ci(repSampleTemp696, p=.05, overlap=FALSE, plot = FALSE, poly = FALSE) rm(repSampleTemp696) gc() #save(list=ls(), file=paste(filepathdata, "repSampledata1.rdata", sep = "")) save(list=ls(), file="/sscc/.../batch/repSampledata1.rdata") rm(list=ls()) gc() # load(paste(filepathdata, "descriptivesData.Rdata", sep = "")) # when using the off-site computing cluster, the following load command was needed: load("/sscc/.../batch/descriptivesData.Rdata") rm(list=setdiff(ls(), c("DataItems", "confSample"))) gc() confSampleTemp696 <- subset(confSample, select = DataItems) rm(confSample, DataItems) gc() confSampleCorCI <- cor.ci(confSampleTemp696, p=.05, overlap=FALSE, plot = FALSE, poly = FALSE) rm(confSampleTemp696) gc() #save(list=ls(), file=paste(filepathdata, "confSampledata1.rdata", sep = "")) save(list=ls(), file="/sscc/.../batch/confSampledata1.rdata") rm(list=ls()) gc() #load(paste(filepathdata, "descriptivesData.Rdata", sep = "")) # when using the off-site computing cluster, the following load command was needed: load("/sscc/.../batch/descriptivesData.Rdata") rm(list=setdiff(ls(), c("DataItems", "expSample", "repSample", "confSample"))) gc() bothSample <- rbind(expSample, repSample) allSample <- rbind(bothSample, confSample) rm(list=setdiff(ls(), c("DataItems", "allSample"))) gc() allSampleTemp696 <- subset(allSample, select = DataItems) rm(allSample, DataItems) gc() allSampleCorCI <- cor.ci(allSampleTemp696, p=.05, overlap=FALSE, plot = FALSE, poly = FALSE) rm(allSampleTemp696) gc() #save(list=ls(), file=paste(filepathdata, "allSampledata.rdata", sep = "")) save(list=ls(), file="/sscc/.../batch/allSampledata.rdata") rm(list=ls()) gc() #load(paste(filepathdata, "descriptivesData.Rdata", sep = "")) # when using the off-site computing cluster, the following load command was needed: load("/sscc/.../batch/descriptivesData.Rdata") rm(list=setdiff(ls(), c("DataItems", "expSample"))) gc() expSampleTemp696 <- subset(expSample, select = DataItems) rm(expSample, DataItems) gc() expSamplePolyCor <- polychoric(expSampleTemp696, smooth=TRUE) rm(expSampleTemp696) gc() #save(list=ls(), file=paste(filepathdata, "expSampledata2.rdata", sep = "")) save(list=ls(), file="/sscc/.../batch/expSampledata2.rdata") rm(list=ls()) gc() #load(paste(filepathdata, "descriptivesData.Rdata", sep = "")) # when using the off-site computing cluster, the following load command was needed: load("/sscc/.../batch/descriptivesData.Rdata") rm(list=setdiff(ls(), c("DataItems", "repSample"))) gc() repSampleTemp696 <- subset(repSample, select = DataItems) rm(repSample, DataItems) gc() repSamplePolyCor <- polychoric(repSampleTemp696, smooth=TRUE) rm(repSampleTemp696) gc() #save(list=ls(), file=paste(filepathdata, "repSampledata2.rdata", sep = "")) save(list=ls(), file="/sscc/.../batch/repSampledata2.rdata") rm(list=ls()) gc() #load(paste(filepathdata, "descriptivesData.Rdata", sep = "")) # when using the off-site computing cluster, the following load command was needed: load("/sscc/.../batch/descriptivesData.Rdata") rm(list=setdiff(ls(), c("DataItems", "confSample"))) gc() confSampleTemp696 <- subset(confSample, select = DataItems) rm(confSample, DataItems) gc() confSamplePolyCor <- polychoric(confSampleTemp696, smooth=TRUE) rm(confSampleTemp696) gc() #save(list=ls(), file=paste(filepathdata, "confSampledata2.rdata", sep = "")) save(list=ls(), file="/sscc/.../batch/confSampledata2.rdata") rm(list=ls()) gc() #load(paste(filepathdata, "expSampledata1.rdata", sep = "")) load("/sscc/.../batch/expSampledata1.rdata") #load(paste(filepathdata, "expSampledata2.rdata", sep = "")) load("/sscc/.../batch/expSampledata2.rdata") #load(paste(filepathdata, "repSampledata1.rdata", sep = "")) load("/sscc/.../batch/repSampledata1.rdata") #load(paste(filepathdata, "repSampledata2.rdata", sep = "")) load("/sscc/.../batch/repSampledata2.rdata") #load(paste(filepathdata, "confSampledata1.rdata", sep = "")) load("/sscc/.../batch/confSampledata1.rdata") #load(paste(filepathdata, "confSampledata2.rdata", sep = "")) load("/sscc/.../batch/confSampledata2.rdata") #load(paste(filepathdata, "allSampledata.rdata", sep = "")) load("/sscc/.../batch/allSampledata.rdata") #save(list=ls(), file=paste(filepathdata, "SPIcorCI.Rdata", sep = "")) save(list=ls(), file="/sscc/.../batch/SPIcorCI.Rdata") @ \end{scriptsize} \begin{scriptsize} <>= # Clean up again; keep the normative correlation matrices rm(list=setdiff(ls(), c("confSample", "confSampleCor696raw", "DataItems", "DemoItems", "expSample", "expSampleCor696raw", "filepathdata", "filepathfig", "ItemInfo696", "pwiseAdmins1C", "pwiseAdmins1E", "pwiseAdmins1R", "repSample", "repSampleCor696raw"))) load(paste(filepathdata, "SPIcorCI.RData", sep = "")) # Evaluate the difference between the normative Pearsons and the normative polychorics cordiff5 <- expSampleCor696raw-expSamplePolyCor$rho # how different are they? diag(cordiff5) <- NA descordiff5 <- psych::describe(cordiff5)[1:8,c(1:4,8:10,13)] descordiff5 # how many differences are greater than |0.2| in magnitude? sum(abs(cordiff5) > .20, na.rm=TRUE) # out of how many? ((696^2)-696)/2 # greater than |0.15| in magnitude? sum(abs(cordiff5) > .15, na.rm=TRUE) # what's the mean difference? cordiff5plot <- cordiff5 mean(cordiff5plot, na.rm=TRUE) # make a frequency plot of the differences cordiff5plot[upper.tri(cordiff5plot, diag=TRUE)] <- NA pdf(paste(filepathfig, "rawVpolyCorsE.pdf", sep = ""), onefile = TRUE) plot(density(cordiff5plot, na.rm=TRUE), main="", ylab="Frequency", xlab="Differences in correlations", xlim=c(-.2,.2), ylim=c(0,30)) dev.off() @ <>= # Repeat for the Replication sample cordiff6 <- repSampleCor696raw-repSamplePolyCor$rho # how different are they? diag(cordiff6) <- NA descordiff6 <- psych::describe(cordiff6)[1:8,c(1:4,8:10,13)] descordiff6 # how many differences are greater than |0.2| in magnitude? sum(abs(cordiff6) > .20, na.rm=TRUE) # out of how many? ((696^2)-696)/2 # greater than |0.15| in magnitude? sum(abs(cordiff6) > .15, na.rm=TRUE) # what's the mean difference? cordiff6plot <- cordiff6 mean(cordiff6plot, na.rm=TRUE) # make a frequency plot of the differences cordiff6plot[upper.tri(cordiff6plot, diag=TRUE)] <- NA pdf(paste(filepathfig, "rawVpolyCorsR.pdf", sep = ""), onefile = TRUE) plot(density(cordiff5plot, na.rm=TRUE), main="", ylab="Frequency", xlab="Differences in correlations", xlim=c(-.2,.2), ylim=c(0,30)) dev.off() # Repeat for the Confirmatory sample cordiff7 <- confSampleCor696raw-confSamplePolyCor$rho # how different are they? diag(cordiff7) <- NA descordiff7 <- psych::describe(cordiff7)[1:8,c(1:4,8:10,13)] descordiff7 # how many differences are greater than |0.2| in magnitude? sum(abs(cordiff7) > .20, na.rm=TRUE) # out of how many? ((696^2)-696)/2 # greater than |0.15| in magnitude? sum(abs(cordiff7) > .15, na.rm=TRUE) # what's the mean difference? cordiff7plot <- cordiff7 mean(cordiff7plot, na.rm=TRUE) # make a frequency plot of the differences cordiff7plot[upper.tri(cordiff7plot, diag=TRUE)] <- NA pdf(paste(filepathfig, "rawVpolyCorsC.pdf", sep = ""), onefile = TRUE) plot(density(cordiff7plot, na.rm=TRUE), main="", ylab="Frequency", xlab="Differences in correlations", xlim=c(-.2,.2), ylim=c(0,30)) dev.off() @ \end{scriptsize} The mean difference between the Pearson and polychoric correlations is \Sexpr{prettyNum(round(mean(cordiff5plot, na.rm=TRUE),4), big.mark=",", scientific=F)} for all correlations in the Exploratory sample, \Sexpr{prettyNum(round(mean(cordiff6plot, na.rm=TRUE),4), big.mark=",", scientific=F)} in the Replication sample, and \Sexpr{prettyNum(round(mean(cordiff7plot, na.rm=TRUE),4), big.mark=",", scientific=F)} in the Confirmatory sample. Figures \ref{fig:PearsonVpolyE} through \ref{fig:PearsonVpolyC} suggest that the differences are randomly distributed around zero. The two methods produce very similar correlation matrices and the similarity increases with larger sample sizes. Given this, I will use the Pearson correlations going forward with confidence that the results would be very similar if the polychorics were used instead. Researchers interested in re-using these data might consider following up on analyses of this sort to evaluate the differences in the two approaches with greater specificity. \begin{figure} \begin{center} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/rawVpolyCorsE.pdf}}{\caption{Exploratory sample Pearson minus polychoric correlations}\label{fig:PearsonVpolyE}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/rawVpolyCorsR.pdf}}{\caption{Replication sample Pearson minus polychoric correlations}\label{fig:PearsonVpolyR}} \end{floatrow} \end{center} \end{figure} \begin{figure} \begin{center} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/rawVpolyCorsC.pdf}}{\caption{Confirmatory sample Pearson minus polychoric correlations}\label{fig:PearsonVpolyC}} \end{floatrow} \end{center} \end{figure} \subsection{Correlation matrices: smoothed vs non-smoothed} The correlation matrix based on Pearson's method will need to be smoothed in order to move ahead with the structural analyses. This is because some of the 696 items are fully predicted by combinations of other variables. In some simple cases, this is due to the presence of highly correlated doublets and triplets, though in more complicated cases it may be due to larger sets of highly correlated groups of items. These high correlations can lead to (slightly) negative eigenvalues, which makes it impossible to derive chi-square based fit statistics. Smoothing the correlation matrices is standard practice in most statistical software programs but the effects are rarely, if ever, evaluated or even made clear to the researchers whose data are affected by this practice. Here I evaluate the difference between smoothed and unsmoothed matrices. \begin{scriptsize} <>= # this will give warnings - as it should! expCor696rawSmoothed <- cor.smooth(expSampleCor696raw) expCor696rawUnsmoothed <- expSampleCor696raw # another warning is in order repCor696rawSmoothed <- cor.smooth(repSampleCor696raw) repCor696rawUnsmoothed <- repSampleCor696raw # and another warning confCor696rawSmoothed <- cor.smooth(confSampleCor696raw) confCor696rawUnsmoothed <- confSampleCor696raw cordiff8 <- expCor696rawSmoothed-expCor696rawUnsmoothed # how different are they? diag(cordiff8) <- NA descordiff8 <- psych::describe(cordiff8)[1:8,c(1:4,8:10,13)] descordiff8 # how many differences are greater than |0.2| in magnitude? sum(abs(cordiff8) > .20, na.rm=TRUE) # out of how many? ((696^2)-696)/2 # greater than |0.15| in magnitude? sum(abs(cordiff8) > .15, na.rm=TRUE) # what's the mean difference? cordiff8plot <- cordiff8 mean(cordiff8plot, na.rm=TRUE) # make a frequency plot of the differences cordiff8plot[upper.tri(cordiff8plot, diag=TRUE)] <- NA pdf(paste(filepathfig, "smoothedVunsmoothedE.pdf", sep = ""), onefile = TRUE) plot(density(cordiff8plot, na.rm=TRUE), main="", ylab="Frequency", xlab="Differences in correlations", xlim=c(-.2,.2), ylim=c(0,30)) dev.off() @ <>= # Repeat for the Replication sample cordiff9 <- repCor696rawSmoothed-repCor696rawUnsmoothed # how different are they? diag(cordiff9) <- NA descordiff9 <- psych::describe(cordiff9)[1:8,c(1:4,8:10,13)] descordiff9 # how many differences are greater than |0.2| in magnitude? sum(abs(cordiff9) > .20, na.rm=TRUE) # out of how many? ((696^2)-696)/2 # greater than |0.15| in magnitude? sum(abs(cordiff9) > .15, na.rm=TRUE) # what's the mean difference? cordiff9plot <- cordiff9 mean(cordiff9plot, na.rm=TRUE) # make a frequency plot of the differences cordiff9plot[upper.tri(cordiff9plot, diag=TRUE)] <- NA pdf(paste(filepathfig, "smoothedVunsmoothedR.pdf", sep = ""), onefile = TRUE) plot(density(cordiff9plot, na.rm=TRUE), main="", ylab="Frequency", xlab="Differences in correlations", xlim=c(-.2,.2), ylim=c(0,30)) dev.off() # Repeat for the Confirmatory sample cordiff10 <- confCor696rawSmoothed-confCor696rawUnsmoothed # how different are they? diag(cordiff10) <- NA descordiff10 <- psych::describe(cordiff10)[1:8,c(1:4,8:10,13)] descordiff10 # how many differences are greater than |0.2| in magnitude? sum(abs(cordiff10) > .20, na.rm=TRUE) # out of how many? ((696^2)-696)/2 # greater than |0.15| in magnitude? sum(abs(cordiff10) > .15, na.rm=TRUE) # what's the mean difference? cordiff10plot <- cordiff10 mean(cordiff10plot, na.rm=TRUE) # make a frequency plot of the differences cordiff10plot[upper.tri(cordiff10plot, diag=TRUE)] <- NA pdf(paste(filepathfig, "smoothedVunsmoothedC.pdf", sep = ""), onefile = TRUE) plot(density(cordiff10plot, na.rm=TRUE), main="", ylab="Frequency", xlab="Differences in correlations", xlim=c(-.2,.2), ylim=c(0,30)) dev.off() @ \end{scriptsize} \begin{figure} \begin{center} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/smoothedVunsmoothedE.pdf}}{\caption{Exploratory sample smoothed minus unsmoothed correlations}\label{fig:smoothedVunsmoothedE}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/smoothedVunsmoothedR.pdf}}{\caption{Replication sample smoothed minus unsmoothed correlations}\label{fig:smoothedVunsmoothedR}} \end{floatrow} \end{center} \end{figure} \begin{figure} \begin{center} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/smoothedVunsmoothedC.pdf}}{\caption{Confirmatory sample smoothed minus unsmoothed correlations}\label{fig:smoothedVunsmoothedC}} \end{floatrow} \end{center} \end{figure} The differences between the smoothed and unsmoothed correlation matrices are surprisingly similar to the differences between the Pearson and polychoric matrices --- and similarly small. The mean difference in the Exploratory sample is \Sexpr{round(mean(cordiff8plot, na.rm=TRUE),4)}, the mean difference in the Replication sample is \Sexpr{round(mean(cordiff9plot, na.rm=TRUE),4)}, and \Sexpr{round(mean(cordiff10plot, na.rm=TRUE),4)} in the Confirmatory sample. Figures \ref{fig:smoothedVunsmoothedE} through \ref{fig:smoothedVunsmoothedC} suggest that the differences are randomly distributed around zero.\\ It is theoretically possible to eliminate the need for smoothing by dropping one (or more) of the items in highly correlated sets. By way of anecdote, I have tried to do this in the Exploratory sample by dropping one item from pairs of items that are correlated above an arbitrary threshold (e.g., 0.95). This invariably altered the correlation matrix in such a way that additional items needed to be removed. By the time several items had been removed, I became concerned that the resulting correlation matrix would be meaningfully theoretically different from the original matrix. As a digression, this phenomena provides useful insight into the nature of dependency in the universe of personality items. As \citet{Goldberg:92} noted: ``like the stars in the sky... there are some more densely concentrated regions, some semantic galaxies, within the widespread distribution of personality descriptors.'' We observe the same situation with these 696 phrased personality items. I don't know where one would begin if they hoped to disentangle whether these concentrated regions result from bias in measurement (e.g., non-representative sampling of items from the extant and potential universe of personality items [\citealp{Goldberg:92}]), a bias caused by observability (see topics related to ``self-other knowledge assymetry'' [\citealp{Vazire:10}]), a reflection of underlying biologically/socially-induced mechanisms \citep{DeYoung:14b, Revelle:2015hg}, some combination of these factors, or others which are not listed here. Though fascinating to consider, these theoretical issues are well beyond the scope of the current work. Still, it is important to acknowledge that the items are not uniformly distributed across the multidimensional personality space. This is a consequence of the item selection procedures described in Section \ref{sec:IPIPselection} and will be important again in Section \ref{sec:rotation} on rotational methods. In any case, the smoothed correlation matrix is necessary to continue with these analyses, so that's what I'll be using. It is reassuring to note that it is very nearly identical to the unsmoothed matrix. \subsection{Factoring techniques: principal components analysis vs factor analysis} The difference between principal components analysis and common factor analysis (typically referred to as merely factor analysis, though often an important differentiation is made between exploratory and confirmatory factor analysis) is both theoretically interesting and practically inconsequential. The practical inconsequence is shown in the code below, where I compare 3 examples of exploratory factor analysis and principal components analysis using randomly generated extraction levels. \begin{scriptsize} <>= rm(list=setdiff(ls(), c("confSample", "confCor696rawSmoothed", "DataItems", "DemoItems", "expSample", "expCor696rawSmoothed", "filepathdata", "filepathfig", "ItemInfo696", "pwiseAdmins1C", "pwiseAdmins1E", "pwiseAdmins1R", "repSample", "repCor696rawSmoothed"))) # identify three levels to extract from 1 to the max # we do this at random as a precaution against manipulation randomExtraction <- sample(1:dim(expCor696rawSmoothed)[1], 3) # what are they randomExtraction # get the PCA solutions example1PC <- principal(expCor696rawSmoothed, randomExtraction[1], n.obs = pwiseAdmins1E$mean, rotate = "none") example2PC <- principal(expCor696rawSmoothed, randomExtraction[2], n.obs = pwiseAdmins1E$mean, rotate = "none") example3PC <- principal(expCor696rawSmoothed, randomExtraction[3], n.obs = pwiseAdmins1E$mean, rotate = "none") # and the FA solutions example1FA <- fa(expCor696rawSmoothed, randomExtraction[1], np.obs = pwiseAdmins1E$mean, rotate = "none") example2FA <- fa(expCor696rawSmoothed, randomExtraction[2], np.obs = pwiseAdmins1E$mean, rotate = "none") example3FA <- fa(expCor696rawSmoothed, randomExtraction[3], np.obs = pwiseAdmins1E$mean, rotate = "none") loadingsDiffexample1 <- example1PC$loadings[1:696, 1:randomExtraction[1]]-example1FA$loadings[1:696, 1:randomExtraction[1]] max(abs(loadingsDiffexample1)) mean(loadingsDiffexample1) loadingsDiffexample2 <- example2PC$loadings[1:696, 1:randomExtraction[2]]-example2FA$loadings[1:696, 1:randomExtraction[2]] max(abs(loadingsDiffexample2)) mean(loadingsDiffexample2) loadingsDiffexample3 <- example3PC$loadings[1:696, 1:randomExtraction[3]]-example3FA$loadings[1:696, 1:randomExtraction[3]] max(abs(loadingsDiffexample3)) mean(loadingsDiffexample3) @ \end{scriptsize} The examples above demonstrate that the loadings are basically identical between the factor analytic and principal components approaches. This is, by far, the most relevant point for our purposes --- it doesn't matter much. It should be noted that this is not \textit{always} the case. See \citet{Fabrigar:1999tr} for more information about the circumstances that affect the magnitude of the differences in results. The theoretically interesting angle comes, in part, from recognition that the code given above invokes the option that `rotate=``none''' for both the factor analyses and the principal components analyses. For both principal components and factor analytic procedures, the rotation of the solution can meaningfully change the interpretability of the solution (more on this in Section \ref{sec:rotation}), but for principal components analysis it makes things really confusing. Rotation of the PCA solution leads to rotated components which are intrinsically different from the principal components (e.g., they're not \textit{principal} components). Let's back up. There are a huge number of psychometric particulars which might be included here but are not particularly relevant to the current work (see \citealp{Comrey:1988wl} for an introduction to the topic; more enthusiastic readers are directed to \citealp{Goldberg:2006wj} and the special issue in \textit{Multivariate Behavioral Research} from 1990). Still, there are a couple of good reasons for trying to understand the theoretical differences even though the practical consequences are not substantial. One good reason is that the default methods in the most widely-used statistical packages are inconsistent; SPSS, for example, performs a PCA when users call for a factor analysis. Another good reason is that principal components analyses were widely used among personality psychologists during (and prior to) the identification of the Big Five/Five Factor Model.\footnote{It's not entirely clear (to the author) why this is the case, though part of the reason may be driven by the fact that PCA is less computationally intensive than factor analysis \citep{Velicer:1990ew, Fabrigar:1999tr}, and this was likely important before the widespread availability of sufficiently powerful computers. It's hard to believe, but for at least 50 years before the mid-1980s or so, researchers computed large factor analyses by hand or with primitive computing technology. Computational difficulty was a very important concern. Obviously, any differences in difficulty between the two approaches are now rendered moot by modern statistical software. I have encountered other arguments in favor of PCA (i.e., factor analyses occasionally produce Heywood cases, etc., [\citealp{Gorsuch:97b}]) but none of these are particularly compelling.} In terms of theoretical rationale, the factor analytic approach should be preferred. This is because factor analysis begins with the presumption that the observed variables are caused by latent factors. So, it allows for \textit{exploration} of the extent to which variance is common to multiple items. PCA, by contrast, does not presume this; it's really just intended as a means of data reduction via the extraction of linear composites of \textit{observed} variables. The biggest practical difference between the two is that decomposition of the correlation matrix with PCA will take into account the full, total variance (common and specific), whereas with factor analysis we identify factors which explain the common variance. In both cases, rotation will improve the interpretability of these approaches --- for factor analysis, this is achieved without changing the meaning of the factors but for PCA... once you have rotated the components to make them more interpretable, the meaning of the results becomes murky. In any case, we are primarily concerned with the loadings and less concerned with the (total) variance explained. I illustrate some of the more rudimentary differences between the two approaches below by demonstrating that the eigenvalues from the prinicipal components are identical despite the number of components extracted (as they should be --- this is fundamental) whereas they differ slightly depending on the number of factors extracted. Also the sum of the eigenvalues in all solutions is equal to the number of items (this is also fundamental as it is merely the sum of the diagonal; all 1's because they include the common and specific variance). The sum of the eigenvalues for the factor analysis solution only sum to the portion of variance described by the common factor variance; this would be a proportion of the total variance if divided by the number of items. \begin{scriptsize} <>= example1PC$values[1:30] example2PC$values[1:30] example3PC$values[1:30] sum(example1PC$values) sum(example2PC$values) sum(example3PC$values) example1FA$values[1:30] sum(example1FA$values) example2FA$values[1:30] sum(example2FA$values) example3FA$values[1:30] sum(example3FA$values) @ \end{scriptsize} To summarize, the distinction between principal components and factor analysis doesn't matter much for our purposes and the difference is rendered even less meaningful when we consider the fact that we will want to rotate the loadings in either case. I proceed by using the factor analytic approach. \subsection{Factoring techniques: factoring methods} Before considering rotational techniques, we must first distinguish among factoring methods. There are many options; these are often confused with rotational methods and the differences between factoring methods are perceived to be highly technical. This perception is accurate --- they are nerdy technical. Essentially, the goal of factor analysis (and principal component analysis) is to identify the factors (or components) that minimize the differences between the correlation matrix that is put into the computational function and a more compact model of the correlations with the specified number of dimensions. In other words, the function calls for reduction of the \textit{given} correlation matrix to a smaller factor matrix with \textit{n} dimensions where \textit{n} is the number of factors being extracted. Many different methods have been developed to accomplish this matrix algebra magic and they are referred to by the criteria on which they focus. These are the most prominently used:\\ \begin{itemize} \vspace{-4mm} \item ``ordinary least squares'' (aka ``minres'', aka ``unweighted least squares'', aka ``minimized residuals'') aims to minimize the residuals between the correlations given and those which are reproduced by the factor matrix\\ \vspace{-4mm} \item ``generalized least squares'' (sometimes aka ``weighted least squares'') minimizes the residuals as above but also disproportionately weights correlation coefficients with high communality (in the current iteration of the algorithm)\\ \vspace{-4mm} \item ``generalized weighted least squares'' is a special case of generalized least squares that is useful when the variances of the observations are (highly) unequal (i.e., the case of heteroscedasticity)\\ \vspace{-4mm} \item ``maximum likelihood'' assumes that the residuals between the given correlation matrix and the model are normally distributed around zero\\ \end{itemize} Practically speaking, the extent to which these factoring methods affect the results is relatively small, as illustrated in the following examples. \begin{scriptsize} <>= # get some more random numbers (between 2 and 10) to use as extraction levels more_randomExtraction <- sample(2:10, 2) # what are they? more_randomExtraction # show some examples of different factoring methods: # minres aka ordinary least squares example4FA <- fa(expCor696rawSmoothed, more_randomExtraction[1], np.obs = pwiseAdmins1E$mean, fm = "minres") # weighted least squares example5FA <- fa(expCor696rawSmoothed, more_randomExtraction[1], np.obs = pwiseAdmins1E$mean, fm = "wls") # generalized weighted least squares example6FA <- fa(expCor696rawSmoothed, more_randomExtraction[1], np.obs = pwiseAdmins1E$mean, fm = "gls") # maximum likelihood example7FA <- fa(expCor696rawSmoothed, more_randomExtraction[1], np.obs = pwiseAdmins1E$mean, fm = "ml") # compare minres loadings to wls loadings loadingsDiffexample4and5 <- example4FA$loadings[1:696, 1:more_randomExtraction[1]]-example5FA$loadings[1:696, 1:more_randomExtraction[1]] max(abs(loadingsDiffexample4and5)) mean(loadingsDiffexample4and5) # compare minres loadings to gls loadings loadingsDiffexample4and6 <- example4FA$loadings[1:696, 1:more_randomExtraction[1]]-example6FA$loadings[1:696, 1:more_randomExtraction[1]] max(abs(loadingsDiffexample4and6)) mean(loadingsDiffexample4and6) # compare minres loadings to ml loadings loadingsDiffexample4and7 <- example4FA$loadings[1:696, 1:more_randomExtraction[1]]-example7FA$loadings[1:696, 1:more_randomExtraction[1]] max(abs(loadingsDiffexample4and7)) mean(loadingsDiffexample4and7) # compare wls loadings to gls loadings loadingsDiffexample5and6 <- example5FA$loadings[1:696, 1:more_randomExtraction[1]]-example6FA$loadings[1:696, 1:more_randomExtraction[1]] max(abs(loadingsDiffexample5and6)) mean(loadingsDiffexample5and6) # compare wls loadings to ml loadings loadingsDiffexample5and7 <- example5FA$loadings[1:696, 1:more_randomExtraction[1]]-example7FA$loadings[1:696, 1:more_randomExtraction[1]] max(abs(loadingsDiffexample5and7)) mean(loadingsDiffexample5and7) # compare gls loadings to ml loadings loadingsDiffexample6and7 <- example6FA$loadings[1:696, 1:more_randomExtraction[1]]-example7FA$loadings[1:696, 1:more_randomExtraction[1]] max(abs(loadingsDiffexample6and7)) mean(loadingsDiffexample6and7) @ \end{scriptsize} The maximum likelihood loadings are most different from the remaining three measures, but they still aren't much different. The mean difference in the loadings for maximum likelihood and the other methods is less than .0001 in all cases. The `gls' and `wls' options are exactly identical in this case and both are effectively identical to the `minres' approach. I use the minres approach with high confidence that the results would be very similar if maximum likelihood were used instead. \subsection{Factoring techniques: oblique vs orthogonal rotations} \label{sec:rotation} Where to begin? The decision to be made here is both theoretically and practically consequential and it is also closely tied to other critical topics like item sampling, validity, and --- believe it or not --- internal consistency (not to mention other factoring options like the potential use of principal components). And, despite the importance of this consideration, there is a historical lack of consensus about the best approach. It is not much of an overstatement to say that this issue is fundamentally important to this project, the personality inventory that results from it and this research area as a whole. As a psychometric topic, this can be generally understood without too much math. Rotation comes into play after the factors have been extracted. Sometimes, the initial solution is hard to interpret by virtue of the iterative procedures for extracting factors (the first factor is extracted and ``located'' based on the strength of covariation among the variables, then the second factor and so on). If rotations are allowed, then various methods can be used to re-position the factors (think vectors or axes) with the goal of making the solution more simple and clear. Mathematically, the solution is transformed to reduce the complexity of the solution by using one of many different algorithms. The biggest distinction among these algorithms relates to orthogonality \citep{Osborne:2015tc}. Orthogonal rotation algorithms force the factors to remain orthogonal; the factors tend to be orthogonal in the original, unrotated solution because of the iterative procedures by which factor analysis unfolds. Spatially, orthogonality means that all factors are at right angles to one another (think of the x, y, and z axes in 3 dimensional space); orthogonal factors are uncorrelated with one another. Oblique rotation algorithms do not force the factors to be orthogonal, though they also do not force the factors to be obliquely oriented. It is possible (and common) to get a roughly orthogonal or even perfectly orthogonal solution when using an oblique rotational algorithm.\footnote{Anecdotally, whenever I have seen someone claim that rotational method makes no difference in interpretability (see \citealp{Brown:2009vf}), it is because they are looking at factors which are largely uncorrelated. This suggests that their interpretation of the impact of rotation is biased due to extrapolation of cases where the impact is low.} The use of an oblique rotation just relaxes the constraint of orthogonality. One additional observation is worth making: the allowance of oblique rotations will not result in \textit{highly} correlated factors because such factors would be more likely to be joined in the original solution. Note the emphasis on highly. The extent to which they might be correlated will depend on the complexity of the solution. By this, I only mean that the extent to which the factors might be expected to correlate will increase as more and more factors are extracted (and, obviously, as the underlying variables demonstrate high correlations by virtue of being clumpy). Now, why does this matter? \citet{Thurstone:47} is the one who first strongly emphasized the usefulness of rotation and he is often cited as laying out the criteria for simple structure (read: low complexity). Not all of his criteria are strictly followed but they are frequently recited in the literature \citep{Carroll:53, Bernaards:2003wg, Pettersson:10, Pettersson:14}. Rotations make the output of factor analysis more meaningful and oblique rotations allow it to be even more meaningful than orthogonal rotations (recall that orthogonal solutions are possible outcomes of oblique rotation algorithms). Rotation improves the interpretability by altering the loadings of items on the factors. The mathematical formula for complexity is based on the loadings, so when the rotational algorithms are iterating through the options for minimizing complexity, they do so by simultaneously changing the loadings of the items (as a set, thanks to the magic of matrix algebra). Orthogonal rotations will minimize complexity with the constraint of orthogonality. Allowance for obliqueness will minimize complexity further. It is often claimed that this further reduction comes at a cost, that correlated factors are more difficult to interpret by virtue of being harder to distinguish in terms of content (because... they're correlated). If this seems a little circular, I agree! The whole point of reducing complexity --- the rationale for rotation --- is to improve interpretability! It seems to me that the most logical approach is to reduce complexity as much as possible with rotation (by allowing for correlated factors) and then evaluate the solution for interpretability. Some solutions just don't make sense. \textbf{From a psychometric perspective, the use of oblique rotational algorithms allows for less complex, more simple solutions and offers the potential to make solutions more interpretable.} From a theoretical perspective, the issue of factor rotation matters because it is my goal to improve upon existing assessment models of personality in terms of predictive utility. \citet[pp. 184-185,][]{Goldberg:93c} has described the essence of this goal based on the presumption of orthogonality and simple structure: \begin{blockquote} ``A perfect hierarchical model would consist of L higher order factors, M additional orthogonal common factors, each associated with a particular facet, and then K specific factors, one for each item. Every item would contain some variance associated with one or more of the L higher order factors, some additional variance associated with one of the M lower order factors, plus some specific variance of its own. The eigenvalues of the matrix of correlations among the items should show a break between the sizes of factors L and L + 1, and then again between Factors M and M + 1. All real data sets, however, will doubtless include variables with various combinations of the four types of variance [the 3 previously mentioned plus error], and thus there will rarely be any huge breaks in the sizes of the eigenvalues. ... It is for this reason that one must think hierarchically when using multiple regression analyses in applied prediction contexts. With any typical data matrix, as one’s sample size increases, one will profit by reaching down lower in the hierarchy to select one’s initial predictor variables. For small samples, one must stay with only the [highest order] variances of the highest order factor scores to avoid capitalizing on the chance vagaries of one’s derivation sample. For large samples, one should use the M facet scales because each contains some [additional] variance beyond the highest level factors. For populations (or huge samples), one should use the items because each item contains some [specific] variance beyond its [additional facet-level] variance and [highest order] variance. ... By partialing out sources of variance at successively lower levels, the aim is to arrive at unidimensional subsets of items, with each subset being orthogonal to each of the others. In such a model, all sources of common variance have been partialed out of the lowest level factors, and the resulting factor representation is that of perfect simple structure. In this idealized unidimensional model, there is no nonarbitrary basis for ordering the items that load on each lowest level factor.'' \end{blockquote} \noindent There is little basis to say anything with certainty regarding the true structure of human personality, but we should be willing to bet the house that it isn't simple. This confidence is partly grounded in considerable evidence against simple structure (see \citet{Condon:14Diss} and \citet{Pettersson:14} for more thorough discussion and evidence of moderate-to-high correlations between facets in the IPIP-NEO, the IPIP-HEXACO and the Big Five Aspect Scales) but, more than that, it's grounded in the astronomical odds against simple structure as the best option among the innumerable possibilities. Goldberg's ``perfect'' model is optimal for the unique case of very simple structure. But, \textit{in the absence of perfectly simple structure}, oblique solutions will tautologically be less complex than perfectly orthogonal solutions \citep{Osborne:2015tc, Fabrigar:1999tr}. The claims that Goldberg makes regarding the utility of hierarchical models are quite compelling as is his description of the most elegant hierarchical model possible, with orthogonal, very simple structure. But these two ideas are not necessarily related.\footnote{We don't believe that Goldberg was claiming this, though it's not clear. Readers should be warned that, if they choose to read this Goldberg article, they run the risk of feeling compelled to study personality structure for the rest of their lives.} The hierarchical approach remains incrementally useful for prediction and, if anything, the ability to dispense with orthogonality at each level makes his observations regarding increased predictive utility more true as the levels are, in some sense, liberated by the opportunity to be derived with structural independence. Previous attempts to identify hierarchical models of personality \citep{DeYoung:07, Ashton:07b, Costa:92a, Costa:95, McCrae:1992ur, Soto:09} have not generally capitalized on this opportunity.\footnote{\citet{Hofstee:92} did begin to approximate the fully oblique hierarchical model by integrating the circumplex and Big Five approaches. The AB5C model acknowledges the considerable covariation that occurs interstitial to the axes of the five factor solution.} In the case of the most widely used hierarchical model, the NEO-PI-R\texttrademark \citep{Costa:92a, McCrae:2010}, this is particularly curious because they \citep[pp. 230-231,][]{McCrae:1992ur} seem to acknowledge the logic: \begin{blockquote} ``The NEO-PI-R is based on an alternative, hierarchical, view of the five-factor model, in which individual traits at the lower level of the hierarchy are assumed to contain both common and specific variance. By measuring separate facets, additional information is thought to be obtained about the individual over and above that which could be obtained from the five common factors. In this view, the five factors themselves are not sufficient for a complete description of personality, except at the global, higher level of the hierarchy.'' \end{blockquote} Yet, the NEO-PI-R\texttrademark (and, similarly, the HEXACO facet structure and, to a lesser extent, the BFAS aspect structure) seems to be hung up on the emphasis on orthogonality at the highest domain level (5 for the NEO-PI-R\texttrademark and the BFAS and 6 for the HEXACO). Orthogonality is emphasized at the highest domain and the scores for this level are equivalent to the sum of the facet level scores that are theoretically proposed to underlie each domain \citep{Costa:95, McCrae:1992ur, Lee:04}. This means that, as a group, the items in each domain are theorized to be orthogonal to the items in all other domains. But, it's not clear what this means for the orientation of the facets/aspects. It does not mean that the facets in each group are orthogonal (as a group or independently) to the facets in the other domains, nor does it say anything about the orientation of the facets to one another (the BFAS aspects are orthogonal within each domain by design). There could in fact be substantial correlations between the facets, which might be misinterpreted as suggesting that these facets describe the interstitial space (between factors) better than the domains --- this is not true because the facets are comprised of the same items as the domains. \textbf{Theoretically, the facets/aspects of the NEO-PI-R\texttrademark, HEXACO, and BFAS are (1) almost certainly not orthogonal to other facets within or across their domains (this is an empirical question which can, has, and should continue to be evaluated); (2) unlikely to describe the variance of the personality space constituted by their items as well as an independent solution that is not tied to their higher level domains; and (3) inherently unable to include constructs which might fall ``beyond'' the higher level domains by virtue of the fact that the facets are derived solely using items which are highly loaded \textit{on} the highest level domains.}\footnote{Forgive the bold font but I can't help raising my voice a little. Argh. More spherical cows.} It is not clear why solutions at levels other than five (or six) need to be confined by reliance on orthogonality at the five/six factor level (or at any level). Indeed, oblique solutions should be as good or better at any level of factoring if the solutions at different levels are independent of orthogonality constraints at other levels. Note that this does not mean that solutions at all levels are equally valid, nor that forced orthogonality invalidates the NEO-PI-R\texttrademark, HEXACO, BFAS or any other solution. As \citet[p. 23,][]{Saucier:2002wx} explains (the emphases are his): ``One element of adequate validity is a congruence between what the instrument is \textit{believed} (or theorized) to measure and what the instrument is \textit{demonstrated} to measure. I noted earlier that the Big Five factors have been \textit{theorized} to be independent factors and that the failure of marker sets to \textit{demonstrate} mutually [sic] orthogonality among the factors creates a validity issue jointly for these marker sets and for the theoretical standpoints they are supposed to operationalize.'' Non-orthogonality of supposedly orthogonal Big Five scales was also a criticism of \citet{Block:95}. Saucier explains that the reasons for non-orthogonality in the scales despite orthogonal rotations during scale development are largely due to scale construction procedures that attempt to maximize reliability (by using the scale scores based on salient items rather than the full factor scores). Evidence for non-orthogonality does not invalidate extant scales but rather suggests that these scales may be improved upon by allowances for correlated factors from the outset. A reasonable reaction to this prolonged discussion regarding just one single analytic option might be to wonder if this minor issue is being over-emphasized. I really don't think so. The fundamental presumption of orthogonality, especially in hierarchical scales, has broad consequences for the field. Three suggestive examples are given below. \subsubsection{Fractal structure at all levels} \label{sec:fractal} An interesting phenomenon occurs when one attempts to factor a sufficiently large set of items and then factor those factors and then factor those factors (and so on until no more salient factors are left). There appears to be some support for the near fractal form of these data and analytic methods \citep{Barrett:2005eh} as the structure looks similarly complex at each nested level of analysis. This is evident in the network plots of Figures \ref{fig:zoom1} to \ref{fig:zoom4} in which the items loading onto the first factor are re-factored into multiple 3 factor solutions. If the item set were larger, it seems likely that this trend would have persisted through further iterations of factoring. For comparison purposes, each network graph is accompanied by a graph that demonstrates correlations between an equivalent number of items with perfectly simple structure (i.e., each item being completely uncorrelated with all others). One way to interpret the empirically complex orientation between the items at each level of analysis is that there is no particular basis for using the structure at one level of a hierarchy to parameterize the structure at the next level. In other words, there does not seem to be more parsimony or elegance gained at the second level (the facets) by restricting measurement at that level to the orthogonally (or even obliquely) organized axes of the first level (the domains/factors). It is equivalently messy, so there is no theoretical basis for limiting measurement at levels beyond the factors to include only those pockets of the multi-dimensional space which fit well at higher levels. Just as astronomers do not limit their research by only studying the internal structure of the five nearest solar systems, nor should personality psychologists focus on the internal structure of the Big Five. Yet, most of the widely-used hierarchical measures do this exactly, presumably because they were limited in their construction by attempts to maintain orthogonality at the highest level. \begin{scriptsize} <>= # For this example, we will use a combined data set with the Exploratory Replication, and Confirmatory samples together. bothSample <- rbind(expSample, repSample) allSample <- rbind(bothSample, confSample) allSampleCor <- cor(subset(allSample, select = c(DataItems)), use="pairwise") allSampleCor <- cor.smooth(allSampleCor) # take out three factors from the Replication dataset fractalExample1 <- fa(allSampleCor, 3) # get the items with primary loadings on the first factor (whatever that is) l <- as.matrix(fa.sort(fractalExample1$loadings)) nrows <- dim(l)[1] ncols <- dim(l)[2] m1 <- matrix(apply(t(apply(l, 1, abs)), 1, which.max), ncol = 1) id <- matrix(c(1:nrows, m1), ncol = 2) factor2keys <- matrix(rep(0, ncols * nrows), ncol = ncols) factor2keys[id] <- sign(l[id]) * (abs(l[id]) + 0) factor2keys[factor2keys > 0] <- 1 factor2keys[factor2keys < 0] <- -1 rownames(factor2keys) <- rownames(l) colnames(factor2keys) <- colnames(l) k <- abs(factor2keys) fractalExample1FA1 <- subset(k, k[,1]>0) zoomIn1 <- rownames(fractalExample1FA1) fractalExample1cor <- allSampleCor[rownames(fractalExample1FA1), rownames(fractalExample1FA1)] dim(fractalExample1cor) ##### # let's plot the items that load on the first factor suppressPackageStartupMessages(library(qgraph)) findcolnumber <- function(df, thecolumnname){ which(colnames(df) == thecolumnname) } colname2colnum <- function(x, scaleList) { colnumList <- list() for (i in 1:length(scaleList)) { jlist <- rep(NA, length(scaleList[[i]])) for (j in 1:length(scaleList[[i]])) { jlist[j] <- findcolnumber(x, scaleList[[i]][j]) } colnumList[[length(colnumList)+1]] <- jlist } names(colnumList) <- names(scaleList) rm(i, j) return(colnumList) } Temperament <- colnames(allSampleCor) TempLesszoomIn1 <- setdiff(Temperament, zoomIn1) suppressPackageStartupMessages(library(RColorBrewer)) shortcolors = c(brewer.pal(6, "Set1")) longcolors = c(rainbow(30)) #title = "Plot the first factor items" title = "" filename = paste(filepathfig, "zoom1", sep = "") zoomIn1num <- unlist(colname2colnum(allSampleCor, zoomIn1)) other <- unlist(colname2colnum(allSampleCor, TempLesszoomIn1)) groups <- list(factor1 = zoomIn1num, other = other) colors = c(shortcolors[1], "grey60") pGraph <- qgraph(allSampleCor, minimum=0.3, cut=.6, groups=groups, legend=FALSE, colors = colors, labels = TRUE, title = title, layout="spring", edge.color = "grey50", filetype = "pdf", filename = filename, GLratio = 3, borders=FALSE) sim <- matrix(0, nrow=dim(allSampleCor)[1], ncol=dim(allSampleCor)[1]) diag(sim) <- 1 filename = paste(filepathfig, "sim1", sep = "") groups <- list(all = colnames(allSampleCor)) colors = c("grey60", "grey60") simGraph <- qgraph(sim, minimum=0.3, cut=.6, groups=TRUE, legend=FALSE, colors = colors, labels = TRUE, title = title, layout="spring", edge.color = "grey50", filetype = "pdf", filename = filename, GLratio = 3, borders=FALSE) # Now do it again - take out three factors from the first factor items fractalExample2 <- fa(fractalExample1cor, 3) # get the items with primary loadings on the first factor (whatever that is) l <- as.matrix(fa.sort(fractalExample2$loadings)) nrows <- dim(l)[1] ncols <- dim(l)[2] m1 <- matrix(apply(t(apply(l, 1, abs)), 1, which.max), ncol = 1) id <- matrix(c(1:nrows, m1), ncol = 2) factor2keys <- matrix(rep(0, ncols * nrows), ncol = ncols) factor2keys[id] <- sign(l[id]) * (abs(l[id]) + 0) factor2keys[factor2keys > 0] <- 1 factor2keys[factor2keys < 0] <- -1 rownames(factor2keys) <- rownames(l) colnames(factor2keys) <- colnames(l) k <- abs(factor2keys) fractalExample2FA1 <- subset(k, k[,1]>0) zoomIn2 <- rownames(fractalExample2FA1) fractalExample2cor <- fractalExample1cor[rownames(fractalExample2FA1), rownames(fractalExample2FA1)] dim(fractalExample2cor) Temperament <- colnames(fractalExample1cor) TempLesszoomIn2 <- setdiff(Temperament, zoomIn2) #title = "Plot the second set of first factor items" title = "" filename = paste(filepathfig, "zoom2", sep = "") zoomIn2num <- unlist(colname2colnum(fractalExample1cor, zoomIn2)) other <- unlist(colname2colnum(fractalExample1cor, TempLesszoomIn2)) groups <- list(factor1 = zoomIn2num, other = other) colors = c(shortcolors[1], "grey60") pGraph <- qgraph(fractalExample1cor, minimum=0.3, cut=.6, groups=groups, legend=FALSE, colors = colors, labels = TRUE, title = title, layout="spring", edge.color = "grey50", filetype = "pdf", filename = filename, GLratio = 3, borders=FALSE) sim <- matrix(0, nrow=dim(fractalExample1cor)[1], ncol=dim(fractalExample1cor)[1]) diag(sim) <- 1 filename = paste(filepathfig, "sim2", sep = "") groups <- list(all = colnames(fractalExample1cor)) colors = c("grey60", "grey60") simGraph <- qgraph(sim, minimum=0.3, cut=.6, groups=TRUE, legend=FALSE, colors = colors, labels = TRUE, title = title, layout="spring", edge.color = "grey50", filetype = "pdf", filename = filename, GLratio = 3, borders=FALSE) ##### # Now do it again - take out three factors from the second factor items fractalExample3 <- fa(fractalExample2cor, 3) # get the items with primary loadings on the first factor (whatever that is) l <- as.matrix(fa.sort(fractalExample3$loadings)) nrows <- dim(l)[1] ncols <- dim(l)[2] m1 <- matrix(apply(t(apply(l, 1, abs)), 1, which.max), ncol = 1) id <- matrix(c(1:nrows, m1), ncol = 2) factor2keys <- matrix(rep(0, ncols * nrows), ncol = ncols) factor2keys[id] <- sign(l[id]) * (abs(l[id]) + 0) factor2keys[factor2keys > 0] <- 1 factor2keys[factor2keys < 0] <- -1 rownames(factor2keys) <- rownames(l) colnames(factor2keys) <- colnames(l) k <- abs(factor2keys) fractalExample3FA1 <- subset(k, k[,1]>0) zoomIn3 <- rownames(fractalExample3FA1) fractalExample3cor <- fractalExample2cor[rownames(fractalExample3FA1), rownames(fractalExample3FA1)] dim(fractalExample3cor) Temperament <- colnames(fractalExample2cor) TempLesszoomIn3 <- setdiff(Temperament, zoomIn3) #title = "Plot the third set of first factor items" title = "" filename = paste(filepathfig, "zoom3", sep = "") zoomIn3num <- unlist(colname2colnum(fractalExample2cor, zoomIn3)) other <- unlist(colname2colnum(fractalExample2cor, TempLesszoomIn3)) groups <- list(factor1 = zoomIn3num, other = other) colors = c(shortcolors[1], "grey60") pGraph <- qgraph(fractalExample2cor, minimum=0.3, cut=.6, groups=groups, legend=FALSE, colors = colors, labels = TRUE, title = title, layout="spring", edge.color = "grey50", filetype = "pdf", filename = filename, GLratio = 3, borders=FALSE) sim <- matrix(0, nrow=dim(fractalExample2cor)[1], ncol=dim(fractalExample2cor)[1]) diag(sim) <- 1 filename = paste(filepathfig, "sim3", sep = "") groups <- list(all = colnames(fractalExample2cor)) colors = c("grey60", "grey60") simGraph <- qgraph(sim, minimum=0, cut=.6, groups=TRUE, legend=FALSE, colors = colors, labels = TRUE, title = title, layout="spring", edge.color = "grey50", filetype = "pdf", filename = filename, GLratio = 3, borders=FALSE) ##### # Now do it again - take out three factors from the third first factor items fractalExample4 <- fa(fractalExample3cor, 3) # get the items with primary loadings on the first factor (whatever that is) l <- as.matrix(fa.sort(fractalExample4$loadings)) nrows <- dim(l)[1] ncols <- dim(l)[2] m1 <- matrix(apply(t(apply(l, 1, abs)), 1, which.max), ncol = 1) id <- matrix(c(1:nrows, m1), ncol = 2) factor2keys <- matrix(rep(0, ncols * nrows), ncol = ncols) factor2keys[id] <- sign(l[id]) * (abs(l[id]) + 0) factor2keys[factor2keys > 0] <- 1 factor2keys[factor2keys < 0] <- -1 rownames(factor2keys) <- rownames(l) colnames(factor2keys) <- colnames(l) k <- abs(factor2keys) fractalExample4FA1 <- subset(k, k[,1]>0) zoomIn4 <- rownames(fractalExample4FA1) fractalExample4cor <- fractalExample3cor[rownames(fractalExample4FA1), rownames(fractalExample4FA1)] dim(fractalExample4cor) Temperament <- colnames(fractalExample3cor) TempLesszoomIn4 <- setdiff(Temperament, zoomIn4) #title = "Plot the fourth set of first factor items" title = "" filename = paste(filepathfig, "zoom4", sep = "") zoomIn4num <- unlist(colname2colnum(fractalExample3cor, zoomIn4)) other <- unlist(colname2colnum(fractalExample3cor, TempLesszoomIn4)) groups <- list(factor1 = zoomIn4num, other = other) colors = c(shortcolors[1], "grey60") pGraph <- qgraph(fractalExample3cor, minimum=0.3, cut=.6, groups=groups, legend=FALSE, colors = colors, labels = TRUE, title = title, layout="spring", edge.color = "grey50", filetype = "pdf", filename = filename, GLratio = 3, borders=FALSE) sim <- matrix(0, nrow=dim(fractalExample3cor)[1], ncol=dim(fractalExample3cor)[1]) diag(sim) <- 1 filename = paste(filepathfig, "sim4", sep = "") groups <- list(all = colnames(fractalExample3cor)) colors = c("grey60", "grey60") simGraph <- qgraph(sim, minimum=0, cut=.6, groups=TRUE, legend=FALSE, colors = colors, labels = TRUE, title = title, layout="spring", edge.color = "grey50", filetype = "pdf", filename = filename, GLratio = 3, borders=FALSE) @ \end{scriptsize} \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/zoom1.pdf}}{\caption{Network plot based on correlations between all 696 items. Note that the structure of all 696 items is considerably different from the simulated simple structure in Figure \ref{fig:sim1}. Items in red are those with primary loadings on the first of 3 extracted factors --- correlations between these items are factored again Figure \ref{fig:zoom2}.}\label{fig:zoom1}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/sim1.pdf}}{\caption{Network plot based on simulated correlations reflecting perfectly simple structure for 696 items.}\label{fig:sim1}} \end{floatrow} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/zoom2.pdf}}{\caption{Network plot based on correlations between the items in red in Figure \ref{fig:zoom1}. Again, there is little evidence for simple structure among these 264 items (compare to Figure \ref{fig:sim2}). Items in red here are those with primary loadings on the first of 3 extracted factors --- correlations between these items are factored again Figure \ref{fig:zoom3}.}\label{fig:zoom2}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/sim2.pdf}}{\caption{Network plot based on simulated correlations reflecting perfectly simple structure for 264 items.}\label{fig:sim2}} \end{floatrow} \end{figure} \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/zoom3.pdf}}{\caption{Network plot based on correlations between the items in red in Figure \ref{fig:zoom2}. Still no evidence for simple structure compared to Figure \ref{fig:sim3}. Items in red here are those with primary loadings on the first of 3 extracted factors in these 136 items.}\label{fig:zoom3}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/sim3.pdf}}{\caption{Network plot based on simulated correlations reflecting perfectly simple structure for 136 items.}\label{fig:sim3}} \end{floatrow} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/zoom4.pdf}}{\caption{Network plot based on correlations between the items in red in Figure \ref{fig:zoom3}. Even when using the 66 items in the first factor of the first factor of the first factor of the 696 item set, simple structure is not evident.}\label{fig:zoom4}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/sim4.pdf}}{\caption{Network plot based on simulated correlations reflecting perfectly simple structure for 66 items.}\label{fig:sim4}} \end{floatrow} \end{figure} \subsubsection{Heterarchical arrangement of items in predictive applications} One consequence --- or perhaps a cause --- of the tendency to invoke orthogonality in structure is the related tendency to organize traits into tidily nested, tree-like structures, sometimes referred to as dendritic-nesting. This line of thinking suggests that the Extraversion items all belong to one and only one facet, each facet is tied to only one domain (factor), and Extraversion itself is perhaps wholly subsumed by the super-ordinate factor ``Beta'' \citep{Digman:97}. There is nothing explicitly wrong with the tendency to organize the traits this way. It can be done but it seems motivated by an unrealistically symmetric and psychometric view of the \href{https://youtu.be/Z0GFRcFm-aY}{world}. In addition to the aforementioned evidence that these neatly nested models do not approximate reality in assessment, they imply a strict and inflexible ``org chart'' like those which might be used at a bureaucratic corporation. It is commonplace for personality psychologists to conduct research by pointing their org chart at the criterion du jour: first, evaluate the associations between the criterion variable and the Big Five and then follow up with further evaluations at the facet level for the one or two most highly related Big-Five factor(s).\footnote{Lest readers get defensive, note that I have used this strategy to good effect as well, though I hope to avoid the practice in the future now that I know better.} This would be a reasonable approach if reality reflected orthogonality. If the NEO-PI-R\texttrademark, for example, were strictly orthogonal, then by the transitive property, it would be true for an outcome which was more highly associated with Conscientiousness than with Extraversion that all facets of Conscientiousness would be more highly associated with the outcome than all facets of Extraversion (note that the rationale here is essentially the same as for the bifactor example given above). Reality requires a predictive model that is far more flexible and dynamic, one which is heterarchical. Though not widely-used, this term simply refers to a subset of hierarchical models that allows for dynamic reorganization of the constituent parts within and, more importantly, across levels of the hierarchy as needed. In the case of predictive modeling, this would be as needed based on different criterion. Several authors have recently made similar observations about the structure of personality items, scales and sets of scales. Heterarchical organization of the traits is indirectly implied, for example, in several features of \citet{DeYoung:14b}'s Cybernetic Big Five Theory, the dynamic three-level model of \citet{Revelle:2015hg}, in the observations of \citet{Mottus:2016kx} regarding the common and unique variance predicted by facets and factors, and above all by the network model of \citet{Cramer:12}. The latter paper provides an excellent description of the heterarchical structure among personality items (without ever using the word heterarchy) and expounds upon several ramifactions with regards to the behavioral and temporal incompatibilities of behaviors (and the items that assess them --- ``you can't like parties if you don't like people''). The trouble is that none of these authors prescribe feasible remedies for personality assessment. To support this claim, it seems worth pulling another long quote, this time from \citet[p. 417,][]{Cramer:12}: \begin{blockquote} ``The model with five latent traits influencing only their respective items ... does not fit the data ... which is mainly due to violations of simple structure: particularly, the correlations between items that belong to different personality dimensions are too high to be accounted for by the model. How can one address this problem? One way, the standard way in personality psychology (e.g., \citealp{Savla:2007kb}) is by tweaking the model `on the basis of the data' so that the basic latent variable hypothesis is preserved (e.g., by allowing cross-loadings, exploratory factor analysis with procrustes rotation; see also \citealp{Borsboom:06b} for an elaborate critique). Another way would be to make the simple structure model more complex, for example, by introducing first-order and second-order latent variables.... Another way, the central tenet of this paper, is to consider the misfit of the un-tweaked model, an indication that the latent variable hypothesis fails as an explanation of the emergence of normal personality dimensions, and to move on towards alternative models.'' \end{blockquote} The authors successfully achieve their aim, but fail to propose feasible alternative models other than the one which is not addressed (more complex models with first- and second-order latent variables). My proposal includes two additional alternatives for assessment purposes. The first of these is highly similar to the more complex alternative suggested by \citet{Cramer:12} except that the levels for the latent variable models (whether the so-called first or second or any other level that may be specified) should be explicitly independent. The second alternative would be for the field of personality psychology to begin developing a repository of measures that are empirically designed to predict specific outcomes. All things being equal, outcome-specific models should be maximally predictive but the development of such a repository would be a massive undertaking. The main challenge is that a very large number of items and outcomes need to be simultaneously assessed in multiple, diverse samples in order to identify generalizable scales which do not either capitalize on chance or predictor/outcome circularity (e.g., predicting body mass from eating frequency or low activity levels). \href{https://youtu.be/aopES6KCBQs}{I believe} that such a repository can be constructed and that personality researchers should provide information about the items (or similarly granular measures) that predict the criterion variables of interest whenever possible. Yet, a more empirically-informed and complex general-purpose model would seem to be more useful in many applications. Instead of ignoring the heterarchical nature of personality structure, such a model would be flexible enough to accomodate it. For further comments on heterarchy, see Section \ref{sec:overfactoring}. \subsubsection{Bifactor confusion} It has recently become more common to examine the extent to which facets like those in the NEO-PI-R\texttrademark can be used to predict outcomes beyond the domain (Five Factor) level using bifactor modeling techniques. These methods tend to produce results suggesting that particular facets are correlated with a criterion ``above and beyond'' the domains to which the facets are purportedly related \citep{Chen:2012gy}. A common criticism is that it is theoretically difficult to interpret the meaning of most facets after removing the common variance of the higher-level trait (e.g., what is gregariousness without Extraversion?). But there is a more subtle problem caused by the fact that these analyses are often conducted without regard to the theoretical and empirical structure of the domains and facets of the measurement model. As previously suggested, there are many reasons to believe that the domains of the NEO-PI-R\texttrademark (for example) are not orthogonal to one another despite the orthogonal rotational method used during their development. There is also no clear explanation of the structural orientation of the facets within any given domain, in part because this was not specified by the test developers at the outset and in part because each sample will likely (almost certainly) reflect idiosyncratic empirical orientation due to sample-size dependent instability in the correlations and other standard replicability concerns. In the case of the NEO-PI-R\texttrademark, it is not even the case that the mean of the facet scores is necessarily equivalent to the domain score (this is somewhat counterintuitive). It is also not the case that facets describe the interstitial spaces between the domains more thoroughly than the domains themselves (again, this is somewhat surprising but necessarily true because both the domains and the facets use the same item set). Bifactor analyses with measures such as these demonstrate only that facets have specific variance beyond the common variance (of the factor under study). Full stop. This may be worth demonstrating as a psychometric exercise. But it says nothing about the location of the outcome in the multi-dimensional personality space. By that, I mean that a forced orthogonal solution between the factor and the facet-level residuals tells us nothing about the orientation of the factor or the facet in relation to all of the other factors and facets. In order to describe the absolute orientation of the scales to other structures in the personality space (factors, facets, or external criterion), considerable exploratory work is needed to identify the structure of each specific sample --- and these results will still be biased unless a broader set of items is administered than those which are in the scales being used! There are circumstances for which bifactor techniques are highly useful, but these generally do not include cases where the domain/factor and facet levels are non-orthogonal (and where there is little evidence of a ``positive manifold''). See \citet{Chen:06} and \citet{Reise:12} for more about useful applications of bifactor models and, especially, \citet[p. 222,][]{Chen:06} for orthogonality-related limitations and guidance. When exploring the relationship between criterion variables and factors/facets/items, there are at least two good alternatives: (1) describe the relationship between the predicted outcome and each level of the hierarchy independently --- the domain level and the facet level (and maybe also the item level, if the sample is large enough); and (2) use residual regression methods to partial out the facet scores from the domain/factor score of interest. Researchers who want to explore the structural relationships of hierarchical facets to the scales (rather than criterion) are encouraged to use an omega factor analysis \citep[without a Schmid-Leiman rotation or similar bifactor approach,][]{Zinbarg:06} or a cluster analysis or a simple factor analysis of the facet scores (though this may not reproduce the purported higher level structure). \subsubsection{Conclusions about oblique vs orthogonal rotations} I hope to have made a thorough and compelling case for the allowance of oblique transformations of the factor analytic results. Further, the process of evaluating this analytic option has caused me to conclude that the identification of a hierarchical model need not depend upon orthogonality at any level and that the identification of optimally descriptive structure at any level should be independent of the structure at other levels. I will focus, therefore, on identifying the levels at which structure is most interpretable, most likely to replicate and most likely to be useful for making predictions. In closing, it should be acknowledged that a very important question remains to be answered. Namely, is the use of more messy empirical models justified by greater predictive power? If not, it may not be worth the trouble to switch and the field should carry on indefinitely using variants of the NEO-PI-R\texttrademark. The only way to know is through model testing. Time will tell. \subsection{Conclusions about the sensitivity analyses} We will proceed with the structural analyses by essentially using the same analytic parameters which are the defaults in the \textit{psych} package \citep{Rpsych} in \texttt{R} \citep{R}. A laissez-faire approach seems preferred, whenever possible. To be explicitly clear, we will conduct exploratory factor analysis using: (1) the `fa' function rather than `principal'; (2) the Pearson correlations rather than the polychorics; (3) the smoothed correlation matrix rather than the unsmoothed; (4) the `oblimin' rotation option which allows for oblique rotations rather than `varimax' or any other option that forces orthogonal rotations; and (5) the `minres' factoring method rather than weighted or generalized least squares (or any of the many other options). The syntax will look like this:\\ fa(\textit{r}, nfactors = \textit{nfactors}, np.obs = \textit{np.obs}, rotate = ``oblimin", fm = ``minres") \indent where:\\ \begin{itemize} \vspace{-5mm} \item \textit{r} is the smoothed correlation matrix based on Pearson correlations of the non-ipsatized data\\ \vspace{-5mm} \item \textit{nfactors} is the number of factors being extracted at each unique iteration of the analyses (we start at 1 and increase by 1, iteratively, in order to allow for comparison across solutions)\\ \vspace{-5mm} \item \textit{np.obs} is the mean number of pairwise observations among the items in the sample\\ \end{itemize} The values for the number of pairwise observations were determined earlier as pwiseAdminsMeanE and pwiseAdminsMeanR in sections \ref{sec:expDesc} and \ref{sec:repDesc}; these values are \Sexpr{prettyNum(round(pwiseAdmins1E$mean, 0), big.mark=",", scientific=F)} for the Exploratory sample and \Sexpr{prettyNum(round(pwiseAdmins1R$mean, 0), big.mark=",", scientific=F)} for the Replication sample. (Mean pairwise observations in the Confirmatory sample were \Sexpr{prettyNum(round(pwiseAdmins1C$mean, 0), big.mark=",", scientific=F)}, though this is not relevant until the CFA is conducted in Section \ref{sec:confirmatory}). There are several additional options which can be specified in the `fa()' function but they are not relevant for our purposes here. Note also that this code is the primary engine that underlies the wrapper function (`autoFactoring()'; see sections \ref{sec:expAutoFactor} and \ref{sec:repAutoFactor}) that I use to streamline the extraction procedures. Before leaving the topic of sensitivity, it's worth noting that the influence of almost all the analytic options considered here (except maybe ipsatization) appear to be highly dependent on sample size. In other words, the effect of these techniques is more noticeable in smaller sample sizes. This is how it should be for sensitivity analyses --- the ``effect size'' gets \textit{smaller} with larger samples. This is the inverse of how we normally think about effect sizes. Large samples are more likely to replicate in part because they are less sensitive to analytic modifications. Bigger samples make it easier to differentiate the true signal from the noise (to paraphrase a vast expanse of the methodological literature). The additional recognition to be made here is that modifications to correlation matrices for the sake of interpretability actually have a smaller effect on matrices formed from larger data sets. This is (retrospectively) intuitive but also serves as a reminder of the imperative need for large samples when relying on factor analyses for scale development. \section{Stability: within and between samples} \label{sec:stability} Stability and replicability are directly related. The likelihood of replication will be a function of all the topics described up to this point, and others that I have not attempted to address. Specifically, failure to replicate the exploratory factor analyses reported below might be caused by: \begin{enumerate} \item the use of inadequate sample sizes \item the use of a different sampling of items/stimuli \item the administration of items to a sample of participants that differs meaningfully from the ones used here on any number of variables, including those for which no data have been collected \item the use of different analytic options for extracting/rotating the factors \item differences in the method of data collection (whether online through a different user interface or offline) and/or differences in the motivation of the participants (i.e., the use of financial or similar incentives that differed from the provision of feedback to participants about their personalities) \item collection of the data across different time periods \end{enumerate} This list is not exhaustive but I feel that these are the most likely reasons why the findings reported here would not replicate. I have listed them in descending order based on the likelihood of reducing replicability of the structural analyses in the following section (based solely on my experience in working with similar datasets over the last few years). There is, of course, the separate issue of generalizability; do these results generalize to other cultures or when using translations of items or when using observer ratings? Those are important topics for future research. All of these issues ultimately relate to the degree of stability in the correlation matrices. Issues 2 through 6 above affect replicability by altering the degree of stability across samples. Issue 1 is somewhat different because it affects the degree of stability across samples \textit{and within}. This is because fluctuations in the associations between variables decrease as the sample size increases. The obvious question is: how big should the sample be? Many factor analytic experts have weighed in on this point and their answers are surprisingly varied (see Osborne and Fitzpatrick [\citeyearpar{Osborne:2012te}] for an nice summary). The challenge is that it depends considerably on the number of variables being analyzed, the number of factors being extracted, and the inherent variability of the items. Given the intention to factor analyze a large pool of items and to extract a large number of factors, the recommended size in this case would seem to be much larger than the typical recommendation of 200+ participants. In any case, it is much more practical to evaluate the stability of the correlation matrices post-hoc. In other words, how stable are the samples we've got? Given the specifics of their actual size and density, I can estimate the stability of the correlations with bootstrapping methods (we do this in \texttt{R} by using the function `cor.ci()' in the psych package \citep{Rpsych}). The analyses in following code chunk use output that was generated from an earlier chunk (this was part of the analyses run on a remote computing cluster --- the code is given in Section \ref{sec:pearsonpoly}). This code chunk pulls those data in again and creates plots that reflect the stability of the correlations (only the Exploratory sample is shown as an example). \begin{scriptsize} <>= load(paste(filepathdata, "SPIcorCI.RData", sep = "")) # Exploratory sample standard errors for item correlations pdf(paste(filepathfig, "expCIs.pdf", sep = ""), onefile = TRUE) plot(density(expSampleCorCI$sds), main="", ylab="Frequency", xlab="Standard error of item level correlations", xlim= c(0,.1), ylim = c(0,160)) dev.off() # Mean sd of correlations in the Exploratory sample. meanSDofRsE <- mean(expSampleCorCI$sds) meanSDofRsE # Median sd of correlations in the Exploratory sample. medianSDofRsE <- median(expSampleCorCI$sds) medianSDofRsE # min sd of correlations in the Exploratory sample. maxSDofRsE <- max(expSampleCorCI$sds) maxSDofRsE # max sd of correlations in the Exploratory sample. minSDofRsE <- min(expSampleCorCI$sds) minSDofRsE @ <>= # Replication sample standard errors for item correlations pdf(paste(filepathfig, "repCIs.pdf", sep = ""), onefile = TRUE) plot(density(repSampleCorCI$sds), main="", ylab="Frequency", xlab="Standard error of item level correlations", xlim= c(0,.1), ylim = c(0,160)) dev.off() # Mean sd of correlations in the Replication sample. meanSDofRsR <- mean(repSampleCorCI$sds) meanSDofRsR # Median sd of correlations in the Replication sample. medianSDofRsR <- median(repSampleCorCI$sds) medianSDofRsR # max sd of correlations in the Replication sample. maxSDofRsR <- max(repSampleCorCI$sds) maxSDofRsR # min sd of correlations in the Replication sample. minSDofRsR <- min(repSampleCorCI$sds) minSDofRsR # Confirmatory sample standard errors for item correlations pdf(paste(filepathfig, "confCIs.pdf", sep = ""), onefile = TRUE) plot(density(confSampleCorCI$sds), main="", ylab="Frequency", xlab="Standard error of item level correlations", xlim= c(0,.1), ylim = c(0,160)) dev.off() # Mean sd of correlations in the Confirmatory sample. meanSDofRsC <- mean(confSampleCorCI$sds) meanSDofRsC # Median sd of correlations in the Confirmatory sample. medianSDofRsC <- median(confSampleCorCI$sds) medianSDofRsC # max sd of correlations in the Confirmatory sample. maxSDofRsC <- max(confSampleCorCI$sds) maxSDofRsC # min sd of correlations in the Confirmatory sample. minSDofRsC <- min(confSampleCorCI$sds) minSDofRsC # Combined sample standard errors for item correlations pdf(paste(filepathfig, "allCIs.pdf", sep = ""), onefile = TRUE) plot(density(allSampleCorCI$sds), main="", ylab="Frequency", xlab="Standard error of item level correlations", xlim= c(0,.1), ylim = c(0,160)) dev.off() # Mean sd of correlations in the combined sample. meanSDofRsAll <- mean(allSampleCorCI$sds) meanSDofRsAll # Median sd of correlations in the combined sample. medianSDofRsAll <- median(allSampleCorCI$sds) medianSDofRsAll # max sd of correlations in the combined sample. maxSDofRsAll <- max(allSampleCorCI$sds) maxSDofRsAll # min sd of correlations in the combined sample. minSDofRsAll <- min(allSampleCorCI$sds) minSDofRsAll @ \end{scriptsize} The mean and median of the standard errors of the correlations (based on bootstrapped estimates with 100 iterations) are \Sexpr{round(meanSDofRsE,2)} and \Sexpr{round(medianSDofRsE,2)} for the Exploratory sample, respectively. For the Replication sample, they are \Sexpr{round(meanSDofRsR,2)} and \Sexpr{round(medianSDofRsR,2)}, respectively. For the Confirmatory sample, they are \Sexpr{round(meanSDofRsC,2)} and \Sexpr{round(medianSDofRsC,2)}, respectively. Figures \ref{fig:expCIs} through \ref{fig:confCIs} show density plots of the standard errors. This implies, for example, that a correlation of .2 in the Replication sample would have an expected 95\% confidence interval of about .17 to .23. For item level correlations, this is a relatively narrow range. \begin{figure} \begin{center} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/expCIs.pdf}}{\caption{Density distribution of the standard errors of the correlations for the exploratory sample}\label{fig:expCIs}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/repCIs.pdf}}{\caption{Density distribution of the standard errors of the correlations for the replication sample}\label{fig:repCIs}} \end{floatrow} \end{center} \end{figure} \begin{figure} \begin{center} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/confCIs.pdf}}{\caption{Density distribution of the standard errors of the correlations for the confirmatory sample}\label{fig:confCIs}} \end{floatrow} \end{center} \end{figure} \section{Structural analyses} \label{sec:structural} Now that I have all that preliminary work out of the way, I can finally begin evaluating the factor structure for both samples. This will be done by using the `autoFactoring' function, the code for which is given below. I will use the output from this function in many ways, including evaluation of traditional fit statistics, then evaluation of fit based on quantitative metrics of the item content for various solutions, and then based on qualitative review of the item content. \subsection{autoFactoring the Exploratory sample} \label{sec:expAutoFactor} First, I have to get the output. Note that it takes a ton of space to show the output for the extraction of 1 to 30 factors (several hundred pages), so I have suppressed the printing of all output here. It is possible to review the results in more detail by running the code in the rnw version of this file and then printing any of the many components of the output from the autoFactoring results. Also note that (1) the hidden code in this section includes the code for the `autoFactoring' function which was not available elsewhere at the time this file was written; and (2) this section of the file takes a while to run (up to a few hours). \begin{scriptsize} <>= # THIS STANDS IN PLACE OF HAVING STEPS 39 THRU 44 SET TO eval=TRUE # I used this at various points to reduce the computing time # between iterations. load(paste(filepathdata, "factorContentData.Rdata", sep = "")) @ \end{scriptsize} \begin{scriptsize} <>= rm(list=setdiff(ls(), c("confCor696rawSmoothed", "expCor696rawSmoothed", "filepathdata", "filepathfig", "pwiseAdmins1C", "pwiseAdmins1E", "repCor696rawSmoothed", "pwiseAdmins1R", "ItemInfo696"))) # This long section of code creates the autoFactoring function autoFactoring <- function (x, nfactors = 3, np.obs = np.obs, dictionary = dictionary, PriLdgMin = 0.3, SecLdgMax = 0.3, rotate = "oblimin") { # autoFactoring is intended to work with the default options in fa. # Additional options to be considered in autoFactoring include: # nfactors: number of factor solutions to be extracted in sequence (begin with 1) # PriLdgMin: minimum cut for primary loading for each item (for QI) # SecLdgMax: maximum cut for secondary loadings for each item (for QI) # x must be a correlation matrix # This first section runs fa on the correlation matrix from 1 to nfactors # For each level of fa, it also finds the QIC, identifies the quality items and # generates list of the highest loading quality items. FAoutput <- list() for (i in 1:nfactors) { n <- i fax <- suppressWarnings(suppressMessages(fa(x, n, np.obs=np.obs, rotate=rotate))) colnames(fax$loadings) <- paste("EV",1:n,sep="") # fixes the ordering issue to eigenvalues # fax is faoutput_nfactors faItems <- fa.lookup(fax, dictionary) faItems <- faItems[,c(1:(n+3))] # faItems is lookup table for fax with item content l <- as.matrix(fa.sort(fax$loadings)) nrows <- dim(l)[1] ncols <- dim(l)[2] if (ncols == 1) { m1 <- matrix(rep(1, nrows), ncol = 1) } else { m1 <- matrix(apply(t(apply(l, 1, abs)), 1, which.max), ncol = 1) } id <- matrix(c(1:nrows, m1), ncol = 2) factor2keys <- matrix(rep(0, ncols * nrows), ncol = ncols) factor2keys[id] <- sign(l[id]) * (abs(l[id]) + 0) factor2keys[factor2keys > 0] <- 1 factor2keys[factor2keys < 0] <- -1 rownames(factor2keys) <- rownames(l) colnames(factor2keys) <- colnames(l) k <- abs(factor2keys) # k is the absolute value (only 1s and 0s) for the keys matrix based on loadings k2 <- scrub(k, isvalue=1, newvalue=2) k1 <- scrub(k2, isvalue=0, newvalue=1) kInv <- scrub(k1, isvalue=2, newvalue=0) # inverted k matrix primary <- (fa.sort(fax$loadings)[, 1:n]*k) # only the primary loadings notPrimary <- (fa.sort(fax$loadings)[, 1:n]*kInv) # only the non-primary loadings iii <- scrub(abs(primary), min = PriLdgMin) # a matrix with primary loadings above abs value of PriLdgMin (all else NAs) iii[iii > (PriLdgMin-0.000001)] <- 1 # now the loadings are switched to 1s iiir <- rowSums(iii, na.rm=TRUE) iiir <- as.data.frame(iiir) # number of 'primary' factors for each item (could be >1) iiirInv <- scrub(iiir, isvalue=1, newvalue=2) iiirInv <- scrub(iiirInv, isvalue=0, newvalue=1) iiirInv <- scrub(iiirInv, isvalue=2, newvalue=0) iii2 <- scrub(abs(notPrimary), min = SecLdgMax) # a matrix with secondary loadings above abs value of SecLdgMax (all else NAs) iii2[iii2 > (SecLdgMax-0.0001)] <- 1 # now the loadings are switched to 1s iii2r <- rowSums(iii2, na.rm=TRUE) # number of high secondary factors for each item DNQ <- data.frame(iiirInv, iii2r, iiirInv+iii2r) # a df of the items that Do Not Qualify factor <- paste("fa", n, sep="") qualPrimary <- as.data.frame(primary[rownames(DNQ[which(DNQ$iiir.1 == 0),]),]) # subset of primary matrix with qualified items best <- list() for (j in 1:n) { qual <- qualPrimary[(abs(qualPrimary[,j]) > 0),] if (ncol(qualPrimary) > 1) { qualified <- as.vector(qual[,j]) names(qualified) <- rownames(qual) } else { qualified <- as.vector(qual) names(qualified) <- rownames(qualPrimary) } bestItems <- qualified[1:min(dim(qual)[1],10)] items <- paste("itemsFa", j, sep="") best[[items]] <- list(qualified = qualified, bestItems = bestItems) rm(bestItems, items, qualified) } QICfactor <- colSums(as.matrix(k[rownames(DNQ[which(DNQ$iiir.1 == 0),]),])) # a list of the number of items that do qualify by factor tmp <- list(QIC = QICfactor, fa = fax, faItems = faItems, best = best) FAoutput[[factor]] <- tmp rm(best, factor, iii, iiir, iiirInv, iii2, iii2r, j, n, fax, faItems, k, k2, k1, kInv, DNQ, notPrimary, primary, QICfactor, tmp, factor2keys, i, id, l, m1, ncols, nrows, qual, qualPrimary) } # Here begins the code to pull out the fit statistics and table them if (rotate == "oblimin") { if (!requireNamespace("GPArotation")) { stop("You must have GPArotation installed to use oblimin rotation") } } old_rotate = rotate # next two functions are needed to get the VSS values (see below) complexrow <- function(x, c) { n = length(x) temp <- x x <- rep(0, n) for (j in 1:c) { locmax <- which.max(abs(temp)) x[locmax] <- sign(temp[locmax]) * max(abs(temp)) temp[locmax] <- 0 } return(x) } complexmat <- function(x, c) { nrows <- dim(x)[1] ncols <- dim(x)[2] for (i in 1:nrows) { x[i, ] <- complexrow(x[i, ], c) } return(x) } # a function for deriving the MAP map <- function(x, n) { nvar <- dim(x)[2] MAP <- rep(NA, n) e <- eigen(x) evect <- e$vectors comp <- evect %*% diag(suppressWarnings(sqrt(e$values))) if (n >= nvar) { n1 <- nvar - 1 } else { n1 <- n } for (i in 1:n1) { c11.star <- x - comp[, 1:i] %*% t(comp[, 1:i]) d <- diag(1/sqrt(diag(c11.star))) rstar <- d %*% c11.star %*% d diag(rstar) <- 0 MAP[i] <- sum(rstar * rstar)/(nvar * (nvar - 1)) } return(MAP) } ifelse (dim(x)[2] < nfactors, n <- dim(x)[2], n <- nfactors) complexfit <- array(0, dim = c(n, n)) complexresid <- array(0, dim = c(n, n)) fitStats.df <- data.frame(vss1 = rep(0, n), vss2 = NA, MAP = NA, dof = NA, chisq = NA, prob = NA, sqresid = NA, fit = NA, RMSEA = NA, BIC = NA, SABIC = NA, complex = NA, eChisq = NA, SRMR = NA, eCRMS = NA, eBIC = NA) if (!is.matrix(x)) x <- as.matrix(x) if (is.null(np.obs)) { message("np.obs was not specified and was arbitrarily set to 1000. This only affects the chi square values.") np.obs <- 1000 } map.values <- map(x, n) if (n > dim(x)[2]) { n <- dim(x)[2] } for (i in 1:n) { PHI <- diag(i) f <- eval(parse(text=paste("FAoutput$fa", i, "$fa", sep=""))) diagonal = FALSE if (i == 1) { original <- x sqoriginal <- original * original totaloriginal <- sum(sqoriginal) - diagonal * sum(diag(sqoriginal)) } load <- as.matrix(f$loadings) model <- load %*% PHI %*% t(load) residual <- original - model sqresid <- residual * residual totalresid <- sum(sqresid) - diagonal * sum(diag(sqresid)) fit <- 1 - totalresid/totaloriginal fitStats.df[i, "MAP"] <- map.values[i] fitStats.df[i, "dof"] <- f$dof fitStats.df[i, "chisq"] <- f$STATISTIC fitStats.df[i, "prob"] <- f$PVAL fitStats.df[i, "eChisq"] <- f$chi fitStats.df[i, "SRMR"] <- f$rms fitStats.df[i, "eRMS"] <- f$rms fitStats.df[i, "eCRMS"] <- f$crms fitStats.df[i, "eBIC"] <- f$EBIC if (!is.null(f$RMSEA)) { fitStats.df[i, "RMSEA"] <- f$RMSEA[1] } else { fitStats.df[i, "RMSEA"] <- NA } if (!is.null(f$BIC)) { fitStats.df[i, "BIC"] <- f$BIC } else { fitStats.df[i, "BIC"] <- NA } if (!is.null(f$SABIC)) { fitStats.df[i, "SABIC"] <- f$SABIC } else { fitStats.df[i, "SABIC"] <- NA } if (!is.null(f$complexity)) { fitStats.df[i, "complex"] <- mean(f$complexity) } else { fitStats.df[i, "complex"] <- NA } fitStats.df[i, "sqresid"] <- totalresid fitStats.df[i, "fit"] <- fit for (c in 1:i) { simpleload <- complexmat(load, c) model <- simpleload %*% PHI %*% t(simpleload) residual <- original - model sqresid <- residual * residual totalsimple <- sum(sqresid) - diagonal * sum(diag(sqresid)) simplefit <- 1 - totalsimple/totaloriginal complexresid[i, c] <- totalsimple complexfit[i, c] <- simplefit } fitStats.df[i, "vss1"] <- complexfit[i,1] fitStats.df[i, "vss2"] <- complexfit[i,2] } # Here begins the code to make the QIC table allfactors <- paste("fa", seq(1, nfactors, 1), sep="") QICtable <- data.frame() for (i in allfactors){ QICrowx <- sapply(FAoutput[i],"[[", "QIC") QICrowx <- as.data.frame(t(QICrowx)) QICrowx <- cbind(i, QICrowx) colnames(QICrowx)[1] <- "factors" if(i == "fa1") colnames(QICrowx)[2] <- "EV1" QICrow <- unlist(QICrowx) names(QICrow) <- names(QICrowx) QICrow <- as.data.frame(t(as.data.frame(QICrow))) QICtable <- rbind.fill(QICtable, QICrow) rm(QICrow, QICrowx) } QICtable <- subset(QICtable, select = -c(factors)) colnames(QICtable) <- paste("EV", seq(1, nfactors, 1), sep="") rownames(QICtable) <- paste("fa", seq(1, nfactors, 1), sep="") # Builds the QIC stats table totalitems <- dim(x)[1] total <- rowSums(QICtable, na.rm=TRUE) ratio <- round(total/totalitems, 4) FAplus1 <- c(1:nfactors)+1 base1 <- round(totalitems/FAplus1, 0) base2 <- round(base1/fitStats.df$complex, 0) QICoverB1 <- rowSums(QICtable > base1, na.rm=TRUE) QICoverB2 <- rowSums(QICtable > base2, na.rm=TRUE) QICB1ratio <- QICoverB1/(FAplus1-1) QICB2ratio <- QICoverB2/(FAplus1-1) QICstats <- data.frame(total, ratio, FAplus1, base1, base2, QICoverB1, QICoverB2, QICB1ratio, QICB2ratio) # Here begins the code to make the eigenvalues table EVtable <- matrix(ncol=nfactors, nrow=nfactors) for (i in 1:nfactors) { eachSSloadings <- paste("colSums((FAoutput$fa", i, "$fa$loadings)^2)", sep="") SSloadingstmp1 <- round(eval(parse(text=eachSSloadings)), 2) SSloadingstmp2 <- rep(NA, nfactors-length(SSloadingstmp1)) EVtable[i,] <- c(SSloadingstmp1, SSloadingstmp2) rm(eachSSloadings, SSloadingstmp1, SSloadingstmp2) } colnames(EVtable) <- paste("EV", seq(1, nfactors, 1), sep="") rownames(EVtable) <- paste("fa", seq(1, nfactors, 1), sep="") results <- list(FAoutput = FAoutput, fits = fitStats.df, QICtable = QICtable, QICstats = QICstats, EVtable = EVtable) return(results) } @ <>= expSampleCor696 <- expCor696rawSmoothed rm(expCor696rawSmoothed) # run a parallel analysis on the Exploratory sample expSampleCor696fapara <- fa.parallel(expSampleCor696, n.obs=pwiseAdmins1E$mean) nfactors = 30 # run autoFactoring to conduct EFA at many levels simultaneously for later comparison of the output expSampleCor696autoFA <- autoFactoring(expSampleCor696, nfactors = nfactors, np.obs = pwiseAdmins1E$mean, dictionary = ItemInfo696) @ \end{scriptsize} The following chunk of code is using the output of autoFactoring to make data frames of fit statistics and complexity. This will be used to generate Table \ref{tab:fitstats} and Figures \ref{fig:expFits} and \ref{fig:expComplexities}. I also pull out the mean absolute correlation between factors for all of the solutions. \begin{scriptsize} <>= Factors <- 1:30 # Plot and table the fit statistics to get a better sense of the structure. expFitstatistics.df <- data.frame(Factors, subset(expSampleCor696autoFA$fits, select = c(MAP, eChisq, eCRMS, eBIC))) colnames(expFitstatistics.df) <- c("Factors", "MAP", "eChiSq", "eCRMS", "eBIC") expComplex.df <- data.frame(Factors, expSampleCor696autoFA$fits$complex) colnames(expComplex.df) <- c("Factors", "complexity") @ <>= # Make plots for complexity (by itself) and MAP, eChiSq, eCRMS, and eBIC (as a 2x2) suppressPackageStartupMessages(library(ggplot2)) suppressPackageStartupMessages(library(grid)) p1 <- ggplot(data = expComplex.df) + geom_line(aes(x = Factors, y = complexity)) + xlab("Factors extracted") + ylab("Complexity") + ylim(1,5.5) pdf(paste(filepathfig, "expComplexities.pdf", sep = ""), onefile = TRUE) grid.newpage() pushViewport(viewport(layout = grid.layout(1, 1, widths = c(300, 300)))) vplayout <- function(x, y) viewport(layout.pos.row = x, layout.pos.col = y) print(p1, vp = vplayout(1,1)) dev.off() p1 <- ggplot(data = expFitstatistics.df) + geom_line(aes(x = Factors, y = MAP)) + xlab("Factors extracted") + ylab("Velicer's MAP") + ylim(.001,.013) p2 <- ggplot(data = expFitstatistics.df) + geom_line(aes(x = Factors, y = eChiSq)) + xlab("Factors extracted") + ylab("Empirically-Derived Chi Square") + ylim(175000,4750000) p3 <- ggplot(data = expFitstatistics.df) + geom_line(aes(x = Factors, y = eCRMS)) + xlab("Factors extracted") + ylab("Empirically-Derived Mean Residual Corrected for df") + ylim(0.024,.10) p4 <- ggplot(data = expFitstatistics.df) + geom_line(aes(x = Factors, y = eBIC)) + xlab("Factors extracted") + ylab("Empirically-Derived Bayesian Information Criterion") + ylim(-1300000,3000000) png(filename = paste(filepathfig, "expFitStatistics.png", sep = ""), width = 600, height = 600, units = "px", res = NA) grid.newpage() pushViewport(viewport(layout = grid.layout(2, 2, widths = c(300, 300)))) vplayout <- function(x, y) viewport(layout.pos.row = x, layout.pos.col = y) print(p1, vp = vplayout(1,1)) print(p2, vp = vplayout(1,2)) print(p3, vp = vplayout(2,1)) print(p4, vp = vplayout(2,2)) dev.off() rm(p1, p2, p3, p4, vplayout) EFAmeanFactorCorrelations <- function (x, nfactors = nfactors) { # A helper function using autoFactoring output. # x is the output from autoFactoring. # nfactors is the number of solutions run in the autoFactoring. FactorCorMeanList <- list() xstring <- deparse(substitute(x)) for (factorSolution in 1:nfactors) { factorCorMatX <- eval(parse(text=paste(xstring, "$FAoutput$fa", factorSolution, "$fa$r.scores", sep=""))) factorCorMatX[upper.tri(factorCorMatX)] <- NA diag(factorCorMatX) <- NA meanX <- mean(abs(factorCorMatX), na.rm=TRUE) FactorCorMeanList[[factorSolution]] <- meanX } results <- unlist(FactorCorMeanList) return(results) rm(factorCorMatx, meanX, xstring, factorSolution) } expMeanFactorCorrelations <- EFAmeanFactorCorrelations(expSampleCor696autoFA, nfactors=nfactors) round(expMeanFactorCorrelations,2) @ \end{scriptsize} \subsection{autoFactoring the Replication sample} \label{sec:repAutoFactor} Here I repeat the same analyses for the Replication sample. First, I run a parallel analysis, then autoFactoring, and then organize the results for tables and figures. \begin{scriptsize} <>= repSampleCor696 <- repCor696rawSmoothed repSampleCor696fapara <- fa.parallel(repSampleCor696, n.obs=pwiseAdmins1R$mean) repSampleCor696autoFA <- autoFactoring(repSampleCor696, nfactors = nfactors, np.obs = pwiseAdmins1R$mean, dictionary = ItemInfo696) Factors <- 1:30 # Plot and table the fit statistics to get a better sense of the structure. repFitstatistics.df <- data.frame(Factors, subset(repSampleCor696autoFA$fits, select = c(MAP, eChisq, eCRMS, eBIC))) colnames(repFitstatistics.df) <- c("Factors", "MAP", "eChiSq", "eCRMS", "eBIC") repComplex.df <- data.frame(Factors, repSampleCor696autoFA$fits$complex) colnames(repComplex.df) <- c("Factors", "complexity") tableFitstatistics <- round(data.frame(expFitstatistics.df, repFitstatistics.df[,c("MAP", "eChiSq", "eCRMS", "eBIC")]),3) tableFitstatistics[,"eBIC"] <- round(tableFitstatistics[,"eBIC"],0) tableFitstatistics[,"eBIC.1"] <- round(tableFitstatistics[,"eBIC.1"],0) tableFitstatistics[,"eChiSq"] <- round(tableFitstatistics[,"eChiSq"],0) tableFitstatistics[,"eChiSq.1"] <- round(tableFitstatistics[,"eChiSq.1"],0) colnames(tableFitstatistics) <- c("Factors", "MAP", "eChiSq", "eCRMS", "eBIC", "MAP", "eChiSq", "eCRMS", "eBIC") @ <>= # Make plots for complexity (by itself) and MAP, eChiSq, eCRMS, and eBIC (as a 2x2) suppressPackageStartupMessages(library(ggplot2)) suppressPackageStartupMessages(library(grid)) p1 <- ggplot(data = repComplex.df) + geom_line(aes(x = Factors, y = complexity)) + xlab("Factors extracted") + ylab("Complexity") + ylim(1,5.5) pdf(paste(filepathfig, "repComplexities.pdf", sep = ""), onefile = TRUE) grid.newpage() pushViewport(viewport(layout = grid.layout(1, 1, widths = c(300, 300)))) vplayout <- function(x, y) viewport(layout.pos.row = x, layout.pos.col = y) print(p1, vp = vplayout(1,1)) dev.off() p1 <- ggplot(data = repFitstatistics.df) + geom_line(aes(x = Factors, y = MAP)) + xlab("Factors extracted") + ylab("Velicer's MAP") + ylim(.001,.013) p2 <- ggplot(data = repFitstatistics.df) + geom_line(aes(x = Factors, y = eChiSq)) + xlab("Factors extracted") + ylab("Empirically-Derived Chi Square") + ylim(175000,4750000) p3 <- ggplot(data = repFitstatistics.df) + geom_line(aes(x = Factors, y = eCRMS)) + xlab("Factors extracted") + ylab("Empirically-Derived Mean Residual Corrected for df") + ylim(0.024,.10) p4 <- ggplot(data = repFitstatistics.df) + geom_line(aes(x = Factors, y = eBIC)) + xlab("Factors extracted") + ylab("Empirically-Derived Bayesian Information Criterion") + ylim(-1300000,3000000) png(filename = paste(filepathfig, "repFitStatistics.png", sep = ""), width = 600, height = 600, units = "px", res = NA) grid.newpage() pushViewport(viewport(layout = grid.layout(2, 2, widths = c(300, 300)))) vplayout <- function(x, y) viewport(layout.pos.row = x, layout.pos.col = y) print(p1, vp = vplayout(1,1)) print(p2, vp = vplayout(1,2)) print(p3, vp = vplayout(2,1)) print(p4, vp = vplayout(2,2)) dev.off() rm(p1, p2, p3, p4, vplayout) repMeanFactorCorrelations <- EFAmeanFactorCorrelations(repSampleCor696autoFA, nfactors=nfactors) round(repMeanFactorCorrelations, 2) confSampleCor696 <- confCor696rawSmoothed save(expSampleCor696fapara, filepathdata, filepathfig, ItemInfo696, nfactors, repSampleCor696fapara, tableFitstatistics, expSampleCor696autoFA, repSampleCor696autoFA, repSampleCor696, expSampleCor696, confSampleCor696, file=paste(filepathdata, "factorContentData.Rdata", sep = "")) @ \end{scriptsize} \subsection{Interpreting the fit statistics in both samples independently} \label{sec:fit} Interpreting the fit statistics is difficult because they tend not to point to clear solutions and are confusingly contradictory. But we'll try anyway. Visual inspection of the fit statistic plots (see Figures \ref{fig:expFits} and \ref{fig:repFits}) shows consistent shifts in slope at around five factors. This provides good support for the broad use of five factors as a parsimonious solution. Parsimony is not the only objective however; I am attempting to empirically identify more narrow constructs which can improve the predictive utility of personality models. Plots of item complexity (see Figures \ref{fig:expComplexities} and \ref{fig:repComplexities}) and, to a lesser extent, parallel analyses and/or traditional fit statistics (shown in Table \ref{tab:fitstats} and Figures \ref{fig:expFits} and \ref{fig:repFits}) can be useful for this purpose as they help to identify the relative fit of solutions at proximal levels of extraction. Parallel analysis for the Exploratory sample suggests \Sexpr{expSampleCor696fapara$nfact} factors and \Sexpr{expSampleCor696fapara$ncomp} components based on a comparison between the scree (eigenvalues) of factors/components of the observed data with that of a random data matrix of the same size. Parallel analysis for the Replication sample suggests \Sexpr{repSampleCor696fapara$nfact} factors and \Sexpr{repSampleCor696fapara$ncomp} components. Table \ref{tab:fitstats} demonstrates that the fit statistics are inconclusive across the samples and highly inconsistent within the samples. Velicer's MAP criterion \citep{Velicer:76} suggests \Sexpr{match(min(expSampleCor696autoFA$fits$MAP), expSampleCor696autoFA$fits$MAP)} and \Sexpr{match(min(repSampleCor696autoFA$fits$MAP), repSampleCor696autoFA$fits$MAP)} factors in the Exploratory and Replication samples, respectively. The empirically calculated RMSEA (eCRMS) \citep{Hu:99} suggests 5 factors as the optimal cut-off (about .05) for both samples. The empirically calculated Bayesian Information Criterion \citep[eBIC,][]{Schwarz:78, Rpsych} suggests \Sexpr{match(min(expSampleCor696autoFA$fits$eBIC), expSampleCor696autoFA$fits$eBIC)} and \Sexpr{match(min(repSampleCor696autoFA$fits$eBIC), repSampleCor696autoFA$fits$eBIC)} factors in the Exploratory and Replication samples, respectively, and the empirically calculated Chi Square values did not reach a localized minimum in either sample. The complexity plots (Figures \ref{fig:expComplexities} and \ref{fig:repComplexities}) are also useful here. As expected, the complexity \citep{Hofmann:77, Hofmann:78} consistently increases in both samples as additional factors are extracted. Decreases in the slope of the complexity plots, however, suggest that the extraction of an additional factor leads to a less complex solution than would be expected based on results from the previous level of extraction. There are two points worth mentioning for the complexity plots for both samples. First, both samples show similar levels of complexity at 5 factors (complexity = 2) and, for both samples, the slope of the complexity lines decreases at 5 factors and then increases rather sharply from 5 to 6 factors. The slope decreases again at 7 in the replication sample, but not much for the exploratory sample. I interpret these circumstances as consistent evidence for the reduced complexity of a 5 factor solution for both samples and inconsistent evidence for a 7 factor solution. Second, both samples show similar patterns of reduced complexity at 15-16 factors and again at 25-27 factors. \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/expFitStatistics.png}}{\caption{Fit Statistics in the Exploratory Sample}\label{fig:expFits}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/repFitStatistics.png}}{\caption{Fit Statistics in the Replication Sample}\label{fig:repFits}} \end{floatrow} \end{figure} \begin{scriptsize} \begin{table}[htdp]\caption{Fit statistics for the Exploratory and Replication Samples} \begin{center} \begin{tabular}{rrrrrrrrr} \hline & \multicolumn{4}{c|}{Exploratory Sample} & \multicolumn{4}{|c}{Replication Sample}\\ Factors & MAP & eChiSq & eCRMS & \multicolumn{1}{r|}{eBIC} & MAP & eChiSq & eCRMS & eBIC \cr \hline 1 & \Sexpr{prettyNum(tableFitstatistics[1,2], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[1,3], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[1,4], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[1,5], big.mark=",", scientific=F, digits=0)} & \Sexpr{prettyNum(tableFitstatistics[1,6], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[1,7], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[1,8], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[1,9], big.mark=",", scientific=F, digits=0)} \cr 2 & \Sexpr{prettyNum(tableFitstatistics[2,2], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[2,3], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[2,4], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[2,5], big.mark=",", scientific=F, digits=0)} & \Sexpr{prettyNum(tableFitstatistics[2,6], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[2,7], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[2,8], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[2,9], big.mark=",", scientific=F, digits=0)} \cr 3 & \Sexpr{prettyNum(tableFitstatistics[3,2], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[3,3], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[3,4], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[3,5], big.mark=",", scientific=F, digits=0)} & \Sexpr{prettyNum(tableFitstatistics[3,6], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[3,7], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[3,8], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[3,9], big.mark=",", scientific=F, digits=0)} \cr 4 & \Sexpr{prettyNum(tableFitstatistics[4,2], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[4,3], big.mark=",", scientific=F, digits=3)} & \textbf{\Sexpr{prettyNum(tableFitstatistics[4,4], big.mark=",", scientific=F, digits=3)}} & \Sexpr{prettyNum(tableFitstatistics[4,5], big.mark=",", scientific=F, digits=0)} & \Sexpr{prettyNum(tableFitstatistics[4,6], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[4,7], big.mark=",", scientific=F, digits=3)} & \textbf{\Sexpr{prettyNum(tableFitstatistics[4,8], big.mark=",", scientific=F, digits=3)}} & \Sexpr{prettyNum(tableFitstatistics[4,9], big.mark=",", scientific=F, digits=0)} \cr 5 & \Sexpr{prettyNum(tableFitstatistics[5,2], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[5,3], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[5,4], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[5,5], big.mark=",", scientific=F, digits=0)} & \Sexpr{prettyNum(tableFitstatistics[5,6], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[5,7], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[5,8], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[5,9], big.mark=",", scientific=F, digits=0)} \cr 6 & \Sexpr{prettyNum(tableFitstatistics[6,2], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[6,3], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[6,4], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[6,5], big.mark=",", scientific=F, digits=0)} & \Sexpr{prettyNum(tableFitstatistics[6,6], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[6,7], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[6,8], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[6,9], big.mark=",", scientific=F, digits=0)} \cr 7 & \Sexpr{prettyNum(tableFitstatistics[7,2], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[7,3], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[7,4], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[7,5], big.mark=",", scientific=F, digits=0)} & \Sexpr{prettyNum(tableFitstatistics[7,6], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[7,7], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[7,8], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[7,9], big.mark=",", scientific=F, digits=0)} \cr 8 & \Sexpr{prettyNum(tableFitstatistics[8,2], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[8,3], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[8,4], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[8,5], big.mark=",", scientific=F, digits=0)} & \Sexpr{prettyNum(tableFitstatistics[8,6], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[8,7], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[8,8], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[8,9], big.mark=",", scientific=F, digits=0)} \cr 9 & \Sexpr{prettyNum(tableFitstatistics[9,2], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[9,3], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[9,4], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[9,5], big.mark=",", scientific=F, digits=0)} & \Sexpr{prettyNum(tableFitstatistics[9,6], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[9,7], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[9,8], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[9,9], big.mark=",", scientific=F, digits=0)} \cr 10 & \Sexpr{prettyNum(tableFitstatistics[10,2], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[10,3], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[10,4], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[10,5], big.mark=",", scientific=F, digits=0)} & \Sexpr{prettyNum(tableFitstatistics[10,6], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[10,7], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[10,8], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[10,9], big.mark=",", scientific=F, digits=0)} \cr 11 & \Sexpr{prettyNum(tableFitstatistics[11,2], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[11,3], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[11,4], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[11,5], big.mark=",", scientific=F, digits=0)} & \Sexpr{prettyNum(tableFitstatistics[11,6], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[11,7], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[11,8], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[11,9], big.mark=",", scientific=F, digits=0)} \cr 12 & \Sexpr{prettyNum(tableFitstatistics[12,2], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[12,3], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[12,4], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[12,5], big.mark=",", scientific=F, digits=0)} & \Sexpr{prettyNum(tableFitstatistics[12,6], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[12,7], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[12,8], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[12,9], big.mark=",", scientific=F, digits=0)} \cr 13 & \Sexpr{prettyNum(tableFitstatistics[13,2], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[13,3], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[13,4], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[13,5], big.mark=",", scientific=F, digits=0)} & \Sexpr{prettyNum(tableFitstatistics[13,6], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[13,7], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[13,8], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[13,9], big.mark=",", scientific=F, digits=0)} \cr 14 & \Sexpr{prettyNum(tableFitstatistics[14,2], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[14,3], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[14,4], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[14,5], big.mark=",", scientific=F, digits=0)} & \Sexpr{prettyNum(tableFitstatistics[14,6], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[14,7], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[14,8], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[14,9], big.mark=",", scientific=F, digits=0)} \cr 15 & \Sexpr{prettyNum(tableFitstatistics[15,2], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[15,3], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[15,4], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[15,5], big.mark=",", scientific=F, digits=0)} & \Sexpr{prettyNum(tableFitstatistics[15,6], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[15,7], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[15,8], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[15,9], big.mark=",", scientific=F, digits=0)} \cr 16 & \Sexpr{prettyNum(tableFitstatistics[16,2], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[16,3], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[16,4], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[16,5], big.mark=",", scientific=F, digits=0)} & \Sexpr{prettyNum(tableFitstatistics[16,6], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[16,7], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[16,8], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[16,9], big.mark=",", scientific=F, digits=0)} \cr 17 & \Sexpr{prettyNum(tableFitstatistics[17,2], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[17,3], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[17,4], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[17,5], big.mark=",", scientific=F, digits=0)} & \Sexpr{prettyNum(tableFitstatistics[17,6], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[17,7], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[17,8], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[17,9], big.mark=",", scientific=F, digits=0)} \cr 18 & \Sexpr{prettyNum(tableFitstatistics[18,2], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[18,3], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[18,4], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[18,5], big.mark=",", scientific=F, digits=0)} & \Sexpr{prettyNum(tableFitstatistics[18,6], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[18,7], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[18,8], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[18,9], big.mark=",", scientific=F, digits=0)} \cr 19 & \Sexpr{prettyNum(tableFitstatistics[19,2], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[19,3], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[19,4], big.mark=",", scientific=F, digits=3)} & \textbf{\Sexpr{prettyNum(tableFitstatistics[19,5], big.mark=",", scientific=F, digits=0)}} & \Sexpr{prettyNum(tableFitstatistics[19,6], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[19,7], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[19,8], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[19,9], big.mark=",", scientific=F, digits=0)} \cr 20 & \Sexpr{prettyNum(tableFitstatistics[20,2], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[20,3], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[20,4], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[20,5], big.mark=",", scientific=F, digits=0)} & \Sexpr{prettyNum(tableFitstatistics[20,6], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[20,7], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[20,8], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[20,9], big.mark=",", scientific=F, digits=0)} \cr 21 & \Sexpr{prettyNum(tableFitstatistics[21,2], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[21,3], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[21,4], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[21,5], big.mark=",", scientific=F, digits=0)} & \Sexpr{prettyNum(tableFitstatistics[21,6], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[21,7], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[21,8], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[21,9], big.mark=",", scientific=F, digits=0)} \cr 22 & \Sexpr{prettyNum(tableFitstatistics[22,2], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[22,3], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[22,4], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[22,5], big.mark=",", scientific=F, digits=0)} & \Sexpr{prettyNum(tableFitstatistics[22,6], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[22,7], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[22,8], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[22,9], big.mark=",", scientific=F, digits=0)} \cr 23 & \Sexpr{prettyNum(tableFitstatistics[23,2], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[23,3], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[23,4], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[23,5], big.mark=",", scientific=F, digits=0)} & \Sexpr{prettyNum(tableFitstatistics[23,6], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[23,7], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[23,8], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[23,9], big.mark=",", scientific=F, digits=0)} \cr 24 & \Sexpr{prettyNum(tableFitstatistics[24,2], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[24,3], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[24,4], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[24,5], big.mark=",", scientific=F, digits=0)} & \Sexpr{prettyNum(tableFitstatistics[24,6], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[24,7], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[24,8], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[24,9], big.mark=",", scientific=F, digits=0)} \cr 25 & \Sexpr{prettyNum(tableFitstatistics[25,2], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[25,3], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[25,4], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[25,5], big.mark=",", scientific=F, digits=0)} & \Sexpr{prettyNum(tableFitstatistics[25,6], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[25,7], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[25,8], big.mark=",", scientific=F, digits=3)} & \textbf{\Sexpr{prettyNum(tableFitstatistics[25,9], big.mark=",", scientific=F, digits=0)}} \cr 26 & \textbf{\Sexpr{prettyNum(tableFitstatistics[26,2], big.mark=",", scientific=F, digits=3)}} & \Sexpr{prettyNum(tableFitstatistics[26,3], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[26,4], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[26,5], big.mark=",", scientific=F, digits=0)} & \Sexpr{prettyNum(tableFitstatistics[26,6], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[26,7], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[26,8], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[26,9], big.mark=",", scientific=F, digits=0)} \cr 27 & \Sexpr{prettyNum(tableFitstatistics[27,2], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[27,3], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[27,4], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[27,5], big.mark=",", scientific=F, digits=0)} & \Sexpr{prettyNum(tableFitstatistics[27,6], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[27,7], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[27,8], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[27,9], big.mark=",", scientific=F, digits=0)} \cr 28 & \Sexpr{prettyNum(tableFitstatistics[28,2], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[28,3], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[28,4], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[28,5], big.mark=",", scientific=F, digits=0)} & \Sexpr{prettyNum(tableFitstatistics[28,6], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[28,7], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[28,8], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[28,9], big.mark=",", scientific=F, digits=0)} \cr 29 & \Sexpr{prettyNum(tableFitstatistics[29,2], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[29,3], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[29,4], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[29,5], big.mark=",", scientific=F, digits=0)} & \textbf{\Sexpr{prettyNum(tableFitstatistics[29,6], big.mark=",", scientific=F, digits=3)}} & \Sexpr{prettyNum(tableFitstatistics[29,7], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[29,8], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[29,9], big.mark=",", scientific=F, digits=0)} \cr 30 & \Sexpr{prettyNum(tableFitstatistics[30,2], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[30,3], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[30,4], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[30,5], big.mark=",", scientific=F, digits=0)} & \Sexpr{prettyNum(tableFitstatistics[30,6], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[30,7], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[30,8], big.mark=",", scientific=F, digits=3)} & \Sexpr{prettyNum(tableFitstatistics[30,9], big.mark=",", scientific=F, digits=0)} \cr \hline \multicolumn{9}{l}{\begin{scriptsize}Bold values indicate optimal fit for each statistic, where relevant.\end{scriptsize}}\\ \multicolumn{9}{l}{\begin{scriptsize}MAP: Velicer's MAP criterion\end{scriptsize}}\\ \multicolumn{9}{l}{\begin{scriptsize}eChiSq: Empirically calculated Chi Square values. Did not resolve in either sample.\end{scriptsize}}\\ \multicolumn{9}{l}{\begin{scriptsize}eCRMS: Empirically calculated mean residual (RMSEA) corrected for the degrees of freedom.\end{scriptsize}}\\ \multicolumn{9}{l}{\begin{scriptsize}eBIC: Empirically calculated Bayesian Information Criterion based upon the empirical Chi Square.\end{scriptsize}}\\ \end{tabular} \end{center} \label{tab:fitstats} \end{table} \end{scriptsize} \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/expComplexities.pdf}}{\caption{Complexity in the Exploratory Sample}\label{fig:expComplexities}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/repComplexities.pdf}}{\caption{Complexity in the Replication Sample}\label{fig:repComplexities}} \end{floatrow} \end{figure} We can also consider the eigenvalues of the factors for each level of extraction (this is redundant with the scree plot approach). Eigenvalues are the sums of the squared loadings down the columns of the loadings matrix. Note that, while the ``eigenvalue greater than 1'' rule has been widely discredited \citep{Shaycoft:1970, Cliff:88}, some researchers feel examination of the magnitudes of the eigenvalues helps to evaluate the ``right" number of factors. The output of autoFactoring makes it easy to generate a \textbf{table of the eigenvalues for the various factors at each level of extraction} from 1 to 30 factors for both samples (see the rnw version for the code). Tables \ref{tab:expEVtable} and \ref{tab:repEVtable} contain the eigenvalues for the Exploratory and Replication samples, respectively. I don't find these tables to be particularly meaningful, but some psychometricians find them informative. I leave it up to the reader to interpret the results. \clearpage \begin{scriptsize} <>= suppressPackageStartupMessages(library(plyr)) suppressPackageStartupMessages(library(xtable)) print(xtable(expSampleCor696autoFA$EVtable, digits = 1, caption = 'Eigenvalues by Factor - Exploratory Sample', label = 'tab:expEVtable', align = c("p{1.05cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}")), size='tiny', floating = TRUE, floating.environment = "sidewaystable") @ \end{scriptsize} \clearpage \begin{scriptsize} <>= print(xtable(repSampleCor696autoFA$EVtable, digits = 1, caption = 'Eigenvalues by Factor - Replication Sample', label = 'tab:repEVtable', align = c("p{1.05cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}")), size='tiny', floating = TRUE, floating.environment = "sidewaystable") @ \end{scriptsize} \clearpage \begin{scriptsize} <>= print(xtable(expSampleCor696autoFA$QICtable, digits=0, caption='Qualified item counts by factor - Exploratory Sample', label='tab:ExpItemCount', align= c("p{1.05cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}")), size='tiny', floating = TRUE, floating.environment = "sidewaystable") @ \end{scriptsize} \clearpage \begin{scriptsize} <>= print(xtable(repSampleCor696autoFA$QICtable, digits=0, caption='Qualified item counts by factor - Replication Sample', label='tab:RepItemCount', align= c("p{1.15cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}", "R{.44cm}")), size='tiny', floating = TRUE, floating.environment = "sidewaystable") @ \end{scriptsize} \clearpage A more useful analysis, in my opinion, evaluates the number of \textit{quality} items produced at the various levels of factor extraction. These results help to identify so-called junk factors. Junk factors are those which do not have many or any items with primary loadings and/or without high secondary loadings. Junk factors are problematic because they are essentially impossible to interpret. The output of autoFactoring is once again used here to generate \textbf{tables of the counts of items (by factor) with a primary factor loading above 0.3 and no secondary factor loadings above 0.3} when I extract between 1 and 30 factors. I refer to these as Qualified Item Counts (QICs). Tables \ref{tab:ExpItemCount} and \ref{tab:RepItemCount} show the QICs in each factor solution. Interpretation of these tables is easy --- strong factors have many qualified items and weak factors have relatively few (see Section \ref{sec:overfactoring} for more on the relative strength of factors). True ``junk'' factors in these results are those with 0 qualified items (note that this does not preclude the possibility that such factors may have multiple items with primary loadings below .30).\footnote{Readers who have not previously pored over factor analysis output should note that it's not uncommon to get solutions with really crappy factors --- these are the ones that most people call junk --- where there are 0 items with primary loadings. These factors serve to improve the statistical fit but are not interpretable. I'm taking a slightly more strict view on junk factors by using this idea of qualified items.} Using liberal --- and somewhat arbitrary --- guidelines, several levels appear to offer useful solutions. My rule of thumb for evaluating the legitimacy of a factor depended on the number of factors being extracted for each solution.\footnote{A brief digression about these guidelines seems warranted. I was not able to discover any prior work regarding the number of items (or magnitude of eigenvalues) needed to constitute a legitimate factor. Clearly, there is an opportunity for further research here. I used thresholds that seemed appropriate given the number of items included in the EFAs (696 items in all cases) and the number of factors extracted (ranging from 1 to 30). In other words, I assumed that the number of items needed to constitute a real/good factor is inversely proportionate to the number of factors extracted; more specifically, the number of parameters being estimated. Fewer items should be needed when extracting 30 factors, for example, than when extracting only 3 factors. It should be possible to specify and substantiate this proportion through various types of simulations. I did not get around to doing this but someone should. This would be a much more useful contribution than the development of another marginally useful fit statistic, of which there are too many.} Specifically, I judged a factor to be legit if it contained at least 10 items for factor solutions with 1-10 factors, at least 7 items for solutions with 11-15 factors, 5 items for solutions with 16-20 factors, and 3-ish items for solutions with 21-30 factors. In cases where a solution had 1 or 2 factors that just missed these thresholds, I erred on the side of leniency by flagging it for further follow-up during my evaluation of the item content in ``good'' solutions (i.e., those that replicated; see the next section). This was done to avoid ruling out any solutions due to the arbitrary nature of these thresholds. Note that failure to meet these thresholds does not necessarily mean that the solution provides poor statistical fit, but it does suggest that the solution is not useful as an assessment model. In the Exploratory sample, these thresholds lent support for solutions with 1, 2, 3, 4, 5, 7, 8, 15, 17, 18, 19, 24, and 27 factors; in the Replication sample, it was solutions with 1, 2, 3, 4, 5, 6, 8, 21, 22, 23, 24, 27 and 30 factors. If I focus only on those which overlapped across samples, I'm left with 1-5, 8, 24, and 27. Given the arbitrary nature of these thresholds and the fit statistics reported earlier, I spent considerable time evaluating nearly all possible solutions --- with a particular focus on the solutions with between 24 and 29 extracted factors --- by attempting to determine whether the factor content for these solutions aligned across samples. Before bothering to make sense of these (many) results regarding the fits \textit{within} each sample, it makes more sense to consider how well the solutions replicate \textit{across} samples. There is little point in considering solutions that fail to hold up in both samples. \subsection{Replication analyses} \label{sec:replication} Evaluating the replicability of the factor structures requires multiple steps. The first and most tedious of these involves creation and examination of the item content in each factor for each solution (for each sample). These tables allowed me to align the factors with matching content across the samples and then carry out several quantitative analyses. Only after completing this step can we move on to quantitative evaluation of replicability and conclusions about the factor structure across samples. \subsubsection{Interpreting and aligning factors} I interpreted the factors by examining the item content at the many levels of extraction. The goals were to evaluate whether or not the solutions at each level form coherent factors based on item content and, if so, to figure out the alignment of these factors across samples at \textit{equivalent} levels of extraction (I paid little attention to maintenance/splitting of factors within samples at increasing levels of extraction --- it was interesting but not very useful). To do this, I created Appendix \hyperref[sec:appendixA]{A} with tables of the top 10 qualified items --- or all qualified items in cases where fewer than 10 were qualified --- for each factor in the solutions from 1 to 30 for both samples. There are \Sexpr{nfactors*(nfactors+1)/2} tables for each sample, \Sexpr{nfactors*(nfactors+1)} in all. I referenced these tables frequently when attempting to align the factor solutions at various levels of extraction and when attempting to evaluate the strength of the factors in each solution. The next section of code shows how these tables were generated for the Exploratory sample. \begin{scriptsize} <>= objects = ls() tablesToMake <- nfactors*(nfactors+1)/2 tablesToMake # Make all the tables for the exploratory item set factorsSolvedList <- list() for (factorSolution in 1:nfactors) { factorsInfo <- list() for (factorNumber in 1:factorSolution) { named <- paste("ItemInfo696[names(expSampleCor696autoFA$FAoutput$fa", factorSolution, "$best$itemsFa", factorNumber, "$bestItems),]", sep="") factorsInfo[[factorNumber]] <- named rm(factorNumber, named) } factorsSolvedList[[factorSolution]] <- factorsInfo rm(factorsInfo, factorSolution) } factorsSolved <- unlist(factorsSolvedList) rm(factorsSolvedList) tableNameList <- list() n <- 0 for (i in 1:nfactors) { for(j in 1:i) { n <- n+1 tableNameList[n] <- paste(i, "_", j, sep="") } } rm(j, n) for (i in 1:tablesToMake) { assign(paste("ExpTable", tableNameList[i], sep=""), eval(parse(text=factorsSolved[i]))) rm(i) } @ <>= # Make all the tables for the replication item set factorsSolvedList <- list() for (factorSolution in 1:nfactors) { factorsInfo <- list() for (factorNumber in 1:factorSolution) { named <- paste("ItemInfo696[names(repSampleCor696autoFA$FAoutput$fa", factorSolution, "$best$itemsFa", factorNumber, "$bestItems),]", sep="") factorsInfo[[factorNumber]] <- named rm(factorNumber, named) } factorsSolvedList[[factorSolution]] <- factorsInfo rm(factorsInfo, factorSolution) } factorsSolved <- unlist(factorsSolvedList) rm(factorsSolvedList) tableNameList <- list() n <- 0 for (i in 1:nfactors) { for(j in 1:i) { n <- n+1 tableNameList[n] <- paste(i, "_", j, sep="") } } rm(j, n) for (i in 1:tablesToMake) { assign(paste("RepTable", tableNameList[i], sep=""), eval(parse(text=factorsSolved[i]))) rm(i) } rm(tableNameList) rm(list=setdiff(objects, c("filepathdata", "filepathfig", "ItemInfo696"))) save(list=ls(), file=paste(filepathdata, "tablesForInterpretation.Rdata", sep = "")) @ \end{scriptsize} These tables allowed me to identify similar/matching factors across the samples. This step was straight-forward but tedious because it could not be automated. While invariance analyses based on CFA techniques is the gold standard for evaluating the consistency of a previously identified model across samples, those analyses are not appropriate for evaluating the consistency of factors in two samples where no a priori model exists.\footnote{Yes, yes, I know, I mean other than the Big Five.} While EFA replication analyses get around this concern, the relative disadvantage of this approach is that we have to compensate for the lack of a specified model by going through the output by hand. This is because there are many cases where effectively equivalent factors are extracted at different rank orders across the two samples. In other words, I had to determine which factor in the 19 factor solution of the Replication sample, for example, matches factor 3 in the 19 factor solution of the Exploratory sample. Unsurprisingly, there were a few factors among the 30 solutions (930 factors in all) that were difficult to line up clearly; this generally occurred for solutions that were poor. Uninterpretable factors were often impossible to align. Better solutions were less subjective and less likely to be biased in terms of factor alignment. In an effort to reduce subjectivity, I completed the content alignment twice, about 3 months apart. It was... a lot of fun.\footnote{If you're interested in evaluating the results of my alignment it can be found in the next chunk of code, not visible in the PDF version.} It should be noted that the rank order of factors was not given any consideration in the quantitative evaluations of replicability as this is dependent on the eigenvalues and can be manipulated via rotation. \subsubsection{Quantifying the replicability of factors} After aligning the factors by content across samples at each level of extraction, (I was very happy. And relieved. So I took a break to work on other projects for a while and then...) I generated quantitative metrics of the extent to which they replicate. Three metrics were used, largely following the procedures outlined by \citet{Osborne:2012te}. First, I checked whether the basic factor structures were similar across samples (at each level of extraction) by evaluating whether the primary loading for each of the qualified items was consistent across the samples. By this, I do not mean consistent in magnitude but rather consistent in terms of the factor content. In other words, I considered whether the items loaded on the same factor. I only used the qualified items because I was less concerned with those that had low primary loadings and/or high cross-loadings. Second, for those items which passed the first evaluation, I assessed the strength of replication by comparing the \textit{magnitude} of the loadings across the samples. This was done by finding the difference in the squared loadings for those items that loaded on the same factors across the samples. Finally, I evaluated the correlations of the loadings for these items. While this last analysis is insensitive to the magnitudes of the loadings compared to the second analyses, it indicates the extent to which the rank orderings of the items are aligned. This is very similar to factor congruence when done for all items but different for cases where I only used the correlations for the loadings among qualified items or structurally invariant or structurally stable items on each factor. The next section of code shows the procedures for deriving these metrics. Only the analyses for the first few factors are shown as an example. The remaining analyses and some helper functions for autoFactoring are included in the rnw version of the file. \begin{scriptsize} <>= rm(list=setdiff(ls(), c("filepathdata", "filepathfig"))) load(paste(filepathdata, "factorContentData.Rdata", sep = "")) rm(list=setdiff(ls(), c("repSampleCor696autoFA", "expSampleCor696autoFA", "filepathdata", "filepathfig", "ItemInfo696", "nfactors"))) EFAreplication <- function (x, y, EFAn = 1, fax = 1, fay = 1, stable=.04) { # A helper function using autoFactoring output # x and y are the outputs from autoFactoring on two samples # EFAn is the total number of factors in the solution of interest (must be equal for x and y) # fax and fay are the factor numbers which correspond in the solutions (may be different in x and y) xstring <- deparse(substitute(x)) ystring <- deparse(substitute(y)) qualX <- eval(parse(text=paste(xstring, "$FAoutput$fa", EFAn, "$best$itemsFa", fax, "$qualified", sep=""))) qualY <- eval(parse(text=paste(ystring, "$FAoutput$fa", EFAn, "$best$itemsFa", fay, "$qualified", sep=""))) ldgmatX <- eval(parse(text=paste(xstring, "$FAoutput$fa", EFAn, "$faItems", sep=""))) ldgmatY <- eval(parse(text=paste(ystring, "$FAoutput$fa", EFAn, "$faItems", sep=""))) ldgmatY <- ldgmatY[match(rownames(ldgmatX),rownames(ldgmatY)),] magldgs <- (abs(ldgmatX[, fax]) + abs(ldgmatY[, fay]))/2 names(magldgs) <- rownames(ldgmatX) qualItems <- union(names(qualX), names(qualY)) qualCount <- length(qualItems) structuralPass <- intersect(names(qualX), names(qualY)) structuralPassCount <- length(structuralPass) structuralFail <- union(setdiff(names(qualX), names(qualY)), setdiff(names(qualY), names(qualX))) structuralFailCount <- length(structuralFail) passedItemsStrength <- (abs(ldgmatX[structuralPass, fax])-abs(ldgmatY[structuralPass, fay]))^2 names(passedItemsStrength) <- structuralPass stableItemsStrength <- passedItemsStrength[abs(passedItemsStrength) < stable] # reorder ldgmatY to match ldgmatX allItemsCor <- round(cor(ldgmatX[, fax], ldgmatY[, fay]), 3) qualItemsCor <- round(cor(ldgmatX[qualItems, fax], ldgmatY[qualItems, fay]), 3) passedItemsCor <- round(cor(ldgmatX[structuralPass, fax], ldgmatY[structuralPass, fay]), 3) stableItemsCor <- round(cor(ldgmatX[names(stableItemsStrength), fax], ldgmatY[names(stableItemsStrength), fay]), 3) output <- list(qualItems = qualItems, qualCount = qualCount, structuralPass = structuralPass, structuralPassCount = structuralPassCount, structuralFail = structuralFail, structuralFailCount = structuralFailCount, passedItemsStrength = passedItemsStrength, stableItems = names(stableItemsStrength), stableItemsCount = length(stableItemsStrength), stableItemsStrength = stableItemsStrength, allItemsCor = allItemsCor, qualItemsCor = qualItemsCor, passedItemsCor = passedItemsCor, stableItemsCor = stableItemsCor, magldgs = magldgs) results <- output return(results) } @ \end{scriptsize} \begin{scriptsize} <>= stable <- .01 replicationEFA1_1 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 1, fax = 1, fay = 1, stable=stable) # show the output for this first example replicationEFA1_1$qualCount # How many qualified items are there total? replicationEFA1_1$qualItems[1:50] # just show the first 50 items replicationEFA1_1$structuralPassCount # How many structural passes total? replicationEFA1_1$structuralPass[1:50] # just show the first 50 items replicationEFA1_1$structuralFailCount # How many `qualifieds' failed? replicationEFA1_1$structuralFail[1:50] # show the first 50 items replicationEFA1_1$stableItemsCount # How many stable items in total? replicationEFA1_1$stableItems[1:50] # just show the first 50 items replicationEFA1_1$passedItemsCor replicationEFA1_1$stableItemsCor # It is possible to write a loop but that does not work when some of the factors are out of order. So, I have done this by hand in order to document the alignment of factors across samples at each level. replicationEFA1_1 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 1, fax = 1, fay = 1, stable=stable) replicationEFA2_1 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 2, fax = 1, fay = 1, stable=stable) replicationEFA2_2 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 2, fax = 2, fay = 2, stable=stable) replicationEFA3_1 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 3, fax = 1, fay = 1, stable=stable) replicationEFA3_2 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 3, fax = 2, fay = 2, stable=stable) replicationEFA3_3 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 3, fax = 3, fay = 3, stable=stable) # And so on and so on @ <>= replicationEFA4_1 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 4, fax = 1, fay = 1, stable=stable) replicationEFA4_2 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 4, fax = 2, fay = 2, stable=stable) replicationEFA4_3 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 4, fax = 3, fay = 4, stable=stable) replicationEFA4_4 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 4, fax = 4, fay = 3, stable=stable) replicationEFA5_1 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 5, fax = 1, fay = 1, stable=stable) replicationEFA5_2 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 5, fax = 2, fay = 2, stable=stable) replicationEFA5_3 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 5, fax = 3, fay = 4, stable=stable) replicationEFA5_4 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 5, fax = 4, fay = 3, stable=stable) replicationEFA5_5 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 5, fax = 5, fay = 5, stable=stable) replicationEFA6_1 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 6, fax = 1, fay = 2, stable=stable) replicationEFA6_2 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 6, fax = 2, fay = 1, stable=stable) replicationEFA6_3 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 6, fax = 3, fay = 3, stable=stable) replicationEFA6_4 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 6, fax = 4, fay = 4, stable=stable) replicationEFA6_5 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 6, fax = 5, fay = 5, stable=stable) replicationEFA6_6 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 6, fax = 6, fay = 6, stable=stable) replicationEFA7_1 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 7, fax = 1, fay = 1, stable=stable) replicationEFA7_2 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 7, fax = 2, fay = 2, stable=stable) replicationEFA7_3 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 7, fax = 3, fay = 3, stable=stable) replicationEFA7_4 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 7, fax = 4, fay = 4, stable=stable) replicationEFA7_5 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 7, fax = 5, fay = 5, stable=stable) replicationEFA7_6 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 7, fax = 6, fay = 6, stable=stable) replicationEFA7_7 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 7, fax = 7, fay = 7, stable=stable) replicationEFA8_1 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 8, fax = 1, fay = 1, stable=stable) replicationEFA8_2 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 8, fax = 2, fay = 2, stable=stable) replicationEFA8_3 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 8, fax = 3, fay = 3, stable=stable) replicationEFA8_4 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 8, fax = 4, fay = 4, stable=stable) replicationEFA8_5 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 8, fax = 5, fay = 5, stable=stable) replicationEFA8_6 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 8, fax = 6, fay = 6, stable=stable) replicationEFA8_7 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 8, fax = 7, fay = 7, stable=stable) replicationEFA8_8 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 8, fax = 8, fay = 8, stable=stable) replicationEFA9_1 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 9, fax = 1, fay = 1, stable=stable) replicationEFA9_2 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 9, fax = 2, fay = 3, stable=stable) replicationEFA9_3 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 9, fax = 3, fay = 2, stable=stable) replicationEFA9_4 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 9, fax = 4, fay = 7, stable=stable) replicationEFA9_5 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 9, fax = 5, fay = 4, stable=stable) replicationEFA9_6 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 9, fax = 6, fay = 5, stable=stable) replicationEFA9_7 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 9, fax = 7, fay = 6, stable=stable) replicationEFA9_8 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 9, fax = 8, fay = 8, stable=stable) replicationEFA9_9 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 9, fax = 9, fay = 9, stable=stable) replicationEFA10_1 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 10, fax = 1, fay = 1, stable=stable) replicationEFA10_2 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 10, fax = 2, fay = 3, stable=stable) replicationEFA10_3 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 10, fax = 3, fay = 2, stable=stable) replicationEFA10_4 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 10, fax = 4, fay = 5, stable=stable) replicationEFA10_5 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 10, fax = 5, fay = 7, stable=stable) replicationEFA10_6 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 10, fax = 6, fay = 4, stable=stable) replicationEFA10_7 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 10, fax = 7, fay = 6, stable=stable) replicationEFA10_8 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 10, fax = 8, fay = 1, stable=stable) replicationEFA10_9 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 10, fax = 9, fay = 10, stable=stable) replicationEFA10_10 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 10, fax = 10, fay = 1, stable=stable) replicationEFA11_1 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 11, fax = 1, fay = 1, stable=stable) replicationEFA11_2 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 11, fax = 2, fay = 2, stable=stable) replicationEFA11_3 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 11, fax = 3, fay = 3, stable=stable) replicationEFA11_4 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 11, fax = 4, fay = 4, stable=stable) replicationEFA11_5 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 11, fax = 5, fay = 5, stable=stable) replicationEFA11_6 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 11, fax = 6, fay = 6, stable=stable) replicationEFA11_7 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 11, fax = 7, fay = 7, stable=stable) replicationEFA11_8 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 11, fax = 8, fay = 8, stable=stable) replicationEFA11_9 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 11, fax = 9, fay = 9, stable=stable) replicationEFA11_10 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 11, fax = 10, fay = 1, stable=stable) replicationEFA11_11 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 11, fax = 11, fay = 1, stable=stable) replicationEFA12_1 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 12, fax = 1, fay = 1, stable=stable) replicationEFA12_2 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 12, fax = 2, fay = 3, stable=stable) replicationEFA12_3 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 12, fax = 3, fay = 2, stable=stable) replicationEFA12_4 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 12, fax = 4, fay = 5, stable=stable) replicationEFA12_5 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 12, fax = 5, fay = 8, stable=stable) replicationEFA12_6 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 12, fax = 6, fay = 7, stable=stable) replicationEFA12_7 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 12, fax = 7, fay = 6, stable=stable) replicationEFA12_8 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 12, fax = 8, fay = 1, stable=stable) replicationEFA12_9 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 12, fax = 9, fay = 9, stable=stable) replicationEFA12_10 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 12, fax = 10, fay = 4, stable=stable) replicationEFA12_11 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 12, fax = 11, fay = 10, stable=stable) replicationEFA12_12 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 12, fax = 12, fay = 1, stable=stable) replicationEFA13_1 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 13, fax = 1, fay = 1, stable=stable) replicationEFA13_2 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 13, fax = 2, fay = 2, stable=stable) replicationEFA13_3 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 13, fax = 3, fay = 3, stable=stable) replicationEFA13_4 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 13, fax = 4, fay = 5, stable=stable) replicationEFA13_5 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 13, fax = 5, fay = 4, stable=stable) replicationEFA13_6 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 13, fax = 6, fay = 8, stable=stable) replicationEFA13_7 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 13, fax = 7, fay = 6, stable=stable) replicationEFA13_8 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 13, fax = 8, fay = 7, stable=stable) replicationEFA13_9 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 13, fax = 9, fay = 10, stable=stable) replicationEFA13_10 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 13, fax = 10, fay = 9, stable=stable) replicationEFA13_11 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 13, fax = 11, fay = 11, stable=stable) replicationEFA13_12 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 13, fax = 12, fay = 1, stable=stable) replicationEFA13_13 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 13, fax = 13, fay = 12, stable=stable) replicationEFA14_1 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 14, fax = 1, fay = 1, stable=stable) replicationEFA14_2 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 14, fax = 2, fay = 2, stable=stable) replicationEFA14_3 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 14, fax = 3, fay = 3, stable=stable) replicationEFA14_4 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 14, fax = 4, fay = 5, stable=stable) replicationEFA14_5 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 14, fax = 5, fay = 4, stable=stable) replicationEFA14_6 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 14, fax = 6, fay = 8, stable=stable) replicationEFA14_7 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 14, fax = 7, fay = 6, stable=stable) replicationEFA14_8 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 14, fax = 8, fay = 7, stable=stable) replicationEFA14_9 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 14, fax = 9, fay = 9, stable=stable) replicationEFA14_10 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 14, fax = 10, fay = 10, stable=stable) replicationEFA14_11 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 14, fax = 11, fay = 11, stable=stable) replicationEFA14_12 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 14, fax = 12, fay = 12, stable=stable) replicationEFA14_13 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 14, fax = 13, fay = 14, stable=stable) replicationEFA14_14 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 14, fax = 14, fay = 13, stable=stable) replicationEFA15_1 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 15, fax = 1, fay = 1, stable=stable) replicationEFA15_2 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 15, fax = 2, fay = 2, stable=stable) replicationEFA15_3 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 15, fax = 3, fay = 3, stable=stable) replicationEFA15_4 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 15, fax = 4, fay = 4, stable=stable) replicationEFA15_5 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 15, fax = 5, fay = 5, stable=stable) replicationEFA15_6 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 15, fax = 6, fay = 6, stable=stable) replicationEFA15_7 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 15, fax = 7, fay = 8, stable=stable) replicationEFA15_8 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 15, fax = 8, fay = 9, stable=stable) replicationEFA15_9 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 15, fax = 9, fay = 7, stable=stable) replicationEFA15_10 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 15, fax = 10, fay = 10, stable=stable) replicationEFA15_11 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 15, fax = 11, fay = 8, stable=stable) replicationEFA15_12 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 15, fax = 12, fay = 14, stable=stable) replicationEFA15_13 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 15, fax = 13, fay = 11, stable=stable) replicationEFA15_14 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 15, fax = 14, fay = 15, stable=stable) replicationEFA15_15 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 15, fax = 15, fay = 13, stable=stable) replicationEFA16_1 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 16, fax = 1, fay = 1, stable=stable) replicationEFA16_2 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 16, fax = 2, fay = 2, stable=stable) replicationEFA16_3 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 16, fax = 3, fay = 3, stable=stable) replicationEFA16_4 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 16, fax = 4, fay = 5, stable=stable) replicationEFA16_5 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 16, fax = 5, fay = 4, stable=stable) replicationEFA16_6 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 16, fax = 6, fay = 8, stable=stable) replicationEFA16_7 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 16, fax = 7, fay = 6, stable=stable) replicationEFA16_8 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 16, fax = 8, fay = 7, stable=stable) replicationEFA16_9 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 16, fax = 9, fay = 9, stable=stable) replicationEFA16_10 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 16, fax = 10, fay = 10, stable=stable) replicationEFA16_11 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 16, fax = 11, fay = 11, stable=stable) replicationEFA16_12 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 16, fax = 12, fay = 16, stable=stable) replicationEFA16_13 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 16, fax = 13, fay = 14, stable=stable) replicationEFA16_14 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 16, fax = 14, fay = 13, stable=stable) replicationEFA16_15 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 16, fax = 15, fay = 12, stable=stable) replicationEFA16_16 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 16, fax = 16, fay = 15, stable=stable) replicationEFA17_1 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 17, fax = 1, fay = 1, stable=stable) replicationEFA17_2 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 17, fax = 2, fay = 2, stable=stable) replicationEFA17_3 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 17, fax = 3, fay = 5, stable=stable) replicationEFA17_4 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 17, fax = 4, fay = 4, stable=stable) replicationEFA17_5 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 17, fax = 5, fay = 3, stable=stable) replicationEFA17_6 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 17, fax = 6, fay = 7, stable=stable) replicationEFA17_7 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 17, fax = 7, fay = 8, stable=stable) replicationEFA17_8 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 17, fax = 8, fay = 9, stable=stable) replicationEFA17_9 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 17, fax = 9, fay = 6, stable=stable) replicationEFA17_10 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 17, fax = 10, fay = 10, stable=stable) replicationEFA17_11 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 17, fax = 11, fay = 11, stable=stable) replicationEFA17_12 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 17, fax = 12, fay = 14, stable=stable) replicationEFA17_13 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 17, fax = 13, fay = 15, stable=stable) replicationEFA17_14 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 17, fax = 14, fay = 16, stable=stable) replicationEFA17_15 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 17, fax = 15, fay = 13, stable=stable) replicationEFA17_16 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 17, fax = 16, fay = 12, stable=stable) replicationEFA17_17 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 17, fax = 17, fay = 17, stable=stable) replicationEFA18_1 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 18, fax = 1, fay = 2, stable=stable) replicationEFA18_2 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 18, fax = 2, fay = 1, stable=stable) replicationEFA18_3 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 18, fax = 3, fay = 5, stable=stable) replicationEFA18_4 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 18, fax = 4, fay = 4, stable=stable) replicationEFA18_5 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 18, fax = 5, fay = 3, stable=stable) replicationEFA18_6 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 18, fax = 6, fay = 6, stable=stable) replicationEFA18_7 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 18, fax = 7, fay = 8, stable=stable) replicationEFA18_8 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 18, fax = 8, fay = 7, stable=stable) replicationEFA18_9 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 18, fax = 9, fay = 9, stable=stable) replicationEFA18_10 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 18, fax = 10, fay = 1, stable=stable) replicationEFA18_11 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 18, fax = 11, fay = 10, stable=stable) replicationEFA18_12 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 18, fax = 12, fay = 11, stable=stable) replicationEFA18_13 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 18, fax = 13, fay = 13, stable=stable) replicationEFA18_14 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 18, fax = 14, fay = 16, stable=stable) replicationEFA18_15 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 18, fax = 15, fay = 1, stable=stable) replicationEFA18_16 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 18, fax = 16, fay = 12, stable=stable) replicationEFA18_17 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 18, fax = 17, fay = 15, stable=stable) replicationEFA18_18 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 18, fax = 18, fay = 17, stable=stable) replicationEFA19_1 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 19, fax = 1, fay = 1, stable=stable) replicationEFA19_2 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 19, fax = 2, fay = 3, stable=stable) replicationEFA19_3 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 19, fax = 3, fay = 2, stable=stable) replicationEFA19_4 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 19, fax = 4, fay = 5, stable=stable) replicationEFA19_5 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 19, fax = 5, fay = 4, stable=stable) replicationEFA19_6 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 19, fax = 6, fay = 8, stable=stable) replicationEFA19_7 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 19, fax = 7, fay = 6, stable=stable) replicationEFA19_8 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 19, fax = 8, fay = 7, stable=stable) replicationEFA19_9 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 19, fax = 9, fay = 9, stable=stable) replicationEFA19_10 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 19, fax = 10, fay = 2, stable=stable) replicationEFA19_11 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 19, fax = 11, fay = 10, stable=stable) replicationEFA19_12 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 19, fax = 12, fay = 1, stable=stable) replicationEFA19_13 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 19, fax = 13, fay = 11, stable=stable) replicationEFA19_14 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 19, fax = 14, fay = 15, stable=stable) replicationEFA19_15 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 19, fax = 15, fay = 18, stable=stable) replicationEFA19_16 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 19, fax = 16, fay = 12, stable=stable) replicationEFA19_17 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 19, fax = 17, fay = 19, stable=stable) replicationEFA19_18 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 19, fax = 18, fay = 16, stable=stable) replicationEFA19_19 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 19, fax = 19, fay = 1, stable=stable) replicationEFA20_1 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 20, fax = 1, fay = 1, stable=stable) replicationEFA20_2 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 20, fax = 2, fay = 2, stable=stable) replicationEFA20_3 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 20, fax = 3, fay = 3, stable=stable) replicationEFA20_4 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 20, fax = 4, fay = 4, stable=stable) replicationEFA20_5 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 20, fax = 5, fay = 1, stable=stable) replicationEFA20_6 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 20, fax = 6, fay = 7, stable=stable) replicationEFA20_7 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 20, fax = 7, fay = 9, stable=stable) replicationEFA20_8 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 20, fax = 8, fay = 5, stable=stable) replicationEFA20_9 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 20, fax = 9, fay = 8, stable=stable) replicationEFA20_10 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 20, fax = 10, fay = 16, stable=stable) replicationEFA20_11 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 20, fax = 11, fay = 11, stable=stable) replicationEFA20_12 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 20, fax = 12, fay = 13, stable=stable) replicationEFA20_13 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 20, fax = 13, fay = 10, stable=stable) replicationEFA20_14 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 20, fax = 14, fay = 6, stable=stable) replicationEFA20_15 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 20, fax = 15, fay = 12, stable=stable) replicationEFA20_16 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 20, fax = 16, fay = 1, stable=stable) replicationEFA20_17 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 20, fax = 17, fay = 15, stable=stable) replicationEFA20_18 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 20, fax = 18, fay = 17, stable=stable) replicationEFA20_19 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 20, fax = 19, fay = 20, stable=stable) replicationEFA20_20 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 20, fax = 20, fay = 14, stable=stable) replicationEFA21_1 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 21, fax = 1, fay = 4, stable=stable) replicationEFA21_2 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 21, fax = 2, fay = 3, stable=stable) replicationEFA21_3 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 21, fax = 3, fay = 1, stable=stable) replicationEFA21_4 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 21, fax = 4, fay = 2, stable=stable) replicationEFA21_5 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 21, fax = 5, fay = 2, stable=stable) replicationEFA21_6 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 21, fax = 6, fay = 7, stable=stable) replicationEFA21_7 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 21, fax = 7, fay = 12, stable=stable) replicationEFA21_8 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 21, fax = 8, fay = 5, stable=stable) replicationEFA21_9 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 21, fax = 9, fay = 9, stable=stable) replicationEFA21_10 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 21, fax = 10, fay = 8, stable=stable) replicationEFA21_11 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 21, fax = 11, fay = 16, stable=stable) replicationEFA21_12 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 21, fax = 12, fay = 13, stable=stable) replicationEFA21_13 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 21, fax = 13, fay = 14, stable=stable) replicationEFA21_14 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 21, fax = 14, fay = 1, stable=stable) replicationEFA21_15 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 21, fax = 15, fay = 10, stable=stable) replicationEFA21_16 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 21, fax = 16, fay = 15, stable=stable) replicationEFA21_17 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 21, fax = 17, fay = 1, stable=stable) replicationEFA21_18 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 21, fax = 18, fay = 18, stable=stable) replicationEFA21_19 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 21, fax = 19, fay = 1, stable=stable) replicationEFA21_20 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 21, fax = 20, fay = 1, stable=stable) replicationEFA21_21 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 21, fax = 21, fay = 1, stable=stable) replicationEFA22_1 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 22, fax = 1, fay = 3, stable=stable) replicationEFA22_2 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 22, fax = 2, fay = 1, stable=stable) replicationEFA22_3 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 22, fax = 3, fay = 4, stable=stable) replicationEFA22_4 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 22, fax = 4, fay = 2, stable=stable) replicationEFA22_5 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 22, fax = 5, fay = 7, stable=stable) replicationEFA22_6 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 22, fax = 6, fay = 5, stable=stable) replicationEFA22_7 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 22, fax = 7, fay = 15, stable=stable) replicationEFA22_8 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 22, fax = 8, fay = 9, stable=stable) replicationEFA22_9 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 22, fax = 9, fay = 2, stable=stable) replicationEFA22_10 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 22, fax = 10, fay = 6, stable=stable) replicationEFA22_11 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 22, fax = 11, fay = 8, stable=stable) replicationEFA22_12 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 22, fax = 12, fay = 13, stable=stable) replicationEFA22_13 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 22, fax = 13, fay = 17, stable=stable) replicationEFA22_14 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 22, fax = 14, fay = 12, stable=stable) replicationEFA22_15 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 22, fax = 15, fay = 14, stable=stable) replicationEFA22_16 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 22, fax = 16, fay = 1, stable=stable) replicationEFA22_17 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 22, fax = 17, fay = 19, stable=stable) replicationEFA22_18 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 22, fax = 18, fay = 11, stable=stable) replicationEFA22_19 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 22, fax = 19, fay = 22, stable=stable) replicationEFA22_20 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 22, fax = 20, fay = 1, stable=stable) replicationEFA22_21 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 22, fax = 21, fay = 18, stable=stable) replicationEFA22_22 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 22, fax = 22, fay = 21, stable=stable) replicationEFA23_1 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 23, fax = 1, fay = 4, stable=stable) replicationEFA23_2 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 23, fax = 2, fay = 2, stable=stable) replicationEFA23_3 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 23, fax = 3, fay = 1, stable=stable) replicationEFA23_4 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 23, fax = 4, fay = 3, stable=stable) replicationEFA23_5 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 23, fax = 5, fay = 5, stable=stable) replicationEFA23_6 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 23, fax = 6, fay = 8, stable=stable) replicationEFA23_7 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 23, fax = 7, fay = 23, stable=stable) replicationEFA23_8 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 23, fax = 8, fay = 6, stable=stable) replicationEFA23_9 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 23, fax = 9, fay = 10, stable=stable) replicationEFA23_10 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 23, fax = 10, fay = 9, stable=stable) replicationEFA23_11 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 23, fax = 11, fay = 13, stable=stable) replicationEFA23_12 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 23, fax = 12, fay = 2, stable=stable) replicationEFA23_13 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 23, fax = 13, fay = 1, stable=stable) replicationEFA23_14 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 23, fax = 14, fay = 11, stable=stable) replicationEFA23_15 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 23, fax = 15, fay = 12, stable=stable) replicationEFA23_16 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 23, fax = 16, fay = 7, stable=stable) replicationEFA23_17 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 23, fax = 17, fay = 20, stable=stable) replicationEFA23_18 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 23, fax = 18, fay = 15, stable=stable) replicationEFA23_19 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 23, fax = 19, fay = 16, stable=stable) replicationEFA23_20 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 23, fax = 20, fay = 14, stable=stable) replicationEFA23_21 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 23, fax = 21, fay = 1, stable=stable) replicationEFA23_22 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 23, fax = 22, fay = 1, stable=stable) replicationEFA23_23 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 23, fax = 23, fay = 21, stable=stable) replicationEFA24_1 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 24, fax = 1, fay = 1, stable=stable) replicationEFA24_2 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 24, fax = 2, fay = 2, stable=stable) replicationEFA24_3 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 24, fax = 3, fay = 3, stable=stable) replicationEFA24_4 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 24, fax = 4, fay = 6, stable=stable) replicationEFA24_5 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 24, fax = 5, fay = 4, stable=stable) replicationEFA24_6 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 24, fax = 6, fay = 8, stable=stable) replicationEFA24_7 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 24, fax = 7, fay = 9, stable=stable) replicationEFA24_8 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 24, fax = 8, fay = 24, stable=stable) replicationEFA24_9 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 24, fax = 9, fay = 5, stable=stable) replicationEFA24_10 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 24, fax = 10, fay = 13, stable=stable) replicationEFA24_11 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 24, fax = 11, fay = 10, stable=stable) replicationEFA24_12 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 24, fax = 12, fay = 1, stable=stable) replicationEFA24_13 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 24, fax = 13, fay = 7, stable=stable) replicationEFA24_14 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 24, fax = 14, fay = 2, stable=stable) replicationEFA24_15 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 24, fax = 15, fay = 12, stable=stable) replicationEFA24_16 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 24, fax = 16, fay = 1, stable=stable) replicationEFA24_17 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 24, fax = 17, fay = 11, stable=stable) replicationEFA24_18 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 24, fax = 18, fay = 16, stable=stable) replicationEFA24_19 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 24, fax = 19, fay = 14, stable=stable) replicationEFA24_20 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 24, fax = 20, fay = 17, stable=stable) replicationEFA24_21 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 24, fax = 21, fay = 15, stable=stable) replicationEFA24_22 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 24, fax = 22, fay = 1, stable=stable) replicationEFA24_23 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 24, fax = 23, fay = 1, stable=stable) replicationEFA24_24 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 24, fax = 24, fay = 19, stable=stable) replicationEFA25_1 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 25, fax = 1, fay = 1, stable=stable) replicationEFA25_2 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 25, fax = 2, fay = 2, stable=stable) replicationEFA25_3 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 25, fax = 3, fay = 3, stable=stable) replicationEFA25_4 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 25, fax = 4, fay = 6, stable=stable) replicationEFA25_5 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 25, fax = 5, fay = 7, stable=stable) replicationEFA25_6 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 25, fax = 6, fay = 4, stable=stable) replicationEFA25_7 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 25, fax = 7, fay = 17, stable=stable) replicationEFA25_8 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 25, fax = 8, fay = 8, stable=stable) replicationEFA25_9 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 25, fax = 9, fay = 9, stable=stable) replicationEFA25_10 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 25, fax = 10, fay = 5, stable=stable) replicationEFA25_11 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 25, fax = 11, fay = 16, stable=stable) replicationEFA25_12 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 25, fax = 12, fay = 1, stable=stable) replicationEFA25_13 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 25, fax = 13, fay = 10, stable=stable) replicationEFA25_14 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 25, fax = 14, fay = 2, stable=stable) replicationEFA25_15 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 25, fax = 15, fay = 11, stable=stable) replicationEFA25_16 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 25, fax = 16, fay = 23, stable=stable) replicationEFA25_17 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 25, fax = 17, fay = 18, stable=stable) replicationEFA25_18 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 25, fax = 18, fay = 1, stable=stable) replicationEFA25_19 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 25, fax = 19, fay = 15, stable=stable) replicationEFA25_20 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 25, fax = 20, fay = 12, stable=stable) replicationEFA25_21 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 25, fax = 21, fay = 21, stable=stable) replicationEFA25_22 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 25, fax = 22, fay = 22, stable=stable) replicationEFA25_23 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 25, fax = 23, fay = 14, stable=stable) replicationEFA25_24 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 25, fax = 24, fay = 24, stable=stable) replicationEFA25_25 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 25, fax = 25, fay = 1, stable=stable) replicationEFA26_1 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 26, fax = 1, fay = 2, stable=stable) replicationEFA26_2 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 26, fax = 2, fay = 3, stable=stable) replicationEFA26_3 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 26, fax = 3, fay = 1, stable=stable) replicationEFA26_4 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 26, fax = 4, fay = 8, stable=stable) replicationEFA26_5 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 26, fax = 5, fay = 6, stable=stable) replicationEFA26_6 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 26, fax = 6, fay = 4, stable=stable) replicationEFA26_7 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 26, fax = 7, fay = 13, stable=stable) replicationEFA26_8 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 26, fax = 8, fay = 5, stable=stable) replicationEFA26_9 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 26, fax = 9, fay = 9, stable=stable) replicationEFA26_10 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 26, fax = 10, fay = 7, stable=stable) replicationEFA26_11 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 26, fax = 11, fay = 14, stable=stable) replicationEFA26_12 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 26, fax = 12, fay = 1, stable=stable) replicationEFA26_13 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 26, fax = 13, fay = 10, stable=stable) replicationEFA26_14 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 26, fax = 14, fay = 18, stable=stable) replicationEFA26_15 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 26, fax = 15, fay = 11, stable=stable) replicationEFA26_16 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 26, fax = 16, fay = 20, stable=stable) replicationEFA26_17 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 26, fax = 17, fay = 19, stable=stable) replicationEFA26_18 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 26, fax = 18, fay = 1, stable=stable) replicationEFA26_19 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 26, fax = 19, fay = 12, stable=stable) replicationEFA26_20 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 26, fax = 20, fay = 16, stable=stable) replicationEFA26_21 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 26, fax = 21, fay = 17, stable=stable) replicationEFA26_22 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 26, fax = 22, fay = 25, stable=stable) replicationEFA26_23 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 26, fax = 23, fay = 23, stable=stable) replicationEFA26_24 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 26, fax = 24, fay = 21, stable=stable) replicationEFA26_25 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 26, fax = 25, fay = 26, stable=stable) replicationEFA26_26 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 26, fax = 26, fay = 1, stable=stable) replicationEFA27_1 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 27, fax = 1, fay = 2, stable=stable) replicationEFA27_2 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 27, fax = 2, fay = 3, stable=stable) replicationEFA27_3 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 27, fax = 3, fay = 1, stable=stable) replicationEFA27_4 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 27, fax = 4, fay = 5, stable=stable) replicationEFA27_5 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 27, fax = 5, fay = 6, stable=stable) replicationEFA27_6 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 27, fax = 6, fay = 7, stable=stable) replicationEFA27_7 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 27, fax = 7, fay = 12, stable=stable) replicationEFA27_8 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 27, fax = 8, fay = 4, stable=stable) replicationEFA27_9 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 27, fax = 9, fay = 11, stable=stable) replicationEFA27_10 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 27, fax = 10, fay = 17, stable=stable) replicationEFA27_11 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 27, fax = 11, fay = 8, stable=stable) replicationEFA27_12 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 27, fax = 12, fay = 9, stable=stable) replicationEFA27_13 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 27, fax = 13, fay = 15, stable=stable) replicationEFA27_14 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 27, fax = 14, fay = 14, stable=stable) replicationEFA27_15 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 27, fax = 15, fay = 16, stable=stable) replicationEFA27_16 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 27, fax = 16, fay = 10, stable=stable) replicationEFA27_17 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 27, fax = 17, fay = 13, stable=stable) replicationEFA27_18 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 27, fax = 18, fay = 18, stable=stable) replicationEFA27_19 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 27, fax = 19, fay = 20, stable=stable) replicationEFA27_20 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 27, fax = 20, fay = 19, stable=stable) replicationEFA27_21 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 27, fax = 21, fay = 21, stable=stable) replicationEFA27_22 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 27, fax = 22, fay = 26, stable=stable) replicationEFA27_23 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 27, fax = 23, fay = 23, stable=stable) replicationEFA27_24 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 27, fax = 24, fay = 22, stable=stable) replicationEFA27_25 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 27, fax = 25, fay = 25, stable=stable) replicationEFA27_26 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 27, fax = 26, fay = 24, stable=stable) replicationEFA27_27 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 27, fax = 27, fay = 14, stable=stable) replicationEFA28_1 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 28, fax = 1, fay = 2, stable=stable) replicationEFA28_2 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 28, fax = 2, fay = 1, stable=stable) replicationEFA28_3 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 28, fax = 3, fay = 3, stable=stable) replicationEFA28_4 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 28, fax = 4, fay = 6, stable=stable) replicationEFA28_5 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 28, fax = 5, fay = 7, stable=stable) replicationEFA28_6 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 28, fax = 6, fay = 5, stable=stable) replicationEFA28_7 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 28, fax = 7, fay = 15, stable=stable) replicationEFA28_8 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 28, fax = 8, fay = 4, stable=stable) replicationEFA28_9 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 28, fax = 9, fay = 10, stable=stable) replicationEFA28_10 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 28, fax = 10, fay = 8, stable=stable) replicationEFA28_11 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 28, fax = 11, fay = 9, stable=stable) replicationEFA28_12 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 28, fax = 12, fay = 17, stable=stable) replicationEFA28_13 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 28, fax = 13, fay = 14, stable=stable) replicationEFA28_14 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 28, fax = 14, fay = 1, stable=stable) replicationEFA28_15 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 28, fax = 15, fay = 12, stable=stable) replicationEFA28_16 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 28, fax = 16, fay = 16, stable=stable) replicationEFA28_17 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 28, fax = 17, fay = 11, stable=stable) replicationEFA28_18 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 28, fax = 18, fay = 18, stable=stable) replicationEFA28_19 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 28, fax = 19, fay = 19, stable=stable) replicationEFA28_20 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 28, fax = 20, fay = 1, stable=stable) replicationEFA28_21 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 28, fax = 21, fay = 1, stable=stable) replicationEFA28_22 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 28, fax = 22, fay = 26, stable=stable) replicationEFA28_23 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 28, fax = 23, fay = 1, stable=stable) replicationEFA28_24 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 28, fax = 24, fay = 1, stable=stable) replicationEFA28_25 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 28, fax = 25, fay = 20, stable=stable) replicationEFA28_26 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 28, fax = 26, fay = 23, stable=stable) replicationEFA28_27 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 28, fax = 27, fay = 25, stable=stable) replicationEFA28_28 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 28, fax = 28, fay = 13, stable=stable) replicationEFA29_1 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 29, fax = 1, fay = 3, stable=stable) replicationEFA29_2 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 29, fax = 2, fay = 1, stable=stable) replicationEFA29_3 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 29, fax = 3, fay = 2, stable=stable) replicationEFA29_4 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 29, fax = 4, fay = 8, stable=stable) replicationEFA29_5 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 29, fax = 5, fay = 5, stable=stable) replicationEFA29_6 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 29, fax = 6, fay = 6, stable=stable) replicationEFA29_7 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 29, fax = 7, fay = 15, stable=stable) replicationEFA29_8 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 29, fax = 8, fay = 9, stable=stable) replicationEFA29_9 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 29, fax = 9, fay = 7, stable=stable) replicationEFA29_10 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 29, fax = 10, fay = 1, stable=stable) replicationEFA29_11 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 29, fax = 11, fay = 4, stable=stable) replicationEFA29_12 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 29, fax = 12, fay = 10, stable=stable) replicationEFA29_13 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 29, fax = 13, fay = 12, stable=stable) replicationEFA29_14 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 29, fax = 14, fay = 17, stable=stable) replicationEFA29_15 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 29, fax = 15, fay = 11, stable=stable) replicationEFA29_16 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 29, fax = 16, fay = 2, stable=stable) replicationEFA29_17 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 29, fax = 17, fay = 13, stable=stable) replicationEFA29_18 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 29, fax = 18, fay = 18, stable=stable) replicationEFA29_19 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 29, fax = 19, fay = 1, stable=stable) replicationEFA29_20 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 29, fax = 20, fay = 19, stable=stable) replicationEFA29_21 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 29, fax = 21, fay = 25, stable=stable) replicationEFA29_22 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 29, fax = 22, fay = 16, stable=stable) replicationEFA29_23 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 29, fax = 23, fay = 23, stable=stable) replicationEFA29_24 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 29, fax = 24, fay = 20, stable=stable) replicationEFA29_25 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 29, fax = 25, fay = 1, stable=stable) replicationEFA29_26 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 29, fax = 26, fay = 1, stable=stable) replicationEFA29_27 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 29, fax = 27, fay = 24, stable=stable) replicationEFA29_28 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 29, fax = 28, fay = 27, stable=stable) replicationEFA29_29 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 29, fax = 29, fay = 26, stable=stable) replicationEFA30_1 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 30, fax = 1, fay = 2, stable=stable) replicationEFA30_2 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 30, fax = 2, fay = 3, stable=stable) replicationEFA30_3 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 30, fax = 3, fay = 1, stable=stable) replicationEFA30_4 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 30, fax = 4, fay = 8, stable=stable) replicationEFA30_5 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 30, fax = 5, fay = 5, stable=stable) replicationEFA30_6 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 30, fax = 6, fay = 6, stable=stable) replicationEFA30_7 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 30, fax = 7, fay = 15, stable=stable) replicationEFA30_8 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 30, fax = 8, fay = 7, stable=stable) replicationEFA30_9 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 30, fax = 9, fay = 9, stable=stable) replicationEFA30_10 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 30, fax = 10, fay = 12, stable=stable) replicationEFA30_11 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 30, fax = 11, fay = 10, stable=stable) replicationEFA30_12 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 30, fax = 12, fay = 4, stable=stable) replicationEFA30_13 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 30, fax = 13, fay = 11, stable=stable) replicationEFA30_14 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 30, fax = 14, fay = 1, stable=stable) replicationEFA30_15 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 30, fax = 15, fay = 18, stable=stable) replicationEFA30_16 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 30, fax = 16, fay = 23, stable=stable) replicationEFA30_17 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 30, fax = 17, fay = 1, stable=stable) replicationEFA30_18 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 30, fax = 18, fay = 13, stable=stable) replicationEFA30_19 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 30, fax = 19, fay = 17, stable=stable) replicationEFA30_20 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 30, fax = 20, fay = 19, stable=stable) replicationEFA30_21 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 30, fax = 21, fay = 16, stable=stable) replicationEFA30_22 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 30, fax = 22, fay = 25, stable=stable) replicationEFA30_23 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 30, fax = 23, fay = 20, stable=stable) replicationEFA30_24 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 30, fax = 24, fay = 1, stable=stable) replicationEFA30_25 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 30, fax = 25, fay = 30, stable=stable) replicationEFA30_26 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 30, fax = 26, fay = 29, stable=stable) replicationEFA30_27 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 30, fax = 27, fay = 22, stable=stable) replicationEFA30_28 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 30, fax = 28, fay = 28, stable=stable) replicationEFA30_29 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 30, fax = 29, fay = 1, stable=stable) replicationEFA30_30 <- EFAreplication(expSampleCor696autoFA, repSampleCor696autoFA, EFAn = 30, fax = 30, fay = 14, stable=stable) qualCountList <- list() for (factorSolution in 1:nfactors) { factorsInfo <- list() for (factorNumber in 1:factorSolution) { label <- paste("replicationEFA", factorSolution, "_", factorNumber, "$qualCount", sep="") factorsInfo[[factorNumber]] <- label rm(factorNumber) } qualCountList[[factorSolution]] <- factorsInfo rm(factorsInfo, factorSolution) } qualCountList <- unlist(qualCountList) qualCount <- list() for (i in 1:length(qualCountList)) { qualCount[[i]] <- eval(parse(text=qualCountList[i])) rm(i) } rm(qualCountList) qualCount <- unlist(qualCount) passCountList <- list() for (factorSolution in 1:nfactors) { factorsInfo <- list() for (factorNumber in 1:factorSolution) { label <- paste("replicationEFA", factorSolution, "_", factorNumber, "$structuralPassCount", sep="") factorsInfo[[factorNumber]] <- label rm(factorNumber) } passCountList[[factorSolution]] <- factorsInfo rm(factorsInfo, factorSolution) } passCountList <- unlist(passCountList) passCount <- list() for (i in 1:length(passCountList)) { passCount[[i]] <- eval(parse(text=passCountList[i])) rm(i) } rm(passCountList) passCount <- unlist(passCount) stableCountList <- list() for (factorSolution in 1:nfactors) { factorsInfo <- list() for (factorNumber in 1:factorSolution) { label <- paste("replicationEFA", factorSolution, "_", factorNumber, "$stableItemsCount", sep="") factorsInfo[[factorNumber]] <- label rm(factorNumber) } stableCountList[[factorSolution]] <- factorsInfo rm(factorsInfo, factorSolution) } stableCountList <- unlist(stableCountList) stableCount <- list() for (i in 1:length(stableCountList)) { stableCount[[i]] <- eval(parse(text=stableCountList[i])) rm(i) } rm(stableCountList) stableCount <- unlist(stableCount) allItemsCorList <- list() for (factorSolution in 1:nfactors) { factorsInfo <- list() for (factorNumber in 1:factorSolution) { label <- paste("replicationEFA", factorSolution, "_", factorNumber, "$allItemsCor", sep="") factorsInfo[[factorNumber]] <- label rm(factorNumber) } allItemsCorList[[factorSolution]] <- factorsInfo rm(factorsInfo, factorSolution) } allItemsCorList <- unlist(allItemsCorList) allItemsCor <- list() for (i in 1:length(allItemsCorList)) { allItemsCor[[i]] <- eval(parse(text=allItemsCorList[i])) rm(i) } rm(allItemsCorList) allItemsCor <- round(unlist(allItemsCor), 3) qualItemsCorList <- list() for (factorSolution in 1:nfactors) { factorsInfo <- list() for (factorNumber in 1:factorSolution) { label <- paste("replicationEFA", factorSolution, "_", factorNumber, "$qualItemsCor", sep="") factorsInfo[[factorNumber]] <- label rm(factorNumber) } qualItemsCorList[[factorSolution]] <- factorsInfo rm(factorsInfo, factorSolution) } qualItemsCorList <- unlist(qualItemsCorList) qualItemsCor <- list() for (i in 1:length(qualItemsCorList)) { qualItemsCor[[i]] <- eval(parse(text=qualItemsCorList[i])) rm(i) } rm(qualItemsCorList) qualItemsCor <- round(unlist(qualItemsCor), 3) passCorList <- list() for (factorSolution in 1:nfactors) { factorsInfo <- list() for (factorNumber in 1:factorSolution) { label <- paste("replicationEFA", factorSolution, "_", factorNumber, "$passedItemsCor", sep="") factorsInfo[[factorNumber]] <- label rm(factorNumber) } passCorList[[factorSolution]] <- factorsInfo rm(factorsInfo, factorSolution) } passCorList <- unlist(passCorList) passCor <- list() for (i in 1:length(passCorList)) { passCor[[i]] <- eval(parse(text=passCorList[i])) rm(i) } rm(passCorList) passCor <- round(unlist(passCor), 3) stableCorList <- list() for (factorSolution in 1:nfactors) { factorsInfo <- list() for (factorNumber in 1:factorSolution) { label <- paste("replicationEFA", factorSolution, "_", factorNumber, "$stableItemsCor", sep="") factorsInfo[[factorNumber]] <- label rm(factorNumber) } stableCorList[[factorSolution]] <- factorsInfo rm(factorsInfo, factorSolution) } stableCorList <- unlist(stableCorList) stableCor <- list() for (i in 1:length(stableCorList)) { stableCor[[i]] <- eval(parse(text=stableCorList[i])) rm(i) } rm(stableCorList) stableCor <- round(unlist(stableCor), 3) allCount <- rep(696, 465) replicationTable <- data.frame(allCount, qualCount, passCount, stableCount, allItemsCor, qualItemsCor, passCor, stableCor) rowNameList <- list() n <- 0 for (i in 1:nfactors) { for(j in 1:i) { n <- n+1 rowNameList[n] <- paste("EFA", i, "_", j, sep="") } } rm(j, n) rownames(replicationTable) <- unlist(rowNameList) colnames(replicationTable) <- c("All Items", "Qual Items", "Pass Items", "Stable Items", "All Items Cor", "Qual Items Cor", "Pass Items Cor", "Stable Cor") AllItemsCorMean <- list() for (i in 1:nfactors) { x <- mean(abs(replicationTable[c(which(rownames(replicationTable)==paste("EFA", i, "_1", sep="")):which(rownames(replicationTable)==paste("EFA", i, "_", i, sep=""))),"All Items Cor"])) AllItemsCorMean[i] <- x } AllItemsCorMean <- unlist(AllItemsCorMean) rm(x) QualItemsCorMean <- list() for (i in 1:nfactors) { x <- mean(abs(replicationTable[c(which(rownames(replicationTable)==paste("EFA", i, "_1", sep="")):which(rownames(replicationTable)==paste("EFA", i, "_", i, sep=""))),"Qual Items Cor"])) QualItemsCorMean[i] <- x } QualItemsCorMean <- unlist(QualItemsCorMean) rm(x) ReplicPropMean1 <- list() for (i in 1:nfactors) { x <- mean(replicationTable[c(which(rownames(replicationTable)==paste("EFA", i, "_1", sep="")):which(rownames(replicationTable)==paste("EFA", i, "_", i, sep=""))),"Pass Items"]) y <- mean(replicationTable[c(which(rownames(replicationTable)==paste("EFA", i, "_1", sep="")):which(rownames(replicationTable)==paste("EFA", i, "_", i, sep=""))),"All Items"]) ReplicPropMean1[i] <- x/y } ReplicPropMean1 <- unlist(ReplicPropMean1) rm(x,y) ReplicPropMean2 <- list() for (i in 1:nfactors) { x <- mean(replicationTable[c(which(rownames(replicationTable)==paste("EFA", i, "_1", sep="")):which(rownames(replicationTable)==paste("EFA", i, "_", i, sep=""))),"Pass Items"]) y <- mean(replicationTable[c(which(rownames(replicationTable)==paste("EFA", i, "_1", sep="")):which(rownames(replicationTable)==paste("EFA", i, "_", i, sep=""))),"Qual Items"]) ReplicPropMean2[i] <- x/y } ReplicPropMean2 <- unlist(ReplicPropMean2) rm(x,y) QualPropMean <- list() for (i in 1:nfactors) { x <- mean(replicationTable[c(which(rownames(replicationTable)==paste("EFA", i, "_1", sep="")):which(rownames(replicationTable)==paste("EFA", i, "_", i, sep=""))),"Qual Items"]) y <- mean(replicationTable[c(which(rownames(replicationTable)==paste("EFA", i, "_1", sep="")):which(rownames(replicationTable)==paste("EFA", i, "_", i, sep=""))),"All Items"]) QualPropMean[i] <- x/y } QualPropMean <- unlist(QualPropMean) rm(x,y) StablePropMean1 <- list() for (i in 1:nfactors) { x <- mean(replicationTable[c(which(rownames(replicationTable)==paste("EFA", i, "_1", sep="")):which(rownames(replicationTable)==paste("EFA", i, "_", i, sep=""))),"Stable Items"]) y <- mean(replicationTable[c(which(rownames(replicationTable)==paste("EFA", i, "_1", sep="")):which(rownames(replicationTable)==paste("EFA", i, "_", i, sep=""))),"All Items"]) StablePropMean1[i] <- x/y } StablePropMean1 <- unlist(StablePropMean1) rm(x,y) StablePropMean2 <- list() for (i in 1:nfactors) { x <- mean(replicationTable[c(which(rownames(replicationTable)==paste("EFA", i, "_1", sep="")):which(rownames(replicationTable)==paste("EFA", i, "_", i, sep=""))),"Stable Items"]) y <- mean(replicationTable[c(which(rownames(replicationTable)==paste("EFA", i, "_1", sep="")):which(rownames(replicationTable)==paste("EFA", i, "_", i, sep=""))),"Qual Items"]) StablePropMean2[i] <- x/y } StablePropMean2 <- unlist(StablePropMean2) rm(x,y) Factors <- 1:30 meanAvailable <- list() for (i in 1:nfactors) { x <- 696/i/696 meanAvailable[i] <- x } meanAvailable <- unlist(meanAvailable) rm(x) suppressPackageStartupMessages(library(tidyr)) suppressPackageStartupMessages(library(ggplot2)) suppressPackageStartupMessages(library(grid)) ReplicationStatistics.df <- data.frame(Factors, QualItemsCorMean, AllItemsCorMean, ReplicPropMean1, ReplicPropMean2, QualPropMean, StablePropMean1, StablePropMean2, meanAvailable) ReplicationStatistics.df2 <- gather(ReplicationStatistics.df, key="variable", value="value", which(colnames(ReplicationStatistics.df) %in% c("ReplicPropMean2", "StablePropMean2", "QualPropMean", "ReplicPropMean1", "StablePropMean1", "meanAvailable"))) ReplicationStatistics.df2$variable <- factor(ReplicationStatistics.df2$variable, levels = c("ReplicPropMean2", "StablePropMean2", "QualPropMean", "ReplicPropMean1", "StablePropMean1", "meanAvailable")) ggplot(data = ReplicationStatistics.df2, aes(x=Factors, y = value, color = variable, lty=variable)) + geom_line() + scale_colour_manual("", labels = c("Replicated/Qualified", "Stable/Qualified", "Qualified/All", "Replicated/All", "Stable/All", "Mean Available"), values = c("black", "blue", "red", "black", "blue", "green")) + scale_linetype_manual("", labels = c("Replicated/Qualified", "Stable/Qualified", "Qualified/All", "Replicated/All", "Stable/All", "Mean Available"), values = c(1, 1, 2, 2, 2, 1)) + xlab("Factors extracted") + scale_y_continuous("Mean proportion of items on same factor", limits=c(0,1)) + theme_bw() ggsave(filename = paste(filepathfig, "MeanPropBySolution.pdf", sep = ""), scale = 1, width = 10, height = 8, units = c("in"), dpi = 300) ggplot(data = ReplicationStatistics.df, aes(x=Factors)) + geom_line(aes(y = AllItemsCorMean, color="All Items")) + geom_line(aes(y = QualItemsCorMean, color="Qualified Items")) + scale_colour_manual("", breaks = c("All Items", "Qualified Items"), values = c("black", "blue")) + xlab("Factors extracted") + scale_y_continuous("Mean abs value of correlations of item loadings by factor solution", limits=c(0,1)) + theme_bw() ggsave(filename = paste(filepathfig, "MeanCorsBySolution.pdf", sep = ""), scale = 1, width = 10, height = 8, units = c("in"), dpi = 300) save(replicationTable, file=paste(filepathdata, "replicationTable.rdata", sep="")) rm(allCount, allItemsCor, AllItemsCorMean, i, passCor, passCount, qualCount, qualItemsCor, QualItemsCorMean, QualPropMean, replicationTable, ReplicPropMean1, ReplicPropMean2, rowNameList, stable, stableCor, stableCount, StablePropMean1, StablePropMean2) @ \end{scriptsize} For clarity, let's review. This part is both critical and a bit confusing. All I'm doing is comparing the matching factors across samples at each level of extraction (from 1 to 30). First, I went through the tables in Appendix \hyperref[sec:appendixA]{A} to figure out how the factors lined up across samples according to item content. After the factors were aligned, I generated quantitative statistics for comparing the solutions. Specifically, these statistics included lists of: \begin{enumerate} \item the number of \textit{qualified} items in common factors across samples \item the number of \textit{qualified} items in common factors across samples that passed the test for structural replicability (denoted in the code/output as a structual PASS). Structural replicability was indicated if a qualified item loaded on the same factor in both solutions. \item the number of \textit{qualified} items in common factors across samples with stable loadings (denoted in the code/output as STABLE). Stable items were those with similar magnitudes of loadings defined by a difference in loadings of less than $\left|0.1\right|$ across samples. \item the correlations among loadings for the items. These included (a) all of the 696 items; (b) all of the qualified items on each factor in each solution; (c) all of the structurally invariant items (qualified items that ``passed'' in \#2 above); and (d) all of the stable items (the items in \#3 above). \end{enumerate} Two figures help to describe these analyses.\footnote{I welcome suggestions about alternative methods for visualizing and interpreting these results. I was not able to readily identify existing methods in the factor analytic literature, presumably because few people bother to conduct EFA replications.} Figure \ref{fig:MeanPropBySolution} plots the proportion of items that replicate. The dotted lines at the bottom show the mean proportion of qualified items (red dotted), structurally invariant items (black dotted), and structurally stable items (blue dotted) per factor relative to the total item count (696 items). The green line is included as reference --- it is the inverse of the number of factors extracted. These lines decrease rapidly as a function of complexity; as more factors are extracted, the number of items with primary loadings on any one factor decreases. The top two lines show the relationship between the qualified items, the structurally invariant items and the structurally stable items. There is considerable variability in the proportion of replicated/stable items to the qualified items. As with any proportion, this is partly driven by changes in the denominator (spikes and dips in the number of qualified items from one solution to the next) or the numerator (changes in the number of replicated/stable item counts) or both. It is important to remember that the values being plotted are means across all of the factors at each level of extraction --- this explains why the qualified/all proportion is sometimes greater than 1/factors. All else being equal, higher mean proportions of stable items are better as this suggests a better distribution of stable items across the factors. \begin{center} \begin{figure} \ffigbox{\includegraphics[width=.75\textwidth]{figures/MeanPropBySolution.pdf}}{\caption{The mean proportion of items loading on the same factor across samples at each level of extraction.}\label{fig:MeanPropBySolution}} \end{figure} \end{center} Figure \ref{fig:MeanPropBySolution} shows the mean correlations of item loadings by factor for each factor solution. For example, the mean correlation for the 22 factor solution is found by determining the correlation between item loadings across samples for the 1st factor and then the 2nd factor and so on through all 22 factors... and then finding the average of those correlations. Note that the plot for the qualified items is missing a value for the 26 factor solution because it contains a junk factor without any qualified items. Higher values indicate better correspondence between samples. As with the proportions, the expected correlation should decrease as a function of the complexity of the solution (i.e., the number of factors extracted). This means it is not accurate to say that solutions with more factors are necessarily worse than those with fewer factors solely because they have lower correlations. \begin{figure} \ffigbox{\includegraphics[width=.75\textwidth]{figures/MeanCorsBySolution.pdf}}{\caption{The mean absolute value of the correlations between factor loadings across samples.}\label{fig:MeanCorsBySolution}} \end{figure} In sum, the replication analyses point to similar conclusions. Both provide excellent support for the 5 factor solution. The proportional approach indicates that the 5 factor solution replicates better than all others beyond 1 factor; the correlational approach indicates that it replicates as well as all solutions with fewer factors extracted. Beyond the Big Five, several solutions indicated locally superior replicability relative to nearby solutions. The 17 and 27 factor solutions were most notable, followed by 14 and 22. While the 17 factor solution had higher mean correlations and higher proportions of replicated/stable items in absolute terms, the 27 factor solution replicated nearly as strongly despite the larger number of factors extracted. Solutions with 10, 15, and 20 to 26 factors replicated poorly. \subsection{How many factors?} At this point, I have identified the fit statistics and eigenvalues for each of the factor solutions in both the Exploratory and Replication samples, identified the number of qualified items for each factor in each factor solution, aligned the factors at each level of extraction across samples, and considered several metrics of replicability for each solution. Enough already. So, what is the conclusion? Well, there appear to be three levels in the data. One at 5 factors, one at around 17, and a third at 26 or 27 factors. The first of these is the most clear and least surprising. While it is reassuring that the Big Five emerged once again, it would have been more surprising --- and a little concerning --- if this hadn't happened. The replicability of this solution has been almost universally supported elsewhere and, in any case, the deck was stacked a bit in that many of the items in this sample were taken from measures based on Big Five and Big Six solutions (though some other types were included and the number of items taken from Big Five and Big Six measures was roughly equal). It is worth noting that the 6 factor solution is not nearly as good as 5 and, more importantly, it does not reflect the familiar six factor structure. Instead of the HEXACO, the replicated six factors in these samples were Sociability, Emotional Reactivity, Compassion, Conscientiousness, Intellect, and... something like Boldness/Surgency (or whatever the reverse-coded label might be; see Tables A1-41 and A1-42 in Appendix \hyperref[sec:appendixA]{A}). The level at around 17 factors is less clear than the 5 factor solution but still quite good. In the Exploratory sample, parallel analysis indicated that 17 was the optimal solution, but this same approach suggested 27 in the larger Replication sample. Similarly, the empirical BIC statistic suggested 19 factors in the Exploratory sample but the value was considerably higher in the Replication sample (25). Complexity plots for both samples suggested slightly less complex solutions at 15 and 16 factors but the reduction in complexity was more noticeable at higher levels (25-27). The number of qualified items in both samples suggested that the 17 factor solutions were acceptable but not clearly better than the nearby solutions (15-19 factors). The most compelling evidence in support of the 17 factor solution stems from the replication analyses --- among factor solutions with more than 9 factors, it had the highest mean proportion of structurally invariant and stable items and the highest average correlations among factor loadings. If considered by itself, I think there is sufficient justification for the 17 factor solution to be used as the basis for an assessment model. However, I did not feel there was sufficient justification to include it as the middle level of a hierarchical model with 3 levels. This is because the 17 factor solution is fully accounted for by the 27 factor solution. 15 of the 17 factors are functionally equivalent to those in the 27 factor solution and the remaining two factors are combinations of factors in the 27 factor solution. One of these two is a combination of Industriousness and Orderliness (see Tables A1-277 and A1-282 in Appendix \hyperref[sec:appendixA]{A}). The other is a combination of Imaginative and Cultured (see Tables A1-287 and A1-290).\footnote{The full list of 17 factors --- using labels assigned by me --- includes Sociability, Positivity, Conscientiousness (a combination of Industriousness and Orderliness), Compassion, Irritability, Honesty, Adventurousness, Openness (a combination of Imaginative and Cultured), Intellect, Worry, Conservatism, Humor, Impulsivity, Trust, Humility, Inactivity, and Self-Discipline.} Given this overlap and the additional utility of the 27 factor solution, I decided to not to include the 17 factor solution in a 3-tier hierarchy, opting instead to focus only on the higher and lower levels. The 27 factor solution replicated well across the samples and also produced a factor structure that was both clear and interpretable. While none of the fit statistics indicated that 27 factors was the best solution, several pointed to nearby solutions (25, 26, 29) and 27 factors was supported by the parallel analysis in the Replication sample. The qualified item counts at 27 factors were excellent in the Exploratory sample and very good in the Replication sample (there was 1 under-identified factor). The replication analyses also provided strong support. The 27 factor solution replicated more strongly, in absolute terms, than all of the solutions with more than 18 factors. By some criteria (the mean proportion of replicated/qualified items and correlations among loadings for qualified items), the 27 factor solution outperformed all levels above 9 except for 14 and 17. I think it's reasonable to claim that the 27 factor solution replicated better than any other (except 5) \textit{after} accounting for the substantially higher levels of complexity.\footnote{Unfortunately, I can't quite figure out how to quantify/adjust for the larger number of parameters being estimated so... until someone does, it's probably wise not to over-interpret that claim.} Certainly, none of the nearby solutions performed nearly as well and several of these contained multiple junk factors. Further consideration of the merits of this solution are provided in the commentary section below. So, how many factors? I advocate for a hierarchical model that has a level with 5 factors and a level with 27 factors. As Robert Sternberg \citeyearpar{Sternberg:1990} observed, academic psychologists tend to be either lumpers or splitters when it comes to factor structure. His reference was specific to the organization of cognitive abilities, but the same might be said for other features of personality. For some researchers (or, more often, some assessment settings), one approach may be preferred. The use of hierarchical measures makes it possible to have both options. The 5 factor level will allow for assessment that is consistent with the massive literature on the relationships between the Big Five and a host of other topics. The 27 factor level allows for more specific and narrow assessment, as is needed for prediction. While this hierarchy does not reflect a beautifully symmetrical theoretical model like the NEO-PI-R\texttrademark \citep{Costa:92a} or HEXACO \citep{Ashton:04, Lee:04}, it offers the alternate benefit of empirical derivation --- a quality that will hopefully make it more replicable, more predictive, and more amenable to iterative improvement. \subsection{Commentary on the levels identified} I did not expect that any of the solutions with more than a dozen factors or so would emerge to be clearly superior than the rest.\footnote{As I mentioned earlier, my first attempt at these analyses was conducted as part of my dissertation. At the time of my defense, only the Exploratory sample had been collected (barely) and I only ran the EFA up to 15 factors because of the time and energy required... this was only one part of my dissertation, after all. I later expanded the scope at the urging of two colleagues who pointed out that 15 was not sufficient given the theoretical structure proposed by Costa \& McCrae.} Nor do I believe that I ``discovered'' a structure because I wanted to find something interesting in the data. I was fully prepared to derive scales based on a more manageable number of factors.\footnote{Again, the evidence for this is my dissertation.} Still, I acknowledge the possibility that bias has crept in because of the incentives to find \textit{something}. If so, I trust that I will be corrected following re-evaluation by others. In any case, the broader goal of these analyses is to develop a predictive assessment model that is incrementally superior to those that are currently available. I think the 5 and 27 factor `heterarchy' is a good start but we won't know until we evaluate validity. It might take a few iterations to move things forward. Now, I want to make some additional observations about future attempts at replication, overfactoring, and validity. \subsubsection{The likelihood for further replication} With regards to interpretation of the factors in terms of the likelihood for further replication... I wish I could say that there is one or more factor models (beyond the regularly referenced five or six factors) that is likely to replicate in \textit{all} reasonably sized samples. Or even many samples. I'm not comfortable saying that without many conditional statements about the characteristics of the participant sample and the item sample. The many sensitivity analyses discussed in Section \ref{sec:sensitivity} were presented for good reasons. The samples used here are large but idiosyncratically different from the populations of all possible participants and items. All other samples will also differ idiosyncratically and these differences will directly affect the covariances between items and, therefore, the resultant EFA output. I think the 27 factor structure is likely to replicate if a similar item set is administered to a large and diverse pool of participants.\footnote{Someone should do this with the ESCS sample. I don't know what to predict about the strength of replication. That sample is considerably less diverse than the ones used here on several variables that \textit{probably} affect the structure, but I don't know whether (or to what extent) that's true. Also note that the ESCS was not administered 79 of the Eysenck items included among the 696 used here. A proper analysis would begin by evaluating the similarity of the correlation matrices for the 617 common items.} \subsubsection{Heterarchy and the potential for overfactoring} \label{sec:overfactoring} With regards to the possibility of overfactoring, it is important to acknowledge that the majority of solutions produced one or more factors that were uninterpretable across samples and this was especially true for the solutions with more factors. Only the solutions with 1-9, 14, 15, 17 and 27 factors were fully interpretable in both solutions due to the presence of uninterpretable or junk factors. It is my anecdotal experience that the term ``overfactoring'' is used in psychometric circles to suggest that a grave error has been committed; factors have been identified that are not likely to replicate or are not meaningful. In the 27 factor solution, there are a few weak factors that I view as problematic.\footnote{These are denoted in the next section as factors 21, 23, and 26.} They are problematic because they include relatively few items and/or include item content that has not received much attention in the personality literature. While I acknowledge the risk of overfactoring (I spent a lot of time worrying about this concern), the replicability of these factors supports their retention despite the weakness. In general, my concerns about the spector of overfactoring have waned considerably during the course of this research. This is partly driven by the recognition that overfactoring will work itself out during the course of replication and validation analyses. Meaningless factors will not survive for long. However my views have also been affected by a more nuanced appreciation for the variable strength of factors. Anyone who has studied a loadings matrix has probably recognized that some factors have more items with primary loadings than others. But the relative strength of factors will change when more or fewer factors are extracted. In this project, I extracted from 1 to 30 factors in two samples and was constantly reminded that the relative strength of factors at any one level depends on (a) the internal structure of the factors at lower levels (solutions with fewer factors extracted), (b) the strength of association among factors at lower levels, and (c) the characteristics of the sample. All of these features (a, b, and c) are directly implied by the statement that personality structure is \textbf{heterarchical}. It's not just a hierarchy with structural dependency across levels. It can be further classified as a heterarchy because the structural associations across AND within levels \textit{change} based on characteristics of the sample from which they were derived.\footnote{And probably also based on characteristics of the situation. This is not particularly relevant to cross sectional evaluations of structure (like these). I only mention it as a preemptive retort to those who would argue that I am confusing the vagaries of sampling for dynamism in structure.} Factors that represent basic, biologically determined differences among all humans (or any animal, for that matter) should be relatively strong in most samples. Eysenck posited Extraversion and Neuroticism as examples. But traits --- and factors --- can be placed on a spectrum of universality that includes emic and even idiosyncratic features of personality. This dimension of universality would be pretty well reflected in the relative strength of factors if the sample (of items and respondents) were perfectly representative of the population (of items and respondents). Of course, they aren't. The relative strength of factors --- that is, the relationship between the factors across and within levels --- will differ depending on the sampling characteristics. This is relevant to concerns about overfactoring because more damage is done by throwing out weak factors than by checking to see if they replicate. Weak factors may indicate the existence of dimensions that are not well covered by any of the frameworks that contribute to the 696 items in these samples.\footnote{Recall the prolonged discussion of representativeness in item sampling in Section \ref{sec:itemsamplingissues}.} Such dimensions may only contain a few items because the \textit{item sample} only contains a few items with relevant content. Similarly, the weakness of some factors may reflect characteristics of the participant sample. For example, at least two of the weak factors in the 27 factor solution are comprised of content that relates to maladaptive traits. These factors may not capture considerable variance in large normative samples (like those used here) but may have high predictive utility for clinical applications. And, as previously mentioned, the case for retaining weak factors in this 27 factor solution is buoyed by the fact that they replicated across both samples. \subsubsection{Expectations about validity} With regards to interpretation of the factors in terms of expectations about validity, I have only three comments. First, it doesn't matter what I think about validity. Validity is an empirical concern and there are many ways and criterion against which it should be evaluated. (But I won't let this stop me from talking about it.) Second, I \textit{expect} a measurement model based on the 27 factor solution described here to be comparatively more valid than measurement tools at similar levels of specificity. This may be a somewhat controversial claim. I only mean to suggest that these factors should be more predictive of individual differences in ``other stuff'' than rationally/theoretically derived models (assessed with these same items) because the 27 factors described here are empirical indicators of \textit{actual differences} in responses to the items. Here again, several considerations should be made about the sample characteristics, but I believe that the empirical approach will ultimately prove more useful than the rational approach. If not these 27 factors, then some better informed revision of them. We'll see. Third, the face validity of the factors in the 27 factor solution (for that matter, in all of the factor solutions) varies considerably. I agonized a great deal about whether solutions with 25 or more factors should be dismissed on the grounds that this \textit{felt} like overfactoring, especially given the identification of a few weaker factors (see above). These concerns evaporated when I realized that \textbf{evaluations of the validity of various factor solutions} is different from \textbf{evaluations of the validity of the underlying factors}. It may be that one or more of these factors should be skipped when conducting assessments because it/they do not relate to ``other stuff'' that is meaningful. Not all factors need to predict Very Important Outcomes. Some of them probably don't and that's fine. But, the removal of factors \textit{at this stage} based on face validity or theory (aka bias) would undermine all of the comparative analyses conducted on these samples. \subsubsection{Open questions for future research on personality structure} There are a couple of questions that need further evaluation. The most obvious is whether we should continue to look beyond 30 factors. I think we definitely should but only if we can increase the scope of data collection by another order of magnitude. Given the effort required to evaluate large solutions in multiple samples (there are few economies of scale in terms of researcher effort) and the increasing frequency of junk factors in larger extractions, I would not bother extracting 40-50 factors unless I had large samples with 1000 items or more. If I could figure out a method to collect multiple large samples with 5000 items or so (I suspect this is roughly the number of public-domain items in existence), I might be willing to extract as many as 100 factors. That said, I don't personally believe there are likely to be more than 40 or 50 traits with generalizable predictive utility\footnote{To be clear, I think there are probably several dozen meaningful traits, but most of these are not univerally salient. Instead, they are salient only in selected groups and they may, in fact, be highly predictive with those groups. I don't mean to say there are only 40 meaningful traits.} and I think that most of those (25-27) are probably in this 27 factor solution. Another critical question relates to the representativeness of the 700 items administered to these samples. I have written extensively about this topic in Section \ref{sec:IPIPselection} and in other places \citep{Condon:14Diss}, but I view this as the most nagging question. We cannot have complete confidence in models at any level of specificity unless someone can demonstrate with some certainty that we've covered all of the important constructs. I'm not yet convinced this can be done. \section{Identify new scales} \label{sec:scales} Now that we have identified the desired level(s) of extraction, we want to make scales for assessing personality at those levels with as few items as possible. The challenge here is to describe the steps used to select the items in a manner that is both consistent and clear. The exact method is described in considerable detail below but, in general, I used items with the highest and \textit{most stable} loadings on each factor, while also taking several other characteristics of the items under consideration. Going into the item selection process, I had a hunch that it would make more sense to select the items first at the more specific level because I figured that many of those items would also work well at the less specific level. I planned to add items for the 5 factor scales as needed to boost internal consistency, but this did not prove necessary because there were sufficient overlapping items to make 5 factor scales with high internal consistency. In this section, we first import results of the item-level qualitative analyses (reported more thoroughly in Condon, Graham, Weston \& Mroczek (in prep)) and consolidate them with the results from the exploratory factor analyses (the loadings). Then we use this info to select the items for the 27 factor scales. Next, we select items for the 5 factor scales from the pool of items in the 27 factor scales. Finally, we make scoring keys for all of the scales before concluding with some comments on the practice of using response styles to influence scale construction. \subsection{Item-level qualitative analyses} \label{sec:itemqualitative} Before constructing the scales, we need to review/clarify which characteristics of the items were considered in addition to the loadings and why. There are several reasons why it can be problematic to rely only on the factor loadings when choosing items. The most basic reason is that this will prioritize internal consistency at the expense of validity. This is because it tends to de-emphasize the least salient aspects of the extracted factors, and this can reduce (1) the fit between the resultant measurement model (the scales) and the structural model (the factors) and (2) the utility (predictive validity) of the measurement model across a wide range of situations and individuals. Second, the selection of items based solely on the magnitudes of the loadings substantially increases the likelihood that two or more items will have highly redundant content. This is not only inefficient but also frequently leads to so-called ``enemy'' items in the context of computer adaptive testing. Of course, items also differ from one another in terms of non-psychometric, qualitative properties. Examples include the readability, clarity, and salience of the items. By this I mean that the items: (1) do not use language and vocabulary that is equivalently sophisticated (they are not equally readable); (2) are not equivalently clear in their wording (clarity); and (3) are not perceived to be equally relevant for personality assessment. While these item properties certainly influence the results of psychometric analyses, it's hard to know how. In an attempt to guard against the potential use of items that are not clear, salient or easily understood by a diverse range of participants, several types of qualitative analyses were conducted before final selection of the items to be included in the scales at each level. These specifically included analyses of readability, perceived clarity, and salience for the 696 items used in these samples (and 104 other items that are not described here). The rationale and results for each of these analyses are described below. \subsubsection{Readability analyses} \label{sec:readability} The rationale for conducting readability analyses on the items should be clear; it is imperative that the item content is as widely comprehensible as possible. Certainly, universal comprehension is not a reasonable goal. Sadly, many individuals are fully or partially illiterate and therefore unable to read most or all of the items. Many more individuals do not speak or read English as their native language and the use of uncommon words or complex sentence structure may pose a problem for these individuals. It is also the case that items might not have uniform clarity due to vague wording, cultural/regional variations in word meaning, and/or the use of colloquialisms. Many methods are available for evaluating readability. One of the most popular tools is the Lexile text analyzer, a proprietary (though freely available) tool owned by MetaMetrics. Estimated lexile scores for each of the items was obtained by entering the text directly into the Lexile text analyzer at \url{https://lexile.com/analyzer/}. We also used the \texttt{R} package `koRpus' \citep{R:koRpus} to evaluate the readability of the items using other non-proprietary metrics. These included the Flesch-Kincaid measure of reading grade level, the Flesch-Kincaid estimate of reading age level, the SMOG reading grade level, the SMOG estimate of reading age level, the Automatic Readability Index (based on reading grade level), and the Coleman Liau reading grade level. It should be noted that no assessment of readability is particularly well-suited to brief phrases or short sentences. This is mainly due to the fact that sentence length is often used as one of the parameters in determining difficulty (along with word frequency). It is possible that very short sentences about complicated or nuanced topics would reflect a low reading level even though responding might require relatively mature or abstract cognitive abilities.\footnote{By way of anecdote regarding enigmatic Lexile scores, consider that all three of these book titles have similar results according the Lexile website: \textit{Anna Karenina} by Leo Tolstoy (Lexile score of 1080), \textit{Ulysses} by James Joyce (1050), and \textit{Diary of a Wimpy Kid} by Jeff Kinney (1060).} The following code chunk shows how the items were analyzed using the koRpus package and also pulls in the Lexile estimates as derived directly on the Lexile website. These values will be used (along with estimates of clarity and salience) for selecting items in the next section. \begin{scriptsize} <>= suppressPackageStartupMessages(library(koRpus)) # Individual users will need to download and install the koRpus package. # The koRpus package also requires a free, open-source app called # TreeTagger to tokenize the text. # See http://www.cis.uni-muenchen.de/~schmid/tools/TreeTagger/ for # more info. The following line simply points to the local installation # of TreeTagger and will need to be customized for those who want # to reproduce this section of code. set.kRp.env(TT.cmd="/Users/dmc174/bin/TreeTagger/cmd/tree-tagger-english", lang="en") FKage <- list() FKgrade <- list() SMOGage <- list() SMOGgrade <- list() ARI <- list() CL <- list() items <- list() @ \end{scriptsize} \begin{scriptsize} <>= # ItemInfo696 contains the item content for all the items for (i in 1:nrow(ItemInfo696)) { write(ItemInfo696[i,1], file = paste(filepathdata, "items/", rownames(ItemInfo696)[i], ".txt", sep ="")) item <- tokenize(paste(filepathdata, "items/", rownames(ItemInfo696)[i], ".txt", sep =""), lang = "kRp.env") rdb <- readability(item) FKage[i] <- rdb@Flesch.Kincaid$age SMOGage[i] <- rdb@SMOG$age FKgrade[i] <- rdb@Flesch.Kincaid$grade SMOGgrade[i] <- rdb@SMOG$grade ARI[i] <- rdb@ARI$grade CL[i] <- rdb@Coleman.Liau$grade items[i] <- rownames(ItemInfo696)[i] } @ \end{scriptsize} \begin{scriptsize} <>= readability <- data.frame(unlist(FKage), unlist(SMOGage), unlist(FKgrade), unlist(SMOGgrade), unlist(ARI), unlist(CL)) readability$readage <- (readability[,1]+readability[,2])/2 readability$readgrade <- (rowSums(readability[,3:6])/4) colnames(readability) <- c("FKage", "SMOGage", "FKgrade", "SMOGgrade", "ARIgrade", "CLgrade", "readage", "readgrade") rownames(readability) <- unlist(items) # add Lexile scores load(paste(filepathdata, "QualitativeInfo.Rdata", sep="")) lexile <- lexile696 readability$lexile <- lexile[rownames(readability), c("lexile")] @ \end{scriptsize} \subsubsection{Item clarity and salience} \label{sec:clarsal} Independent of reading level evaluations, data were collected to assess the perceived clarity and salience of each item. The rationale for this data collection was based on recognition that the first author's familiarity with (indeed, passion for) the corpus of personality items may have allowed for more frequent inclusion of items with personality psychology jargon. For example, anecdotal experience suggests that many well-educated people are not familiar with the phrase ``shirk my duties.''\footnote{This phrase has occasionally elicited inexplicable snickering from undergraduates in the lab.} The gold-standard for evaluating these sorts of issues is to use focus groups. Unfortunately, these methods tend to be expensive and time-consuming. Online surveys were used as a cost-effective, albeit lower-quality, alternative. Data about the clarity and salience of each item were collected from a sample of Amazon Mechanical Turk workers (\textit{N} $=$ 2,649). Each worker was presented with a random subset of all of the items. Participants were asked to rate 18 items for clarity (``For the items on this page, please describe the clarity of the wording for each of the phrases or sentences given below.'') and an additional 18 items for salience (``Please rate the importance of each item on this page relative to your views about personality.''). Note that much more information about these analyses can be found in the supplement for a related paper (Condon, Graham, Weston \& Mroczek, (in prep)) on qualitative analyses of personality items and scales. The following code chunk pulls in the means, standard deviations and sample sizes for the ratings of each item --- we'll use this info along with the readability information during the selection of items for scales (up next). \begin{scriptsize} <>= qualitative <- Qtable696 # Describe the size of the sample for item clarity ratings mean(qualitative$c_n) median(qualitative$c_n) sd(qualitative$c_n) # Describe the size of the sample for item salience ratings mean(qualitative$s_n) median(qualitative$s_n) sd(qualitative$s_n) @ \end{scriptsize} \subsection{Twenty-seven factor scales} This section documents the procedures used for creating scales based on the 27 factors extracted (and replicated) in the 27 factor solutions. I aimed to identify 5 items for each factor based on my experience with CAT-based tools in other domains (namely, PROMIS\texttrademark, the Patient Reported Outcome Measurement Information System). In my experience with those measurement tools, 5 items is usually sufficient to locate respondents who are within two standard deviations of the mean. In fact, most respondents can be located using computer adaptive testing with only 3 or 4 items and there are ``stopping rules'' in place for most measures after 5 items have been administered. But... these CATs also make use of item selection algorithms that choose from larger item pools --- a few of the domain-specific PROMIS item pools contain more than 100 calibrated items.\footnote{This is IRT lingo. In two parameter logistic item-response models, the calibrations are the sample-specific difficulty and discrimination parameters.} It would have been better to derive 10 items for each factor, but that didn't make sense with 27 facets for several reasons. The most important is that it's not possible to identify 10 or more items with high primary loadings for all the factors. Other reasons include the fact that, with 270 items, the resultant scales would retain nearly 40\% of the original 696-item set and would produce a test that is longer than most participants are willing and able to complete in cases where CATs are not possible. An alternate approach would have been to identify a variable number of items for each scale, using as many as needed to demonstrate sufficient internal consistency. I think this approach places too much emphasis on alpha\footnote{After several simulations, I re-learned what I had previously been taught by Bill Revelle. The best way to increase the alpha of a scale is to add more items but this does not necessarily increase the general factor saturation.} and is confusing for many researchers to score. Still, this is a legitimate alternative to be considered for future revisions. Anyway, here's what I did. I identified 5 items for the 27 fixed-length scales. I also chose an additional set of items for each of the 27 factors (5 in most cases but fewer in a few cases where I couldn't get to 5) that were also reasonably good candidates for CAT-based item pools. The creation of a larger/secondary item set and further discussion of CAT development will definitely be addressed in future research but will not be discussed again here. The first step in the identification of items is to create tables of the candidate items for each of the 27 scales. By looping through the code chunk provided below, tables of items are generated for each factor and saved as a separate .csv file. I ran these loops manually but the process could easily be automated. \begin{scriptsize} <>= options(width=110) # Start by making lists of items from the five factor solution. # First, all of the items with primary loadings on each of the five factors b5_qual <- c(replicationEFA5_1$qualItems, replicationEFA5_2$qualItems, replicationEFA5_3$qualItems, replicationEFA5_4$qualItems, replicationEFA5_5$qualItems) # Then the items that PASSed the test of structural replicability across samples for each factor b5_pass <- c(replicationEFA5_1$structuralPass, replicationEFA5_2$structuralPass, replicationEFA5_3$structuralPass, replicationEFA5_4$structuralPass, replicationEFA5_5$structuralPass) # Then those with stable factor loadings across the samples b5_stable <- c(replicationEFA5_1$stableItems, replicationEFA5_2$stableItems, replicationEFA5_3$stableItems, replicationEFA5_4$stableItems, replicationEFA5_5$stableItems) ItemInfo696$b5_stable <- ifelse(rownames(ItemInfo696) %in% b5_stable, 1, 0) ItemInfo696$b5_pass <- ifelse(rownames(ItemInfo696) %in% b5_pass, 1, 0) ItemInfo696$b5_qual <- ifelse(rownames(ItemInfo696) %in% b5_qual, 1, 0) ItemTable <- data.frame(ItemInfo696[,c(1,10:12)], qualitative) # Now here is our example. # Here is the correspondence across samples. This means exploratory factor 1 was replication factor 2, etc. SPI_27exploratoryFactorCorrespondence = c(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27) SPI_27replicationFactorCorrespondence = c(2, 3, 1, 5, 6, 7, 12, 4, 11, 17, 8, 9, 15, 14, 16, 10, 13, 18, 20, 19, 21, 26, 23, 22, 25, 24, 14) # Which factor extraction solution are we looking at? EFAn = 27 # give a generic name to the factor correspondence vector matchrep = SPI_27replicationFactorCorrespondence # Now just run the code below to identify the items for that factor. exp <- eval(parse(text=paste("expSampleCor696autoFA$FAoutput$fa", EFAn, "$fa$loadings", sep=""))) rep <- eval(parse(text=paste("repSampleCor696autoFA$FAoutput$fa", EFAn, "$fa$loadings", sep=""))) for (n in 1:EFAn) { stable <- eval(parse(text=paste("replicationEFA", EFAn, "_", n, "$stableItems", sep=""))) pass <- eval(parse(text=paste("replicationEFA", EFAn, "_", n, "$structuralPass", sep=""))) qual <- eval(parse(text=paste("replicationEFA", EFAn, "_", n, "$qualItems", sep=""))) ldgs <- eval(parse(text=paste("replicationEFA", EFAn, "_", n, "$magldgs", sep=""))) # Sort the items in descending order by the magnitude of the average primary loading sortedLdgs <- sort(ldgs, decreasing=TRUE) factorTable <- ItemTable[names(sortedLdgs),] factorTable$stable <- ifelse(rownames(factorTable) %in% stable, 1, 0) factorTable$pass <- ifelse(rownames(factorTable) %in% pass, 1, 0) factorTable$qual <- ifelse(rownames(factorTable) %in% qual, 1, 0) factorTable$sortedLdgs <- sortedLdgs factorTable$stability <- ifelse(factorTable$stable==1 & factorTable$b5_stable==1, 1, ifelse(factorTable$pass==1 & factorTable$b5_stable==1, 2, ifelse(factorTable$qual==1 & factorTable$b5_stable==1, 3, ifelse(factorTable$stable==1 & factorTable$b5_pass==1, 4, ifelse(factorTable$pass==1 & factorTable$b5_pass==1, 5, ifelse(factorTable$qual==1 & factorTable$b5_pass==1, 6, ifelse(factorTable$stable==1 & factorTable$b5_qual==1, 7, ifelse(factorTable$pass==1 & factorTable$b5_qual==1, 8, ifelse(factorTable$qual==1 & factorTable$b5_qual==1, 9, ifelse(factorTable$stable==1, 10, ifelse(factorTable$pass==1, 11, ifelse(factorTable$qual==1, 12, 13)))))))))))) factorTable$qualitativeRank <- round((scale(factorTable$clarity)+scale(factorTable$salience)-scale(factorTable$readgrade)-scale(factorTable$readage)-scale(factorTable$lexile))[,1], 2) factorTable <- subset(factorTable, select = c(Item, sortedLdgs, stable, pass, qual, b5_stable:b5_qual, stability, clarity, salience, lexile, readage, readgrade, qualitativeRank)) assign(paste("factorTable", n, sep=""), factorTable) write.csv(factorTable, file=paste(filepathdata, "factorTable/factorTable", n, ".csv", sep="")) } @ \end{scriptsize} Note that as a matter of protocol, I opened each of the .csv files generated in the previous chunk of code and annotated them to indicate which items were selected or excluded. I have archived screenshots of each of these files in Appendix \hyperref[sec:appendixB]{B}. (Screenshots are shown because I used some of the convenient formatting tools available in Microsoft Excel but, since this is not freely available software, I chose not to share the Excel files directly). The same three decision criteria were used for each facet. These were: \begin{itemize} \item \textbf{Item loading.} Items were considered in descending order based on the magnitude of the \textit{average} of the loadings on the 27 factor solutions for each sample (the item tables were sorted on this basis in the prior code chunk). I did not use the weighted average loading because doing so would have given more considerably more weight to the structure of the replication sample and I didn't feel this was justified. I also might have used the loadings for the 27 factor solution after combining the data but this would have further ignored the merits of replicability. \item \textbf{Structural stability.} In an attempt to increase replicability for these scales and improve assessment efficiency, I developed a stability index that ranked each item from 1 to 13. The rankings for each item are labeled `stability' in the tables for each factor. These rankings were formulaically derived based on the replicability and similarity of loading magnitudes across the 27 factor solutions and the 5 factor solutions. These features are indicated for each item by the columns labeled `stable,' `pass,' `qual,' `b5\_stable,' `b5\_pass, and `b5\_qual.' Items are coded as `qual' if the item is `qualified' in at least one sample --- recall that this means that it has a primary loading with an absolute value above 0.3 and no secondary loadings with absolute values above 0.3. Note that some items are qualified in one sample but not both. Items are coded as `pass' if the item is qualified on the same factor in both samples. Items are coded as `stable' if coded as `pass' and the difference in loadings is less than 0.1. Note that this is twice as stringent as the recommended values for exploratory factor analysis replication \citep[the recommended difference threshold is 0.2, ][]{Osborne:2012te}. The stable items are a subset of the pass items which are a subset of the qual items. Codings for `b5\_stable' and `b5\_pass' represent those items with stable loadings (difference of less than 0.1 across samples) or less stable but still structurally equivalent loadings across samples for the 5 factor solutions. `b5\_qual' is less strict than `b5\_pass' because it indicates that the item was qualified at the level of 5 factors of extraction but it was not structurally invariant. These latter three criteria are relevant because I'm attempting to identify items at the facet level (27) which may also be useful at the factor level (5). While it is not particularly critical that we use items for the 27 factor scales that are also representative of the 5 factor structure, it is more efficient because it allows for the derivation of scores on multiple levels with a single assessment. The codings for these variables are used to generate the stability ranking for each item as follows: \begin{enumerate} \item stable and b5\_stable \vspace{-2mm} \item pass and b5\_stable \vspace{-2mm} \item qual and b5\_stable \vspace{-2mm} \item stable and b5\_pass \vspace{-2mm} \item pass and b5\_pass \vspace{-2mm} \item qual and b5\_pass \vspace{-2mm} \item stable and b5\_qual \vspace{-2mm} \item pass and b5\_qual \vspace{-2mm} \item qual and b5\_qual \vspace{-2mm} \item stable \vspace{-2mm} \item pass \vspace{-2mm} \item qual \vspace{-2mm} \item items that are not qualified in either sample at the 27 factor level \end{enumerate} \item \textbf{Qualitative rank.} I also developed an index based on the qualitative data collected on each item. This index used equal weightings for five characteristics of the items. The `clarity' and `salience' values were based on the ratings described in Section \ref{sec:clarsal}. Clarity ratings reflect the perceived clarity of the wording for each item (from 1 to 4) and salience ratings reflect the perceived importance of each item to the raters views about personality. The `lexile,' `readage,' `readgrade' values were based on those described in Section \ref{sec:readability}. The Lexile values are taken exactly as described. The `readage' and `readgrade' values are the averages of the age and grade measures generated by the koRpus package. These five values are all scaled. The `qualitativeRank' value in the table for each of the 27 factors is the sum of the scaled clarity and salience values minus the scaled values for lexile, readage, and readlevel. \end{itemize} In addition to these criteria, I also considered whether the item content was highly redundant with a previously selected item or included colloquial language. I tried to avoid the use of colloquialisms as these tend to cause problems when translating items. Note that I did \textit{not} consider the direction of item content when choosing items. There is some debate about this practice; I discuss it in more detail at the end of this section (see sub-section \ref{sec:acquiescence}). The code below shows the items that were chosen for each of the 27 facet level scales \textit{by item number only}. The item content is shown in Table \ref{tab:orgtable} in Section \ref{sec:naming}. \begin{scriptsize} <>= options(width=110) #SPI_27 # `prelim' labels are mine, not the final crowd-sourced names. # Factor 1 items - prelim label: compassion SPI_27_1 <- c("q_90", "q_253", "q_1763", "q_851", "q_1832") # other acceptable items: q_844, q_298, q_460, q_98, q_1962 x1 <- factorTable1[SPI_27_1,] y1 <- psych::describe(x1[,c(10:14)], fast=TRUE) # Factor 2 items - prelim label: irritability SPI_27_2 <- c("q_952", "q_1357", "q_1585", "q_1683", "q_176") # other acceptable items: q_926, q_1591, q_4300, q_965, q_4235 x2 <- factorTable2[SPI_27_2,] y2 <- psych::describe(x2[,c(10:14)], fast=TRUE) # Factor 3 items - prelim label: sociability SPI_27_3 <- c("q_1904", "q_312", "q_684", "q_4243", "q_1923") # other acceptable items: q_1899, q_1531, q_4247, q_4267, q_314 x3 <- factorTable3[SPI_27_3,] y3 <- psych::describe(x3[,c(10:14)], fast=TRUE) # Factor 4 items - prelim label: well-being SPI_27_4 <- c("q_578", "q_811", "q_1371", "q_2765", "q_820") # other acceptable items: q_285, q_1044, q_1038, q_1578, q_832 x4 <- factorTable4[SPI_27_4,] y4 <- psych::describe(x4[,c(10:14)], fast=TRUE) # Factor 5 items - prelim label: sensation-seeking SPI_27_5 <- c("q_1367", "q_1664", "q_1662", "q_1781", "q_598") # other acceptable items: q_1306, q_292, q_1780, q_2011, q_804 x5 <- factorTable5[SPI_27_5,] y5 <- psych::describe(x5[,c(10:14)], fast=TRUE) # Factor 6 items - prelim label: worry SPI_27_6 <- c("q_4249", "q_1989", "q_4252", "q_1505", "q_808") # other acceptable items: q_4256, q_1596, q_986, q_85, q_332 x6 <- factorTable6[SPI_27_6,] y6 <- psych::describe(x6[,c(10:14)], fast=TRUE) # Factor 7 items - prelim label: honesty SPI_27_7 <- c("q_4296", "q_1812", "q_2853", "q_1896", "q_501") # other acceptable items: q_1765, q_1808, q_2021, q_393, q_2003 x7 <- factorTable7[SPI_27_7,] y7 <- psych::describe(x7[,c(10:14)], fast=TRUE) # Factor 8 items - prelim label: industry SPI_27_8 <- c("q_904", "q_1744", "q_1979", "q_1452", "q_1444") # other acceptable items: q_1087, q_992, q_985, q_637, q_602 x8 <- factorTable8[SPI_27_8,] y8 <- psych::describe(x8[,c(10:14)], fast=TRUE) # Factor 9 items - prelim label: intellect SPI_27_9 <- c("q_1253", "q_240", "q_493", "q_422", "q_1834") # other acceptable items: q_722, q_1235, q_423, q_132, q_157 x9 <- factorTable9[SPI_27_9,] y9 <- psych::describe(x9[,c(10:14)], fast=TRUE) # Factor 10 items - prelim label: imaginative SPI_27_10 <- c("q_128","q_2745", "q_2754", "q_1392", "q_1058") # other acceptable items: q_516, q_1090, q_609, q_1083, q_1232 x10 <- factorTable10[SPI_27_10,] y10 <- psych::describe(x10[,c(10:14)], fast=TRUE) # Factor 11 items - prelim label: impulsivity SPI_27_11 <- c("q_35", "q_1424", "q_1173", "q_4223","q_4276") # other acceptable items: q_636, q_1638, q_4286, q_708, q_22 x11 <- factorTable11[SPI_27_11,] y11 <- psych::describe(x11[,c(10:14)], fast=TRUE) # Factor 12 items - prelim label: attention-seeking SPI_27_12 <- c("q_1416", "q_1296", "q_565", "q_1555", "q_1027") # other acceptable items: q_1667, q_1871, q_690, q_698, q_154 x12 <- factorTable12[SPI_27_12,] y12 <- psych::describe(x12[,c(10:14)], fast=TRUE) # Factor 13 items - prelim label: orderliness SPI_27_13 <- c("q_1201", "q_1254", "q_1483", "q_169", "q_1290") # other acceptable items: q_1333, q_1255, q_170, q_4254, q_1374 x13 <- factorTable13[SPI_27_13,] y13 <- psych::describe(x13[,c(10:14)], fast=TRUE) # Factor 14 items - prelim label: conventionalism SPI_27_14 <- c("q_369", "q_398", "q_1624", "q_1867", "q_1609") # other acceptable items: q_4236, q_1752, q_4244, q_1657, q_623 x14 <- factorTable14[SPI_27_14,] y14 <- psych::describe(x14[,c(10:14)], fast=TRUE) # Factor 15 items - prelim label: charisma # This is a weak factor. # The first 3 items listed below are great: q_254, q_1045, and q_131. # q_901 loads -.45 in the exp sample and -.29 in the rep sample but has cross-loadings with sociability of -.30 in the rep sample (only .14 in the exp sample). # q_1242 is retained for the same reasons: loading of -.38 in the rep sample but only -.25 in the exp sample SPI_27_15 <- c("q_254", "q_901", "q_1045", "q_131", "q_1242") # other marginally acceptable items: q_158, q_3090, q_1205, q_1110, and q_241 x15 <- factorTable15[SPI_27_15,] y15 <- psych::describe(x15[,c(10:14)], fast=TRUE) # Factor 16 items - prelim label: trust SPI_27_16 <- c("q_1855", "q_4289", "q_377", "q_871", "q_379") # other acceptable items: q_1854, q_594, q_1758, q_343, q_286 x16 <- factorTable16[SPI_27_16,] y16 <- psych::describe(x16[,c(10:14)], fast=TRUE) # Factor 17 items - prelim label: humor SPI_27_17 <- c("q_1243", "q_1244", "q_1248", "q_296", "q_1685") # other acceptable items: q_4265, q_1064, q_1709, q_4295, q_799 x17 <- factorTable17[SPI_27_17,] y17 <- psych::describe(x17[,c(10:14)], fast=TRUE) # Factor 18 items - prelim label: expressiveness SPI_27_18 <- c("q_219", "q_1081", "q_1706", "q_1635", "q_803") # other acceptable items: q_403, q_1810, q_55, q_1668, q_140 x18 <- factorTable18[SPI_27_18,] y18 <- psych::describe(x18[,c(10:14)], fast=TRUE) # Factor 19 items - prelim label: culture SPI_27_19 <- c("q_607", "q_348", "q_610", "q_1132", "q_612") # other acceptable items: q_1369, q_776, q_1388, q_608, q_964 x19 <- factorTable19[SPI_27_19,] y19 <- psych::describe(x19[,c(10:14)], fast=TRUE) # Factor 20 items - prelim label: reflection SPI_27_20 <- c("q_1389", "q_1738", "q_1880", "q_755", "q_1310") # other acceptable items: q_778, q_1587, q_322, q_197, q_4232 x20 <- factorTable20[SPI_27_20,] y20 <- psych::describe(x20[,c(10:14)], fast=TRUE) # Factor 21 items - prelim label: perfectionism # This is a problematic factor. In the exploratory sample, this factor has considerable overlap with the 13th factor (order); the overlap is far less in the replication sample (in fact, this is a very strong factor in the replication sample). SPI_27_21 <- c("q_571", "q_1915", "q_1694", "q_530", "q_142") # other acceptable items: maybe q_4278, q_554, q_71, q_1984, q_604 x21 <- factorTable21[SPI_27_21,] y21 <- psych::describe(x21[,c(10:14)], fast=TRUE) # Factor 22 items - prelim label: selfdiscipline # A weak factor in both samples (few items with strong loadings); stability across samples is okay. SPI_27_22 <- c("q_1462", "q_1590", "q_1461", "q_56", "q_736") # other acceptable items: maybe q_1120, q_1474 x22 <- factorTable22[SPI_27_22,] y22 <- psych::describe(x22[,c(10:14)], fast=TRUE) # Factor 23 items - prelim label: modesty # This factor is problematic bc it was much stronger in the replication sample than the exploratory sample. SPI_27_23 <- c("q_1300", "q_2005", "q_747", "q_152", "q_1653") # other somewhat acceptable items: q_1691, q_525, q_1151, q_1913, q_319 x23 <- factorTable23[SPI_27_23,] y23 <- psych::describe(x23[,c(10:14)], fast=TRUE) # Factor 24 items - prelim label: adjustability SPI_27_24 <- c("q_566", "q_689", "q_1542", "q_39", "q_1303") # other acceptable items: q_38, q_582, q_47, q_77, q_1540 x24 <- factorTable24[SPI_27_24,] y24 <- psych::describe(x24[,c(10:14)], fast=TRUE) # Factor 25 items - prelim label: inactivity # This factor is has few items in both samples SPI_27_25 <- c("q_1329", "q_1281", "q_1052", "q_1024", "q_1280") # other acceptable items: q_1372 x25 <- factorTable25[SPI_27_25,] y25 <- psych::describe(x25[,c(10:14)], fast=TRUE) # Factor 26 items - prelim label: emotionlability # This factor is problematic, predictably related to Factor 2 (irritability). They are correlated and there may be an argument for collapsing the two, though it will depend on the evidence for discriminant validity. SPI_27_26 <- c("q_1840", "q_797", "q_979", "q_793", "q_174") # other acceptable items: q_1099, q_4224, q_497, q_52 x26 <- factorTable26[SPI_27_26,] y26 <- psych::describe(x26[,c(10:14)], fast=TRUE) # Factor 27 items - prelim label: conservatism # This factor replicates in both samples. That said, the administration of these items is likely to be problematic in many settings, particularly those which are governed by legal oversight (employment and possibly healthcare settings). These items explicitly ask about political views and religiosity. SPI_27_27 <- c("q_1824", "q_660", "q_345", "q_1825","q_1328") # other acceptable items: q_4260, q_394, q_398, q_1301, q_1223 x27 <- factorTable27[SPI_27_27,] y27 <- psych::describe(x27[,c(10:14)], fast=TRUE) SPI_27_Items <- c(SPI_27_1, SPI_27_2, SPI_27_3, SPI_27_4, SPI_27_5, SPI_27_6, SPI_27_7, SPI_27_8, SPI_27_9, SPI_27_10, SPI_27_11, SPI_27_12, SPI_27_13, SPI_27_14, SPI_27_15, SPI_27_16, SPI_27_17, SPI_27_18, SPI_27_19, SPI_27_20, SPI_27_21, SPI_27_22, SPI_27_23, SPI_27_24, SPI_27_25, SPI_27_26, SPI_27_27) @ \end{scriptsize} For the majority of factors (18 of 27), all five of the items selected had average loadings with an absolute value greater than 0.45. Of the remaining 9 factors, I would describe 3 as moderate in strength, 3 as weak, and 3 as problematic (see Section \ref{sec:overfactoring} for commentary regarding the impetus to retain weak factors). Factor 21 had only 2 items that were qualified (primary loadings above absolute value of 0.3; no secondary loadings above absolute value of 0.3), structurally invariant on the same factor and stable. Factors 23 and 26 had only one of these items each. Still, for all three of these factors, there were 5 or more items that were qualified in one or both samples and at least 3 items with average factor loadings with an absolute value above 0.3. Let's review the items that are retained for the scales identified and describe their characteristics as a set. \begin{scriptsize} <>= # How many items are there total? length(SPI_27_Items) # How do they compare with the 5 factor item lists? # How many are qualified items in the 5 factor solutions? length(intersect(SPI_27_Items, b5_qual)) # How many are structurally invariant in the 5 factor solutions? length(intersect(SPI_27_Items, b5_pass)) # How many are structurally invariant with stable loadings? length(intersect(SPI_27_Items, b5_stable)) #Make a table with all the qualitative data for the full set and for the individual scales. allItems <- ItemTable[SPI_27_Items,c(6,9,12,15,16)] descAllItems <- psych::describe(allItems, fast=TRUE) #rbind all the y tables, give them rownames, and show them. descAll <- rbind(t(descAllItems$mean), t(y1$mean), t(y2$mean), t(y3$mean), t(y4$mean), t(y5$mean), t(y6$mean), t(y7$mean), t(y8$mean), t(y9$mean), t(y10$mean), t(y11$mean), t(y12$mean), t(y13$mean), t(y14$mean), t(y15$mean), t(y16$mean), t(y17$mean), t(y18$mean), t(y19$mean), t(y20$mean), t(y21$mean), t(y22$mean), t(y23$mean), t(y24$mean), t(y25$mean), t(y26$mean), t(y27$mean)) rownames(descAll) <- c("All_Items", "SPI_27_1", "SPI_27_2", "SPI_27_3", "SPI_27_4", "SPI_27_5", "SPI_27_6", "SPI_27_7", "SPI_27_8", "SPI_27_9", "SPI_27_10", "SPI_27_11", "SPI_27_12", "SPI_27_13", "SPI_27_14", "SPI_27_15", "SPI_27_16", "SPI_27_17", "SPI_27_18", "SPI_27_19", "SPI_27_20", "SPI_27_21", "SPI_27_22", "SPI_27_23", "SPI_27_24", "SPI_27_25", "SPI_27_26", "SPI_27_27") colnames(descAll) <- c("clarity", "salience", "lexile", "readage", "readgrade") round(descAll, 2) @ \end{scriptsize} As noted above, all of information used to select items is provided in Appendix \hyperref[sec:appendixB]{B} (see Figures \ref{fig:ft27_1} through \ref{fig:ft27_27}). I hope this detail will encourage others to (tactfully) identify shortcomings and suggest revisions. \subsection{Five factor scales} To be clear, the procedure described above could be used to identify scales at ANY level of extraction. The decision to repeat the procedure for the five factor solution was warranted because: (1) it provides a pretty impressive solution, as shown in Section \ref{sec:fit}; (2) the five factor solution is very frequently used by other researchers and extensively referenced in the literature; and (3) I believe that assessment at the level of five factors (by others in the field) will benefit from using empirically-derived blends of existing measures, especially if these 5 factor scales fit hierarchically with the (27) facet level scales. To reiterate a critically important methodological point made at the beginning of this section, I gave priority during the selection of items for the 27 factor solution to those items that were qualified/invariant/stable on the 5 factor solution. The reason for this was because I planned to identify items that work at both levels of the hierarchy. Here, I chose items for the 5 factor scales from those in the 27 factor set. More than 90\% of these items have primary loadings on their respective factor in the 5 factor solution and all of them have stable loadings across both samples. But, I certainly would have chosen different items if I was only concerned with high internal consistency at the 5 factor level without regard for hierarchy. Aside from this distinction, items were chosen for the 5 factor scales using the same general criteria as those used for the 27 factor scales. The ``stability'' index had fewer values because I was not concerned with the nuances of each items loadings at the 27 factor level. Items were ranked 1 if they had stable 5 factor loadings on the same factor and were included among the 27 factor scales. Items were ranked 2 if they had stable 5 factor loadings on the same factor but were \textbf{not} included among the 27 factor scales. A ranking of 3 indicated structural invariance but not stable loadings with an absolute value of 0.1 in both samples. A 4 indicated that the item was only qualified on both items and a 5 indicated that it was not a replicable 5 factor item. An important observation needs to be made before presenting the code used to select the candidate items for the five factor scales: the fourth factor in this solution is essentially Agreeableness,\footnote{In truth, I think this is a terrible name. The items do not specifically relate to the tendency to ``agree,'' nor is it true that the content in Agreeableness scales developed by others reflect this. A more appropriate name would be Compassion and that is the label I have used for the facet scale that is most highly correlated with the 5 factor scale of Agreeableness. I'm not going to change the name for the 5 factor solution but I would certainly support a movement away from this misleading label.} but it also includes many items that I believe to be strongly gendered. This includes items about compassion and empathy but also items about danger-seeking and Machiavellianism. In the six factor solution, the danger and Machiavellianism items appear to split off from the compassion and empathy items. This suggests to me that there may be an effect of gender on the factor structure at the 5/6 factor level. It is a top priority to explore this question further in future research.\footnote{It's reasonable to wonder why I haven't explored this issue before deriving the scales and/or whether I should somehow control for characteristics like gender in the exploratory factor analyses. My answer is that this only serves to emphasize the importance of using representative samples. I think it's a little heavy handed for me to say what constitutes representativeness, much less what constitutes ``gender.'' Plus, a bit of poking around in the data has suggested that the factor structure may be more-than-a-little different across other variables --- age \& race/ethnicity. So, I have resisted the temptation to pull these strings, for now.} The key point for the current work is that I have decided to use only the top 14 items (from those in the 27 factor scales) for each of the 5 factor scales. This avoids the inclusion of 2 items relating to ``danger.'' Even with 14 items, the Agreeableness scale includes 5 items that have lower primary loadings than the other factors (5 items have primary loadings below 0.4). While I might have speculated that Agreeableness would be a problematic factor given the inclusion of 6 factor scales in this sample of items,\footnote{I didn't.} I would not have predicted that evidence of underlying structure in this factor would include danger-seeking and Machiavellianism. The next section of code shows the procedures for identifying items at the 5 factor level. Figures \ref{fig:ft5_1} through \ref{fig:ft5_5} in Appendix \hyperref[sec:appendixB]{B} provide specific details. \begin{scriptsize} <>= options(width=110) ItemTable$SPI27 <- ifelse(rownames(ItemTable) %in% rownames(allItems), 1, 0) SPI_5exploratoryFactorCorrespondence = c(1, 2, 3, 4, 5) SPI_5replicationFactorCorrespondence = c(1, 2, 4, 3, 5) EFAn = 5 matchrep = SPI_5replicationFactorCorrespondence exp <- eval(parse(text=paste("expSampleCor696autoFA$FAoutput$fa", EFAn, "$fa$loadings", sep=""))) rep <- eval(parse(text=paste("repSampleCor696autoFA$FAoutput$fa", EFAn, "$fa$loadings", sep=""))) for (n in 1:EFAn) { stable <- eval(parse(text=paste("replicationEFA", EFAn, "_", n, "$stableItems", sep=""))) pass <- eval(parse(text=paste("replicationEFA", EFAn, "_", n, "$structuralPass", sep=""))) qual <- eval(parse(text=paste("replicationEFA", EFAn, "_", n, "$qualItems", sep=""))) ldgs <- eval(parse(text=paste("replicationEFA", EFAn, "_", n, "$magldgs", sep=""))) # Sort the items in descending order by the magnitude of the average primary loading sortedLdgs <- sort(ldgs, decreasing=TRUE) factorTable <- ItemTable[names(sortedLdgs),] factorTable$sortedLdgs <- sortedLdgs factorTable$stable <- ifelse(rownames(factorTable) %in% stable, 1, 0) factorTable$pass <- ifelse(rownames(factorTable) %in% pass, 1, 0) factorTable$qual <- ifelse(rownames(factorTable) %in% qual, 1, 0) factorTable$stability <- ifelse(factorTable$stable==1 & factorTable$pass==1 & factorTable$qual==1 & factorTable$SPI27==1, 1, ifelse(factorTable$stable==1 & factorTable$pass==1 & factorTable$qual==1, 2, ifelse(factorTable$pass==1 & factorTable$qual==1, 3, ifelse(factorTable$qual==1, 4, 5)))) factorTable$qualitativeRank <- round((scale(factorTable$clarity)+scale(factorTable$salience)-scale(factorTable$readgrade)-scale(factorTable$readage)-scale(factorTable$lexile))[,1], 2) factorTable <- subset(factorTable, select = c(Item, sortedLdgs, stable, pass, qual, SPI27, stability, clarity, salience, lexile, readage, readgrade, qualitativeRank)) assign(paste("factors5Table", n, sep=""), factorTable) write.csv(factorTable, file=paste(filepathdata, "factorTable/factors5Table", n, ".csv", sep="")) } # Factor 1 items - prelim label: Extraversion SPI_5_1 <- c("q_1904", "q_4243", "q_312", "q_565", "q_1416", "q_1923", "q_1027", "q_684", "q_254", "q_1296", "q_901", "q_1243", "q_803", "q_1244") # all spi27 items invariant on this factor: # q_1904, q_4243, q_312, q_565, q_1416, q_1923, q_1027, q_684, q_254, q_1296, q_901, q_1243, q_803, q_1244, q_1635, q_219, q_296, q_1781, q_1248, q_598, q_1662, q_1045, q_1371, q_1052, q_1081, q_1242, q_1555 # Note: all of the above were also stable # qualified (but not inv.) in Exp or Rep sample: q_2765, q_39 x1_5 <- data.frame(factors5Table1[SPI_5_1,c(1,2,7:12)],ItemInfo696[SPI_5_1,c(2:4,6,7)]) y1_5 <- psych::describe(x1_5[,c(3:7)], fast=TRUE) # Factor 2 items - prelim label: Neuroticism SPI_5_2 <- c("q_979", "q_4252", "q_1989", "q_1505", "q_4249", "q_808", "q_793", "q_1840", "q_811", "q_1585", "q_578", "q_176", "q_797", "q_1683") # all spi27 items invariant on this factor: # q_979, q_4252, q_1989, q_1505, q_4249, q_808, q_793, q_1840, q_811, q_1585, q_578, q_176, q_797, q_1683, q_820, q_1706, q_566, q_1357 # Note: q_1357 was not stable # qualified (but not inv.) in Exp or Rep sample: q_174, q_689, q_56 x2_5 <- data.frame(factors5Table2[SPI_5_2,c(1,2,7:12)],ItemInfo696[SPI_5_2,c(2:4,6,7)]) y2_5 <- psych::describe(x2_5[,c(3:7)], fast=TRUE) # Factor 3 items - prelim label: Conscientiousness SPI_5_3 <- c("q_1290", "q_1744", "q_1979", "q_1452", "q_1915", "q_1201", "q_530", "q_904", "q_1867", "q_1694", "q_369", "q_1444", "q_1483", "q_1254") # all spi27 items invariant on this factor: # q_1290, q_1744, q_1979, q_1452, q_1915, q_1201, q_530, q_904, q_1867, q_1694, q_369, q_1444, q_1483, q_1254, q_571, q_398, q_169, q_736, q_1024 # Note: all of the above were also stable # qualified (but not inv.) in Exp or Rep sample: q_4223, q_1424, q_1328 x3_5 <- data.frame(factors5Table3[SPI_5_3,c(1,2,7:12)],ItemInfo696[SPI_5_3,c(2:4,6,7)]) y3_5 <- psych::describe(x3_5[,c(3:7)], fast=TRUE) # Factor 4 items - prelim label: Agreeableness SPI_5_4 <- c("q_90", "q_1763", "q_253", "q_1896", "q_851", "q_1832", "q_501", "q_377", "q_871", "q_1855", "q_4296", "q_142", "q_379", "q_4289") # all spi27 items invariant on this factor: # q_90, q_1763, q_253, q_1896, q_851, q_1832, q_501, q_377, q_871, q_1855, q_4296, q_142, q_379, q_4289, q_1367, q_1664 # Note: all of the above were also stable # qualified (but not inv.) in Exp or Rep sample: q_2853 x4_5 <- data.frame(factors5Table4[SPI_5_4,c(1,2,7:12)],ItemInfo696[SPI_5_4,c(2:4,6,7)]) y4_5 <- psych::describe(x4_5[,c(3:7)], fast=TRUE) # Factor 5 items - prelim label: Openness SPI_5_5 <- c("q_128", "q_2745", "q_2754", "q_1392", "q_1058", "q_240", "q_1738", "q_422", "q_1389", "q_1310", "q_1880", "q_747", "q_1609", "q_1834") # all spi27 items invariant on this factor: # q_128, q_2745, q_2754, q_1392, q_1058, q_240, q_1738, q_422, q_1389, q_1310, q_1880, q_747, q_1609, q_1834, q_1132, q_493, q_1303, q_610, q_755, q_1300, q_1653, q_2005, q_607, q_612, q_152, q_1253, q_1542 # Note: all of the above were also stable # qualified (but not inv.) in Exp or Rep sample: q_348, q_1825 x5_5 <- data.frame(factors5Table5[SPI_5_5,c(1,2,7:12)],ItemInfo696[SPI_5_5,c(2:4,6,7)]) y5_5 <- psych::describe(x5_5[,c(3:7)], fast=TRUE) SPI_5_Items <- c(SPI_5_1, SPI_5_2, SPI_5_3, SPI_5_4, SPI_5_5) # How many SPI items in total? length(SPI_5_Items) # Confirm they're all unique... length(unique(SPI_5_Items)) # What are the item labels? unique(SPI_5_Items) # How many of them overlap with 27 factor items? length(intersect(SPI_5_Items, SPI_27_Items)) @ \end{scriptsize} \subsection{Scoring keys and item lists} After identifying all of the items in the scales, it's time to clean up a bit. This includes making scoring keys and item lists for later use. In the following code chunk, I have written a helper function to streamline this process and make it less subject to human error (mine, specifically). It does this by using the factor output for the items in each scale to get the correct scoring direction --- the alternative is to do this by hand, but... don't say I didn't warn you. \begin{scriptsize} <>= SPI_5_Scales <- c("SPI_5_1", "SPI_5_2", "SPI_5_3", "SPI_5_4", "SPI_5_5") SPI_27_Scales <- c("SPI_27_1", "SPI_27_2", "SPI_27_3", "SPI_27_4", "SPI_27_5", "SPI_27_6", "SPI_27_7", "SPI_27_8", "SPI_27_9", "SPI_27_10", "SPI_27_11", "SPI_27_12", "SPI_27_13", "SPI_27_14", "SPI_27_15", "SPI_27_16", "SPI_27_17", "SPI_27_18", "SPI_27_19", "SPI_27_20", "SPI_27_21", "SPI_27_22", "SPI_27_23", "SPI_27_24", "SPI_27_25", "SPI_27_26", "SPI_27_27") autoKeying <- function (EFAn = EFAn) { EFAn = EFAn keying.list <- list(list()) item.list <- list(list()) n <- 0 for (i in 1:EFAn) { n <- n+1 expLdgs <- eval(parse(text=paste("expSampleCor696autoFA$FAoutput$fa", EFAn, "$fa$loadings", sep=""))) repLdgs <- eval(parse(text=paste("repSampleCor696autoFA$FAoutput$fa", EFAn, "$fa$loadings", sep=""))) if (EFAn == 5) { rows <- eval(parse(text=SPI_5_Scales[i])) colExp <- SPI_5exploratoryFactorCorrespondence[i] colRep <- SPI_5replicationFactorCorrespondence[i] named <- SPI_5_Scales } if (EFAn == 27) { rows <- eval(parse(text=SPI_27_Scales[i])) colExp <- SPI_27exploratoryFactorCorrespondence[i] colRep <- SPI_27replicationFactorCorrespondence[i] named <- SPI_27_Scales } expVals <- expLdgs[rows, colExp] repVals <- repLdgs[rows, colRep] expVals[expVals < 0] <- -1 expVals[expVals > 0] <- 1 repVals[repVals < 0] <- -1 repVals[repVals > 0] <- 1 ifelse(sum(repVals-expVals) == 0, x <- expVals, ifelse(sum(repVals+expVals) == 0, x <- expVals, x <- c("NA"))) z <- list() for (j in 1:length(x)) { z[[j]] <- paste("", names(x[j]), sep="") } z <- unlist(z) item.list[[i]] <- z y <- list() for (j in 1:length(x)) { ifelse(x[j] == -1, y[[j]] <- paste("-", names(x[j]), sep=""), y[[j]] <- paste("", names(x[j]), sep="")) } y <- unlist(y) keying.list[[i]] <- y rm(colExp, colRep, expLdgs, expVals, i, j, repLdgs, repVals, rows) } names(keying.list) <- named names(item.list) <- named rm(n) results <- list(keys = keying.list, items = item.list) return(results) } keys.list27 <- autoKeying(EFAn = 27) keys.list5 <- autoKeying(EFAn = 5) # Some manual changes to the direction of scoring are needed. # The following 7 lines of code were added post-hoc, # after the crowd-sourced names were derived. # All we are doing here is flipping the direction of the items. keys.list27$keys$SPI_27_3 <- c("q_1904", "-q_312", "-q_684", "q_4243", "-q_1923") keys.list27$keys$SPI_27_4 <- c("-q_578", "-q_811", "q_1371", "q_2765", "q_820") keys.list27$keys$SPI_27_7 <- c("-q_4296", "q_1812", "q_2853", "-q_1896", "-q_501") keys.list27$keys$SPI_27_8 <- c("-q_904", "q_1744", "q_1979", "-q_1452", "-q_1444") keys.list27$keys$SPI_27_18 <- c("q_219", "-q_1081", "q_1706", "-q_1635", "q_803") keys.list27$keys$SPI_27_19 <- c("-q_607", "q_348", "-q_610", "q_1132", "-q_612") keys.list5$keys$SPI_5_4 <- c("q_90", "q_1763", "q_253", "-q_1896", "q_851", "q_1832", "-q_501", "q_377", "-q_871", "q_1855", "-q_4296", "-q_142", "q_379", "q_4289") @ \end{scriptsize} \subsection{Comments on `the challenge of response sets'} \label{sec:acquiescence} People differ. That's the fundamental reason for assessing personality in the first place. Unfortunately, these differences include the ways in which people respond to surveys and this inconvenient truth introduces several potential confounds in personality measurement by survey. There's not much to say about these confounds that wasn't covered by Block \citeyearpar{Block:65} and Rorer \citeyearpar{Rorer:65} more than 50 years ago or more recently by \citet{Johnson:2014}. Careful survey construction can help with some of these issues (i.e., readability, clarity, and colloquialisms) but it also seems worth commenting briefly on the topics of social desirability bias and acquiescence. Social desirability is particularly vexing. I have not encountered any good solutions nor do any seem likely. It seems to me that impression management is an integral part of the human experience (and a fascinating source of individual differences). It is critical to remember that survey responses do not necessarily reflect actual behavior and the extent to which responses align with behavior varies across persons as well as items/scales. It is beyond the scope of the current work to estimate the extent of this variability among the 27 facet scales (this will require substantial data from observer reports of personality and other criterion variables), but it seems worth noting that a few of the facets seem particularly vulnerable to socially desirable responding. Factors 7 (Honesty) and 1 (Compassion) are the best examples. In general, I believe that responses to the 696 items administered to these samples should be \textit{relatively} less biased by social desirability than most administrations on the grounds that the primary motivation --- typically, the \textit{only} motivation --- for responding to these items was to receive algorithmically-generated feedback about their personality. In other words, participants should have been motivated to respond accurately and without expectation that their responses would be evaluated by anyone else. Still, a substantial proportion of socially-desirable responding probably occurs unconsciously and there is no reason to believe these samples are less affected by these unconscious biases than participants in other samples. By comparison, the issue of acquiescence is much less concerning, though still worth mentioning because it has received some attention recently with respect to the BFI \citep{Soto:08, Soto:2016jr}. First of all, it should be obvious that the influence of acquiescence, if any, should not affect the comparison of EFA solutions within a sample because the solutions are based on the same underlying correlational structure. While it might affect the \textit{replicability} of a given solution if the samples differ in terms of the tendency to acquiesce, the magnitude of the effect in the samples described here is likely to be small given that the items were administered in random order, without regard to positively- and negatively-oriented content (this is supported by evaluation of the effects of ipsatization in Section \ref{sec:ipsatize}). The more relevant question is whether I should have considered the threat of acquiescence when identifying items to be included in the scales. This would have meant choosing items such that the number of positively- and negatively-keyed items was balanced. With 5 items in each scale, the best I could have done would be to choose 2 negatively-keyed items. The odd number of items would make it impossible to have perfect balance so all the scales would still be somewhat biased in one direction or the other. There are several reasons why I did not do this: 1. \textbf{Emphasis on internal consistency.} In general, I used items with the greatest magnitude of mean loadings. I can only say ``in general'' because I made some efforts to avoid the inclusion of enemy items (those that are highly-redundant) and items that were unclear or likely to pose translation issues. But, I mainly aimed to develop a broad personality assessment tool built on many narrow, unidimensional scales. Unidimensionality would have been reduced for several of the facet-level scales if I had made an intentional effort to choose items that were keyed in both directions. This is partly due to the fact that I was constrained by these 696 items and partly due to the fact that the introduction of reverse-keyed items sometimes changes the very nature of a dimension. 2. \textbf{The questionable logic of ``balanced'' scales.} The claim that balanced scales are free of acquiescence strikes me as specious because it presumes that items are formulaic and people are rational. I don't believe either is true. Let's consider an example in which we determine the location of an individual on the Conscientiousness dimension. So we'll administer 12 items to that individual --- 8 of them will be positively keyed (higher agreement/scores indicates higher Conscientiousness) and 4 of them will be negatively keyed. Now we can construct two overlapping 8-item scales for Conscientiousness. One will have 8 positively-keyed items and the second will have 4 positively- and 4 negatively-keyed items. What will we learn if the scores are different? In theory, we could claim evidence for acquiescence --- this individual is either a nay-sayer or a yea-sayer when it comes to Conscientiousness items(depending on the direction of the difference in scores). But there are many other reasons why the scores might differ including the fact that positively and negatively keyed items are rarely perfect opposites in terms of meaning or difficulty (consider ``I keep a messy room'' vs ``I keep a clean room'' vs ``I do not keep a messy room''). Even when people are given items that are highly similar --- in terms of syntax and/or highly correlated response patterns in large samples --- many individuals will not respond identically to both items (or identically opposite, if they are keyed in opposite directions). People are not perfectly rational. Sometimes we are uncertain and/or \href{https://youtu.be/S9WMncHTtEI}{behave inexplicably} and/or fail to pay attention and/or interpret reverse-coded items to have much different meanings and it is impossible to accurately distinguish these issues from acquiescence. It seems to me that the use of balanced scales to ``prevent'' acquiescence presumes that these things don't happen, that the true score in this example must be the balanced 8-item scale. It's also not clear to me (or Block and others) what conclusions we could reach if we extended this example to include assessment across multiple domains. If the individual acquiesces in different directions across domains or even in the same direction to different extents, then I would be concerned that the so-called acquiescence is, to some extent, domain-specific. And that (domain-specific variation) is what we're trying to measure in the first place! To be clear, I am not arguing against the existence of acquiescence as a phenomenon; differences between ipsatized and non-ipsatized scores suggest that response styles do exist. However, it is my view is that more compelling evidence is needed to demonstrate the measurement effects of acquiescence on validity \citep{Johnson:2014} and, more importantly, the ways that balanced scales remove bias. 3. \textbf{The merits of ipsative scoring procedures.} In cases where high levels of acquiescence are expected \citep{Soto:08}, the identification of differences in response style across sub-samples may well be informative. Fortunately, the effects of response style can be minimized (from a statistical perspective, eliminated) through the use of ipsative scoring. This practice is common in survey research \citep{TenBerge:1999bg}, including personality research \citep{Goldberg:92}, and the procedures for ipsatization are straight-forward (I have demonstrated how to ipsatize scores and evaluate the results for these samples in Section \ref{sec:ipsatize}). In fact, I would recommend routinely evaluating the effects of ipsatization in sensitivity analyses because there is information to be gained by doing so. I don't believe there is any marginal utility to be gained through the use of balanced scales. If anything, the use of balanced scales may mask the evidence for differences in response styles across sub-samples. Given this, the development of carefully balanced scales at the expense of internal consistency does not seem warranted. \section{Confirmatory factor analysis} \label{sec:confirmatory} The fact that CFAs are viewed by most psychometricians as the standard methodological protocol for testing structural models serves as the primary rational for conducting a confirmatory factor analyis in this project. Clearly, my first choice for testing the structure identified in the Exploratory sample was to use EFA replication techniques, but I also acknowledge that these are uncommonly used techniques. So, a third and final sample was collected in order to conduct a CFA. The distinction between EFA replication and CFA \textit{for these specific purposes} is not trivial. The different approaches for evaluating structural fit are not much different than those which might be used for other fit evaluations. The CFA approach is a little like taking your favorite pair of jeans to the store and then evaluating pairs in the store's inventory based on the degree to which they match the ones you brought in. The EFA replication approach is more closely aligned with the shopping process most people actually use --- you try on a few (or more) pairs and buy the ones that match your body best. It seems to me that the latter method is preferred because it more directly measures the structure of reality. But, it's better still to use both approaches instead of choosing only one. \subsection{Model fits} The following code chunk shows how the 27 factor model is specified and then used to evaluate fit in the Confirmatory sample. The same procedures are then used with the 5 factor model (not shown in the pdf version). \begin{scriptsize} <>= library(lavaan) library(semTools) load(paste(filepathdata, "descriptivesData.Rdata", sep = "")) load(paste(filepathdata, "factorContentData.Rdata", sep = "")) rm(list=setdiff(ls(), c("confSampleCor696", "DataItems", "filepathdata", "filepathfig", "ItemInfo696", "keys.list5", "keys.list27", "pwiseAdmins1C"))) for (i in 1:27) { assign(paste("CFA27_", i, sep=""), paste("F", i, " =~ ", eval(keys.list27$items[[i]][1]), " + ", eval(keys.list27$items[[i]][2]), " + ", eval(keys.list27$items[[i]][3]), " + ", eval(keys.list27$items[[i]][4]), " + ", eval(keys.list27$items[[i]][5]), sep="")) } assign(paste("SPI27model"), paste(eval(CFA27_1), " \n ", eval(CFA27_2), " \n ", eval(CFA27_3), " \n ", eval(CFA27_4), " \n ", eval(CFA27_5), " \n ", eval(CFA27_6), " \n ", eval(CFA27_7), " \n ", eval(CFA27_8), " \n ", eval(CFA27_9), " \n ", eval(CFA27_10), " \n ", eval(CFA27_11), " \n ", eval(CFA27_12), " \n ", eval(CFA27_13), " \n ", eval(CFA27_14), " \n ", eval(CFA27_15), " \n ", eval(CFA27_16), " \n ", eval(CFA27_17), " \n ", eval(CFA27_18), " \n ", eval(CFA27_19), " \n ", eval(CFA27_20), " \n ", eval(CFA27_21), " \n ", eval(CFA27_22), " \n ", eval(CFA27_23), " \n ", eval(CFA27_24), " \n ", eval(CFA27_25), " \n ", eval(CFA27_26), " \n ", eval(CFA27_27), sep = "")) SPI27cfaOblique <- cfa(SPI27model, sample.cov = confSampleCor696, sample.nobs = 697, std.lv = TRUE, orthogonal = FALSE) fitsSPI27cfaOblique <- fitMeasures(SPI27cfaOblique) @ \end{scriptsize} \begin{scriptsize} <>= SPI27cfaOrthogonal <- cfa(SPI27model, sample.cov = confSampleCor696, sample.nobs = 697, std.lv = TRUE, orthogonal = TRUE) for (i in 1:5) { assign(paste("CFA5_", i, sep=""), paste("F", i, " =~ ", eval(keys.list5$items[[i]][1]), " + ", eval(keys.list5$items[[i]][2]), " + ", eval(keys.list5$items[[i]][3]), " + ", eval(keys.list5$items[[i]][4]), " + ", eval(keys.list5$items[[i]][5]), " + ", eval(keys.list5$items[[i]][6]), " + ", eval(keys.list5$items[[i]][7]), " + ", eval(keys.list5$items[[i]][8]), " + ", eval(keys.list5$items[[i]][9]), " + ", eval(keys.list5$items[[i]][10]), " + ", eval(keys.list5$items[[i]][11]), " + ", eval(keys.list5$items[[i]][12]), " + ", eval(keys.list5$items[[i]][13]), " + ", eval(keys.list5$items[[i]][14]), sep="")) } assign(paste("SPI5model"), paste(eval(CFA5_1), " \n ", eval(CFA5_2), " \n ", eval(CFA5_3), " \n ", eval(CFA5_4), " \n ", eval(CFA5_5), sep = "")) SPI5cfaOblique <- cfa(SPI5model, sample.cov = confSampleCor696, sample.nobs = 697, std.lv = TRUE) fitsSPI5cfaOblique <- fitMeasures(SPI5cfaOblique) fitsSPI27cfaObliquermsea <- fitsSPI27cfaOblique["rmsea"] fitsSPI5cfaObliquermsea <- fitsSPI5cfaOblique["rmsea"] fitsSPI27cfaObliquesrmr <- fitsSPI27cfaOblique["srmr"] fitsSPI5cfaObliquesrmr <- fitsSPI5cfaOblique["srmr"] fitsSPI27cfaObliqueaic <- fitsSPI27cfaOblique["aic"] fitsSPI5cfaObliqueaic <- fitsSPI5cfaOblique["aic"] fitsSPI27cfaObliquebic <- fitsSPI27cfaOblique["bic"] fitsSPI5cfaObliquebic <- fitsSPI5cfaOblique["bic"] RMSEAnull27Oblique <- suppressWarnings(suppressMessages(nullRMSEA(SPI27cfaOblique))) RMSEAnull27Oblique RMSEAnull5Oblique <- suppressWarnings(suppressMessages(nullRMSEA(SPI5cfaOblique))) RMSEAnull5Oblique fitsTable <- round(data.frame(fitsSPI27cfaOblique, fitsSPI5cfaOblique), 4) fitsTableCleaner <- fitsTable[c("chisq", "df", "pvalue", "rmsea", "rmsea.ci.lower", "rmsea.ci.upper", "rmr", "srmr", "cfi", "tli", "nnfi", "rfi", "nfi", "pnfi", "ifi", "rni", "aic", "bic", "bic2", "gfi", "agfi", "pgfi", "mfi", "ecvi"),] rownames(fitsTableCleaner) <- c("Chi-Square", "Degrees of freedom", "p value (proposed v null)", "Root Mean Square Error of Approximation (RMSEA)", "RMSEA - confidence interval lower bound", "RMSEA - confidence interval upper bound", "Root Mean Square Residual (RMR)", "Standardized Root Mean Square Residual (SRMR)", "Comparative Fit Index (CFI)", "Tucker-Lewis Index (TLI)", "Bentler-Bonett Non-normed Fit Index (NNFI)", "Bentler-Bonett Normed Fit Index (NFI)", "Parsimony Normed Fit Index (PNFI)", "Bollen's Relative Fit Index (RFI)", "Bollen's Incremental Fit Index (IFI)", "Relative Noncentrality Index (RNI)", "Akaike Information Criterion (AIC)", "Bayesian Information Criterion (BIC)", "Sample-size adjusted Bayesian (SABIC)", "Goodness of Fit Index (GFI)", "Adjusted Goodness of Fit Index (AGFI)", "Parsimony Goodness of Fit Index (PGFI)", "McDonald Fit Index (MFI)", "Expected Cross-Validation Index (ECVI)") colnames(fitsTableCleaner) <- c("SPI-27", "SPI-5") @ \end{scriptsize} Unfortunately, the most widely-used fit statistics are not very useful for evaluations of fit based on samples with many participants and items. That includes the Confirmatory sample which has nearly 700 mean pairwise observations between items and 696 variables. The null models in the CFAs reported here include 9,000 degrees of freedom for the 27 factor model and 2,500 degrees of freedom for the 5 factor model. Given this, the chi-square statistics for evaluations of both models are highly likely to be statistically significant and... virtually all other fit statistics are fundamentally based on the chi-square. It's safe to assume --- even before I have conducted the CFAs --- that the fit statistics will be difficult to interpret. To avoid cherry-picking, I have shown most of the fit statistics provided by the lavaan package \citep{lavaan:12} for both the 27 factor and 5 factor models in Table \ref{tab:fitsTable}. Some of these do not appear to be compelling using traditional standards, but this is mainly because these standards were derived based on models with small sample sizes and far fewer degrees of freedom \citep{Kenny:2014jc}. For example, \citet{Kenny:12} does not even recommend reporting the so-called ``incremental'' measures of fit --- these include the non-normed fit index (NNFI; aka the Tucker-Lewis Index [TLI]), the normed fit index (NFI), and the comparative fit index (CFI) --- when the RMSEA of the null model is below .158 as the range of possible values is restricted below thresholds that are generally considered to indicate good fit. I've reported them anyway and he's right; the values for these statistics range from .54 to .64 for the 5 factor model and .61 to .80 for the 27 factor model (see Table \ref{tab:fitsTable}). The RMSEA for the null model with 27 factors and 5 factors is \Sexpr{prettyNum(round(RMSEAnull27Oblique, 4), big.mark=",", scientific=F)} and \Sexpr{prettyNum(round(RMSEAnull5Oblique, 4), big.mark=",", scientific=F)}, respectively. The absolute measures of fit (RMSEA and standardized root mean square residual [SRMR]) are very good for both of the proposed models. RMSEA for the 27 and 5 factor models are \Sexpr{prettyNum(round(fitsSPI27cfaObliquermsea, 4), big.mark=",", scientific=F)} and \Sexpr{prettyNum(round(fitsSPI5cfaObliquermsea, 4), big.mark=",", scientific=F)}, respectively. SRMR is \Sexpr{prettyNum(round(fitsSPI27cfaObliquesrmr, 4), big.mark=",", scientific=F)} and \Sexpr{prettyNum(round(fitsSPI5cfaObliquesrmr, 4), big.mark=",", scientific=F)}, respectively. \begin{scriptsize} <>= suppressPackageStartupMessages(library(plyr)) suppressPackageStartupMessages(library(xtable)) digits <- matrix(c(rep(0,(2*3)), rep(3,(14*3)), rep(0,(3*3)), rep(3,(4*3)), rep(2,(1*3))), nrow = 24, ncol=3, byrow=TRUE) print(xtable(fitsTableCleaner, digits = digits, caption = 'CFA fit statistics for the 27 and 5 factor models', label = 'tab:fitsTable', align = c("l", "r", "r")), size='footnotesize', floating = TRUE) @ \end{scriptsize} Comparative fit statistics are --- at least in theory --- useful for comparing the 5 and 27 factor models because they allow for comparisons based on differing numbers of parameters. The 27 factor model has 135 parameters while the 5 factor model uses only 70 of those same parameters. The Akaike Information Criterion (AIC) is \Sexpr{prettyNum(round(fitsSPI27cfaObliqueaic, 0), big.mark=",", scientific=F)} for the 27 factor model and \Sexpr{prettyNum(round(fitsSPI5cfaObliqueaic, 0), big.mark=",", scientific=F)} for the 5 factor model; the Bayesian Information Criterion (BIC) values are \Sexpr{prettyNum(round(fitsSPI27cfaObliquebic, 0), big.mark=",", scientific=F)} and \Sexpr{prettyNum(round(fitsSPI5cfaObliquebic, 0), big.mark=",", scientific=F)}, respectively. Both of these suggest that the 5 factor model provides a ``better'' fitting solution. I say that this is only theoretically useful because these statistics emphasize parsimony. In other words, they indicate that the 5 factor model is more parsimonious than the 27 factor model, a point which I find rather obvious and largely irrelevant to the issue of validity (and other aspects of utility). \begin{scriptsize} <>= library(semPlot) pdf(file=paste(filepathfig, "SPI5cfaPath.pdf", sep=""), width = 9, height = 9) semPaths(SPI5cfaOblique, "std", layout = "circle", residuals = FALSE, title = FALSE, edge.label.cex=0.5, curvePivot = TRUE, filetype = "", sizeMan = 1.5, sizeLat = 3, sizeInt = .5) dev.off() SPI5cfaCor <- cov2cor(inspect(SPI5cfaOblique, "coefficients")$psi) pdf(paste(filepathfig, "SPI5cfaFactors.pdf", sep = ""), onefile = TRUE) cor.plot(SPI5cfaCor, numbers=TRUE, cex.axis=.7, main="", cex=1) dev.off() pdf(file=paste(filepathfig, "SPI27cfaPath.pdf", sep=""), width = 9, height = 9) semPaths(SPI27cfaOblique, "std", layout = "circle", residuals = FALSE, title = FALSE, edge.label.cex=0.5, curvePivot = TRUE, filetype = "", sizeMan = 1.5, sizeLat = 3, sizeInt = .5) dev.off() SPI27cfaCorOrtho <- cov2cor(inspect(SPI27cfaOrthogonal, "coefficients")$psi) pdf(paste(filepathfig, "SPI27cfaFactorsOrtho.pdf", sep = ""), onefile = TRUE) cor.plot(SPI27cfaCorOrtho, numbers=TRUE, cex.axis=.7, main="", cex=.4) dev.off() SPI27cfaCor <- cov2cor(inspect(SPI27cfaOblique, "coefficients")$psi) pdf(paste(filepathfig, "SPI27cfaFactors.pdf", sep = ""), onefile = TRUE) cor.plot(SPI27cfaCor, numbers=TRUE, cex.axis=.7, main="", cex=.4) dev.off() @ \end{scriptsize} \begin{figure} \ffigbox{\includegraphics[trim={3cm 1cm 3cm 1cm}, clip, width=.95\textwidth]{figures/SPI27cfaPath.pdf}}{\caption{Path diagram for the 27 factor model in the Confirmatory sample}\label{fig:SPI27cfaPath}} \end{figure} \begin{figure} \ffigbox{\includegraphics[trim={1cm 0 1cm 0}, clip, width=.95\textwidth]{figures/SPI27cfaFactors.pdf}}{\caption{Correlations among the SPI-27 factors in the Confirmatory sample}\label{fig:SPI27cfaFactors}} \end{figure} Aside from overall fit, the models generally performed as expected. Figure \ref{fig:SPI27cfaPath} shows the standardized item loadings (estimates) for the 27 factor model (note that the numeric values are hidden for the sake of legibility; magnitudes are indicated by the intensity of the arrow shading). All but 5 of the 135 item-factor loadings were greater than 0.4 in absolute magnitude. Only 1 of these was particularly problematic (less than 0.3 in absolute magnitude) --- item ``q\_142'' had a standardized loading of only 0.18 on factor 21. Recall that factor 21 was also the most problematic factor in the replication EFA. The remaining four items in factor 21 loaded very highly (between .52 and .67). Consistent with my prior commentary (see Section \ref{sec:}), I think it's best to retain this factor --- and this item --- for now with the intention to rigorously evaluate it's predictive validity in future projects. It's likely that this will be revised in future versions of the measure. The factor correlations for the 27 factor solution are shown in Figure \ref{fig:SPI27cfaFactors}. Entire manuscripts might be written about the relationships between these factors,\footnote{In some sense, they already have been written, using slightly different hierarchical taxonomies.} but I will limit myself to only a few observations for now. First, \textbf{the factors are correlated}.\footnote{Unless you have begun reading in this section, I hope you might say to yourself ``right, we knew that.'' I have also conducted these CFAs using orthogonal rotations but have omitted a summary of the results to prevent confusion (see the Rnw version). Guess what happens? The factors are uncorrelated (duh, orthogonal) and the fit is worse, as expected (the 90\% confidence interval for the 27 factor model is RMSEA is .045 to .047).} Second, the correlations are \textit{mainly} low to moderate in strength (absolute magnitudes less than 0.3) but there are many intriguing exceptions. Factor 8, to pick one example, is correlated 0.67 with factor 25 and 0.62 with factor 13. I have chosen this example because the evidence for redundancy in this case is low (i.e., the remaining correlations for these factors are not consistent and the correlation between factors 13 and 25 is only 0.3). Third, there is evidence for higher order factors. Factors 15 through 18 (and, to a lesser extent, factors 1, 3, 4, and 12) clearly hang together. These are all facets of the most salient factor in the 5 factor model --- Extraversion.\footnote{From a measurement perspective, this introduces the question of whether all these factors need to be assessed separately. Are they all independently useful predictors of important outcomes? For predicting the size of one's social network, probably not. But it's not hard to imagine situations where the demand characteristics for Extraversion are very strong and it may be useful to differentiate among the underlying factors in these circumstances.} If one takes the time to \href{https://youtu.be/Pv_78in5iGE}{rearrange} the factors in this correlation matrix and/or factor the scores, it is possible to identify much more evidence of higher order structure, even beyond the 5 factor level. See \citep{Condon:14Diss} for discussion of the evidence at the 3 factor level; I referred to these factors as Socialability, Emotionality, and Social Cohesion --- not much different from the P-E-N model \citep{Eysenck:81, Eysenck:91}. In general, the evidence for unnested hierarchical structure at multiple levels in this independent sample is consistent with the findings from the Exploratory and Replication samples. How many factors of personality do you want? \begin{figure} \ffigbox{\includegraphics[trim={3cm 3cm 3cm 3cm}, clip, width=.95\textwidth]{figures/SPI5cfaPath.pdf}}{\caption{Path diagram for the 5 factor model in the Confirmatory sample}\label{fig:SPI5cfaPath}} \end{figure} \begin{figure} \ffigbox{\includegraphics[trim={1cm 0 1cm 0}, clip, width=.95\textwidth]{figures/SPI5cfaFactors.pdf}}{\caption{Correlations among the SPI-5 factors in the Confirmatory sample}\label{fig:SPI5cfaFactors}} \end{figure} Figure \ref{fig:SPI5cfaPath} shows the standardized item loadings (estimates) and factor correlations for the 5 factor model. In this case, all but 3 of the 70 standardized estimates were greater than 0.3 in absolute magnitude. Two of these had loadings of only .2 and both were in factor 5. Both of these items relate to ``unconventionality,'' which is (ironically) an uncommonly measured facet of Openness. I feel these items should be retained because they likely add incremental validity, though they may not fit well at the 5 factor level. The correlations between factors are shown in Figure \ref{fig:SPI5cfaFactors} (this is slightly redundant with Figure \ref{fig:SPI5cfaPath}). The factors all reflect low to moderate correlations with absolute magnitudes between .08 and .34. \subsection{CFA conclusions} First, the RMSEA and SRMR values allow us to state that, in absolute terms, the 27 factor model fits very well and the 5 factor model fits well. Within each model, the standardized item loadings are generally quite high, with few exceptions. The correlations between factors are mainly low to moderate, but there are several strong correlations in the 27 factor model. Second, from a methodological perspective, traditional fit statistics are not useful when fitting models with many parameters and large samples. This is because virtually all of them are variations on the chi-square test and therefore sensitive to sample size or --- to a lesser extent --- degrees of freedom. The absolute fit indices (RMSEA and SRMR) are preferred in circumstances like these because they allow for comparisons against the standard of perfect fit, though these should be used cautiously as well because the large degrees of freedom and sample size make even the null/independent models appear reasonable. Finally, I think these analyses help to reinforce the approach of using EFA replication techniques in addition to CFA (or even in place of it). Particularly with large samples, EFA replication helps to demonstrate whether the proposed model represents the best structure. \section{Naming procedures for the scales} \label{sec:naming} For many of the SPI-27 scales, it was relatively easy to generate names for the 5-item scales that were consistent with constructs previously described in the personality literature. However, there were a few scales which described seemingly-coherent traits that were hard to name or that might have been paired with more than one construct\footnote{This is a persistent problem in psychology and personality psychologists are, in my opinion, among the worst offenders. This \href{https://www.brookings.edu/blog/social-mobility-memos/2014/12/19/jingle-jangle-fallacies-for-non-cognitive-factors/}{post} provides a nice overview.}. One controversial example was the SPI-5 factor that maps closely onto Agreeableness. The items in this factor did not explicitly relate to the tendency to agree with others or to be consenting or even deferential. They were more related to empathy and compassion. While the attribute of agreeableness (with a lower case A) is probably more highly correlated with this trait than with the remaining 4 SPI-5 factors, it is not \textit{the} defining attribute. This is not only true for the items in the SPI-5 operationalization of this scale, but also for the items in the IPIP-NEO, the IPIP items measuring the Big Five Factor Markers and the IPIP-HEXACO. An example from the 27 factor solution included a factor where my opinion about the most accurate label (``laziness'') was likely to be unfairly biased (and more than a little judgmental). To avoid problems like these during the naming of the 27 factor scales, I decided to employ a method from biomedical research that is sometimes referred to as ``citizen-engagement,'' also known as crowd-sourcing. I designed an ad hoc four-step method that was intended to incorporate the views of both ordinary citizens and personality psychologists. \begin{scriptsize} <>= load(paste(filepathdata, "QualitativeInfo.Rdata", sep="")) cohPs @ \end{scriptsize} In step 1, I generated names for each of the factors on my own. These were pre-registered on the Open Science Framework prior to the collection of any data (see \href{https://osf.io/e7q3q/register/565fb3678c5e4a66b5582f67}{the pre-registration page}). In step 2, participants recruited from Amazon's Mechanical Turk service were asked to consider a random subset of the 27 factor scales. This sample included \Sexpr{prettyNum(round(cohPs, 0), big.mark=",", scientific=F)} participants from countries where English is the national language. For each scale, participants were presented with all of the items and then asked to rate the coherence of each set (from 1 - very incoherent to 6 - very coherent) and to give it a one- or two-word label. Each participant rated 12 of the 27 factors in addition to 4 random item sets (and additional questions that attempted to confirm their English fluency). This step produced a mean of \Sexpr{prettyNum(round(nomsS1mean, 0), big.mark=",", scientific=F)} (\textit{sd} = \Sexpr{prettyNum(round(nomsS1sd, 1), big.mark=",", scientific=F)}) non-unique nominations for the name of each factor. In the third step, the nominations from step 2 were used to generate a wordcloud of potential names for each factor. This was planned because I was concerned that relatively little consensus would emerge from the first survey --- I anticipated that there would be at least a couple of factors where most of the 100 nominations would be unique. The use of wordclouds as stimuli instead of the full text of all items in each scale was mainly intended to encourage participants to choose appealling labels for personality traits rather than focusing on the best possible label for the 5 item sets. Secondary benefits included the fact that I wanted to make the task a bit less cognitively challenging and reduce the amount of time required for participants to generate nominations --- thus making data collection more efficient. The point about getting ``appealling labels'' warrants more explanation. In the first round of nominations (Step 2 of the naming process), I asked for nominations based on the item content. The task was to generate labels that matched the items. In this process, it was possible for some very creative and appealling labels to be nominated but it was unlikely that such labels would receive \textit{many} nominations. In the second round, by asking for nominations based on the previous round of nominations, I was allowing for rare-but-clever labels to receive more nominations by using word clouds from the first survey.\footnote{Yes, I was concerned that the introduction of this intermediate step would introduce unforeseeable confounds but I also hoped to mitigate this risk somewhat in the last step by using nominations from both surveys as candidates for voting.} An example of the wordclouds used in the second survey is shown for Factor 6 in Figure \ref{fig:wordcloud}. All of the words in each cloud were the same size font in order to discourage respondents from simply selecting the most frequent (i.e., the largest) word in the image. This survey was administered to \Sexpr{prettyNum(round(nomsS2, 0), big.mark=",", scientific=F)} participants, each of whom was asked to nominate one- or two-word names for all 27 wordclouds. Data from 1 participant was omitted because they did not follow directions. Participants were not asked to rate coherence in this round. \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[trim={8cm 0 0 0}, clip, width=.75\textwidth]{figures/wordcloudF6.png}}{\caption{Wordcloud used in the second survey with nominations for Factor 6}\label{fig:wordcloud}} \end{floatrow} \end{figure} In the last step, a third and final voting survey was constructed using my nominations in step one and the most frequently suggested nominations in steps two and three. Five multiple choice response options were chosen to be presented along with the full text of the items in each of the 27 scales. These choices were selected from the most frequently nominated labels in the first two surveys (without using different versions of the same word). If my nomination was not among the most frequently nominated labels in the surveys, they were added as one of the five choices. This final survey was administered to \Sexpr{cohPs} participants who were recruited to participate voluntarily using email lists of members in several professional associations to which personality psychologists are members. \footnote{These include the European Association of Personality Psychology, the Association for Research in Personality, the Society of Personality and Social Psychologists, the Australian Psychology Society, the Ibero-American Association for the Investigation of Individual Differences, the British Society of the Psychology of Individual Differences, the Canadian Psychological Association, the division of personality research in the Chinese Society of Psychology, the International Society for the Study of Individual Differences, and the Society for Personality Assessment.} The nomination that received the most votes in this step was used as the name for each factor. More details regarding these procedures and the data collected in each step are given in Condon, Graham, Weston \& Mroczek (in prep). In the following code chunk (not shown), the names for each scale are combined with the item content to generate Table \ref{tab:orgtable}. \begin{scriptsize} <>= names27 <- c("Compassion", "Irritability", "Sociability", "WellBeing", "SensationSeeking", "Anxiety", "Honesty", "Industry", "Intellect", "Creativity", "Impulsivity", "AttentionSeeking", "Order", "Authoritarianism", "Charisma", "Trust", "Humor", "EmotionalExpressiveness", "ArtAppreciation", "Introspection", "Perfectionism", "SelfControl", "Conformity", "Adaptability", "EasyGoingness", "EmotionalStability", "Conservatism") namesList27 <- list() for (i in 1:length(names27)) { namesList27 <- list(namesList27, rep(names27[[i]], 5)) } namesList27 <- unlist(namesList27) names(keys.list27$items) <- names27 names(keys.list27$keys) <- names27 SPI27items <- as.vector(unlist(keys.list27$items)) SPI27content <- ItemInfo696[SPI27items,] itemContent27 <- data.frame(SPI27content[, "Item"], namesList27) rownames(itemContent27) <- rownames(SPI27content) colnames(itemContent27) <- c("Item", "SPI27 Factor") names5 <- c("Extraversion", "Neuroticism", "Conscientiousness", "Agreeableness", "Openness") namesList5 <- list() for (i in 1:length(names5)) { namesList5 <- list(namesList5, rep(names5[[i]], 14)) } namesList5 <- unlist(namesList5) names(keys.list5$items) <- names5 names(keys.list5$keys) <- names5 SPI5items <- as.vector(unlist(keys.list5$items)) SPI5content <- ItemInfo696[SPI5items,] itemContent5 <- data.frame(SPI5content[, "Item"], namesList5) rownames(itemContent5) <- rownames(SPI5content) colnames(itemContent5) <- c("Item", "SPI5 Factor (70 items)") namesDiff1 <- rep("NA", 65) SPIdiffcontent1 <- ItemInfo696[setdiff(SPI27items, SPI5items),] itemContentdiff1 <- data.frame(SPIdiffcontent1[, "Item"], namesDiff1) rownames(itemContentdiff1) <- rownames(SPIdiffcontent1) colnames(itemContentdiff1) <- c("Item", "SPI5 Factor (70 items)") itemContentTemp1 <- rbind(itemContent5, itemContentdiff1) itemContentTemp1 <- itemContentTemp1[SPI27items,] SPI5content2 <- c("q_1904", "q_4243", "q_312", "q_565", "q_1416", "q_1923", "q_1027", "q_684", "q_254", "q_1296", "q_901", "q_1243", "q_803", "q_1244", "q_1635", "q_219", "q_296", "q_1781", "q_1248", "q_598", "q_1662", "q_1045", "q_1371", "q_1052", "q_1081", "q_1242", "q_1555", "q_979", "q_4252", "q_1989", "q_1505", "q_4249", "q_808", "q_793", "q_1840", "q_811", "q_1585", "q_578", "q_176", "q_797", "q_1683", "q_820", "q_1706", "q_566", "q_1357", "q_1290", "q_1744", "q_1979", "q_1452", "q_1915", "q_1201", "q_530", "q_904", "q_1867", "q_1694", "q_369", "q_1444", "q_1483", "q_1254", "q_571", "q_398", "q_169", "q_736", "q_1024", "q_90", "q_1763", "q_253", "q_1896", "q_851", "q_1832", "q_501", "q_377", "q_871", "q_1855", "q_4296", "q_142", "q_379", "q_4289", "q_1367", "q_1664", "q_128", "q_2745", "q_2754", "q_1392", "q_1058", "q_240", "q_1738", "q_422", "q_1389", "q_1310", "q_1880", "q_747", "q_1609", "q_1834", "q_1132", "q_493", "q_1303", "q_610", "q_755", "q_1300", "q_1653", "q_2005", "q_607", "q_612", "q_152", "q_1253", "q_1542") leftoverItems <- setdiff(SPI27items, SPI5content2) namesListDiff2 <- list() namesListDiff2 <- list(namesListDiff2, rep(names5[1], 27)) namesListDiff2 <- list(namesListDiff2, rep(names5[2], 18)) namesListDiff2 <- list(namesListDiff2, rep(names5[3], 19)) namesListDiff2 <- list(namesListDiff2, rep(names5[4], 16)) namesListDiff2 <- list(namesListDiff2, rep(names5[5], 27)) namesListDiff2 <- list(namesListDiff2, rep("NA", length(leftoverItems))) namesDiff2 <- unlist(namesListDiff2) SPIdiffcontent2 <- ItemInfo696[c(SPI5content2, leftoverItems),] itemContentdiff2 <- data.frame(SPIdiffcontent2[, "Item"], namesDiff2) rownames(itemContentdiff2) <- rownames(SPIdiffcontent2) colnames(itemContentdiff2) <- c("Item", "SPI5 Factor (invariant)") itemContentTemp2 <- itemContentdiff2[SPI27items,] coding <- c("+", "+", "+", "+", "+", "+", "+", "-", "-", "-", "+", "-", "-", "+", "-", "-", "-", "+", "+", "+", "+", "+", "+", "+", "+", "+", "+", "+", "+", "+", "-", "+", "+", "-", "-", "+", "+", "+", "-", "-", "-", "+", "+", "+", "+", "+", "+", "+", "+", "+", "+", "+", "+", "-", "+", "+", "+", "-", "+", "-", "+", "+", "-", "-", "+", "+", "+", "+", "+", "-", "+", "-", "+", "+", "-", "+", "+", "+", "-", "+", "+", "+", "+", "+", "-", "+", "-", "+", "-", "+", "-", "+", "-", "+", "-", "+", "+", "+", "+", "+", "+", "+", "+", "+", "+", "+", "+", "+", "+", "+", "+", "+", "+", "+", "+", "-", "-", "+", "+", "+", "+", "+", "+", "+", "+", "-", "-", "+", "+", "-", "+", "-", "+", "-", "+") itemContent <- data.frame(itemContent27, itemContentTemp1[,"SPI5 Factor (70 items)"], itemContentTemp2[,"SPI5 Factor (invariant)"], coding) colnames(itemContent) <- c("Item Content", "SPI27 Factor", "SPI5 Factor (70 items)", "SPI5 Factor (invariant)", "SPI27 Coding") @ \end{scriptsize} \clearpage \begin{tiny} <>= suppressPackageStartupMessages(library(xtable)) addtorow <- list(pos = list(0), command = NULL) command <- paste0("\\hline\n\\endhead\n", "\\hline\n", "\\multicolumn{5}{c}", "{\\textit{(continued on next page)}}\n", "\\endfoot\n", "\\endlastfoot\n") addtorow$command <- command print(xtable(itemContent[,1:4], caption='Organization of items by factor', label='tab:orgtable', align = c("l", "l", "l", "l", "l")), size='tiny', caption.placement = "top", tabular.environment="longtable", floating=FALSE, add.to.row = addtorow) @ \end{tiny} \section{Describe the scales} \label{sec:psychodescribe} \subsection{Internal consistency} \label{sec:ic} Here we consider the internal consistency values for the scales at each level. To do this (and also to evaluate other properties of these scales in this section), I combine the Exploratory, Replication, and Confirmatory samples. \begin{scriptsize} <>= load(paste(filepathdata, "descriptivesData.Rdata", sep = "")) bothSample <- rbind(expSample, repSample) allSample <- rbind(bothSample, confSample) allSampleCor <- cor(subset(allSample, select = c(DataItems)), use="pairwise") allSampleCor <- cor.smooth(allSampleCor) SPIKey <- make.keys(allSample[,DataItems], c(keys.list27$keys, keys.list5$keys)) allSampleDataItemsScoredSPI <- scoreItems(SPIKey, allSample[,DataItems], totals=FALSE, impute="none", min=1, max=6) allSampleDataItemsScoredOverlapSPI <- scoreOverlap(SPIKey, allSampleCor, impute="none") rely <- data.frame() for (i in 1:27) { alpha1 <- suppressWarnings(alpha(allSample[,keys.list27[[2]][[i]]], check.keys=TRUE)) alpha <- alpha1$total$raw_alpha unidim <- alpha1$Unidim$Unidim rows <- c(alpha, unidim) rows <- round(rows, 2) rely <- rbind(rely, rows) } rely <- data.frame(rely, cohScore) rownames(rely) <- names27 #colnames(rely) <- c("Alpha", "Unidimensionality") colnames(rely) <- c("Alpha", "Unidimensionality", "Coherence") SPI_27_ICs <- rely rely <- data.frame() for (i in 1:5) { alpha1 <- suppressWarnings(alpha(allSample[,keys.list5[[2]][[i]]], check.keys=TRUE)) alpha <- alpha1$total$raw_alpha unidim <- alpha1$Unidim$Unidim rows <- c(alpha, unidim) rows <- round(rows, 2) rely <- rbind(rely, rows) } rownames(rely) <- names5 colnames(rely) <- c("Alpha", "Unidimensionality") SPI_5_ICs <- rely names(keys.list27$items) <- names27 names(keys.list27$keys) <- names27 names(keys.list5$items) <- names5 names(keys.list5$keys) <- names5 save(allSampleDataItemsScoredSPI, allSampleDataItemsScoredOverlapSPI, allSample, allSampleCor, filepathdata, filepathfig, itemContent, ItemInfo696, keys.list27, keys.list5, SPI_27_ICs, SPI_5_ICs, SPIKey, file = paste(filepathdata, "keysAndScalesInfo.Rdata", sep = "")) @ \end{scriptsize} The internal consistency and unidimensionality values are more than adequate for all but three cases (see Tables \ref{tab:SPI_27_ICs} and \ref{tab:SPI_5_ICs}). Two of the 27 facets and one of the 5 factors have unidimensionality values below 0.7 --- Perfectionism (0.61) and Easy-Goingness (0.64) and Openness (SPI-5) (0.59). While I think it is marginally acceptable to use values around 0.75 or higher (0.8 is preferred), the unidimensionality in these two cases is much lower. This is relevant to the extent that we intend to use item-response modeling for the development of CATs and for evaluations of differential item functioning. Note that Table \ref{tab:SPI_27_ICs} also includes estimates of the mean and confidence intervals for the ratings of ``coherence.'' Recall from Section \ref{sec:naming} that participants rated the coherence of each scale while suggesting nominations for the names of the scales. These ratings have been scaled such that a coherence rating of 0 reflects the mean rating given to 5 item sets grouped at random. A rating of 1 would reflect a consensus rating of very coherent (using a 6 point scale) across all raters. All of the SPI-27 scales were rated significantly higher than random (p $<$ .001). While the correlation between alpha and unidimensionality (omega) was nearly perfect (\textit{r} $=$ .99), the coherence ratings were moderately negatively correlated with both alpha (\textit{r} $=$ -.20) and omega (\textit{r} $=$ -.19). There are several possible interpretations. Part of the issue seems to be that raters struggle with reverse coded items --- the correlations are slightly less negative (\textit{r} $=$ -.10) if the coherence ratings are adjusted by the proportion of oppositely coded items (3/5, 4/5, or 5/5). But, the bigger problem seems to be that raters tend to see \textit{more} coherence than can be demonstrated psychometrically. If the scales with unidimensionality below .75 are excluded (Introspection, Perfectionism, Self-Control, and Easy-Goingness), the coherence correlations are moderately positive (\textit{r} $=$ .36 with both alpha and unidimensionality). The ratings are somewhat biased relative to the underlying structure.\footnote{Additional evidence for this bias stems from recognition that even random groupings of items are not rated as fully incoherent (\textit{m} $=$ 2.3 on a 1 to 6 scale. People see stronger themes in groups of items than actually exists.} See Condon, Graham, Weston \& Mroczek (in prep) for more discussion of this topic. \begin{tiny} <>= suppressPackageStartupMessages(library(xtable)) addtorow <- list() addtorow$pos <- list() addtorow$pos[[1]] <- 27 addtorow$pos[[2]] <- 27 addtorow$command <- c("\\hline\n\\multicolumn{4}{l}{\\begin{scriptsize}Coherence coefficients range from 0 (items grouped at random)\\end{scriptsize}}\\\\\n", "\\multicolumn{4}{l}{\\begin{scriptsize}to 1 (the items are maximally similar).\\end{scriptsize}}\\\\\n") print(xtable(SPI_27_ICs, caption='Internal consistencies for SPI-27', label='tab:SPI_27_ICs', align = c("l", "c", "c", "c")), caption.placement = "top", add.to.row = addtorow, hline.after = c(-1,0,27)) @ \end{tiny} \begin{tiny} <>= print(xtable(SPI_5_ICs, caption='Internal consistencies for SPI-5', label='tab:SPI_5_ICs', align = c("l", "c", "c"))) @ \end{tiny} \subsection{Correlations between scales} An important consideration is the degree to which the scales correlate. Figures \ref{fig:scalesCor} and \ref{fig:hetdiag} show the correlations between the scales for the 27 and 5 factor solutions \textit{after adjusting for overlapping items} \citep{Bashaw:67}. This correlation plot contains a lot of information but I want to highlight two features in particular. First, the magnitudes between the 5 factor scales are substantial (in part because we allowed them to be oblique), ranging from $\left|0.05\right|$ to $\left|0.3\right|$. Second, there are several high correlations between the 27 and 5 factor levels. Specifically, Extraversion is correlated (using the fully corrected correlations) above $\left|0.3\right|$ with Factors 3 (Sociability), 4 (Well-Being), 5 (Sensation-Seeking), 6 (Anxiety), 12 (Attention-Seeking), 15 (Charisma), 16 (Trust), 17 (Humor), 18 (Emotional Expressiveness), and 24 (Adaptability); Neuroticism is correlated above $\left|0.3\right|$ with Factors 2 (Irritability), 4 (Well-Being), 6 (Anxiety), 8 (Industry), 9 (Intellect), 15 (Charisma), 22 (Self-Control), 24 (Adaptability), and 26 (Emotional Stability); Conscientiousness is correlated above $\left|0.3\right|$ with Factors 4 (Well-Being), 7 (Honesty), 8 (Industry), 11 (Impulsivity), 13 (Order), 14 (Authoritarianism), 21 (Perfectionism), 22 (Self-Control), 25 (Easy-Goingness), and 27 (Conservatism); Agreeableness is correlated above $\left|0.3\right|$ with Factors 1 (Compassion), 2 (Irritability), 3 (Sociability), 4 (Well-Being), 7 (Honesty), 8 (Industry), 14 (Authoritarianism), 16 (Trust), 17 (Humor), and 18 (Emotional Expressiveness); and Openess is correlated above $\left|0.3\right|$ with Factors 5 (Sensation-Seeking), 9 (Intellect), 10 (Creativity), 14 (Authoritarianism), 15 (Charisma), 19 (Art Appreciation), 20 (Introspection), 23 (Conformity), and 24 (Adaptability). All of the 27 facets are correlated above $\left|0.3\right|$ with at least one of the 5 factors; most are correlated with 2 or 3 and Well-Being is correlated above this threshold with 4 of the 5 factors. These cross-loadings are demonstrated more clearly in Figure \ref{fig:hetdiag}. The arrows indicate only the correlations above $\left|0.3\right|$ after correcting for item overlap. \begin{scriptsize} <>= pwiseAdmins1all <- pairwiseDescribe(allSample[,DataItems]) pwiseAdminsAll <- prettyNum(pwiseAdmins1all, big.mark=",", scientific=F) pwiseAdminsMeanAll <- round(pwiseAdmins1all$mean, 0) pwiseAdminsMeanAll <- prettyNum(pwiseAdminsMeanAll, big.mark=",", scientific=F) # The mean number of pairwise administrations for every possible pairing of items pwiseAdminsMeanAll pwiseAdminsMedianAll <- round(pwiseAdmins1all$median, 0) pwiseAdminsMedianAll <- prettyNum(pwiseAdminsMedianAll, big.mark=",", scientific=F) # The median number of pairwise administrations for every possible pairing of items pwiseAdminsMedianAll pwiseAdminsSDAll <- round(pwiseAdmins1all$sd, 0) pwiseAdminsSDAll <- prettyNum(pwiseAdminsSDAll, big.mark=",", scientific=F) # The SD of the number of pairwise administrations for every possible pairing of items pwiseAdminsSDAll scalesCor <- allSampleDataItemsScoredOverlapSPI$corrected scalesCor2 <- scalesCor[c("Compassion", "Trust", "Honesty", "Conservatism", "Authoritarianism", "EasyGoingness", "Perfectionism", "Order", "Industry", "Impulsivity", "SelfControl", "EmotionalStability", "Anxiety", "Irritability", "WellBeing", "EmotionalExpressiveness", "Sociability", "Adaptability", "Charisma", "Humor", "AttentionSeeking", "SensationSeeking", "Conformity", "Introspection", "ArtAppreciation", "Creativity", "Intellect", "Agreeableness", "Conscientiousness", "Neuroticism", "Extraversion", "Openness"),c("Compassion", "Trust", "Honesty", "Conservatism", "Authoritarianism", "EasyGoingness", "Perfectionism", "Order", "Industry", "Impulsivity", "SelfControl", "EmotionalStability", "Anxiety", "Irritability", "WellBeing", "EmotionalExpressiveness", "Sociability", "Adaptability", "Charisma", "Humor", "AttentionSeeking", "SensationSeeking", "Conformity", "Introspection", "ArtAppreciation", "Creativity", "Intellect", "Agreeableness", "Conscientiousness", "Neuroticism", "Extraversion", "Openness")] pdf(paste(filepathfig, "scalesCor.pdf", sep = ""), onefile = TRUE) cor.plot(scalesCor2, numbers=TRUE, cex.axis=.7, main="") dev.off() scalesCor3 <- scalesCor[c("Compassion", "Trust", "Honesty", "Authoritarianism", "EasyGoingness", "Perfectionism", "Order", "Impulsivity", "Industry", "EmotionalExpressiveness", "WellBeing", "SelfControl", "EmotionalStability", "Anxiety", "Irritability", "Adaptability", "Charisma", "Humor", "AttentionSeeking", "SensationSeeking", "Sociability", "Conformity", "Introspection", "ArtAppreciation", "Creativity", "Intellect", "Conservatism", "Agreeableness", "Conscientiousness", "Neuroticism", "Extraversion", "Openness"),c("Compassion", "Trust", "Honesty", "Authoritarianism", "EasyGoingness", "Perfectionism", "Order", "Impulsivity", "Industry", "EmotionalExpressiveness", "WellBeing", "SelfControl", "EmotionalStability", "Anxiety", "Irritability", "Adaptability", "Charisma", "Humor", "AttentionSeeking", "SensationSeeking", "Sociability", "Conformity", "Introspection", "ArtAppreciation", "Creativity", "Intellect", "Conservatism", "Agreeableness", "Conscientiousness", "Neuroticism", "Extraversion", "Openness")] SPI5nums = c(which(colnames(scalesCor3) == "Agreeableness"):which(colnames(scalesCor3) == "Openness")) SPI27nums = c(which(colnames(scalesCor3) == "Compassion"):which(colnames(scalesCor3) == "Conservatism")) pdf(paste(filepathfig, "hetdiag.pdf", sep = ""), onefile = TRUE) op <- par(mfrow = c(1,1), pty = "s", mai = c(0.15,0.45,0.45,0.45)) # square plotting region with specified margins het.diagram(scalesCor3, levels = list(SPI5 = SPI5nums, SPI27 = SPI27nums), cut=.3, digits=2, main="", cex=.6, l.cex = .7) par(op) dev.off() @ \end{scriptsize} \begin{figure} \ffigbox{\includegraphics[width=.95\textwidth]{figures/scalesCor.pdf}}{\caption{Correlations between scales for the SPI-27 and the SPI-5. Correlations below the diagonal are corrected for item overlap, where needed, and scores above the diagonal are corrected for item overlap and attenuation. Cronbach's alpha values are shown on the diagonal.}\label{fig:scalesCor}} \end{figure} \begin{figure} \ffigbox{\includegraphics[width=.95\textwidth]{figures/hetdiag.pdf}}{\caption{Correlations between scales for the SPI-27 and the SPI-5. Correlations corrected for item overlap but not for attenuation. These match the values below the diagonal in Figure \ref{fig:scalesCor}. Only values greater than $\left|0.3\right|$ are shown.}\label{fig:hetdiag}} \end{figure} \subsection{Coverage of the multidimensional space} Perhaps the most important goal of scale development is to develop measures that explain a good proportion of the variance without being inefficiently redundant. One manner of depicting how well any given set of scales achieves this goal is by use of a network plot, as shown in Figures \ref{fig:spi27qoverlay} and \ref{fig:IPIPneoAllfacets300overlay}. These plots are essentially showing the residual correlations of the items after the general factor has been removed. Recall that similar plots were used to describe the fractal nature of the multidimensional personality space in Section \ref{sec:fractal}; refer to Figure \ref{fig:sim1} for a depiction of the hypothetically perfect simple structure of a simulated 696-item space. Figure \ref{fig:spi27qoverlay} shows the 135 items of the SPI-27 scales (in color) amongst the larger set of 696 items (all others are in grey). The shaded ovals are included to help indicate the regions of the multidimensional space which are covered by each of the 5-item scales. Of course, this depiction is somewhat misleading in that it is a 2 dimensional representation of a geometric space with much higher dimensionality. Nevertheless, the SPI-27 appears to cover much of the space without considerable overlap. For comparison purposes, a similar figure is shown for the 300-items of the IPIP-NEO \citep[Figure \ref{fig:IPIPneoAllfacets300overlay},][]{Goldberg:99a}. The 300-item version is both inefficient and confusing; several of the facets are highly overlapping and/or entirely nested inside the regions covered by other facets. In the image for the 27 facet SPI, the large circle at the top of the image is Easy-Goingness. This is also one of the weaker factors and has relatively low unidimensionality. The extent to which this factor overlaps with other factors is unclear based on the two-dimensional representation but it appears to be less well-defined than all the rest. The code needed to draw these plots is somewhat involved; I have tucked it away in the rnw version of the file. \begin{scriptsize} <>= suppressPackageStartupMessages(library(qgraph)) findcolnumber <- function(df, thecolumnname){ which(colnames(df) == thecolumnname) } colname2colnum <- function(x, scaleList) { colnumList <- list() for (i in 1:length(scaleList)) { jlist <- rep(NA, length(scaleList[[i]])) for (j in 1:length(scaleList[[i]])) { jlist[j] <- findcolnumber(x, scaleList[[i]][j]) } colnumList[[length(colnumList)+1]] <- jlist } names(colnumList) <- names(scaleList) rm(i, j) return(colnumList) } ##### # Load the data load(paste(filepathdata, "keysAndScalesInfo.Rdata", sep = "")) ##### # Make lists of items Temperament <- colnames(allSampleCor) NEOscales300 <- list( NEO_A = c("q_286", "q_594", "q_1758", "q_1854", "q_1855", "q_343", "q_377", "q_379", "q_381", "q_1835", "q_501", "q_1214", "q_1467", "q_1543", "q_1556", "q_1752", "q_1765", "q_1894", "q_1896", "q_2016", "q_90", "q_146", "q_298", "q_1041", "q_1354", "q_1385", "q_1420", "q_1421", "q_1774", "q_1881", "q_957", "q_1051", "q_1150", "q_121", "q_479", "q_531", "q_1031", "q_1163", "q_1359", "q_2036", "q_364", "q_401", "q_525", "q_565", "q_586", "q_1042", "q_1235", "q_1416", "q_1691", "q_1830", "q_195", "q_339", "q_383", "q_487", "q_851", "q_1757", "q_1764", "q_1818", "q_1858", "q_1910"), NEO_C = c("q_786", "q_258", "q_515", "q_520", "q_708", "q_722", "q_1023", "q_1113", "q_1215", "q_1435", "q_169", "q_170", "q_619", "q_1254", "q_1255", "q_1290", "q_1333", "q_1374", "q_1483", "q_1917", "q_1867", "q_404", "q_405", "q_618", "q_977", "q_1190", "q_1341", "q_1436", "q_1508", "q_1812", "q_191", "q_602", "q_604", "q_1519", "q_1979", "q_554", "q_1012", "q_1553", "q_1694", "q_1883", "q_76", "q_491", "q_904", "q_962", "q_992", "q_1087", "q_1444", "q_1521", "q_1744", "q_1949", "q_1173", "q_1292", "q_1424", "q_1488", "q_598", "q_35", "q_321", "q_507", "q_1638", "q_1748"), NEO_E = c("q_20", "q_140", "q_313", "q_1196", "q_1410", "q_1943", "q_200", "q_505", "q_819", "q_1480", "q_744", "q_1923", "q_314", "q_684", "q_1169", "q_1370", "q_1376", "q_1531", "q_1666", "q_1803", "q_1114", "q_455", "q_690", "q_1180", "q_1768", "q_1151", "q_1670", "q_1769", "q_1872", "q_1913", "q_71", "q_74", "q_429", "q_595", "q_1280", "q_1281", "q_1329", "q_1330", "q_1599", "q_1600", "q_294", "q_746", "q_1662", "q_1664", "q_2017", "q_31", "q_576", "q_745", "q_1360", "q_1368", "q_1043", "q_1371", "q_1567", "q_175", "q_296", "q_799", "q_1244", "q_1248", "q_1350", "q_1685"), NEO_O = c("q_780", "q_1058", "q_1310", "q_1381", "q_1682", "q_609", "q_1083", "q_1161", "q_1675", "q_1738", "q_610", "q_612", "q_1648", "q_348", "q_607", "q_608", "q_611", "q_776", "q_1289", "q_1369", "q_174", "q_228", "q_721", "q_755", "q_793", "q_797", "q_844", "q_1589", "q_1681", "q_1880", "q_148", "q_47", "q_82", "q_566", "q_581", "q_689", "q_1303", "q_1334", "q_1540", "q_1542", "q_194", "q_197", "q_316", "q_322", "q_422", "q_778", "q_1050", "q_1088", "q_1327", "q_1388", "q_345", "q_359", "q_369", "q_394", "q_398", "q_1328", "q_1824", "q_1825", "q_395", "q_397"), NEO_N = c("q_177", "q_248", "q_727", "q_986", "q_1989", "q_38", "q_66", "q_179", "q_808", "q_960", "q_1571", "q_1585", "q_1683", "q_208", "q_176", "q_952", "q_974", "q_995", "q_1184", "q_1357", "q_875", "q_824", "q_578", "q_820", "q_1099", "q_1677", "q_1044", "q_1479", "q_206", "q_285", "q_63", "q_67", "q_69", "q_89", "q_114", "q_168", "q_184", "q_901", "q_1496", "q_1755", "q_56", "q_632", "q_673", "q_736", "q_1009", "q_1384", "q_1461", "q_1462", "q_1474", "q_1590", "q_1505", "q_85", "q_332", "q_423", "q_469", "q_867", "q_979", "q_1209", "q_1606", "q_1616"), NEOa1 = c("q_286", "q_594", "q_1758", "q_1854", "q_1855", "q_343", "q_377", "q_379", "q_381", "q_1835"), NEOa2 = c("q_501", "q_1214", "q_1467", "q_1543", "q_1556", "q_1752", "q_1765", "q_1894", "q_1896", "q_2016"), NEOa3 = c("q_90", "q_146", "q_298", "q_1041", "q_1354", "q_1385", "q_1420", "q_1421", "q_1774", "q_1881"), NEOa4 = c("q_957", "q_1051", "q_1150", "q_121", "q_479", "q_531", "q_1031", "q_1163", "q_1359", "q_2036"), NEOa5 = c("q_364", "q_401", "q_525", "q_565", "q_586", "q_1042", "q_1235", "q_1416", "q_1691", "q_1830"), NEOa6 = c("q_195", "q_339", "q_383", "q_487", "q_851", "q_1757", "q_1764", "q_1818", "q_1858", "q_1910"), NEOc1 = c("q_786", "q_258", "q_515", "q_520", "q_708", "q_722", "q_1023", "q_1113", "q_1215", "q_1435"), NEOc2 = c("q_169", "q_170", "q_619", "q_1254", "q_1255", "q_1290", "q_1333", "q_1374", "q_1483", "q_1917"), NEOc3 = c("q_1867", "q_404", "q_405", "q_618", "q_977", "q_1190", "q_1341", "q_1436", "q_1508", "q_1812"), NEOc4 = c("q_191", "q_602", "q_604", "q_1519", "q_1979", "q_554", "q_1012", "q_1553", "q_1694", "q_1883"), NEOc5 = c("q_76", "q_491", "q_904", "q_962", "q_992", "q_1087", "q_1444", "q_1521", "q_1744", "q_1949"), NEOc6 = c("q_1173", "q_1292", "q_1424", "q_1488", "q_598", "q_35", "q_321", "q_507", "q_1638", "q_1748"), NEOe1 = c("q_20", "q_140", "q_313", "q_1196", "q_1410", "q_1943", "q_200", "q_505", "q_819", "q_1480"), NEOe2 = c("q_744", "q_1923", "q_314", "q_684", "q_1169", "q_1370", "q_1376", "q_1531", "q_1666", "q_1803"), NEOe3 = c("q_1114", "q_455", "q_690", "q_1180", "q_1768", "q_1151", "q_1670", "q_1769", "q_1872", "q_1913"), NEOe4 = c("q_71", "q_74", "q_429", "q_595", "q_1280", "q_1281", "q_1329", "q_1330", "q_1599", "q_1600"), NEOe5 = c("q_294", "q_746", "q_1662", "q_1664", "q_2017", "q_31", "q_576", "q_745", "q_1360", "q_1368"), NEOe6 = c("q_1043", "q_1371", "q_1567", "q_175", "q_296", "q_799", "q_1244", "q_1248", "q_1350", "q_1685"), NEOn1 = c("q_177", "q_248", "q_727", "q_986", "q_1989", "q_38", "q_66", "q_179", "q_808", "q_960"), NEOn2 = c("q_1571", "q_1585", "q_1683", "q_208", "q_176", "q_952", "q_974", "q_995", "q_1184", "q_1357"), NEOn3 = c("q_875", "q_824", "q_578", "q_820", "q_1099", "q_1677", "q_1044", "q_1479", "q_206", "q_285"), NEOn4 = c("q_63", "q_67", "q_69", "q_89", "q_114", "q_168", "q_184", "q_901", "q_1496", "q_1755"), NEOn5 = c("q_56", "q_632", "q_673", "q_736", "q_1009", "q_1384", "q_1461", "q_1462", "q_1474", "q_1590"), NEOn6 = c("q_1505", "q_85", "q_332", "q_423", "q_469", "q_867", "q_979", "q_1209", "q_1606", "q_1616"), NEOo1 = c("q_780", "q_1058", "q_1310", "q_1381", "q_1682", "q_609", "q_1083", "q_1161", "q_1675", "q_1738"), NEOo2 = c("q_610", "q_612", "q_1648", "q_348", "q_607", "q_608", "q_611", "q_776", "q_1289", "q_1369"), NEOo3 = c("q_174", "q_228", "q_721", "q_755", "q_793", "q_797", "q_844", "q_1589", "q_1681", "q_1880"), NEOo4 = c("q_148", "q_47", "q_82", "q_566", "q_581", "q_689", "q_1303", "q_1334", "q_1540", "q_1542"), NEOo5 = c("q_194", "q_197", "q_316", "q_322", "q_422", "q_778", "q_1050", "q_1088", "q_1327", "q_1388"), NEOo6 = c("q_345", "q_359", "q_369", "q_394", "q_398", "q_1328", "q_1824", "q_1825", "q_395", "q_397")) # Plot all 300 NEO items only NEO300 <- c(NEOscales300$NEO_N, NEOscales300$NEO_E, NEOscales300$NEO_O, NEOscales300$NEO_A, NEOscales300$NEO_C) # Plot all non-NEO items TempLessNEO300 <- setdiff(Temperament, NEO300) # Plot 696 items colored by NEO or non NEO PersonalityNEOall300 <- list(IPIP_NEO_300 = NEO300, Other = TempLessNEO300) # Plot 696 items colored by NEO domain or non NEO PersonalityNEOdomains300 <- list(Neuroticism = NEOscales300$NEO_N, Extraversion = NEOscales300$NEO_E, Openness = NEOscales300$NEO_O, Agreeableness = NEOscales300$NEO_A, Conscientiousness = NEOscales300$NEO_C, Other = TempLessNEO300) # Plot 696 items colored by NEO facet or non NEO PersonalityNEOfacets300 <- list(Anxiety = NEOscales300$NEOn1, Anger = NEOscales300$NEOn2, Depression = NEOscales300$NEOn3, SelfConsciousness = NEOscales300$NEOn4, Immoderation = NEOscales300$NEOn5, Vulnerability = NEOscales300$NEOn6, Friendliness = NEOscales300$NEOe1, Gregariousness = NEOscales300$NEOe2, Assertiveness = NEOscales300$NEOe3, ActivityLevel = NEOscales300$NEOe4, ExcitementSeeking = NEOscales300$NEOe5, Cheerfulness = NEOscales300$NEOe6, Imagination = NEOscales300$NEOo1, ArtisticInterests = NEOscales300$NEOo2, Emotionality = NEOscales300$NEOo3, Adventurousness = NEOscales300$NEOo4, Intellect = NEOscales300$NEOo5, Liberalism = NEOscales300$NEOo6, Trust = NEOscales300$NEOa1, Morality = NEOscales300$NEOa2, Altruism = NEOscales300$NEOa3, Cooperation = NEOscales300$NEOa4, Modesty = NEOscales300$NEOa5, Sympathy = NEOscales300$NEOa6, SelfEfficacy = NEOscales300$NEOc1, Orderliness = NEOscales300$NEOc2, Dutifulness = NEOscales300$NEOc3, AchievementStriving = NEOscales300$NEOc4, SelfDiscipline = NEOscales300$NEOc5, Cautiousness = NEOscales300$NEOc6, Other = TempLessNEO300) PersonalityNEOfacetsLabels = c("Anxiety", "Anger", "Depression", "Self-Consciousness", "Immoderation", "Vulnerability", "Friendliness", "Gregariousness", "Assertiveness", "Activity Level", "Excitement-Seeking", "Cheerfulness", "Imagination", "Artistic Interests", "Emotionality", "Adventurousness", "Intellect", "Liberalism", "Trust", "Morality", "Altruism", "Cooperation", "Modesty", "Sympathy", "Self-Efficacy", "Orderliness", "Dutifulness", "Achievement-Striving", "Self-Discipline", "Cautiousness", "Other") # Plot 696 items colored by NEO Neuro facet or not (other NEO and non NEO combined) TempNEOlessNeuro300 <- c(TempLessNEO300, setdiff(NEO300, NEOscales300$NEO_N)) PersonalityNEOneuro300 <- list(Anxiety = NEOscales300$NEOn1, Anger = NEOscales300$NEOn2, Depression = NEOscales300$NEOn3, SelfConsciousness = NEOscales300$NEOn4, Immoderation = NEOscales300$NEOn5, Vulnerability = NEOscales300$NEOn6, Other = TempNEOlessNeuro300) PersonalityNEOneuroLabels = c("Anxiety", "Anger", "Depression", "Self-Consciousness", "Immoderation", "Vulnerability", "Other") # Plot 696 items colored by NEO Extra facet or not (other NEO and non NEO combined) TempNEOlessExtra300 <- c(TempLessNEO300, setdiff(NEO300, NEOscales300$NEO_E)) PersonalityNEOextra300 <- list(Friendliness = NEOscales300$NEOe1, Gregariousness = NEOscales300$NEOe2, Assertiveness = NEOscales300$NEOe3, ActivityLevel = NEOscales300$NEOe4, ExcitementSeeking = NEOscales300$NEOe5, Cheerfulness = NEOscales300$NEOe6, Other = TempNEOlessExtra300) PersonalityNEOextraLabels = c("Friendliness", "Gregariousness", "Assertiveness", "Activity Level", "Excitement-Seeking", "Cheerfulness", "Other") # Plot 696 items colored by NEO Open facet or not (other NEO and non NEO combined) TempNEOlessOpen300 <- c(TempLessNEO300, setdiff(NEO300, NEOscales300$NEO_O)) PersonalityNEOopen300 <- list(Imagination = NEOscales300$NEOo1, ArtisticInterests = NEOscales300$NEOo2, Emotionality = NEOscales300$NEOo3, Adventurousness = NEOscales300$NEOo4, Intellect = NEOscales300$NEOo5, Liberalism = NEOscales300$NEOo6, Other = TempNEOlessOpen300) PersonalityNEOopenLabels = c("Imagination", "Artistic Interests", "Emotionality", "Adventurousness", "Intellect", "Liberalism", "Other") # Plot 696 items colored by NEO Agree facet or not (other NEO and non NEO combined) TempNEOlessAgree300 <- c(TempLessNEO300, setdiff(NEO300, NEOscales300$NEO_A)) PersonalityNEOagree300 <- list(Trust = NEOscales300$NEOa1, Morality = NEOscales300$NEOa2, Altruism = NEOscales300$NEOa3, Cooperation = NEOscales300$NEOa4, Modesty = NEOscales300$NEOa5, Sympathy = NEOscales300$NEOa6, Other = TempNEOlessAgree300) PersonalityNEOagreeLabels = c("Trust", "Morality", "Altruism", "Cooperation", "Modesty", "Sympathy", "Other") # Plot 696 items colored by NEO Consc facet or not (other NEO and non NEO combined) TempNEOlessConsc300 <- c(TempLessNEO300, setdiff(NEO300, NEOscales300$NEO_C)) PersonalityNEOconsc300 <- list(SelfEfficacy = NEOscales300$NEOc1, Orderliness = NEOscales300$NEOc2, Dutifulness = NEOscales300$NEOc3, AchievementStriving = NEOscales300$NEOc4, SelfDiscipline = NEOscales300$NEOc5, Cautiousness = NEOscales300$NEOc6, Other = TempNEOlessConsc300) PersonalityNEOconscLabels = c("Self-Efficacy", "Orderliness", "Dutifulness", "Achievement-Striving", "Self-Discipline", "Cautiousness", "Other") ##### # Repeat the above for the SPI items # Insert the crowd-sourced names SPIscales <- list( Extraversion = keys.list5$items$Extraversion, Neuroticism = keys.list5$items$Neuroticism, Conscientiousness = keys.list5$items$Conscientiousness, Agreeableness = keys.list5$items$Agreeableness, Openness = keys.list5$items$Openness, Compassion = keys.list27$items$Compassion, Irritability = keys.list27$items$Irritability, Sociability = keys.list27$items$Sociability, WellBeing = keys.list27$items$WellBeing, SensationSeeking = keys.list27$items$SensationSeeking, Anxiety = keys.list27$items$Anxiety, Honesty = keys.list27$items$Honesty, Industry = keys.list27$items$Industry, Intellect = keys.list27$items$Intellect, Creativity = keys.list27$items$Creativity, Impulsivity = keys.list27$items$Impulsivity, AttentionSeeking = keys.list27$items$AttentionSeeking, Order = keys.list27$items$Order, Authoritarianism = keys.list27$items$Authoritarianism, Charisma = keys.list27$items$Charisma, Trust = keys.list27$items$Trust, Humor = keys.list27$items$Humor, EmotionalExpressiveness = keys.list27$items$EmotionalExpressiveness, ArtAppreciation = keys.list27$items$ArtAppreciation, Introspection = keys.list27$items$Introspection, Perfectionism = keys.list27$items$Perfectionism, SelfControl = keys.list27$items$SelfControl, Conformity = keys.list27$items$Conformity, Adaptability = keys.list27$items$Adaptability, EasyGoingness = keys.list27$items$EasyGoingness, EmotionalStability = keys.list27$items$EmotionalStability, Conservatism = keys.list27$items$Conservatism) # Plot all 135 SPI items only SPI135 <- c(Compassion = keys.list27$items$Compassion, Irritability = keys.list27$items$Irritability, Sociability = keys.list27$items$Sociability, WellBeing = keys.list27$items$WellBeing, SensationSeeking = keys.list27$items$SensationSeeking, Anxiety = keys.list27$items$Anxiety, Honesty = keys.list27$items$Honesty, Industry = keys.list27$items$Industry, Intellect = keys.list27$items$Intellect, Creativity = keys.list27$items$Creativity, Impulsivity = keys.list27$items$Impulsivity, AttentionSeeking = keys.list27$items$AttentionSeeking, Order = keys.list27$items$Order, Authoritarianism = keys.list27$items$Authoritarianism, Charisma = keys.list27$items$Charisma, Trust = keys.list27$items$Trust, Humor = keys.list27$items$Humor, EmotionalExpressiveness = keys.list27$items$EmotionalExpressiveness, ArtAppreciation = keys.list27$items$ArtAppreciation, Introspection = keys.list27$items$Introspection, Perfectionism = keys.list27$items$Perfectionism, SelfControl = keys.list27$items$SelfControl, Conformity = keys.list27$items$Conformity, Adaptability = keys.list27$items$Adaptability, EasyGoingness = keys.list27$items$EasyGoingness, EmotionalStability = keys.list27$items$EmotionalStability, Conservatism = keys.list27$items$Conservatism) # Plot all non-SPI items TempLessSPI135 <- setdiff(Temperament, SPI135) # Plot 696 items colored by SPI or non SPI PersonalitySPIall135 <- list(SPI_135 = SPI135, Other = TempLessSPI135) # Plot 696 items colored by SPI domain or non SPI # Insert the crowd-sourced names PersonalitySPI5 <- list(Neuroticism = SPIscales$Neuroticism, Extraversion = SPIscales$Extraversion, Openness = SPIscales$Openness, Agreeableness = SPIscales$Agreeableness, Conscientiousness = SPIscales$Conscientiousness, Other = TempLessSPI135) # Plot 696 items colored by SPI facet or non SPI # Insert the crowd-sourced names PersonalitySPIfacets135 <- list(Intellect = keys.list27$items$Intellect, Adaptability = keys.list27$items$Adaptability, Creativity = keys.list27$items$Creativity, Introspection = keys.list27$items$Introspection, ArtAppreciation = keys.list27$items$ArtAppreciation, Conformity = keys.list27$items$Conformity, Humor = keys.list27$items$Humor, EmotionalExpressiveness = keys.list27$items$EmotionalExpressiveness, Sociability = keys.list27$items$Sociability, Charisma = keys.list27$items$Charisma, SensationSeeking = keys.list27$items$SensationSeeking, AttentionSeeking = keys.list27$items$AttentionSeeking, Trust = keys.list27$items$Trust, Compassion = keys.list27$items$Compassion, Conservatism = keys.list27$items$Compassion, Honesty = keys.list27$items$Honesty, Authoritarianism = keys.list27$items$Authoritarianism, Impulsivity = keys.list27$items$Impulsivity, EmotionalStability = keys.list27$items$EmotionalStability, WellBeing = keys.list27$items$WellBeing, Industry = keys.list27$items$Industry, Irritability = keys.list27$items$Irritability, Order = keys.list27$items$Order, Perfectionism = keys.list27$items$Perfectionism, SelfControl = keys.list27$items$SelfControl, Anxiety = keys.list27$items$Anxiety, EasyGoingness = keys.list27$items$EasyGoingness, Other = TempLessSPI135) ##### # Set some plotting options suppressPackageStartupMessages(library(RColorBrewer)) shortcolors = c(brewer.pal(6, "Set1")) longcolors = c(rainbow(30)) ##### # For testing formatting #test <- allSampleCor[c(1:20),c(1:20)] #title = "Test Graph" #filename = paste(filepathfig, "test", sep = "") #groups <- list(fake1 = c(1:5), fake2 = c(6:10), fake3 = c(11:15), fake4 = c(16:20)) #colors = shortcolors[1:length(groups)] #testGraph <- qgraph(test, minimum=0.3, groups=groups, colors = colors, labels = FALSE, title = title, layout="spring", edge.color = "grey50", filetype = "pdf", filename = filename) ##### # Make graphs #title = "IPIP-NEO (300-item) Neuroticism facets" title = "" filename = paste(filepathfig, "IPIPneoNeuro300", sep = "") groups <- colname2colnum(allSampleCor, PersonalityNEOneuro300) colors = c(shortcolors[1:(length(groups)-1)], "grey60") labels = PersonalityNEOneuroLabels pGraph <- qgraph(allSampleCor, minimum=0.3, cut=.6, groups=groups, colors = colors, labels = TRUE, title = title, layout="spring", legend.cex=.5, edge.color = "grey50", filetype = "pdf", filename = filename, GLratio = 3, borders=FALSE) #title = "IPIP-NEO (300-item) Extraversion facets" title = "" filename = paste(filepathfig, "IPIPneoExtra300", sep = "") groups <- colname2colnum(allSampleCor, PersonalityNEOextra300) colors = c(shortcolors[1:(length(groups)-1)], "grey60") labels = PersonalityNEOextraLabels pGraph <- qgraph(allSampleCor, minimum=0.3, cut=.6, groups=groups, colors = colors, labels = TRUE, title = title, layout="spring", legend.cex=.5, edge.color = "grey50", filetype = "pdf", filename = filename, GLratio = 3, borders=FALSE) #title = "IPIP-NEO (300-item) Openness facets" title = "" filename = paste(filepathfig, "IPIPneoOpen300", sep = "") groups <- colname2colnum(allSampleCor, PersonalityNEOopen300) colors = c(shortcolors[1:(length(groups)-1)], "grey60") labels = PersonalityNEOopenLabels pGraph <- qgraph(allSampleCor, minimum=0.3, cut=.6, groups=groups, colors = colors, labels = TRUE, title = title, layout="spring", legend.cex=.5, edge.color = "grey50", filetype = "pdf", filename = filename, GLratio = 3, borders=FALSE) #title = "IPIP-NEO (300-item) Agreeableness facets" title = "" filename = paste(filepathfig, "IPIPneoAgree300", sep = "") groups <- colname2colnum(allSampleCor, PersonalityNEOagree300) colors = c(shortcolors[1:(length(groups)-1)], "grey60") labels = PersonalityNEOagreeLabels pGraph <- qgraph(allSampleCor, minimum=0.3, cut=.6, groups=groups, colors = colors, labels = TRUE, title = title, layout="spring", legend.cex=.5, edge.color = "grey50", filetype = "pdf", filename = filename, GLratio = 3, borders=FALSE) #title = "IPIP-NEO (300-item) Conscientiousness facets" title = "" filename = paste(filepathfig, "IPIPneoConsc300", sep = "") groups <- colname2colnum(allSampleCor, PersonalityNEOconsc300) colors = c(shortcolors[1:(length(groups)-1)], "grey60") labels = PersonalityNEOconscLabels pGraph <- qgraph(allSampleCor, minimum=0.3, cut=.6, groups=groups, colors = colors, labels = TRUE, title = title, layout="spring", legend.cex=.5, edge.color = "grey50", filetype = "pdf", filename = filename, GLratio = 3, borders=FALSE) #title = "IPIP-NEO (300-item) all facets with overlay" title = "" filename = paste(filepathfig, "IPIPneoAllfacets300overlay", sep = "") groups <- colname2colnum(allSampleCor, PersonalityNEOfacets300) colors = c(longcolors[1:(length(groups)-1)], "grey60") pGraph <- qgraph(allSampleCor, minimum=0.3, cut=.6, groups=groups, colors = colors, labels = TRUE, title = title, layout="spring", legend.cex=.5, edge.color = "grey50", filetype = "pdf", filename = filename, GLratio = 3, borders=FALSE, overlay=TRUE) #title = "IPIP-NEO (300-item) all facets" title = "" filename = paste(filepathfig, "IPIPneoAllfacets300", sep = "") groups <- colname2colnum(allSampleCor, PersonalityNEOfacets300) colors = c(longcolors[1:(length(groups)-1)], "grey60") pGraph <- qgraph(allSampleCor, minimum=0.3, cut=.6, groups=groups, colors = colors, labels = TRUE, title = title, layout="spring", legend.cex=.5, edge.color = "grey50", filetype = "pdf", filename = filename, GLratio = 3, borders=FALSE, overlay=FALSE) #title = "IPIP-NEO (300-item) Domains with overlay" title = "" filename = paste(filepathfig, "IPIPneoDomains300overlay", sep = "") groups <- colname2colnum(allSampleCor, PersonalityNEOdomains300) colors = c(shortcolors[1:(length(groups)-1)], "grey60") pGraph <- qgraph(allSampleCor, minimum=0.3, cut=.6, groups=groups, colors = colors, labels = TRUE, title = title, layout="spring", legend.cex=.5, edge.color = "grey50", filetype = "pdf", filename = filename, GLratio = 3, borders=FALSE, overlay=TRUE) #title = "IPIP-NEO (300-item) Domains" title = "" filename = paste(filepathfig, "IPIPneoDomains300", sep = "") groups <- colname2colnum(allSampleCor, PersonalityNEOdomains300) colors = c(shortcolors[1:(length(groups)-1)], "grey60") pGraph <- qgraph(allSampleCor, minimum=0.3, cut=.6, groups=groups, colors = colors, labels = TRUE, title = title, layout="spring", legend.cex=.5, edge.color = "grey50", filetype = "pdf", filename = filename, GLratio = 3, borders=FALSE, overlay=FALSE) #title = "IPIP-NEO (300-item) Items" title = "" filename = paste(filepathfig, "IPIPneo300All", sep = "") groups <- colname2colnum(allSampleCor, PersonalityNEOall300) colors = c(shortcolors[1:(length(groups)-1)], "grey60") pGraph <- qgraph(allSampleCor, minimum=0.3, cut=.6, groups=groups, colors = colors, labels = TRUE, title = title, layout="spring", legend.cex=.5, edge.color = "grey50", filetype = "pdf", filename = filename, GLratio = 3, borders=FALSE) ##### # SPI135 #title = "SPI27 (135-item) all 27 factors" title = "" filename = paste(filepathfig, "SPI27Allfacets135labeled", sep = "") groups <- colname2colnum(allSampleCor, PersonalitySPIfacets135) colors = c(longcolors[1:(length(groups)-1)], "grey60") labels = colnames(allSampleCor) pGraph <- qgraph(allSampleCor, minimum=0.3, cut=.6, groups=groups, colors = colors, labels = labels, title = title, layout="spring", legend.cex=.5, edge.color = "grey50", filetype = "pdf", filename = filename, GLratio = 3, borders=FALSE) #title = "SPI27 (135-item) all 27 factors" title = "" filename = paste(filepathfig, "SPI27Allfacets135unlabeled", sep = "") groups <- colname2colnum(allSampleCor, PersonalitySPIfacets135) colors = c(longcolors[1:(length(groups)-1)], "grey60") labels = colnames(allSampleCor) pGraph <- qgraph(allSampleCor, minimum=0.3, cut=.6, groups=groups, colors = colors, labels = FALSE, legend = FALSE, title = title, layout="spring", legend.cex=.5, edge.color = "grey50", filetype = "pdf", filename = filename, GLratio = 3, borders=FALSE, overlay=FALSE) #title = "SPI27 (135-item) all 27 factors with overlay" title = "" filename = paste(filepathfig, "SPI27Allfacets135overlay", sep = "") groups <- colname2colnum(allSampleCor, PersonalitySPIfacets135) colors = c(longcolors[1:(length(groups)-1)], "grey60") labels = colnames(allSampleCor) pGraph <- qgraph(allSampleCor, minimum=0.3, cut=.6, groups=groups, colors = colors, labels = labels, title = title, layout="spring", legend.cex=.5, edge.color = "grey50", filetype = "pdf", filename = filename, GLratio = 3, borders=FALSE, overlay=TRUE) #title = "SPI (135-item) Items" title = "" filename = paste(filepathfig, "SPIall135", sep = "") groups <- colname2colnum(allSampleCor, PersonalitySPIall135) colors = c(shortcolors[1:(length(groups)-1)], "grey60") pGraph <- qgraph(allSampleCor, minimum=0.3, cut=.6, groups=groups, colors = colors, labels = TRUE, title = title, layout="spring", legend.cex=.5, edge.color = "grey50", filetype = "pdf", filename = filename, GLratio = 3, borders=FALSE) @ \end{scriptsize} \begin{figure} \ffigbox{\includegraphics[width=.95\textwidth]{figures/SPI27Allfacets135overlay.pdf}}{\caption{Network graph for the 135 items and 27 facets of the SPI}\label{fig:spi27qoverlay}} \end{figure} \begin{figure} \ffigbox{\includegraphics[width=.95\textwidth]{figures/IPIPneoAllfacets300overlay.pdf}}{\caption{Network graph for the 300 items and 30 facets of the IPIP-NEO-300}\label{fig:IPIPneoAllfacets300overlay}} \end{figure} \subsection{Age and gender norms for the SPI items} Age and gender specific norms for the SPI items are provided in Table \ref{tab:MeansSDs}. These norms are based on the data in all three samples combined. As discussed in Section \ref{sec:samples}, these data should not be considered as wholly representative of any population other than English-speaking individuals who were willing to complete a web survey about personality in 2014 or 2015. It is trivial to identify the norms (or, if desired, the full distribution of responses) for each item at each year of age, but this is an overwhelming amount of output (Table \ref{tab:MeansSDs} only shows age bands of about 10 years). To get more narrow age bands, simply alter the code below by changing the lines which collapse the age categories (using `mapvalues'). In Appendix \hyperref[sec:appendixC]{C}, we show the scale means and sds for the SPI-27 and SPI-5 scales (rather than the item means and sds) in Tables \ref{tab:MeansSDsByScale27} and \ref{tab:MeansSDsByScale5}, respectively. Plots of the scores by age and gender are given for all ages from 14 to 90 in Figures \ref{fig:scale1} through \ref{fig:scale32}. \begin{scriptsize} <>= load(paste(filepathdata, "keysAndScalesInfo.Rdata", sep = "")) SPI27items <- as.vector(unlist(keys.list27$items)) SPI27content <- ItemInfo696[SPI27items,] normsData <- allSample[,c("age", "gender", SPI27items)] normsData$ageGroup <- mapvalues(normsData$age, from = c(14:18), to = rep(16, 5)) normsData$ageGroup <- mapvalues(normsData$ageGroup, from = c(19:25), to = rep(22, 7)) normsData$ageGroup <- mapvalues(normsData$ageGroup, from = c(26:35), to = rep(30, 10)) normsData$ageGroup <- mapvalues(normsData$ageGroup, from = c(36:45), to = rep(40, 10)) normsData$ageGroup <- mapvalues(normsData$ageGroup, from = c(46:55), to = rep(50, 10)) normsData$ageGroup <- mapvalues(normsData$ageGroup, from = c(56:65), to = rep(60, 10)) normsData$ageGroup <- mapvalues(normsData$ageGroup, from = c(66:90), to = rep(77, 25)) normsData$gender <- as.numeric(normsData[,"gender"]) normsStatsByItem <- statsBy(subset(normsData, select = -c(age)), c("gender", "ageGroup"), cor=TRUE) Means <- round(subset(normsStatsByItem$mean, select = -c(gender, ageGroup)), 2) rownames(Means) <- c("14-18 M", "14-18 F", "19-25 M", "19-25 F", "26-35 M", "26-35 F", "36-45 M", "36-45 F", "46-55 M", "46-55 F", "56-65 M", "56-65 F", "66+ M", "66+ F") Means <- t(Means) SDs <- round(subset(normsStatsByItem$sd, select = -c(gender, ageGroup)), 2) rownames(SDs) <- c("14-18 M", "14-18 F", "19-25 M", "19-25 F", "26-35 M", "26-35 F", "36-45 M", "36-45 F", "46-55 M", "46-55 F", "56-65 M", "56-65 F", "66+ M", "66+ F") SDs <- t(SDs) MeansSDs <- data.frame(matrix(ncol=2*ncol(Means), nrow = nrow(Means))) colsMeansSDs <- list() for (i in 1:ncol(Means)) { MeansSDs[,(2*i-1)] <- Means[,i] MeansSDs[,(2*i)] <- SDs[,i] } MeansSDs <- data.frame(SPI27content[,"Item"], MeansSDs) colnames(MeansSDs) <- c("Item", "14-18M_M", "14-18M_sd", "14-18F_M", "14-18F_sd", "19-25M_M", "19-25M_sd", "19-25F_M", "19-25F_sd", "26-35M_M", "26-35M_sd", "26-35F_M", "26-35F_sd", "36-45M_M", "36-45M_sd", "36-45F_M", "36-45F_sd", "46-55M_M", "46-55M_sd", "46-55F_M", "46-55F_sd", "56-65M_M", "56-65M_sd", "56-65F_M", "56-65F_sd", "66+M_M", "66+M_sd", "66+F_M", "66+F_sd") rownames(MeansSDs) <- rownames(SPI27content) @ \end{scriptsize} \clearpage \begin{tiny} \newgeometry{left=2.54cm,right=2.54cm, bottom=2.54cm, top=2.54cm} \begin{landscape} \pagestyle{empty} \setlength\LTcapwidth{\textwidth} \setlength\LTleft{0pt} \setlength\LTright{0pt} <>= suppressPackageStartupMessages(library(xtable)) addtorow <- list(pos = list(0), command = NULL) command <- paste0(" & & \\multicolumn{4}{c}{14 --- 18 yrs old} & \\multicolumn{4}{c}{19 --- 25 yrs old} & \\multicolumn{4}{c}{26 --- 35 yrs old} & \\multicolumn{4}{c}{36 --- 45 yrs old} & \\multicolumn{4}{c}{46 --- 55 yrs old} & \\multicolumn{4}{c}{56 --- 65 yrs old} & \\multicolumn{4}{c}{66 yrs old and up} \\cr", " & & \\multicolumn{2}{c}{Male} & \\multicolumn{2}{c}{Female} & \\multicolumn{2}{c}{Male} & \\multicolumn{2}{c}{Female} & \\multicolumn{2}{c}{Male} & \\multicolumn{2}{c}{Female} & \\multicolumn{2}{c}{Male} & \\multicolumn{2}{c}{Female} & \\multicolumn{2}{c}{Male} & \\multicolumn{2}{c}{Female} & \\multicolumn{2}{c}{Male} & \\multicolumn{2}{c}{Female} & \\multicolumn{2}{c}{Male} & \\multicolumn{2}{c}{Female} \\cr", " & & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} \\cr", "\\hline\n\\endhead\n", "\\hline\n", "\\multicolumn{30}{c}", "{\\textit{(continued on next page)}}\n", "\\endfoot\n", "\\endlastfoot\n") addtorow$command <- command print(xtable(MeansSDs, digits = 2, caption='SPI item means and standard deviations by age range and gender', label='tab:MeansSDs', align = c("l", ">{\\hangindent=1em}p{4.5cm}", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r")), size='tiny', caption.placement = "top", tabular.environment = "longtable", include.colnames = FALSE, floating = FALSE, add.to.row = addtorow) @ \end{landscape} \restoregeometry \end{tiny} \subsection{Item Response Theory-based parameters} The following section of code derives the IRT parameters for each of the 27 scales in the SPI-27 using the two parameter logistic model. IRT parameters are not derived for the SPI-5 because the use of such parameters is not supported for all 5 factors based on the unidimensionality estimates (see Table \ref{tab:SPI_5_ICs}). After considerable reflection, I decided to derive the parameters for all of the 27 factor scales because it would be very confusing to users if they were omitted for the two factors with inadequate unidimensionality (see Section \ref{sec:ic}). I think future research with these scales will likely lead in one of two directions. Either these scales (or any of the others) will not prove to have much predictive utility or the unidimensionality of these factors will be strengthened by future analyses of personality structure. Basically, I think we should assume these factors are ``real'' until we have more information but remain skeptical about their utility. The best uses of the IRT parameters for the SPI-27 items (note the unidimensionality in Table \ref{tab:SPI_27_ICs}) would be to identify Differential Item Functioning and/or for computer adaptive assessment (aka computer adaptive testing or CATs). Recommendations for such procedures extend beyond the scope of this paper, but it seems worth noting that it would be possible to administer CATs that provide concise and reliable scores on all 27 scales with about 81-108 items. Since most of these items are also valid for scoring on the SPI-5 scales, this methodology would simultaneously provide scores on that level as well. \begin{scriptsize} <>= rm(list=setdiff(ls(), c("filepathdata", "filepathfig"))) load(paste(filepathdata, "keysAndScalesInfo.Rdata", sep="")) # Get the IRT parameters for all of the SPI27 scales irt.fa.loop <- function (x, y, nscales=c(1:length(y))) { # Loops through all of the item lists in y # x is the raw data frame used to derive the parameters # nscales is a vector of the numbers of the lists in y to be run IRTlist <- list() for (i in nscales) { IRT <- irt.fa(subset(x, select = c(y[[i]]))) IRTlist[[i]] <- IRT } names(IRTlist) <- names(y)[nscales] results <- IRTlist return(results) } IRToutputSPI27 <- irt.fa.loop(allSample, keys.list27$items) for (i in 1:27) { calTable <- cbind(IRToutputSPI27[i][[1]]$irt[2][[1]], IRToutputSPI27[i][[1]]$irt[1][[1]][[1]]) colnames(calTable) <- c("slope", "thresh1", "thresh2", "thresh3", "thresh4", "thresh5") assign(paste("calTable", i, sep=""), round(calTable, 2)) rm(calTable) } calTableCombined <- rbind(calTable1, calTable2, calTable3, calTable4, calTable5, calTable6, calTable7, calTable8, calTable9, calTable10, calTable11, calTable12, calTable13, calTable14, calTable15, calTable16, calTable17, calTable18, calTable19, calTable20, calTable21, calTable22, calTable23, calTable24, calTable25, calTable26, calTable27) save(allSample, calTable1, calTable2, calTable3, calTable4, calTable5, calTable6, calTable7, calTable8, calTable9, calTable10, calTable11, calTable12, calTable13, calTable14, calTable15, calTable16, calTable17, calTable18, calTable19, calTable20, calTable21, calTable22, calTable23, calTable24, calTable25, calTable26, calTable27, calTableCombined, IRToutputSPI27, keys.list27, file = paste(filepathdata, "IRToutputSPI27.Rdata", sep="")) @ \end{scriptsize} Now that I have derived the IRT parameters, I can plot the Item Information Functions and the Test Information Functions for each of the SPI-27 scales. These are shown by factor in Appendix \hyperref[sec:appendixD]{D} (Figures \ref{fig:SPI27_1IIF} through \ref{fig:SPI27_27TIF}). Appendix \hyperref[sec:appendixD]{D} also includes tables (\ref{tab:SPI_27cals1} through \ref{tab:SPI_27cals27}) showing the discrimination and difficulty thresholds for each of the 27 scales. \section{Topics for further research} \label{sec:future} \href{http://youtu.be/QRJ38y4Jn6k}{Are you still here? It's over.} I can't believe you made it this far. The only explanation is that you're an incurable insomniac with a strong interest in personality structure. Well, the bad news (depending on your perspective) is that there's not much more to say at this point. All I can offer you are some ideas for getting down to \href{https://youtu.be/559eWB93jW4}{work} yourself. My first suggestion is to evaluate the extent to which the results reported here are replicable. As I have already pointed out, several of the 27 factors are weak/moderate in strength. It'd be good to understand why. Equally important is the issue of predictive utility. I expect different facets of personality to be predictive of many important outcomes in many domains --- health outcomes, economic/financial outcomes, education, employment, social, and relationship outcomes. But I don't expect that all differences are predictive of outcomes that matter to most people. Given the inefficiency of measuring irrelevant differences, it would be very useful to determine whether each of the 27 predict anything important. This leads directly to the topic of differential predictive utility across groups. This includes grouping by one or more variables such as age, gender, SES, cognitive ability, educational attainment level, health status and many more. DIF analyses (based on IRT scoring) are also needed to ascertain whether the scales derived here perform consistently across groups or if there evidence for bias. Not all DIF leads to bias and there are many methods for addressing DIF at the scale and item level. To the extent that differences across groups can be substantiated, it would be useful to consider the mechanisms underlying these differences. This might be pursued by considering item level differences or by considering the contribution of external factors. The latter case might include social roles, biological factors, and situations. Just one example of the former can be found in the last row of Table \ref{tab:orgtable} --- women of all age groups endorse ``like to stand during the national anthem'' at significantly higher rates than men. Why? There are many similar differences to be considered in the remaining 695 items. I intend to focus on these topics in my own research because I think item-level analyses are generally under-utilized in personality assessment, but there's a ton of work to do here --- have at it. There are also several practical avenues for further research. The data described here could be used for linking analyses that would allow for the development of equivalence scores across closely related measures of the Big Five and Big Six. This contribution would enable more direct translation of previously reported findings that are based on different measures and would \textit{potentially} allow for immediate extension of the nomological network for the SPI27 scales. Another means of evaluating the generalizability of these measures (or, more precisely, the structure they represent) would be to administer the items in many different languages and modes (including those which may not require reading). This will first require translation of the items into as many languages as possible. I would advocate prioritization of the 135 items in the 27 factor scales but many more should be translated in order to enable EFA replications. Unfortunately, high quality translation work requires considerable resources. The gold standard (or a very good one) in translation methodology \citep[FACIT,][]{Eremenco:2005ks} uses a multi-step process to maximize equivalence in both meaning and measurement. The steps include (1) two independent forward translations performed simultaneously by separate individuals, preferably by persons with slightly different nationalities and/or bilingual dialect; (2) reconciliation of the original translations by a third person who is a native speaker of the target language; (3) back-translation by one or more native (bilingual) speaker(s); (4) quality control to ensure that the back-translation closely matches the source document; and (5) independent review by three or four bilingual experts. While the the prospect of translating hundreds (or possibly thousands) of items into multiple languages is daunting, many IPIP items have already been translated; see \href{http://ipip.ori.org/newItemTranslations.htm}{http://ipip.ori.org/newItemTranslations.htm} for more information. Evaluations of structure would also benefit from several advanced psychometric analyses. As previously discussed, IRT-based analyses of differential item functioning (DIF) are a priority to determine the extent to which various items perform inconsistently across variously-defined groups; revision of the scales may be needed if differential functioning suggests biased scales. IRT-based scoring also facilitates the development of computer adaptive testing procedures, though this is as much of a technological challenge as it is psychometric. The benefit of CAT assessment is a reduction in administration time without loss of precision. It would also be insightful to consider the longitudinal invariance of this proposed structure. While I have focused on the inclusion of items that are structurally stable across samples, I have no information regarding stability over time. Using multiple timepoints of longitudinal data collection and item-level change scores, it is possible to fit a measurement model of derivatives \citep{Estabrook:2015bu} that enables evaluation of within and between person change. Differing magnitudes and directions of change among items over time should be evaluated as predictors of important outcomes and, possibly, for evidence that the scales could be improved.\footnote{I say ``possibly'' because it's not clear to me if or how much items should be invariant over time.} From my perspective, the most exciting avenue for further research involves the continued application of the methods used here. This will include administration of the remaining IPIP items to evaluate their relationship to the 696 administered here. A likely outcome is that the many of the remaining IPIP items can be ``located'' among the 27 factor scales using factor extension procedures. But, these data may also lead to the replacement or elimination of existing factors or even the identification of new ones. More importantly, I look forward to using these methods to integrate these items with other frameworks for assessing individual differences. These include domains like cognitive abilities, interests (including hobbies and pasttimes), and values as well as important demographic/psychographic characteristics. The predictive utility for each of these domains of measurement is supported by a long history of research and validation (see \citep{Condon:14Diss} for more details about the prospects for integration in differential/personality psychology). Unfortunately, very few attempts have previously been made to integrate these domains; the extent to which they overlap or are incrementally valid is not clear. I view this as the biggest opportunity in personality assessment as it suggests to me that our current tools could be improved --- either made more efficient or more predictive --- by use of SAPA-like methods for evaluating them holistically. Of course, I am happy to provide more or more specific feedback about these suggestions for further research just as I hope that many people will reach out with suggestions of their own. I truly believe that the most efficient path forward for personality assessment as a field is through collaboration. While I recognize the importance of maintaining a diversity of data collection methods, it seems to me that the best way to evaluate personality structure is with very large international samples. This might be achieved if the personality research community collectively directed as much traffic as possible to a collaboratively developed and maintained administration portal (in my very biased opinion, \href{SAPA-Project.org}{SAPA-Project.org} would be a good model but I'd be pleased just to be involved). Under such a collaboration, all data \textit{and measures} would be made \href{https://youtu.be/9l0LZAXLP9U}{publicly available}, publications would include anyone who contributes time or resources to the project, and the constructs under study at any point in time would be targeted based on input from a steering committee. Who knows? Someday, maybe. In the meantime, we will continue to do what we can with the resources we've got. Thank you for reading all this --- I look forward to your feedback and your \href{https://youtu.be/55zAxY-Ad5o}{iterative contribution}. \clearpage \begin{thebibliography}{} \bibitem[Anderson et~al., 2007]{Anderson:07} Anderson, J.~O., Lin, H.-S., Treagust, D.~F., Ross, S.~P., and Yore, L.~D. (2007). \newblock Using large-scale assessment datasets for research in science and mathematics education: {P}rogramme for {I}nternational {S}tudent {A}ssessment ({PISA}). \newblock {\em International Journal of Science and Mathematics Education}, 5(4):591--614. \bibitem[Ashton et~al., 2007]{Ashton:07b} Ashton, M.~C., Lee, K., and Goldberg, L.~R. (2007). \newblock {The IPIP{\textendash}HEXACO scales: An alternative, public-domain measure of the personality constructs in the HEXACO model}. \newblock {\em Personality and Individual Differences}, 42(8):1515--1526. \bibitem[Ashton et~al., 2004]{Ashton:04} Ashton, M.~C., Lee, K., Perugini, M., Szarota, P., de~Vries, R.~E., Di~Blas, L., Boies, K., and De~Raad, B. (2004). \newblock {A six-factor structure of personality-descriptive adjectives: Solutions from psycholexical studies in seven languages}. \newblock {\em Journal of Personality and Social Psychology}, 86(2):356--366. \bibitem[Barrett, 2005]{Barrett:2005eh} Barrett, P. (2005). \newblock {What if there were no psychometrics?: Constructs, complexity, and measurement}. \newblock {\em Journal of Personality Assessment}, 85(2):134--140. \bibitem[Barrick and Mount, 1991]{Barrick:91} Barrick, M.~R. and Mount, M.~K. (1991). \newblock {{The big five personality dimensions and job performance: a meta-analysis}}. \newblock {\em Personnel Psychology}, 44(1):1--26. \bibitem[Bashaw and Anderson, 1967]{Bashaw:67} Bashaw, W.~L. and Anderson, Jr, H.~E. (1967). \newblock {A correction for replicated error in correlation coefficients}. \newblock {\em Psychometrika}, 32(4):435--441. \bibitem[Baumeister, 2016]{Baumeister:2016} Baumeister, R.~F. (2016). \newblock {Charting the future of social psychology on stormy seas: Winners, losers, and recommendations}. \newblock {\em Journal of Experimental Social Psychology}, pages 1--6. \bibitem[Bernaards and Jennrich, 2003]{Bernaards:2003wg} Bernaards, C.~A. and Jennrich, R.~I. (2003). \newblock {Orthomax rotation and perfect simple structure}. \newblock {\em Psychometrika}. \bibitem[Block, 1965]{Block:65} Block, J. (1965). \newblock {\em {The challenge of response sets: Unconfounding meaning, acquiescence, and social desirability in the MMPI}}. \newblock Appleton-Century-Crofts, New York. \bibitem[Block, 1995]{Block:95} Block, J. (1995). \newblock {A contrarian view of the five-factor approach to personality description.} \newblock {\em Psychological Bulletin}, 117(2):187--215. \bibitem[Borsboom, 2006]{Borsboom:06b} Borsboom, D. (2006). \newblock {The attack of the psychometricians}. \newblock {\em Psychometrika}, 71(3):425--440. \bibitem[Breen, 2012]{Breen:2012aa} Breen, J. (2012). \newblock {\em zipcode: U.S. ZIP Code database for geocoding}. \newblock R package version 1.0. \bibitem[Briggs, 1992]{Briggs:92} Briggs, S.~R. (1992). \newblock {{Assessing the five-factor model of personality description}}. \newblock {\em Journal of Personality}, 60(2):253--293. \bibitem[Brown, 2009]{Brown:2009vf} Brown, J.~D. (2009). \newblock {Choosing the right type of rotation in PCA and EFA}. \newblock {\em JALT Testing {\&} Evaluation SIG Newsletter}, 13(3):20--25. \bibitem[Buhrmester et~al., 2011]{Buhrmester:11} Buhrmester, M., Kwang, T., and Gosling, S.~D. (2011). \newblock {Amazon's Mechanical Turk: A new source of inexpensive, yet high-quality, data?} \newblock {\em Perspectives on Psychological Science}, 6(1):3--5. \bibitem[Butcher, 2001]{Butcher:2001tq} Butcher, J.~N. (2001). \newblock {\em {Minnesota Multiphasic Personality Inventory-2}}. \newblock Manual for Administration, Scoring, and Interpretation. University of Minnesota Press. \bibitem[Carroll, 1953]{Carroll:53} Carroll, J.~B. (1953). \newblock {An analytical solution for approximating simple structure in factor analysis}. \newblock {\em Psychometrika}, 18(1):23--38. \bibitem[Cattell and Chevrier, 2002]{Cattell:2002wh} Cattell, R.~B. and Chevrier, J.~M. (2002). \newblock {\em {Manual for the 16-PF}}. \newblock Institute for Personality and Ability Testing, 5 edition. \bibitem[Chen et~al., 2012]{Chen:2012gy} Chen, F.~F., Hayes, A., and Carver, C.~S. (2012). \newblock {Modeling general and specific variance in multifaceted constructs: A comparison of the bifactor model to other approaches}. \newblock {\em Journal of Personality}, 80(1):219--251. \bibitem[Chen et~al., 2006]{Chen:06} Chen, F.~F., West, S.~G., and Sousa, K.~H. (2006). \newblock {A comparison of bifactor and second-order models of quality of life}. \newblock {\em Multivariate Behavioral Research}, 41(2):189--225. \bibitem[Cliff, 1988]{Cliff:88} Cliff, N. (1988). \newblock {The eigenvalues-greater-than-one rule and the reliability of components}. \newblock {\em Psychological Bulletin}, 103(2):276--279. \bibitem[Cloninger et~al., 1994]{Cloninger:1994to} Cloninger, C.~R., Przybeck, T.~R., Svrakic, D.~M., and Wetzel, R.~D. (1994). \newblock {The Temperament and Character Inventory (TCI): A Guide to Its Development and Use}. \newblock Center for Psychobiology of Personality, St Louis, MO. \bibitem[code~by Richard A.~Becker et~al., 2016]{Richard-A.-Becker:2016aa} code~by Richard A.~Becker, O.~S., version by Ray Brownrigg. Enhancements~by Thomas P~Minka, A. R. W.~R., and Deckmyn., A. (2016). \newblock {\em maps: Draw Geographical Maps}. \newblock R package version 3.1.1. \bibitem[Comrey, 1988]{Comrey:1988wl} Comrey, A.~L. (1988). \newblock {Factor-analytic methods of scale development in personality and clinical psychology.} \newblock {\em Journal of Consulting and Clinical Psychology}, 56(5):754--761. \bibitem[Condon, 2014]{Condon:14Diss} Condon, D.~M. (2014). \newblock {\em {An organizational framework for the psychological individual differences: Integrating the affective, cognitive, and conative domains }}. \newblock PhD thesis, Northwestern University, Evanston, IL. \bibitem[Condon and Revelle, 2014]{Condon:14} Condon, D.~M. and Revelle, W. (2014). \newblock {The International Cognitive Ability Resource: Development and initial validation of a public-domain measure}. \newblock {\em Intelligence}, 43:52--64. \bibitem[Costa and McCrae, 1992]{Costa:92a} Costa, P.~T. and McCrae, R.~R. (1992). \newblock {\em {Revised NEO Personality Inventory (NEO-PI-R) and NEO Five-Factor Inventory (NEO-FFI) Professional Manual}}. \newblock Psychological Assessment Resources, Odessa, FL. \bibitem[Costa and McCrae, 1995]{Costa:95} Costa, P.~T. and McCrae, R.~R. (1995). \newblock {Domains and facets: Hierarchical personality assessment using the Revised NEO Personality Inventory}. \newblock {\em Journal of Personality Assessment}, 64(1):21--50. \bibitem[Cramer et~al., 2012]{Cramer:12} Cramer, A. O.~J., van~der Sluis, S., Noordhof, A., Wichers, M., Geschwind, N., Aggen, S.~H., Kendler, K.~S., and Borsboom, D. (2012). \newblock {Dimensions of normal personality as networks in search of equilibrium: You can't like parties if you don't like people}. \newblock {\em European Journal of Personality}, 26(4):414--431. \bibitem[Dahl, 2016]{Dahl:2016aa} Dahl, D.~B. (2016). \newblock {\em xtable: Export Tables to LaTeX or HTML}. \newblock R package version 1.8-2. \bibitem[De~Raad et~al., 2010]{DeRaad:10} De~Raad, B., Barelds, D. P.~H., Levert, E., Ostendorf, F., Mla{\v c}i{\'c}, B., Blas, L.~D., Hreb{\'\i}ckov{\'a}, M., Szirm{\'a}k, Z., Szarota, P., Perugini, M., Church, A.~T., and Katigbak, M.~S. (2010). \newblock {Only three factors of personality description are fully replicable across languages: A comparison of 14 trait taxonomies}. \newblock {\em Journal of Personality and Social Psychology}, 98(1):160--173. \bibitem[DeYoung, 2010]{DeYoung:10a} DeYoung, C.~G. (2010). \newblock {Toward a theory of the Big Five}. \newblock {\em Psychological Inquiry}, 21(1):26--33. \bibitem[DeYoung, 2014]{DeYoung:14b} DeYoung, C.~G. (2014). \newblock {Cybernetic Big Five Theory}. \newblock pages 1--81. \bibitem[DeYoung et~al., 2007]{DeYoung:07} DeYoung, C.~G., Quilty, L.~C., and Peterson, J.~B. (2007). \newblock {Between facets and domains: 10 aspects of the Big Five}. \newblock {\em Journal of Personality and Social Psychology}, 93(5):880--896. \bibitem[Digman, 1990]{Digman:90} Digman, J.~M. (1990). \newblock {Personality structure: Emergence of the five-factor model}. \newblock {\em Annual Review of Psychology}, 41(1):417--440. \bibitem[Digman, 1997]{Digman:97} Digman, J.~M. (1997). \newblock {Higher-order factors of the Big Five}. \newblock {\em Journal of Personality and Social Psychology}, 73(6):1246--1256. \bibitem[Digman and Inouye, 1986]{Digman:86} Digman, J.~M. and Inouye, J. (1986). \newblock {Further specification of the five robust factors of personality.} \newblock {\em Journal of Personality and Social Psychology}, 50(1):116. \bibitem[Donnellan et~al., 2006]{Donnellan:06} Donnellan, M.~B., Oswald, F.~L., Baird, B.~M., and Lucas, R.~E. (2006). \newblock {The Mini-IPIP Scales: Tiny-yet-effective measures of the Big Five factors of personality.} \newblock {\em Psychological Assessment}, 18(2):192--203. \bibitem[{Douglas Nychka} et~al., 2015]{Douglas-Nychka:2015aa} {Douglas Nychka}, {Reinhard Furrer}, {John Paige}, and {Stephan Sain} (2015). \newblock fields: Tools for spatial data. \newblock R package version 8.4-1. \bibitem[Epskamp, 2014]{Epskamp:2014aa} Epskamp, S. (2014). \newblock {\em semPlot: Path diagrams and visual analysis of various SEM packages' output}. \newblock R package version 1.0.1. \bibitem[Epskamp et~al., 2012]{Epskamp:2012aa} Epskamp, S., Cramer, A. O.~J., Waldorp, L.~J., Schmittmann, V.~D., and Borsboom, D. (2012). \newblock {qgraph}: Network visualizations of relationships in psychometric data. \newblock {\em Journal of Statistical Software}, 48(4):1--18. \bibitem[Eremenco, 2005]{Eremenco:2005ks} Eremenco, S.~L. (2005). \newblock {A comprehensive method for the translation and cross-cultural validation of health status questionnaires}. \newblock {\em Evaluation {\&} the Health Professions}, 28(2):212--232. \bibitem[Estabrook, 2015]{Estabrook:2015bu} Estabrook, R. (2015). \newblock {Evaluating measurement of dynamic constructs: Defining a measurement model of derivatives.} \newblock {\em Psychological Methods}, 20(1):117--141. \bibitem[Eysenck, 1981]{Eysenck:81} Eysenck, H.~J. (1981). \newblock {\em {A model for personality}}. \newblock Springer-Verlag, New York. \bibitem[Eysenck, 1991]{Eysenck:91} Eysenck, H.~J. (1991). \newblock {Dimensions of personality: 16, 5 or 3?---Criteria for a taxonomic paradigm}. \newblock {\em Personality and Individual Differences}, 12(8):773--790. \bibitem[Eysenck, 1994]{Eysenck:94} Eysenck, H.~J. (1994). \newblock {The Big Five or Giant Three: Criteria for a paradigm}. \newblock In Halverson, C.~F., Kohnstamm, G.~A., and Martin, R.~P., editors, {\em The developing structure of temperament and personality from infancy to adulthood}, pages 37--51. Lawrence Erlbaum, Hillsdale, NJ. \bibitem[Eysenck et~al., 1985]{Eysenck:85} Eysenck, S., Eysenck, H., and Barrett, P. (1985). \newblock {A revised version of the psychoticism scale}. \newblock {\em Personality and Individual Differences}, 6(1):21--29. \bibitem[Fabrigar et~al., 1999]{Fabrigar:1999tr} Fabrigar, L.~R., Wegener, D.~T., and MacCallum, R.~C. (1999). \newblock {Evaluating the use of exploratory factor analysis in psychological research.} \newblock {\em Psychological Science}. \bibitem[Galton, 1884]{Galton:1884} Galton, F. (1884). \newblock {The measurement of character}. \newblock {\em Fortnightly Review}, 42:179--185. \bibitem[Gandrud, 2015]{Gandrud:2015ux} Gandrud, C. (2015). \newblock {\em {Reproducible Research with R and R Studio, Second Edition}}. \newblock CRC Press, Berlin, Germany. \bibitem[Gerber and Furrer, 2015]{Gerber:2015} Gerber, F. and Furrer, R. (2015). \newblock Pitfalls in the implementation of bayesian hierarchical modeling of areal count data: An illustration using bym and leroux models. \newblock {\em Journal of Statistical Software}, 63(1):1--32. \bibitem[Goldberg, 1981]{Goldberg:81a} Goldberg, L.~R. (1981). \newblock {Language and individual differences: The search for universals in personality lexicons}. \newblock In Wheeler, L., editor, {\em Review of personality and scoial psychology}, pages 141--165. Sage Publications, Beverly Hills. \bibitem[Goldberg, 1990]{Goldberg:90} Goldberg, L.~R. (1990). \newblock {An alternative ``description of personality": The Big-Five factor structure}. \newblock {\em Journal of Personality and Social Psychology}, 59(6):1216--1229. \bibitem[Goldberg, 1992]{Goldberg:92} Goldberg, L.~R. (1992). \newblock {The development of markers for the Big-Five factor structure.} \newblock {\em Psychological Assessment}, 4(1):26--42. \bibitem[Goldberg, 1993a]{Goldberg:93c} Goldberg, L.~R. (1993a). \newblock {The structure of personality traits: Vertical and horizontal}. \newblock In Funder, D.~C., Parke, R.~D., Tomlinson-Keasey, C., and Widaman, K., editors, {\em Studying lives through time: Personality and development}, pages 169--188. American Psychological Association, Washington, DC. \bibitem[Goldberg, 1993b]{Goldberg:93} Goldberg, L.~R. (1993b). \newblock {The structure of phenotypic personality traits}. \newblock {\em American Psychologist}, 48:26--34. \bibitem[Goldberg, 1999]{Goldberg:99a} Goldberg, L.~R. (1999). \newblock {A broad-bandwidth, public domain, personality inventory measuring the lower-level facets of several Five-Factor Models}. \newblock In Mervielde, I., Deary, I., De~Fruyt, F., and Ostendorf, F., editors, {\em Personality and Individual Differences}, pages 7--28. Tilburg University Press, The Netherlands. \bibitem[Goldberg, 2008]{Goldberg:2008vg} Goldberg, L.~R. (2008). \newblock {The Eugene-Springfield community sample: Information available from the research participants}. \newblock Technical Report~1. \bibitem[Goldberg, 2009]{Goldberg:09} Goldberg, L.~R. (2009). \newblock {How to win a career achievement award in five easy lessons}. \newblock {\em Journal of Personality Assessment}, 91(6):506--517. \bibitem[Goldberg, 2016]{ipip} Goldberg, L.~R. (2016). \newblock {International Personality Item Pool: A scientific collaboratory for the development of advanced measures of personality traits and other individual differences}. \bibitem[Goldberg and Johnson, 2006]{Goldberg:06} Goldberg, L.~R. and Johnson, J.~A. (2006). \newblock {The international personality item pool and the future of public-domain personality measures}. \newblock {\em Journal of Research in Personality}, 40(1):84--96. \bibitem[Goldberg and Saucier, 2016]{Goldberg:2016vg} Goldberg, L.~R. and Saucier, G. (2016). \newblock {The~Eugene-Springfield~Community~Sample:~Information~Available~from~the~Research~Participants}. \newblock Technical report. \bibitem[Goldberg and Velicer, 2006]{Goldberg:2006wj} Goldberg, L.~R. and Velicer, W.~F. (2006). \newblock {Differentiating normal and abnormal personality}. \newblock In Strack, S., editor, {\em Principles of exploratory factor analysis}, pages 209--237. Springer Publishing Company, New York. \bibitem[Gorsuch, 1997]{Gorsuch:97b} Gorsuch, R.~L. (1997). \newblock {Exploratory factor analysis: Its role in item analysis}. \newblock {\em Journal of Personality Assessment}, 68(3):532--560. \bibitem[Gosling et~al., 2003]{Gosling:03b} Gosling, S.~D., Rentfrow, P.~J., and Swann, W.~B. (2003). \newblock {A very brief measure of the Big-Five personality domains}. \newblock {\em Journal of Research in Personality}, 37:504--528. \bibitem[Gough, 1975]{Gough:75} Gough, H.~G. (1975). \newblock {Manual for the California Psychological Inventory}. \newblock Consulting Psychologists Press. \bibitem[Gough and Bradley, 1996]{Gough:1996wm} Gough, H.~G. and Bradley, P. (1996). \newblock {California Psychological Inventory}. \newblock Consulting Psychologists Press, Palo Alto, CA. \bibitem[Groth-Marnat, 2016]{GrothMarnat:2009} Groth-Marnat, G. (2016). \newblock {California Psychological Inventory}. \newblock In {\em Handbook of Psychological Assessment}, pages 335--384. Hoboken, NJ. \bibitem[Haisken-DeNew and Frick, 2005]{HaiskenDeNew:05} Haisken-DeNew, J. and Frick, J.~R. (2005). \newblock {\em {DTC Desktop Companion to the German Socio-Economic Panel (SOEP)}}. \newblock Deutsches Institut f{\"u}r Wirtschaftsforschung, Berlin. \bibitem[Hathaway and Mckinley, 1940]{Hathaway:1940id} Hathaway, S.~R. and Mckinley, J.~C. (1940). \newblock {A Multiphasic Personality Schedule (Minnesota) : I. Construction of the Schedule}. \newblock {\em The Journal of Psychology}, 10(2):249--254. \bibitem[Hendriks, 1997]{Hendriks:97} Hendriks, A.~J. (1997). \newblock {\em {The construction of the five-factor personality inventory (FFPI)}}. \newblock PhD thesis, University of Groningen, Groningen. \bibitem[Hendriks et~al., 1999]{Hendriks:99} Hendriks, A.~J., Hofstee, W.~K., and Raad, B.~D. (1999). \newblock {The five-factor personality inventory (FFPI)}. \newblock {\em Personality and Individual Differences}, 27(2):307--325. \bibitem[Hendriks et~al., 2003]{Hendriks:03} Hendriks, A.~J., Perugini, M., Angleitner, A., Ostendorf, F., Johnson, J.~A., De~Fruyt, F., Hrebickov{\'a}, M., Kreitler, S., Murakami, T., Bratko, D., Conner, M., Nagy, J., Rodriguez-Fornells, A., and Ruisel, I. (2003). \newblock {The five-factor personality inventory: Cross-cultural generalizability across 13 countries}. \newblock {\em European Journal of Personality}, 17(5):347--373. \bibitem[Hofmann, 1977]{Hofmann:77} Hofmann, R.~J. (1977). \newblock {Indices descriptive of factor complexity}. \newblock {\em The Journal of General Psychology}, 96(1):103--110. \bibitem[Hofmann, 1978]{Hofmann:78} Hofmann, R.~J. (1978). \newblock {Complexity and simplicity as objective indices descriptive of factor solutions}. \newblock {\em Multivariate Behavioral Research}, 13(2):247--250. \bibitem[Hofstee et~al., 1998]{Hofstee:98} Hofstee, W., Berge, J.~T., and Hendriks, A. (1998). \newblock {How to score questionnaires}. \newblock {\em Personality and Individual Differences}, 25(5):897--909. \bibitem[Hofstee et~al., 1992]{Hofstee:92} Hofstee, W.~K., De~Raad, B., and Goldberg, L.~R. (1992). \newblock {Integration of the Big Five and circumplex approaches to trait structure}. \newblock {\em Journal of Personality and Social Psychology}, 63(1):146. \bibitem[Hogan, 1992]{Hogan:92} Hogan, R. (1992). \newblock {\em {Hogan Personality Inventory manual}}. \newblock Hogan Assessment Systems, Tulsa, OK. \bibitem[Hogan and Hogan, 2007]{Hogan:2007tg} Hogan, R. and Hogan, J. (2007). \newblock {\em {Hogan Personality Inventory Manual}}. \newblock Hogan Assessment Systems, Tulsa, OK. \bibitem[Hogan et~al., 1996]{Hogan:96} Hogan, R., Hogan, J., and Roberts, B.~W. (1996). \newblock {Personality measurement and employment decisions: Questions and answers}. \newblock {\em American Psychologist}, 51(5):469. \bibitem[Hu and Bentler, 1999]{Hu:99} Hu, L. and Bentler, P.~M. (1999). \newblock {Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives}. \newblock {\em Structural Equation Modeling}, 6(1):1--55. \bibitem[Jackson, 2004]{Jackson:2004tx} Jackson, D.~N. (2004). \newblock {Jackson Personality Inventory, Revised}. \newblock Sigma Assessment Systems, Port Huron, MI. \bibitem[Jackson et~al., 2000]{Jackson:2000tj} Jackson, D.~N., Paunonen, S.~V., and Tremblay, P.~F. (2000). \newblock {Six Factor Personality Questionnaire: Technical Manual }. \newblock Sigma Assessment Systems, Port Huron, MI. \bibitem[John and Srivastava, 1999]{John:1999} John, O.~P. and Srivastava, S. (1999). \newblock {The Big Five trait taxonomy: History, measurement, and theoretical perspectives}. \newblock In {\em The Handbook of Personality: Theory and Research}, pages 102--138. Guilford Press, London, UK. \bibitem[Johnson, 2014a]{Johnson:2014} Johnson, J.~A. (2014a). \newblock {\em {Acquiescence and social desirability: Psychometric bogeymen}}. \newblock Retrieved August 8, 2016, {from https://www.psychologytoday.com/blog/cui-bono/201404/acquiescence-and-social-desirability-psychometric-bogeymen}. \bibitem[Johnson, 2014b]{Johnson:14} Johnson, J.~A. (2014b). \newblock {Measuring thirty facets of the Five Factor Model with a 120-Item public domain inventory: Development of the IPIP-NEO-120}. \newblock {\em Journal of Research in Personality}, 51:78--89. \bibitem[{JSI}, 2014]{ZIP2ZCTA} {JSI} (2014). \newblock {ZIP Code to ZCTA Crosswalk Table}. \newblock {\em {John Snow, Inc. and the JSI Research \& Training Institute, Inc.}} \bibitem[Kenny, 2016]{Kenny:12} Kenny, D.~A. (2016). \newblock {\em {Measuring model fit.}} \newblock Retrieved October 27, 2016, {from http://www.davidakenny.net/cm/fit.htm}. \bibitem[Kenny et~al., 2014]{Kenny:2014jc} Kenny, D.~A., Kaniskan, B., and McCoach, D.~B. (2014). \newblock {The performance of RMSEA in models with small degrees of freedom}. \newblock {\em Sociological Methods {\&} Research}. \bibitem[Lee and Ashton, 2004]{Lee:04} Lee, K. and Ashton, M.~C. (2004). \newblock {Psychometric Properties of the HEXACO Personality Inventory}. \newblock {\em Multivariate Behavioral Research}, 39(2):329--358. \bibitem[Lee et~al., 2005]{Lee:05} Lee, K., Ogunfowora, B., and Ashton, M.~C. (2005). \newblock {Personality traits beyond the Big Five: Are they within the HEXACO space?} \newblock {\em Journal of Personality}, 73(5):1437--1463. \bibitem[Leisch, 2002]{Sweave} Leisch, F. (2002). \newblock {Sweave: Dynamic generation of statistical reports using literate data analysis}. \newblock In H{\"a}rdle, W. and R{\"o}nz, B., editors, {\em Compstat 2002 - Proceedings in Computational Statistics}, pages 575--580. Physica Verlag, Heidelberg. \bibitem[Lord, 1955]{Lord:55} Lord, F.~M. (1955). \newblock {Sampling fluctuations resulting from the sampling of test items}. \newblock {\em Psychometrika}, 20(1):1--22. \bibitem[Maples et~al., 2014]{Maples:2014hr} Maples, J.~L., Guan, L., Carter, N.~T., and Miller, J.~D. (2014). \newblock {A test of the International Personality Item Pool representation of the Revised NEO Personality Inventory and development of a 120-item IPIP-based measure of the five-factor model.} \newblock {\em Psychological Assessment}, 26(4):1070--1084. \bibitem[Markey and Markey, 2009]{Markey:2009bc} Markey, P.~M. and Markey, C.~N. (2009). \newblock {A brief assessment of the Interpersonal Circumplex: The IPIP-IPC}. \newblock {\em Assessment}, 16(4):352--361. \bibitem[McCrae and Costa, 2010]{McCrae:2010} McCrae, R.~J. and Costa, P.~T. (2010). \newblock {\em {NEO Inventories: Professional manual}}. \newblock Psychological Assessment Resources, Inc., Lutz, FL. \bibitem[McCrae and Costa, 1992]{McCrae:1992ur} McCrae, R.~R. and Costa, P.~T. (1992). \newblock {Discriminant validity of NEO-PI-R facet scales}. \newblock {\em Educational and Psychological Measurement}, 52(1):229--237. \bibitem[McCrae and Terracciano, 2005]{McCrae:2005jz} McCrae, R.~R. and Terracciano, A. (2005). \newblock {Universal features of personality traits from the observer's perspective: data from 50 cultures.} \newblock {\em Journal of Personality and Social Psychology}, 88(3):547--561. \bibitem[Michalke, 2016]{R:koRpus} Michalke, M. (2016). \newblock {\em koRpus: An R Package for Text Analysis}. \newblock (Version 0.06-4). \bibitem[Mislevy et~al., 1992]{Mislevy:92} Mislevy, R.~J., Beaton, A.~E., Kaplan, B., and Sheehan, K.~M. (1992). \newblock {Estimating population characteristics from sparse matrix samples of item responses}. \newblock {\em Journal of Educational Measurement}, 29:133--161. \bibitem[M{\~o}ttus, 2016]{Mottus:2016kx} M{\~o}ttus, R. (2016). \newblock {Towards more rigorous personality trait-outcome research}. \newblock {\em European Journal of Personality}, 30(4):292--303. \bibitem[Myers et~al., 1998]{MBTI:1998} Myers, I.~B., McCaulley, M.~H., Quenk, N.~L., and Hammer, A.~L. (1998). \newblock {\em {MBTI Manual: A guide to the development and use of the Myers Briggs type indicator}}. \newblock Consulting Psychologists Press, 3rd edition. \bibitem[Navarro and Griffin, 2002]{Navarro:93} Navarro, A. and Griffin, R.~A. (2002). \newblock {Matrix sampling designs for the year 2000 census}. \newblock In {\em American Statistical Association, proceedings of the section on survey research methods}, pages 480--485. \bibitem[Neuwirth, 2014]{Neuwirth:2014aa} Neuwirth, E. (2014). \newblock {\em RColorBrewer: ColorBrewer Palettes}. \newblock R package version 1.1-2. \bibitem[Obschonka et~al., 2013]{Obschonka:13} Obschonka, M., Schmitt-Rodermund, E., Silbereisen, R.~K., Gosling, S.~D., and Potter, J. (2013). \newblock {The regional distribution and correlates of an entrepreneurship-prone personality profile in the United States, Germany, and the United Kingdom: A socioecological perspective}. \newblock {\em Journal of Personality and Social Psychology}. \bibitem[{OECD}, 2012]{PISA:09} {OECD} (2012). \newblock {\em {PISA 2009 Technical Report}}. \newblock OECD Publishing. \bibitem[Osborne, 2015]{Osborne:2015tc} Osborne, J.~W. (2015). \newblock {What is rotating in exploratory factor analysis?} \newblock {\em Practical Assessment}, 20(2). \bibitem[Osborne and Fitzpatrick, 2012]{Osborne:2012te} Osborne, J.~W. and Fitzpatrick, D.~C. (2012). \newblock {Replication analysis in exploratory factor analysis: What it is and why it makes your analysis better.} \newblock {\em Practical Assessment, Research {\&} Evaluation}, 17:1--8. \bibitem[Paunonen, 2003]{Paunonen:03} Paunonen, S.~V. (2003). \newblock {Big Five factors of personality and replicated predictions of behavior.} \newblock {\em Journal of Personality and Social Psychology}, 84(2):411--424. \bibitem[Pettersson and Turkheimer, 2010]{Pettersson:10} Pettersson, E. and Turkheimer, E. (2010). \newblock {Item selection, evaluation, and simple structure in personality data}. \newblock {\em Journal of Research in Personality}, 44(4):407--420. \bibitem[Pettersson and Turkheimer, 2014]{Pettersson:14} Pettersson, E. and Turkheimer, E. (2014). \newblock {Self-reported personality pathology has complex structure and imposing simple structure degrades test information}. \newblock {\em Multivariate Behavioral Research}. \bibitem[{R Core Team}, 2016a]{R} {R Core Team} (2016a). \newblock {\em R: A Language and Environment for Statistical Computing}. \newblock R Foundation for Statistical Computing, Vienna, Austria. \newblock {ISBN} 3-900051-07-0. \bibitem[{R Core Team}, 2016b]{R-Core-Team:2016aa} {R Core Team} (2016b). \newblock {\em R: A Language and Environment for Statistical Computing}. \newblock R Foundation for Statistical Computing, Vienna, Austria. \bibitem[Reise, 2012]{Reise:12} Reise, S.~P. (2012). \newblock {The rediscovery of bifactor measurement models}. \newblock {\em Multivariate Behavioral Research}, 47(5):667--696. \bibitem[Revelle, 2016]{Rpsych} Revelle, W. (2016). \newblock {\em psych: Procedures for psychological, psychometric, and personality research}. \newblock Northwestern University, Evanston, Illinois. \newblock R package version 1.6. \bibitem[Revelle and Condon, 2015]{Revelle:2015hg} Revelle, W. and Condon, D.~M. (2015). \newblock {A model for personality at three levels}. \newblock {\em Journal of Research in Personality}, 56:70--81. \bibitem[Revelle et~al., 2016]{Revelle:2016chap} Revelle, W., Condon, D.~M., Wilt, J.~A., French, J.~A., Brown, A.~D., and Elleman, L.~G. (2016). \newblock {Web and phone based data collection using planned missing designs}. \newblock In Fielding, N.~G., Lee, R.~M., and Blank, G., editors, {\em Handbook of Online Research Methods}. Thousand Oaks, CA. \bibitem[Revelle and Wilt, 2013]{Revelle:2013hs} Revelle, W. and Wilt, J. (2013). \newblock {The general factor of personality: A general critique}. \newblock {\em Journal of Research in Personality}, 47(5):493--504. \bibitem[Rhemtulla et~al., 2012]{Rhemtulla:2012jv} Rhemtulla, M., Brosseau-Liard, P.~{\'E}., and Savalei, V. (2012). \newblock {When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions.} \newblock {\em Psychological Methods}, 17(3):354--373. \bibitem[Roberts et~al., 2007]{Roberts:07} Roberts, B.~W., Kuncel, N.~R., Shiner, R., Caspi, A., and Goldberg, L.~R. (2007). \newblock {The power of personality: The comparative validity of personality traits, socioeconomic status, and cognitive ability for predicting important life outcomes}. \newblock {\em Perspectives on Psychological Science}, 2(4):313. \bibitem[Rorer, 1965]{Rorer:65} Rorer, L.~G. (1965). \newblock {The great response-style myth.} \newblock {\em Psychological Bulletin}. \bibitem[Rosseel, 2012a]{Rosseel:2012aa} Rosseel, Y. (2012a). \newblock {lavaan}: An {R} package for structural equation modeling. \newblock {\em Journal of Statistical Software}, 48(2):1--36. \bibitem[Rosseel, 2012b]{lavaan:12} Rosseel, Y. (2012b). \newblock {lavaan: An R package for structural equation modeling}. \newblock {\em Journal of statistical software}, 48(2):1--36. \bibitem[Saucier, 1997]{Saucier:1997uw} Saucier, G. (1997). \newblock {Effects of variable selection on the factor structure of person descriptors.} \newblock {\em Journal of Personality and Social Psychology}. \bibitem[Saucier, 2002]{Saucier:2002wx} Saucier, G. (2002). \newblock {Orthogonal markers for orthogonal factors: The case of the Big Five}. \newblock {\em Journal of Research in Personality}, 36:1--31. \bibitem[Saucier, 2009]{Saucier:09a} Saucier, G. (2009). \newblock {What are the most important dimensions of personality? Evidence from studies of descriptors in diverse languages}. \newblock {\em Social and Personality Psychology Compass}, 3(4):620--637. \bibitem[Savla et~al., 2007]{Savla:2007kb} Savla, J., Davey, A., Costa~Jr, P.~T., and Whitfield, K.~E. (2007). \newblock {Replicating the NEO-PI-R factor structure in African-American older adults}. \newblock {\em Personality and Individual Differences}, 43(5):1279--1288. \bibitem[Schwarz, 1978]{Schwarz:78} Schwarz, G. (1978). \newblock {Estimating the dimension of a model}. \newblock {\em The annals of statistics}, 6(2):461--464. \bibitem[semTools Contributors, 2016]{Contributors:2016aa} semTools Contributors (2016). \newblock {\em {semTools}: Useful tools for structural equation modeling}. \newblock R package version 0.4-12. \bibitem[Shaycoft, 1970]{Shaycoft:1970} Shaycoft, M. (1970). \newblock {The eigenvalue myth and the dimension-reduction fallacy}. \newblock pages 1--16. \bibitem[Sherman, 2015]{Sherman:2015aa} Sherman, R.~A. (2015). \newblock {\em multicon: Multivariate Constructs}. \newblock R package version 1.6. \bibitem[Soto and John, 2009]{Soto:09} Soto, C.~J. and John, O.~P. (2009). \newblock {Ten facet scales for the Big Five Inventory: Convergence with NEO PI-R facets, self-peer agreement, and discriminant validity}. \newblock {\em Journal of Research in Personality}, 43(1):84--90. \bibitem[Soto and John, 2016]{Soto:2016jr} Soto, C.~J. and John, O.~P. (2016). \newblock {The Next Big Five Inventory (BFI-2): Developing and Assessing a Hierarchical Model With 15 Facets to Enhance Bandwidth, Fidelity, and Predictive Power.} \newblock {\em Journal of Personality and Social Psychology}, pages 1--27. \bibitem[Soto et~al., 2008]{Soto:08} Soto, C.~J., John, O.~P., Gosling, S.~D., and Potter, J. (2008). \newblock {The developmental psychometrics of big five self-reports: Acquiescence, factor structure, coherence, and differentiation from ages 10 to 20.} \newblock {\em Journal of Personality and Social Psychology}, 94(4):718--737. \bibitem[Soto et~al., 2010]{Soto:10} Soto, C.~J., John, O.~P., Gosling, S.~D., and Potter, J. (2010). \newblock {Age differences in personality traits from 10 to 65: Big five domains and facets in a large cross-sectional sample.} \newblock {\em Journal of Personality and Social Psychology}, 100(2):330--348. \bibitem[South, 2011]{South:2011aa} South, A. (2011). \newblock rworldmap: A new r package for mapping global data. \newblock {\em The R Journal}, 3(1):35--43. \bibitem[Sternberg, 1990]{Sternberg:1990} Sternberg, R.~J. (1990). \newblock {Metaphors of mind}. \newblock Cambridge University Press, Cambridge, UK. \bibitem[Tellegen, 2003]{Tellegen:2003ue} Tellegen, A. (2003). \newblock {Multidimensional Personality Questionnaire}. \newblock University of Minnesota Press, Minneapolis. \bibitem[Tellegen and Waller, 2008]{Tellegen:08} Tellegen, A. and Waller, N.~G. (2008). \newblock {Exploring personality through test construction: Development of the Multidimensional Personality Questionnaire}. \newblock In {\em The SAGE Handbook of Personality Theory and Assessment: Volume 2 {\textemdash} Personality Measurement and Testing}, pages 261--292. SAGE Publications Ltd, 1 Oliver's Yard,~ 55 City Road,~ London~ ~ EC1Y 1SP~ United Kingdom~. \bibitem[Ten~Berge, 1999]{TenBerge:1999bg} Ten~Berge, J. M.~F. (1999). \newblock {A legitimate case of component analysis of ipsative measures, and partialling the mean as an alternative to ipsatization}. \newblock {\em Multivariate Behavioral Research}, 34(1):89--102. \bibitem[Thalmayer et~al., 2011]{Thalmayer:11} Thalmayer, A.~G., Saucier, G., and Eigenhuis, A. (2011). \newblock {Comparative validity of brief to medium-length Big Five and Big Six personality questionnaires}. \newblock {\em Psychological Assessment}, 23(4):995--1009. \bibitem[Thurstone, 1947]{Thurstone:47} Thurstone, L.~L. (1947). \newblock {\em {Multiple factor analysis: A development and expansion of vectors of the mind}}. \newblock University of Chicago Press, Chicago, IL. \bibitem[{U. S. Census Bureau}, 2012a]{censusJobstatus} {U. S. Census Bureau} (2012a). \newblock Table 13: Labor force and employment status of the civilian population 16 years and over by sex and age: 2012. \newblock {\em Age and Sex Composition in the United States: 2012}. \bibitem[{U. S. Census Bureau}, 2012b]{censusEdustatus} {U. S. Census Bureau} (2012b). \newblock Table 5: Educational attainment of the population 15 years and over by sex and age: 2012. \newblock {\em Age and Sex Composition in the Unites States: 2012}. \bibitem[{U. S. Census Bureau}, 2014]{censusRace} {U. S. Census Bureau} (2014). \newblock Annual estimates of the resident population by sex, age, race, and hispanic origin: April 1, 2010 to {July 1}, 2014. \newblock {\em Median Age and Age by Sex, Race, and Hispanic Origin}. \bibitem[{U.S. Census Bureau}, 2012]{censusMarstatus} {U.S. Census Bureau} (2012). \newblock Table 3: Marital status of the population 15 years and over by sex and age: 2012. \newblock {\em Age and Sex Composition in the United States: 2012}. \bibitem[Vazire, 2010]{Vazire:10} Vazire, S. (2010). \newblock {Who knows what about a person? The self{\textendash}other knowledge asymmetry (SOKA) model.} \newblock {\em Journal of Personality and Social Psychology}, 98:281--300. \bibitem[Velicer, 1976]{Velicer:76} Velicer, W.~F. (1976). \newblock {Determining the number of components from the matrix of partial correlations}. \newblock {\em Psychometrika}, 41(3):321--327. \bibitem[Velicer and Jackson, 1990]{Velicer:1990ew} Velicer, W.~F. and Jackson, D.~N. (1990). \newblock {Component analysis versus common factor analysis: Some issues in selecting an appropriate procedure}. \newblock {\em Multivariate Behavioral Research}, 25(1):1--28. \bibitem[Wickham, 2009]{Wickham:2009aa} Wickham, H. (2009). \newblock {\em ggplot2: Elegant Graphics for Data Analysis}. \newblock Springer-Verlag New York. \bibitem[Wickham, 2011]{Wickham:2011aa} Wickham, H. (2011). \newblock The split-apply-combine strategy for data analysis. \newblock {\em Journal of Statistical Software}, 40(1):1--29. \bibitem[Wickham, 2016]{Wickham:2016aa} Wickham, H. (2016). \newblock {\em tidyr: Easily Tidy Data with `spread()` and `gather()` Functions}. \newblock R package version 0.6.0. \bibitem[Wilt et~al., 2011]{Wilt:11b} Wilt, J., Condon, D.~M., and Revelle, W. (2011). \newblock {Telemetrics and online data collection: Collecting data at a distance}. \newblock In Laursen, B., Little, T.~D., and Card, N.~A., editors, {\em Handbook of Developmental Research Methods}, chapter~10, pages 163--180. Guilford Press, New York. \bibitem[Wood, 2011]{Wood:2011} Wood, S. (2011). \newblock Fast stable restricted maximum likelihood and marginal likelihood estimation of semiparametric generalized linear models. \newblock {\em Journal of the Royal Statistical Society (B)}, 73(1):3--36. \bibitem[Xie, 2015]{Xie:2015vc} Xie, Y. (2015). \newblock {\em {Dynamic Documents with R and knitr, Second Edition}}. \newblock CRC Press. \bibitem[Yarkoni, 2013]{Yarkoni:13} Yarkoni, T. (2013). \newblock {\em {Principled failure to detect a five-factor structure in a canonical Big Five measure}}. \newblock Retrieved January 18, 2014, {from http://figshare.com/}. \bibitem[Zinbarg et~al., 2006]{Zinbarg:06} Zinbarg, R.~E., Yovel, I., Revelle, W., and McDonald, R.~P. (2006). \newblock {Estimating generalizability to a latent variable common to all of a scale's indicators: A comparison of estimators for omega hierarchical}. \newblock {\em Applied Psychological Measurement}, 30(2):121--144. \end{thebibliography} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \clearpage \begin{center} \appendix \section*{Appendices} \addcontentsline{toc}{section}{Appendices} \end{center} \clearpage \begin{center} \begin{Large} \textbf{Appendix A} \subsection*{Item-factor loadings from exploratory factor analyses} \addcontentsline{toc}{subsection}{A: Item-factor loadings from exploratory factor analyses} \label{sec:appendixA} \end{Large} {\bf David M. Condon}\footnote[$\star$]{This document is an appendix to \textit{The SAPA Personality Inventory}. Correspondence should be directed to \href{mailto:david-condon@northwestern.edu}{david-condon@northwestern.edu}, Department of Medical Social Sciences, Northwestern University, Chicago, Illinois. Please use the following citation information: Condon, D.M. (2017). The SAPA Personality Inventory: an empirically-derived, hierarchically-organized self-report personality assessment model.} \end{center} %\clearpage \pagenumbering{arabic}% resets `page` counter to 1 \renewcommand*{\thepage}{A1-\arabic{page}} %\setlength{\parindent}{0.0in} %\clearpage \renewcommand{\thetable}{A1-\arabic{table}} \setcounter{table}{0} %\clearpage \renewcommand{\thefigure}{A1-\arabic{figure}} \setcounter{figure}{0} \vskip -0.5cm %%%%%%%%%%%%%%%%%% %Add Breaking Line \begin{center} \linethickness{0.5mm} \line(1,0){300} \end{center} %%%%%%%%%%%%%%%%%% This appendix contains many pages of tables that show the content for items loading onto each factor when 1 to 30 factors are extracted from the exploratory and replication samples. For ease of navigation, a listing of the tables (with hotlinks) follows this introduction. The methods for extraction (the exact \texttt{R} code used) is provided in Section \ref{sec:replication}. Table lists the item content for every extracted factor, labeled by the name of the table. Each table shows the 10 items with the highest primary loadings (or all of the items if fewer than 10 items have primary loadings above 0.3) on the given factor. Items are not listed if they have secondary loadings on any other factor(s) above 0.3. These are described elsewhere as the ``qualified'' items. Note that I am merely presenting all the tables here, without differentiating between the quality of the solutions. There are 465 tables for each sample --- (nfactors)*(nfactors+1)/2 where nfactors is set at 30 for these analyses. There are 2 samples; 930 tables in all. Note that some of the tables are merely place holders for instances where the factor does not have any qualified items. Each table also indicates the other IPIP-based personality scales in which the item can be found, if applicable. To keep things legible, I've used abbreviations for the column headers to denote the measures as follows: \vspace{2mm} \begin{hangparas}{.25in}{1} \textbf{IPIP100}: the 100 IPIP items corresponding to the Big Five factor marker \citep[][]{Goldberg:99a} scales \textbf{BFAS}: the 100 items of the 10 Big Five Aspect Scales \citep[][]{DeYoung:07} \textbf{HEXACO}: the 240 items of the 6 factor scales and 24 facet scales of the IPIP-HEXACO inventory \citep[][]{Ashton:07b} \textbf{QB6}: the 48 items of the 6 Questionnaire Big Six \citep[][]{Thalmayer:11} scales \textbf{IPIP-NEO}: the 300 items of the 5 factor scales and 30 facet scales of the IPIP-NEO \citep{Goldberg:99a} \textbf{MPQ}: the 127 items of the 12 IPIP-Multidimensional Personality Questionnaire \citep[][]{ipip, Tellegen:08} scales \textbf{EPQr}: revisions based on the 79 items of the 3 Eysenck Personality Questionnaire (Revised) \citep{Eysenck:85} scales (items from the Lie scale have been intentionally omitted) \textbf{PS}: the 40 items of the 2 Plasticity/Stability \citep{DeYoung:10a} scales \end{hangparas} \vspace{2mm} \noindent Administration of these scales also implies the administration of several other measures which are abbreviations of these scales, including the 24 and 36 item Questionnaire Big Six scales \citep{Thalmayer:11}, the 50 item IPIP scales corresponding to the Big Five factor markers \citep{ipip}, and the 20 item ``mini-IPIP'' scales \citep{Donnellan:06}. If one includes these items, the item sets administered represent all of the items in 114 publicly available personality scales. Several other IPIP-based scales are partially represented by this subset (e.g., the Hogan Personality Inventory [``HPI], Gough's California Psychological Inventory [CPI], Cloninger's Temperament and Character Inventory [TCI], Cattell's 16 Factor Personality Questionnaire [16PF]). See Table Table \ref{tab:IPIPscales} in Section \ref{sec:IPIPselection} for a (partial) listing of the scales represented in this 696 item set. \noindent Here are the abbreviations used to denote the scales within each measure: \vspace{2mm} \begin{scriptsize} \begin{hangparas}{.25in}{1} \hspace{6.5mm}\textbf{IPIP100:B5:A} = IPIP100 - Agreeableness\\ \textbf{IPIP100:B5:C} = IPIP100 - Conscientiousness\\ \textbf{IPIP100:B5:E} = IPIP100 - Extraversion\\ \textbf{IPIP100:B5:ES} = IPIP100 - Emotional Stability\\ \textbf{IPIP100:B5:I} = IPIP100 - Intellect\\ \textbf{BFAS:A:C} = BFAS Agreeableness - Compassion\\ \textbf{BFAS:A:P} = BFAS Agreeableness - Politeness\\ \textbf{BFAS:C:I} = BFAS Conscientiousness - Industriousness\\ \textbf{BFAS:C:O} = BFAS Conscientiousness - Orderliness\\ \textbf{BFAS:E:A} = BFAS Extraversion - Assertiveness\\ \textbf{BFAS:E:E} = BFAS Extraversion - Enthusiasm\\ \textbf{BFAS:N:W} = BFAS Neuroticism - Withdrawal\\ \textbf{BFAS:N:W} = BFAS Neuroticism - Volatility\\ \textbf{BFAS:O:V} = BFAS Openness - Openness\\ \textbf{BFAS:O:I} = BFAS Openness - Intellect\\ \textbf{H:H:F} = HEXACO Honesty-Humility - Fairness\\ \textbf{H:H:GA} = HEXACO Honesty-Humility - Greed Avoidance\\ \textbf{H:H:M} = HEXACO Honesty-Humility - Modesty\\ \textbf{H:H:S} = HEXACO Honesty-Humility - Sincerity\\ \textbf{H:E:A} = HEXACO Emotionality - Anxiety\\ \textbf{H:E:D} = HEXACO Emotionality - Dependence\\ \textbf{H:E:F} = HEXACO Emotionality - Fearfulness\\ \textbf{H:E:S} = HEXACO Emotionality - Sentimentality\\ \textbf{H:X:E} = HEXACO eXtraversion - Expressiveness\\ \textbf{H:X:L} = HEXACO eXtraversion - Liveliness\\ \textbf{H:X:S} = HEXACO eXtraversion - Sociability\\ \textbf{H:X:SB} = HEXACO eXtraversion - Social Boldness\\ \textbf{H:A:FL} = HEXACO Agreeableness - Flexibility\\ \textbf{H:A:FO} = HEXACO Agreeableness - Forgiveness\\ \textbf{H:A:G} = HEXACO Agreeableness - Gentleness\\ \textbf{H:A:P} = HEXACO Agreeableness - Patience\\ \textbf{H:C:D} = HEXACO Conscientiousness - Diligence\\ \textbf{H:C:O} = HEXACO Conscientiousness - Organization\\ \textbf{H:C:PE} = HEXACO Conscientiousness - Perfectionism\\ \textbf{H:C:PR} = HEXACO Conscientiousness - Prudence\\ \textbf{H:O:AA} = HEXACO Openness - Aesthetic Appreciation\\ \textbf{H:O:C} = HEXACO Openness - Creativity\\ \textbf{H:O:I} = HEXACO Openness - Inquisitiveness\\ \textbf{H:O:U} = HEXACO Openness - Unconventionality\\ \textbf{QB6:A} = QB6 Agreeableness\\ \textbf{QB6:C} = QB6 Conscientiousness\\ \textbf{QB6:ES} = QB6 Emotional Stability\\ \textbf{QB6:H} = QB6 Honesty/Propriety\\ \textbf{QB6:O} = QB6 Openness\\ \textbf{QB6:X} = QB6 Extraversion\\ \textbf{N1:AN}: = IPIP-NEO Neuroticism - Anxiety\\ \textbf{N2:AN}: = IPIP-NEO Neuroticism - Anger\\ \textbf{N3:DE}: = IPIP-NEO Neuroticism - Depression\\ \textbf{N4:SC}: = IPIP-NEO Neuroticism - Self-Consciousness\\ \textbf{N5:IM}: = IPIP-NEO Neuroticism - Immoderation\\ \textbf{N6:VU}: = IPIP-NEO Neuroticism - Vulnerability\\ \textbf{E1:FR}: = IPIP-NEO Extraversion - Friendliness\\ \textbf{E2:GR}: = IPIP-NEO Extraversion - Gregariousness\\ \textbf{E3:AS}: = IPIP-NEO Extraversion - Assertiveness\\ \textbf{E4:AL}: = IPIP-NEO Extraversion - Activity Level\\ \textbf{E5:ES}: = IPIP-NEO Extraversion - Excitement-Seeking\\ \textbf{E6:CH}: = IPIP-NEO Extraversion - Cheerfulness\\ \textbf{O1:IM}: = IPIP-NEO Openness - Imagination\\ \textbf{O2:AI}: = IPIP-NEO Openness - Artistic Interests\\ \textbf{O3:EM}: = IPIP-NEO Openness - Emotionality\\ \textbf{O4:AD}: = IPIP-NEO Openness - Adventurousness\\ \textbf{O5:IN}: = IPIP-NEO Openness - Intellect\\ \textbf{O6:LI}: = IPIP-NEO Openness - Liberalism\\ \textbf{A1:TR}: = IPIP-NEO Agreeableness - Trust\\ \textbf{A2:MO}: = IPIP-NEO Agreeableness - Morality\\ \textbf{A3:AL}: = IPIP-NEO Agreeableness - Altruism\\ \textbf{A4:CO}: = IPIP-NEO Agreeableness - Cooperation\\ \textbf{A5:MO}: = IPIP-NEO Agreeableness - Modesty\\ \textbf{A6:SY}: = IPIP-NEO Agreeableness - Sympathy\\ \textbf{C1:SE}: = IPIP-NEO Conscientiousness - Self-Efficacy\\ \textbf{C2:OR}: = IPIP-NEO Conscientiousness - Orderliness\\ \textbf{C3:DU}: = IPIP-NEO Conscientiousness - Dutifulness\\ \textbf{C4:AS}: = IPIP-NEO Conscientiousness - Achievement Striving\\ \textbf{C5:SD}: = IPIP-NEO Conscientiousness - Self-Discipline\\ \textbf{C6:CA}: = IPIP-NEO Conscientiousness - Cautiousness\\ \textbf{MPQ:WB} MPQ Wellbeing (Joyfulness)\\ \textbf{MPQ:SP} MPQ Social Potency (Power-Seeking)\\ \textbf{MPQ:AC} MPQ Achievement (Achievement-Seeking)\\ \textbf{MPQ:SC} MPQ Social Closeness (Friendliness)\\ \textbf{MPQ:SR} MPQ Stress Reaction (Emotional Instability)\\ \textbf{MPQ:AG} MPQ Aggression (Belligerence)\\ \textbf{MPQ:AL} MPQ Alienation (Distrust)\\ \textbf{MPQ:CO} MPQ Control (Planfulness)\\ \textbf{MPQ:HA} MPQ Harm Avoidance (Risk Avoidance)\\ \textbf{MPQ:TR} MPQ Traditionalism (Conservatism)\\ \textbf{MPQ:AB} MPQ Absorption (Imagination)\\ \textbf{MPQ:UV} MPQ Unlikely Virtues (Unlikely Virtues)\\ \textbf{EPQ:P} EPQrevised - Psychoticism\\ \textbf{EPQ:P} EPQrevised - Extraversion\\ \textbf{EPQ:P} EPQrevised - Neuroticism\\ \textbf{PS:P} Plasticity/Stability - Plasticity\\ \textbf{PS:S} Plasticity/Stability - Stability\\ \end{hangparas} \end{scriptsize} \clearpage \begin{scriptsize} <>= # In this first step we are simply getting things ready by installing and loading necessary packages and functions. These are hidden in the PDF output. rm(list=setdiff(ls(), c("filepathdata", "filepathfig"))) suppressPackageStartupMessages(library(psych)) suppressPackageStartupMessages(library(xtable)) load(paste(filepathdata, "tablesForInterpretation.Rdata", sep="")) alignList = c("p{0.7cm}", ">{\\hangindent=1em}p{6.5cm}", "p{1.0cm}", "p{0.95cm}", "p{0.9cm}", "p{0.65cm}", "p{0.8cm}", "p{1.3cm}", "p{0.6cm}", "p{0.6cm}") @ \end{scriptsize} \title{EFA1} \begin{tiny} <>= print(xtable(ExpTable1_1, caption = 'Exploratory Sample - EFA1:Factor1', label = 'ExpTable1_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable1_1, caption = 'Replication Sample - EFA1:Factor1', label = 'RepTable1_1', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \title{EFA2} \begin{tiny} <>= print(xtable(ExpTable2_1, caption = 'Exploratory Sample - EFA2:Factor1', label = 'ExpTable2_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable2_1, caption = 'Replication Sample - EFA2:Factor1', label = 'RepTable2_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable2_2, caption = 'Exploratory Sample - EFA2:Factor2', label = 'ExpTable2_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable2_2, caption = 'Replication Sample - EFA2:Factor2', label = 'RepTable2_2', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \title{EFA3} \begin{tiny} <>= print(xtable(ExpTable3_1, caption = 'Exploratory Sample - EFA3:Factor1', label = 'ExpTable3_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable3_1, caption = 'Replication Sample - EFA3:Factor1', label = 'RepTable3_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable3_2, caption = 'Exploratory Sample - EFA3:Factor2', label = 'ExpTable3_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable3_2, caption = 'Replication Sample - EFA3:Factor2', label = 'RepTable3_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable3_3, caption = 'Exploratory Sample - EFA3:Factor3', label = 'ExpTable3_3', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable3_3, caption = 'Replication Sample - EFA3:Factor3', label = 'RepTable3_3', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \title{EFA4} \begin{tiny} <>= print(xtable(ExpTable4_1, caption = 'Exploratory Sample - EFA4:Factor1', label = 'ExpTable4_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable4_1, caption = 'Replication Sample - EFA4:Factor1', label = 'RepTable4_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable4_2, caption = 'Exploratory Sample - EFA4:Factor2', label = 'ExpTable4_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable4_2, caption = 'Replication Sample - EFA4:Factor2', label = 'RepTable4_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable4_3, caption = 'Exploratory Sample - EFA4:Factor3', label = 'ExpTable4_3', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable4_3, caption = 'Replication Sample - EFA4:Factor3', label = 'RepTable4_3', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable4_4, caption = 'Exploratory Sample - EFA4:Factor4', label = 'ExpTable4_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable4_4, caption = 'Replication Sample - EFA4:Factor4', label = 'RepTable4_4', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \title{EFA5} \begin{tiny} <>= print(xtable(ExpTable5_1, caption = 'Exploratory Sample - EFA5:Factor1', label = 'ExpTable5_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable5_1, caption = 'Replication Sample - EFA5:Factor1', label = 'RepTable5_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable5_2, caption = 'Exploratory Sample - EFA5:Factor2', label = 'ExpTable5_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable5_2, caption = 'Replication Sample - EFA5:Factor2', label = 'RepTable5_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable5_3, caption = 'Exploratory Sample - EFA5:Factor3', label = 'ExpTable5_3', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable5_3, caption = 'Replication Sample - EFA5:Factor3', label = 'RepTable5_3', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable5_4, caption = 'Exploratory Sample - EFA5:Factor4', label = 'ExpTable5_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable5_4, caption = 'Replication Sample - EFA5:Factor4', label = 'RepTable5_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable5_5, caption = 'Exploratory Sample - EFA5:Factor5', label = 'ExpTable5_5', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable5_5, caption = 'Replication Sample - EFA5:Factor5', label = 'RepTable5_5', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \title{EFA6} \begin{tiny} <>= print(xtable(ExpTable6_1, caption = 'Exploratory Sample - EFA6:Factor1', label = 'ExpTable6_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable6_1, caption = 'Replication Sample - EFA6:Factor1', label = 'RepTable6_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable6_2, caption = 'Exploratory Sample - EFA6:Factor2', label = 'ExpTable6_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable6_2, caption = 'Replication Sample - EFA6:Factor2', label = 'RepTable6_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable6_3, caption = 'Exploratory Sample - EFA6:Factor3', label = 'ExpTable6_3', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable6_3, caption = 'Replication Sample - EFA6:Factor3', label = 'RepTable6_3', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable6_4, caption = 'Exploratory Sample - EFA6:Factor4', label = 'ExpTable6_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable6_4, caption = 'Replication Sample - EFA6:Factor4', label = 'RepTable6_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable6_5, caption = 'Exploratory Sample - EFA6:Factor5', label = 'ExpTable6_5', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable6_5, caption = 'Replication Sample - EFA6:Factor5', label = 'RepTable6_5', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable6_6, caption = 'Exploratory Sample - EFA6:Factor6', label = 'ExpTable6_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable6_6, caption = 'Replication Sample - EFA6:Factor6', label = 'RepTable6_6', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \title{EFA7} \begin{tiny} <>= print(xtable(ExpTable7_1, caption = 'Exploratory Sample - EFA7:Factor1', label = 'ExpTable7_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable7_1, caption = 'Replication Sample - EFA7:Factor1', label = 'RepTable7_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable7_2, caption = 'Exploratory Sample - EFA7:Factor2', label = 'ExpTable7_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable7_2, caption = 'Replication Sample - EFA7:Factor2', label = 'RepTable7_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable7_3, caption = 'Exploratory Sample - EFA7:Factor3', label = 'ExpTable7_3', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable7_3, caption = 'Replication Sample - EFA7:Factor3', label = 'RepTable7_3', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable7_4, caption = 'Exploratory Sample - EFA7:Factor4', label = 'ExpTable7_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable7_4, caption = 'Replication Sample - EFA7:Factor4', label = 'RepTable7_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable7_5, caption = 'Exploratory Sample - EFA7:Factor5', label = 'ExpTable7_5', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable7_5, caption = 'Replication Sample - EFA7:Factor5', label = 'RepTable7_5', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable7_6, caption = 'Exploratory Sample - EFA7:Factor6', label = 'ExpTable7_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable7_6, caption = 'Replication Sample - EFA7:Factor6', label = 'RepTable7_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable7_7, caption = 'Exploratory Sample - EFA7:Factor7', label = 'ExpTable7_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable7_7, caption = 'Replication Sample - EFA7:Factor7', label = 'RepTable7_7', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \title{EFA8} \begin{tiny} <>= print(xtable(ExpTable8_1, caption = 'Exploratory Sample - EFA8:Factor1', label = 'ExpTable8_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable8_1, caption = 'Replication Sample - EFA8:Factor1', label = 'RepTable8_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable8_2, caption = 'Exploratory Sample - EFA8:Factor2', label = 'ExpTable8_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable8_2, caption = 'Replication Sample - EFA8:Factor2', label = 'RepTable8_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable8_3, caption = 'Exploratory Sample - EFA8:Factor3', label = 'ExpTable8_3', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable8_3, caption = 'Replication Sample - EFA8:Factor3', label = 'RepTable8_3', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable8_4, caption = 'Exploratory Sample - EFA8:Factor4', label = 'ExpTable8_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable8_4, caption = 'Replication Sample - EFA8:Factor4', label = 'RepTable8_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable8_5, caption = 'Exploratory Sample - EFA8:Factor5', label = 'ExpTable8_5', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable8_5, caption = 'Replication Sample - EFA8:Factor5', label = 'RepTable8_5', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable8_6, caption = 'Exploratory Sample - EFA8:Factor6', label = 'ExpTable8_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable8_6, caption = 'Replication Sample - EFA8:Factor6', label = 'RepTable8_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable8_7, caption = 'Exploratory Sample - EFA8:Factor7', label = 'ExpTable8_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable8_7, caption = 'Replication Sample - EFA8:Factor7', label = 'RepTable8_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable8_8, caption = 'Exploratory Sample - EFA8:Factor8', label = 'ExpTable8_8', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable8_8, caption = 'Replication Sample - EFA8:Factor8', label = 'RepTable8_8', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \title{EFA9} \begin{tiny} <>= print(xtable(ExpTable9_1, caption = 'Exploratory Sample - EFA9:Factor1', label = 'ExpTable9_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable9_1, caption = 'Replication Sample - EFA9:Factor1', label = 'RepTable9_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable9_2, caption = 'Exploratory Sample - EFA9:Factor2', label = 'ExpTable9_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable9_2, caption = 'Replication Sample - EFA9:Factor2', label = 'RepTable9_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable9_3, caption = 'Exploratory Sample - EFA9:Factor3', label = 'ExpTable9_3', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable9_3, caption = 'Replication Sample - EFA9:Factor3', label = 'RepTable9_3', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable9_4, caption = 'Exploratory Sample - EFA9:Factor4', label = 'ExpTable9_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable9_4, caption = 'Replication Sample - EFA9:Factor4', label = 'RepTable9_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable9_5, caption = 'Exploratory Sample - EFA9:Factor5', label = 'ExpTable9_5', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable9_5, caption = 'Replication Sample - EFA9:Factor5', label = 'RepTable9_5', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable9_6, caption = 'Exploratory Sample - EFA9:Factor6', label = 'ExpTable9_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable9_6, caption = 'Replication Sample - EFA9:Factor6', label = 'RepTable9_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable9_7, caption = 'Exploratory Sample - EFA9:Factor7', label = 'ExpTable9_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable9_7, caption = 'Replication Sample - EFA9:Factor7', label = 'RepTable9_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable9_8, caption = 'Exploratory Sample - EFA9:Factor8', label = 'ExpTable9_8', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable9_8, caption = 'Replication Sample - EFA9:Factor8', label = 'RepTable9_8', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable9_9, caption = 'Exploratory Sample - EFA9:Factor9', label = 'ExpTable9_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable9_9, caption = 'Replication Sample - EFA9:Factor9', label = 'RepTable9_9', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \title{EFA10} \begin{tiny} <>= print(xtable(ExpTable10_1, caption = 'Exploratory Sample - EFA10:Factor1', label = 'ExpTable10_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable10_1, caption = 'Replication Sample - EFA10:Factor1', label = 'RepTable10_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable10_2, caption = 'Exploratory Sample - EFA10:Factor2', label = 'ExpTable10_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable10_2, caption = 'Replication Sample - EFA10:Factor2', label = 'RepTable10_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable10_3, caption = 'Exploratory Sample - EFA10:Factor3', label = 'ExpTable10_3', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable10_3, caption = 'Replication Sample - EFA10:Factor3', label = 'RepTable10_3', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable10_4, caption = 'Exploratory Sample - EFA10:Factor4', label = 'ExpTable10_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable10_4, caption = 'Replication Sample - EFA10:Factor4', label = 'RepTable10_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable10_5, caption = 'Exploratory Sample - EFA10:Factor5', label = 'ExpTable10_5', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable10_5, caption = 'Replication Sample - EFA10:Factor5', label = 'RepTable10_5', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable10_6, caption = 'Exploratory Sample - EFA10:Factor6', label = 'ExpTable10_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable10_6, caption = 'Replication Sample - EFA10:Factor6', label = 'RepTable10_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable10_7, caption = 'Exploratory Sample - EFA10:Factor7', label = 'ExpTable10_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable10_7, caption = 'Replication Sample - EFA10:Factor7', label = 'RepTable10_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable10_8, caption = 'Exploratory Sample - EFA10:Factor8', label = 'ExpTable10_8', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable10_8, caption = 'Replication Sample - EFA10:Factor8', label = 'RepTable10_8', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable10_9, caption = 'Exploratory Sample - EFA10:Factor9', label = 'ExpTable10_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable10_9, caption = 'Replication Sample - EFA10:Factor9', label = 'RepTable10_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable10_10, caption = 'Exploratory Sample - EFA10:Factor10', label = 'ExpTable10_10', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable10_10, caption = 'Replication Sample - EFA10:Factor10', label = 'RepTable10_10', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \title{EFA11} \begin{tiny} <>= print(xtable(ExpTable11_1, caption = 'Exploratory Sample - EFA11:Factor1', label = 'ExpTable11_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable11_1, caption = 'Replication Sample - EFA11:Factor1', label = 'RepTable11_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable11_2, caption = 'Exploratory Sample - EFA11:Factor2', label = 'ExpTable11_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable11_2, caption = 'Replication Sample - EFA11:Factor2', label = 'RepTable11_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable11_3, caption = 'Exploratory Sample - EFA11:Factor3', label = 'ExpTable11_3', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable11_3, caption = 'Replication Sample - EFA11:Factor3', label = 'RepTable11_3', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable11_4, caption = 'Exploratory Sample - EFA11:Factor4', label = 'ExpTable11_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable11_4, caption = 'Replication Sample - EFA11:Factor4', label = 'RepTable11_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable11_5, caption = 'Exploratory Sample - EFA11:Factor5', label = 'ExpTable11_5', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable11_5, caption = 'Replication Sample - EFA11:Factor5', label = 'RepTable11_5', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable11_6, caption = 'Exploratory Sample - EFA11:Factor6', label = 'ExpTable11_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable11_6, caption = 'Replication Sample - EFA11:Factor6', label = 'RepTable11_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable11_7, caption = 'Exploratory Sample - EFA11:Factor7', label = 'ExpTable11_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable11_7, caption = 'Replication Sample - EFA11:Factor7', label = 'RepTable11_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable11_8, caption = 'Exploratory Sample - EFA11:Factor8', label = 'ExpTable11_8', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable11_8, caption = 'Replication Sample - EFA11:Factor8', label = 'RepTable11_8', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable11_9, caption = 'Exploratory Sample - EFA11:Factor9', label = 'ExpTable11_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable11_9, caption = 'Replication Sample - EFA11:Factor9', label = 'RepTable11_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable11_10, caption = 'Exploratory Sample - EFA11:Factor10', label = 'ExpTable11_10', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable11_10, caption = 'Replication Sample - EFA11:Factor10', label = 'RepTable11_10', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable11_11, caption = 'Exploratory Sample - EFA11:Factor11', label = 'ExpTable11_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable11_11, caption = 'Replication Sample - EFA11:Factor11', label = 'RepTable11_11', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \title{EFA12} \begin{tiny} <>= print(xtable(ExpTable12_1, caption = 'Exploratory Sample - EFA12:Factor1', label = 'ExpTable12_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable12_1, caption = 'Replication Sample - EFA12:Factor1', label = 'RepTable12_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable12_2, caption = 'Exploratory Sample - EFA12:Factor2', label = 'ExpTable12_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable12_2, caption = 'Replication Sample - EFA12:Factor2', label = 'RepTable12_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable12_3, caption = 'Exploratory Sample - EFA12:Factor3', label = 'ExpTable12_3', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable12_3, caption = 'Replication Sample - EFA12:Factor3', label = 'RepTable12_3', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable12_4, caption = 'Exploratory Sample - EFA12:Factor4', label = 'ExpTable12_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable12_4, caption = 'Replication Sample - EFA12:Factor4', label = 'RepTable12_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable12_5, caption = 'Exploratory Sample - EFA12:Factor5', label = 'ExpTable12_5', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable12_5, caption = 'Replication Sample - EFA12:Factor5', label = 'RepTable12_5', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable12_6, caption = 'Exploratory Sample - EFA12:Factor6', label = 'ExpTable12_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable12_6, caption = 'Replication Sample - EFA12:Factor6', label = 'RepTable12_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable12_7, caption = 'Exploratory Sample - EFA12:Factor7', label = 'ExpTable12_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable12_7, caption = 'Replication Sample - EFA12:Factor7', label = 'RepTable12_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable12_8, caption = 'Exploratory Sample - EFA12:Factor8', label = 'ExpTable12_8', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable12_8, caption = 'Replication Sample - EFA12:Factor8', label = 'RepTable12_8', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable12_9, caption = 'Exploratory Sample - EFA12:Factor9', label = 'ExpTable12_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable12_9, caption = 'Replication Sample - EFA12:Factor9', label = 'RepTable12_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable12_10, caption = 'Exploratory Sample - EFA12:Factor10', label = 'ExpTable12_10', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable12_10, caption = 'Replication Sample - EFA12:Factor10', label = 'RepTable12_10', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable12_11, caption = 'Exploratory Sample - EFA12:Factor11', label = 'ExpTable12_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable12_11, caption = 'Replication Sample - EFA12:Factor11', label = 'RepTable12_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable12_12, caption = 'Exploratory Sample - EFA12:Factor12', label = 'ExpTable12_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable12_12, caption = 'Replication Sample - EFA12:Factor12', label = 'RepTable12_12', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \title{EFA13} \begin{tiny} <>= print(xtable(ExpTable13_1, caption = 'Exploratory Sample - EFA13:Factor1', label = 'ExpTable13_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable13_1, caption = 'Replication Sample - EFA13:Factor1', label = 'RepTable13_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable13_2, caption = 'Exploratory Sample - EFA13:Factor2', label = 'ExpTable13_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable13_2, caption = 'Replication Sample - EFA13:Factor2', label = 'RepTable13_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable13_3, caption = 'Exploratory Sample - EFA13:Factor3', label = 'ExpTable13_3', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable13_3, caption = 'Replication Sample - EFA13:Factor3', label = 'RepTable13_3', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable13_4, caption = 'Exploratory Sample - EFA13:Factor4', label = 'ExpTable13_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable13_4, caption = 'Replication Sample - EFA13:Factor4', label = 'RepTable13_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable13_5, caption = 'Exploratory Sample - EFA13:Factor5', label = 'ExpTable13_5', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable13_5, caption = 'Replication Sample - EFA13:Factor5', label = 'RepTable13_5', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable13_6, caption = 'Exploratory Sample - EFA13:Factor6', label = 'ExpTable13_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable13_6, caption = 'Replication Sample - EFA13:Factor6', label = 'RepTable13_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable13_7, caption = 'Exploratory Sample - EFA13:Factor7', label = 'ExpTable13_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable13_7, caption = 'Replication Sample - EFA13:Factor7', label = 'RepTable13_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable13_8, caption = 'Exploratory Sample - EFA13:Factor8', label = 'ExpTable13_8', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable13_8, caption = 'Replication Sample - EFA13:Factor8', label = 'RepTable13_8', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable13_9, caption = 'Exploratory Sample - EFA13:Factor9', label = 'ExpTable13_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable13_9, caption = 'Replication Sample - EFA13:Factor9', label = 'RepTable13_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable13_10, caption = 'Exploratory Sample - EFA13:Factor10', label = 'ExpTable13_10', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable13_10, caption = 'Replication Sample - EFA13:Factor10', label = 'RepTable13_10', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable13_11, caption = 'Exploratory Sample - EFA13:Factor11', label = 'ExpTable13_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable13_11, caption = 'Replication Sample - EFA13:Factor11', label = 'RepTable13_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable13_12, caption = 'Exploratory Sample - EFA13:Factor12', label = 'ExpTable13_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable13_12, caption = 'Replication Sample - EFA13:Factor12', label = 'RepTable13_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable13_13, caption = 'Exploratory Sample - EFA13:Factor13', label = 'ExpTable13_13', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable13_13, caption = 'Replication Sample - EFA13:Factor13', label = 'RepTable13_13', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \title{EFA14} \begin{tiny} <>= print(xtable(ExpTable14_1, caption = 'Exploratory Sample - EFA14:Factor1', label = 'ExpTable14_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable14_1, caption = 'Replication Sample - EFA14:Factor1', label = 'RepTable14_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable14_2, caption = 'Exploratory Sample - EFA14:Factor2', label = 'ExpTable14_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable14_2, caption = 'Replication Sample - EFA14:Factor2', label = 'RepTable14_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable14_3, caption = 'Exploratory Sample - EFA14:Factor3', label = 'ExpTable14_3', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable14_3, caption = 'Replication Sample - EFA14:Factor3', label = 'RepTable14_3', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable14_4, caption = 'Exploratory Sample - EFA14:Factor4', label = 'ExpTable14_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable14_4, caption = 'Replication Sample - EFA14:Factor4', label = 'RepTable14_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable14_5, caption = 'Exploratory Sample - EFA14:Factor5', label = 'ExpTable14_5', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable14_5, caption = 'Replication Sample - EFA14:Factor5', label = 'RepTable14_5', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable14_6, caption = 'Exploratory Sample - EFA14:Factor6', label = 'ExpTable14_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable14_6, caption = 'Replication Sample - EFA14:Factor6', label = 'RepTable14_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable14_7, caption = 'Exploratory Sample - EFA14:Factor7', label = 'ExpTable14_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable14_7, caption = 'Replication Sample - EFA14:Factor7', label = 'RepTable14_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable14_8, caption = 'Exploratory Sample - EFA14:Factor8', label = 'ExpTable14_8', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable14_8, caption = 'Replication Sample - EFA14:Factor8', label = 'RepTable14_8', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable14_9, caption = 'Exploratory Sample - EFA14:Factor9', label = 'ExpTable14_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable14_9, caption = 'Replication Sample - EFA14:Factor9', label = 'RepTable14_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable14_10, caption = 'Exploratory Sample - EFA14:Factor10', label = 'ExpTable14_10', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable14_10, caption = 'Replication Sample - EFA14:Factor10', label = 'RepTable14_10', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable14_11, caption = 'Exploratory Sample - EFA14:Factor11', label = 'ExpTable14_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable14_11, caption = 'Replication Sample - EFA14:Factor11', label = 'RepTable14_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable14_12, caption = 'Exploratory Sample - EFA14:Factor12', label = 'ExpTable14_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable14_12, caption = 'Replication Sample - EFA14:Factor12', label = 'RepTable14_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable14_13, caption = 'Exploratory Sample - EFA14:Factor13', label = 'ExpTable14_13', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable14_13, caption = 'Replication Sample - EFA14:Factor13', label = 'RepTable14_13', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable14_14, caption = 'Exploratory Sample - EFA14:Factor14', label = 'ExpTable14_14', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable14_14, caption = 'Replication Sample - EFA14:Factor14', label = 'RepTable14_14', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \title{EFA15} \begin{tiny} <>= print(xtable(ExpTable15_1, caption = 'Exploratory Sample - EFA15:Factor1', label = 'ExpTable15_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable15_1, caption = 'Replication Sample - EFA15:Factor1', label = 'RepTable15_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable15_2, caption = 'Exploratory Sample - EFA15:Factor2', label = 'ExpTable15_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable15_2, caption = 'Replication Sample - EFA15:Factor2', label = 'RepTable15_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable15_3, caption = 'Exploratory Sample - EFA15:Factor3', label = 'ExpTable15_3', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable15_3, caption = 'Replication Sample - EFA15:Factor3', label = 'RepTable15_3', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable15_4, caption = 'Exploratory Sample - EFA15:Factor4', label = 'ExpTable15_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable15_4, caption = 'Replication Sample - EFA15:Factor4', label = 'RepTable15_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable15_5, caption = 'Exploratory Sample - EFA15:Factor5', label = 'ExpTable15_5', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable15_5, caption = 'Replication Sample - EFA15:Factor5', label = 'RepTable15_5', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable15_6, caption = 'Exploratory Sample - EFA15:Factor6', label = 'ExpTable15_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable15_6, caption = 'Replication Sample - EFA15:Factor6', label = 'RepTable15_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable15_7, caption = 'Exploratory Sample - EFA15:Factor7', label = 'ExpTable15_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable15_7, caption = 'Replication Sample - EFA15:Factor7', label = 'RepTable15_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable15_8, caption = 'Exploratory Sample - EFA15:Factor8', label = 'ExpTable15_8', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable15_8, caption = 'Replication Sample - EFA15:Factor8', label = 'RepTable15_8', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable15_9, caption = 'Exploratory Sample - EFA15:Factor9', label = 'ExpTable15_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable15_9, caption = 'Replication Sample - EFA15:Factor9', label = 'RepTable15_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable15_10, caption = 'Exploratory Sample - EFA15:Factor10', label = 'ExpTable15_10', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable15_10, caption = 'Replication Sample - EFA15:Factor10', label = 'RepTable15_10', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable15_11, caption = 'Exploratory Sample - EFA15:Factor11', label = 'ExpTable15_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable15_11, caption = 'Replication Sample - EFA15:Factor11', label = 'RepTable15_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable15_12, caption = 'Exploratory Sample - EFA15:Factor12', label = 'ExpTable15_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable15_12, caption = 'Replication Sample - EFA15:Factor12', label = 'RepTable15_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable15_13, caption = 'Exploratory Sample - EFA15:Factor13', label = 'ExpTable15_13', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable15_13, caption = 'Replication Sample - EFA15:Factor13', label = 'RepTable15_13', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable15_14, caption = 'Exploratory Sample - EFA15:Factor14', label = 'ExpTable15_14', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable15_14, caption = 'Replication Sample - EFA15:Factor14', label = 'RepTable15_14', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable15_15, caption = 'Exploratory Sample - EFA15:Factor15', label = 'ExpTable15_15', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable15_15, caption = 'Replication Sample - EFA15:Factor15', label = 'RepTable15_15', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \title{EFA16} \begin{tiny} <>= print(xtable(ExpTable16_1, caption = 'Exploratory Sample - EFA16:Factor1', label = 'ExpTable16_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable16_1, caption = 'Replication Sample - EFA16:Factor1', label = 'RepTable16_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable16_2, caption = 'Exploratory Sample - EFA16:Factor2', label = 'ExpTable16_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable16_2, caption = 'Replication Sample - EFA16:Factor2', label = 'RepTable16_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable16_3, caption = 'Exploratory Sample - EFA16:Factor3', label = 'ExpTable16_3', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable16_3, caption = 'Replication Sample - EFA16:Factor3', label = 'RepTable16_3', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable16_4, caption = 'Exploratory Sample - EFA16:Factor4', label = 'ExpTable16_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable16_4, caption = 'Replication Sample - EFA16:Factor4', label = 'RepTable16_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable16_5, caption = 'Exploratory Sample - EFA16:Factor5', label = 'ExpTable16_5', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable16_5, caption = 'Replication Sample - EFA16:Factor5', label = 'RepTable16_5', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable16_6, caption = 'Exploratory Sample - EFA16:Factor6', label = 'ExpTable16_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable16_6, caption = 'Replication Sample - EFA16:Factor6', label = 'RepTable16_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable16_7, caption = 'Exploratory Sample - EFA16:Factor7', label = 'ExpTable16_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable16_7, caption = 'Replication Sample - EFA16:Factor7', label = 'RepTable16_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable16_8, caption = 'Exploratory Sample - EFA16:Factor8', label = 'ExpTable16_8', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable16_8, caption = 'Replication Sample - EFA16:Factor8', label = 'RepTable16_8', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable16_9, caption = 'Exploratory Sample - EFA16:Factor9', label = 'ExpTable16_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable16_9, caption = 'Replication Sample - EFA16:Factor9', label = 'RepTable16_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable16_10, caption = 'Exploratory Sample - EFA16:Factor10', label = 'ExpTable16_10', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable16_10, caption = 'Replication Sample - EFA16:Factor10', label = 'RepTable16_10', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable16_11, caption = 'Exploratory Sample - EFA16:Factor11', label = 'ExpTable16_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable16_11, caption = 'Replication Sample - EFA16:Factor11', label = 'RepTable16_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable16_12, caption = 'Exploratory Sample - EFA16:Factor12', label = 'ExpTable16_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable16_12, caption = 'Replication Sample - EFA16:Factor12', label = 'RepTable16_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable16_13, caption = 'Exploratory Sample - EFA16:Factor13', label = 'ExpTable16_13', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable16_13, caption = 'Replication Sample - EFA16:Factor13', label = 'RepTable16_13', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable16_14, caption = 'Exploratory Sample - EFA16:Factor14', label = 'ExpTable16_14', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable16_14, caption = 'Replication Sample - EFA16:Factor14', label = 'RepTable16_14', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable16_15, caption = 'Exploratory Sample - EFA16:Factor15', label = 'ExpTable16_15', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable16_15, caption = 'Replication Sample - EFA16:Factor15', label = 'RepTable16_15', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable16_16, caption = 'Exploratory Sample - EFA16:Factor16', label = 'ExpTable16_16', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable16_16, caption = 'Replication Sample - EFA16:Factor16', label = 'RepTable16_16', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \title{EFA17} \begin{tiny} <>= print(xtable(ExpTable17_1, caption = 'Exploratory Sample - EFA17:Factor1', label = 'ExpTable17_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable17_1, caption = 'Replication Sample - EFA17:Factor1', label = 'RepTable17_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable17_2, caption = 'Exploratory Sample - EFA17:Factor2', label = 'ExpTable17_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable17_2, caption = 'Replication Sample - EFA17:Factor2', label = 'RepTable17_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable17_3, caption = 'Exploratory Sample - EFA17:Factor3', label = 'ExpTable17_3', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable17_3, caption = 'Replication Sample - EFA17:Factor3', label = 'RepTable17_3', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable17_4, caption = 'Exploratory Sample - EFA17:Factor4', label = 'ExpTable17_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable17_4, caption = 'Replication Sample - EFA17:Factor4', label = 'RepTable17_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable17_5, caption = 'Exploratory Sample - EFA17:Factor5', label = 'ExpTable17_5', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable17_5, caption = 'Replication Sample - EFA17:Factor5', label = 'RepTable17_5', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable17_6, caption = 'Exploratory Sample - EFA17:Factor6', label = 'ExpTable17_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable17_6, caption = 'Replication Sample - EFA17:Factor6', label = 'RepTable17_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable17_7, caption = 'Exploratory Sample - EFA17:Factor7', label = 'ExpTable17_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable17_7, caption = 'Replication Sample - EFA17:Factor7', label = 'RepTable17_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable17_8, caption = 'Exploratory Sample - EFA17:Factor8', label = 'ExpTable17_8', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable17_8, caption = 'Replication Sample - EFA17:Factor8', label = 'RepTable17_8', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable17_9, caption = 'Exploratory Sample - EFA17:Factor9', label = 'ExpTable17_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable17_9, caption = 'Replication Sample - EFA17:Factor9', label = 'RepTable17_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable17_10, caption = 'Exploratory Sample - EFA17:Factor10', label = 'ExpTable17_10', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable17_10, caption = 'Replication Sample - EFA17:Factor10', label = 'RepTable17_10', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable17_11, caption = 'Exploratory Sample - EFA17:Factor11', label = 'ExpTable17_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable17_11, caption = 'Replication Sample - EFA17:Factor11', label = 'RepTable17_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable17_12, caption = 'Exploratory Sample - EFA17:Factor12', label = 'ExpTable17_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable17_12, caption = 'Replication Sample - EFA17:Factor12', label = 'RepTable17_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable17_13, caption = 'Exploratory Sample - EFA17:Factor13', label = 'ExpTable17_13', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable17_13, caption = 'Replication Sample - EFA17:Factor13', label = 'RepTable17_13', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable17_14, caption = 'Exploratory Sample - EFA17:Factor14', label = 'ExpTable17_14', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable17_14, caption = 'Replication Sample - EFA17:Factor14', label = 'RepTable17_14', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable17_15, caption = 'Exploratory Sample - EFA17:Factor15', label = 'ExpTable17_15', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable17_15, caption = 'Replication Sample - EFA17:Factor15', label = 'RepTable17_15', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable17_16, caption = 'Exploratory Sample - EFA17:Factor16', label = 'ExpTable17_16', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable17_16, caption = 'Replication Sample - EFA17:Factor16', label = 'RepTable17_16', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable17_17, caption = 'Exploratory Sample - EFA17:Factor17', label = 'ExpTable17_17', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable17_17, caption = 'Replication Sample - EFA17:Factor17', label = 'RepTable17_17', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \title{EFA18} \begin{tiny} <>= print(xtable(ExpTable18_1, caption = 'Exploratory Sample - EFA18:Factor1', label = 'ExpTable18_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable18_1, caption = 'Replication Sample - EFA18:Factor1', label = 'RepTable18_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable18_2, caption = 'Exploratory Sample - EFA18:Factor2', label = 'ExpTable18_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable18_2, caption = 'Replication Sample - EFA18:Factor2', label = 'RepTable18_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable18_3, caption = 'Exploratory Sample - EFA18:Factor3', label = 'ExpTable18_3', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable18_3, caption = 'Replication Sample - EFA18:Factor3', label = 'RepTable18_3', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable18_4, caption = 'Exploratory Sample - EFA18:Factor4', label = 'ExpTable18_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable18_4, caption = 'Replication Sample - EFA18:Factor4', label = 'RepTable18_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable18_5, caption = 'Exploratory Sample - EFA18:Factor5', label = 'ExpTable18_5', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable18_5, caption = 'Replication Sample - EFA18:Factor5', label = 'RepTable18_5', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable18_6, caption = 'Exploratory Sample - EFA18:Factor6', label = 'ExpTable18_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable18_6, caption = 'Replication Sample - EFA18:Factor6', label = 'RepTable18_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable18_7, caption = 'Exploratory Sample - EFA18:Factor7', label = 'ExpTable18_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable18_7, caption = 'Replication Sample - EFA18:Factor7', label = 'RepTable18_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable18_8, caption = 'Exploratory Sample - EFA18:Factor8', label = 'ExpTable18_8', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable18_8, caption = 'Replication Sample - EFA18:Factor8', label = 'RepTable18_8', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable18_9, caption = 'Exploratory Sample - EFA18:Factor9', label = 'ExpTable18_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable18_9, caption = 'Replication Sample - EFA18:Factor9', label = 'RepTable18_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable18_10, caption = 'Exploratory Sample - EFA18:Factor10', label = 'ExpTable18_10', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable18_10, caption = 'Replication Sample - EFA18:Factor10', label = 'RepTable18_10', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable18_11, caption = 'Exploratory Sample - EFA18:Factor11', label = 'ExpTable18_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable18_11, caption = 'Replication Sample - EFA18:Factor11', label = 'RepTable18_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable18_12, caption = 'Exploratory Sample - EFA18:Factor12', label = 'ExpTable18_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable18_12, caption = 'Replication Sample - EFA18:Factor12', label = 'RepTable18_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable18_13, caption = 'Exploratory Sample - EFA18:Factor13', label = 'ExpTable18_13', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable18_13, caption = 'Replication Sample - EFA18:Factor13', label = 'RepTable18_13', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable18_14, caption = 'Exploratory Sample - EFA18:Factor14', label = 'ExpTable18_14', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable18_14, caption = 'Replication Sample - EFA18:Factor14', label = 'RepTable18_14', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable18_15, caption = 'Exploratory Sample - EFA18:Factor15', label = 'ExpTable18_15', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable18_15, caption = 'Replication Sample - EFA18:Factor15', label = 'RepTable18_15', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable18_16, caption = 'Exploratory Sample - EFA18:Factor16', label = 'ExpTable18_16', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable18_16, caption = 'Replication Sample - EFA18:Factor16', label = 'RepTable18_16', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable18_17, caption = 'Exploratory Sample - EFA18:Factor17', label = 'ExpTable18_17', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable18_17, caption = 'Replication Sample - EFA18:Factor17', label = 'RepTable18_17', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable18_18, caption = 'Exploratory Sample - EFA18:Factor18', label = 'ExpTable18_18', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable18_18, caption = 'Replication Sample - EFA18:Factor18', label = 'RepTable18_18', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \title{EFA19} \begin{tiny} <>= print(xtable(ExpTable19_1, caption = 'Exploratory Sample - EFA19:Factor1', label = 'ExpTable19_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable19_1, caption = 'Replication Sample - EFA19:Factor1', label = 'RepTable19_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable19_2, caption = 'Exploratory Sample - EFA19:Factor2', label = 'ExpTable19_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable19_2, caption = 'Replication Sample - EFA19:Factor2', label = 'RepTable19_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable19_3, caption = 'Exploratory Sample - EFA19:Factor3', label = 'ExpTable19_3', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable19_3, caption = 'Replication Sample - EFA19:Factor3', label = 'RepTable19_3', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable19_4, caption = 'Exploratory Sample - EFA19:Factor4', label = 'ExpTable19_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable19_4, caption = 'Replication Sample - EFA19:Factor4', label = 'RepTable19_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable19_5, caption = 'Exploratory Sample - EFA19:Factor5', label = 'ExpTable19_5', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable19_5, caption = 'Replication Sample - EFA19:Factor5', label = 'RepTable19_5', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable19_6, caption = 'Exploratory Sample - EFA19:Factor6', label = 'ExpTable19_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable19_6, caption = 'Replication Sample - EFA19:Factor6', label = 'RepTable19_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable19_7, caption = 'Exploratory Sample - EFA19:Factor7', label = 'ExpTable19_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable19_7, caption = 'Replication Sample - EFA19:Factor7', label = 'RepTable19_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable19_8, caption = 'Exploratory Sample - EFA19:Factor8', label = 'ExpTable19_8', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable19_8, caption = 'Replication Sample - EFA19:Factor8', label = 'RepTable19_8', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable19_9, caption = 'Exploratory Sample - EFA19:Factor9', label = 'ExpTable19_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable19_9, caption = 'Replication Sample - EFA19:Factor9', label = 'RepTable19_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable19_10, caption = 'Exploratory Sample - EFA19:Factor10', label = 'ExpTable19_10', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable19_10, caption = 'Replication Sample - EFA19:Factor10', label = 'RepTable19_10', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable19_11, caption = 'Exploratory Sample - EFA19:Factor11', label = 'ExpTable19_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable19_11, caption = 'Replication Sample - EFA19:Factor11', label = 'RepTable19_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable19_12, caption = 'Exploratory Sample - EFA19:Factor12', label = 'ExpTable19_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable19_12, caption = 'Replication Sample - EFA19:Factor12', label = 'RepTable19_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable19_13, caption = 'Exploratory Sample - EFA19:Factor13', label = 'ExpTable19_13', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable19_13, caption = 'Replication Sample - EFA19:Factor13', label = 'RepTable19_13', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable19_14, caption = 'Exploratory Sample - EFA19:Factor14', label = 'ExpTable19_14', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable19_14, caption = 'Replication Sample - EFA19:Factor14', label = 'RepTable19_14', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable19_15, caption = 'Exploratory Sample - EFA19:Factor15', label = 'ExpTable19_15', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable19_15, caption = 'Replication Sample - EFA19:Factor15', label = 'RepTable19_15', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable19_16, caption = 'Exploratory Sample - EFA19:Factor16', label = 'ExpTable19_16', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable19_16, caption = 'Replication Sample - EFA19:Factor16', label = 'RepTable19_16', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable19_17, caption = 'Exploratory Sample - EFA19:Factor17', label = 'ExpTable19_17', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable19_17, caption = 'Replication Sample - EFA19:Factor17', label = 'RepTable19_17', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable19_18, caption = 'Exploratory Sample - EFA19:Factor18', label = 'ExpTable19_18', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable19_18, caption = 'Replication Sample - EFA19:Factor18', label = 'RepTable19_18', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable19_19, caption = 'Exploratory Sample - EFA19:Factor19', label = 'ExpTable19_19', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable19_19, caption = 'Replication Sample - EFA19:Factor19', label = 'RepTable19_19', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \title{EFA20} \begin{tiny} <>= print(xtable(ExpTable20_1, caption = 'Exploratory Sample - EFA20:Factor1', label = 'ExpTable20_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable20_1, caption = 'Replication Sample - EFA20:Factor1', label = 'RepTable20_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable20_2, caption = 'Exploratory Sample - EFA20:Factor2', label = 'ExpTable20_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable20_2, caption = 'Replication Sample - EFA20:Factor2', label = 'RepTable20_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable20_3, caption = 'Exploratory Sample - EFA20:Factor3', label = 'ExpTable20_3', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable20_3, caption = 'Replication Sample - EFA20:Factor3', label = 'RepTable20_3', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable20_4, caption = 'Exploratory Sample - EFA20:Factor4', label = 'ExpTable20_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable20_4, caption = 'Replication Sample - EFA20:Factor4', label = 'RepTable20_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable20_5, caption = 'Exploratory Sample - EFA20:Factor5', label = 'ExpTable20_5', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable20_5, caption = 'Replication Sample - EFA20:Factor5', label = 'RepTable20_5', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable20_6, caption = 'Exploratory Sample - EFA20:Factor6', label = 'ExpTable20_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable20_6, caption = 'Replication Sample - EFA20:Factor6', label = 'RepTable20_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable20_7, caption = 'Exploratory Sample - EFA20:Factor7', label = 'ExpTable20_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable20_7, caption = 'Replication Sample - EFA20:Factor7', label = 'RepTable20_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable20_8, caption = 'Exploratory Sample - EFA20:Factor8', label = 'ExpTable20_8', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable20_8, caption = 'Replication Sample - EFA20:Factor8', label = 'RepTable20_8', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable20_9, caption = 'Exploratory Sample - EFA20:Factor9', label = 'ExpTable20_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable20_9, caption = 'Replication Sample - EFA20:Factor9', label = 'RepTable20_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable20_10, caption = 'Exploratory Sample - EFA20:Factor10', label = 'ExpTable20_10', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable20_10, caption = 'Replication Sample - EFA20:Factor10', label = 'RepTable20_10', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable20_11, caption = 'Exploratory Sample - EFA20:Factor11', label = 'ExpTable20_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable20_11, caption = 'Replication Sample - EFA20:Factor11', label = 'RepTable20_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable20_12, caption = 'Exploratory Sample - EFA20:Factor12', label = 'ExpTable20_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable20_12, caption = 'Replication Sample - EFA20:Factor12', label = 'RepTable20_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable20_13, caption = 'Exploratory Sample - EFA20:Factor13', label = 'ExpTable20_13', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable20_13, caption = 'Replication Sample - EFA20:Factor13', label = 'RepTable20_13', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable20_14, caption = 'Exploratory Sample - EFA20:Factor14', label = 'ExpTable20_14', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable20_14, caption = 'Replication Sample - EFA20:Factor14', label = 'RepTable20_14', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable20_15, caption = 'Exploratory Sample - EFA20:Factor15', label = 'ExpTable20_15', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable20_15, caption = 'Replication Sample - EFA20:Factor15', label = 'RepTable20_15', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable20_16, caption = 'Exploratory Sample - EFA20:Factor16', label = 'ExpTable20_16', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable20_16, caption = 'Replication Sample - EFA20:Factor16', label = 'RepTable20_16', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable20_17, caption = 'Exploratory Sample - EFA20:Factor17', label = 'ExpTable20_17', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable20_17, caption = 'Replication Sample - EFA20:Factor17', label = 'RepTable20_17', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable20_18, caption = 'Exploratory Sample - EFA20:Factor18', label = 'ExpTable20_18', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable20_18, caption = 'Replication Sample - EFA20:Factor18', label = 'RepTable20_18', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable20_19, caption = 'Exploratory Sample - EFA20:Factor19', label = 'ExpTable20_19', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable20_19, caption = 'Replication Sample - EFA20:Factor19', label = 'RepTable20_19', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable20_20, caption = 'Exploratory Sample - EFA20:Factor20', label = 'ExpTable20_20', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable20_20, caption = 'Replication Sample - EFA20:Factor20', label = 'RepTable20_20', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \title{EFA21} \begin{tiny} <>= print(xtable(ExpTable21_1, caption = 'Exploratory Sample - EFA21:Factor1', label = 'ExpTable21_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable21_1, caption = 'Replication Sample - EFA21:Factor1', label = 'RepTable21_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable21_2, caption = 'Exploratory Sample - EFA21:Factor2', label = 'ExpTable21_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable21_2, caption = 'Replication Sample - EFA21:Factor2', label = 'RepTable21_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable21_3, caption = 'Exploratory Sample - EFA21:Factor3', label = 'ExpTable21_3', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable21_3, caption = 'Replication Sample - EFA21:Factor3', label = 'RepTable21_3', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable21_4, caption = 'Exploratory Sample - EFA21:Factor4', label = 'ExpTable21_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable21_4, caption = 'Replication Sample - EFA21:Factor4', label = 'RepTable21_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable21_5, caption = 'Exploratory Sample - EFA21:Factor5', label = 'ExpTable21_5', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable21_5, caption = 'Replication Sample - EFA21:Factor5', label = 'RepTable21_5', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable21_6, caption = 'Exploratory Sample - EFA21:Factor6', label = 'ExpTable21_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable21_6, caption = 'Replication Sample - EFA21:Factor6', label = 'RepTable21_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable21_7, caption = 'Exploratory Sample - EFA21:Factor7', label = 'ExpTable21_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable21_7, caption = 'Replication Sample - EFA21:Factor7', label = 'RepTable21_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable21_8, caption = 'Exploratory Sample - EFA21:Factor8', label = 'ExpTable21_8', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable21_8, caption = 'Replication Sample - EFA21:Factor8', label = 'RepTable21_8', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable21_9, caption = 'Exploratory Sample - EFA21:Factor9', label = 'ExpTable21_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable21_9, caption = 'Replication Sample - EFA21:Factor9', label = 'RepTable21_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable21_10, caption = 'Exploratory Sample - EFA21:Factor10', label = 'ExpTable21_10', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable21_10, caption = 'Replication Sample - EFA21:Factor10', label = 'RepTable21_10', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable21_11, caption = 'Exploratory Sample - EFA21:Factor11', label = 'ExpTable21_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable21_11, caption = 'Replication Sample - EFA21:Factor11', label = 'RepTable21_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable21_12, caption = 'Exploratory Sample - EFA21:Factor12', label = 'ExpTable21_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable21_12, caption = 'Replication Sample - EFA21:Factor12', label = 'RepTable21_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable21_13, caption = 'Exploratory Sample - EFA21:Factor13', label = 'ExpTable21_13', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable21_13, caption = 'Replication Sample - EFA21:Factor13', label = 'RepTable21_13', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable21_14, caption = 'Exploratory Sample - EFA21:Factor14', label = 'ExpTable21_14', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable21_14, caption = 'Replication Sample - EFA21:Factor14', label = 'RepTable21_14', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable21_15, caption = 'Exploratory Sample - EFA21:Factor15', label = 'ExpTable21_15', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable21_15, caption = 'Replication Sample - EFA21:Factor15', label = 'RepTable21_15', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable21_16, caption = 'Exploratory Sample - EFA21:Factor16', label = 'ExpTable21_16', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable21_16, caption = 'Replication Sample - EFA21:Factor16', label = 'RepTable21_16', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable21_17, caption = 'Exploratory Sample - EFA21:Factor17', label = 'ExpTable21_17', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable21_17, caption = 'Replication Sample - EFA21:Factor17', label = 'RepTable21_17', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable21_18, caption = 'Exploratory Sample - EFA21:Factor18', label = 'ExpTable21_18', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable21_18, caption = 'Replication Sample - EFA21:Factor18', label = 'RepTable21_18', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable21_19, caption = 'Exploratory Sample - EFA21:Factor19', label = 'ExpTable21_19', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable21_19, caption = 'Replication Sample - EFA21:Factor19', label = 'RepTable21_19', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable21_20, caption = 'Exploratory Sample - EFA21:Factor20', label = 'ExpTable21_20', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable21_20, caption = 'Replication Sample - EFA21:Factor20', label = 'RepTable21_20', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable21_21, caption = 'Exploratory Sample - EFA21:Factor21', label = 'ExpTable21_21', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable21_21, caption = 'Replication Sample - EFA21:Factor21', label = 'RepTable21_21', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \title{EFA22} \begin{tiny} <>= print(xtable(ExpTable22_1, caption = 'Exploratory Sample - EFA22:Factor1', label = 'ExpTable22_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable22_1, caption = 'Replication Sample - EFA22:Factor1', label = 'RepTable22_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable22_2, caption = 'Exploratory Sample - EFA22:Factor2', label = 'ExpTable22_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable22_2, caption = 'Replication Sample - EFA22:Factor2', label = 'RepTable22_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable22_3, caption = 'Exploratory Sample - EFA22:Factor3', label = 'ExpTable22_3', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable22_3, caption = 'Replication Sample - EFA22:Factor3', label = 'RepTable22_3', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable22_4, caption = 'Exploratory Sample - EFA22:Factor4', label = 'ExpTable22_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable22_4, caption = 'Replication Sample - EFA22:Factor4', label = 'RepTable22_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable22_5, caption = 'Exploratory Sample - EFA22:Factor5', label = 'ExpTable22_5', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable22_5, caption = 'Replication Sample - EFA22:Factor5', label = 'RepTable22_5', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable22_6, caption = 'Exploratory Sample - EFA22:Factor6', label = 'ExpTable22_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable22_6, caption = 'Replication Sample - EFA22:Factor6', label = 'RepTable22_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable22_7, caption = 'Exploratory Sample - EFA22:Factor7', label = 'ExpTable22_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable22_7, caption = 'Replication Sample - EFA22:Factor7', label = 'RepTable22_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable22_8, caption = 'Exploratory Sample - EFA22:Factor8', label = 'ExpTable22_8', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable22_8, caption = 'Replication Sample - EFA22:Factor8', label = 'RepTable22_8', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable22_9, caption = 'Exploratory Sample - EFA22:Factor9', label = 'ExpTable22_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable22_9, caption = 'Replication Sample - EFA22:Factor9', label = 'RepTable22_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable22_10, caption = 'Exploratory Sample - EFA22:Factor10', label = 'ExpTable22_10', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable22_10, caption = 'Replication Sample - EFA22:Factor10', label = 'RepTable22_10', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable22_11, caption = 'Exploratory Sample - EFA22:Factor11', label = 'ExpTable22_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable22_11, caption = 'Replication Sample - EFA22:Factor11', label = 'RepTable22_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable22_12, caption = 'Exploratory Sample - EFA22:Factor12', label = 'ExpTable22_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable22_12, caption = 'Replication Sample - EFA22:Factor12', label = 'RepTable22_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable22_13, caption = 'Exploratory Sample - EFA22:Factor13', label = 'ExpTable22_13', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable22_13, caption = 'Replication Sample - EFA22:Factor13', label = 'RepTable22_13', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable22_14, caption = 'Exploratory Sample - EFA22:Factor14', label = 'ExpTable22_14', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable22_14, caption = 'Replication Sample - EFA22:Factor14', label = 'RepTable22_14', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable22_15, caption = 'Exploratory Sample - EFA22:Factor15', label = 'ExpTable22_15', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable22_15, caption = 'Replication Sample - EFA22:Factor15', label = 'RepTable22_15', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable22_16, caption = 'Exploratory Sample - EFA22:Factor16', label = 'ExpTable22_16', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable22_16, caption = 'Replication Sample - EFA22:Factor16', label = 'RepTable22_16', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable22_17, caption = 'Exploratory Sample - EFA22:Factor17', label = 'ExpTable22_17', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable22_17, caption = 'Replication Sample - EFA22:Factor17', label = 'RepTable22_17', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable22_18, caption = 'Exploratory Sample - EFA22:Factor18', label = 'ExpTable22_18', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable22_18, caption = 'Replication Sample - EFA22:Factor18', label = 'RepTable22_18', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable22_19, caption = 'Exploratory Sample - EFA22:Factor19', label = 'ExpTable22_19', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable22_19, caption = 'Replication Sample - EFA22:Factor19', label = 'RepTable22_19', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable22_20, caption = 'Exploratory Sample - EFA22:Factor20', label = 'ExpTable22_20', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable22_20, caption = 'Replication Sample - EFA22:Factor20', label = 'RepTable22_20', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable22_21, caption = 'Exploratory Sample - EFA22:Factor21', label = 'ExpTable22_21', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable22_21, caption = 'Replication Sample - EFA22:Factor21', label = 'RepTable22_21', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable22_22, caption = 'Exploratory Sample - EFA22:Factor22', label = 'ExpTable22_22', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable22_22, caption = 'Replication Sample - EFA22:Factor22', label = 'RepTable22_22', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \title{EFA23} \begin{tiny} <>= print(xtable(ExpTable23_1, caption = 'Exploratory Sample - EFA23:Factor1', label = 'ExpTable23_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable23_1, caption = 'Replication Sample - EFA23:Factor1', label = 'RepTable23_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable23_2, caption = 'Exploratory Sample - EFA23:Factor2', label = 'ExpTable23_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable23_2, caption = 'Replication Sample - EFA23:Factor2', label = 'RepTable23_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable23_3, caption = 'Exploratory Sample - EFA23:Factor3', label = 'ExpTable23_3', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable23_3, caption = 'Replication Sample - EFA23:Factor3', label = 'RepTable23_3', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable23_4, caption = 'Exploratory Sample - EFA23:Factor4', label = 'ExpTable23_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable23_4, caption = 'Replication Sample - EFA23:Factor4', label = 'RepTable23_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable23_5, caption = 'Exploratory Sample - EFA23:Factor5', label = 'ExpTable23_5', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable23_5, caption = 'Replication Sample - EFA23:Factor5', label = 'RepTable23_5', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable23_6, caption = 'Exploratory Sample - EFA23:Factor6', label = 'ExpTable23_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable23_6, caption = 'Replication Sample - EFA23:Factor6', label = 'RepTable23_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable23_7, caption = 'Exploratory Sample - EFA23:Factor7', label = 'ExpTable23_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable23_7, caption = 'Replication Sample - EFA23:Factor7', label = 'RepTable23_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable23_8, caption = 'Exploratory Sample - EFA23:Factor8', label = 'ExpTable23_8', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable23_8, caption = 'Replication Sample - EFA23:Factor8', label = 'RepTable23_8', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable23_9, caption = 'Exploratory Sample - EFA23:Factor9', label = 'ExpTable23_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable23_9, caption = 'Replication Sample - EFA23:Factor9', label = 'RepTable23_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable23_10, caption = 'Exploratory Sample - EFA23:Factor10', label = 'ExpTable23_10', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable23_10, caption = 'Replication Sample - EFA23:Factor10', label = 'RepTable23_10', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable23_11, caption = 'Exploratory Sample - EFA23:Factor11', label = 'ExpTable23_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable23_11, caption = 'Replication Sample - EFA23:Factor11', label = 'RepTable23_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable23_12, caption = 'Exploratory Sample - EFA23:Factor12', label = 'ExpTable23_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable23_12, caption = 'Replication Sample - EFA23:Factor12', label = 'RepTable23_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable23_13, caption = 'Exploratory Sample - EFA23:Factor13', label = 'ExpTable23_13', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable23_13, caption = 'Replication Sample - EFA23:Factor13', label = 'RepTable23_13', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable23_14, caption = 'Exploratory Sample - EFA23:Factor14', label = 'ExpTable23_14', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable23_14, caption = 'Replication Sample - EFA23:Factor14', label = 'RepTable23_14', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable23_15, caption = 'Exploratory Sample - EFA23:Factor15', label = 'ExpTable23_15', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable23_15, caption = 'Replication Sample - EFA23:Factor15', label = 'RepTable23_15', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable23_16, caption = 'Exploratory Sample - EFA23:Factor16', label = 'ExpTable23_16', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable23_16, caption = 'Replication Sample - EFA23:Factor16', label = 'RepTable23_16', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable23_17, caption = 'Exploratory Sample - EFA23:Factor17', label = 'ExpTable23_17', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable23_17, caption = 'Replication Sample - EFA23:Factor17', label = 'RepTable23_17', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable23_18, caption = 'Exploratory Sample - EFA23:Factor18', label = 'ExpTable23_18', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable23_18, caption = 'Replication Sample - EFA23:Factor18', label = 'RepTable23_18', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable23_19, caption = 'Exploratory Sample - EFA23:Factor19', label = 'ExpTable23_19', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable23_19, caption = 'Replication Sample - EFA23:Factor19', label = 'RepTable23_19', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable23_20, caption = 'Exploratory Sample - EFA23:Factor20', label = 'ExpTable23_20', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable23_20, caption = 'Replication Sample - EFA23:Factor20', label = 'RepTable23_20', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable23_21, caption = 'Exploratory Sample - EFA23:Factor21', label = 'ExpTable23_21', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable23_21, caption = 'Replication Sample - EFA23:Factor21', label = 'RepTable23_21', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable23_22, caption = 'Exploratory Sample - EFA23:Factor22', label = 'ExpTable23_22', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable23_22, caption = 'Replication Sample - EFA23:Factor22', label = 'RepTable23_22', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable23_23, caption = 'Exploratory Sample - EFA23:Factor23', label = 'ExpTable23_23', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable23_23, caption = 'Replication Sample - EFA23:Factor23', label = 'RepTable23_23', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \title{EFA24} \begin{tiny} <>= print(xtable(ExpTable24_1, caption = 'Exploratory Sample - EFA24:Factor1', label = 'ExpTable24_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable24_1, caption = 'Replication Sample - EFA24:Factor1', label = 'RepTable24_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable24_2, caption = 'Exploratory Sample - EFA24:Factor2', label = 'ExpTable24_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable24_2, caption = 'Replication Sample - EFA24:Factor2', label = 'RepTable24_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable24_3, caption = 'Exploratory Sample - EFA24:Factor3', label = 'ExpTable24_3', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable24_3, caption = 'Replication Sample - EFA24:Factor3', label = 'RepTable24_3', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable24_4, caption = 'Exploratory Sample - EFA24:Factor4', label = 'ExpTable24_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable24_4, caption = 'Replication Sample - EFA24:Factor4', label = 'RepTable24_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable24_5, caption = 'Exploratory Sample - EFA24:Factor5', label = 'ExpTable24_5', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable24_5, caption = 'Replication Sample - EFA24:Factor5', label = 'RepTable24_5', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable24_6, caption = 'Exploratory Sample - EFA24:Factor6', label = 'ExpTable24_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable24_6, caption = 'Replication Sample - EFA24:Factor6', label = 'RepTable24_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable24_7, caption = 'Exploratory Sample - EFA24:Factor7', label = 'ExpTable24_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable24_7, caption = 'Replication Sample - EFA24:Factor7', label = 'RepTable24_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable24_8, caption = 'Exploratory Sample - EFA24:Factor8', label = 'ExpTable24_8', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable24_8, caption = 'Replication Sample - EFA24:Factor8', label = 'RepTable24_8', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable24_9, caption = 'Exploratory Sample - EFA24:Factor9', label = 'ExpTable24_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable24_9, caption = 'Replication Sample - EFA24:Factor9', label = 'RepTable24_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable24_10, caption = 'Exploratory Sample - EFA24:Factor10', label = 'ExpTable24_10', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable24_10, caption = 'Replication Sample - EFA24:Factor10', label = 'RepTable24_10', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable24_11, caption = 'Exploratory Sample - EFA24:Factor11', label = 'ExpTable24_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable24_11, caption = 'Replication Sample - EFA24:Factor11', label = 'RepTable24_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable24_12, caption = 'Exploratory Sample - EFA24:Factor12', label = 'ExpTable24_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable24_12, caption = 'Replication Sample - EFA24:Factor12', label = 'RepTable24_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable24_13, caption = 'Exploratory Sample - EFA24:Factor13', label = 'ExpTable24_13', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable24_13, caption = 'Replication Sample - EFA24:Factor13', label = 'RepTable24_13', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable24_14, caption = 'Exploratory Sample - EFA24:Factor14', label = 'ExpTable24_14', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable24_14, caption = 'Replication Sample - EFA24:Factor14', label = 'RepTable24_14', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable24_15, caption = 'Exploratory Sample - EFA24:Factor15', label = 'ExpTable24_15', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable24_15, caption = 'Replication Sample - EFA24:Factor15', label = 'RepTable24_15', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable24_16, caption = 'Exploratory Sample - EFA24:Factor16', label = 'ExpTable24_16', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable24_16, caption = 'Replication Sample - EFA24:Factor16', label = 'RepTable24_16', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable24_17, caption = 'Exploratory Sample - EFA24:Factor17', label = 'ExpTable24_17', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable24_17, caption = 'Replication Sample - EFA24:Factor17', label = 'RepTable24_17', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable24_18, caption = 'Exploratory Sample - EFA24:Factor18', label = 'ExpTable24_18', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable24_18, caption = 'Replication Sample - EFA24:Factor18', label = 'RepTable24_18', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable24_19, caption = 'Exploratory Sample - EFA24:Factor19', label = 'ExpTable24_19', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable24_19, caption = 'Replication Sample - EFA24:Factor19', label = 'RepTable24_19', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable24_20, caption = 'Exploratory Sample - EFA24:Factor20', label = 'ExpTable24_20', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable24_20, caption = 'Replication Sample - EFA24:Factor20', label = 'RepTable24_20', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable24_21, caption = 'Exploratory Sample - EFA24:Factor21', label = 'ExpTable24_21', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable24_21, caption = 'Replication Sample - EFA24:Factor21', label = 'RepTable24_21', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable24_22, caption = 'Exploratory Sample - EFA24:Factor22', label = 'ExpTable24_22', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable24_22, caption = 'Replication Sample - EFA24:Factor22', label = 'RepTable24_22', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable24_23, caption = 'Exploratory Sample - EFA24:Factor23', label = 'ExpTable24_23', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable24_23, caption = 'Replication Sample - EFA24:Factor23', label = 'RepTable24_23', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable24_24, caption = 'Exploratory Sample - EFA24:Factor24', label = 'ExpTable24_24', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable24_24, caption = 'Replication Sample - EFA24:Factor24', label = 'RepTable24_24', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \title{EFA25} \begin{tiny} <>= print(xtable(ExpTable25_1, caption = 'Exploratory Sample - EFA25:Factor1', label = 'ExpTable25_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable25_1, caption = 'Replication Sample - EFA25:Factor1', label = 'RepTable25_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable25_2, caption = 'Exploratory Sample - EFA25:Factor2', label = 'ExpTable25_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable25_2, caption = 'Replication Sample - EFA25:Factor2', label = 'RepTable25_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable25_3, caption = 'Exploratory Sample - EFA25:Factor3', label = 'ExpTable25_3', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable25_3, caption = 'Replication Sample - EFA25:Factor3', label = 'RepTable25_3', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable25_4, caption = 'Exploratory Sample - EFA25:Factor4', label = 'ExpTable25_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable25_4, caption = 'Replication Sample - EFA25:Factor4', label = 'RepTable25_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable25_5, caption = 'Exploratory Sample - EFA25:Factor5', label = 'ExpTable25_5', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable25_5, caption = 'Replication Sample - EFA25:Factor5', label = 'RepTable25_5', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable25_6, caption = 'Exploratory Sample - EFA25:Factor6', label = 'ExpTable25_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable25_6, caption = 'Replication Sample - EFA25:Factor6', label = 'RepTable25_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable25_7, caption = 'Exploratory Sample - EFA25:Factor7', label = 'ExpTable25_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable25_7, caption = 'Replication Sample - EFA25:Factor7', label = 'RepTable25_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable25_8, caption = 'Exploratory Sample - EFA25:Factor8', label = 'ExpTable25_8', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable25_8, caption = 'Replication Sample - EFA25:Factor8', label = 'RepTable25_8', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable25_9, caption = 'Exploratory Sample - EFA25:Factor9', label = 'ExpTable25_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable25_9, caption = 'Replication Sample - EFA25:Factor9', label = 'RepTable25_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable25_10, caption = 'Exploratory Sample - EFA25:Factor10', label = 'ExpTable25_10', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable25_10, caption = 'Replication Sample - EFA25:Factor10', label = 'RepTable25_10', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable25_11, caption = 'Exploratory Sample - EFA25:Factor11', label = 'ExpTable25_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable25_11, caption = 'Replication Sample - EFA25:Factor11', label = 'RepTable25_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable25_12, caption = 'Exploratory Sample - EFA25:Factor12', label = 'ExpTable25_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable25_12, caption = 'Replication Sample - EFA25:Factor12', label = 'RepTable25_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable25_13, caption = 'Exploratory Sample - EFA25:Factor13', label = 'ExpTable25_13', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable25_13, caption = 'Replication Sample - EFA25:Factor13', label = 'RepTable25_13', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable25_14, caption = 'Exploratory Sample - EFA25:Factor14', label = 'ExpTable25_14', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable25_14, caption = 'Replication Sample - EFA25:Factor14', label = 'RepTable25_14', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable25_15, caption = 'Exploratory Sample - EFA25:Factor15', label = 'ExpTable25_15', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable25_15, caption = 'Replication Sample - EFA25:Factor15', label = 'RepTable25_15', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable25_16, caption = 'Exploratory Sample - EFA25:Factor16', label = 'ExpTable25_16', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable25_16, caption = 'Replication Sample - EFA25:Factor16', label = 'RepTable25_16', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable25_17, caption = 'Exploratory Sample - EFA25:Factor17', label = 'ExpTable25_17', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable25_17, caption = 'Replication Sample - EFA25:Factor17', label = 'RepTable25_17', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable25_18, caption = 'Exploratory Sample - EFA25:Factor18', label = 'ExpTable25_18', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable25_18, caption = 'Replication Sample - EFA25:Factor18', label = 'RepTable25_18', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable25_19, caption = 'Exploratory Sample - EFA25:Factor19', label = 'ExpTable25_19', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable25_19, caption = 'Replication Sample - EFA25:Factor19', label = 'RepTable25_19', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable25_20, caption = 'Exploratory Sample - EFA25:Factor20', label = 'ExpTable25_20', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable25_20, caption = 'Replication Sample - EFA25:Factor20', label = 'RepTable25_20', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable25_21, caption = 'Exploratory Sample - EFA25:Factor21', label = 'ExpTable25_21', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable25_21, caption = 'Replication Sample - EFA25:Factor21', label = 'RepTable25_21', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable25_22, caption = 'Exploratory Sample - EFA25:Factor22', label = 'ExpTable25_22', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable25_22, caption = 'Replication Sample - EFA25:Factor22', label = 'RepTable25_22', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable25_23, caption = 'Exploratory Sample - EFA25:Factor23', label = 'ExpTable25_23', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable25_23, caption = 'Replication Sample - EFA25:Factor23', label = 'RepTable25_23', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable25_24, caption = 'Exploratory Sample - EFA25:Factor24', label = 'ExpTable25_24', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable25_24, caption = 'Replication Sample - EFA25:Factor24', label = 'RepTable25_24', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable25_25, caption = 'Exploratory Sample - EFA25:Factor25', label = 'ExpTable25_25', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable25_25, caption = 'Replication Sample - EFA25:Factor25', label = 'RepTable25_25', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \title{EFA26} \begin{tiny} <>= print(xtable(ExpTable26_1, caption = 'Exploratory Sample - EFA26:Factor1', label = 'ExpTable26_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable26_1, caption = 'Replication Sample - EFA26:Factor1', label = 'RepTable26_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable26_2, caption = 'Exploratory Sample - EFA26:Factor2', label = 'ExpTable26_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable26_2, caption = 'Replication Sample - EFA26:Factor2', label = 'RepTable26_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable26_3, caption = 'Exploratory Sample - EFA26:Factor3', label = 'ExpTable26_3', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable26_3, caption = 'Replication Sample - EFA26:Factor3', label = 'RepTable26_3', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable26_4, caption = 'Exploratory Sample - EFA26:Factor4', label = 'ExpTable26_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable26_4, caption = 'Replication Sample - EFA26:Factor4', label = 'RepTable26_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable26_5, caption = 'Exploratory Sample - EFA26:Factor5', label = 'ExpTable26_5', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable26_5, caption = 'Replication Sample - EFA26:Factor5', label = 'RepTable26_5', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable26_6, caption = 'Exploratory Sample - EFA26:Factor6', label = 'ExpTable26_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable26_6, caption = 'Replication Sample - EFA26:Factor6', label = 'RepTable26_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable26_7, caption = 'Exploratory Sample - EFA26:Factor7', label = 'ExpTable26_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable26_7, caption = 'Replication Sample - EFA26:Factor7', label = 'RepTable26_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable26_8, caption = 'Exploratory Sample - EFA26:Factor8', label = 'ExpTable26_8', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable26_8, caption = 'Replication Sample - EFA26:Factor8', label = 'RepTable26_8', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable26_9, caption = 'Exploratory Sample - EFA26:Factor9', label = 'ExpTable26_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable26_9, caption = 'Replication Sample - EFA26:Factor9', label = 'RepTable26_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable26_10, caption = 'Exploratory Sample - EFA26:Factor10', label = 'ExpTable26_10', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable26_10, caption = 'Replication Sample - EFA26:Factor10', label = 'RepTable26_10', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable26_11, caption = 'Exploratory Sample - EFA26:Factor11', label = 'ExpTable26_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable26_11, caption = 'Replication Sample - EFA26:Factor11', label = 'RepTable26_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable26_12, caption = 'Exploratory Sample - EFA26:Factor12', label = 'ExpTable26_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable26_12, caption = 'Replication Sample - EFA26:Factor12', label = 'RepTable26_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable26_13, caption = 'Exploratory Sample - EFA26:Factor13', label = 'ExpTable26_13', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable26_13, caption = 'Replication Sample - EFA26:Factor13', label = 'RepTable26_13', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable26_14, caption = 'Exploratory Sample - EFA26:Factor14', label = 'ExpTable26_14', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable26_14, caption = 'Replication Sample - EFA26:Factor14', label = 'RepTable26_14', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable26_15, caption = 'Exploratory Sample - EFA26:Factor15', label = 'ExpTable26_15', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable26_15, caption = 'Replication Sample - EFA26:Factor15', label = 'RepTable26_15', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable26_16, caption = 'Exploratory Sample - EFA26:Factor16', label = 'ExpTable26_16', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable26_16, caption = 'Replication Sample - EFA26:Factor16', label = 'RepTable26_16', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable26_17, caption = 'Exploratory Sample - EFA26:Factor17', label = 'ExpTable26_17', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable26_17, caption = 'Replication Sample - EFA26:Factor17', label = 'RepTable26_17', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable26_18, caption = 'Exploratory Sample - EFA26:Factor18', label = 'ExpTable26_18', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable26_18, caption = 'Replication Sample - EFA26:Factor18', label = 'RepTable26_18', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable26_19, caption = 'Exploratory Sample - EFA26:Factor19', label = 'ExpTable26_19', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable26_19, caption = 'Replication Sample - EFA26:Factor19', label = 'RepTable26_19', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable26_20, caption = 'Exploratory Sample - EFA26:Factor20', label = 'ExpTable26_20', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable26_20, caption = 'Replication Sample - EFA26:Factor20', label = 'RepTable26_20', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable26_21, caption = 'Exploratory Sample - EFA26:Factor21', label = 'ExpTable26_21', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable26_21, caption = 'Replication Sample - EFA26:Factor21', label = 'RepTable26_21', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable26_22, caption = 'Exploratory Sample - EFA26:Factor22', label = 'ExpTable26_22', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable26_22, caption = 'Replication Sample - EFA26:Factor22', label = 'RepTable26_22', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable26_23, caption = 'Exploratory Sample - EFA26:Factor23', label = 'ExpTable26_23', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable26_23, caption = 'Replication Sample - EFA26:Factor23', label = 'RepTable26_23', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable26_24, caption = 'Exploratory Sample - EFA26:Factor24', label = 'ExpTable26_24', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable26_24, caption = 'Replication Sample - EFA26:Factor24', label = 'RepTable26_24', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable26_25, caption = 'Exploratory Sample - EFA26:Factor25', label = 'ExpTable26_25', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable26_25, caption = 'Replication Sample - EFA26:Factor25', label = 'RepTable26_25', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable26_26, caption = 'Exploratory Sample - EFA26:Factor26', label = 'ExpTable26_26', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable26_26, caption = 'Replication Sample - EFA26:Factor26', label = 'RepTable26_26', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \title{EFA27} \begin{tiny} <>= print(xtable(ExpTable27_1, caption = 'Exploratory Sample - EFA27:Factor1', label = 'ExpTable27_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable27_1, caption = 'Replication Sample - EFA27:Factor1', label = 'RepTable27_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable27_2, caption = 'Exploratory Sample - EFA27:Factor2', label = 'ExpTable27_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable27_2, caption = 'Replication Sample - EFA27:Factor2', label = 'RepTable27_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable27_3, caption = 'Exploratory Sample - EFA27:Factor3', label = 'ExpTable27_3', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable27_3, caption = 'Replication Sample - EFA27:Factor3', label = 'RepTable27_3', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable27_4, caption = 'Exploratory Sample - EFA27:Factor4', label = 'ExpTable27_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable27_4, caption = 'Replication Sample - EFA27:Factor4', label = 'RepTable27_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable27_5, caption = 'Exploratory Sample - EFA27:Factor5', label = 'ExpTable27_5', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable27_5, caption = 'Replication Sample - EFA27:Factor5', label = 'RepTable27_5', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable27_6, caption = 'Exploratory Sample - EFA27:Factor6', label = 'ExpTable27_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable27_6, caption = 'Replication Sample - EFA27:Factor6', label = 'RepTable27_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable27_7, caption = 'Exploratory Sample - EFA27:Factor7', label = 'ExpTable27_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable27_7, caption = 'Replication Sample - EFA27:Factor7', label = 'RepTable27_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable27_8, caption = 'Exploratory Sample - EFA27:Factor8', label = 'ExpTable27_8', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable27_8, caption = 'Replication Sample - EFA27:Factor8', label = 'RepTable27_8', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable27_9, caption = 'Exploratory Sample - EFA27:Factor9', label = 'ExpTable27_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable27_9, caption = 'Replication Sample - EFA27:Factor9', label = 'RepTable27_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable27_10, caption = 'Exploratory Sample - EFA27:Factor10', label = 'ExpTable27_10', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable27_10, caption = 'Replication Sample - EFA27:Factor10', label = 'RepTable27_10', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable27_11, caption = 'Exploratory Sample - EFA27:Factor11', label = 'ExpTable27_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable27_11, caption = 'Replication Sample - EFA27:Factor11', label = 'RepTable27_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable27_12, caption = 'Exploratory Sample - EFA27:Factor12', label = 'ExpTable27_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable27_12, caption = 'Replication Sample - EFA27:Factor12', label = 'RepTable27_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable27_13, caption = 'Exploratory Sample - EFA27:Factor13', label = 'ExpTable27_13', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable27_13, caption = 'Replication Sample - EFA27:Factor13', label = 'RepTable27_13', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable27_14, caption = 'Exploratory Sample - EFA27:Factor14', label = 'ExpTable27_14', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable27_14, caption = 'Replication Sample - EFA27:Factor14', label = 'RepTable27_14', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable27_15, caption = 'Exploratory Sample - EFA27:Factor15', label = 'ExpTable27_15', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable27_15, caption = 'Replication Sample - EFA27:Factor15', label = 'RepTable27_15', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable27_16, caption = 'Exploratory Sample - EFA27:Factor16', label = 'ExpTable27_16', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable27_16, caption = 'Replication Sample - EFA27:Factor16', label = 'RepTable27_16', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable27_17, caption = 'Exploratory Sample - EFA27:Factor17', label = 'ExpTable27_17', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable27_17, caption = 'Replication Sample - EFA27:Factor17', label = 'RepTable27_17', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable27_18, caption = 'Exploratory Sample - EFA27:Factor18', label = 'ExpTable27_18', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable27_18, caption = 'Replication Sample - EFA27:Factor18', label = 'RepTable27_18', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable27_19, caption = 'Exploratory Sample - EFA27:Factor19', label = 'ExpTable27_19', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable27_19, caption = 'Replication Sample - EFA27:Factor19', label = 'RepTable27_19', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable27_20, caption = 'Exploratory Sample - EFA27:Factor20', label = 'ExpTable27_20', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable27_20, caption = 'Replication Sample - EFA27:Factor20', label = 'RepTable27_20', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable27_21, caption = 'Exploratory Sample - EFA27:Factor21', label = 'ExpTable27_21', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable27_21, caption = 'Replication Sample - EFA27:Factor21', label = 'RepTable27_21', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable27_22, caption = 'Exploratory Sample - EFA27:Factor22', label = 'ExpTable27_22', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable27_22, caption = 'Replication Sample - EFA27:Factor22', label = 'RepTable27_22', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable27_23, caption = 'Exploratory Sample - EFA27:Factor23', label = 'ExpTable27_23', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable27_23, caption = 'Replication Sample - EFA27:Factor23', label = 'RepTable27_23', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable27_24, caption = 'Exploratory Sample - EFA27:Factor24', label = 'ExpTable27_24', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable27_24, caption = 'Replication Sample - EFA27:Factor24', label = 'RepTable27_24', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable27_25, caption = 'Exploratory Sample - EFA27:Factor25', label = 'ExpTable27_25', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable27_25, caption = 'Replication Sample - EFA27:Factor25', label = 'RepTable27_25', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable27_26, caption = 'Exploratory Sample - EFA27:Factor26', label = 'ExpTable27_26', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable27_26, caption = 'Replication Sample - EFA27:Factor26', label = 'RepTable27_26', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable27_27, caption = 'Exploratory Sample - EFA27:Factor27', label = 'ExpTable27_27', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable27_27, caption = 'Replication Sample - EFA27:Factor27', label = 'RepTable27_27', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \title{EFA28} \begin{tiny} <>= print(xtable(ExpTable28_1, caption = 'Exploratory Sample - EFA28:Factor1', label = 'ExpTable28_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable28_1, caption = 'Replication Sample - EFA28:Factor1', label = 'RepTable28_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable28_2, caption = 'Exploratory Sample - EFA28:Factor2', label = 'ExpTable28_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable28_2, caption = 'Replication Sample - EFA28:Factor2', label = 'RepTable28_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable28_3, caption = 'Exploratory Sample - EFA28:Factor3', label = 'ExpTable28_3', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable28_3, caption = 'Replication Sample - EFA28:Factor3', label = 'RepTable28_3', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable28_4, caption = 'Exploratory Sample - EFA28:Factor4', label = 'ExpTable28_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable28_4, caption = 'Replication Sample - EFA28:Factor4', label = 'RepTable28_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable28_5, caption = 'Exploratory Sample - EFA28:Factor5', label = 'ExpTable28_5', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable28_5, caption = 'Replication Sample - EFA28:Factor5', label = 'RepTable28_5', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable28_6, caption = 'Exploratory Sample - EFA28:Factor6', label = 'ExpTable28_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable28_6, caption = 'Replication Sample - EFA28:Factor6', label = 'RepTable28_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable28_7, caption = 'Exploratory Sample - EFA28:Factor7', label = 'ExpTable28_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable28_7, caption = 'Replication Sample - EFA28:Factor7', label = 'RepTable28_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable28_8, caption = 'Exploratory Sample - EFA28:Factor8', label = 'ExpTable28_8', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable28_8, caption = 'Replication Sample - EFA28:Factor8', label = 'RepTable28_8', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable28_9, caption = 'Exploratory Sample - EFA28:Factor9', label = 'ExpTable28_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable28_9, caption = 'Replication Sample - EFA28:Factor9', label = 'RepTable28_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable28_10, caption = 'Exploratory Sample - EFA28:Factor10', label = 'ExpTable28_10', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable28_10, caption = 'Replication Sample - EFA28:Factor10', label = 'RepTable28_10', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable28_11, caption = 'Exploratory Sample - EFA28:Factor11', label = 'ExpTable28_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable28_11, caption = 'Replication Sample - EFA28:Factor11', label = 'RepTable28_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable28_12, caption = 'Exploratory Sample - EFA28:Factor12', label = 'ExpTable28_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable28_12, caption = 'Replication Sample - EFA28:Factor12', label = 'RepTable28_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable28_13, caption = 'Exploratory Sample - EFA28:Factor13', label = 'ExpTable28_13', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable28_13, caption = 'Replication Sample - EFA28:Factor13', label = 'RepTable28_13', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable28_14, caption = 'Exploratory Sample - EFA28:Factor14', label = 'ExpTable28_14', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable28_14, caption = 'Replication Sample - EFA28:Factor14', label = 'RepTable28_14', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable28_15, caption = 'Exploratory Sample - EFA28:Factor15', label = 'ExpTable28_15', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable28_15, caption = 'Replication Sample - EFA28:Factor15', label = 'RepTable28_15', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable28_16, caption = 'Exploratory Sample - EFA28:Factor16', label = 'ExpTable28_16', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable28_16, caption = 'Replication Sample - EFA28:Factor16', label = 'RepTable28_16', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable28_17, caption = 'Exploratory Sample - EFA28:Factor17', label = 'ExpTable28_17', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable28_17, caption = 'Replication Sample - EFA28:Factor17', label = 'RepTable28_17', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable28_18, caption = 'Exploratory Sample - EFA28:Factor18', label = 'ExpTable28_18', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable28_18, caption = 'Replication Sample - EFA28:Factor18', label = 'RepTable28_18', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable28_19, caption = 'Exploratory Sample - EFA28:Factor19', label = 'ExpTable28_19', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable28_19, caption = 'Replication Sample - EFA28:Factor19', label = 'RepTable28_19', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable28_20, caption = 'Exploratory Sample - EFA28:Factor20', label = 'ExpTable28_20', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable28_20, caption = 'Replication Sample - EFA28:Factor20', label = 'RepTable28_20', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable28_21, caption = 'Exploratory Sample - EFA28:Factor21', label = 'ExpTable28_21', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable28_21, caption = 'Replication Sample - EFA28:Factor21', label = 'RepTable28_21', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable28_22, caption = 'Exploratory Sample - EFA28:Factor22', label = 'ExpTable28_22', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable28_22, caption = 'Replication Sample - EFA28:Factor22', label = 'RepTable28_22', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable28_23, caption = 'Exploratory Sample - EFA28:Factor23', label = 'ExpTable28_23', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable28_23, caption = 'Replication Sample - EFA28:Factor23', label = 'RepTable28_23', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable28_24, caption = 'Exploratory Sample - EFA28:Factor24', label = 'ExpTable28_24', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable28_24, caption = 'Replication Sample - EFA28:Factor24', label = 'RepTable28_24', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable28_25, caption = 'Exploratory Sample - EFA28:Factor25', label = 'ExpTable28_25', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable28_25, caption = 'Replication Sample - EFA28:Factor25', label = 'RepTable28_25', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable28_26, caption = 'Exploratory Sample - EFA28:Factor26', label = 'ExpTable28_26', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable28_26, caption = 'Replication Sample - EFA28:Factor26', label = 'RepTable28_26', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable28_27, caption = 'Exploratory Sample - EFA28:Factor27', label = 'ExpTable28_27', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable28_27, caption = 'Replication Sample - EFA28:Factor27', label = 'RepTable28_27', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable28_28, caption = 'Exploratory Sample - EFA28:Factor28', label = 'ExpTable28_28', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable28_28, caption = 'Replication Sample - EFA28:Factor28', label = 'RepTable28_28', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \title{EFA29} \begin{tiny} <>= print(xtable(ExpTable29_1, caption = 'Exploratory Sample - EFA29:Factor1', label = 'ExpTable29_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable29_1, caption = 'Replication Sample - EFA29:Factor1', label = 'RepTable29_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable29_2, caption = 'Exploratory Sample - EFA29:Factor2', label = 'ExpTable29_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable29_2, caption = 'Replication Sample - EFA29:Factor2', label = 'RepTable29_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable29_3, caption = 'Exploratory Sample - EFA29:Factor3', label = 'ExpTable29_3', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable29_3, caption = 'Replication Sample - EFA29:Factor3', label = 'RepTable29_3', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable29_4, caption = 'Exploratory Sample - EFA29:Factor4', label = 'ExpTable29_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable29_4, caption = 'Replication Sample - EFA29:Factor4', label = 'RepTable29_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable29_5, caption = 'Exploratory Sample - EFA29:Factor5', label = 'ExpTable29_5', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable29_5, caption = 'Replication Sample - EFA29:Factor5', label = 'RepTable29_5', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable29_6, caption = 'Exploratory Sample - EFA29:Factor6', label = 'ExpTable29_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable29_6, caption = 'Replication Sample - EFA29:Factor6', label = 'RepTable29_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable29_7, caption = 'Exploratory Sample - EFA29:Factor7', label = 'ExpTable29_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable29_7, caption = 'Replication Sample - EFA29:Factor7', label = 'RepTable29_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable29_8, caption = 'Exploratory Sample - EFA29:Factor8', label = 'ExpTable29_8', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable29_8, caption = 'Replication Sample - EFA29:Factor8', label = 'RepTable29_8', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable29_9, caption = 'Exploratory Sample - EFA29:Factor9', label = 'ExpTable29_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable29_9, caption = 'Replication Sample - EFA29:Factor9', label = 'RepTable29_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable29_10, caption = 'Exploratory Sample - EFA29:Factor10', label = 'ExpTable29_10', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable29_10, caption = 'Replication Sample - EFA29:Factor10', label = 'RepTable29_10', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable29_11, caption = 'Exploratory Sample - EFA29:Factor11', label = 'ExpTable29_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable29_11, caption = 'Replication Sample - EFA29:Factor11', label = 'RepTable29_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable29_12, caption = 'Exploratory Sample - EFA29:Factor12', label = 'ExpTable29_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable29_12, caption = 'Replication Sample - EFA29:Factor12', label = 'RepTable29_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable29_13, caption = 'Exploratory Sample - EFA29:Factor13', label = 'ExpTable29_13', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable29_13, caption = 'Replication Sample - EFA29:Factor13', label = 'RepTable29_13', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable29_14, caption = 'Exploratory Sample - EFA29:Factor14', label = 'ExpTable29_14', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable29_14, caption = 'Replication Sample - EFA29:Factor14', label = 'RepTable29_14', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable29_15, caption = 'Exploratory Sample - EFA29:Factor15', label = 'ExpTable29_15', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable29_15, caption = 'Replication Sample - EFA29:Factor15', label = 'RepTable29_15', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable29_16, caption = 'Exploratory Sample - EFA29:Factor16', label = 'ExpTable29_16', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable29_16, caption = 'Replication Sample - EFA29:Factor16', label = 'RepTable29_16', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable29_17, caption = 'Exploratory Sample - EFA29:Factor17', label = 'ExpTable29_17', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable29_17, caption = 'Replication Sample - EFA29:Factor17', label = 'RepTable29_17', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable29_18, caption = 'Exploratory Sample - EFA29:Factor18', label = 'ExpTable29_18', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable29_18, caption = 'Replication Sample - EFA29:Factor18', label = 'RepTable29_18', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable29_19, caption = 'Exploratory Sample - EFA29:Factor19', label = 'ExpTable29_19', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable29_19, caption = 'Replication Sample - EFA29:Factor19', label = 'RepTable29_19', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable29_20, caption = 'Exploratory Sample - EFA29:Factor20', label = 'ExpTable29_20', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable29_20, caption = 'Replication Sample - EFA29:Factor20', label = 'RepTable29_20', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable29_21, caption = 'Exploratory Sample - EFA29:Factor21', label = 'ExpTable29_21', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable29_21, caption = 'Replication Sample - EFA29:Factor21', label = 'RepTable29_21', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable29_22, caption = 'Exploratory Sample - EFA29:Factor22', label = 'ExpTable29_22', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable29_22, caption = 'Replication Sample - EFA29:Factor22', label = 'RepTable29_22', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable29_23, caption = 'Exploratory Sample - EFA29:Factor23', label = 'ExpTable29_23', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable29_23, caption = 'Replication Sample - EFA29:Factor23', label = 'RepTable29_23', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable29_24, caption = 'Exploratory Sample - EFA29:Factor24', label = 'ExpTable29_24', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable29_24, caption = 'Replication Sample - EFA29:Factor24', label = 'RepTable29_24', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable29_25, caption = 'Exploratory Sample - EFA29:Factor25', label = 'ExpTable29_25', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable29_25, caption = 'Replication Sample - EFA29:Factor25', label = 'RepTable29_25', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable29_26, caption = 'Exploratory Sample - EFA29:Factor26', label = 'ExpTable29_26', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable29_26, caption = 'Replication Sample - EFA29:Factor26', label = 'RepTable29_26', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable29_27, caption = 'Exploratory Sample - EFA29:Factor27', label = 'ExpTable29_27', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable29_27, caption = 'Replication Sample - EFA29:Factor27', label = 'RepTable29_27', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable29_28, caption = 'Exploratory Sample - EFA29:Factor28', label = 'ExpTable29_28', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable29_28, caption = 'Replication Sample - EFA29:Factor28', label = 'RepTable29_28', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable29_29, caption = 'Exploratory Sample - EFA29:Factor29', label = 'ExpTable29_29', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable29_29, caption = 'Replication Sample - EFA29:Factor29', label = 'RepTable29_29', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \title{EFA30} \begin{tiny} <>= print(xtable(ExpTable30_1, caption = 'Exploratory Sample - EFA30:Factor1', label = 'ExpTable30_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable30_1, caption = 'Replication Sample - EFA30:Factor1', label = 'RepTable30_1', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable30_2, caption = 'Exploratory Sample - EFA30:Factor2', label = 'ExpTable30_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable30_2, caption = 'Replication Sample - EFA30:Factor2', label = 'RepTable30_2', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable30_3, caption = 'Exploratory Sample - EFA30:Factor3', label = 'ExpTable30_3', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable30_3, caption = 'Replication Sample - EFA30:Factor3', label = 'RepTable30_3', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable30_4, caption = 'Exploratory Sample - EFA30:Factor4', label = 'ExpTable30_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable30_4, caption = 'Replication Sample - EFA30:Factor4', label = 'RepTable30_4', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable30_5, caption = 'Exploratory Sample - EFA30:Factor5', label = 'ExpTable30_5', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable30_5, caption = 'Replication Sample - EFA30:Factor5', label = 'RepTable30_5', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable30_6, caption = 'Exploratory Sample - EFA30:Factor6', label = 'ExpTable30_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable30_6, caption = 'Replication Sample - EFA30:Factor6', label = 'RepTable30_6', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable30_7, caption = 'Exploratory Sample - EFA30:Factor7', label = 'ExpTable30_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable30_7, caption = 'Replication Sample - EFA30:Factor7', label = 'RepTable30_7', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable30_8, caption = 'Exploratory Sample - EFA30:Factor8', label = 'ExpTable30_8', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable30_8, caption = 'Replication Sample - EFA30:Factor8', label = 'RepTable30_8', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable30_9, caption = 'Exploratory Sample - EFA30:Factor9', label = 'ExpTable30_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable30_9, caption = 'Replication Sample - EFA30:Factor9', label = 'RepTable30_9', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable30_10, caption = 'Exploratory Sample - EFA30:Factor10', label = 'ExpTable30_10', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable30_10, caption = 'Replication Sample - EFA30:Factor10', label = 'RepTable30_10', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable30_11, caption = 'Exploratory Sample - EFA30:Factor11', label = 'ExpTable30_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable30_11, caption = 'Replication Sample - EFA30:Factor11', label = 'RepTable30_11', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable30_12, caption = 'Exploratory Sample - EFA30:Factor12', label = 'ExpTable30_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable30_12, caption = 'Replication Sample - EFA30:Factor12', label = 'RepTable30_12', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable30_13, caption = 'Exploratory Sample - EFA30:Factor13', label = 'ExpTable30_13', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable30_13, caption = 'Replication Sample - EFA30:Factor13', label = 'RepTable30_13', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable30_14, caption = 'Exploratory Sample - EFA30:Factor14', label = 'ExpTable30_14', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable30_14, caption = 'Replication Sample - EFA30:Factor14', label = 'RepTable30_14', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable30_15, caption = 'Exploratory Sample - EFA30:Factor15', label = 'ExpTable30_15', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable30_15, caption = 'Replication Sample - EFA30:Factor15', label = 'RepTable30_15', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable30_16, caption = 'Exploratory Sample - EFA30:Factor16', label = 'ExpTable30_16', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable30_16, caption = 'Replication Sample - EFA30:Factor16', label = 'RepTable30_16', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable30_17, caption = 'Exploratory Sample - EFA30:Factor17', label = 'ExpTable30_17', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable30_17, caption = 'Replication Sample - EFA30:Factor17', label = 'RepTable30_17', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable30_18, caption = 'Exploratory Sample - EFA30:Factor18', label = 'ExpTable30_18', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable30_18, caption = 'Replication Sample - EFA30:Factor18', label = 'RepTable30_18', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable30_19, caption = 'Exploratory Sample - EFA30:Factor19', label = 'ExpTable30_19', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable30_19, caption = 'Replication Sample - EFA30:Factor19', label = 'RepTable30_19', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable30_20, caption = 'Exploratory Sample - EFA30:Factor20', label = 'ExpTable30_20', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable30_20, caption = 'Replication Sample - EFA30:Factor20', label = 'RepTable30_20', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable30_21, caption = 'Exploratory Sample - EFA30:Factor21', label = 'ExpTable30_21', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable30_21, caption = 'Replication Sample - EFA30:Factor21', label = 'RepTable30_21', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable30_22, caption = 'Exploratory Sample - EFA30:Factor22', label = 'ExpTable30_22', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable30_22, caption = 'Replication Sample - EFA30:Factor22', label = 'RepTable30_22', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable30_23, caption = 'Exploratory Sample - EFA30:Factor23', label = 'ExpTable30_23', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable30_23, caption = 'Replication Sample - EFA30:Factor23', label = 'RepTable30_23', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable30_24, caption = 'Exploratory Sample - EFA30:Factor24', label = 'ExpTable30_24', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable30_24, caption = 'Replication Sample - EFA30:Factor24', label = 'RepTable30_24', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable30_25, caption = 'Exploratory Sample - EFA30:Factor25', label = 'ExpTable30_25', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable30_25, caption = 'Replication Sample - EFA30:Factor25', label = 'RepTable30_25', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(ExpTable30_26, caption = 'Exploratory Sample - EFA30:Factor26', label = 'ExpTable30_26', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable30_26, caption = 'Replication Sample - EFA30:Factor26', label = 'RepTable30_26', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable30_27, caption = 'Exploratory Sample - EFA30:Factor27', label = 'ExpTable30_27', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable30_27, caption = 'Replication Sample - EFA30:Factor27', label = 'RepTable30_27', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable30_28, caption = 'Exploratory Sample - EFA30:Factor28', label = 'ExpTable30_28', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} \clearpage \begin{tiny} <>= print(xtable(RepTable30_28, caption = 'Replication Sample - EFA30:Factor28', label = 'RepTable30_28', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable30_29, caption = 'Exploratory Sample - EFA30:Factor29', label = 'ExpTable30_29', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable30_29, caption = 'Replication Sample - EFA30:Factor29', label = 'RepTable30_29', align = alignList), size='tiny', table.placement = "!h") print(xtable(ExpTable30_30, caption = 'Exploratory Sample - EFA30:Factor30', label = 'ExpTable30_30', align = alignList), size='tiny', table.placement = "!h") print(xtable(RepTable30_30, caption = 'Replication Sample - EFA30:Factor30', label = 'RepTable30_30', align = alignList), size='tiny', table.placement = "!h") @ \end{tiny} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \clearpage \pagenumbering{arabic}% resets `page` counter to 1 \renewcommand*{\thepage}{A2-\arabic{page}} \renewcommand{\thefigure}{A2-\arabic{figure}} \setcounter{figure}{0} \renewcommand{\thetable}{A2-\arabic{table}} \setcounter{table}{0} \begin{center} \begin{Large} \textbf{Appendix B} \subsection*{Annotated tables of candidate items} \addcontentsline{toc}{subsection}{B: Annotated tables of candidate items} \label{sec:appendixB} \end{Large} {\bf David M. Condon}\footnote[$\star$]{This document is an appendix to \textit{The SAPA Personality Inventory}. Correspondence should be directed to \href{mailto:david-condon@northwestern.edu}{david-condon@northwestern.edu}, Department of Medical Social Sciences, Northwestern University, Chicago, Illinois. Please use the following citation information: Condon, D.M. (2017). The SAPA Personality Inventory: an empirically-derived, hierarchically-organized self-report personality assessment model.} \end{center} \vskip -0.5cm %%%%%%%%%%%%%%%%%% %Add Breaking Line \begin{center} \linethickness{0.5mm} \line(1,0){300} \end{center} %%%%%%%%%%%%%%%%%% In each of the following figures, the selected items for the facets (Figures \ref{fig:ft27_1} through \ref{fig:ft27_27}) and factors (Figures \ref{fig:ft5_1} through \ref{fig:ft5_5}) are highlighted in green. The yellow items are the ``secondary'' recommendations for each facet/factor. It is generally not the case that the green and yellow highlighted items are the top items in each figure --- interspersed among the green and yellow items for most facets/factors are un-highlighted items that have been excluded For these items, I have indicated the rationale behind my decision to omit the items. The reasons include: redundancy with previously selected items for each factor, the use of colloquial language, a lack of overlapping utility for 5 factor assessment, or concerns about the strength of the loadings and consistency of loadings across samples. \clearpage \begin{figure} \ffigbox{\includegraphics[trim = 0mm 90mm 30mm 60mm, clip, width=.95\textwidth]{figures/appendices/factorTable27_1.png}}{\caption{}\label{fig:ft27_1}} \end{figure} \begin{figure} \ffigbox{\includegraphics[trim = 0mm 90mm 30mm 60mm, clip, width=.95\textwidth]{figures/appendices/factorTable27_2.png}}{\caption{label 27\_2}\label{fig:ft27_2}} \end{figure} \begin{figure} \ffigbox{\includegraphics[trim = 0mm 90mm 30mm 60mm, clip, width=.95\textwidth]{figures/appendices/factorTable27_3.png}}{\caption{label 27\_3}\label{fig:ft27_3}} \end{figure} \clearpage \begin{figure} \ffigbox{\includegraphics[trim = 0mm 90mm 30mm 60mm, clip, width=.95\textwidth]{figures/appendices/factorTable27_4.png}}{\caption{label 27\_4}\label{fig:ft27_4}} \end{figure} \begin{figure} \ffigbox{\includegraphics[trim = 0mm 90mm 30mm 60mm, clip, width=.95\textwidth]{figures/appendices/factorTable27_5.png}}{\caption{label 27\_5}\label{fig:ft27_5}} \end{figure} \begin{figure} \ffigbox{\includegraphics[trim = 0mm 90mm 30mm 60mm, clip, width=.95\textwidth]{figures/appendices/factorTable27_6.png}}{\caption{label 27\_6}\label{fig:ft27_6}} \end{figure} \clearpage \begin{figure} \ffigbox{\includegraphics[trim = 0mm 90mm 30mm 60mm, clip, width=.95\textwidth]{figures/appendices/factorTable27_7.png}}{\caption{label 27\_7}\label{fig:ft27_7}} \end{figure} \begin{figure} \ffigbox{\includegraphics[trim = 0mm 90mm 30mm 60mm, clip, width=.95\textwidth]{figures/appendices/factorTable27_8.png}}{\caption{label 27\_8}\label{fig:ft27_8}} \end{figure} \begin{figure} \ffigbox{\includegraphics[trim = 0mm 90mm 30mm 60mm, clip, width=.95\textwidth]{figures/appendices/factorTable27_9.png}}{\caption{label 27\_9}\label{fig:ft27_9}} \end{figure} \clearpage \begin{figure} \ffigbox{\includegraphics[trim = 0mm 90mm 30mm 60mm, clip, width=.95\textwidth]{figures/appendices/factorTable27_10.png}}{\caption{label 27\_10}\label{fig:ft27_10}} \end{figure} \begin{figure} \ffigbox{\includegraphics[trim = 0mm 90mm 30mm 60mm, clip, width=.95\textwidth]{figures/appendices/factorTable27_11.png}}{\caption{label 27\_11}\label{fig:ft27_11}} \end{figure} \begin{figure} \ffigbox{\includegraphics[trim = 0mm 90mm 30mm 60mm, clip, width=.95\textwidth]{figures/appendices/factorTable27_12.png}}{\caption{label 27\_12}\label{fig:ft27_12}} \end{figure} \clearpage \begin{figure} \ffigbox{\includegraphics[trim = 0mm 90mm 30mm 60mm, clip, width=.95\textwidth]{figures/appendices/factorTable27_13.png}}{\caption{label 27\_13}\label{fig:ft27_13}} \end{figure} \begin{figure} \ffigbox{\includegraphics[trim = 0mm 90mm 30mm 60mm, clip, width=.95\textwidth]{figures/appendices/factorTable27_14.png}}{\caption{label 27\_14}\label{fig:ft27_14}} \end{figure} \begin{figure} \ffigbox{\includegraphics[trim = 0mm 90mm 30mm 60mm, clip, width=.95\textwidth]{figures/appendices/factorTable27_15.png}}{\caption{label 27\_15}\label{fig:ft27_15}} \end{figure} \clearpage \begin{figure} \ffigbox{\includegraphics[trim = 0mm 90mm 30mm 60mm, clip, width=.95\textwidth]{figures/appendices/factorTable27_16.png}}{\caption{label 27\_16}\label{fig:ft27_16}} \end{figure} \begin{figure} \ffigbox{\includegraphics[trim = 0mm 90mm 30mm 60mm, clip, width=.95\textwidth]{figures/appendices/factorTable27_17.png}}{\caption{label 27\_17}\label{fig:ft27_17}} \end{figure} \begin{figure} \ffigbox{\includegraphics[trim = 0mm 90mm 30mm 60mm, clip, width=.95\textwidth]{figures/appendices/factorTable27_18.png}}{\caption{label 27\_18}\label{fig:ft27_18}} \end{figure} \clearpage \begin{figure} \ffigbox{\includegraphics[trim = 0mm 90mm 30mm 60mm, clip, width=.95\textwidth]{figures/appendices/factorTable27_19.png}}{\caption{label 27\_19}\label{fig:ft27_19}} \end{figure} \begin{figure} \ffigbox{\includegraphics[trim = 0mm 90mm 30mm 60mm, clip, width=.95\textwidth]{figures/appendices/factorTable27_20.png}}{\caption{label 27\_20}\label{fig:ft27_20}} \end{figure} \begin{figure} \ffigbox{\includegraphics[trim = 0mm 90mm 30mm 60mm, clip, width=.95\textwidth]{figures/appendices/factorTable27_21.png}}{\caption{label 27\_21}\label{fig:ft27_21}} \end{figure} \clearpage \begin{figure} \ffigbox{\includegraphics[trim = 0mm 90mm 30mm 60mm, clip, width=.95\textwidth]{figures/appendices/factorTable27_22.png}}{\caption{label 27\_22}\label{fig:ft27_22}} \end{figure} \begin{figure} \ffigbox{\includegraphics[trim = 0mm 90mm 30mm 60mm, clip, width=.95\textwidth]{figures/appendices/factorTable27_23.png}}{\caption{label 27\_23}\label{fig:ft27_23}} \end{figure} \begin{figure} \ffigbox{\includegraphics[trim = 0mm 90mm 30mm 60mm, clip, width=.95\textwidth]{figures/appendices/factorTable27_24.png}}{\caption{label 27\_24}\label{fig:ft27_24}} \end{figure} \clearpage \begin{figure} \ffigbox{\includegraphics[trim = 0mm 90mm 30mm 60mm, clip, width=.95\textwidth]{figures/appendices/factorTable27_25.png}}{\caption{label 27\_25}\label{fig:ft27_25}} \end{figure} \begin{figure} \ffigbox{\includegraphics[trim = 0mm 90mm 30mm 60mm, clip, width=.95\textwidth]{figures/appendices/factorTable27_26.png}}{\caption{label 27\_26}\label{fig:ft27_26}} \end{figure} \begin{figure} \ffigbox{\includegraphics[trim = 0mm 90mm 30mm 60mm, clip, width=.95\textwidth]{figures/appendices/factorTable27_27.png}}{\caption{label 27\_27}\label{fig:ft27_27}} \end{figure} \begin{figure} \ffigbox{\includegraphics[trim = 0mm 50mm 30mm 60mm, clip, width=.95\textwidth]{figures/appendices/factorTable5_1.png}}{\caption{}\label{fig:ft5_1}} \end{figure} \begin{figure} \ffigbox{\includegraphics[trim = 0mm 50mm 30mm 60mm, clip, width=.95\textwidth]{figures/appendices/factorTable5_2.png}}{\caption{}\label{fig:ft5_2}} \end{figure} \begin{figure} \ffigbox{\includegraphics[trim = 0mm 50mm 30mm 60mm, clip, width=.95\textwidth]{figures/appendices/factorTable5_3.png}}{\caption{}\label{fig:ft5_3}} \end{figure} \begin{figure} \ffigbox{\includegraphics[trim = 0mm 50mm 30mm 60mm, clip, width=.95\textwidth]{figures/appendices/factorTable5_4.png}}{\caption{}\label{fig:ft5_4}} \end{figure} \begin{figure} \ffigbox{\includegraphics[trim = 0mm 50mm 30mm 60mm, clip, width=.95\textwidth]{figures/appendices/factorTable5_5.png}}{\caption{}\label{fig:ft5_5}} \end{figure} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \clearpage \pagenumbering{arabic}% resets `page` counter to 1 \renewcommand*{\thepage}{A3-\arabic{page}} \renewcommand{\thefigure}{A3-\arabic{figure}} \setcounter{figure}{0} \renewcommand{\thetable}{A3-\arabic{table}} \setcounter{table}{0} \begin{center} \begin{Large} \textbf{Appendix C} \subsection*{Norms by Age and Gender for the SPI} \addcontentsline{toc}{subsection}{C: Norms by Age and Gender for the SPI.} \label{sec:appendixC} \end{Large} {\bf David M. Condon}\footnote[$\star$]{This document is an appendix to \textit{The SAPA Personality Inventory}. Correspondence should be directed to \href{mailto:david-condon@northwestern.edu}{david-condon@northwestern.edu}, Department of Medical Social Sciences, Northwestern University, Chicago, Illinois. Please use the following citation information: Condon, D.M. (2017). The SAPA Personality Inventory: an empirically-derived, hierarchically-organized self-report personality assessment model.} \end{center} \vskip -0.5cm %%%%%%%%%%%%%%%%%% %Add Breaking Line \begin{center} \linethickness{0.5mm} \line(1,0){300} \end{center} %%%%%%%%%%%%%%%%%% Tables \ref{tab:MeansSDsByScale27} through \ref{tab:MeansSDsByScale5} shows the raw scores by scale for the SPI-27 and SPI-5 scales using several age bands and both genders. Figures \ref{fig:scale1} through \ref{fig:scale32} show plots of the scores by age and gender from ages 14-90. Note that the code used to generate the tables and figures in this appendix (not shown) also creates two other large tables that may be useful for end-users of the SPI who are looking for norms data. These tables contain item and scale level means, standard deviations and sample sizes for all ages (by year, 14-90) for both genders. \begin{scriptsize} <>= rm(list=setdiff(ls(), c("filepathdata", "filepathfig"))) load(paste(filepathdata, "keysAndScalesInfo.Rdata", sep = "")) SPI27items <- as.vector(unlist(keys.list27$items)) SPI27content <- ItemInfo696[SPI27items,] normsData <- allSample[,c("age", "gender", SPI27items)] normsData$ageGroup <- mapvalues(normsData$age, from = c(14:18), to = rep(16, 5)) normsData$ageGroup <- mapvalues(normsData$ageGroup, from = c(19:25), to = rep(22, 7)) normsData$ageGroup <- mapvalues(normsData$ageGroup, from = c(26:35), to = rep(30, 10)) normsData$ageGroup <- mapvalues(normsData$ageGroup, from = c(36:45), to = rep(40, 10)) normsData$ageGroup <- mapvalues(normsData$ageGroup, from = c(46:55), to = rep(50, 10)) normsData$ageGroup <- mapvalues(normsData$ageGroup, from = c(56:65), to = rep(60, 10)) normsData$ageGroup <- mapvalues(normsData$ageGroup, from = c(66:90), to = rep(77, 25)) normsData$gender <- as.numeric(normsData[,"gender"]) ### demo <- list( age = c("age"), gender = c("gender"), ageGroup = c("ageGroup")) SPIKey2 <- make.keys(normsData, c(demo, keys.list27$keys, keys.list5$keys)) scoredNormsData <- scoreItems(SPIKey2, normsData, totals=FALSE, impute="none", min=1, max=6) scoredNormsDataScores <- as.data.frame(scoredNormsData$scores) # Make the plots of age by gender scoredNormsDataScoresAgeSubset <- subset(scoredNormsDataScores, age < 71) scoredNormsDataScoresAgeSubset <- subset(scoredNormsDataScoresAgeSubset, select = c(age, gender, Compassion:Openness)) for (i in 3:ncol(scoredNormsDataScoresAgeSubset)) { ggplot(scoredNormsDataScoresAgeSubset, aes(x = age, y = scoredNormsDataScoresAgeSubset[, i], color = as.factor(gender))) + geom_smooth(aes(fill = as.factor(gender)), alpha = 0.25) + geom_smooth(se = F) + scale_y_continuous(colnames(scoredNormsDataScoresAgeSubset)[i], limit = c(1, 6)) + guides(color = F, fill = F) + scale_color_manual(values = c("#0072B2", "#D55E00")) + scale_fill_manual(values = c("#0072B2", "#D55E00")) + theme_bw() ggsave(filename = paste(filepathfig, "norms/scale", i - 2, ".pdf", sep = ""), scale = 1, width = 3.3, height = 3.3, units = c("in"), dpi = 300) } # Now make the scale score table by age group and gender scoredNormsDataScoresAgegroupSubset <- subset(scoredNormsDataScores, select = c(gender, ageGroup, Compassion:Openness)) normsStatsByScaleAgegroup <- statsBy(scoredNormsDataScoresAgegroupSubset, c("gender", "ageGroup")) MeansByScale <- round(subset(normsStatsByScaleAgegroup$mean, select = -c(gender, ageGroup)), 2) rownames(MeansByScale) <- c("14-18 M", "14-18 F", "19-25 M", "19-25 F", "26-35 M", "26-35 F", "36-45 M", "36-45 F", "46-55 M", "46-55 F", "56-65 M", "56-65 F", "66+ M", "66+ F") MeansByScale <- t(MeansByScale) SDsByScale <- round(subset(normsStatsByScaleAgegroup$sd, select = -c(gender, ageGroup)), 2) rownames(SDsByScale) <- c("14-18 M", "14-18 F", "19-25 M", "19-25 F", "26-35 M", "26-35 F", "36-45 M", "36-45 F", "46-55 M", "46-55 F", "56-65 M", "56-65 F", "66+ M", "66+ F") SDsByScale <- t(SDsByScale) MeansSDsByScale <- data.frame(matrix(ncol=2*ncol(MeansByScale), nrow = nrow(MeansByScale))) for (i in 1:ncol(MeansByScale)) { MeansSDsByScale[,(2*i-1)] <- MeansByScale[,i] MeansSDsByScale[,(2*i)] <- SDsByScale[,i] } rownames(MeansSDsByScale) <- rownames(MeansByScale) colnames(MeansSDsByScale) <- c("14-18M_M", "14-18M_sd", "14-18F_M", "14-18F_sd", "19-25M_M", "19-25M_sd", "19-25F_M", "19-25F_sd", "26-35M_M", "26-35M_sd", "26-35F_M", "26-35F_sd", "36-45M_M", "36-45M_sd", "36-45F_M", "36-45F_sd", "46-55M_M", "46-55M_sd", "46-55F_M", "46-55F_sd", "56-65M_M", "56-65M_sd", "56-65F_M", "56-65F_sd", "66+M_M", "66+M_sd", "66+F_M", "66+F_sd") # Make a huge table that is the item means, sds and ns for all ages by gender normsStatsByItemAllAges <- statsBy(subset(normsData, select = -c(ageGroup)), c("age", "gender"), cor=TRUE) genderLabels <- ifelse(normsStatsByItemAllAges$mean[,"gender"] == 1, "M", "F") ageGenderLabels <- paste(normsStatsByItemAllAges$mean[,"age"], "_", genderLabels, sep = "") MeansByItemAge <- round(subset(normsStatsByItemAllAges$mean, select = -c(age, gender)), 2) rownames(MeansByItemAge) <- ageGenderLabels MeansByItemAge <- t(MeansByItemAge) SDsByItemAge <- round(subset(normsStatsByItemAllAges$sd, select = -c(age, gender)), 2) rownames(SDsByItemAge) <- ageGenderLabels SDsByItemAge <- t(SDsByItemAge) NsByItemAge <- round(subset(normsStatsByItemAllAges$n, select = -c(age, gender)), 2) rownames(NsByItemAge) <- ageGenderLabels NsByItemAge <- t(NsByItemAge) MeansSDsNsByItemAge <- data.frame(matrix(ncol=3*ncol(MeansByItemAge), nrow = nrow(MeansByItemAge))) for (i in 1:ncol(MeansByItemAge)) { MeansSDsNsByItemAge[,(3*i-2)] <- MeansByItemAge[,i] MeansSDsNsByItemAge[,(3*i-1)] <- SDsByItemAge[,i] MeansSDsNsByItemAge[,(3*i)] <- NsByItemAge[,i] } rownames(MeansSDsNsByItemAge) <- rownames(MeansByItemAge) colsMeansSDsNsByItemAge <- list() for (i in 1:length(ageGenderLabels)) { colsMeansSDsNsByItemAge[3*i-2] <- paste(colnames(MeansByItemAge)[i], "_m", sep="") colsMeansSDsNsByItemAge[3*i-1] <- paste(colnames(SDsByItemAge)[i], "_sd", sep="") colsMeansSDsNsByItemAge[3*i] <- paste(colnames(NsByItemAge)[i], "_n", sep="") } colsMeansSDsNsByItemAge <- unlist(colsMeansSDsNsByItemAge) colnames(MeansSDsNsByItemAge) <- colsMeansSDsNsByItemAge # We should have a table with 135 rows and slightly less than 456 columns # 456 = 76 years * 2 genders * 3 variables (M, sd, n) # Slightly less bc a few of the higher ages will be empty. dim(MeansSDsNsByItemAge) # And one more (slightly less) huge table that is the scale means, sds and ns for all ages by gender scoredNormsDataScoresAgeSubset2 <- subset(scoredNormsDataScores, select = c(age, gender, Compassion:Openness)) normsStatsByScaleAge <- statsBy(scoredNormsDataScoresAgeSubset2, c("age", "gender")) genderLabels2 <- ifelse(normsStatsByScaleAge$mean[,"gender"] == 1, "M", "F") ageGenderLabels2 <- paste(normsStatsByScaleAge$mean[,"age"], "_", genderLabels2, sep = "") MeansByScaleAge <- round(subset(normsStatsByScaleAge$mean, select = -c(age, gender)), 2) rownames(MeansByScaleAge) <- ageGenderLabels2 MeansByScaleAge <- t(MeansByScaleAge) SDsByScaleAge <- round(subset(normsStatsByScaleAge$sd, select = -c(age, gender)), 2) rownames(SDsByScaleAge) <- ageGenderLabels2 SDsByScaleAge <- t(SDsByScaleAge) NsByScaleAge <- round(subset(normsStatsByScaleAge$n, select = -c(age, gender)), 2) rownames(NsByScaleAge) <- ageGenderLabels2 NsByScaleAge <- t(NsByScaleAge) MeansSDsNsByScaleAge <- data.frame(matrix(ncol=3*ncol(MeansByScaleAge), nrow = nrow(MeansByScaleAge))) for (i in 1:ncol(MeansByScaleAge)) { MeansSDsNsByScaleAge[,(3*i-2)] <- MeansByScaleAge[,i] MeansSDsNsByScaleAge[,(3*i-1)] <- SDsByScaleAge[,i] MeansSDsNsByScaleAge[,(3*i)] <- NsByScaleAge[,i] } rownames(MeansSDsNsByScaleAge) <- rownames(MeansByScaleAge) colsMeansSDsNsByScaleAge <- list() for (i in 1:length(ageGenderLabels2)) { colsMeansSDsNsByScaleAge[3*i-2] <- paste(colnames(MeansByScaleAge)[i], "_m", sep="") colsMeansSDsNsByScaleAge[3*i-1] <- paste(colnames(SDsByScaleAge)[i], "_sd", sep="") colsMeansSDsNsByScaleAge[3*i] <- paste(colnames(NsByScaleAge)[i], "_n", sep="") } colsMeansSDsNsByScaleAge <- unlist(colsMeansSDsNsByScaleAge) colnames(MeansSDsNsByScaleAge) <- colsMeansSDsNsByScaleAge # Should be 32 rows and slightly less than 456 columns dim(MeansSDsNsByScaleAge) @ \end{scriptsize} \clearpage \begin{tiny} \newgeometry{left=2.54cm,right=2.54cm, bottom=2.54cm, top=2.54cm} \begin{landscape} \pagestyle{empty} \setlength\LTcapwidth{\textwidth} \setlength\LTleft{0pt} \setlength\LTright{0pt} <>= suppressPackageStartupMessages(library(xtable)) addtorow <- list(pos = list(0), command = NULL) command <- paste0(" & \\multicolumn{4}{c}{14 --- 18 yrs old} & \\multicolumn{4}{c}{19 --- 25 yrs old} & \\multicolumn{4}{c}{26 --- 35 yrs old} & \\multicolumn{4}{c}{36 --- 45 yrs old} & \\multicolumn{4}{c}{46 --- 55 yrs old} & \\multicolumn{4}{c}{56 --- 65 yrs old} & \\multicolumn{4}{c}{66 yrs old and up} \\cr", " & \\multicolumn{2}{c}{Male} & \\multicolumn{2}{c}{Female} & \\multicolumn{2}{c}{Male} & \\multicolumn{2}{c}{Female} & \\multicolumn{2}{c}{Male} & \\multicolumn{2}{c}{Female} & \\multicolumn{2}{c}{Male} & \\multicolumn{2}{c}{Female} & \\multicolumn{2}{c}{Male} & \\multicolumn{2}{c}{Female} & \\multicolumn{2}{c}{Male} & \\multicolumn{2}{c}{Female} & \\multicolumn{2}{c}{Male} & \\multicolumn{2}{c}{Female} \\cr", " & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} \\cr", "\\hline\n\\endhead\n", "\\hline\n", "\\multicolumn{29}{c}", "{\\textit{(continued on next page)}}\n", "\\endfoot\n", "\\endlastfoot\n") addtorow$command <- command print(xtable(MeansSDsByScale[1:27,], digits = 2, caption='SPI scale raw means and standard deviations by age range and gender', label='tab:MeansSDsByScale27', align = c("l", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r")), size='tiny', caption.placement = "top", tabular.environment = "longtable", include.colnames = FALSE, floating = FALSE, add.to.row = addtorow) @ <>= addtorow <- list(pos = list(0), command = NULL) command <- paste0(" & \\multicolumn{4}{c}{14 --- 18 yrs old} & \\multicolumn{4}{c}{19 --- 25 yrs old} & \\multicolumn{4}{c}{26 --- 35 yrs old} & \\multicolumn{4}{c}{36 --- 45 yrs old} & \\multicolumn{4}{c}{46 --- 55 yrs old} & \\multicolumn{4}{c}{56 --- 65 yrs old} & \\multicolumn{4}{c}{66 yrs old and up} \\cr", " & \\multicolumn{2}{c}{Male} & \\multicolumn{2}{c}{Female} & \\multicolumn{2}{c}{Male} & \\multicolumn{2}{c}{Female} & \\multicolumn{2}{c}{Male} & \\multicolumn{2}{c}{Female} & \\multicolumn{2}{c}{Male} & \\multicolumn{2}{c}{Female} & \\multicolumn{2}{c}{Male} & \\multicolumn{2}{c}{Female} & \\multicolumn{2}{c}{Male} & \\multicolumn{2}{c}{Female} & \\multicolumn{2}{c}{Male} & \\multicolumn{2}{c}{Female} \\cr", " & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} & \\textit{m} & \\textit{sd} \\cr", "\\hline\n\\endhead\n", "\\hline\n", "\\multicolumn{29}{c}", "{\\textit{(continued on next page)}}\n", "\\endfoot\n", "\\endlastfoot\n") addtorow$command <- command print(xtable(MeansSDsByScale[28:32,], digits = 2, caption='SPI scale raw means and standard deviations by age range and gender', label='tab:MeansSDsByScale5', align = c("l", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r", "r")), size='tiny', caption.placement = "top", tabular.environment = "longtable", include.colnames = FALSE, floating = FALSE, add.to.row = addtorow) @ \end{landscape} \restoregeometry \end{tiny} \clearpage \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/norms/scale1.pdf}}{\caption{SPI-135-27 scale scores by age and gender; Males in blue and Females in red}\label{fig:scale1}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/norms/scale2.pdf}}{\caption{SPI-135-27 scale scores by age and gender; Males in blue and Females in red}\label{fig:scale2}} \end{floatrow} \end{figure} \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/norms/scale3.pdf}}{\caption{SPI-135-27 scale scores by age and gender; Males in blue and Females in red}\label{fig:scale3}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/norms/scale4.pdf}}{\caption{SPI-135-27 scale scores by age and gender; Males in blue and Females in red}\label{fig:scale4}} \end{floatrow} \end{figure} \clearpage \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/norms/scale5.pdf}}{\caption{SPI-135-27 scale scores by age and gender; Males in blue and Females in red}\label{fig:scale5}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/norms/scale6.pdf}}{\caption{SPI-135-27 scale scores by age and gender; Males in blue and Females in red}\label{fig:scale6}} \end{floatrow} \end{figure} \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/norms/scale7.pdf}}{\caption{SPI-135-27 scale scores by age and gender; Males in blue and Females in red}\label{fig:scale7}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/norms/scale8.pdf}}{\caption{SPI-135-27 scale scores by age and gender; Males in blue and Females in red}\label{fig:scale8}} \end{floatrow} \end{figure} \clearpage \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/norms/scale9.pdf}}{\caption{SPI-135-27 scale scores by age and gender; Males in blue and Females in red}\label{fig:scale9}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/norms/scale10.pdf}}{\caption{SPI-135-27 scale scores by age and gender; Males in blue and Females in red}\label{fig:scale10}} \end{floatrow} \end{figure} \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/norms/scale11.pdf}}{\caption{SPI-135-27 scale scores by age and gender; Males in blue and Females in red}\label{fig:scale11}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/norms/scale12.pdf}}{\caption{SPI-135-27 scale scores by age and gender; Males in blue and Females in red}\label{fig:scale12}} \end{floatrow} \end{figure} \clearpage \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/norms/scale13.pdf}}{\caption{SPI-135-27 scale scores by age and gender; Males in blue and Females in red}\label{fig:scale13}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/norms/scale14.pdf}}{\caption{SPI-135-27 scale scores by age and gender; Males in blue and Females in red}\label{fig:scale14}} \end{floatrow} \end{figure} \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/norms/scale15.pdf}}{\caption{SPI-135-27 scale scores by age and gender; Males in blue and Females in red}\label{fig:scale15}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/norms/scale16.pdf}}{\caption{SPI-135-27 scale scores by age and gender; Males in blue and Females in red}\label{fig:scale16}} \end{floatrow} \end{figure} \clearpage \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/norms/scale17.pdf}}{\caption{SPI-135-27 scale scores by age and gender; Males in blue and Females in red}\label{fig:scale17}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/norms/scale18.pdf}}{\caption{SPI-135-27 scale scores by age and gender; Males in blue and Females in red}\label{fig:scale18}} \end{floatrow} \end{figure} \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/norms/scale19.pdf}}{\caption{SPI-135-27 scale scores by age and gender; Males in blue and Females in red}\label{fig:scale19}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/norms/scale20.pdf}}{\caption{SPI-135-27 scale scores by age and gender; Males in blue and Females in red}\label{fig:scale20}} \end{floatrow} \end{figure} \clearpage \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/norms/scale21.pdf}}{\caption{SPI-135-27 scale scores by age and gender; Males in blue and Females in red}\label{fig:scale21}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/norms/scale22.pdf}}{\caption{SPI-135-27 scale scores by age and gender; Males in blue and Females in red}\label{fig:scale22}} \end{floatrow} \end{figure} \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/norms/scale23.pdf}}{\caption{SPI-135-27 scale scores by age and gender; Males in blue and Females in red}\label{fig:scale23}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/norms/scale24.pdf}}{\caption{SPI-135-27 scale scores by age and gender; Males in blue and Females in red}\label{fig:scale24}} \end{floatrow} \end{figure} \clearpage \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/norms/scale25.pdf}}{\caption{SPI-135-27 scale scores by age and gender; Males in blue and Females in red}\label{fig:scale25}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/norms/scale26.pdf}}{\caption{SPI-135-27 scale scores by age and gender; Males in blue and Females in red}\label{fig:scale26}} \end{floatrow} \end{figure} \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/norms/scale27.pdf}}{\caption{SPI-135-27 scale scores by age and gender; Males in blue and Females in red}\label{fig:scale27}} \end{floatrow} \end{figure} \clearpage \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/norms/scale28.pdf}}{\caption{SPI-135-5 scale scores by age and gender; Males in blue and Females in red}\label{fig:scale28}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/norms/scale29.pdf}}{\caption{SPI-135-5 scale scores by age and gender; Males in blue and Females in red}\label{fig:scale29}} \end{floatrow} \end{figure} \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/norms/scale30.pdf}}{\caption{SPI-135-5 scale scores by age and gender; Males in blue and Females in red}\label{fig:scale30}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/norms/scale31.pdf}}{\caption{SPI-135-5 scale scores by age and gender; Males in blue and Females in red}\label{fig:scale31}} \end{floatrow} \end{figure} \clearpage \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/norms/scale32.pdf}}{\caption{SPI-135-5 scale scores by age and gender; Males in blue and Females in red}\label{fig:scale32}} \end{floatrow} \end{figure} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \clearpage \pagenumbering{arabic}% resets `page` counter to 1 \renewcommand*{\thepage}{A4-\arabic{page}} \renewcommand{\thefigure}{A4-\arabic{figure}} \setcounter{figure}{0} \renewcommand{\thetable}{A4-\arabic{table}} \setcounter{table}{0} \begin{center} \begin{Large} \textbf{Appendix D} \subsection*{SPI27-135 item calibrations and information curves} \addcontentsline{toc}{subsection}{D: SPI27-135 item calibrations and information curves} \label{sec:appendixD} \end{Large} {\bf David M. Condon}\footnote[$\star$]{This document is an appendix to \textit{The SAPA Personality Inventory}. Correspondence should be directed to \href{mailto:david-condon@northwestern.edu}{david-condon@northwestern.edu}, Department of Medical Social Sciences, Northwestern University, Chicago, Illinois. Please use the following citation information: Condon, D.M. (2017). The SAPA Personality Inventory: an empirically-derived, hierarchically-organized self-report personality assessment model.} \end{center} \vskip -0.5cm %%%%%%%%%%%%%%%%%% %Add Breaking Line \begin{center} \linethickness{0.5mm} \line(1,0){300} \end{center} %%%%%%%%%%%%%%%%%% This appendix contains many pages of figures and tables that show the Item Information Functions, Test Information Functions and item calibrations for the SPI135-27 scales. \begin{scriptsize} <>= # In this first step we are simply getting things ready by installing and loading necessary packages and functions. These are hidden in the pdf output. rm(list=setdiff(ls(), c("filepathdata", "filepathfig"))) suppressPackageStartupMessages(library(psych)) suppressPackageStartupMessages(library(parallel)) suppressPackageStartupMessages(library(plyr)) suppressPackageStartupMessages(library(multicon)) @ \end{scriptsize} \begin{scriptsize} <>= # keep eval=FALSE # This section of code is run in the main document. # Shown here only for reference. load(paste(filepathdata, "keysAndScalesInfo.Rdata", sep="")) # Get the IRT parameters for all of the SPI27 scales irt.fa.loop <- function (x, y, nscales=c(1:length(y))) { # Loops through all of the item lists in y # x is the raw data frame used to derive the parameters # nscales is a vector of the numbers of the lists in y to be run IRTlist <- list() for (i in nscales) { IRT <- irt.fa(subset(x, select = c(y[[i]]))) IRTlist[[i]] <- IRT } names(IRTlist) <- names(y)[nscales] results <- IRTlist return(results) } IRToutputSPI27 <- irt.fa.loop(allSample, keys.list27$items) for (i in 1:27) { calTable <- cbind(IRToutputSPI27[i][[1]]$irt[2][[1]], IRToutputSPI27[i][[1]]$irt[1][[1]][[1]]) colnames(calTable) <- c("slope", "thresh1", "thresh2", "thresh3", "thresh4", "thresh5") assign(paste("calTable", i, sep=""), round(calTable, 2)) rm(calTable) } calTableCombined <- rbind(calTable1, calTable2, calTable3, calTable4, calTable5, calTable6, calTable7, calTable8, calTable9, calTable10, calTable11, calTable12, calTable13, calTable14, calTable15, calTable16, calTable17, calTable18, calTable19, calTable20, calTable21, calTable22, calTable23, calTable24, calTable25, calTable26, calTable27) @ \end{scriptsize} \begin{scriptsize} <>= load(paste(filepathdata, "IRToutputSPI27.Rdata", sep="")) pdf(paste(filepathfig, "IRT/SPI27_1IIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[1]], type="IIC", main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_1TIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[1]], type="test", ylim=c(0,7), main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_2IIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[2]], type="IIC", main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_2TIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[2]], type="test", ylim=c(0,7), main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_3IIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[3]], type="IIC", main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_3TIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[3]], type="test", ylim=c(0,7), main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_4IIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[4]], type="IIC", main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_4TIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[4]], type="test", ylim=c(0,7), main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_5IIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[5]], type="IIC", main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_5TIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[5]], type="test", ylim=c(0,7), main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_6IIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[6]], type="IIC", main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_6TIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[6]], type="test", ylim=c(0,7), main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_7IIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[7]], type="IIC", main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_7TIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[7]], type="test", ylim=c(0,7), main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_8IIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[8]], type="IIC", main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_8TIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[8]], type="test", ylim=c(0,7), main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_9IIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[9]], type="IIC", main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_9TIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[9]], type="test", ylim=c(0,7), main=" ") dev.off() @ \end{scriptsize} \begin{scriptsize} <>= pdf(paste(filepathfig, "IRT/SPI27_10IIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[10]], type="IIC", main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_10TIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[10]], type="test", ylim=c(0,7), main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_11IIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[11]], type="IIC", main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_11TIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[11]], type="test", ylim=c(0,7), main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_12IIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[12]], type="IIC", main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_12TIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[12]], type="test", ylim=c(0,7), main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_13IIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[13]], type="IIC", main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_13TIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[13]], type="test", ylim=c(0,7), main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_14IIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[14]], type="IIC", main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_14TIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[14]], type="test", ylim=c(0,7), main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_15IIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[15]], type="IIC", main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_15TIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[15]], type="test", ylim=c(0,7), main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_16IIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[16]], type="IIC", main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_16TIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[16]], type="test", ylim=c(0,7), main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_17IIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[17]], type="IIC", main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_17TIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[17]], type="test", ylim=c(0,7), main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_18IIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[18]], type="IIC", main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_18TIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[18]], type="test", ylim=c(0,7), main=" ") dev.off() @ \end{scriptsize} \begin{scriptsize} <>= pdf(paste(filepathfig, "IRT/SPI27_19IIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[19]], type="IIC", main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_19TIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[19]], type="test", ylim=c(0,7), main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_20IIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[20]], type="IIC", main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_20TIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[20]], type="test", ylim=c(0,7), main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_21IIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[21]], type="IIC", main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_21TIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[21]], type="test", ylim=c(0,7), main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_22IIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[22]], type="IIC", main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_22TIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[22]], type="test", ylim=c(0,7), main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_23IIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[23]], type="IIC", main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_23TIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[23]], type="test", ylim=c(0,7), main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_24IIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[24]], type="IIC", main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_24TIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[24]], type="test", ylim=c(0,7), main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_25IIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[25]], type="IIC", main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_25TIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[25]], type="test", ylim=c(0,7), main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_26IIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[26]], type="IIC", main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_26TIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[26]], type="test", ylim=c(0,7), main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_27IIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[27]], type="IIC", main=" ") dev.off() pdf(paste(filepathfig, "IRT/SPI27_27TIF.pdf", sep=""), onefile = TRUE) plot(IRToutputSPI27[[27]], type="test", ylim=c(0,7), main=" ") dev.off() @ \end{scriptsize} \begin{tiny} <>= suppressPackageStartupMessages(library(xtable)) print(xtable(calTable1, caption='Item calibrations for Compassion (27-F)', label='tab:SPI_27cals1')) print(xtable(calTable2, caption='Item calibrations for Irritability (27-F)', label='tab:SPI_27cals2')) print(xtable(calTable3, caption='Item calibrations for Sociability (27-F)', label='tab:SPI_27cals3')) print(xtable(calTable4, caption='Item calibrations for Well-Being (27-F)', label='tab:SPI_27cals4')) print(xtable(calTable5, caption='Item calibrations for Sensation-Seeking (27-F)', label='tab:SPI_27cals5')) print(xtable(calTable6, caption='Item calibrations for Anxiety (27-F)', label='tab:SPI_27cals6')) print(xtable(calTable7, caption='Item calibrations for Honesty (27-F)', label='tab:SPI_27cals7')) print(xtable(calTable8, caption='Item calibrations for Industry (27-F)', label='tab:SPI_27cals8')) print(xtable(calTable9, caption='Item calibrations for Intellect (27-F)', label='tab:SPI_27cals9')) print(xtable(calTable10, caption='Item calibrations for Creativity (27-F)', label='tab:SPI_27cals10')) print(xtable(calTable11, caption='Item calibrations for Impulsivity (27-F)', label='tab:SPI_27cals11')) print(xtable(calTable12, caption='Item calibrations for Attention-Seeking (27-F)', label='tab:SPI_27cals12')) print(xtable(calTable13, caption='Item calibrations for Order (27-F)', label='tab:SPI_27cals13')) @ <>= print(xtable(calTable14, caption='Item calibrations for Authoritarianism (27-F)', label='tab:SPI_27cals14')) print(xtable(calTable15, caption='Item calibrations for Charisma (27-F)', label='tab:SPI_27cals15')) print(xtable(calTable16, caption='Item calibrations for Trust (27-F)', label='tab:SPI_27cals16')) print(xtable(calTable17, caption='Item calibrations for Humor (27-F)', label='tab:SPI_27cals17')) print(xtable(calTable18, caption='Item calibrations for Emotional Expressiveness (27-F)', label='tab:SPI_27cals18')) print(xtable(calTable19, caption='Item calibrations for Art Appreciation (27-F)', label='tab:SPI_27cals19')) print(xtable(calTable20, caption='Item calibrations for Introspection (27-F)', label='tab:SPI_27cals20')) print(xtable(calTable21, caption='Item calibrations for Perfectionism (27-F)', label='tab:SPI_27cals21')) print(xtable(calTable22, caption='Item calibrations for Self-Control (27-F)', label='tab:SPI_27cals22')) print(xtable(calTable23, caption='Item calibrations for Conformity (27-F)', label='tab:SPI_27cals23')) print(xtable(calTable24, caption='Item calibrations for Adaptability (27-F)', label='tab:SPI_27cals24')) print(xtable(calTable25, caption='Item calibrations for Easy-Goingness (27-F)', label='tab:SPI_27cals25')) print(xtable(calTable26, caption='Item calibrations for Emotional Stability (27-F)', label='tab:SPI_27cals26')) print(xtable(calTable27, caption='Item calibrations for Conservatism (27-F)', label='tab:SPI_27cals27')) @ \end{tiny} \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_1IIF.pdf}}{\caption{Compassion (27-F) Item Information Functions}\label{fig:SPI27_1IIF}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_1TIF.pdf}}{\caption{Compassion (27-F) Test Information Function}\label{fig:SPI27_1TIF}} \end{floatrow} \end{figure} \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_2IIF.pdf}}{\caption{Irritability (27-F) Item Information Functions}\label{fig:SPI27_2IIF}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_2TIF.pdf}}{\caption{Irritability (27-F) Test Information Function}\label{fig:SPI27_2TIF}} \end{floatrow} \end{figure} \clearpage \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_3IIF.pdf}}{\caption{Sociability (27-F) Item Information Functions}\label{fig:SPI27_3IIF}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_3TIF.pdf}}{\caption{Sociability (27-F) Test Information Function}\label{fig:SPI27_3TIF}} \end{floatrow} \end{figure} \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_4IIF.pdf}}{\caption{Well-Being (27-F) Item Information Functions}\label{fig:SPI27_4IIF}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_4TIF.pdf}}{\caption{Well-Being (27-F) Test Information Function}\label{fig:SPI27_4TIF}} \end{floatrow} \end{figure} \clearpage \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_5IIF.pdf}}{\caption{Sensation-Seeking (27-F) Item Information Functions}\label{fig:SPI27_5IIF}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_5TIF.pdf}}{\caption{Sensation-Seeking (27-F) Test Information Function}\label{fig:SPI27_5TIF}} \end{floatrow} \end{figure} \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_6IIF.pdf}}{\caption{Anxiety (27-F) Item Information Functions}\label{fig:SPI27_6IIF}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_6TIF.pdf}}{\caption{Anxiety (27-F) Test Information Function}\label{fig:SPI27_6TIF}} \end{floatrow} \end{figure} \clearpage \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_7IIF.pdf}}{\caption{Honesty (27-F) Item Information Functions}\label{fig:SPI27_7IIF}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_7TIF.pdf}}{\caption{Honesty (27-F) Test Information Function}\label{fig:SPI27_7TIF}} \end{floatrow} \end{figure} \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_8IIF.pdf}}{\caption{Industry (27-F) Item Information Functions}\label{fig:SPI27_8IIF}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_8TIF.pdf}}{\caption{Industry (27-F) Test Information Function}\label{fig:SPI27_8TIF}} \end{floatrow} \end{figure} \clearpage \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_9IIF.pdf}}{\caption{Intellect (27-F) Item Information Functions}\label{fig:SPI27_9IIF}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_9TIF.pdf}}{\caption{Intellect (27-F) Test Information Function}\label{fig:SPI27_9TIF}} \end{floatrow} \end{figure} \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_10IIF.pdf}}{\caption{Creativity (27-F) Item Information Functions}\label{fig:SPI27_10IIF}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_10TIF.pdf}}{\caption{Creativity (27-F) Test Information Function}\label{fig:SPI27_10TIF}} \end{floatrow} \end{figure} \clearpage \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_11IIF.pdf}}{\caption{Impulsivity (27-F) Item Information Functions}\label{fig:SPI27_11IIF}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_11TIF.pdf}}{\caption{Impulsivity (27-F) Test Information Function}\label{fig:SPI27_11TIF}} \end{floatrow} \end{figure} \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_12IIF.pdf}}{\caption{Attention-Seeking (27-F) Item Information Functions}\label{fig:SPI27_12IIF}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_12TIF.pdf}}{\caption{Attention-Seeking (27-F) Test Information Function}\label{fig:SPI27_12TIF}} \end{floatrow} \end{figure} \clearpage \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_13IIF.pdf}}{\caption{Order (27-F) Item Information Functions}\label{fig:SPI27_13IIF}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_13TIF.pdf}}{\caption{Order (27-F) Test Information Function}\label{fig:SPI27_13TIF}} \end{floatrow} \end{figure} \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_14IIF.pdf}}{\caption{Authoritarianism (27-F) Item Information Functions}\label{fig:SPI27_14IIF}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_14TIF.pdf}}{\caption{Authoritarianism (27-F) Test Information Function}\label{fig:SPI27_14TIF}} \end{floatrow} \end{figure} \clearpage \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_15IIF.pdf}}{\caption{Charisma (27-F) Item Information Functions}\label{fig:SPI27_15IIF}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_15TIF.pdf}}{\caption{Charisma (27-F) Test Information Function}\label{fig:SPI27_15TIF}} \end{floatrow} \end{figure} \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_16IIF.pdf}}{\caption{Trust (27-F) Item Information Functions}\label{fig:SPI27_16IIF}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_16TIF.pdf}}{\caption{Trust (27-F) Test Information Function}\label{fig:SPI27_16TIF}} \end{floatrow} \end{figure} \clearpage \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_17IIF.pdf}}{\caption{Humor (27-F) Item Information Functions}\label{fig:SPI27_17IIF}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_17TIF.pdf}}{\caption{Humor (27-F) Test Information Function}\label{fig:SPI27_17TIF}} \end{floatrow} \end{figure} \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_18IIF.pdf}}{\caption{Emotional Expressiveness (27-F) Item Information Functions}\label{fig:SPI27_18IIF}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_18TIF.pdf}}{\caption{Emotional Expressiveness (27-F) Test Information Function}\label{fig:SPI27_18TIF}} \end{floatrow} \end{figure} \clearpage \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_19IIF.pdf}}{\caption{Art Appreciation (27-F) Item Information Functions}\label{fig:SPI27_19IIF}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_19TIF.pdf}}{\caption{Art Appreciation (27-F) Test Information Function}\label{fig:SPI27_19TIF}} \end{floatrow} \end{figure} \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_20IIF.pdf}}{\caption{Introspection (27-F) Item Information Functions}\label{fig:SPI27_20IIF}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_20TIF.pdf}}{\caption{Introspection (27-F) Test Information Function}\label{fig:SPI27_20TIF}} \end{floatrow} \end{figure} \clearpage \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_21IIF.pdf}}{\caption{Perfectionism (27-F) Item Information Functions}\label{fig:SPI27_21IIF}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_21TIF.pdf}}{\caption{Perfectionism (27-F) Test Information Function}\label{fig:SPI27_21TIF}} \end{floatrow} \end{figure} \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_22IIF.pdf}}{\caption{Self-Control (27-F) Item Information Functions}\label{fig:SPI27_22IIF}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_22TIF.pdf}}{\caption{Self-Control (27-F) Test Information Function}\label{fig:SPI27_22TIF}} \end{floatrow} \end{figure} \clearpage \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_23IIF.pdf}}{\caption{Conformity (27-F) Item Information Functions}\label{fig:SPI27_23IIF}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_23TIF.pdf}}{\caption{Conformity (27-F) Test Information Function}\label{fig:SPI27_23TIF}} \end{floatrow} \end{figure} \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_24IIF.pdf}}{\caption{Adaptability (27-F) Item Information Functions}\label{fig:SPI27_24IIF}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_24TIF.pdf}}{\caption{Adaptability (27-F) Test Information Function}\label{fig:SPI27_24TIF}} \end{floatrow} \end{figure} \clearpage \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_25IIF.pdf}}{\caption{Easy-Goingness (27-F) Item Information Functions}\label{fig:SPI27_25IIF}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_25TIF.pdf}}{\caption{Easy-Goingness (27-F) Test Information Function}\label{fig:SPI27_25TIF}} \end{floatrow} \end{figure} \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_26IIF.pdf}}{\caption{Emotional Stability (27-F) Item Information Functions}\label{fig:SPI27_26IIF}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_26TIF.pdf}}{\caption{Emotional Stability (27-F) Test Information Function}\label{fig:SPI27_26TIF}} \end{floatrow} \end{figure} \clearpage \begin{figure} \begin{floatrow} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_27IIF.pdf}}{\caption{Conservatism (27-F) Item Information Functions}\label{fig:SPI27_27IIF}} \ffigbox{\includegraphics[width=.45\textwidth]{figures/IRT/SPI27_27TIF.pdf}}{\caption{Conservatism (27-F) Test Information Function}\label{fig:SPI27_27TIF}} \end{floatrow} \end{figure} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \clearpage \pagenumbering{arabic}% resets `page` counter to 1 \renewcommand*{\thepage}{A5-\arabic{page}} \renewcommand{\thefigure}{A5-\arabic{figure}} \setcounter{figure}{0} \renewcommand{\thetable}{A5-\arabic{table}} \setcounter{table}{0} \begin{center} \begin{Large} \textbf{Appendix E} \subsection*{Recommendations for Personality Assessment Using SPI Forms} \addcontentsline{toc}{subsection}{E: Recommendations for Personality Assessment Using SPI Forms} \label{sec:appendixE} \end{Large} {\bf David M. Condon}\footnote[$\star$]{This document is an appendix to \textit{The SAPA Personality Inventory}. Correspondence should be directed to \href{mailto:david-condon@northwestern.edu}{david-condon@northwestern.edu}, Department of Medical Social Sciences, Northwestern University, Chicago, Illinois. Please use the following citation information: Condon, D.M. (2017). The SAPA Personality Inventory: an empirically-derived, hierarchically-organized self-report personality assessment model.} \end{center} \vskip -0.5cm %%%%%%%%%%%%%%%%%% %Add Breaking Line \begin{center} \linethickness{0.5mm} \line(1,0){300} \end{center} %%%%%%%%%%%%%%%%%% \begin{scriptsize} <>= # In this first step we are simply getting things ready by installing and loading necessary packages and functions. These are hidden in the pdf output. rm(list=setdiff(ls(), c("filepathdata", "filepathfig"))) suppressPackageStartupMessages(library(psych)) load(paste(filepathdata, "keysAndScalesInfo.Rdata", sep = "")) @ \end{scriptsize} Since beginning this project, I've been increasingly asked for advice about the ``best'' way to assess personality. No surprise -- people want a simple answer to a complicated question. Of course, there isn't a simple answer but I figured it might save me time in the long run to write out a decision-tree-like response that works for most circumstances. My responses are limited to the SPI measures because (a) I feel most qualified to advocate for these tools; (b) I think they're pretty good (though there are other useful options too\footnote{Consider the \href{http://www.colby.edu/psych/personality-lab/}{60-item BFI-2} and the \href{http://www.personal.psu.edu/~j5j/IPIP/ipipneo120.htm}{120-Item version of the IPIP-NEO}.}); and (c) I am biased by a desire to see them used more widely. For each of the questions below (that is, questions from me to you), I have provided advice based on the most likely responses. I reserve the right to update my answers later and, in fact, it would be useful for me know if these recommendations are not helpful, for whatever reason. Feel free to contact me with this feedback; courtesy is always appreciated. Please remember that these items and scales are freely-available for use in the public domain. I make absolutely no warranties about their use or mis-use by others. %%%%%%%%%%%%%%%%%% %Add Breaking Line \begin{center} \linethickness{0.5mm} \line(1,0){300} \end{center} %%%%%%%%%%%%%%%%%% \noindent \textbf{Q1: Why do you want to assess personality?} \vspace{2mm} \noindent \uline{\textit{1. To understand myself better}}: Great. I applaud this. Know thyself, etc. I recommend you take a personality assessment like those available for free online at \href{https://www.sapa-project.org}{sapa-project.org} or \href{http://www.outofservice.com/bigfive}{outofservice.com} or \href{http://www.personal.psu.edu/~j5j/IPIP/ipipneo300.htm}{IPIP-NEO}. These sites are managed by academic personality researchers and they will give you feedback on your personality that is based on your responses. You might also consider how these sites compare to other tools you can find on the internet. I dislike the MBTI\textsuperscript{\textregistered} and would caution anyone against paying a lot of money to take it\footnote{This is not an uninformed opinion. I think it's fine as a means of encouraging people to recognize the importance of personality. But it's less valid and less insightful than more recently developed tools of personality science. See \href{https://sapa-project.org/dmc/blog/myersbriggs.html}{https://sapa-project.org/dmc/blog/myersbriggs.html}} But, it is among the most widely-used measures, thanks to having been around for 75 years and a cult-like following among non-scientists. My own view is that, if you're going to use a measure with poor validity, you might as well use something more fun --- why not astrology or personality-typologies based on your favorite color, \href{https://youtu.be/IxQS6lfn0yU}{comic}, book,\footnote{\href{https://my.pottermore.com/patronus}{Grey squirrel}.} or film? \vspace{2mm} \centerline{[STOP]} \centerline{If this is your response, my advice ends here.} \vspace{2mm} \vspace{2mm} \noindent \uline{\textit{2. To make an important decision based upon your own or someone else's personality}}: Yep. I think most people --- myself included --- choose to believe that many decisions in life can be optimized by identifying the choice that fits who we are as a person... based on our individual differences. Unfortunately, I am reluctant to give advice about personality assessment for this type of purpose. Others will do it (often for a fee) but, with a few exceptions, I don't think our field has advanced enough yet to be prescriptive about important outcomes at the level of the individual (though I do think we're getting pretty good at making personality-based predictions \textit{for groups}). Those who use personality assessments as criteria in important decisions also take other factors into account. Incidentally, I got started on this project because it seemed logical to me that the development of more empirically-derived measurement tools will (eventually) lead to more sophisticated models for predicting important life outcomes. The tools described here are a step in that direction but not an endpoint. \vspace{2mm} \centerline{[STOP]} \centerline{If this is your response, my advice ends here.} \centerline{Many people who give this response also endorse response 3 below.} \centerline{If this includes you, keep reading.} \vspace{2mm} \vspace{2mm} \noindent \uline{\textit{3. To explore the ways that personality relates to other topics}}: I like you. There are lots of options and I am happy to talk through the many alternatives if the following recommendations don't work for your needs. I believe the measures described here are as good or better than other widely-used options. But, I'm biased, so don't take my word for it! This Appendix is part of a very long manual that shows every detail of the steps taken to develop these tools, and I encourage you to spend some time understanding my approach. As I'm writing this, preliminary validation analyses have been completed but not yet published. Much more validation work is needed. To be clear, \textbf{my long-term goal is to maximize the predictive utility of self-report personality assessments}. Like you, I want to understand how and how much personality relates to other stuff. I hope that you benefit from the use of these scales but also that you will provide informed critiques and suggestions that help me and others to achieve this goal. \vspace{2mm} \centerline{[MOVE ON TO QUESTION 2.]} \vspace{2mm} \vspace{2mm} \noindent \textbf{Q2: How many items are you willing to give each respondent? In other words, how much time do you have to assess each respondent?} Note that my answers to this question assume that you intend to administer all of the same items to each participant. If you're willing and able to use a computer adaptive test or a planned missingness design, my answers would differ --- see question 3. \vspace{2mm} \noindent \uline{\textit{1. Very few items - less than 3 minutes}}: You don't have time for much. If you have reason to believe that one or more specific personality traits are relevant for your research, look for a match among the 27 factor scales listed in Table \ref{tab:orgtable2}. Each of these scales contains 5 items and you have enough time to ask roughly 15 or fewer items in total, depending on your sample. \vspace{2mm} \centerline{[STOP]} \centerline{If this is your response, my advice ends here.} \centerline{Try a \href{https://scholar.google.com/}{literature review} to identify personality traits that match your criterion.} \vspace{2mm} \vspace{2mm} \noindent \uline{\textit{2. 20 to 45 items - 5 minutes $+/-$ 2 minutes}}: One option is the \hyperref[sec:SPI205]{SPI-20-5} (this is the 5th assessment form listed in Appendix \hyperref[sec:appendixF]{F}). It takes 3 to 5 minutes to complete and it's the shortest SPI measure that I'm willing to recommend. As denoted by the name, it contains 20 items and can be scored to provide 5-factor scores that are consistent with Big Five research. The factors are Extraversion, Neuroticism, Conscientiousness, Openness and Agreeableness. These scales correlate reasonably well with the ``mini-IPIP'' measures \citep{Donnellan:06} as shown in Figure \ref{fig:scalesCorValid}, though they actually correlate \emph{considerably} better with the 50 and 100 item versions of the IPIP Big Five Factor Markers. A better option would be the \hyperref[sec:SPI455]{SPI-45-5} (the 4th assessment form in Appendix \hyperref[sec:appendixF]{F}). These 45 items provide scores on the same 5-factors with greater precision (i.e., higher correlations with IPIP-based alternatives) and higher internal consistency (each form in Appendix \hyperref[sec:appendixF]{F} includes a table of internal consistencies at the end). Figure \ref{fig:scalesCorValid} shows that correlations with the 50 items of the Big Five Factor Markers \citep{Goldberg:92, Goldberg:99a} after correcting for item overlap range from .79 to .84.\footnote{Correlations with the 44-item Big Five Inventory \citep{John:1999} and the 60-item Big Five Inventory-2 \citep{Soto:2016jr} are forthcoming.} It takes about 7 minutes. If you are interested in more narrowly-defined facets, you might consider giving the ``\hyperref[sec:SPI2727]{signal items}'' listed in the 6th form in Appendix \hyperref[sec:appendixF]{F}. This list includes one item from each of the 27 facets. The items were identified based on informativeness using item-response scoring in the full sample of \Sexpr{prettyNum(round(dim(allSample)[1],0), big.mark=",", scientific=F)} participants (see the manuscript associated with this Appendix for more information about the sample). While single-item measures can be problematic for many reasons (particularly with small samples), this approach could be useful for hypothesis generation and/or exploratory analyses. I have also been asked to develop an option that blends the 20-item 5-factor form and the 27-factor signal items form. In response, I have developed a 35-item measure that contains all of the signal items and 8 additional items in order to produce acceptably reliable 5-factor scores. I do not believe these 5-factor scales are as good as those in the 20-item measure but they remain a good option for researchers who want the shortest possible hierarchical measure. This is the last form in Appendix \hyperref[sec:appendixF]{F}. \vspace{2mm} \centerline{[STOP]} \centerline{If this is your response, my advice ends here.} \vspace{2mm} \vspace{2mm} \noindent \uline{\textit{3. I want a good measure and I'm willing to give up to 15 minutes}}: Nice. I recommend the \hyperref[sec:SPI81275]{SPI-81-27\&5} (this is the 2nd assessment form in Appendix \hyperref[sec:appendixF]{F}). This measure has 81 items and it can be scored to produce good 5-factor scores \textit{and} coarse measures of the 27-factor scales. The 27-factor scores provide measurement of more narrowly defined constructs than the broader 5-factor scores; for most outcomes, the 27-factor scores are likely to be more highly predictive than the 5-factor outcomes. Correlations between the 5-factor scores for these 81 items and the 100 items of the IPIP Big Five Factor Markers are shown in Figure \ref{fig:scalesCorValid} and range from .78 to .85. These 81 items will take most participants 10 to 14 minutes. \vspace{2mm} \centerline{[STOP]} \centerline{If this is your response, my advice ends here.} \vspace{2mm} \vspace{2mm} \noindent \uline{\textit{4. I want to do it right because personality is a critical component of my research}}: Yay! Give the \hyperref[sec:SPI135275]{SPI-135-27\&5} (the 1st assessment form in Appendix \hyperref[sec:appendixF]{F}). Or give an even larger item set in order to compare the SPI scales with others and/or conduct exploratory analyses of structure. \vspace{2mm} \centerline{[MOVE ON TO QUESTION 3.]} \vspace{2mm} \vspace{2mm} \noindent \textbf{Q3: How will you collect and score the data?} There are really \textit{two} questions to consider here --- mode and method. Electronic survey administration is generally preferred because it helps to reduce errors stemming from human coding and higher rates of accidental data loss. Unfortunately, some populations lack familiarity with or access to the electronic equipment needed for assessments in this mode. In these cases, assessment via paper forms or proxy administration (in-person or by telephone) are good alternatives. Most researchers using paper forms administer all of the same items to all participants, though some occasionally do use two or more forms with overlapping subsets of items. Proxy administration (e.g., verbal administration of the items by another person or by interactive voice response survey software) is particularly useful when administering items to illiterate populations. With both proxy and electronic administrations, researchers have considerable flexibility in terms of the method of administration. It is most common to administer all of the same items in the same sequential order to all participants. However, there are often good reasons for administering various subsets of the items using a planned missingness or computer adaptive testing design. Planned missingness can be useful because it allows for the administration of more items than any single participant is willing or able to answer. Computer adaptive tests are useful because they allow for precise and valid individual assessments with relatively few items. This reduces participant burden as well as the time and resources needed for data collection. \vspace{2mm} \centerline{[MOVE ON TO QUESTION 3.]} \vspace{2mm} \vspace{2mm} \noindent \textbf{Q4: How should the scores be interpreted?} This is a deceptively complicated question. The technical, statistical answer is easy enough --- the scores should be compared to some reference group. The trouble is that it's hard to say which one. The only generalizable answer seems to be "Well, it depends on the rationale for assessment in the first place." We know that there are age, gender, and cultural effects on personality. Many other grouping variables are also likely to be relevant as are combinations of grouping variables (old male Australians, young female Peruvians). The extent to which these variables should be ``controlled for'' when evaluating an individual's score(s) should depend on the extent to which these variables are relevant for the outcome under considerations. We have generated SPI-27 and SPI-5 norms based on age and gender (see Appendix \hyperref[sec:appendixF]{F}); the publicly-available data used to develop these norms could be used to generate many other categories of norms as well. Of course, the use of norms based on ``normal'' populations tends to indicate that most people are mainly average on most personality traits. Many people find this disappointing, presumably because we all suffer from biased anecdotal experiences that lead us to conclude that we're unique in many ways. Yet, when all the dimensions of individual differences are reduced down to only 5 factors and our scores are compared to others of our age and gender... it turns out that most people are mostly average. There is also the question of what should be done when statistically deviant scores are identified. The development of guidance will require considerably more validation data than is currently available and the nature of the guidance itself will likely vary considerably across the facets. Extreme scores on many of the facets are likely to indicate maladaptive behaviors, but the nature and consequence of this maladaptiveness will differ by facet. Extremely low self-discipline, for example, may generally be positively correlated with poor physical health, but the merits of an immediate intervention will likely depend on many other factors (e.g., more direct physical evidence of poor health). For the time being, the most advisable course of action in response to extreme scores in one or more facets is follow-up assessment in the domains where problems are suspected. \begin{scriptsize} <>= load(paste(filepathdata, "IRToutputSPI27.Rdata", sep = "")) load(paste(filepathdata, "keysAndScalesInfo.Rdata", sep = "")) ### LET'S FIGURE OUT THE BEST ORDERING OF THE 5 FACTOR ITEMS ### ### THE ADMIN ORDER MAY VARY DEPENDING ON AUC FOR IRT OUTPUT ### factors <- 5 vecNames <- list() for (i in 1:factors) { x <- irt.fa(allSample[,keys.list5$items[[i]]]) y <- rownames(x$plot$AUC) vecNames <- append(vecNames, y) } SPI5itemOrderByFactor <- unlist(vecNames) ### BREAK THAT LIST UP TO GET LISTS OF THE ITEMS BY FACTOR ### for (i in 1:factors) { assign(paste("SPI_5_", i, "items", sep=""), SPI5itemOrderByFactor[(1+(14*(i-1))):(14+(14*(i-1)))]) } itemsPerRound <- length(SPI5itemOrderByFactor)/factors round1 <- list() for (i in 1:factors) { x <- 1+(itemsPerRound*(i-1)) round1 <- append(round1, x) } round1 <- unlist(round1) round1 SPI5itemsRound1 <- SPI5itemOrderByFactor[round1] for (i in 2:itemsPerRound) { assign(paste("SPI5itemsRound", i, sep=""), SPI5itemOrderByFactor[round1+(1*(i-1))]) } rm(factors, i, round1, vecNames, x, y) ### HERE WE HAVE THE ORDERING OF THE 70 ITEMS ### SPI70_5itemsAdminOrder <- c(SPI5itemsRound1, SPI5itemsRound2, SPI5itemsRound3, SPI5itemsRound4, SPI5itemsRound5, SPI5itemsRound6, SPI5itemsRound7, SPI5itemsRound8, SPI5itemsRound9, SPI5itemsRound10, SPI5itemsRound11, SPI5itemsRound12, SPI5itemsRound13, SPI5itemsRound14) ### NOW FIGURE OUT THE BEST ORDERING OF THE 27 FACTOR ITEMS ### factors <- 27 vecValues <- list() facetNum <- list() vecAUC <- list() vecNames <- list() for (i in 1:factors) { x <- rownames(IRToutputSPI27[[i]]$plot$AUC) for (j in 1:5) { y <- ifelse((x[j] %in% SPI5itemOrderByFactor), 1, 0) vecValues <- append(vecValues, y) facetNum <- append(facetNum, i) vecAUC <- append(vecAUC, IRToutputSPI27[[i]]$plot$AUC[j]) vecNames <- append(vecNames, x[j]) } SPI27vs5df <- data.frame(unlist(vecValues), unlist(facetNum), unlist(vecAUC)) colnames(SPI27vs5df) <- c("inSPI5", "facetNum", "AUCbyFacet") rownames(SPI27vs5df) <- vecNames SPI27vs5df <- SPI27vs5df[order(SPI27vs5df[,"facetNum"], -SPI27vs5df[,"AUCbyFacet"]),] } ### First, we have a list of the items in order by FACTOR ### SPI27itemOrderByFactor <- rownames(SPI27vs5df) ### BREAK THAT LIST UP TO GET LISTS OF THE ITEMS BY FACTOR ### for (i in 1:factors) { assign(paste("SPI_27_", i, "items", sep=""), SPI27itemOrderByFactor[(1+(5*(i-1))):(5+(5*(i-1)))]) } itemsPerRound <- length(SPI27itemOrderByFactor)/factors round1 <- list() for (i in 1:factors) { x <- 1+(itemsPerRound*(i-1)) round1 <- append(round1, x) } round1 <- unlist(round1) round1 SPI27itemsRound1 <- SPI27itemOrderByFactor[round1] for (i in 2:itemsPerRound) { assign(paste("SPI27itemsRound", i, sep=""), SPI27itemOrderByFactor[round1+(1*(i-1))]) } rm(i, itemsPerRound, j, round1, x, y) ### HERE WE HAVE THE ORDERING OF THE 135 ITEMS ### SPI135_27_5itemsAdminOrder <- c(SPI27itemsRound1, SPI27itemsRound2, SPI27itemsRound3, SPI27itemsRound4, SPI27itemsRound5) ### CALCULATE THE ICs FOR THE 70 AND 135 ITEM SOLUTIONS ### ### WE HAVE ALREADY DONE THIS BUT CONFIRM THAT THEY MATCH ### rely <- data.frame() for (i in 1:27) { alpha1 <- suppressWarnings(alpha(allSample[,eval(parse(text=paste("SPI_27_", i, "items", sep="")))], check.keys=TRUE)) alpha <- round(alpha1$total$raw_alpha, 2) unidim <- round(alpha1$Unidim$Unidim, 2) count <- length(alpha1$keys) rows <- c(alpha, unidim, count) rely <- rbind(rely, rows) } colnames(rely) <- c("Alpha_135", "Unidimensionality_135", "Item Count_135") rownames(rely) <- c("Factor1", "Factor2", "Factor3", "Factor4", "Factor5", "Factor6", "Factor7", "Factor8", "Factor9", "Factor10", "Factor11", "Factor12", "Factor13", "Factor14", "Factor15", "Factor16", "Factor17", "Factor18", "Factor19", "Factor20", "Factor21", "Factor22", "Factor23", "Factor24", "Factor25", "Factor26", "Factor27") SPI_27_135_ICs <- rely SPI_27_135_ICs ### THESE SHOULD ALL BE ZERO SPI_27_135_ICs[,1:2]-SPI_27_ICs[,1:2] ### NOW FOR THE 5 FACTORS ### rely <- data.frame() for (i in 1:5) { alpha1 <- suppressWarnings(alpha(allSample[,eval(parse(text=paste("SPI_5_", i, "items", sep="")))], check.keys=TRUE)) alpha <- round(alpha1$total$raw_alpha, 2) unidim <- round(alpha1$Unidim$Unidim, 2) count <- length(alpha1$keys) rows <- c(alpha, unidim, count) rely <- rbind(rely, rows) } colnames(rely) <- c("Alpha_135", "Unidimensionality_135", "Item Count_135") rownames(rely) <- c("Extraversion", "Neuroticism", "Conscientiousness", "Agreeableness", "Openness") SPI_5_135_ICs <- rely SPI_5_135_ICs ### THESE SHOULD ALL BE ZERO SPI_5_135_ICs[,1:2]-SPI_5_ICs ### NOW, JUST DO WHAT IS NEEDED TO IDENTIFY SHORTER FORMS AND ICs ### ### HERE WE HAVE THE ORDERING OF THE 108 ITEMS ### SPI108_27_5itemsAdminOrder <- c(SPI27itemsRound1, SPI27itemsRound2, SPI27itemsRound3, SPI27itemsRound4) ### GET THE ICs ### for (i in 1:27) { assign(paste("SPI_27_", i, "items_108", sep=""), intersect(eval(parse(text=paste("SPI_27_", i, "items", sep=""))), SPI108_27_5itemsAdminOrder)) } rely <- data.frame() for (i in 1:27) { alpha1 <- suppressWarnings(alpha(allSample[,eval(parse(text=paste("SPI_27_", i, "items_108", sep="")))], check.keys=TRUE)) alpha <- round(alpha1$total$raw_alpha, 2) unidim <- round(alpha1$Unidim$Unidim, 2) count <- length(alpha1$keys) rows <- c(alpha, unidim, count) rely <- rbind(rely, rows) } colnames(rely) <- c("Alpha_108", "Unidimensionality_108", "Item Count_108") rownames(rely) <- c("Factor1", "Factor2", "Factor3", "Factor4", "Factor5", "Factor6", "Factor7", "Factor8", "Factor9", "Factor10", "Factor11", "Factor12", "Factor13", "Factor14", "Factor15", "Factor16", "Factor17", "Factor18", "Factor19", "Factor20", "Factor21", "Factor22", "Factor23", "Factor24", "Factor25", "Factor26", "Factor27") SPI_27_108_ICs <- rely SPI_27_108_ICs for (i in 1:5) { assign(paste("SPI_5_", i, "items_108", sep=""), intersect(eval(parse(text=paste("SPI_5_", i, "items", sep=""))), SPI108_27_5itemsAdminOrder)) } rely <- data.frame() for (i in 1:5) { alpha1 <- suppressWarnings(alpha(allSample[,eval(parse(text=paste("SPI_5_", i, "items_108", sep="")))], check.keys=TRUE)) alpha <- round(alpha1$total$raw_alpha, 2) unidim <- round(alpha1$Unidim$Unidim, 2) count <- length(alpha1$keys) rows <- c(alpha, unidim, count) rely <- rbind(rely, rows) } colnames(rely) <- c("Alpha_108", "Unidimensionality_108", "Item Count_108") rownames(rely) <- c("Extraversion", "Neuroticism", "Conscientiousness", "Agreeableness", "Openness") SPI_5_108_ICs <- rely SPI_5_108_ICs ### HERE WE HAVE THE ORDERING OF THE 81 ITEMS ### SPI81_27_5itemsAdminOrder <- c(SPI27itemsRound1, SPI27itemsRound2, SPI27itemsRound3) ### GET THE ICs ### for (i in 1:27) { assign(paste("SPI_27_", i, "items_81", sep=""), intersect(eval(parse(text=paste("SPI_27_", i, "items", sep=""))), SPI81_27_5itemsAdminOrder)) } rely <- data.frame() for (i in 1:27) { alpha1 <- suppressWarnings(alpha(allSample[,eval(parse(text=paste("SPI_27_", i, "items_81", sep="")))], check.keys=TRUE)) alpha <- round(alpha1$total$raw_alpha, 2) unidim <- round(alpha1$Unidim$Unidim, 2) count <- length(alpha1$keys) rows <- c(alpha, unidim, count) rely <- rbind(rely, rows) } colnames(rely) <- c("Alpha_81", "Unidimensionality_81", "Item Count_81") rownames(rely) <- c("Factor1", "Factor2", "Factor3", "Factor4", "Factor5", "Factor6", "Factor7", "Factor8", "Factor9", "Factor10", "Factor11", "Factor12", "Factor13", "Factor14", "Factor15", "Factor16", "Factor17", "Factor18", "Factor19", "Factor20", "Factor21", "Factor22", "Factor23", "Factor24", "Factor25", "Factor26", "Factor27") SPI_27_81_ICs <- rely SPI_27_81_ICs for (i in 1:5) { assign(paste("SPI_5_", i, "items_81", sep=""), intersect(eval(parse(text=paste("SPI_5_", i, "items", sep=""))), SPI81_27_5itemsAdminOrder)) } rely <- data.frame() for (i in 1:5) { alpha1 <- suppressWarnings(alpha(allSample[,eval(parse(text=paste("SPI_5_", i, "items_81", sep="")))], check.keys=TRUE)) alpha <- round(alpha1$total$raw_alpha, 2) unidim <- round(alpha1$Unidim$Unidim, 2) count <- length(alpha1$keys) rows <- c(alpha, unidim, count) rely <- rbind(rely, rows) } colnames(rely) <- c("Alpha_81", "Unidimensionality_81", "Item Count_81") rownames(rely) <- c("Extraversion", "Neuroticism", "Conscientiousness", "Agreeableness", "Openness") SPI_5_81_ICs <- rely SPI_5_81_ICs ### HERE WE HAVE THE ORDERING OF THE 54 ITEMS ### SPI54_27_5itemsAdminOrder <- c(SPI27itemsRound1, SPI27itemsRound2) ### GET THE ICs ### for (i in 1:27) { assign(paste("SPI_27_", i, "items_54", sep=""), intersect(eval(parse(text=paste("SPI_27_", i, "items", sep=""))), SPI54_27_5itemsAdminOrder)) } rely <- data.frame() for (i in 1:27) { alpha1 <- suppressWarnings(alpha(allSample[,eval(parse(text=paste("SPI_27_", i, "items_54", sep="")))], check.keys=TRUE)) alpha <- round(alpha1$total$raw_alpha, 2) unidim <- round(alpha1$Unidim$Unidim, 2) count <- length(alpha1$keys) rows <- c(alpha, unidim, count) rely <- rbind(rely, rows) } colnames(rely) <- c("Alpha_54", "Unidimensionality_54", "Item Count_54") rownames(rely) <- c("Factor1", "Factor2", "Factor3", "Factor4", "Factor5", "Factor6", "Factor7", "Factor8", "Factor9", "Factor10", "Factor11", "Factor12", "Factor13", "Factor14", "Factor15", "Factor16", "Factor17", "Factor18", "Factor19", "Factor20", "Factor21", "Factor22", "Factor23", "Factor24", "Factor25", "Factor26", "Factor27") SPI_27_54_ICs <- rely SPI_27_54_ICs for (i in 1:5) { assign(paste("SPI_5_", i, "items_54", sep=""), intersect(eval(parse(text=paste("SPI_5_", i, "items", sep=""))), SPI54_27_5itemsAdminOrder)) } rely <- data.frame() for (i in 1:5) { alpha1 <- suppressWarnings(alpha(allSample[,eval(parse(text=paste("SPI_5_", i, "items_54", sep="")))], check.keys=TRUE)) alpha <- round(alpha1$total$raw_alpha, 2) unidim <- round(alpha1$Unidim$Unidim, 2) count <- length(alpha1$keys) rows <- c(alpha, unidim, count) rely <- rbind(rely, rows) } colnames(rely) <- c("Alpha_54", "Unidimensionality_54", "Item Count_54") rownames(rely) <- c("Extraversion", "Neuroticism", "Conscientiousness", "Agreeableness", "Openness") SPI_5_54_ICs <- rely SPI_5_54_ICs ### HERE WE HAVE THE ORDERING OF THE 27 ITEMS ### SPI27_27_5itemsAdminOrder <- c(SPI27itemsRound1) ### GET THE ICs ### for (i in 1:27) { assign(paste("SPI_27_", i, "items_27", sep=""), intersect(eval(parse(text=paste("SPI_27_", i, "items", sep=""))), SPI27_27_5itemsAdminOrder)) } alpha <- rep(NA, 27) unidim <- rep(NA, 27) count <- rep(1, 27) rely <- data.frame(alpha, unidim, count) colnames(rely) <- c("Alpha_27", "Unidimensionality_27", "Item Count_27") rownames(rely) <- c("Factor1", "Factor2", "Factor3", "Factor4", "Factor5", "Factor6", "Factor7", "Factor8", "Factor9", "Factor10", "Factor11", "Factor12", "Factor13", "Factor14", "Factor15", "Factor16", "Factor17", "Factor18", "Factor19", "Factor20", "Factor21", "Factor22", "Factor23", "Factor24", "Factor25", "Factor26", "Factor27") SPI_27_27_ICs <- rely SPI_27_27_ICs for (i in 1:5) { assign(paste("SPI_5_", i, "items_27", sep=""), intersect(eval(parse(text=paste("SPI_5_", i, "items", sep=""))), SPI27_27_5itemsAdminOrder)) } rely <- data.frame() for (i in 1:5) { alpha1 <- suppressWarnings(alpha(allSample[,eval(parse(text=paste("SPI_5_", i, "items_27", sep="")))], check.keys=TRUE)) alpha <- round(alpha1$total$raw_alpha, 2) unidim <- round(alpha1$Unidim$Unidim, 2) count <- length(alpha1$keys) rows <- c(alpha, unidim, count) rely <- rbind(rely, rows) } colnames(rely) <- c("Alpha_27", "Unidimensionality_27", "Item Count_27") rownames(rely) <- c("Extraversion", "Neuroticism", "Conscientiousness", "Agreeableness", "Openness") SPI_5_27_ICs <- rely SPI_5_27_ICs ### GET THE ICs FOR THE 45 AND 20 ITEM 5 FACTOR SCALES ### SPI_5_70_ICs <- SPI_5_135_ICs colnames(SPI_5_70_ICs) <- c("Alpha_70", "Unidimensionality_70", "Item Count_70") SPI45_5itemsAdminOrder <- c(SPI5itemsRound1, SPI5itemsRound2, SPI5itemsRound3, SPI5itemsRound4, SPI5itemsRound5, SPI5itemsRound6, SPI5itemsRound7, SPI5itemsRound8, SPI5itemsRound9) for (i in 1:5) { assign(paste("SPI_5_", i, "items_45", sep=""), intersect(eval(parse(text=paste("SPI_5_", i, "items", sep=""))), SPI45_5itemsAdminOrder)) } rely <- data.frame() for (i in 1:5) { alpha1 <- suppressWarnings(alpha(allSample[,eval(parse(text=paste("SPI_5_", i, "items_45", sep="")))], check.keys=TRUE)) alpha <- round(alpha1$total$raw_alpha, 2) unidim <- round(alpha1$Unidim$Unidim, 2) count <- length(alpha1$keys) rows <- c(alpha, unidim, count) rely <- rbind(rely, rows) } colnames(rely) <- c("Alpha_45", "Unidimensionality_45", "Item Count_45") rownames(rely) <- c("Extraversion", "Neuroticism", "Conscientiousness", "Agreeableness", "Openness") SPI_5_45_ICs <- rely SPI_5_45_ICs SPI20_5itemsAdminOrder <- c(SPI5itemsRound1, SPI5itemsRound2, SPI5itemsRound3, SPI5itemsRound4) for (i in 1:5) { assign(paste("SPI_5_", i, "items_20", sep=""), intersect(eval(parse(text=paste("SPI_5_", i, "items", sep=""))), SPI20_5itemsAdminOrder)) } rely <- data.frame() for (i in 1:5) { alpha1 <- suppressWarnings(alpha(allSample[,eval(parse(text=paste("SPI_5_", i, "items_20", sep="")))], check.keys=TRUE)) alpha <- round(alpha1$total$raw_alpha, 2) unidim <- round(alpha1$Unidim$Unidim, 2) count <- length(alpha1$keys) rows <- c(alpha, unidim, count) rely <- rbind(rely, rows) } colnames(rely) <- c("Alpha_20", "Unidimensionality_20", "Item Count_20") rownames(rely) <- c("Extraversion", "Neuroticism", "Conscientiousness", "Agreeableness", "Openness") SPI_5_20_ICs <- rely SPI_5_20_ICs SPI_5_all_ICs <- cbind(SPI_5_20_ICs, SPI_5_27_ICs, SPI_5_45_ICs, SPI_5_54_ICs, SPI_5_70_ICs, SPI_5_81_ICs, SPI_5_108_ICs, SPI_5_135_ICs) SPI_27_all_ICs <- cbind(SPI_27_27_ICs, SPI_27_54_ICs, SPI_27_81_ICs, SPI_27_108_ICs, SPI_27_135_ICs) ### MAKE KEYS FOR THE 5 FACTOR SOLUTIONS ### for (i in 1:5) { list <- eval(parse(text=paste("SPI_5_", i, "items", sep=""))) alpha1 <- suppressWarnings(alpha(allSample[,list], check.keys=TRUE)) keys <- alpha1$keys keyed <- list() for (j in 1:length(keys)) { keyed <- append(keyed, ifelse(keys[j] == 1, list[j], paste("-", list[j], sep=""))) } assign(paste("SPI_5_135_", i, "keyed", sep=""), unlist(keyed)) } keys.list135 = list( SPI5_135_E = SPI_5_135_1keyed, SPI5_135_N = SPI_5_135_2keyed, SPI5_135_C = SPI_5_135_3keyed, SPI5_135_A = SPI_5_135_4keyed, SPI5_135_O = SPI_5_135_5keyed) for (i in 1:5) { list <- eval(parse(text=paste("SPI_5_", i, "items_108", sep=""))) alpha1 <- suppressWarnings(alpha(allSample[,list], check.keys=TRUE)) keys <- alpha1$keys keyed <- list() for (j in 1:length(keys)) { keyed <- append(keyed, ifelse(keys[j] == 1, list[j], paste("-", list[j], sep=""))) } assign(paste("SPI_5_108_", i, "keyed", sep=""), unlist(keyed)) } keys.list108 = list( SPI5_108_E = SPI_5_108_1keyed, SPI5_108_N = SPI_5_108_2keyed, SPI5_108_C = SPI_5_108_3keyed, SPI5_108_A = SPI_5_108_4keyed, SPI5_108_O = SPI_5_108_5keyed) for (i in 1:5) { list <- eval(parse(text=paste("SPI_5_", i, "items_81", sep=""))) alpha1 <- suppressWarnings(alpha(allSample[,list], check.keys=TRUE)) keys <- alpha1$keys keyed <- list() for (j in 1:length(keys)) { keyed <- append(keyed, ifelse(keys[j] == 1, list[j], paste("-", list[j], sep=""))) } assign(paste("SPI_5_81_", i, "keyed", sep=""), unlist(keyed)) } keys.list81 = list( SPI5_81_E = SPI_5_81_1keyed, SPI5_81_N = SPI_5_81_2keyed, SPI5_81_C = SPI_5_81_3keyed, SPI5_81_A = SPI_5_81_4keyed, SPI5_81_O = SPI_5_81_5keyed) for (i in 1:5) { list <- eval(parse(text=paste("SPI_5_", i, "items_54", sep=""))) alpha1 <- suppressWarnings(alpha(allSample[,list], check.keys=TRUE)) keys <- alpha1$keys keyed <- list() for (j in 1:length(keys)) { keyed <- append(keyed, ifelse(keys[j] == 1, list[j], paste("-", list[j], sep=""))) } assign(paste("SPI_5_54_", i, "keyed", sep=""), unlist(keyed)) } keys.list54 = list( SPI5_54_E = SPI_5_54_1keyed, SPI5_54_N = SPI_5_54_2keyed, SPI5_54_C = SPI_5_54_3keyed, SPI5_54_A = SPI_5_54_4keyed, SPI5_54_O = SPI_5_54_5keyed) for (i in 1:5) { list <- eval(parse(text=paste("SPI_5_", i, "items_45", sep=""))) alpha1 <- suppressWarnings(alpha(allSample[,list], check.keys=TRUE)) keys <- alpha1$keys keyed <- list() for (j in 1:length(keys)) { keyed <- append(keyed, ifelse(keys[j] == 1, list[j], paste("-", list[j], sep=""))) } assign(paste("SPI_5_45_", i, "keyed", sep=""), unlist(keyed)) } keys.list45 = list( SPI5_45_E = SPI_5_45_1keyed, SPI5_45_N = SPI_5_45_2keyed, SPI5_45_C = SPI_5_45_3keyed, SPI5_45_A = SPI_5_45_4keyed, SPI5_45_O = SPI_5_45_5keyed) for (i in 1:5) { list <- eval(parse(text=paste("SPI_5_", i, "items_27", sep=""))) alpha1 <- suppressWarnings(alpha(allSample[,list], check.keys=TRUE)) keys <- alpha1$keys keyed <- list() for (j in 1:length(keys)) { keyed <- append(keyed, ifelse(keys[j] == 1, list[j], paste("-", list[j], sep=""))) } assign(paste("SPI_5_27_", i, "keyed", sep=""), unlist(keyed)) } keys.list27 = list( SPI5_27_E = SPI_5_27_1keyed, SPI5_27_N = SPI_5_27_2keyed, SPI5_27_C = SPI_5_27_3keyed, SPI5_27_A = SPI_5_27_4keyed, SPI5_27_O = SPI_5_27_5keyed) for (i in 1:5) { list <- eval(parse(text=paste("SPI_5_", i, "items_20", sep=""))) alpha1 <- suppressWarnings(alpha(allSample[,list], check.keys=TRUE)) keys <- alpha1$keys keyed <- list() for (j in 1:length(keys)) { keyed <- append(keyed, ifelse(keys[j] == 1, list[j], paste("-", list[j], sep=""))) } assign(paste("SPI_5_20_", i, "keyed", sep=""), unlist(keyed)) } keys.list20 = list( SPI5_20_E = SPI_5_20_1keyed, SPI5_20_N = SPI_5_20_2keyed, SPI5_20_C = SPI_5_20_3keyed, SPI5_20_A = SPI_5_20_4keyed, SPI5_20_O = SPI_5_20_5keyed) # I had to manually create the SPI_35 item lists keys.list35 <- list( SPI5_35_E = c("q_1904", "q_565", "q_1243", "q_312"), SPI5_35_N = c("q_578", "q_4252", "q_979", "q_1989"), SPI5_35_C = c("q_904", "q_1201", "q_1744", "q_1979"), SPI5_35_A = c("q_253", "q_90", "q_1763"), SPI5_35_O = c("q_240", "q_2745", "q_2754", "q_128")) rely <- data.frame() for (i in 1:5) { alpha1 <- suppressWarnings(alpha(allSample[,keys.list35[[i]]], check.keys=TRUE)) alpha <- round(alpha1$total$raw_alpha, 2) unidim <- round(alpha1$Unidim$Unidim, 2) count <- length(alpha1$keys) rows <- c(alpha, unidim, count) rely <- rbind(rely, rows) } colnames(rely) <- c("Alpha_35", "Unidimensionality_35", "Item Count_35") rownames(rely) <- c("Extraversion", "Neuroticism", "Conscientiousness", "Agreeableness", "Openness") SPI_5_35_ICs <- rely SPI_5_35_ICs SPI_5_all_ICs <- cbind(SPI_5_all_ICs, SPI_5_35_ICs) # For some reason, Extraversion is oppositely keyed for the 45 item version # Fix it here. keys.list45$SPI5_45_E = c("q_1904", "q_4243", "-q_312", "-q_565", "q_1416", "-q_1923", "-q_1027", "-q_684", "q_254") keys.listExternal = list( IPIP20Extra = c("q_262", "-q_712", "q_1803", "-q_1180"), IPIP20EmoSt = c("-q_1099", "q_248", "-q_995", "q_1677"), IPIP20Consc = c("q_962", "-q_1483", "q_1290", "-q_1397"), IPIP20Agree = c("q_1763", "-q_195", "q_844", "-q_200"), IPIP20Intel = c("q_1058", "-q_194", "-q_1088", "-q_609"), IPIP50Extra = c("q_819", "q_698", "q_262", "q_1803", "q_1742", "-q_712", "-q_690", "-q_241", "-q_1180", "-q_1114"), IPIP50EmoSt = c("-q_108", "-q_1099", "-q_1479", "-q_1989", "-q_497", "-q_974", "-q_986", "-q_995", "q_1677", "q_248"), IPIP50Consc = c("-q_1255", "-q_1397", "-q_1483", "-q_1696", "q_124", "q_1290", "q_1507", "q_76", "q_931", "q_962"), IPIP50Agree = c("q_844", "q_1792", "q_1763", "q_150", "q_1419", "q_1053", "-q_838", "-q_200", "-q_195", "-q_1163"), IPIP50Intel = c("q_128", "q_240", "q_1893", "q_1738", "q_1090", "q_1058", "q_1050", "-q_609", "-q_194", "-q_1088"), IPIP100Extra = c("-q_241", "q_254", "q_262", "-q_403", "-q_690", "q_698", "-q_712", "q_815", "q_819", "-q_901", "-q_1114", "-q_1180", "q_1205", "q_1410", "-q_1480", "q_1742", "q_1768", "q_1803", "-q_1913"), IPIP100EmoSt = c("-q_108", "q_177", "q_248", "-q_497", "-q_890", "-q_952", "-q_960", "-q_974", "-q_979", "-q_986", "-q_995", "-q_1020", "-q_1099", "-q_1479", "-q_1505", "q_1585", "q_1677", "q_1683", "-q_1775", "-q_1989"), IPIP100Consc = c("q_76", "q_124", "q_530", "q_619", "-q_626", "-q_904", "q_931", "q_962", "-q_1254", "-q_1255", "q_1290", "q_1333", "q_1374", "-q_1397", "q_1422", "-q_1452", "-q_1483", "q_1507", "-q_1696", "-q_1949"), IPIP100Agree = c("-q_140", "-q_146", "q_150", "-q_195", "-q_200", "q_217", "-q_838", "q_844", "q_1041", "q_1053", "q_1162", "-q_1163", "q_1206", "q_1364", "q_1385", "q_1419", "q_1705", "q_1763", "q_1792", "q_1832"), IPIP100Intel = c("q_128", "q_132", "-q_194", "q_240", "-q_316", "q_422", "q_492", "q_493", "-q_609", "q_1050", "q_1058", "-q_1083", "-q_1088", "q_1090", "q_1388", "q_1392", "q_1738", "-q_1861", "q_1893", "-q_1964"), IPIPNEO120E = c("q_1410", "q_819", "-q_313", "-q_1196", "q_1370", "q_1803", "-q_1531", "-q_314", "q_1768", "q_1872", "q_1769", "-q_1913", "q_71", "q_74", "q_595", "-q_1329", "q_1368", "q_1662", "q_746", "q_31", "-q_1567", "q_1043", "q_1371", "q_1350"), IPIPNEO120N = c("q_1989", "q_808", "q_66", "q_986", "q_952", "q_974", "q_1357", "-q_176", "q_1479", "q_578", "q_206", "-q_820", "q_901", "q_69", "q_1496", "-q_168", "q_1009", "-q_1590", "-q_736", "-q_56", "q_1505", "q_332", "q_867", "-q_1616"), IPIPNEO120C = c("q_520", "q_786", "q_1023", "q_1215", "q_1333", "-q_1483", "-q_1254", "-q_1255", "q_1190", "q_1812", "-q_405", "-q_404", "q_604", "q_1979", "-q_1553", "-q_602", "q_76", "q_491", "-q_1949", "-q_1087", "-q_1173", "-q_1424", "-q_1638", "-q_35"), IPIPNEO120A = c("q_1854", "q_377", "q_1855", "-q_594", "-q_1896", "-q_501", "-q_1765", "-q_1467", "q_90", "q_1385", "-q_146", "-q_1774", "-q_1359", "-q_2036", "-q_1163", "-q_957", "-q_364", "-q_1830", "-q_1042", "-q_401", "q_1764", "q_851", "-q_195", "-q_1858"), IPIPNEO120O = c("q_1058", "q_780", "q_1381", "q_1310", "q_348", "q_1648", "-q_612", "-q_607", "q_793", "q_844", "-q_1589", "-q_721", "q_1542", "-q_1540", "-q_566", "-q_82", "q_1388", "-q_322", "-q_1088", "-q_197", "q_1825", "q_394", "-q_1824", "-q_398")) keys.list = c( keys.list135, keys.list108, keys.list81, keys.list54, keys.list45, keys.list35, keys.list27, keys.list20, keys.listExternal) ScalesKey <- make.keys(subset(allSample, select = c(q_6:q_4300)), keys.list) ScalesScoredOverlapSPI <- scoreOverlap(ScalesKey, allSampleCor, impute="none") ScalesScoredOverlapSPIcor <- ScalesScoredOverlapSPI$cor rownames(ScalesScoredOverlapSPIcor)[1:5] <- c("SPI Extraversion","SPI Neuroticism","SPI Conscientiousness","SPI Agreeableness","SPI Openness") pdf(paste(filepathfig, "scalesCorValid.pdf", sep = ""), onefile = TRUE) cor.plot(ScalesScoredOverlapSPIcor[c(6:25,31:40,1:5),c(41:60)], numbers=TRUE, cex.axis=.7, main="", cex=.5) dev.off() @ \end{scriptsize} \clearpage \begin{scriptsize} <>= suppressPackageStartupMessages(library(xtable)) addtorow <- list(pos = list(0), command = NULL) command <- paste0("\\hline\n\\endhead\n", "\\hline\n", "\\multicolumn{4}{c}", "{\\textit{(continued on next page)}}\n", "\\endfoot\n", "\\endlastfoot\n") addtorow$command <- command print(xtable(itemContent[,c(1,2,5,4)], caption='SPI27 items by factor', label='tab:orgtable2', align = c("l", "l", "l", "c", "l")), size='scriptsize', caption.placement = "top", tabular.environment="longtable", floating=FALSE, add.to.row = addtorow) @ \end{scriptsize} \begin{scriptsize} <>= colnames(itemContent)[3] <- "SPI5" AgreeItems <- subset(itemContent, SPI5 == "Agreeableness") AgreeCoding <- c("+", "+", "+", "+", "+", "-", "-", "-", "+", "+", "+", "-", "+", "-") AgreeablenessItems <- data.frame(AgreeItems[,c(1,3)], AgreeCoding, AgreeItems[,c(2)]) colnames(AgreeablenessItems) <- c("Item Content", "SPI5 Factor", "SPI5 Coding", "SPI27 Factor") ConscItems <- subset(itemContent, SPI5 == "Conscientiousness") ConscCoding <- c("-", "+", "+", "-", "-", "+", "-", "-", "+", "+", "+", "+", "+", "+") ConscientiousnessItems <- data.frame(ConscItems[,c(1,3)], ConscCoding, ConscItems[,c(2)]) colnames(ConscientiousnessItems) <- c("Item Content", "SPI5 Factor", "SPI5 Coding", "SPI27 Factor") OpenItems <- subset(itemContent, SPI5 == "Openness") OpenCoding <- c("+", "+", "+", "+", "+", "+", "+", "+", "+", "+", "+", "+", "+", "-") OpennessItems <- data.frame(OpenItems[,c(1,3)], OpenCoding, OpenItems[,c(2)]) colnames(OpennessItems) <- c("Item Content", "SPI5 Factor", "SPI5 Coding", "SPI27 Factor") NeuroItems <- subset(itemContent, SPI5 == "Neuroticism") NeuroCoding <- c("-", "-", "-", "+", "+", "+", "+", "+", "+", "+", "-", "-", "+", "+") NeuroticismItems <- data.frame(NeuroItems[,c(1,3)], NeuroCoding, NeuroItems[,c(2)]) colnames(NeuroticismItems) <- c("Item Content", "SPI5 Factor", "SPI5 Coding", "SPI27 Factor") ExtraItems <- subset(itemContent, SPI5 == "Extraversion") ExtraCoding <- c("+", "-", "-", "+", "-", "+", "+", "-", "-", "+", "-", "+", "+", "+") ExtraversionItems <- data.frame(ExtraItems[,c(1,3)], ExtraCoding, ExtraItems[,c(2)]) colnames(ExtraversionItems) <- c("Item Content", "SPI5 Factor", "SPI5 Coding", "SPI27 Factor") itemContent5 <- rbind(AgreeablenessItems, ConscientiousnessItems, OpennessItems, NeuroticismItems, ExtraversionItems) @ \end{scriptsize} \clearpage \begin{scriptsize} <>= addtorow <- list(pos = list(0), command = NULL) command <- paste0("\\hline\n\\endhead\n", "\\hline\n", "\\multicolumn{4}{c}", "{\\textit{(continued on next page)}}\n", "\\endfoot\n", "\\endlastfoot\n") addtorow$command <- command print(xtable(itemContent5, caption='SPI5 items by factor', label='tab:orgtable3', align = c("l", "l", "l", "c", "l")), size='scriptsize', caption.placement = "top", tabular.environment="longtable", floating=FALSE, add.to.row = addtorow) @ \end{scriptsize} \clearpage \begin{figure} \ffigbox{\includegraphics[width=.95\textwidth]{figures/scalesCorValid.pdf}}{\caption{Correlations (corrected for item overlap) among the various SPI scales and extant IPIP-based sets of measures.}\label{fig:scalesCorValid}} \end{figure} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \clearpage \pagenumbering{arabic}% resets `page` counter to 1 \renewcommand*{\thepage}{A6-\arabic{page}} \renewcommand{\thefigure}{A6-\arabic{figure}} \setcounter{figure}{0} \renewcommand{\thetable}{A6-\arabic{table}} \setcounter{table}{0} \begin{center} \begin{Large} \textbf{Appendix F} \subsection*{SAPA Personality Inventory Forms} \addcontentsline{toc}{subsection}{F: SAPA Personality Inventory Forms} \label{sec:appendixF} \end{Large} {\bf David M. Condon}\footnote[$\star$]{This document is an appendix to \textit{The SAPA Personality Inventory}. Correspondence should be directed to \href{mailto:david-condon@northwestern.edu}{david-condon@northwestern.edu}, Department of Medical Social Sciences, Northwestern University, Chicago, Illinois. Please use the following citation information: Condon, D.M. (2017). The SAPA Personality Inventory: an empirically-derived, hierarchically-organized self-report personality assessment model.} \end{center} \vskip -0.5cm %%%%%%%%%%%%%%%%%% %Add Breaking Line \begin{center} \linethickness{0.5mm} \line(1,0){300} \end{center} %%%%%%%%%%%%%%%%%% Several forms have been developed for personality assessment using the 135 items identified as part of the SAPA Personality Inventory. For recommendations about personality assessment and comparisons of the SPI forms, see Appendix \hyperref[sec:appendixA]{A}. In order, this appendix includes the following six forms: \vspace{2mm} \vspace{2mm} \noindent \textbf{1. SPI-135-27\&5 version 1.1 - A 135-item measure for assessment of the 27-factor and 5-factor scales.} Everyone wants you to use this version --- your clients and your boss and your advisor and your collaborators and your mother. The best reason not to use all 135 of these items is because you intend to use the computer adaptive version of them. \vspace{2mm} \noindent \textbf{2. SPI-81-27\&5 version 1.1 - A 81-item measure for assessment of the 27-factor and 5-factor scales.} This is an excellent general purpose measure because it balances length with reasonable measurement at both the 5 factor and 27 factor levels. \vspace{2mm} \noindent \textbf{3. SPI-70-5 version 1.0 - A 70-item measure for assessment of the 5-factor scales.} The Big Five are important! If you are mainly interested in Big Five scores, use these but note that these items are not recommended for assessment of the 27 factor solution. \vspace{2mm} \noindent \textbf{4. SPI-45-5 version 1.0 - A 45-item measure for assessment of the 5-factor scales.} This measure is highly correlated with other Big Five measures of the same approximate length (yawn) but does not provide facet level scores. Are you sure you can't use the 81-item measure (or a CAT)? What about using the 20-item Big Five measure plus more in-depth measures of key constructs? \vspace{2mm} \noindent \textbf{5. SPI-20-5 version 1.0 - A 20-item measure for assessment of the 5-factor scales.} This 20-item measure only takes about 3 min and produces scores with reasonable psychometric properties for the Big Five. It's generally not a good idea to use a Big Five measure shorter than this one. \vspace{2mm} \noindent \textbf{6. SPI-27-27 version 1.1 - 27 ``signal'' items.} An appropriate use of this 27-item set would be to gauge the need for further follow-up or to conduct exploratory analyses. I do not recommend this item set as a substitute for more lengthy assessment of the 27 factor solution. These items can also be scored on the 5-factor level but the internal consistencies are poor. \vspace{2mm} \noindent \textbf{7. SPI-35-27\&5 version 1.0 - A 35-item measure that represents the shortest possible form for hierarchical assessment.} The 27 signal items have been supplemented by 8 additional items to allow for assessment at both the 27- and 5-factor levels. The 5-factor scales have reasonably good internal consistency --- in fact, it is tempting to argue that these scales are \textit{better} than those in the 20- or even 45-item measures (based on the unidimensionality values) but they are less valid because they measure very narrow interpretations of the 5 factors. \vspace{2mm} <>= # The code below is used to get the ICs for all of the SPI scales rm(list=setdiff(ls(), c("filepathdata", "filepathfig"))) BigKeysList <- list( SPI_20_5_Agree = c("q_90", "q_1763", "q_253", "-q_1896"), SPI_20_5_Consc = c("q_1744", "q_1979", "q_1290", "-q_1452"), SPI_20_5_Extra = c("q_1904", "q_4243", "-q_312", "-q_565"), SPI_20_5_Neuro = c("q_979", "q_4252", "q_1989", "q_1505"), SPI_20_5_Open = c("q_128", "q_2745", "q_2754", "q_1392"), SPI_27_27_5_Agree = c("q_253", "-q_4296", "q_1855"), SPI_27_27_5_Consc = c("q_1201", "q_530", "-q_904"), SPI_27_27_5_Extra = c("q_1904", "-q_565", "q_1243"), SPI_27_27_5_Neuro = c("q_979", "q_4252", "q_578"), SPI_27_27_5_Open = c("q_2745", "q_240", "q_1389"), SPI_27_27_5_Compassion = c("q_253"), SPI_27_27_5_Irritability = c("q_952"), SPI_27_27_5_Sociability = c("q_1904"), SPI_27_27_5_WellBeing = c("-q_578"), SPI_27_27_5_SensationSeeking = c("q_1367"), SPI_27_27_5_Anxiety = c("q_4252"), SPI_27_27_5_Honesty = c("-q_4296"), SPI_27_27_5_Industry = c("-q_904"), SPI_27_27_5_Intellect = c("q_240"), SPI_27_27_5_Creativity = c("q_2745"), SPI_27_27_5_Impulsivity = c("q_35"), SPI_27_27_5_AttentionSeeking = c("-q_565"), SPI_27_27_5_Order = c("q_1201"), SPI_27_27_5_Authoritarianism = c("q_1624"), SPI_27_27_5_Charisma = c("q_1045"), SPI_27_27_5_Trust = c("q_1855"), SPI_27_27_5_Humor = c("q_1243"), SPI_27_27_5_EmotionalExpressiveness = c("q_219"), SPI_27_27_5_ArtAppreciation = c("-q_610"), SPI_27_27_5_Introspection = c("q_1389"), SPI_27_27_5_Perfectionism = c("q_530"), SPI_27_27_5_SelfControl = c("q_56"), SPI_27_27_5_Conformity = c("q_152"), SPI_27_27_5_Adaptability = c("-q_566"), SPI_27_27_5_EasyGoingness = c("q_1329"), SPI_27_27_5_EmotionalStability = c("-q_979"), SPI_27_27_5_Conservatism = c("q_345"), SPI_35_27_5_Agree = c("q_253", "q_90", "q_1763"), SPI_35_27_5_Consc = c("-q_904", "q_1201", "q_1744", "q_1979"), SPI_35_27_5_Extra = c("q_1904", "-q_565", "q_1243", "-q_312"), SPI_35_27_5_Neuro = c("q_578", "q_4252", "q_979", "q_1989"), SPI_35_27_5_Open = c("q_240", "q_2745", "q_2754", "q_128"), SPI_35_27_5_Compassion = c("q_253"), SPI_35_27_5_Irritability = c("q_952"), SPI_35_27_5_Sociability = c("q_1904"), SPI_35_27_5_WellBeing = c("-q_578"), SPI_35_27_5_SensationSeeking = c("q_1367"), SPI_35_27_5_Anxiety = c("q_4252"), SPI_35_27_5_Honesty = c("-q_4296"), SPI_35_27_5_Industry = c("-q_904"), SPI_35_27_5_Intellect = c("q_240"), SPI_35_27_5_Creativity = c("q_2745"), SPI_35_27_5_Impulsivity = c("q_35"), SPI_35_27_5_AttentionSeeking = c("-q_565"), SPI_35_27_5_Order = c("q_1201"), SPI_35_27_5_Authoritarianism = c("q_1624"), SPI_35_27_5_Charisma = c("q_1045"), SPI_35_27_5_Trust = c("q_1855"), SPI_35_27_5_Humor = c("q_1243"), SPI_35_27_5_EmotionalExpressiveness = c("q_219"), SPI_35_27_5_ArtAppreciation = c("-q_610"), SPI_35_27_5_Introspection = c("q_1389"), SPI_35_27_5_Perfectionism = c("q_530"), SPI_35_27_5_SelfControl = c("q_56"), SPI_35_27_5_Conformity = c("q_152"), SPI_35_27_5_Adaptability = c("-q_566"), SPI_35_27_5_EasyGoingness = c("q_1329"), SPI_35_27_5_EmotionalStability = c("-q_979"), SPI_35_27_5_Conservatism = c("q_345"), SPI_45_5_Agree = c("q_90", "q_1763", "q_253", "-q_1896", "q_851", "q_1832", "-q_501", "q_377", "-q_871"), SPI_45_5_Consc = c("q_1290", "q_1744", "q_1979", "-q_1452", "q_1915", "q_1201", "q_530", "-q_904", "q_1867"), SPI_45_5_Extra = c("q_1904", "q_4243", "-q_312", "-q_565", "q_1416", "-q_1923", "-q_1027", "-q_684", "q_254"), SPI_45_5_Neuro = c("q_979", "q_4252", "q_1989", "q_1505", "q_4249", "q_808", "q_793", "-q_1840", "q_811"), SPI_45_5_Open = c("q_128", "q_2745", "q_2754", "q_1392", "q_1058", "q_240", "q_1738", "q_422", "q_1389"), SPI_54_27_5_Agree = c("q_90", "q_253", "q_1855", "-q_4296", "q_4289"), SPI_54_27_5_Consc = c("q_1744", "q_1915", "q_1201", "q_530", "-q_904", "q_1867", "-q_1254"), SPI_54_27_5_Extra = c("q_1904", "-q_312", "-q_565", "-q_1027", "q_254", "q_1243", "q_1244"), SPI_54_27_5_Neuro = c("q_979", "q_4252", "q_1989", "q_811", "q_578"), SPI_54_27_5_Open = c("q_128", "q_2745", "q_240", "q_1738", "q_1389"), SPI_54_27_5_Compassion = c("q_253", "q_90"), SPI_54_27_5_Irritability = c("q_952", "q_1357"), SPI_54_27_5_Sociability = c("q_1904", "-q_312"), SPI_54_27_5_WellBeing = c("-q_578", "-q_811"), SPI_54_27_5_SensationSeeking = c("q_1367", "q_1664"), SPI_54_27_5_Anxiety = c("q_4252", "q_1989"), SPI_54_27_5_Honesty = c("-q_4296", "q_1812"), SPI_54_27_5_Industry = c("-q_904", "q_1744"), SPI_54_27_5_Intellect = c("q_240", "-q_1253"), SPI_54_27_5_Creativity = c("q_2745", "q_128"), SPI_54_27_5_Impulsivity = c("q_35", "q_1173"), SPI_54_27_5_AttentionSeeking = c("-q_565", "-q_1027"), SPI_54_27_5_Order = c("q_1201", "-q_1254"), SPI_54_27_5_Authoritarianism = c("q_1624", "q_1867"), SPI_54_27_5_Charisma = c("q_1045", "q_254"), SPI_54_27_5_Trust = c("q_1855", "q_4289"), SPI_54_27_5_Humor = c("q_1243", "q_1244"), SPI_54_27_5_EmotionalExpressiveness = c("q_219", "-q_1081"), SPI_54_27_5_ArtAppreciation = c("-q_610", "q_348"), SPI_54_27_5_Introspection = c("q_1389", "q_1738"), SPI_54_27_5_Perfectionism = c("q_530", "q_1915"), SPI_54_27_5_SelfControl = c("q_56", "q_736"), SPI_54_27_5_Conformity = c("q_152", "q_1300"), SPI_54_27_5_Adaptability = c("-q_566", "-q_689"), SPI_54_27_5_EasyGoingness = c("q_1329", "q_1281"), SPI_54_27_5_EmotionalStability = c("-q_979", "q_174"), SPI_54_27_5_Conservatism = c("q_345", "-q_660"), SPI_70_5_Agree = c("q_90", "q_1763", "q_253", "-q_1896", "q_851", "q_1832", "-q_501", "q_377", "-q_871", "q_1855", "-q_4296", "-q_142", "q_379", "q_4289"), SPI_70_5_Consc = c("q_1290", "q_1744", "q_1979", "-q_1452", "q_1915", "q_1201", "q_530", "-q_904", "q_1867", "q_1694", "q_369", "-q_1444", "-q_1483", "-q_1254"), SPI_70_5_Extra = c("q_1904", "q_4243", "-q_312", "-q_565", "q_1416", "-q_1923", "-q_1027", "-q_684", "q_254", "-q_1296", "-q_901", "q_1243", "q_803", "q_1244"), SPI_70_5_Neuro = c("q_979", "q_4252", "q_1989", "q_1505", "q_4249", "q_808", "q_793", "-q_1840", "q_811", "-q_1585", "q_578", "-q_176", "-q_797", "-q_1683"), SPI_70_5_Open = c("q_128", "q_2745", "q_2754", "q_1392", "q_1058", "q_240", "q_1738", "q_422", "q_1389", "q_1310", "q_1880", "-q_747", "q_1609", "q_1834"), SPI_81_27_5_Agree = c("q_90", "q_1763", "q_253", "-q_501", "q_377", "q_1855", "-q_4296", "q_4289"), SPI_81_27_5_Consc = c("q_1744", "q_1915", "q_1201", "q_530", "-q_904", "q_1867", "-q_1444", "-q_1483", "-q_1254"), SPI_81_27_5_Extra = c("q_1904", "-q_312", "-q_565", "q_1416", "-q_1923", "-q_1027", "q_254", "q_1243", "q_803", "q_1244"), SPI_81_27_5_Neuro = c("q_979", "q_4252", "q_1989", "q_4249", "q_793", "q_811", "q_578", "-q_1683"), SPI_81_27_5_Open = c("q_128", "q_2745", "q_2754", "q_240", "q_1738", "q_1389", "q_1609"), SPI_81_27_5_Compassion = c("q_253", "q_90", "q_1763"), SPI_81_27_5_Irritability = c("q_952", "q_1357", "-q_1683"), SPI_81_27_5_Sociability = c("q_1904", "-q_312", "-q_1923"), SPI_81_27_5_WellBeing = c("-q_578", "-q_811", "q_2765"), SPI_81_27_5_SensationSeeking = c("q_1367", "q_1664", "q_1781"), SPI_81_27_5_Anxiety = c("q_4252", "q_1989", "q_4249"), SPI_81_27_5_Honesty = c("-q_4296", "q_1812", "-q_501"), SPI_81_27_5_Industry = c("-q_904", "q_1744", "-q_1444"), SPI_81_27_5_Intellect = c("q_240", "-q_1253", "q_493"), SPI_81_27_5_Creativity = c("q_2745", "q_128", "q_2754"), SPI_81_27_5_Impulsivity = c("q_35", "q_1173", "q_1424"), SPI_81_27_5_AttentionSeeking = c("-q_565", "-q_1027", "q_1416"), SPI_81_27_5_Order = c("q_1201", "-q_1254", "-q_1483"), SPI_81_27_5_Authoritarianism = c("q_1624", "q_1867", "-q_1609"), SPI_81_27_5_Charisma = c("q_1045", "q_254", "-q_1242"), SPI_81_27_5_Trust = c("q_1855", "q_4289", "q_377"), SPI_81_27_5_Humor = c("q_1243", "q_1244", "q_1248"), SPI_81_27_5_EmotionalExpressiveness = c("q_219", "-q_1081", "q_803"), SPI_81_27_5_ArtAppreciation = c("-q_610", "q_348", "-q_607"), SPI_81_27_5_Introspection = c("q_1389", "q_1738", "q_755"), SPI_81_27_5_Perfectionism = c("q_530", "q_1915", "q_571"), SPI_81_27_5_SelfControl = c("q_56", "q_736", "q_1590"), SPI_81_27_5_Conformity = c("q_152", "q_1300", "q_1653"), SPI_81_27_5_Adaptability = c("-q_566", "-q_689", "q_39"), SPI_81_27_5_EasyGoingness = c("q_1329", "q_1281", "q_1052"), SPI_81_27_5_EmotionalStability = c("-q_979", "q_174", "-q_793"), SPI_81_27_5_Conservatism = c("q_345", "-q_660", "q_1824"), SPI_108_27_5_Agree = c("q_90", "q_1763", "q_253", "q_851", "-q_501", "q_377", "q_1855", "-q_4296", "q_379", "q_4289"), SPI_108_27_5_Consc = c("q_1290", "q_1744", "-q_1452", "q_1915", "q_1201", "q_530", "-q_904", "q_1867", "q_1694", "q_369", "-q_1444", "-q_1483", "-q_1254"), SPI_108_27_5_Extra = c("q_1904", "q_4243", "-q_312", "-q_565", "q_1416", "-q_1923", "-q_1027", "q_254", "q_1296", "-q_901", "q_1243", "q_803", "q_1244"), SPI_108_27_5_Neuro = c("q_979", "q_4252", "q_1989", "q_1505", "q_4249", "q_793", "q_811", "-q_1585", "q_578", "-q_797", "-q_1683"), SPI_108_27_5_Open = c("q_128", "q_2745", "q_2754", "q_1392", "q_240", "q_1738", "q_422", "q_1389", "q_1880", "-q_747", "q_1609"), SPI_108_27_5_Compassion = c("q_253", "q_90", "q_1763", "q_851"), SPI_108_27_5_Irritability = c("q_952", "q_1357", "-q_1683", "-q_1585"), SPI_108_27_5_Sociability = c("q_1904", "-q_312", "-q_1923", "q_4243"), SPI_108_27_5_WellBeing = c("-q_578", "-q_811", "q_2765", "q_820"), SPI_108_27_5_SensationSeeking = c("q_1367", "q_1664", "q_1781", "q_598"), SPI_108_27_5_Anxiety = c("q_4252", "q_1989", "q_4249", "q_1505"), SPI_108_27_5_Honesty = c("-q_4296", "q_1812", "-q_501", "q_2853"), SPI_108_27_5_Industry = c("-q_904", "q_1744", "-q_1444", "-q_1452"), SPI_108_27_5_Intellect = c("q_240", "-q_1253", "q_493", "q_422"), SPI_108_27_5_Creativity = c("q_2745", "q_128", "q_2754", "q_1392"), SPI_108_27_5_Impulsivity = c("q_35", "q_1173", "q_1424", "q_4276"), SPI_108_27_5_AttentionSeeking = c("-q_565", "-q_1027", "q_1416", "q_1296"), SPI_108_27_5_Order = c("q_1201", "-q_1254", "-q_1483", "q_1290"), SPI_108_27_5_Authoritarianism = c("q_1624", "q_1867", "-q_1609", "q_369"), SPI_108_27_5_Charisma = c("q_1045", "q_254", "-q_1242", "-q_901"), SPI_108_27_5_Trust = c("q_1855", "q_4289", "q_377", "q_379"), SPI_108_27_5_Humor = c("q_1243", "q_1244", "q_1248", "q_296"), SPI_108_27_5_EmotionalExpressiveness = c("q_219", "-q_1081", "q_803", "-q_1635"), SPI_108_27_5_ArtAppreciation = c("-q_610", "q_348", "-q_607", "-q_612"), SPI_108_27_5_Introspection = c("q_1389", "q_1738", "q_755", "q_1880"), SPI_108_27_5_Perfectionism = c("q_530", "q_1915", "q_571", "q_1694"), SPI_108_27_5_SelfControl = c("q_56", "q_736", "q_1590", "q_1462"), SPI_108_27_5_Conformity = c("q_152", "q_1300", "q_1653", "q_747"), SPI_108_27_5_Adaptability = c("-q_566", "-q_689", "q_39", "q_1542"), SPI_108_27_5_EasyGoingness = c("q_1329", "q_1281", "q_1052", "q_1024"), SPI_108_27_5_EmotionalStability = c("-q_979", "q_174", "-q_793", "q_797"), SPI_108_27_5_Conservatism = c("q_345", "-q_660", "q_1824", "-q_1825"), SPI_135_27_5_Agree = c("q_90", "q_1763", "q_253", "-q_1896", "q_851", "q_1832", "-q_501", "q_377", "-q_871", "q_1855", "-q_4296", "-q_142", "q_379", "q_4289"), SPI_135_27_5_Consc = c("q_1290", "q_1744", "q_1979", "-q_1452", "q_1915", "q_1201", "q_530", "-q_904", "q_1867", "q_1694", "q_369", "-q_1444", "-q_1483", "-q_1254"), SPI_135_27_5_Extra = c("q_1904", "q_4243", "-q_312", "-q_565", "q_1416", "-q_1923", "-q_1027", "-q_684", "q_254", "q_1296", "-q_901", "q_1243", "q_803", "q_1244" ), SPI_135_27_5_Neuro = c("q_979", "q_4252", "q_1989", "q_1505", "q_4249", "q_808", "q_793", "-q_1840", "q_811", "-q_1585", "q_578", "-q_176", "-q_797", "-q_1683"), SPI_135_27_5_Open = c("q_128", "q_2745", "q_2754", "q_1392", "q_1058", "q_240", "q_1738", "q_422", "q_1389", "q_1310", "q_1880", "-q_747", "q_1609", "q_1834"), SPI_135_27_5_Compassion = c("q_253", "q_90", "q_1763", "q_851", "q_1832"), SPI_135_27_5_Irritability = c("q_952", "q_1357", "-q_1683", "-q_1585", "-q_176"), SPI_135_27_5_Sociability = c("q_1904", "-q_312", "-q_1923", "q_4243", "-q_684"), SPI_135_27_5_WellBeing = c("-q_578", "-q_811", "q_2765", "q_820", "q_1371"), SPI_135_27_5_SensationSeeking = c("q_1367", "q_1664", "q_1781", "q_598", "q_1662"), SPI_135_27_5_Anxiety = c("q_4252", "q_1989", "q_4249", "q_1505", "q_808"), SPI_135_27_5_Honesty = c("-q_4296", "q_1812", "-q_501", "q_2853", "-q_1896"), SPI_135_27_5_Industry = c("-q_904", "q_1744", "-q_1444", "-q_1452", "q_1979"), SPI_135_27_5_Intellect = c("q_240", "-q_1253", "q_493", "q_422", "q_1834"), SPI_135_27_5_Creativity = c("q_2745", "q_128", "q_2754", "q_1392", "q_1058"), SPI_135_27_5_Impulsivity = c("q_35", "q_1173", "q_1424", "q_4276", "-q_4223"), SPI_135_27_5_AttentionSeeking = c("-q_565", "-q_1027", "q_1416", "q_1296", "q_1555"), SPI_135_27_5_Order = c("q_1201", "-q_1254", "-q_1483", "q_1290", "-q_169"), SPI_135_27_5_Authoritarianism = c("q_1624", "q_1867", "-q_1609", "q_369", "q_398"), SPI_135_27_5_Charisma = c("q_1045", "q_254", "-q_1242", "-q_901", "q_131"), SPI_135_27_5_Trust = c("q_1855", "q_4289", "q_377", "q_379", "-q_871"), SPI_135_27_5_Humor = c("q_1243", "q_1244", "q_1248", "q_296", "-q_1685"), SPI_135_27_5_EmotionalExpressiveness = c("q_219", "-q_1081", "q_803", "-q_1635", "q_1706"), SPI_135_27_5_ArtAppreciation = c("-q_610", "q_348", "-q_607", "-q_612", "q_1132"), SPI_135_27_5_Introspection = c("q_1389", "q_1738", "q_755", "q_1880", "q_1310"), SPI_135_27_5_Perfectionism = c("q_530", "q_1915", "q_571", "q_1694", "q_142"), SPI_135_27_5_SelfControl = c("q_56", "q_736", "q_1590", "q_1462", "q_1461"), SPI_135_27_5_Conformity = c("q_152", "q_1300", "q_1653", "q_747", "q_2005"), SPI_135_27_5_Adaptability = c("-q_566", "-q_689", "q_39", "q_1542", "q_1303"), SPI_135_27_5_EasyGoingness = c("q_1329", "q_1281", "q_1052", "q_1024", "q_1280"), SPI_135_27_5_EmotionalStability = c("-q_979", "q_174", "-q_793", "q_797", "q_1840"), SPI_135_27_5_Conservatism = c("q_345", "-q_660", "q_1824", "-q_1825", "q_1328")) SPIlists <- BigKeysList[c(2:11, 39:176)] load(paste(filepathdata, "descriptivesData.Rdata", sep = "")) bothSample <- rbind(expSample, repSample) allSample <- rbind(bothSample, confSample) allSampleCor <- cor(subset(allSample, select = c(DataItems)), use="pairwise") allSampleCor <- cor.smooth(allSampleCor) # The code below will not work because you have negatively # coded items in the SPIlists object. This is intentional. # If you want to run it, you have to make another list object that # drops all of the "-"s. # I have retained this code so others can use it if they want. # spiICs <- data.frame() # for (i in 1:length(SPIlists)) { # alpha1 <- suppressWarnings(alpha(allSample[,SPIlists[[i]]], check.keys=TRUE)) # alpha <- alpha1$total$raw_alpha # unidim <- alpha1$Unidim$Unidim # rows <- c(alpha, unidim) # rows <- round(rows, 2) # spiICs <- rbind(spiICs, rows) # } # rownames(spiICs) <- names(SPIlists) # colnames(spiICs) <- c("Alpha", "Unidimensionality") # spiICs @ \vspace{2mm} \vspace{2mm} \noindent \textbf{The most updated versions of these forms are available in electronic formats (useful for copying and pasting into other applications) at \href{https://www.sapa-project.org/research/SPI/}{sapa-project.org/research/SPI/}} \clearpage \includepdf[pages={1-}, scale=1, link, linkname=SPI135275]{figures/appendices/SPI_135_27_&_5_v1_1.pdf} \label{sec:SPI135275} \clearpage \includepdf[pages={1-}, scale=1, link, linkname=SPI81275]{figures/appendices/SPI_81_27_&_5_v1_1.pdf} \label{sec:SPI81275} \clearpage \includepdf[pages={1-}, scale=1, link, linkname=SPI705]{figures/appendices/SPI_70_5_v1_1.pdf} \label{sec:SPI705} \clearpage \includepdf[pages={1-}, scale=1, link, linkname=SPI455]{figures/appendices/SPI_45_5_v1_1.pdf} \label{sec:SPI455} \clearpage \includepdf[pages={1-}, scale=1, link, linkname=SPI205]{figures/appendices/SPI_20_5_v1_1.pdf} \label{sec:SPI205} \clearpage \includepdf[pages={1-}, scale=1, link, linkname=SPI27275]{figures/appendices/SPI_27_27_&_5_v1_1.pdf} \label{sec:SPI27275} \clearpage \includepdf[pages={1-}, scale=1, link, linkname=SPI35275]{figures/appendices/SPI_35_27_&_5_v1_1.pdf} \label{sec:SPI35275} \clearpage \end{document}