Megan Welsh

Megan Welsh joined the School in July 2014 as an assistant professor in educational assessment and measurement. Since 2008, she was an assistant professor at the University of Connecticut. Her primary areas of research include test validity analysis, the use of assessment as an educational reform lever, grading, and evaluation of educational programs.

Education

• 2009: Ph.D. in Educational Psychology, University of Arizona, Tucson, Arizona
• 1998: M.P.P. in Public Policy, University of California, Berkeley, Berkeley, California
• 1993: B.A., cum laude, in American Studies, Wellesley College, Wellesley, Massachusetts

Experience

2014-Present: Assistant Professor, University of California, Davis. Davis, CA.

2008-2014: Assistant Professor, University of Connecticut, Storrs, CT.

2008: Research Associate, Brown University, Providence, RI

2006-07: Educational Research Specialist, San Diego Unified School District, San Diego, CA.

2003-04: Program Assessment and Evaluation Analyst, Amphitheater Public Schools, Tucson AZ.

1998-03: Research Associate, Policy Studies Associates, Washington, DC.

1993-1997: Third/Fourth-Grade Bilingual Cantonese Teacher. Bella Vista Elementary School Oakland Unified School District, Oakland, California

Publications

Miller, F. G., Cohen, D., Chafouleas, S. M., Riley-Tillman, T. C., Welsh, M. E., & Fabiano, G. A. (2014). A comparison of measures to screen for social, emotional, and behavioral risk. School Psychology Quarterly. PMID: 25111469

Welsh, M. E., Eastwood, M. & D’Agostino, J. V. (2014). Conceptualizing teaching to the test under standards-based reform. Applied Measurement in Education, 27(2), 98-114.

Briesch, A. M., Swaminathan, H., Welsh, M. E., & Chafouleas, S. M. (2014). A guide to generalizability theory: Understanding current practice and facilitating use in assessment. Journal of School Psychology, 52(1), 13-35.

Kilgus, S. P., Riley-Tillman, T. C., Chafouleas, S. M., Christ, T. J., & Welsh, M. E. (2014). Direct Behavior Rating as a school-based behavior screener: Replication across sites. Journal of School Psychology, 52(1), 63-82.

Welsh, M. E., D’Agostino, J. V. &. Kaniskan, R. B. (2013). Standards-based progress reports and standards-based assessment score convergence. Educational Measurement: Issues and Practice, 32(2), 26-36.

Chafouleas, S. M., Kilgus, S. P., Jaffrey, R., Riley-Tilman, T. C. & Welsh, M. E. (2013). Direct behavior rating as a school-based behavior screener for elementary and middle grades. Journal of School Psychology, 51(3), 367-85.

McCoach, D. B., Rambo, K., & Welsh, M. E. (2013). Assessing the growth of gifted students. Gifted Child Quarterly, 57(1), 56-67.

Kilgus, S. P., Chafouleas, S. M., Riley-Tilman, T. C., & Welsh, M. E. (2012). Direct Behavior Rating scales as screeners: A preliminary investigation of diagnostic accuracy in elementary school. School Psychology Quarterly, 27, 41-50.

Welsh, M.E. (2011). Measuring teacher effectiveness in gifted education: Some challenges and suggestions. Journal of Advanced Academics, 23(1), 1-21.

D’Agostino,J. V., Karpinski, A.  & Welsh, M. E.  (2011). A method to examine content domain structures. International Journal of Testing, 11(4), 295-307.

D’Agostino, J. V., Welsh, M. E., Cimetta, A. D., Falco, L. D., Hester, W. S., Smith, S. & Powers, S. J. (2008). Comparing the rating and matching item-objective alignment methods. Applied Measurement in Education, 21(1), 1-21.

D’Agostino, J. V., Schwartz, K. L., Cimetta, A. D., & Welsh, M. E. (2007). Using a Partitioned Treatment Design to Examine Project WET Impact. Journal of Environmental Education, 38(4), 43-50.

D’Agostino, J. V., Welsh, M. E. & Corson, N. M. (2007). Instructional validity of a state’s standards-based assessment. Educational Assessment, 12(1), 1-22.

Book Chapters

Howard, E., Welsh, M. E., Levine, T. & Moss, D. M. (2014). How can we create forms of assessment and data collection to support accreditation, research, and program improvement? In T. Levine, E. Howard, & D. M. Moss (Eds.) Preparing Classroom Teachers to Succeed with Second Language Learners: Lessons from a Faculty Learning Community. New York: Routledge.

McCoach, D. B., Madura, J., Rambo, K., O’Connell, A. A., & Welsh, M. E. (2013).  Longitudinal Data Analysis.  In T. Teo (Ed.) Handbook of Quantitative Methods for Educational Research. Rotterdam, Netherlands: Sense Publishers.

McCoach, D. B., Rambo, K., & Welsh, M. E. (2011). Issues in the analysis of change. In C. Secolsky & D. B. Denision (Eds.) Handbook on measurement, assessment, and evaluation in higher education. New York, NY: Routledge.

Welsh, M. E. & D’Agostino, J. V. (2008). Fostering consistency between standards-based grades and large scale assessment results. In T.R. Guskey (Ed.), Problems and solutions in standards-based grading.  Thousand Oaks, CA: Corwin.                           

Technical Reports

D’Agostino, J.V., Schwartz, K.L., Cimetta, A.D., & Welsh, M.E. (2006). Arizona Project WET Evaluation: Examining Impact and Developing Water Education Assessments.  Tucson, AZ: University of Arizona.

D’Agostino, J. V., Welsh, M.E., & Cimetta, A.D. (2005). Alignment analysis of Arizona academic standards and assessments.  Tucson, AZ: University of Arizona.

Welsh, M.E., Russell, C. Williams, I., Reisner, E. & White, R. (2002). Promoting learning and school attendance through after-school programs: student-level changes in educational performance across TASC’s first three years. Washington, DC: Policy Studies Associates.

Reisner, E., Russell, C., Welsh, M.E., Birmingham, J. & White, R. (2002). Supporting quality and scale in after-school services to urban youth: Evaluation of program implementation and student engagement in the TASC after-school program’s third year. Washington, DC: Policy Studies Associates.

Reisner, E., White, R., Birmingham, J. & Welsh, M.E. (2001). Building quality and supporting expansion of after-school projects: Evaluation results from the TASC after-school program’s second year. Washington, DC: Policy Studies Associates.

White, R., Reisner, E., Welsh, M.E., & Russell, C. (2001). Patterns of student-level change linked to TASC participation, based on TASC projects in Year 2. Washington, DC: Policy Studies Associates.

Anderson, L. & Welsh, M.E. (2000). Making progress: An update on state implementation of federal education laws enacted in 1994. Washington, DC: U.S. Department of Education.

Reisner, E., Fiester, L., White, R., Welsh, M.E., & Birmingham, J. (2000). Preliminary evaluation results from the TASC after-school program’s second year. Washington, DC: Policy Studies Associates.

Turnbull, B.J., Welsh, M.E., Heid, C., Davis, W, & Ratnofsky, A. (1999). The Longitudinal Evaluation of School Change and Performance (LESCP) in Title I schools: Interim report to Congress. Washington, DC: U.S. Department of Education.

Conference Presentations

Flake, J. K., Strauts, E., McCoach, D. B., Rogers, H. E., Welsh, M. E. (May, 2014) An investigation of the alignment method for detecting measurement non-invariance across many groups with dichotomous indicators. Paper presentation at the 2014 Modern Modeling Methods Conference. Storrs, CT.

Flake, J. K., Barron, K. E., Hulleman, C. H., McCoach, D. B., Welsh, M. E. (April, 2014). Initial validity evidence for a new scale to measure cost. Poster presentation at the 2014 Annual Meeting of the American Educational Research Association.  Philadelphia, PA.

Welsh, M. E., Kooken, J. W, Miller, F. G.; Rosen, R., Chafouleas, S. M., Fabiano, G. A., Riley-Tillman, T. C. (2014, April). Examining bias on a single item scale with think alouds. Paper presentation at the Annual Meeting of the National Council on Measurement in Education, Philadelphia, PA.

Yu, H., Welsh, M. E., McCoach, D. B., & D’Agostino, J. V. (2014, April). Relationship between teacher judgments and student achievement: A multilevel Bayesian structural equation modeling study. Paper presentation at the Annual Meeting of the National Council on       Measurement in Education, Philadelphia, PA.

Miller, F. G., Johnson, A. H., Rosen, R., Chafouleas, S. M., Welsh, M. E., Riley-Tillman, T. C., & Fabiano, G. A. (2014, February). Construct validation of DBR-SIS: A multi-trait multi-method analysis. Paper presentation at the National Association of School Psychologists Annual Convention, Washington, DC.

Rosen, R., Johnson, A. H., Chafouleas, S. M., Welsh, M. E., Riley-Tillman, T. C., & Fabiano, G. A., Miller, F. G. (2014, February). Exploring Direct Behavior Ratings in a middle school population. Paper presentation at the National Association of School Psychologists Annual Convention, Washington, DC.

Welsh, M.E. & Flake, J. K.  (2013, Nov.) The teacher effect: Conceptualizing sensitivity as non-invariance. The First International Conference on Instructional Sensitivity. Lawrence, KS.

Kooken, J. W., Welsh, M. E. & McCoach, D. B. (2013, October). Bi-Factor and Higher-Order Factor Analysis of the Mathematical Resilience Scale. Paper presentation at the Annual Meeting of the Northeastern Educational Research Association, Rocky Hill, CT.

Johnson, A. H., Miller, F. G., Chafouleas, S. M., Riley-Tillman, T. C., Fabiano, G. A., & Welsh, M. E. (2013, August). Using composite scores to determine behavioral risk with Direct Behavior Rating. Poster presentation at the American Psychological Association Annual Convention, Honolulu, HI.

Miller, F. G., Welsh, M. E., Johnson, A. H., Chafouleas, S. M., Riley-Tillman, T. C., & Fabiano, G. A. (2013, August). An evaluation of universal screening methods to identify behavioral risk. Poster presentation at the American Psychological Association Annual Convention, Honolulu, HI.

Welsh, M. E., Chafouleas, S. M., Fabiano, G. A., Riley-Tillman, T. C., & Miller, F. G. (2013, August). Evaluating rater bias with only one rater per target. Poster presentation at the American Psychological Association Annual Convention, Honolulu, HI.

Kooken, J. W., Welsh, M. E., Miller, F. G., Riley-Tillman, T. C., & Chafouleas, S. M., (2013, August). The effect of test order when administering multiple rating scales to a single rater. Poster presentation at the American Psychological Association Annual Convention, Honolulu, HI.

Kooken, J. W., Miller, F. G., Welsh, M. E., Fabiano, G. A.,  Riley-Tillman, T. C., & Chafouleas, S. M. (2013, August). Classroom behavior over time: Do student trajectories vary? Poster presentation at the American Psychological Association Annual Convention, Honolulu, HI.

Welsh, M. E., Miller, F. G., Kooken, J. W., & Chafouleas, S. M. (2013, April). The kindergarten transition: Behavioral trajectories and school adjustment in the first year of school. Paper presentation at the annual meeting of the American Educational Research Association, San Francisco, CA.

Welsh, M. E., Yu, H., & Kooken, J. W. (2013, April). Multiple measures of student performance: Generalizability of a standards-based report card-state assessment composite. Poster presentation at the annual meeting of the American Educational Research Association, San Francisco, CA.

Kooken, J. W., Welsh, M. E., Mccoach, D. B., Johnston-Wilder, S. & Lee, C. (2013, April). Measuring Mathematical Resilience: An application of the construct of resilience to the study of mathematics. Paper presentation at the annual meeting of the American Educational Research Association, San Francisco, CA.

Miller, F.G., Johnson, A.H., Welsh, M.E., Chafouleas, S.M., Riley-Tillman, T.C., & Fabiano, G.A. (2013, February). Comparing biannual and triannual behavioral screenings using DBR-SIS. Poster presentation at the National Association of School Psychologists Annual Convention, Seattle, WA.

Johnson, A. H., Miller, F., Welsh, M. E., Chafouleas, S. M., Riley-Tillman, T. C., & Fabiano, G. A. (2013, February). Using ROC curves to determine risk with Direct Behavior Rating. Poster presentation at the National Association of School Psychologists Annual Convention, Seattle, WA.

Fabiano, G.A., Norton, R L., Welsh, M.E., Miller, F.G., Parham, B.R., Chafouleas, S.M., & Riley-Tillman, T.C. (2012, Nov.). Effective Screening Approaches for Social/Behavioral Challenges: Seasonal Influences. Poster presentation at the 46th Annual Convention of the Association for the Advancement of Behavioral and Cognitive Therapies, National Harbor, MD.

Miller, F.G., Neugebauer, S.R., Chafouleas, S.M., Briesch, A.M., Welsh, M.E., Riley-Tillman, T.C., & Fabiano, G.A. (2012, August). Teacher perceptions of behavior screening assessments. Poster presentation at the American Psychological Association Annual Convention, Orlando, FL.

Kilgus, S. P., Chafouleas, S.M., Jaffery, R., Riley-Tillman, T.C., & Welsh, M.E. (2012, Feb). Direct Behavior Rating as a screener of social behavior risk. Paper presentation at the National Association of School Psychologists Annual Convention, Philadelphia, PA

Johnson, A. H., Kilgus, S. P., Chafouleas, S. M., Riley-Tillman, T. C., Christ, T. J., Welsh, M.E., Brooks, S., & Amon, J. (2012, Feb). Direct Behavior Rating as screener of student behavioral risk. Poster presentation at the National Association of School Psychologists Annual Convention, Philadelphia, PA

Welsh, M.E. (2011, Oct.). Efforts to teach in a way that tests detect: Pointless or profitable? Invited presentation at the annual meeting of the Northeast Educational Research Association. Rocky Hill, CT.

Coleman, A., Welsh, M.E. & Farris, A. (2011, Oct.). Evaluating the role of teachers beyond gatekeeping at school-based health interventions. Poster presented at the annual meeting of the Public Health Association. Southington, CT.

Welsh, M.E. & Sen, R. (2011, Apr.). Impact of panelists’ characteristics on setting cutoff scores during the first round of a standard setting study. Paper presented at the annual meeting of the American Educational Research Association. New Orleans, LA.

Welsh, M.E. & Eastwood, M. (2011, Apr.). Conceptualizing teaching to the test under standards-based reform. Paper presented at the annual meeting of the American Educational Research Association. New Orleans, LA.

Welsh, M.E. (2010, Oct.). An exploration of the instructional sensitivity of items on one state test. Paper presented at the annual meeting of the Northeastern Educational Research Association. Rocky Hill, CT.

Sen, R & Welsh, M.E. (2009, Oct.). Impact of panelists’ characteristics on setting cutoff scores during the first round of a standard setting study. Paper presented at the annual meeting of the Northeastern Educational Research Association. Rocky Hill, CT.

Eastwood, M. & Welsh, M.E. (2009, Oct.). Do states inadvertently encourage “teaching to the test”? Paper presented at the annual meeting of the Northeastern Educational Research Association. Rocky Hill, CT.

Welsh, M.E., Wong, K.K., Freedman, K. & Cancell, E.R. (2009, Apr.). Using embedded   researchers to support reform: One participatory evaluation model. Paper presented at the annual meeting of the American Educational Research Association. San Diego, CA.

D’Agostino, J.V., & Welsh. M.E. (2007, Apr.). Standards-based progress reports and standards-based assessment score convergence. In T.R. Guskey (Chair), The challenges of standards-based grading and reporting. Symposium conducted at the meeting of the American Educational Research Association, Chicago, IL.

Welsh, M.E. & D’Agostino, J.V. (2006, Apr.). Examining instructional validity: An interview-based approach. Paper presented at the annual meeting of the American Educational Research Association. San Francisco, CA.

D’Agostino, J.V. & Welsh, M.E. (2004, Oct.). Alignment between standards reports and AIMS scores in Marana Public Schools. Paper presented at the annual meeting of the Arizona Educational Research Organization. Phoenix, AZ.

Welsh, M.E. (2003, Oct.). Attainability and accountability: Pay for performance under Proposition 301.  Paper presented at the annual meeting of the Arizona Educational Research Organization. Phoenix, AZ.

Welsh, M.E. & White, R. (2002, Nov.). Evaluation of the TASC after-school program: outcomes associated with TASC participation. Paper presented at the annual meeting of the American Evaluation Association. Washington, DC.

Turnbull, B.J., Welsh, M.E., Ratnofsky, A. & Troppe, P. (2002, Apr.). Classroom practices associated with improved student achievement on the closed-ended portion of the SAT-9.  Paper presented at the annual meeting of the American Educational Research Association. New Orleans, LA.

Ratnofsky, A., Welsh, M.E. & Troppe, P. (2001, June). The Longitudinal Evaluation of School Change and Performance (LESCP). Paper presented at the Council for Chief State School Officers National Conference on Large-Scale Assessment. Houston, TX.             

Invited Talks

Welsh, M.E. (2012, Aug.) Standards-based grading. Invited talk at the first annual Connecticut Assessment Forum. Rocky Hill, CT.

Welsh, M.E. (2011, Jan.) Developing effective multiple choice tests. Invited talk at the University of Connecticut’s Institute for Teaching and Learning’s Teaching Institute. Storrs, CT.

Welsh, M.E. (2010, Aug.) Formative assessment. Invited talk at the first annual Connecticut Assessment Forum. Rocky Hill, CT.

Welsh, M.E. (2010, Mar.) Item Analysis. Invited talk at the University of Connecticut’s Institute for Teaching and Learning’s Brown Bag Seminars. Storrs, CT.

Welsh, M.E. (2010, Feb.) Developing effective multiple choice tests. Invited talk at the University of Connecticut’s Institute for Teaching and Learning’s Brown Bag Seminars. Storrs, CT.

Wood, C. & Welsh, M.E. (2004, June). Designing and evaluating school counseling interventions. Invited presentation at the 2004 Arizona School Counselor’s Academy. Tucson, AZ.

Welsh, M.E. (2004, June). Collecting data: When counselor time and expertise are realistic factors. Invited presentation at the 2004 Arizona School Counselor’s Academy. Tucson, AZ.

Bleiberg, M. & Welsh, M.E. (2003, Jan.). Evaluation of the TASC after-school program: Outcomes associated with TASC participation. Invited roundtable presentation at the University of Arizona Institute for Children, Youth and Families Conference “Bridging the gap between research and practice:  Creating common ground in youth development.” Tucson, AZ.

University Teaching Experience

• EPSY 4010:  Assessment of Learning I
• EPSY 4015:  Assessment of Learning II
• EPSY 5602: Educational Tests and Measurements
• EPSY 5605: Quantitative Methods in Research I
• EPSY 6194: Advanced Assessment
• EPSY 6621: Program Evaluation
• EPSY 6636: Measurement Theory and Application
• EDP 357/358: Methods of Educational Assessment of Students (University of Arizona, Tucson, AZ)

Funded Grants

Co-Principal Investigator.  Project PAPER: Preparing Academics in Psychometrics and Educational Research. Funding Source: US Department of Education. Duration of Funding: 8/12-8/15. Total amount funded: $400,000.

Principal Investigator. Validation of Student Learning Objectives as a Teacher Evaluation Tool. Funding Source: University of Connecticut Large Faculty Grant. Duration of Funding: 1/1/12-12/31/13. Total amount funded: $22,084.

Co-Principal Investigator.  Project VIABLE-II: Unified validation of Direct Behavior Rating (DBR) in a problem-solving model. Funding Source: US Department of Education, Institute for Education Sciences. Duration of Funding: 7/1/11-6/30/15. Total amount funded: $ 2,332,829.

Senior Personnel/Evaluator. Exploratory Curriculum for Trustable Computing Systems Security Education. Funding Source: National Science Foundation. Duration of funding: 8/2011-8/2013. Total amount funded: $199,996.

Evaluator. Graduate Assistance in Areas of National Need: Cloud Computing.  Funding Source: U.S. Department of Education. Duration of funding: 8/2009-8/2012. Total amount funded: $393,795.

Evaluator. Graduate Assistance in Areas of National Need: Sustainable Energy.  Funding Source: U.S. Department of Education. Duration of funding: 8/2009-8/2012. Total amount funded: $522,624.

Evaluator. Graduate Assistance in Areas of National Need: Advanced Computing.  Funding Source: U.S. Department of Education. Duration of funding: 8/2009-8/2012. Total amount funded: $522,624.

Evaluator. Graduate Assistance in Areas of National Need: Computer Systems Security.  Funding Source: U.S. Department of Education. Duration of funding: 8/2009-8/2012. Total amount funded: $522,624.

Evaluator. Graduate Assistance in Areas of National Need: Biomaterials for Tissue Regeneration.  Funding Source: U.S. Department of Education. Duration of funding: 8/2009-8/2012. Total amount funded: $653,280.

National Service

Associate Editor (2013-2017), Educational Administration Quarterly

Editorial Board, Journal of Advanced Academics

Editorial Board, Gifted Child Quarterly

Editorial Board, Frontiers in Quantitative Psychology and Measurement

Reviewer, Educational Measurement: Issues and Practice

Reviewer, Educational Assessment Journal

Reviewer, Educational and Psychological Measurement

Member, Behavioral Progress Monitoring Technical Review Committee

Member, Connecticut State Network of Educators Smarter Balanced Digital Library

Distinguished Paper Award Reviewer, AERA Classroom Assessment SIG

Proposal Reviewer, AERA Annual Conference

Selected as a NERA Mentor, 2011 Northeastern Educational Research Association Annual Meeting

University Service

Program Coordinator, Graduate Certificate in Program Evaluation, 2010-2013

Dissertation Proposal Guidelines Committee, Educational Psychology Department, 2011-12

Teacher Education Outcomes Committee, Teacher Preparation Program, 2011-12

Scholarship Committee, Educational Psychology Department, 2011-present

MEA Search Committee, Educational Psychology Department, 2010-11

MEA Search Committee, Educational Psychology Department, 2011-12

MEA Search Committee (Cluster Hire), Educational Psychology Department, 2012-13

MEA Search Committee (Associate/Full Hire), Educational Psychology Department, 2012-13

MEA Search Committee, Educational Psychology Department, 2013-14

Community Building Committee, School of Education, 2009-present

Neag Assessment Committee, School of Education, 2008-present

Honors and Fellowships

National Council on Measurement in Education (NCME) Outstanding Service Award (2019-20)

Organisation for Economic Co-operation and Development (OECD) Thomas J. Alexander Fellow (2019-20)

UC Davis Chancellor’s Fellow (2019-20)

University of Arizona Graduate College Fellowship (2004-05, 2005-06)

University of Arizona College of Education Dean’s Office Stipend (2004-05, 2005-06)

Memberships

American Educational Research Association

• Division D
• Classroom Assessment SIG
• Large Scale Assessment SIG
• Cognition and Assessment SIG
• Test Validity SIG

American Psychological Association

National Council on Measurement in Education

Northeastern Educational Research Association