Aprimorando a qualidade em surveys online: descobrindo temas e estratégias fundamentais
DOI:
https://doi.org/10.5585/remark.v23i4.25692Palavras-chave:
Metodologia, Questionário online, Qualidade da pesquisa, Qualidade dos dadosResumo
Objetivo: Este artigo teve como objetivo identificar quais são os temas e estratégias fundamentais que podem aumentar a qualidade das respostas em surveys online.
Método: O termo "survey online" e outras variações foram utilizados como palavras-chave amplas no processo de seleção para identificar artigos metodológicos e empíricos sobre a qualidade das pesquisas online. O banco de dados selecionado foi descrito utilizando técnicas bibliométricas. Os temas fundamentais foram identificados com a análise de cocitação, enquanto e as estratégias recomendadas foram determinadas utilizando a análise de acoplamento bibliográfico.
Resultados: Os temas fundamentais na literatura sobre pesquisas online são: Dispositivos, Modalidade de Administração, Design de Perguntas, Respostas Descuidada, Taxa de Resposta, Paradados, Ajuste Estatístico, Incentivos e Survey Domiciliar. Os temas Dispositivos e Modalidade de Administração enfatizam na descrição e na comparação dos métodos de coleta online, os métodos com abordagens tradicionais e o uso de diferentes dispositivos. Os demais temas investigam estratégias voltadas para a aprimorar respostas em surveys online, com o foco em estratégias específicas, indicadores de qualidade ou comportamentos dos participantes.
Originalidade/Valor: Este estudo serve como um guia valioso para pesquisadores de survey. Até onde sabemos, esta é a primeira revisão utilizando a análise de cocitação para identificar as principais estratégias para aprimorar a qualidade das respostas em survey online.
Contribuições Teóricas/Metodológicas: Esta pesquisa contribui em várias disciplinas ao identificar as principais estratégias para melhorar a qualidade das respostas em survey online e fornecer orientações para pesquisadores de survey.
Contribuições Sociais/Gerenciais: Ao destacar a importância dos protocolos de survey e os potenciais vieses e erros associados à pesquisa não planejada, os resultados oferecem insights práticos para contextos sociais e gerenciais.
Downloads
Referências
Al-Salom, P., & Miller, C. J. (2019). The Problem with Online Data Collection: Predicting Invalid Responding in Undergraduate Samples. Current Psychology, 38(5), 1258–1264. https://doi.org/10.1007/s12144-017-9674-9
Alwin, D. F., & Krosnick, J. A. (1991). The Reliability of Survey Attitude Measurement: The Influence of Question and Respondent Attributes. Sociological Methods & Research, 20(1). https://doi.org/10.1177/0049124191020001005
Bethlehem, J. (2010). Selection bias in web surveys. International Statistical Review, 78(2), 161-188. https://doi.org/10.1111/j.1751-5823.2010.00112.x
Biemer, P. P. (2010). Total survey error: Design, implementation, and evaluation. Public Opinion Quarterly, 74(5), 817–848. https://doi.org/10.1093/poq/nfq058
Bosnjak, M., Dannwolf, T., Enderle, T., Schaurer, I., Struminskaya, B., Tanner, A., & Weyandt, K. W. (2018). Establishing an Open Probability-Based Mixed-Mode Panel of the General Population in Germany: The GESIS Panel. Social Science Computer Review, 36(1). https://doi.org/10.1177/0894439317697949
Bowling, A. (2005). Mode of questionnaire administration can have serious effects on data quality. Journal of Public Health, 27(3), 281–291. https://doi.org/10.1093/pubmed/fdi031
Bybee, S., Cloyes, K., Baucom, B., Supiano, K., Mooney, K., & Ellington, L. (2022). Bots and nots: safeguarding online survey research with underrepresented and diverse populations. Psychology & Sexuality, 13(4), 901-911. https://doi.org/10.1080/19419899.2021.1936617
Callegaro, M. (2013). Paradata in Web Surveys. In Kreuter, F. (Ed.), Improving surveys with paradata: Analytic uses of process information (pp. 259–279). John Wiley & Sons. https://doi.org/10.1002/9781118596869.ch11
Cornesse, C., & Bosnjak, M. (2018, April). Is there an association between survey characteristics and representativeness? A meta-analysis. Survey Research Methods. 12(1), 1-13. https://doi.org/10.18148/srm/2018.v12i1.7205
Couper, M. P. (2011). The future of modes of data collection. Public Opinion Quarterly, 75(5), 889-908. https://doi.org/10.1093/poq/nfr046
Couper, M. P., Kapteyn, A., Schonlau, M., & Winter, J. (2007). Noncoverage and nonresponse in an Internet survey. Social Science Research, 36(1), 131-148. https://doi.org/10.1016/j.ssresearch.2005.10.002
Crawford, S. D., Couper, M. P., & Lamias, M. J. (2001). Web surveys: Perceptions of burden. Social Science Computer Review, 19(2), 146-162. https://doi.org/10.1177/089443930101900202
Curran, P. G. (2016). Methods for the detection of carelessly invalid responses in survey data. Journal of Experimental Social Psychology, 66, 4-9. https://doi.org/10.1016/j.jesp.2015.07.006
Daikeler, J., Bosnjak, M., & Manfreda, K. L. (2020). Web versus other survey modes: An updated and extended meta-analysis comparing response rates. Journal of Survey Statistics and Methodology, 8(3), 513–539. https://doi.org/10.1093/jssam/smz008
De Man, J., Campbell, L., Tabana, H., & Wouters, E. (2021). The pandemic of online research in times of COVID-19. BMJ Open, 11(2). https://doi.org/10.1136/bmjopen-2020-043866
DeCastellarnau, A. (2018). A classification of response scale characteristics that affect data quality: a literature review. Quality & quantity, 52(4), 1523-1559. https://doi.org/10.1007/s11135-017-0533-4
DeLeeuw, E. D. (2018, August). Mixed-mode: Past, present, and future. Survey Research Methods. 12(2), 75-89. https://doi.org/10.18148/srm/2018.v12i2.7402
Deutskens, E., de Ruyter, K., Wetzels, M., & Oosterveld, P. (2004). Response Rate and Response Quality of Internet-Based Surveys: An Experimental Study. Marketing Letters, 15(1), 21–36. https://doi.org/10.1023/B:MARK.0000021968.86465.00
Dever, J. A., Rafferty, A., & Valliant, R. (2008). Internet surveys: Can statistical adjustments eliminate coverage bias? Survey Research Methods, 2(2), 47–60. https://doi.org/10.18148/srm/2008.v2i2.128
Dillman, D. A., & Christian, L. M. (2005). Survey Mode as a Source of Instability in Responses across Surveys. Field Methods, 17(1), 30-52. https://doi.org/10.1177/1525822X04269550
Donthu, N., Kumar, S., Mukherjee, D., Pandey, N., & Lim, W. M. (2021). How to conduct a bibliometric analysis: An overview and guidelines. Journal of business research, 133, 285-296. https://doi.org/10.1016/j.jbusres.2021.04.070
Evans, J. R., & Mathur, A. (2018). The value of online surveys: a look back and a look ahead. Internet Research, 28(4), 854–887. https://doi.org/10.1108/IntR-03-2018-0089
Ferri-García, R., & Rueda, M. D. M. (2020). Propensity score adjustment using machine learning classification algorithms to control selection bias in online surveys. PloS one, 15(4), e0231500. https://doi.org/10.1371/journal.pone.0231500
Freire, O., Senise, D. D. S. V., dos Reis, W. B., & Ono, H. S. (2017). # Step@ by# Step: recomendações para o desenvolvimento de pesquisa online de alta qualidade. Revista Gestão & Tecnologia, 17(3), 36-62. https://doi.org/10.20397/2177-6652/2017.v17i3.1281
Galesic, M., Tourangeau, R., Couper, M. P., & Conrad, F. G. (2008). Eye-tracking data: New insights on response order effects and other cognitive shortcuts in survey responding. Public Opinion Quarterly, 72(5), 892–913. https://doi.org/10.1093/poq/nfn059
Göritz, A. S. (2006). Incentives in web studies: Methodological issues and a review. International Journal of Internet Science, 1(1), 58-70. https://jwolf-ra.wdfiles.com/local--files/incentives/goritz_2006.pdf
Groves, R. M. (2006). Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly, 70(5), 646–675. https://doi.org/10.1093/poq/nfl033
Groves, R. M., & Lyberg, L. (2010). Total survey error: Past, present, and future. Public Opinion Quarterly, 74(5), 849–879. https://doi.org/10.1093/poq/nfq065
Groves, R. M., Presser, S., & Dipko, S. (2004). The role of topic interest in survey participation decisions. Public Opinion Quarterly, 68, 2–31. https://doi.org/10.1093/poq/nfh002
Groves, R. M., Singer, E., & Corning, A. (2000). Leverage-saliency theory of survey participation: Description and an illustration. Public Opinion Quarterly, 64, 299–308. https://doi.org/10.1086/317990
Hair, J. F. (2009). Multivariate data analysis (7th ed.). Prentice Hall.
Hulland, J., Baumgartner, H., & Smith, K. M. (2018). Marketing survey research best practices: evidence and recommendations from a review of JAMS articles. Journal of the Academy of Marketing Science, 46(1), 92–108. https://doi.org/10.1007/s11747-017-0532-y
Iannelli, L., Giglietto, F., Rossi, L., & Zurovac, E. (2020). Facebook digital traces for survey research: Assessing the efficiency and effectiveness of a Facebook Ad–based procedure for recruiting online survey respondents in niche and difficult-to-reach populations. Social Science Computer Review, 38(4), 462-476. https://doi.org/10.1177/0894439318816638
Johnson, J. A. (2005). Ascertaining the validity of individual protocols from Web-based personality inventories. Journal of Research in Personality, 39(1), 103-129. https://doi.org/10.1016/j.jrp.2004.09.009
Keusch, F. (2015). Why do people participate in Web surveys? Applying survey participation theory to Internet survey data collection. Management Review Quarterly, 65(3), 183–216. https://doi.org/10.1007/s11301-014-0111-y
Keusch, F., & Zhang, C. (2017). A review of issues in gamified surveys. Social Science Computer Review, 35(2), 147-166. https://doi.org/10.1177/0894439315608451
Keusch, F., Bähr, S., Haas, G. C., Kreuter, F., & Trappmann, M. (2023). Coverage error in data collection combining mobile surveys with passive measurement using apps: Data from a German national survey. Sociological Methods & Research, 52(2), 841-878. https://doi.org/10.1177/0049124120914924
Krosnick, J. A., Holbrook, A. L., Berent, M. K., Carson, R. T., Michael Hanemann, W., Kopp, R. J., … Conaway, M. (2002). The Impact Of “No Opinion” Response Options On Data Quality Non-Attitude Reduction Or An Invitation To Satisfice? Public Opinion Quarterly, 66(3), 371–403. https://doi.org/10.1086/341394
Lee, S. (2006). Propensity score adjustment as a weighting scheme for volunteer panel web surveys. Journal of Official Statistics, 22(2), 329-349. https://www.proquest.com/docview/1266792612
Leeuw, E. D. De. (2005). To Mix or Not to Mix Data Collection Modes in Surveys. Journal of Official Statistics, 21(2), 233-255. https://www.proquest.com/docview/1266791766
Lehdonvirta, V., Oksanen, A., Räsänen, P., & Blank, G. (2021). Social Media, Web, and Panel Surveys: Using Non-Probability Samples in Social and Policy Research. Policy and Internet, 13(1), 134–155. https://doi.org/10.1002/poi3.238
Leiner, D. J. (2019). Too fast, too straight, too weird: Non-reactive indicators for meaningless data in internet surveys. Survey Research Methods, 13(3), 229–248. https://doi.org/10.18148/srm/2019.v13i3.7403
Liebe, U., Glenk, K., von Meyer-Höfer, M., & Spiller, A. (2019). A web survey application of real choice experiments. Journal of choice modelling, 33, 100150. https://doi.org/10.1016/j.jocm.2018.07.003
Lugtig, P., & Toepoel, V. (2016). The use of PCs, smartphones, and tablets in a probability-based panel survey: Effects on survey measurement error. Social science computer review, 34(1), 78-94. https://doi.org/10.1177/0894439315574248
Maniaci, M. R., & Rogge, R. D. (2014). Caring about carelessness: Participant inattention and its effects on research. Journal of Research in Personality, 48(1), 61-83. https://doi.org/10.1016/j.jrp.2013.09.008
Marcus, B., Bosnjak, M., Lindner, S., Pilischenko, S., & Schütz, A. (2007). Compensating for low topic interest and long surveys: A field experiment on nonresponse in web surveys. Social Science Computer Review, 25(3), 372-383. https://doi.org/10.1177/0894439307297606
Matias, J., Kalamara, E., Mathis, F., Skarupova, K., Noor, A., Singleton, N., … Grp, E. W. S. D. (2019). The use of multi-national web surveys for comparative analysis: Lessons from the European Web Survey on Drugs. International Journal of Drug Policy, 73, 235–244. https://doi.org/10.1016/j.drugpo.2019.03.014
Mavletova, A., & Couper, M. P. (2014). Mobile web survey design: Scrolling versus paging, sms versus e-mail invitations. Journal of Survey Statistics and Methodology, 2(4), 498–518. https://doi.org/10.1093/jssam/smu015
McCain, K. W. (1990). Mapping authors in intellectual space: A technical overview. Journal of the American Society for Information Science, 41(6), 433-443. https://doi.org/10.1002/(SICI)1097-4571(199009)41:6<433::AID-ASI11>3.0.CO;2-Q
Meade, A. W., & Craig, S. B. (2012). Identifying careless responses in survey data. Psychological Methods, 17(3), 437–455. https://doi.org/10.1037/a0028085
Olson, K., Smyth, J. D., Horwitz, R., Keeter, S., Lesser, V., Marken, S., … Wagner, J. (2021). Transitions From Telephone Surveys To Self-Administered And Mixed-Mode Surveys: Aapor Task Force Report. Journal of Survey Statistics and Methodology, 9(3), 381–411. https://doi.org/10.1093/jssam/smz062
Palaniappan, K., & Kum, I. Y. S. (2019). Underlying Causes behind Research Study Participants’ Careless and Biased Responses in the Field of Sciences. Current Psychology, 38(6), 1737–1747. https://doi.org/10.1007/s12144-017-9733-2
Paul, J., Lim, W. M., O’Cass, A., Hao, A. W., & Bresciani, S. (2021). Scientific procedures and rationales for systematic literature reviews (SPAR-4-SLR). International Journal of Consumer Studies, 45(4), O1-O16. https://doi.org/10.1111/ijcs.12695
Peytchev, A., & Hill, C. A. (2010). Experiments in mobile web survey design: Similarities to other modes and unique considerations. Social Science Computer Review, 28(3), 319-335. https://doi.org/10.1177/0894439309353037
Pickert, R. (2023). Survey Response Rates Are Down Since Covid. That’s Worrying for US Economic Data. Bloomberg. https://www.bloomberg.com/news/articles/2023-02-15/us-economic-data-threatened-by-survey-non-responses
Quevedo-Silva, F., Almeida Santos, E. B., Brandão, M. M., & Vils, L. (2016). Estudo Bibliométrico: Orientações sobre sua Aplicação. Revista Brasileira de Marketing, 15(2), 246–262. https://doi.org/10.5585/remark.v15i2.3274
Revilla, M., & Ochoa, C. (2017). Ideal and maximum length for a web survey. International Journal of Market Research, 59(5), 557-565. https://doi.org/10.2501/IJMR-2017-039
Roßmann, J., Gummer, T., & Silber, H. (2018). Mitigating satisficing in cognitively demanding grid questions: Evidence from two web-based experiments. Journal of Survey Statistics and Methodology, 6(3), 376-400. https://doi.org/10.1093/jssam/smx020
Rouse, S. V. (2019). Reliability of MTurk data from masters and workers. Journal of Individual Differences. 41(1), 30–36. https://doi.org/10.1027/1614-0001/a000300
Sammut, R., Griscti, O., & Norman, I. J. (2021). Strategies to improve response rates to web surveys: a literature review. International Journal of Nursing Studies, 123, 104058. https://doi.org/10.1016/j.ijnurstu.2021.104058
Saris, W. E., Revilla, M., Krosnick, J. A., & Shaeffer, E. M. (2010). Comparing questions with agree/disagree response options to questions with item-specific response options. Survey Research Methods, 4(1), 61–79.. https://doi.org/10.18148/srm/2010.v4i1.2682
Sauer, P. C., & Seuring, S. (2023). How to conduct systematic literature reviews in management research: a guide in 6 steps and 14 decisions. Review of Managerial Science, 17, 1899–1933. https://doi.org/10.1007/s11846-023-00668-3
Sauermann, H., & Roach, M. (2013). Increasing web survey response rates in innovation research: An experimental study of static and dynamic contact design features. Research Policy, 42(1), 273-286. https://doi.org/10.1016/j.respol.2012.05.003
Shamon, H., & Berning, C. C. (2020). Attention Check Items and Instructions in Online Surveys with Incentivized and Non-Incentivized Samples: Boon or Bane for Data Quality?. Survey Research Methods, 14(1), 55-77. https://doi.org/10.18148/srm/2020.v14i1.7374
Singer, E., & Ye, C. (2013). The Use and Effects of Incentives in Surveys. Annals of the American Academy of Political and Social Science, 645(1), 112-141. https://doi.org/10.1177/0002716212458082
Smith, S. M., Roster, C. A., Golden, L. L., & Albaum, G. S. (2016). A multi-group analysis of online survey respondent data quality: Comparing a regular USA consumer panel to MTurk samples. Journal of Business Research, 69(8), 3139–3148. https://doi.org/10.1016/j.jbusres.2015.12.002
Stoet, G. (2017). PsyToolkit: A Novel Web-Based Method for Running Online Questionnaires and Reaction-Time Experiments. Teaching of Psychology, 44(1), 24–31. https://doi.org/10.1177/0098628316677643
Storozuk, A., Ashley, M., Delage, V., & Maloney, E. A. (2020). Got bots? Practical recommendations to protect online survey data from bot attacks. The Quantitative Methods for Psychology, 16(5), 472-481. https://doi.org/10.20982/tqmp.16.5.p472
Struminskaya, B., Weyandt, K., & Bosnjak, M. (2015). The Effects of Questionnaire Completion Using Mobile Devices on Data Quality. Evidence from a Probability-based General Population Panel. Methods, Data, Analyses, 9(2). https://doi.org/10.12758/mda.2015.014
Tourangeau, R. (2003). Cognitive aspects of survey measurement and mismeasurement. International Journal of Public Opinion Research, 15(1), 3–7. https://doi.org/10.1093/ijpor/15.1.3
Tourangeau, R., Couper, M. P., & Conrad, F. (2004). Spacing, position, and order: Interpretive heuristics for visual features of survey questions. Public Opinion Quarterly, 68(3), 368–393. https://doi.org/10.1093/poq/nfh035
Tourangeau, R., Couper, M. P., & Conrad, F. (2007). Color, labels, and interpretive heuristics for response scales. Public Opinion Quarterly, 71(1), 91–112. https://doi.org/10.1093/poq/nfl046
Walzenbach, S. (2019). Hiding Sensitive Topics by Design? An Experiment on the Reduction of Social Desirability Bias in Factorial Surveys. Survey Research Methods, 13(1), 103–121. https://doi.org/10.18148/srm/2019.v1i1.7243
Weber, S. (2021). A step-by-step procedure to implement discrete choice experiments in Qualtrics. Social Science Computer Review, 39(5), 903-921. https://doi.org/10.1177/0894439319885317
Wells, T., Bailey, J. T., & Link, M. W. (2014). Comparison of Smartphone and Online Computer Survey Administration. Social Science Computer Review, 32(2), 238-255. https://doi.org/10.1177/0894439313505829
Zupic, I., & Čater, T. (2015). Bibliometric Methods in Management and Organization. Organizational Research Methods, 18(3), 429-472. https://doi.org/10.1177/1094428114562629
Zwarun, L., & Hall, A. (2014). What’s going on? Age, distraction, and multitasking during online survey taking. Computers in Human Behavior, 41, 236-244. https://doi.org/10.1016/j.chb.2014.09.041
Downloads
Publicado
Como Citar
Edição
Seção
Licença
Copyright (c) 2024 Fernanda Sayuri Yoda, Otávio Bandeira de Lamônica Freire
Este trabalho está licenciado sob uma licença Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
