ORIGINAL

Implementation of the Electronic Medical Record based on the Theory of the Innovation Diffusion: a case study

Daniela Sousa Gomes1, Regina Consolação dos Santos2, Tarcísio Laerte Gontijo1, Valéria Conceição de Oliveira1, Eliete Albano Azevedo Guimarães1, Ricardo Bezerra Cavalcante2

1 Federal University of São João Del Rei, MG, Brazil

2 Federal University of Juiz de Fora, MG, Brazil

ABSTRACT

Objective: To analyze the diffusion of the Electronic Medical Record of the Citizen within family health teams at a district in the West Region of Minas Gerais State. Method: A case study, qualitative approach, based on the Innovation Diffusion Theory. We carried out 17 structured interviews with professionals and observations of the daily routine of eight family health teams that used the Electronic Medical Record of the Citizen (EMRC). Data were analyzed by means of Content Analysis, the Thematic-Categorical modality. Results: The participants identified the Electronic Medical Record of the Citizen as more advantageous than the previous technology – compatible with their needs, not a complex technology. Despite the short experimentation period, the results and the contributions of this technology were acknowledged. Conclusion: Over its diffusion period, the Electronic Medical Record of the Citizen has been proving to be a potentially able innovation to transform daily activities, furthering receptive attitudes. Nevertheless, there are fragile situations to be overcome.

Descriptors: Electronic Health Records; Innovations Diffusion; Family Health Strategy.

INTRODUCTION

In Brazil, the Primary Health Care – PHC is currently going through a computer-aided process identified as e-SUS Basic Attention (e-SUS AB), meant to re-structure information and the Information System in Health, aimed at one single electronic Health System. The process is expected to lead to the individualization of the users’ data, the integration of the entire national SIS, the elimination of re-working of records and the optimization of the working process(1).

 For this national strategy, one of the technologies being introduced is the Electronic Medical Record of the Citizen – EMRC –, a software system that allows health professionals to develop the care management providing assertive decisions. Furthermore, the EMRC is also expected to provide support for both the planning and the programming of local actions, as well as agenda organization and the assistance flow for the users.

In this sense, the international experiences of implementation of electronic devices in the PHS confirm expectations posed on the EMRC being introduced in Brazil. Some studies proved that the electronic medical record assigns agility to assistance procedures, reduces information records mistakes as well as the working load of the administrative team. Information are more organized, can be read and shared more easily, thus optimizing the follow-up of patients(2,4). Furthermore, the EMRC helps when prescribing medicines and may reduce risks to the patients’ safety, preventing adverse events(3,4).The professionals’ perceptions about the technology, deficiencies involving the access to the internet (connectivity) and the lack of capability of those who deal with the technology affect its functionalities(5).

In the context of Brazilian PHC, EMRC represents a technological innovation with the potential to promote changes to the daily work and the behavior of both the health professionals and the patients. The innovation may be considered an idea, a process, an object or a technology that is perceived as new or still unknown by the individuals(6). Therefore, the success of the EMRC as an innovation depends on being used daily in PHC activities, as well as being accepted by those who shall use it in the future and their motivations.

An innovation carries “attributes” that might influence its diffusion, affecting either its acceptance or its rejection by those who will use it. Attributes are characteristics that are exclusive of the innovation itself: relative advantage, compatibility, complexity, experimentation and possibility of being perceived(6). Such attributes are commonly used to elucidate acceptance or rejection determinants of innovations in experiences of incorporation of electronic medical records(7,9).

Therefore, considering an innovation meant to support the computerization of the PHC in a continental country such as Brazil, one must understand how its diffusion is taking place, leading to its acceptance or rejection. Studies on the diffusion of the EMRC are vital to elucidate situations that might harm the health teams in the process of acceptance of this technology, risking potential benefits its use may lead to. In this sense, the present research is guided by the following question: “How the EMRC has been diffused among Family Health Teams (FHTs)?”  In this article, we try to analyze the diffusion of EMRC among family health teams of a municipality in the West Region of Minas Gerais State.

METHOD

A case study with qualitative approach, based on the Theory of Innovation Diffusion (TID)(6). We use the “attributes” of an innovation that potentiates its acceptance or rejection in the diffusion process (Figure 1).

Figure 1 – Attributes and their definitions based on the Theory of Innovation Diffusion. Brazil, 2019

Source: Developed by the authors, 2019.

The scenery of the research was defined to be the FHTs (Family Health Teams) of a municipality in the West Region of Minas Gerais State, in Brazil. That was an intentional choice, as the local process of implementation and use of the EMRC was just starting out. In that municipality, 18 family health teams assist 59% of the population. The following inclusion criteria were defined for the EMRC selection: teams at the initial stage of the EMRC implementation process; and teams whose members already used the EMRC in their daily work. The analysis of an innovation at the initial stage of implementation helps understanding its diffusion process among those who shall adopt a social system in the future(16).

Once those criteria were defined, one of the 18 FHTs selected was excluded, as the professionals were not using the EMRC. Next, we carried out a random lottery to define the sequence of 17 eligible FHTs that would take part in the data collection. For those 17 FHTs selected to build up the board of participants of the study, the following inclusion criteria were defined: professionals who already used the EMRC; who were present at the Basic Health Unit (BHU) at the moment data were collected; and who did accept to take part in the study. The exclusion criteria were: professionals out on vacation, out in leave, not present at the unit at the moment data were collected, besides those who did not use the EMRC. Furthermore, data saturation technique was used to close the sample of participants(11). Thus, saturation was achieved with the number of professionals at the seventeenth interview. All 17 participants were codified according to their professional category and their respective number, followed by the FHT code they worked with. The group of 17 participants was composed by four medical doctors, six nurses, three nursing technicians and four dentists.

In order to save participants’ anonymity, the following coding systematics was assigned: the professional category and its respective numbering, followed by the coding of the family health team he/she took part of. Data collection was carried out from November, 2018, to March, 2019. We realized structured interviews and direct observation, non-participant. Both evidence sources were built up based on the innovation attributes, as defined by the Theory of Diffusion and Innovation (TDI) (Figure 2).

Figure 2 – Itinerary for questions of interviews and items of observation according to the Innovation Attributes. Divinópolis, MG, Brazil, 2019

Source: Developed by the authors, 2019.

Data were collected after participants did receive the Term of Free and Explained Consent (TFEC). The collection took place in day shift, at the participants’ working location, and previously scheduled. Interviews were carried out in a private room, in the presence of just the researcher and the participant, and the audio was taped, with the interviewee’s consent. The professionals were observed on three typical days of one working week at each FHT, when using the EMRP. Over five months of collection, 224 observation hours were carried out. Observations were recorded in a field diary and produced Observation Notes (ON).

Data collected were submitted to a Content Analysis, The Thematic Category modality(12). At the first stage – the pre-analysis –, a “flowing” reading took place: the first contact with the interviews and observations submitted to analysis. Later on, the material was prepared. At the second stage – the phase of material exploration –, the record units were codified, producing the sense nuclei (interpretations of the researcher based on the theoretical system of references). At this second stage, the sense nuclei were allocated into categories that were previously defined based on the Innovation Attributes established at the TDI: Relative Advantage; Compatibility; Complexity; Experimentation; Possibility to be observed. The previous categorization is also a possibility in the Content Analysis(12).

At the third stage of the process of content analysis – the results treatment –, gross results were interpreted so as to make them meaningful and valid, respecting the inference of interviews and observations, as well as the interpretation of concepts and propositions. The study followed Resolution 466/2012 by the National Health Council, and was approved by the Ethic Committee in Research of the Federal University of São João Del Rei, under Report Nr. 3,297,521.

RESULTS

Relative advantage: contribution of the Electronic Medical Report of the Citizen (EMRC) when compared to entry registers on paper

Concerning the attribute “relative advantage”, interviewees reported that the EMRC provides information organization:

[...] as formerly medical records on paper could be lost [...] (ENF1-ESFA).

Another advantage was agility in the assistance:

[...] you are already at the computer, typing. One is already dropping information at that very moment. It’s very fast [...] (D4-ESFF).

The credibility of information entered into the EMRC was also mentioned as an advantage by the professionals:

[...] for legal purposes, it is much more reliable than on paper [...] For instance, one patient called the office of the auditor complaining about lack of attention, claiming there was no attention at all. I printed everything in the EMRC and sent to the auditor. [...] (ENF4-ESFD).

Professionals felt safer when recording data on electronic media, as the access to the system requires a password:

[...] I think the EMRC is more worthy than the written alternative. Writing on paper is at anyones’ reach. But as it is on your Login, on your data, I believe it is more difficult for anyone to try to do anything else [...] (TENF3- ESFD).

The agility when recording provided by the EMRC was also mentioned as a relative advantage: (ENF6-ESF).

Compatibilities and incompatibilities of the Electronic Medical Record of the Citizen regarding the daily work

As to the attribute “compatibility”, some professionals pointed out that the EMRC is compatible with their working needs.

[...] They are perfectly compatible. Everything we need is in the program [...] (ENF6- ESFF).

Other interviewees also remarked the compatibility of the EMRC when considering the need to systematize the assistance, besides fulfilling requirements of specific councils of professionals, such as the nursing:

[...] COREN (Regional Council of Nursing) firmly requires the systematization in nursing. It is quite similar in the SOAP with the systematization method. You have data collection, nursing diagnosis and prescription [...] (ENF4- ESFD).

On the other hand, some functionalities of the EMRC do not match the professionals’ working reality, mainly the daily work on Basic Attention, producing incompleteness and the need to keep on recording some things on paper:

[...] for me, the more prejudiced field is the procedure field. For instance, cerume remotion is a very usual procedure in primary attention, and you do not find it in the EMRC [...] (D2- ESFC).

Another participant remarks:

[...] some things are lacking, we register them on paper, separately [...] (D3- ESFE).

Problems have also occurred involving the internet (connectivity) that did harm the work of the professionals, affecting the relationship with the patients:

[...] There were times we’ve come to assist the patient and as you try to enter the system, the signal (internet) is already down, and this somehow disturbs the work (M2- ESFB).

This situation reinforces the co-existence between the use of both technology and paper in the daily work of the professionals:

[...] not even the EMRC did operate, as they were undergoing maintenance. Even more when it rains. That is one more reason for me to use the written medical record, on paper. Because one cannot trust [...] (ENF5-ESFE).

Moreover, some professionals highlighted the extra time at the computer, to the detriment of the assistance itself:

[...] the difficult part is for you to be at the computer all day long. At this very moment, you are getting LRE (Lesion for Recurrent Effort), and you could be with the patient instead [...] (TENF2-ESFA).

Another participat remarks:

[...] my big issue today when using the EMRC does not involve the program itself, it has to do with ergonomics. Due to the position at the computer, the table size, the mouse [...] (D2-ESFC).

Observing the flow of assistances to patients since the EMRC was introduced, we did often hear professionals arguing about “system laziness”, “hardware jammed”, which led to delays when assisting users of the service, “conflicts and professional unsatisfaction, thus weakening the use of the EMRC (ON).

Complexities related to the Electronic Medical Report of the Citizen and its use

The attribute “complexity” was related at first with difficulties concerning the comprehension of some items required by the program during the procedure of recording information:

[...] the EMRC requires an input choice that sometimes is not clear for me (ENF2- ESFB);

Another participant confirms: [...] if I do record [the information] in the EMRC just as assistance, does it go straight to the CDS, for invoicing? And what about the monthly productivity? [...] (M2- ESFB).

Fragilities concerning the qualification process to use the EMRC also contributed to make its operation more complex:

[...] because we lack qualification, we actually have to learn by ourselves, and sometimes it is very difficult [...] Situations like erasing, deleting, inactivating one family, we are not learning (M1- ESFA).

Some professionals faced difficulties to exclude information from the EMRC. They would ask the unit nurse for help, and many times she would face difficulties as well, and would say: “I’ll call the central office” (ON).

Another interviewee reported:

[...] there was a meeting there, they passed on for us [...] we couldn’t understand a thing [...] then we go on trying to find out how things work [...] (ENF1- ESFA).

The lack of qualification of departments responsible for the implementation process of the EMRC was also mentioned by the participant:

[...] no one came from the ministry, no one came from de regional department to prepare the staff. There was no one, we didn’t have that [...] (ENF4- ESFD).

The acknowledgement by the professionals that the EMRC is extensive and detailed also contributed to enlarge the perception of its complexity:

[...] it is a very extensive medical record, plenty with details. So that’s what makes it complicated [...] (M3- ESFC); [...] I consider it very extensive, very detailed. Those parts on subjective, objective evaluation, they could be more concise [...] (ENF3- ESFC).

The remarks also endorsed the interviews findings: qualification fragilities; the EMRC considered too extensive and requiring too many information and time for one to enter the records; professionals’ difficulties to understand the items to be filled in in the EMRC. At the end, all those situations increased the time the patients would wait for assistance (ON).

Experimentation as a short adaptation period

During the observation period, we often heard professionals mention the lack of a testing period when they could have tried the program and expressed their opinion about possibilities of adequation to the actual demands of the daily work at the Basic Attention. Instead, they had just a short adaptation period. This situation was related during the interviews as well:

[...] we had no adaptation period. It had to be fast, we could not even talk about what should be changed, adapted, improved [...] (D3- ESFE); [...] I do not even consider that a testing period, as from de very first moment we were told: start using it because we are abandoning paper, it is the government’s determination [...] (ENF6- ESFF).

Possibility to be observed: contributions of the Electronic Medical Record of the Citizen to integrate the patients’ information to the attendance network and among professionals

As to the attribute “possibility to be observed”, professionals at first reported that the EMRC was already providing the integration of the attendance network:

[...] the patient saw a doctor here, if he/she goes to another unit, the doctor can access his/her information, his/her history will be there. [...] (ENF5- ESFE). The integration goes beyond the assistance points of the network in that municipality: the patient is cared for as a whole, no matter if he/she is here in Formiga, in Divinópolis or in Belo Horizonte [...] (M1- ESFA).

The information integration can also be observed among health professionals:

[...] with the EMRC I need no more to go to the medical report, this was no longer that ordinary medical repor.t Ours remained here, the doctor’s remained at the reception, and there was also the nurse’s report. (ENF5- ESFE).

Another interviewee confirms: [...] all in one single place, you enter the consultation, pressure and weight. Now procedures are no longer bureaucratic, everything is integrated [...] (ENF4- ESFD).

Interviewees also observed as a result of the EMRC the easiness to recover information, making assistance more agile and contributing with the patients’ follow-up:

[...] you have quick access to medicines prescriptions, need no longer to be looking for them on paper. You just reprint to update medical prescriptions for accompanied diabetic and hypertensive patients [...] (M4- ESFG).

The EMRC was acknowledged as a fundamental and indispensable working tool:

[...] We now depend on it. Today it is fundamental in our work [...] (M1- ESFA); [...] in my reality, I can no longer imagine our work without the EMRC [...] (ENF3- ESFC).

In this sense, we observe that, among the changes provoked by the introduction of the EMRC, contributions that potentize its acceptance prevail (ON). Nevertheless, there are also fragilities related to the EMRC that potentize its rejection, as shown in Figure 3.

Figure 3 – Attributes, contributions and fragilities related to the EMRC in the scenery analyzed. Divinópolis, MG, Brazil, 2019

Source: Developed by the authors, 2019.

DISCUSSION

The analysis of the EMRC diffusion based on its attributes (Relative Advantage, Compatibility, Complexity, Experimentation, Possibility to be observed) reveals contributions of the technology favoring its adoption among professionals. Nevertheless, fragilities also emerged related to EMRC attributes, potentizing its rejection.

Concerning the attribute “relative advantage”, the EMRC has been acknowledged as more advantageous that procedures on paper, particularly when it comes to the information organization, the agility to record and restore information, credibility and information safety, and the legal support granted by the registration of the assistance provided to the users. In this sense, the acknowledgement of the advantages provided by the innovation adds to the adoption of the EMRC, as professionals will adopt an innovation if they realize it may offer advantages on previous practices(9,13). The adoption of technological innovations is potentized when a technology proves to be better than the previous one(7).

The compatibility of the EMRC with both the professionals’ and the patients’ demands is another advancement related to the innovation. The attribute “compatibility” highlights the consistence of an innovation regarding the existing social and technical environment(6). The better a innovation integrates or coexists with existing values, the previous experience and the needs of potential adopters, the larger will be its perspectives of diffusion and adoption(14,6). In this sense, the EMRC proved compatible with the reality of the PHC, only lacking some blanks required that were not considered in the software, and that may be corrected in future versions. As it starts being introduced, a technological innovation does not behave as a definite product: it goes through adaptations over the time, as it is used and changed to fulfill the users’ needs in their daily work(14).

The results of the EMRC observed on the daily work were also acknowledged: the integration of the patients’ information to the attendance network and among the professionals, the more agile assistance and the systematization of the follow-up of patients, turning the EMRC into an indispensable working tool. The results identified potentizes the quality of the assistance, the care management and the patients’ safety(15,16).

As to the EMRC diffusion, we did verify as well fragilities related to their attributes, potentizing the rejection to the innovation. First, we identified the need to widen the access to the internet, the connectivity, the furniture availability and the adequate infrastructure to maintain the technology in perfect conditions to be used. The national plan for the implementation of the EMRC includes the availability of computers and printers in attendance areas, besides access to the internet and stable connectivity(1). Brazil faces deficit in the communication area: the access to the internet and the connectivity are barriers for the diffusion of technological innovations in the health area, due to high costs and unviable network conditions(1). Besides, in distant localities, such as rural areas, the access to the internet is even more difficult, and the connectivity speed is less than what would be required, situations that are typical of Brazilian scenery(18).  

Another fragility verified was related to the software of the EMRC and the suitability to work demands provided in PHC services. Blanks to enter necessary information produced in daily work are missing, and there are too many items to be registered, some of them not understood by the professionals. Worsened by the insufficient experimentation period, such situations turn the innovation into a complex one and potentize its rejection. The evaluation by EMRC users adds to the improvement of its functionalities and, consequently, to its adoption. The innovation should not be just a technical ability: users should try and evaluate the innovation prior to its implementation(19,20).

As to restrains involved in the present research, its results are to be applicable to the scenery under analysis, not open for generalizations. Nevertheless, its originality may contribute for the continuity of the process of diffusion of the innovation EMRC.

CONCLUSION

The analysis of the diffusion process of the EMRC based on its attributes revealed it is more advantageous than the previous technology, is compatible with the users’ needs, is not perceived as a complex technology and, despite the short experimentation period, its results and contributions for the daily work can be perceived by the professionals: information organization; agility to record and restore information; credibility, safety of information and legal backing provided by the systematized registration; integration of information to the assistance network and among professionals; agile assistance and systematization of patients follow-up.

Nevertheless, the analysis of the EMRC diffusion based on its attributes did reveal fragilities as well, and they must be overcome: the EMRC carries incompatibilities regarding needs of the professionals; access and connectivity to the internet in the context analyzed are restricted; there are infrastructure and furniture deficiencies that compromise the diffusion; influences on the relationship professional-patient that potentize resistances; the fragile qualification process and the lack of experimentation make the use of the innovation and the comprehension of its functionalities more difficult. Those fragilities have been contributing for the professionals’ perception that the innovation is complex, thus leading to possible rejections.   

ACKNOWLEDGEMENTS

We express our gratitude to Ricardo Bezerra Cavalcante and Daniela Sousa Gomes for their substantial contribution for the Project conception and planning, for data collection and interpretation. We express our gratitude to Ricardo Bezerra Cavalcante, Daniela Sousa, Tarcisio Laerte Gontijo, Valeria da Conceição Oliveira, Eliete Albano de Azevedo Guimarães and Regina Consolação dos Santos for their expressive contribution to the preliminary sketch, the critical review of the content and the approval of the final version of the manuscript.

CONFLICT OF INTEREST

The authors have declared that there is no conflict of interest.

FINANCIAL SUPPORT

FAPEMIG (APQ-00337-15) and CNPq (404653/2016-2).

REFERENCES

1. Ministério da Saúde (BR). Secretaria de Atenção à Saúde. E-SUS atenção básica: manual do sistema com prontuário eletrônico do cidadão PEC (versão 3.1) [Internet]. Brasília: Ministério da Saúde; 2018 [cited 2019 Oct 10]. Available from: http://aps.saude.gov.br/ape/esus/manual/

2. Compeau DR, Terry A. Connecting medical records: na evaluation o benefit sand challenges for primary care practices. J Innov Health Inform. 2017;24(2):855. http://dx.doi.org/10.14236/jhi.v24i2.855.  PMid:28749315.

3. Price J, Man SL, Bartlett S, Taylor K, Dinwoodie M, Bowie P. Repeat prescribing of medications: a system- centred risk management model for primary care organisations. J Eval Clin Pract. 2017;23(4):779-96. http://dx.doi.org/10.1111/jep.12718. PMid:28370904.

4. Hoon SEM, Hek K, Van-Dijk L, Verheij RA. Adverse events recording in electronic health record systems in primary care. BMC Med Inform Decis Mak.2017;17(1):163.http://dx.doi.org/10.1186/s12911-017-0565-7. PMid:29212497.

5. Jeffries M, Phipps DL, Howard RL, Avery AJ, Rodgers S, Ashcroft DM. Understanding the implementation and adoption of a technological intervention to improve medication safety in primary care: a realist evaluation. BMC Health Serv Res. 2017;17(1):196. http://dx.doi.org/10.1186/s12913-017-2131-5. PMid:28288634.

6. Currie G, Spyridonidis D. Sharing leadership for diffusion of innovation in professionalized settings. Sage Journals. 2019;72(7):1209-33. http://dx.doi.org/10.1177/0018726718796175.

7. Olsson A, Skovdahl K, Engström M. Using diffusion of innovation theory to describe perceptions of a passive position ingalar mamong persons with milddementia: a repeated interview study. BMC Geriatr. 2016;16:3. http://dx.doi.org/10.1186/s12877-016-0183-8. PMid:26745961.

8. Mohammadi MM, Poursaberi R, Salahshoor MR. Evaluating the adoption of evidence-based practice using Rogers’s diffusion of innovation theory: a model testing study. Health Promot Perspect. 2018;8(1):25-32. http://dx.doi.org/10.15171/hpp.2018.03. PMid:29423359.

9. Emani S, Peters E, Desai S, Karson AS, Lipsitz SR, La Rocca R, et al. Perception sofadopters versus non-adoptersof a patient portal: na applicationof diffusion of innovation theory. J Innov Health Inform. 2018;25(3):149-57. http://dx.doi.org/10.14236/jhi.v25i3.991. PMid:30398458.

10. Minas Gerais. Assembléia Legislativa. PMDI plano mineiro de desenvolvimento integrado 2015- 2017: perfis territoriais [Internet]. Belo Horizonte: Governo do Estado de Minas Gerais; 2016 [cited 2019 Oct 10]. Available from: https://www.mg.gov.br/sites/default/files/transicao/governamental/Cat%C3%A1logo%20PMDI%20Volume%201.pdf

11. Minayo MCS. Amostragem e saturação em pesquisa qualitativa: consensos e controvérsias. Rev Pesq Qual [Internet]. 2017 [cited 2019 Oct 10];5(7):1-12. Available from: https://editora.sepq.org.br/rpq/article/view/82/59

12. Bardin L. Análise de conteúdo. 5ª ed. Lisboa: Edições 70; 2016.

13. Gomes PAR, Farah BF, Rocha RS, Friedrich DBC, Dutra HS. Prontuário eletrônico do cidadão: instrumento para o cuidado de enfermagem. Rev Fund Care. 2019;11(5):1226-35. http://dx.doi.org/10.9789/2175-5361.2019.v11i5.1226-1235.

14. Chang F, Paramsothy T, Roche M, Gupta NS. Patient, staff, and clinician perspectives on implementing electronic communications in an interdisciplinary rural family health practice. Prim Health Care Res Dev. 2017;18(2):149-60. http://dx.doi.org/10.1017/S1463423616000414. PMid:27995826.

15. Kruse CS, Mileski M, Vijaykumar AG, Viswanathan SV, Suskandla U, Chidambaram Y. Impact of electronic health records on long-term care facilities: systematic review. JMIR Med Inform. 2017;5(3):e35. http://dx.doi.org/10.2196/medinform.7958. PMid:28963091.

16. Escalda P, Parreira CM SF. Dimensions of interprofessional work and of collaborative practices developed at a primary care unit by a family health team. Interface Commun Heal Educ. 2018;22:1717-27. http://dx.doi.org/10.1590/1807-57622017.0818.

17. Associação Brasileira de Saúde Coletiva (ABRASCO). Grupo Temático Informação em Saúde e População (GTISP). Plano Diretor para o desenvolvimento da Informação e Tecnologia de Informação em Saúde [Internet]. 3ª ed. Brasília: ABRASCO; 2016 [cited 2021 June 17]. Available from: https://www.abrasco.org.br/site/gtinformacoesemsaudeepopulacao/wp-content/uploads/sites/13/2021/04/plad5.pdf

18. Bjelke M, Martinsson AK, Lendahls L, Oscarsson M. Using the internet as a source of information during pregnancy - a descriptive cross-sectional study in sweden. Midwifery. 2016;40:187-91. http://dx.doi.org/10.1016/j.midw.2016.06.020. PMid:27450590.

19. Leider JP, Shah GH, Williams KS, Gupta A, Castrucci BC. Data, Staff, and Money: leader ship reflection son the future of public health informatics. J Public Health Manag Pract. 2017;23(3):302-10. http://dx.doi.org/10.1097/PHH.0000000000000580. PMid:28350627.

20. Mazzucca S, Parks RG, Tabak RG, Allen P, Dobbins M, Stamatakis KA, et al. Assessing organizational supports for evidence-based decision making in local public health departments in the United States: development and psychometric properties of a new measure. J Public Health Manag Pract. 2019;25(5):454-63. http://dx.doi.org/10.1097/PHH.0000000000000952. PMid:31348160.

Submission: 06/10/2021

Approved: 11/04/2021