Article front-matter

Does the Open Access Business Model Have a Significant Impact on the Citation of Publications? Case Study in the Field of Civil Engineering

1 University of Ljubljana, Faculty of Civil and Geodetic Engineering – Library, Jamova c. 2, SI-1000 Ljubljana (Slovenia)

2 University of Ljubljana, Faculty of Civil and Geodetic Engineering, Jamova c. 2, SI-1000 Ljubljana (Slovenia)

3 Faculty of Arts, Department of Library and Information Science and Book Science, Aškerčeva c. 2, SI-1000 Ljubljana (Slovenia)


Abstract: One of the possible benefits of open access (OA) might be the better visibility of articles, which is usually measured by the number of citations of the article. In order to realistically estimate the effect of OA on citation, it is not enough to compare OA and non-OA ISI journals. Thus, as Harnad and Brody (2004) suggested, the numbers of citations of OA and non-OA articles from the same journals were compared. Therefore, we have chosen to analyze the publications in three international journals in the field of civil engineering. All of them have an ISI impact factor in the Civil engineering subject category in the ISI/Web of science database (WOS). The articles were classified into two groups − the OA publications and the non-OA publications. We analyzed all the articles published in the same year and the number of their citations until the end of February 2012, seeking to find out if these two groups differ from each other.

Keywords: Open Access; impact factor; citations; civil engineering; articles; databases; funding; libraries; scientists; WoS; Google


Article body

Open Access (OA) was developed as a reaction to the increasing prices of scholarly and scientific journals. This was first emphasized by librarians, as libraries were the first to be hit by this development. The movement evolved when scientists themselves started to doubt the existing economic model of journals as the main means of scientific communication and information. The most important boost of OA was when a number of funding bodies, research councils and governmental bodies began transforming their view on the publication of the research they were financing. The most well-known example is a 2008 mandate of the US National Institutes of Health, which requires researchers in receipt of federal funding to deposit their final, peer-reviewed manuscripts in PubMed Central (NIH Public Access Policy Details, 2008, 2009).

There are two types of OA: the “golden” road (to publish your article in an OA journal) and the “green” road (to publish your article in a non-OA journal, but to also self-archive it in an OA archive). According to a 2008 research (Harnad et al., 2008), about 10% of journals were gold, but over 90% were already green (i.e. they have given the authors the right to self-archive); however, only a fraction of the articles have been self-archived. Most of the traditional publishers have started to offer a special “open access” option, which involves paying a publication fee for the right to archive the material online. This fee, which is actually an additional one, as subscription is paid as well, can be quite substantial – as far as 3000 € per article. Publication fees or article-processing charges (APC) have become the predominant means for funding professional OA journals and recent findings are suggesting that OA publishing funded through APC is likely to increase furthermore (Solomon and Björk, 2012).

The main point − as journals are a part of the scientific information and communication system − is how a typical scientist seeks information. Researchers submit their research results in the form of publication after peer-review, with the main idea that other researchers can use and apply their findings. Other researchers must first find the published articles of other researchers and be able to access them. The second step is to find these research results useful, which is exhibited by using and citing them.

Typically, a scientist will begin by using a bibliographic tool, e.g. the Web of Science. In Slovenia, the Slovenian Research Agency evaluates scientific work through publications and citations included in WOS. In Slovenia, where WOS was bought by the Slovenian Research Agency in the early nineties of the last century, all members of Slovenian universities can easily access the WOS database and look up the publications and their citations. In other countries, this may often not be possible, since their libraries might not have subscriptions (Guédon, 2004; Guédon, 2008). Libraries simply cannot afford to pay a subscription for all valuable information resources, such as journals. The principal two arguments in support of open access publishing rest on the belief that the subscription-based publishing model has produced a crisis of accessibility to scientific literature and that research article access is a problem (Harnad et al., 2008). Availability of the article is not a sufficient condition for citation, but a necessary one. OA increases the number of potential users of any given article. The researchers who would otherwise have been unable to access articles because their institution could not afford the access-tolls of the journal can easily use OA articles. Therefore, OA can only increase both usage and impact.

There are different opinions about the extent of the problem. A recent review article (Davis and Walters, 2011) did not support the notion that there is a problem, as it stated that recent studies provide little evidence to support the idea that there is a crisis in access to scientific literature, at least for scientists themselves. Scientists do not perceive access to scientific literature as an especially important problem (Davis and Walters, 2011). Access to scientific literature is not a serious concern for most scientists in developed nations, and they actually feel that their access to the literature is steadily improving. A survey on information-seeking behavior of Slovenian researchers has confirmed that OA has obviously not gained a lot of scientists' attention. One of the reasons might also be the traditionally well-organized access to academic journals in Slovenia (Vilar, Juznic, Bartol, 2012).

Surveys are one way to estimate the effect of OA on the impact of the article but may be biased due to subjective views. This is the main reason why OA impact on the availability of information sources for researchers was also a topic which was researched by bibliometrical methods, namely citation analysis.

The citation analysis is based on the information seeking behavior of the researchers as they cite their information resources. Citations identify earlier research results which have been used by the authors in the process of performing and representing their own research. A citation can also be understood as the manifestation of scientific influence. As stated by Moed (2005), “Outcomes of citation analysis must be valued in terms of a qualitative, evaluative framework that takes into account the substantive contents of the works under evaluation”.

Open accessibility is also a very important political issue, since it can improve the awareness of researchers and their funding bodies for OA. The published results tend to differ, although most find positive correlations between OA and the impact/citation advantage. The comparison of the citation counts of individual OA and non-OA articles published in the same non-OA journals was suggested as the correct method (Harnad and Brody, 2004). The authors claimed that the advantage of OA was very obvious in fields such as computer science and physics, and suggested that self-archiving increases citations by 50% or more (Harnad, 2006). Many authors have reported that online availability considerably increases a paper’s impact (Harnad and Brody, 2004; Antelman, 2004; Eysenbach, 2006). There are also some analysis which do not confirm the link betwen OA and the increasing number of citations (Craig et al., 2007; Norris et al., 2008; Miguel et al., 2011). Some previous studies have indicated that self-archiving (Gargouri et al., 2010) and open accessibility (Lin, 2007; Lin, 2009) substantially increase citation impact, but the citation advantage effect of OA publications varies between disciplines (Norris et al., 2008). The conclusions of Moed (2007) for articles in solid state physics are the same as Kurtz’s (Kurtz et al., 2005) for articles in astronomy. The quality effect (authors post their most quality articles as openly accessible) and an early view effect (preprints are visible earlier and are cited more often) impacts the number of citations, and therefore the OA articles receive more citations than non-OA articles. Eysenbach (2006) published the results of an analysis of articles in a hybrid OA journal, the Proceedings of the National Academy of Sciences (PNAS). While OA status was found to remain a significant predictor, there were also other factors − among these were the number of authors of the paper and funding from competitive grants. One very important conclusion of this study is the appeal to funders. They might provide financial support to authors for paying the publishers for immediate OA to their publication.

In our study on the influence of open-access on articles’ citations we relied on objective bibliometric evidence. We compared OA articles with non-OA articles in the same journal and hypothesized that OA would have a positive impact on the number of citations. We analyzed articles in three research journals published in 2007 in the field of civil engineering. These journals were chosen on the basis of two criteria. Firstly, the journals in which the researchers from the University of Ljubljana, Faculty of Civil and Geodetic Engineering (UL FGG) have published were chosen. Secondly, all three journals have an impact factor (IF) and are categorized in the same subject category “engineering, civil”. This subject category included 88 journals in 2007. Their IFs were different, but generally ranked higher than average in this category. Two of them were in the second quarter (Automation in Construction (hereinafter AutCon), IF 2007 = 0.61 and The Journal of Computing in Civil Engineering (hereinafter JCCE), IF 2007 = 0.71), and one (Computers & Structures (hereinafter Com&S), IF 2007 = 0.93) in the first quarter. In a previous study (Koler-Povh et al., 2011), it was found that researchers of UL FGG published most of their scientific articles (72%) in journals that are ranked in the first or second quarter. Two databases were used as a source of citation data: Google Scholar (GS), which is available to common users and allows them the open or free access to full texts of journal articles without the condition of toll-access, and the Web of Science (WOS), as a world known system for bibliometric evaluation of scientific work that is relevant in Slovenia too. By choosing GS instead of Google, we assumed that scholarly information is sought by users who typically use the specialized GS database rather than Google. As Jacso (2005) found, the GS database includes many scientific publications, like thesis, research papers, reports of scientific projects etc., which are not included in the Google search engine, which includes many commercial publications that are not useful for scientific work.Comparing the databases of WOS, Scopus and GS, Bar-Ilan (2010) found that GS gives the highest number of citations. However, the fact that GS often includes repeated citations of the same article should not be neglected, i.e. in bilingual articles/journals, or articles with the same title published in conference procceedings and in the journal, or the publishing of the summary and the publishing of full text at the same time. When counting journal citation only, WOS gives the most relevant results.

We studied articles published in 2007 in the three journals which are present in both systems, with an emphasis on the citation of each of them with citation data until February 29th 2012.

Firstly, the GS database was used. The search was conducted outside the domain of the University of Ljubljana, as we emulated a user who uses the GS tool for Internet access to scholarly information and has no access to the journals subscribed to by the libraries of the University of Ljubljana. We collected data in the period between February 26th and 28th 2012. The data were sorted by journal and categorized into two groups. The first group included OA articles, whereas the second comprised non-OA articles. For each article we collected the number of cumulative citations until the end of February 2012. The data obtained were compared to those obtained with the same analysis in WOS. The data for each journal are presented in Table 1.

No. of
OA articles
Share
%
No. of
non-OA articles
Share
%
Total Share
%
AutCon 30 34 59 66 89 100
JCCE 11 23 37 77 48 100
Com&S 40 27 109 73 149 100
Table 1. The number of articles by journals and by the open access business model

The share of OA in the three journals, published in 2007, was substantial. The lowest share was 23% in the JCCE-ASCE, and the highest was in the AutCon (34%). Mainly, the articles are made OA mostly via different institutional repositories, and only a few of them are openly accessible through the author’s websites. The articles are usually in the form of a peer-reviewed author's final version in pdf format. Rarely are they identical to the printed version of the publisher.

Does the Open Access Business Model Have a Significant Impact on the Citation of Publications? Case Study in the Field of Civil Engineering
Figure 1. The arithmetic mean of citations and 95% confidence intervals
Download as Power Point slide

As shown in Figure 1, open-access articles received more citations than those which were not openly accessible. However, the t-test reveals that this applies to the Com&S journal only (the significance level is α=5%). This is the journal with the highest IF ranked into the first quarter. In the case of JCCE, open access significantly (with the significance level being 5 percent or less) affects the number of citations only in the GS database. The effect of OA on the citation number is not significant for AutCon in neither the GS nor the WOS database.

In order to conclude if the business model of OA has a significant effect on the citation number, we will perform an analysis with a larger sample. Certainly, the large variability of data influenced the results. Namely, the numbers of citations are very different for either OA or non-OA articles in the same journal. The data are given in Table 2.

Interestingly, the most cited article is a non-OA article (from Perez and Behdinan, published in Com&S). This article gets the maximum of citations in both information systems, GS and WOS. That might support other findings that, indeed, the quality of the article is the most important factor which influences the authors' decision to deposit the articles of highest quality into repositories. This is also observed by Kurtz et al. (2005), Davis and Fromerth (2007), Craig et al. (2007) and Gargouri et al. (2010).

OA non-OA
AutCon GS max 41 65
min 1 1
median 13 10
WOS max 23 40
min 0 0
median 5 4
JCCE GS max 35 22
min 3 0
median 16 3
WOS max 15 11
min 0 0
median 8 3
Com&S GS max 47 103
min 1 0
median 13 8
WOS max 37 64
min 1 0
median 10 5
Table 2. Number of citations

The analysis of articles’ citations was the main scope of the presented paper. The articles published in 2007 in three civil engineering non-OA journals were considered. Google Scholar and Web of Science databases were utilized. It was found that OA significantly influenced the citation counts for the articles published in the Computers & Structures journal, which is ranked in the first quarter – according to both databases. Only the GS database showed a significant effect of OA on citations for the articles published in the Journal of Computing in Civil Engineering. Neither GS nor WOS databases indicate a significant effect of OA on the citation counts of articles in the Automation in Construction journal. These two journals are ranked in the second quarter among 88 journals in the same subject category, civil engineering. The present results indicate that more research is needed to give a final answer to the principle question of the paper: does open access have a significant impact on citations in the field of civil engineering. Some other potentially influential factors will be tested as well.


Article back-matter

1. Antelman, K. (2004). Do open-access articles have a greater citation impact? College & Research Libraries, 65(5), 372-382. Retrieved from [Full Text].

2. Bar-Ilan, J. (2010). Citations to the “Introduction to informetrics” indexed by WOS, Scopus and Google Scholar. Scientometrics, 82, 495-506. [CrossRef].

3. Craig, I. D., Plume, A. M., Mcveigh, M. E., Pringle, M., & Amin, M. (2007). Do Open Access Articles Have Greater Citation Impact? Journal of Informetrics, 1, 239-248. Retrieved from [Full Text].

4. Davis, P. M., & Fromerth, M. J. (2007). Does the arXiv lead to higher citations and reduced publisher downloads for mathematics articles? Scientometrics, 71(2), 203-215. Retrieved from [Full Text].

5. Davis, P. M., & Walters, W. H. (2011). The impact of free access to the scientific literature: A review of recent research. Journal of the Medical Library Association, 99(3), 208-217. [CrossRef].

6. Eysenbach, G. (2006). Citation Advantage of Open Access Articles. PLoS Biology, 4(5), e157. [CrossRef].

7. Gargouri, Y., Hajjem, C., Larivière, V, Gingras, Y, Carr, L., Brody, T., & Harnad, S. (2010). Self-Selected or Mandated, Open Access Increases Citation Impact for Higher Quality Research. PLoS One, 5(10), e13636. [CrossRef].

8. Guédon, J. (2004). The "Green" and "Gold" Roads to Open Access: The Case for Mixing and Matching. Serials Review, 30(4), 315-328. [CrossRef].

9. Guédon, J. (2008). Mixing and Matching the Green and Gold Roads to Open Access:Take 2. Serials Review, 34(1), 41-51. [CrossRef].

10. Harnad, S., & Brody, T. (2004). Comparing the impact of open access (OA) vs. non-OA articles in the same journals. D-Lib Magazine, 10(6), 1-6. Retrieved from [Full Text], 2012 Feb 29.

11. Harnad, S. (2006). Publish or perish- self-archive to flourish: The green route to open access. ERCIM News [64]. Retrieved from [Full Text], 2012 Feb 29.

12. Harnad, S., Brody, T., Vallières, F., Carr, L., Hitchcock, S., Gingras, Y., et al. (2008). The Access/Impact Problem and the Green and Gold Roads to Open Access: An Update. Serials Review, 34(1), 36-40. [CrossRef].

13. Jacso, P. (2005). As we may search: Comparison of major features of Web of Science, Scopus and Google Scholar citation-based and citation-enhanced databases. Current Science, 89(9), 1537-1547. Retrieved from [Full Text].

14. Koler-Povh, T., Južnić, P., Turk, Ž., & Turk, G. (2011). Analysis of scientific publications in civil and geodetic engineering in Slovenia, in the case of the Faculty of civil and geodetic engineering in University of Ljubljana. Geodetski vestnik, 55(4), 764-779. Retrieved from [Full Text], 2012 Jan 7.

15. Kurtz, M. J., Eichhorn, G., Accomazzi, A., Grant, C., Demleitner, M., Henneken, E., & Murray, S. S. (2005). The effect of use and access on citations. Information Processing & Management, 41(6), 1395-1402. [CrossRef].

16. Lin, S. (2007). Non-Open Access and Its Adverse Impact on Molecules. Molecules, 12(7), 1436-1437. [CrossRef].

17. Lin, S. (2009). Full Open Access Journals Have Increased Impact Factors. Molecules, 14(6), 2254-2255. [CrossRef].

18. Miguel, S., Chinchilla-Rodriguez, Z., & De, M.F. (2011). Open access and Scopus: A new approach to scientific visibility from the standpoint of access. Journal of the American Society for Information Science and Technology, 62(6), 1130-1145. [CrossRef].

19. Moed, H. F. (2007). The effect of “open access” on citation impact: An analysis of ArXiv’s condensed matter section. Journal of the American Society for Information Science and Technology, 58(13), 2047-2054. [CrossRef].

20. Moed, H.F. (2005). Citation analysis in Research Evaluation. In: Information Science and Knowledge Management, 9, 346. Retrieved from [Full Text].

21. Norris, M., Oppenheim, C., & Rowland, F. (2008). The citation advantage of open-access articles. Journal of the American Society for Information Science and Technology, 59(12), 1963-1972. [CrossRef].

22. NIH Public Access Policy Details. Retrieved from [Full Text], 2012 Feb 29.

23. Solomon, D. J., & Björk, B. C. (2012). Publication fees in open access publishing: Sources of funding and factors influencing choice of journal. Journal of the American Society for Information Science and Technology, 63(1), 98-107. [CrossRef].

24. Vilar, P., Juznic, P., & Bartol, T. (2012). Information-Seeking Behaviour of Slovenian Researchers: Implications for information services. The Grey Journal, International Journal on Grey Literature, 8(1), 43-53. Retrieved from [Full Text].

doi:10.5937/BIOAC-68

About the author

How to cite item

Indexing metadata

Finding Citations

View reader comments (0)

Download article