Introduction

Background/rationale

A large-scale survey was conducted on open access (OA) diamond journals from June 2020 to February 2021 by a consortium of 10 organizations to facilitate an understanding of the present situation of OA diamond journals throughout the world.

Objectives

This study aimed to identify whether OA diamond journals were compliant with the standards specified in the Plan S technical requirements. Specifically, the following were analyzed for adherence to the industry standards: licenses; peer review; author copyright; unique article identifiers; digital archiving; machine-readable licenses; author identifiers; self-archiving policy; full texts in JATS XML; compliance with OpenAIRE metadata standards; linking to data, code, and other research outputs; standards of the Initiative for Open Citations; and Creative Commons license types.

Methods

Ethics statement

This was a literature database and survey-based study on journal publishing, and no sensitive personal information was included in the survey items. No approval by the Institutional Review Board was required. Participants agreed to participate in the survey voluntarily.

Study design

This was a cross-sectional descriptive study based on a literature database search and survey.

Setting

From mid-June to mid-July 2020, an online survey listing 94 questions collected data on the different components of diamond journals. Information in the Directory of Open Access Journals (DOAJ) was also searched. The structure and questions of the survey are available from the previous article on the landscape of OA diamond journals [1] (Suppl. 1). The analysis was based on data from two datasets: DOAJ metadata, which contained a large amount of information about DOAJ journals; and survey data, in which journals not in DOAJ had given much of the same information present in the DOAJ metadata, as well as some more, and where DOAJ journals had given information not found in the DOAJ metadata. Several comparisons were made between survey journals and DOAJ journals, as well as between OA diamond and article processing charge (APC)-based journals in DOAJ. APC-based journals were not asked to participate in the survey. In instances where the DOAJ metadata contained no relevant cinformation, a comparison was made between survey journals that were also listed in DOAJ and those not listed in DOAJ. The other aspects of the setting were also the same as in the previous article on the landscape of OA diamond journals [1].

Participants

The total number of surveyed journals was 1,619, consisting of 532 journals not listed in DOAJ, and 1,087 journals listed in DOAJ. The number of DOAJ journals analyzed was 14,368. Out of them, 10,449 (72.7%) were OA diamond journals. These data are the same as in the previous article on the journal landscape [1].

Variables

The variables were the items of the survey questionnaire.

Data sources/measurement

The data sources and measurements were the same as in the previous landscape article, including survey, database analysis, and literature review [1]. Raw response data from the survey participants are available from Dataset 1, with a readme text containing the variable list for the survey data file in Dataset 2. Data files for Figs. 1–18 are available in Suppl. 2.

Bias

The potential sources of bias were also the same as in the previous study [1]. Different circumstances and motivations of journals to participate in the survey may have been a source of bias in participation.

Study size

Study size estimation was not done because this was not a randomized experimental study, but rather a study based on voluntary participation.

Statistical methods

Descriptive statistics were applied for the interpretation of the results.

Results

Scientific and editorial quality

Compliance with COPE principles (source: survey Q52)

Plan S requirements specify “a solid system in place for review according to the standards within the relevant discipline and guided by the core practices and policies outlined by the Committee on Publication Ethics (COPE).” COPE represents good standards for review and other editorial practices, and issues guidelines and other resources to help editors. Out of 1,417 surveyed journals, 1,137 (70.2%) reported following the COPE guidelines, whereas 51 journals (3.1%) did not (Fig. 1).

Information on the peer review procedure (source: DOAJ, survey Q26)

In DOAJ, all journals except one indicated that they conducted peer review in a form that meets Plan S requirements. Fig. 2 shows the distribution of the various types of reviews listed by the journals over the two categories of journals (OA diamond and APC-based). Blind and double-blind reviews were the most frequently used types, totaling more than 80% for both journal groups. A comparison of the answers between journals in DOAJ and journals not in DOAJ is presented in Fig. 3. Double-blind peer review was performed by more than 50% of journals in both groups and emerged as the predominant review process by a wide margin. All review processes used by both DOAJ and survey journals that answered this question were Plan S-compliant.

Information on editorial management and submission/rejection (source: DOAJ, survey Q50)

In the survey, Q50 asks, “Does the journal publish annually at least basic statistics, covering in particular: …,” with five options plus an “other” alternative. Nearly half of all journals did not publish any of the statistics offered as an alternative. Still, some of them had some information under “other,” which included various published information, predominantly usage statistics (Fig. 4). Journals in DOAJ selected more than one answer more frequently (39%) than OA diamond journals (23%). Responses of “blank” and “none” were also higher for survey-only journals than for DOAJ journals in the survey.

Technical requirements and recommendations

Persistent identifiers (source: DOAJ, survey Q42)

A persistent identifier (PID) is an identifier that remains constant over time and always points to the resource referred to, irrespective of renaming or moving to new domains or URLs. In DOAJ, only PIDs for articles are listed in the journal metadata. In total, 9,037 out of the 14,368 DOAJ journals (62.9%) offered article PIDs in the form of digital object identifiers (DOIs) (Fig. 5). Among 3,919 APC-based DOAJ journals, 3,335 (85.0%) had DOIs. Uniform resource names (URNs) were used in 14 journals and handles in 65 journals.

The use of PIDs was the theme for Q42 in the survey. A journal could check more than one answer, so the numbers did not add up to the total number of journals surveyed. The DOAJ journals in the survey scored higher for Crossref DOIs, other DOIs, ORCIDs, and grant IDs, while survey-only journals had a higher percentage of Datacite DOIs and other PIDs (Fig. 6). Journals using other DOIs mentioned handles, mEDRA, and Researcher IDs. In total, 960 journals (59.3%) in the survey used Crossref DOIs, while 124 (7.7%) mentioned Datacite DOIs and 400 (24.7%) stated that they used other DOIs (Fig. 6).

Long-term digital preservation or archiving (source: DOAJ, survey Q28)

A total of 3,361 OA diamond journals out of 10,449 (32.1%) appeared to satisfy this requirement, as well as 2,639 of the 3,919 APC-based journals (67.3%) (Fig. 7). An interesting observation is that 6,000 journals reported some form of archiving in place.

In the survey, journals could choose more than one option; hence, the numbers do not equal the total number of journals surveyed. The majority of survey journals had no archiving policies (855 of 1,619 respondents) (Fig. 8). In addition, only 381 respondents used a standard archiving system (LOCKSS, PKP PN, CLOCKSS, and Portico) that would comply with cOAlition S requirements. Local solutions such as national libraries (170 respondents) were frequently quoted.

Machine-readable metadata in CC0 (source: DOAJ)

Most DOAJ journals deposited article-level metadata in DOAJ. However, it is unclear from the data to what extent this is a continuing process for individual journals or a one-off or rare occurrence. It was found that 2,304 out of 10,449 OA diamond journals (78.0%) in DOAJ had deposited one or more article-level records compared to 3,420 out of 3,919 APC-based journals (87.3%) (Fig. 9). This high deposit rate suggests that DOAJ could be the best way to solve this requirement for many OA diamond journals.

Author and grant PIDs

Information about the author and grant PIDs was not available in DOAJ metadata. The subsection “Persistent identifiers (PIDs)” above contains information about the use of ORCID among survey journals. Only 135 journals out of 1,619 surveyed journals (32.4%) used ORCID (Fig. 6).

Self-archiving policy in Sherpa Romeo (source: DOAJ)

Sherpa Romeo is the only self-archiving policy service accepted by Plan S. Some journals used other services. Of DOAJ diamond journals, 1,942 out of 10,449 (18.6%) had a policy to use Sherpa Romeo, compared to 2,041 out of 3,919 APC-based journals (52.1%) (Fig. 10).

Full text in JATS XML (source: DOAJ, survey Q27)

Many journals offered full text in multiple formats. Here, only PDF, XML, and HTML formats were considered. Plan S recommends “a machine-readable community standard format such as JATS XML.”

PDF was the most common format, used by more than 99% of all OA journals. OA diamond journals were slightly less likely to offer this format. Still, 98.9% of such journals offered PDF (Table S1). XML was used by 859 of the 10,449 DOAJ OA diamond journals (8.2%) compared to 815 of the 3,919 APC-based journals (20.8%) (Table S2). A total of 1,674 of the 14,368 DOAJ journals (11.7%) produced full-text XML, and larger journals were more likely to offer full text in XML. HTML is another full-text format that could satisfy the Plan S requirement (Table S3). It was found that 22.9% of DOAJ OA diamond journals used the HTML format compared to 59.7% of APC-based journals. Table S4 shows the numbers of DOAJ journals offering either XML or HTML. Among the DOAJ OA diamond journals, 25.6% offered XML and/or HTML compared to 63.4% of APC-based journals.

The full-text formats by survey journal category are presented in Table S5. The results generally conform to what was found in DOAJ, but there are some differences. PDF was offered by only 78.2% of the survey-only journals, compared to 99.1% of all DOAJ journals. A higher percentage of survey journals provided XML—including both the survey DOAJ journals and survey-only journals—than the DOAJ OA diamond journals (8.2%). The same was the case for HTML, which was offered by 22.9% of DOAJ OA diamond journals. More than 30% of survey journals offered HTML and/or XML, compared to 25% of DOAJ OA diamond journals.

Automatic deposit of JATS XML in an author-designated repository (source: survey Q47)

While the information on author-designated repositories was not available in the DOAJ metadata, the survey data showed that the JATS XML compliance rate was 35.1% (Fig. 11). However, since more than 40% of journals responded either “unknown” or “no answer,” it is difficult to draw exact conclusions on the actual status. DOAJ journals in the survey were more compliant with JATS XML automatic deposit than survey-only journals, but both groups had an “unknown” share of around one-third (Fig. 11).

Compliance with OpenAIRE metadata standards (source: survey Q46)

Although no information was available on compliance with OpenAIRE metadata standards in DOAJ, we found the following in the survey data: the compliance rate was over 40%, but nearly 50% of responses were “unknown” or “no answer.” The fact that only 6.5% of respondents answered “no” can be interpreted as a positive sign, as shown in Fig. 12. Survey journals in DOAJ were more compliant than survey-only journals; the latter group had a higher rate of “unknown” or “blank” answers.

Does the journal require linking to data, code, and other research outputs? (source: survey Q540)

Although no information was available in DOAJ on whether journals required links to data, code, and other research outputs, from the survey data, we found that nearly half of respondents reported not requiring this, versus 24.8% that did. Despite more than 25% of answers being “unknown” or “no,” this points to a low level of compliance. DOAJ journals in the survey were slightly more compliant than survey-only journals (Fig. 13).

Does the journal provide openly accessible data on citations according to the standards of the Initiative for Open Citations? (source: survey Q55)

No information was available in DOAJ on whether journals provided openly accessible data on citations. Fewer than 25% of journals in the survey did indeed provide such citations, indicating a low level of compliance (Table S6). DOAJ journals in the survey were somewhat more compliant than survey-only journals.

Copyright and licensing

Is the license made visible or embedded in the article? (source: DOAJ, survey Q19)

DOAJ asks journals whether a machine-readable CC license is embedded or displayed in articles. Assuming that all journals that have answered “yes” are indeed compliant with the Plan S requirement, 43.4% of DOAJ OA diamond journals were found to be compliant compared to 73.6% of APC-based ones (Fig. 14). A more detailed analysis showed that compliant journals were, on average, larger than noncompliant ones. Hence, 49% of articles in OA diamond journals were published in compliant journals, versus 86.4% of APC-based articles.

In the survey, 793 of 1,619 journals (49.0%) stated that they embedded or displayed licenses in the article. Survey-only journals were more compliant (53.9%) than DOAJ journals (46.6%) in the survey, and DOAJ journals in the survey were also more compliant than OA diamond journals in DOAJ (43.4%) (Fig. 15).

To what extent do OA journals allow reuse and remixing of content, and which CC licenses do they use? (source: DOAJ, survey Q20, Q59)

DOAJ asks journals to list their most restrictive license. We know, though, that some journals allow a number of licenses to be chosen. Although certain journals limit the choices, some alternatives are allowed for authors to choose if mandated by funders. DOAJ is working on allowing journals to list a number of licenses for the author to choose. The listing of the most restrictive license in DOAJ metadata makes it likely that the compliance rate is higher than seen in Fig. 16. We assume that cOAlition S is satisfied if a Plan S-compliant license is available to the author, without all content in the journal being compliant.

Among DOAJ OA diamond journals, 44.2% satisfied the Plan S requirement (CC BY, CC BY-SA, or CC0), while 57.1% of APC-based journals complied. CC BY was the most widely used license; it was used by more than half of APC-based journals and 37.4% of DOAJ OA diamond journals. Some journals listing a restrictive license may also offer a compliant license, but DOAJ asks journals to list only the most restrictive license, which is often the least Plan S-compliant option (Fig. 16).

The NC clause is a significant problem for compliance. CC BY-NC and CC BY-NC-SA licenses, where the NC clause is the reason for the license being noncompliant, were applied by 27.8% of DOAJ OA diamond journals and 26.8% of APC-based journals. If OA diamond journals chose not to use the NC clause, 72.1% of DOAJ OA diamond journals and 80.9% of APC-based journals would be compliant. Furthermore, 23.6% of DOAJ OA diamond journals and 17.3% of APC-based journals used the CC BY-NC-ND license, where both the NC and the ND clauses represent a problem for compliance. The CC BY-ND license (which can be accepted as an individual exception) was used by only 1.4% of all OA diamond journals and hardly any APC-based journals. In the complete survey data, we found that 1,350 of 1,619 journals (83.4%) reported allowing reuse in accordance with a CC license or a license with a similar condition (Table S7).

It should be noted that, in the survey, unlike in DOAJ, journals could list more than one license. Hence, the 1,350 journals listed 1,363 responses to this question, and the sum of percentages reflect this. Of the 1,350 respondents that replied “yes” to Q20, 48 did not provide information about their license. CC BY was the most widely used among survey journals, with CC BY-NC-ND in second place. Nearly 50% of these journals were compliant with the Plan S requirements.

To what extent is copyright retention without restrictions allowed, and if not, what plans to introduce this? (source: DOAJ, survey Q22)

Among the DOAJ OA diamond journals, 48.7% stated that authors hold the copyright without restrictions compared to 53.0% of APC-based DOAJ journals (Fig. 17). OA diamond journals were slightly less compliant (48.7%) than APC-based journals (53.0%).

In the survey, Q22 asked whether the journal allows authors to retain copyright without restrictions. DOAJ journals in the survey allowed author copyright retention to a somewhat larger extent than survey-only journals; DOAJ journals had a compliance rate of 57.2% compared to 55.3% for survey-only journals (Fig. 18). Unlike the previously discussed technical questions, this policy question had few “unknown” or “blank” answers. A majority of responses were positive, meaning that the journal conformed to Plan S requirements.

Those who did not answer “yes” to the above question were asked in Q23 to indicate whether they intended to allow authors to retain copyright in the future. The responses indicated that not many journals plan to change their policies to align better with Plan S requirements (Table S8).

Discussion

Key results

This study investigated how well journals in DOAJ and the survey conformed to Plan S requirements and recommendations. Six requirements were confirmed based on the DOAJ metadata, which included license, peer review, author copyright, article permanent ID, permanent preservation, and machine-readable license. Peer review was the one requirement that all but one journal satisfied. Permanent preservation was the requirement with the lowest compliance among journals, at 28.9%, and only 19.1% for OA diamond journals (Table S9). APC-based journals met more requirements than OA diamond journals. The journals that met few criteria were predominated by OA diamond journals, while APC-based journals predominated among the journals that satisfied all requirements.

Interpretation

The most striking difference in peer review requirements is that double-blind peer review was more commonly used by OA diamond journals, while APC-based journals more commonly used blind peer review (Fig. 2). However, this phenomenon might have been more a matter of semantics than reality, as labels for authors’ and reviewers’ anonymization processes vary over time [2]. For small journals, the expense of DOI deposits to Crossref is a problem (Fig. 6). The DOI deposition cost to Crossref is USD 1 per article; however, the annual membership fee to Crossref is USD 275 per year, which may be burdensome to small societies or institutions. The publisher pays the annual fee, so that many journals could be covered by a single annual fee if appropriately organized. One method is for an organization, such as a publishing company or an editor’s organization, to sponsor publishers of small journals.

As for digital archiving, the Plan S requirement for content archiving is unclear as to what services conform to the requirement (Fig. 8). Journals need guidance on what is meant more specifically by archiving in this context, what possibilities exist, and how journals can use archives at low or no cost. Some groups of journals might need financial support to find a working solution to the archiving requirement.

DOAJ metadata do not provide information on whether the journal makes article-level metadata available under a CC0 license (Fig. 9). However, if a journal deposits article-level metadata with DOAJ, the metadata are made available under a CC0 license in various ways, including API, OAI-PMH, and a full data dump of all journal metadata. Therefore, journals depositing article-level metadata with DOAJ fulfill the article metadata requirement. cOAlition S requires these metadata to include funding information, but such information is not yet generally available in DOAJ.

A more detailed analysis indicated that larger journals tended to offer XML or HTML to a more considerable extent than smaller ones. Even among APC-based journals with income that can be used to pay for XML or secure in-house competence, XML is only offered by a fifth of them. This means that full-text JATS XML is still not a concern for many OA journals.

The low adherence of OA diamond journals to industry standards, especially to the Plan S technical requirements, might originate from the publishers’ or editors’ lack of knowledge and skills in journal publishing. Out of the items analyzed, full-text JATS XML file production and DOI deposit require payment. Archiving also requires a fee if the journal is archived in Portico, LOCKSS, or CLOCKSS. Archiving in a country’s national library usually requires no cost if the library has an archiving policy. All other items can be easily adopted if the publisher or editor understands the above requirements. Other descriptions or policies can be easily achieved by the editor or publisher through prompt action. It is necessary to inform OA diamond journal publishers about the relevant knowledge and technologies in light of their incomplete adherence to industry standards (Table S9).

Comparison with previous studies

The literature has shown that so-called international standards (COPE, International Committee of Medical Journal Editors [ICMJE]) were far from universally practiced. For example, “top-ranked” or Web of Science (WoS)-endorsed journals, even when they formally declared that they followed standards, had various authorship policies [3,4], as well as distinct duplicate and salami-slicing policies—or even no policies at all [5]. However, those three studies were not large-scale investigations. There is no previous study directly comparable to the present study.

Limitations

There may have been sampling bias as a limitation of voluntary participation.

Generalizability

These large-scale survey results may be able to provide information on the current status of adherence to the Plan S requirements of OA diamond journals throughout the world.

Conclusion

It was found how well journals in DOAJ and in the survey conformed to Plan S requirements and recommendations. In general, smaller journals scored lower on these criteria than larger ones, and OA diamond had poorer results than APC-based journals. Structurally, smaller journals are more likely to be OA diamond, so these results imply that the same factors may manifest themselves in various ways. Size relates to the likelihood and operational need for gaining competence—that is, the larger the journal, the larger the need for competence, and the better the chances of achieving competence. APCs enable the journal to pay costs and buy competence, either by outsourcing functions or by hiring persons in the organization. This situation does not mean that APCs are the solution, but it indicates that funding beyond in-kind contributions must be considered vital to ensure strong and healthy OA diamond journals. It also points to a need for journal owners of all kinds to organize journals so that resources are pooled, and competence is built up collectively among many journals.

Notes

Conflict of Interest

No potential conflict of interest relevant to this article was reported.

Funding

The original work was supported by Science Europe and cOAlition S (2020–2021).

Data Availability

This dataset contains data used by and partly generated by the OA Diamond Journals Study on open access journals that do not charge authors. It contains the data files themselves, as well as some readme texts with variable lists available from https://doi.org/10.5281/zenodo.4553103. All data are available for reuse under a CC0 license. Additionally, an online version of the survey results (excluding DOAJ data and excluding free text answers) is available from SurveyMonkey https://ko.surveymonkey.com/results/SM-5RY8WNNP7/.

Dataset files are available from https://doi.org/10.7910/DVN/HVGNGY

Dataset 1. Raw response data from the survey participants of 94 questions for open access diamond journals’ status from June 2020 to February 2021 without identifying information and without free-text answers (CSV). This dataset includes, for some questions, data from DOAJ for journals present in that database.

kcse-295-dataset-1.csv

Dataset 2. Readme text with the variable list for the survey data file (TXT).

kcse-295-dataset-2.txt

Acknowledgements

OPERA, Open Access Scholarly Publishing Association (OASPA), SPARC Europe, Directory of Open Access Journals (DOAJ), Utrecht University Library, Redalyc/AmeliCA, UiT The Arctic University of Norway, and Ligue des Bibliothèques Européennes de Recherche-Association of European Research Libraries (LIBER) supported this research by providing personnel, databases, or systems.

Supplementary Material

Fig. 1.

Compliance with Committee on Publication Ethics (COPE) principles. DOAJ, Directory of Open Access Journals.

Fig. 2.

Review types used by journal groups in Directory of Open Access Journals (DOAJ). OA, open access; APC, article processing charge.

Fig. 3.

Review types used by survey journals organized by those in Directory of Open Access Journals (DOAJ) and those not in DOAJ.

Fig. 4.

Basic statistics published on editorial management related to submission and rejection. DOAJ, Directory of Open Access Journals.

Fig. 5.

Use of article identifiers by journal category in Directory of Open Access Journals (DOAJ). DOI, digital object identifier; URN, uniform resource name; APC, article processing charge; OA, open access.

Fig. 6.

Use of article identifiers by journal category in the survey. DOAJ, Directory of Open Access Journals; DOI, digital object identifier.

Fig. 7.

Archiving in place by journal category in Directory of Open Access Journals (DOAJ). OA, open access; APC, article processing charge.

Fig. 8.

Archiving solution by journal category in the survey. PMC, PubMed Central.

Fig. 9.

Article-level metadata deposition in Directory of Open Access Journals (DOAJ) by journal category. OA, open access; APC, article processing charge.

Fig. 10.

Self-archiving policy in Sherpa Romeo by journal category. OA, open access; APC, article processing charge.

Fig. 11.

JATS XML automatic deposit by journal type in the survey. DOAJ, Directory of Open Access Journals.

Fig. 12.

OpenAIRE metadata standards compliance by survey journal category. DOAJ, Directory of Open Access Journals.

Fig. 13.

Journal requirements on linking to data by survey journal category. DOAJ, Directory of Open Access Journals.

Fig. 14.

Embedded licenses by journal category in Directory of Open Access Journals (DOAJ). OA, open access; APC, article processing charge.

Fig. 15.

Embedded licenses by survey journal category. DOAJ, Directory of Open Access Journals.

Fig. 16.

License types by journal category in Directory of Open Access Journals (DOAJ). APC, article processing charge; OA, open access.

Fig. 17.

Author copyright retention policy by journal category in Directory of Open Access Journals (DOAJ). OA, open access; APC, article processing charge.

Fig. 18.

Survey journals that allow authors to retain copyright without restrictions by journal group. DOAJ, Directory of Open Access Journals.

References

• 1. Korean Council of Science Editors. Charting variety, scope, and impact of open access diamond journals in various disciplines and regions: a survey-based observational study. Sci Ed 2022;9:120-35.https://doi.org/10.6087/kcse.277. Article
• 2. Pontile D, Torny D. From manuscript evaluation to article valuation: the changing technologies of journal peer review. Hum Stud 2015;38:57-79.https://doi.org/10.1007/s10746-014-9335-z. Article
• 3. Bosch X, Pericas JM, Hernández C, Torrents A. A comparison of authorship policies at top-ranked peer-reviewed biomedical journals. Arch Intern Med 2012;172:70-2.https://doi.org/10.1001/archinternmed.2011.600. ArticlePubMed
• 4. Bošnjak L, Marušić A. Prescribed practices of authorship: review of codes of ethics from professional bodies and journal guidelines across disciplines. Scientometrics 2012;93:751-63.https://doi.org/10.1007/s11192-012-0773-y. Article
• 5. Ding D, Nguyen B, Gebel K, Bauman A, Bero L. Duplicate and salami publication: a prevalence study of journal policies. Int J Epidemiol 2020;49:281-8.https://doi.org/10.1093/ije/dyz187. ArticlePubMed

Figure & Data

References

Citations

Citations to this article as recorded by