Introduction

This article explores the growing influence of artificial intelligence (AI) tools, particularly those leveraging large language models (LLMs) [1] on the research and specifically editing landscapes. While AI tools hold immense promise in accelerating scientific discovery and innovation, their adoption raises significant concerns related to ethics and biases. I emphasize the need for an eyes opened, balanced approach to harness the advantages of AI while mitigating associated risks.

Tools powered by AI, in particular those using LLMs, promise to speed up the often time-consuming work of scientific discovery and innovation. Most researchers spend a majority of their work days reading and analyzing vast amounts of data and literature [2]—tasks which take AI applications a fraction of the time. Thus, given the potential time and cost savings, it is no wonder legacy scientific databases are rolling out AI enhancements for search and discovery tools. Formatting datasets and publications eat up an estimated 52 hours per year. AI tools for every use case are being released at lightning speed. However, automated solutions are prone to error, bias, and other ethical concerns. Unattributed use of chatbots in research is on the rise. What are some of the benefits and drawbacks of using these tools in research?

AI Tools for Creating Content as Well as Writing and Editing Articles Are All Over the Place

AI is constantly in the news : “OpenAI is now working on a model so powerful it alarms staff [3], able to even solve basic math problems it’s not seen before! You’ll need to have been hibernating, or in a coma, for the last 12-month to have missed out on all the hype surrounding ChatGPT and how it can help us poor struggling academics. If we don’t use it, we’ll lose time and fall far behind the career curve.”

The utilization of AI in research endeavors has garnered substantial attention due to its potential to streamline time-consuming tasks, such as data analysis and literature review. This paper delves into the implications of incorporating AI tools in research workflows, shedding light on the advancements made, as well as the challenges that researchers currently face.

This section highlights the positive contributions of AI tools to various aspects of the research process. From aiding in writing and presentation tasks to facilitating literature reviews, the advantages of tools like ChatGPT (OpenAI; https://openai.com/chatgpt/), DeepL (DeepL; https://www.deepl.com/en/translator), WritefullX (Writefull; https://x.writefull.com/), SciSpace (PubGenius Inc; https://scispace.com/), Microsoft Copilot (Microsoft Corp; https://copilot.microsoft.com), and others are explored.

And There Are Many More Than Just ChatGPT

A veritable cornucopia of other tools supposed to help us researchers with our English writing, especially non-native speakers. A number of these tools were developed, and are now sold back to researchers, by the large “author services” (hereafter “editing”) companies like Cactus (Editage, https://www.editage.com; PaperPal, https://paperpal.com), Enago (Trinka AI; https://www.trinka.ai), WordVice (WordVice AI; https://wordvice.ai/), and AJE (Curie; https://www.aje.com/curie/). This is a very exciting time for researchers, especially if English is not your native language: AI tools can now be used to reduce—even completely remove—the English language barrier to publication success

“Editing” companies know this well. In recent years we have seen a race to create, own, and operate AI tools specifically targeted at academics, hand-in-hand with a “race to the bottom” to offer “human” editing services at the cheapest possible prices. Companies which used to employ “native speaking PhD-level editors” are now outsourcing this work more and more to regions in the world where workers are available much cheaper (e.g., India, the Philippines—nothing against these countries, not at all, but this fact is never disclosed when you order an editing job). Outsourcing (or “offshoring” as it is known in publishing) is understandable from a business perspective: margins on editing, especially direct to researchers (business to consumer), have always been extremely tight (< 10%). The “AI tools explosion” has therefore created huge opportunities for our friends at the “editing” companies.

Indeed, the marketing messaging is clear: Use OUR TOOL because it’s been TRAINED specifically on academic research papers and is therefore better than generic English correcting software FOR YOUR ARTICLE’S SUCCESS. Here is an example, a message received yesterday from an editing which said, “Don’t use Grammarly! Use our tool to help with your next paper.”

This “unique selling point (USP)” is real, by the way. The differentiator is that the AI tools operated by “editing” companies were actually trained using real research papers. But whose? Answer: articles submitted to them as editing jobs. Your research, potentially. Did you realize that? Well, no, and me neither. It has never been made particularly clear, as we’ll see in a moment.

Fun Fact: “Editing” Companies Have Been Using AI Tools in Their Workflows for Years

In fact, in most cases, the driver for their initial creation was not to help researchers per se, but rather to help speed up and streamline in-house editing processes. So that they could charge you the same to work on a paper but have their editors spend less time and therefore get paid less money, increasing those small margins. Genius.

In the same way, publishers have also been using AI tools for years as part of their workflows: proofreading, typesetting, peer reviewer selection, and general production, for example. Not to help researchers but to help themselves.

In other words, if you have submitted one of your research articles to an “editing” company over the last few years then it is almost certain that your work will have been “edited” (either at the start or at the end of the “human process”) by an AI tool. You would have been told this fact but nevertheless probably remained blissfully unaware of it: notices about your IP (internet protocol) address and the presence of an automated step would have been buried in the fine print within those “terms and conditions” that nobody reads.

Take Home Message

You most likely thought you were paying for “an expert edit” but at least part of the process was in fact carried out by an AI tool, built to increase editing workflow efficiency and learning from your article. Because that is how AI algorithms work: they learn and improve themselves based on real written language (turned into a USP, as mentioned earlier), in this case your research.

Is this fair? Is this ethical? Tools built and trained on researcher writing are now being sold back to those same researchers. Whatever, it is a genius business model.

AI for Research: the Good News

Writing and presentation tasks are often a challenge and time-suck for researchers, where tools like ChatGPT can get a jump on a draft paper or refine an article in development. Automated translation tools like DeepL are proving valuable, especially for non-English speakers looking to publish in leading journals. Summaries of papers for non-specialist audiences are not very widespread, given the already time-intensive task of writing articles; tools like WritefullX, SciSpace Microsoft Copilot, and Semantic Scholar (Ai2; https://www.semanticscholar.org) are looking to make that an easier step in the publishing process.

Faster, more targeted literature or data search is an obvious area to flex AI muscles, as well as analytics tasks like identifying patterns across texts. This is the focus of System, which surfaces relevant content as well as offers background context to deepen understanding. Services like LookUp promise to shorten the runway for data analysts and AI is proving useful to speed up quantitative calculations in some proof-of-concept studies.

AI-facilitated literature reviews, using tools like Elicit (Elicit Research; https://elicit.com), can save researchers a great deal of time, more quickly identifying research gaps across large batches of articles than any human could achieve. Litmaps (Litmap Ltd; https://www.litmaps.com) also offers visual knowledge graphs and some platforms include chatbot assistance, such as Consensus (Consensus NLP; https://consensus.app) and Iris.ai (Iris AI; https://iris.ai).

ChatGPT itself has proven to help scholars with a range of research tasks, from peer review to manuscript formatting. Data visualization can now be achieved with Olli.ai (Realize Inc; https://www.olli.ai). Publishing tasks can be supported by the likes of Grammarly (Grammarly Inc; https://www.grammarly.com) and Turnitin (Turnitin LLC; https://www.turnitin.com). And the AI behind Research Rabbit (Research Rabbit; https://www.researchrabbit.ai) can facilitate research collaboration and networking.

AI Tools Terms and Conditions

Let’s take a moment to look at the “terms and conditions” of ChatGPT to illustrate this point:

Your Content. You may provide input to the Services (‘Input’), and receive output generated and returned by the Services based on the Input (‘Output’). Input and Output are collectively “Content.” As between the parties and to the extent permitted by applicable law, you own all Input, and subject to your compliance with these Terms, OpenAI hereby assigns to you all its right, title and interest in and to Output. OpenAI may use Content as necessary to provide and maintain the Services, comply with applicable law, and enforce our policies. You are responsible for Content, including for ensuring that it does not violate any applicable law or these Terms.

If you carefully examine this definition, you will notice that OpenAI declares that it can use your content as it deems necessary to maintain its services, including complying with applicable laws and enforcing its policies. This is a handy catchall for them. Indeed, in a later portion of the terms, labeled section c, they mention this: “One of the main benefits of machine learning models is that they can be improved over time. To help OpenAI provide and maintain the Services, you agree and instruct that we may use Content to develop and improve the Services.” This is akin to the earlier discussed one-line caution that appears when you log into ChatGPT. Paperpal has a similar “terms and conditions”:

The Content presented on this Site (including but not limited to text, design, software, graphics, audio, video, HTML code, and data) is protected by copyright law, patent law, trademark law, and other applicable intellectual property laws and is the exclusive property of us. You agree to follow all instructions on this Site limiting the way you may use the Content. You affirm that you own the copyrights of the source files you provide us for availing the services or you affirm that you have the necessary authority to provide the source files to us for availing our services and in either case you authorize us to use the source files for rendering the services envisaged by you from us.

It is worth repeating this last part: “[…] you authorize us to use the source files for rendering the services envisaged by you from us.” What is “the service” you “envisage” from this “editing company”? An edited paper, of course. Note that ChatGPT does offer users the option to “opt out” of your work being added to their training model. We will be returning to the key legal concept of “fair use” in a later essay—a number of landmark legal cases are ongoing. Their outcomes will likely exert a huge impact on this “your work in, your work out” AI training model in the future. In short, at the moment, the concept of “fair use” allows these companies to get away with almost anything as long as the “derivative output” is not too similar to an actual article used in the training process.

The fact that most researchers (and others) remain unaware of how AI tools train and learn is illustrated by a recent cautionary tale from our own experience. We were chatting about the issue of IP and training with a marketing leader from a major publisher who expressed surprise that this was actually the case: (paraphrasing) “Shucks, we recently ran a marketing workshop involving the use of AI tools.” We then asked the well-known and very widely used tool in question to give us some notes on that particular publisher’s early career researcher marketing personas. What it spat out was almost word-for-word what the marketing team was using (and had developed based on lots of research). This information had presumably been fed into the tool during the recent workshop. It is noteworthy that for exactly this reason a growing array of pharmaceutical and other IP-dependent companies now forbid their employees from using certain “your work in, your work out” learning-based AI tools.

AI for Research: the Drawbacks

It is not all upside; there are many hazards to navigate when using AI tools in advanced research contexts. Anyone who has tried ChatGPT has encountered its limitations and maybe even “hallucinations,” where AI conjures plausible content or citations that are incorrect. In its current state, the free version of ChatGPT is trained on superficial content, not the type of highly specialized material required for professional use cases. Even when trained for research purposes, AI tools do not produce the same results. Inaccuracies have even been found with scientific tools, like Consensus.

And there are many legal unknowns. The US Congress is looking into regulations, under pressure from the White House, and lawsuits around the world are disputing whether training AI with copyrighted materials should be considered fair use. Debates are raging about whether chatbot outputs should be considered plagiarism—or if publishing chatbot-produced content as your own is legit. As far as scholarly publishing is concerned, the industry came out quickly to affirm that chatbots are not authors—but the jury is still out on how copyright policies apply to generative AI.

Some are concerned about risks to privacy and security, as well as encoded biases that will eventually undermine the credibility and value of research advancements. Similar to the anxieties that arose with the advent of the calculator or automated transmission, some worry that regular use of generative AI will degrade researchers’ critical thinking skills in specialty fields. Could AI become a crutch that will eventually erode overall knowledge generation and scientific breakthroughs?

Conclusion

We are living in interesting AI times for sure. OpenAI is launching more and more GPTs (generative pre-trained transformers), trainable assistants which can be embedded inside your accounts. ChatGPT (Copilot) is already integrated across the entire Microsoft Office suite (Microsoft Corp) and even within Windows 11 (Microsoft Corp), while Claude 2 (Anthropic PBC; https://www.anthropic.com/claude) has become much smarter and was bought by the mega-powerful Amazon. X (formally known as Twitter) is launching Grok, the latest AI from Elon Musk and dozens and dozens of new AI solutions are popping up all the time across various fields. Almost every day we come across cool online solutions for making videos, upscaling images, creating tools from data in minutes, and writing music with only our voices. The dawning AI era is all pretty cool. If only people would learn to get along with each other and treat each other “fairly,” these times would be so great. Come on “editing companies”!

Emphasizing the importance of human oversight, the paper recommends careful review and validation of AI outputs. Additionally, the role of AI in quality assurance is explored, with examples such as AI Reviewer (Logarix; https://www.aireviewer.com/), ImageTwin (ImageTwin AI GmbH; https://imagetwin.ai/), and Penelope.ai (PNLP Ltd; https://www.penelope.ai/). Seeking professional guidance and continuous learning are also highlighted as essential strategies for navigating the evolving landscape of AI in research.

So, how does one mitigate these risks while incorporating the benefits of AI into the research workflow? Here are a few expert tips about how to make the most of these innovations while minimizing the drawbacks.

Keep humans in the loop

All AI outputs should be carefully reviewed and validated, and likely expanded and edited to meet the necessary standards. AI tools must be carefully trained and maintained, which involves human interaction and, therefore, regular opportunities to check for errors or biases. Also, experts advise to expect engaging with AI to be somewhat iterative and, for example, to not give up after one or two attempts at a chatbot prompt, which can be a key mechanism to continually refine and train AI tools.

Use AI for quality assurance

AI can generate content, but it can also validate content. AI Reviewer is a startup offering presubmission checks. We can leverage AI to check for errors, gaps, or manipulated data, for example using ImageTwin and Penelope.ai. Some of these validation tasks are better suited to AI than humans, for example in the speed and volume of memorization and quickly spotting patterns in large datasets. This can free up human time for things AI cannot do well, like effective, situational, and experiential intelligence, common sense, or rationality. But, be sure to heed the advice of experts and do not assume AI tools are perfect.

Seek professional guidance

Just as we continually train AI models, we must all continually learn how to make the most of these new technologies. Look for best practices and training from experts like librarians and information practitioners. Alongside other professional development goals, we can all hone our AI skills, or human intelligence tasks, as these will be key to how research is done in the very near future.

Above all, this article paper underscores the transformative potential of AI in research while acknowledging the imperative of addressing associated challenges. By adopting a proactive and informed approach, researchers can harness the power of AI to enhance productivity and innovation while ensuring the responsible and ethical use of these tools in advancing scientific knowledge.

Notes

Conflict of Interest

No potential conflict of interest relevant to this article was reported.

Funding

The author received no financial support for this work.

Data Availability

Data sharing is not applicable to this article as no new data were created or analyzed.

Disclosure

No AI tools were used, or harmed, during the writing of this article.

Supplementary Materials

References

• 1. IBM. What are large language models (LLMs)? [Internet]. IBM; [cited 2024 Aug 1]. Available from: https://www.ibm.com/topics/large-language-models.
• 2. McGill B. How much time should you spend reading scholarly literature? (a poll). Dynamic Ecology [blog at WordPress.com]; 2016 [cited 2024 Aug 1]. Available from: https://dynamicecology.wordpress.com/2016/01/27/how-muchtime-should-you-spend-reading-scholarly-literature/.
• 3. Milmo D. OpenAI ‘was working on advanced model so powerful it alarmed staff’ [Internet]. Guardian; 2023 [cited 2024 Aug 1]. Available from: https://www.theguardian.com/business/2023/nov/23/openai-was-working-on-advanced-model-so-powerful-it-alarmed-staff.

Figure & Data

References

Citations

Citations to this article as recorded by