From Keyword Matching to Semantic Understanding: Technical Evolution and Methodological Boundaries of Academic Search

Abstract

Academic search underpins literature investigation, review writing, and research topic selection. Many researchers reduce academic search to simply entering keywords to retrieve papers. Yet in real‑world research workflows, the core retrieval challenge lies not in locating papers with known titles, but in identifying genuinely relevant outputs from massive scholarly records that address a given research question. Scientific concepts are expressed in diverse ways; disciplinary terminology systems diverge across fields; and researchers’ information needs shift dynamically with tasks. This paper systematically traces the evolutionary trajectory of academic retrieval technologies, analyzes the technical principles, applicable scenarios, and inherent limitations of keyword‑based retrieval, vector‑driven semantic retrieval, and hybrid retrieval. It further discusses the value and boundaries of citation networks and knowledge graphs as relevance evidence beyond plain text, maps retrieval strategies to distinct research tasks, and examines new shifts brought by generative AI to academic search. It offers methodological guidance for researchers to select appropriate retrieval tools and mitigate retrieval biases.

Keywords: academic search; keyword retrieval; semantic retrieval; hybrid retrieval; knowledge graph; literature investigation; information retrieval

I. Core Tensions in Academic Retrieval: More Than “Finding Known Documents”

1.1 Two Distinct Retrieval Tasks

When researchers possess unambiguous metadata such as full paper titles, authors, DOIs, or journal volume‑issue numbers, locating target literature is straightforward via metadata‑driven precise lookup. The real practical difficulty lies in topical retrieval: given a research question, the retrieval system must filter massive document corpora to surface papers that meaningfully respond to that question. This is where major literature‑investigation tools face their central challenge.

For instance, a query on “extreme climate and food security” returns numerous papers containing the terms “climate”, “food”, and “agriculture”. Many of these matches are merely lexical overlaps and do not actually address the underlying scientific question. One concept may have multiple alternative formulations: some studies use “climate shocks” instead of “extreme climate”, or “crop yield stability” in place of “food security”. Terminology systems also differ substantially across disciplines.

1.2 Two Fundamental Contradictions in Academic Retrieval

The first contradiction: researchers submit limited search terms, while dozens of expressions may describe the same scientific problem within a research domain. Information retrieval literature terms this lexical mismatch: queries and papers share thematic proximity but diverge in surface wording, which readily causes omission of relevant literature.

The second contradiction: lexical overlap does not guarantee thematic alignment. Many terms carry multi‑disciplinary polysemy. Take “network” as an example: it may refer to social networks, neural networks, computer networks, or ecological networks. Accordingly, academic retrieval is not merely checking whether a document contains certain words, but identifying the specific scientific concept a term denotes within its scholarly context.

II. Technical Logic and Practical Boundaries of Keyword Retrieval

2.1 Operating Principles of the BM25 Probabilistic Relevance Model

Against the rise of semantic retrieval, keyword retrieval is sometimes dismissed as outdated. Nevertheless, it remains a foundational pillar of academic search. The representative BM25 probabilistic relevance model computes relevance scores between queries and documents by integrating multiple dimensions: term occurrence frequency, term discriminative power across the whole corpus, and document length, rather than relying purely on raw term counts.

BM25 saturates term‑frequency weights so that documents laden with repeated jargon cannot achieve artificially inflated relevance scores. High‑frequency generic words such as “study” or “analysis” are down‑weighted, whereas rare specialized terminology receives higher discriminative weight.

2.2 Applicable Scenarios and Inherent Shortcomings of Keyword Retrieval

Keyword retrieval excels when information requirements are well‑defined and can be articulated using standardized domain terminology. It works well for retrieving specific compounds, genes, algorithms, disease names, experimental standards, particular authors, paper titles, and exact phrases. In these cases, strict literal matching filters out semantically similar yet thematically irrelevant noise.

Its drawbacks are equally prominent: when researchers lack precise conceptual descriptions or familiarity with domain terminology, literal matching alone leads to substantial missed relevant papers (failure modes elaborated in Section III).

For method‑demanding work such as systematic reviews and evidence synthesis that demand full reproducibility, keyword search, controlled vocabularies, and exact‑phrase lookup remain indispensable to avoid untraceable semantic generalization. Such platforms must preserve full support for Boolean logic and field‑restricted searching to enable replicable search strategies. Intelligent academic research platforms such as UniResearch leverage comprehensive large‑scale literature repositories while retaining Boolean and field‑filtering capabilities for constructing rigorous, controllable search queries.

III. Three Typical Failure Modes of Literal‑Term Matching

Traditional lexical retrieval failures fall into three classic categories: synonymous expressions, polysemous ambiguity, and breakdown in complex conceptual combinatorial relationships. These pain points are particularly acute in academic search and constitute key targets for AI‑powered literature interpretation and structured intelligent reading tools.

3.1 Synonymous Expressions: One Concept, Multiple Terminological Forms

Different authors and sub‑fields coin divergent terms for identical scientific concepts. For example, “adverse drug reaction (ADR)” may also appear as “drug adverse reaction” or “drug‑related harm events”; “named entity recognition” is alternatively termed “proper‑name recognition”. Such terminological divergence is especially pronounced across sub‑disciplines. Since keyword matching only detects surface‑level word overlap, papers using synonymous phrasing are omitted, resulting in loss of critical scholarship.

3.2 Polysemous Ambiguity: Identical Terms, Distinct Scholarly Contexts

Words such as “model”, “stress”, “structure”, and “memory” span entirely separate research domains. Retrieval systems based purely on term presence cannot distinguish disciplinary contexts, returning large volumes of off‑topic literature and increasing manual screening workload.

3.3 Complex Compositional Meanings: Relationships Trump Isolated Terms

The meaning of many research queries derives from logical relations among multiple terms rather than individual words in isolation. For a query representing “a method applied to a certain research object”, traditional systems decompose the query into separate terms and return documents containing each term but lacking the intended logical relationship.

To mitigate these limitations, Semantic Scholar incorporates knowledge‑graph embedding: queries and papers are mapped onto standardized conceptual entities, and ranking is performed based on semantic relations between entities. This improves performance for queries where lexical search struggles. The retrieval paradigm thus shifts from counting overlapping words between queries and documents to judging whether queries and documents point toward comparable scientific concepts.

IV. Technical Approaches and Intrinsic Limitations of Semantic Retrieval

This section discusses an alternative semantic‑retrieval paradigm built on dense vector representations.

4.1 Vector Representations: Encoding Literature into Semantic Vectors in Latent Space

Dense vector representation is the dominant technical route: full documents are projected into high‑dimensional vector space. Thematically similar papers lie close to one another in this space and can be retrieved as relevant even with zero surface‑word overlap.

In scholarly document representation learning, the SPECTER model stands out. Besides textual content, it draws heavily on citation signals during training, building on the premise that mutually citing papers tend to share scholarly connections, thereby learning each paper’s position within the knowledge ecosystem. It is paired with the SciDocs evaluation benchmark covering seven document‑level tasks including citation prediction, document classification, and recommendation. Subsequent work argues that a single paper should yield different vector embeddings for different tasks. To this end, the SciRepEval multi‑format benchmark was released, covering classification, regression, nearest‑neighbor retrieval, and search to evaluate multi‑task paper representation‑learning methods.

4.2 Four Inherent Limitations of Vector‑Based Semantic Retrieval

Vector‑based semantic retrieval excels at synonym expansion and cross‑concept recall. Yet its efficacy rests on the hypothesis that conceptual similarity can be approximated by geometric distance in vector space — an assumption that frequently breaks down in real‑world scenarios, giving rise to four persistent limitations:

1. Weak performance for precise entity matching: For queries targeting concrete entities such as molecular identifiers, gene names, algorithm versions, standard abbreviations, drug names, or specific study subjects, vector retrieval tends to return topically analogous yet entity‑mismatched papers. For example, searching for applications of a given method to one disease yields many results applying the same method to unrelated diseases, inflating irrelevant outputs. This bias arises from training objectives: vectorization prioritizes smooth conceptual proximity and is inherently misaligned with exact string matching.
2. Loss of fine‑grained details for long documents: Constrained by Transformer context windows and GPU memory costs, vector models mostly encode only titles, abstracts, or partial excerpts. Full‑text details — especially parameters, conditional constraints, and tabular data within Methods and Results sections — are truncated or compressed. Consequently, semantic matching for long documents cannot approach the fidelity of full human reading.
3. Poor handling of negation, numerical comparison, and temporal constraints: Vector‑space distance metrics struggle to encode logical operators such as “not”, “less than”, “earlier than”, or “exclude X”. Queries containing negation, magnitude comparisons, or temporal filters suffer marked drops in hit‑rate and precision, often requiring post‑processing rules or secondary retrieval for remediation.
4. Poor interpretability and representational drift across domains: Vector proximity reflects purely geometric closeness in high‑dimensional latent space. Researchers cannot intuitively interpret why two papers are deemed similar; result interpretability is far lower than keyword‑ or Boolean‑logic‑based retrieval. When deployed in new research domains, vector distributions drift toward themes seen in training corpora, undermining cross‑disciplinary retrieval stability.

Moreover, semantic similarity is multi‑faceted: two papers may resemble each other in theme, methodology, study subject, citation lineage, or recommendation suitability. No single vector embedding can capture all these dimensions simultaneously. Semantic retrieval adds extra evidence for relevance assessment but does not equate to machines fully comprehending scholarly content.

Many research tools have adopted this design philosophy. AI research‑assistant platforms such as UniResearch leverage semantic‑search capabilities based on document vectors for conceptual recall, while retaining term‑level, year‑based, and journal‑level filtering. This expands conceptual search scope while preserving controllability, making it well‑suited for exploratory literature investigation in unfamiliar domains.

V. Citation Networks: Scholarly Relations Beyond Text

5.1 Scholarly Information Embedded in Citation Relationships

Distinct from ordinary webpages, academic papers form structured citation networks. A paper’s references reveal its intellectual origins; multiple works citing the same foundational paper often belong to a shared research tradition. A paper cited broadly across disparate fields suggests its methods are diffusing into other disciplines. Tracing references backward uncovers theoretical and methodological roots; tracing forward citations reveals subsequent revisions, extensions, and critiques of original work.

Modern academic retrieval therefore analyzes not only document text but also relational links between papers. Semantic Scholar constructs heterogeneous scholarly graphs integrating papers, authors, entities, and citation links. OpenAlex likewise builds interconnected scholarly entity graphs supporting analysis of works, institutions, topics, and citation relations.

A critical boundary must be emphasized: co‑citation between two papers only demonstrates some form of scholarly association. It does not mean their research conclusions are identical. Citation networks map intellectual genealogy but cannot directly judge paper quality or factual correctness.

5.2 Practical Value of Graph Tools for Domain Exploration

For newcomers to an unfamiliar research area, pure keyword retrieval often confines results to popular buzzwords. Literature‑association graphs enable outward exploration starting from a small set of seed papers, identifying research branches, cross‑disciplinary connections, and landmark foundational works.

Researchers can expand outward from seed papers via relational graphs to map intellectual lineages, sub‑fields, and potential cross‑disciplinary links for high‑level domain orientation. UniResearch offers visualized literature‑association graphs, automatic knowledge‑graph construction, and literature provenance tracing. Nonetheless, graph outputs serve only as analytical clues; all core judgments must be validated against original full‑text papers.

VI. Hybrid Retrieval: Complementary Fusion of Keyword and Semantic Retrieval

6.1 Shortcomings of Single‑Mode Retrieval

Keyword retrieval delivers precise matching yet suffers high omission risks for synonymous terminology and cross‑disciplinary concepts. Vector‑based semantic retrieval enables synonym expansion and cross‑concept recall yet exhibits intrinsic drawbacks regarding exact‑entity matching, capturing fine‑grained long‑document content, processing negation and numerical conditions, result interpretability, and cross‑domain stability (see Section 4.2).

Neither paradigm fully supersedes the other. The mainstream industry solution is hybrid retrieval: sparse term‑based retrieval (e.g., BM25) and dense‑vector semantic search run in parallel to generate multiple candidate document pools, followed by fusion strategies and cross‑encoder re‑ranking. This yields a multi‑stage retrieval pipeline: multi‑source initial recall → candidate pool → re‑ranking → final output. Recent studies show that fusion mechanisms and weight configurations heavily shape final performance; conventional Reciprocal Rank Fusion (RRF) weighting does not always outperform well‑tuned standalone dense retrievers. Fusion strategies require task‑specific tuning. Many all‑in‑one research platforms adopt hybrid retrieval as their core architecture.

6.2 Researcher Competency Requirements under Hybrid Retrieval

With hybrid‑retrieval architectures widely deployed, researchers must recognize which retrieval mode serves their current task. For exact‑entity lookup and reproducible literature reviews, keyword and field‑restricted search take priority. For exploratory work in unfamiliar domains or cross‑disciplinary mining, semantic capabilities and citation networks should be emphasized. Semantic expansion functions act as supplementary tools; controllable lexical‑search logic cannot be entirely discarded.

VII. Four‑Dimensional Evaluation Metrics for Academic Retrieval Performance

Ordinary users tend to judge retrieval‑system quality subjectively from top‑page outputs. For scholarly literature investigation — particularly systematic reviews and evidence synthesis — subjective impressions are insufficient. Four core metrics should be jointly assessed: recall, precision, ranking quality, and reproducibility.

• Recall: the proportion of all genuinely relevant documents retrieved by the system. In systematic reviews, missing key literature can directly alter review conclusions.
• Precision: the share of returned documents genuinely relevant to the research topic. Low precision imposes heavy manual screening burdens.
• Ranking quality: positional placement of high‑value relevant papers within result lists.
• Reproducibility: the capacity to fully record, replicate, and explain search strategies. This represents a hard requirement for systematic reviews and meta‑analyses.

Gusenbauer and Haddaway empirically evaluated 28 academic search systems and found substantial performance variance across recall, precision, and reproducibility. Approximately half of those systems (and very few open‑access databases) are suitable for evidence‑synthesis workflows. Google Scholar is not recommended as a primary retrieval resource for systematic reviews. This empirical evidence underpins the four‑dimensional evaluation framework outlined above, furnishing evaluative criteria for researchers selecting literature‑investigation tools.

VIII. Task‑Oriented Adaptation of Academic Retrieval Strategies

The four task categories below reflect different priority trade‑offs among the four metrics: systematic evidence retrieval prioritizes recall and reproducibility; exploration in unfamiliar domains prioritizes recall with tolerable precision degradation.

8.1 Retrieving Known Target Literature: Prioritize Exact Lookup

When paper titles, DOIs, authors, gene identifiers, chemical‑substance IDs, or fixed specialized terms are available, use keyword plus field‑restricted exact search. Semantic expansion should be minimized to avoid introducing extraneous noise.

8.2 Systematic Evidence Retrieval: Prioritize Controllable, Reproducible Workflows

Systematic reviews and meta‑analyses demand traceable retrieval workflows. Researchers should carefully select databases, apply field constraints, Boolean logic, synonym tables, and controlled vocabularies, and document search timelines. Every included candidate document should admit an interpretable inclusion rationale. Database searching should be complemented by forward‑ and backward‑citation tracking rather than relying on any single method. For instance, PubMed’s “Best Match” ranking launched in June 2017 represents an industrial‑grade implementation combining lexical matching, term‑frequency signals, and learned ranking. Even so, evidence‑synthesis workflows should adhere to the principles laid out in this section.

8.3 Exploring Unfamiliar Domains: Prioritize Semantic Retrieval and Citation Networks

When domain terminology is poorly understood and effective keywords are hard to formulate: obtain a small set of representative seed papers, deploy AI‑powered and structured intelligent literature interpretation to distill core arguments, apply semantic retrieval to find conceptually analogous papers, and use citation graphs to trace intellectual origins and subsequent derivative work. Iteratively refine keywords in a “search‑learn‑search‑again” feedback loop.

8.4 Cross‑Disciplinary Retrieval: Attend to Concept‑Level Semantic Links

Different disciplines employ disparate terminology to describe comparable scientific phenomena, rendering pure keyword retrieval prone to omissions. Semantic retrieval and knowledge graphs uncover conceptual associations. Multi‑paper comparative reading and literature‑matrix analysis can rapidly map similarities and divergences across fields. Nevertheless, candidate papers retrieved cross‑disciplinarily demand human critical assessment: conceptual similarity does not guarantee transferable research hypotheses, experimental conditions, or evidentiary standards.

No universal optimal retrieval method exists. Tool selection must align with specific research objectives.

IX. New Transformations Brought by Generative‑AI‑Assisted Research to Academic Search

Generative AI reshapes retrieval interfaces: users may input full natural‑language research questions instead of only keywords. Still, fundamental retrieval challenges persist: source of literature, relevance judgment, ranking, evidence provenance, and risk of omitting counter‑evidence papers.

Generative interfaces constitute outer‑layer wrappers; underlying retrieval systems still perform document recall. Large language models cannot invent missing literature if the base retrieval step omits key sources. When fed many weakly relevant documents, LLMs may produce logically coherent yet factually suspect synthesized answers.

Evidence traceability therefore becomes even more critical within AI‑augmented retrieval workflows. Researchers must be able to inspect original source papers cited by AI outputs and revisit raw retrieval pools to verify whether important scholarship has been overlooked. Users of AI‑assisted literature‑review tools should remain alert to conclusion biases stemming from incomplete underlying retrieval.

Conclusion

The technical evolution of academic search is not a simple linear replacement of keyword search. Instead, it reflects a shift from single‑source relevance evidence toward multi‑evidence fusion. Keyword retrieval captures surface‑level lexical features of documents; semantic vectors model conceptual similarity; citation networks map scholarly intellectual lineage; hybrid retrieval merges these multiple signals.

Keyword‑based exact matching, Boolean search, and citation tracing retain irreplaceable research value. Each technology illuminates only one facet of scholarly documents, and no universal retrieval paradigm fits all research tasks. For modern researchers, research efficiency means more than mastering query formulation. It entails understanding what relevance evidence underpins returned results and actively governing retrieval strategies rather than delegating relevance judgment entirely to black‑box retrieval systems. As scholarly research grows increasingly intelligent, all‑in‑one research platforms integrate retrieval, interpretation, and graph‑based analysis. Researchers should flexibly combine keyword search, semantic retrieval, and citation‑graph analysis tailored to task goals to mitigate omission and mis‑retrieval risks and support high‑quality literature investigation.

References

[1] Robertson S, Zaragoza H. The Probabilistic Relevance Framework: BM25 and Beyond [J]. Foundations and Trends in Information Retrieval, 2009, 3(4): 333‑389. DOI: 10.1561/1500000019.

[2] Xiong C, Power R, Callan J. Explicit Semantic Ranking for Academic Search via Knowledge Graph Embedding [C]//Proceedings of the 26th International Conference on World Wide Web (WWW 2017). 2017: 1271‑1279. DOI: 10.1145/3038912.3052558.

[3] Cohan A, Feldman S, Beltagy I, Downey D, Weld D S. SPECTER: Document‑level Representation Learning using Citation‑informed Transformers [C]//Proceedings of ACL 2020. 2020: 2270‑2282. DOI: 10.18653/v1/2020.acl‑main.207.

[4] Singh A, D’Arcy M, Cohan A, Downey D, Feldman S. SciRepEval: A Multi‑Format Benchmark for Scientific Document Representations [C]//Proceedings of EMNLP 2023. 2023: 5548‑5566. DOI: 10.18653/v1/2023.emnlp‑main.338.

[5] Ammar W, Groeneveld D, Bhagavatula C, et al. Construction of the Literature Graph in Semantic Scholar [C]//Proceedings of NAACL‑HLT 2018, Volume 3 (Industry Papers). 2018: 84‑91. DOI: 10.18653/v1/N18‑3011.

[6] Lee D, Hwang S W, Lee K, Choi S, Park S. On Complementarity Objectives for Hybrid Retrieval [C]//Proceedings of ACL 2023. 2023: 13357‑13368. DOI: 10.18653/v1/2023.acl‑long.746.

[7] Al‑Joofi W, Sagheer A, Hamdoun H. A Multi‑Stage Hybrid Retrieval Framework for the Scientific Literature with Cross‑Encoder Re‑Ranking [J]. Applied Sciences, 2026, 16(10): 4813. DOI: 10.3390/app16104813.

[8] Gusenbauer M, Haddaway N R. Which academic search systems are suitable for systematic reviews or meta‑analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources [J]. Research Synthesis Methods, 2020, 11(2): 181‑217. DOI: 10.1002/jrsm.1378.

[9] Mourão E, Pimentel J F, Murta L, et al. On the performance of hybrid search strategies for systematic literature reviews in software engineering [J]. Information and Software Technology, 2020, 123: 106294. DOI: 10.1016/j.infsof.2020.123.106294.

[10] Fiorini N, Canese K, Starchenko G, et al. Best Match: New relevance search for PubMed [J]. PLOS Biology, 2018, 16(8): e2005343. DOI: 10.1371/journal.pbio.2005343.

[11] Priem J, Piwowar H, Orr R. OpenAlex: A fully‑open index of scholarly works, authors, venues, institutions, and concepts [EB/OL]. arXiv:2205.01833, 2022.