# Impact of Tokenization on Large Language Models

## Mechanics of Language Model Tokenization

Tokenization serves as the foundational preprocessing layer of modern large language models, dictating the fundamental boundaries of how artificial neural networks perceive natural language. Before a transformer architecture can process text, the raw string of characters must be systematically transformed into a discrete sequence of numerical identifiers [cite: 1, 2, 3]. Language models operate entirely through mathematical operations on high-dimensional vector spaces; they possess no inherent understanding of syntax, orthography, or character relationships [cite: 4, 5, 6]. Consequently, the tokenization process defines the absolute atomic units of the model’s linguistic universe. The segmentation boundaries established by a given tokenizer constrain downstream performance, computational efficiency, multilinguality, and logical reasoning capabilities [cite: 2, 7]. 

The pipeline from raw text to computable inputs involves three distinct phases: text segmentation, integer vectorization, and dense vector embedding [cite: 2, 3, 8].

[image delta #1, 0 bytes]

 Initially, raw text is partitioned into smaller constituent chunks, known as tokens, utilizing a predefined statistical algorithm [cite: 1, 9]. Depending on the specific tokenizer architecture, a token may encompass an entire word, a morphological subword, a single character, or an individual raw byte [cite: 1, 9, 10]. 

Following segmentation, each discrete token is mapped to a unique integer identifier corresponding to its exact index within the tokenizer's fixed, pre-compiled vocabulary [cite: 3, 8, 9]. For example, the complex word "untokenizable" might be fractured into the subword components "un", "token", and "izable", which are subsequently assigned arbitrary integer identifiers from the vocabulary matrix [cite: 3]. 

The final, mathematically critical step is the projection of these integer identifiers into dense, continuous vectors known as embeddings. The embedding layer functions as an expansive lookup matrix, with dimensions determined by the total vocabulary size multiplied by the model's chosen hidden dimension parameter [cite: 8, 11]. An embedding captures profound semantic relationships, grammatical roles, and contextual nuances learned iteratively during the model's pretraining phase [cite: 1]. During this phase, models learn to map tokens with similar semantic weight into close proximity within the multi-dimensional embedding space [cite: 6, 8]. Ultimately, the tokenization protocol directly governs the granularity and structural integrity of the semantic representations the language model relies upon for text generation and comprehension.



## Evolution of Subword Algorithms and Vocabulary Architectures

Historically, natural language processing frameworks relied primarily on word-level or character-level tokenization paradigms. Word-level tokenization preserves high semantic value but suffers from catastrophic out-of-vocabulary rates [cite: 3, 7]. To accommodate conjugation, varying punctuation, and morphological richness, word-level tokenizers require unmanageably large vocabulary matrices [cite: 3, 7]. Conversely, character-level tokenization resolves the out-of-vocabulary problem by utilizing a highly constrained vocabulary restricted to individual letters and symbols, but it heavily inflates the sequence length of the input data [cite: 7]. Because the computational cost of the transformer's self-attention mechanism scales quadratically relative to sequence length, character-level processing rapidly exhausts memory buffers and severely limits the model's capacity to learn long-range semantic dependencies [cite: 7, 12, 13].

Modern large language models bridge this operational gap by utilizing subword tokenization algorithms, most notably Byte-Pair Encoding and its advanced variants such as WordPiece and SentencePiece [cite: 2, 3, 14, 15]. Adapted for neural machine translation in 2016, Byte-Pair Encoding relies fundamentally on data compression principles [cite: 3, 14]. The algorithm initializes with a base vocabulary of individual characters or raw bytes, then iteratively scans a massive reference training corpus to identify the most frequently occurring adjacent pairs of symbols. The most statistically common pair is merged into a single new subword symbol, which is permanently added to the evolving vocabulary [cite: 2, 15]. This greedy merging process repeats iteratively until the vocabulary expands to a predetermined target size [cite: 2, 3, 15].

The result is a highly efficient linguistic representation. Common, high-frequency elements are preserved intact as single tokens, ensuring computational efficiency [cite: 3, 5, 7, 16]. Simultaneously, rare, complex, or entirely unknown words are fragmented into recognizable subword pieces [cite: 3, 17]. By operating at the foundational byte level, modern tokenizers ensure that any arbitrary string of text—including executable code snippets, emojis, mathematical formatting, and non-Latin scripts—can be encoded losslessly without returning a definitive out-of-vocabulary error [cite: 3, 18, 19].

The selection of a tokenizer's overall vocabulary size introduces a profound architectural trade-off regarding model parameters and computational overhead. Smaller vocabularies result in fewer neural network parameters required for the embedding and prediction layers, but they generate longer, highly fragmented token sequences, thereby inflating inference computation costs [cite: 14]. Larger vocabularies compress text more efficiently into fewer tokens, allowing models to process more contextual information within their fixed context windows [cite: 14]. However, vast vocabularies demand massive embedding matrices. For example, specific commercial architectures utilizing a hidden dimension of 12,288 alongside a sprawling vocabulary of 256,000 tokens require approximately 6.3 billion model parameters solely dedicated to maintaining the embedding and prediction layers [cite: 14].

Despite this immense parameter cost, the trajectory of artificial intelligence engineering has aggressively favored larger vocabularies to maximize compression rates, reduce sequence length, and enhance multilingual representation.

| Model Architecture | Tokenizer Implementation | Approximate Vocabulary Size | Primary Tokenization Algorithm | Architectural Notes |
| :--- | :--- | :--- | :--- | :--- |
| **GPT-3 / GPT-3.5** | `r50k_base` / `p50k_base` | 50,257 | Byte-Pair Encoding (BPE) | Early generation tokenizer architecture utilized for OpenAI Codex and the original GPT-3 series [cite: 3, 20, 21]. |
| **GPT-4 / GPT-4 Turbo** | `cl100k_base` | 100,256 | Byte-Pair Encoding (BPE) | High-speed tokenizer implemented via the open-source `tiktoken` library. Highly optimized for English prose and code generation [cite: 18, 21, 22]. |
| **GPT-4o / o1 / o3** | `o200k_base` | 199,997 | Byte-Pair Encoding (BPE) | Substantially improves compression across non-English scripts, averaging four characters per token in English text [cite: 18, 19, 22]. |
| **Llama 1 / Llama 2** | Custom SentencePiece | 32,000 | SentencePiece (BPE) | Features a constrained vocabulary that struggled with out-of-domain text and non-Latin scripts [cite: 15]. |
| **Llama 3 / 3.1** | Custom TikToken | 128,256 | Byte-Pair Encoding (BPE) | Integrated 100,000 standard tokens and 28,000 dedicated non-English tokens, yielding a 15% reduction in token sequence lengths compared to previous iterations [cite: 23, 24, 25]. |
| **Mistral / Mixtral** | Tekken | 131,072 | Byte-Pair Encoding (BPE) | Expanded vocabularies explicitly designed to support multilingual deployment efficiently across European languages [cite: 15, 26]. |
| **Aya-Expanse** | Custom Multilingual | 255,029 | Byte-Pair Encoding (BPE) | Massively expanded vocabulary engineered to force token parity across diverse global languages [cite: 27]. |

Table 1: Vocabulary Size and Tokenizer Architecture Across Modern Large Language Models [cite: 15, 18, 21, 22, 23, 24, 25, 26, 27].

## Sublexical and Structural Processing Limitations

While subword tokenization elegantly balances the rigid computational constraints of transformer self-attention mechanisms with the semantic richness of natural language, the methodology fundamentally obscures the internal structure of words from the model's perception [cite: 11]. Because the language model only evaluates sequences of integer identifiers referencing atomic subword units, it lacks inherent awareness of character counts, morphological roots, acoustic phonology, and numerical positional notation [cite: 11, 28]. 

### Character-Level Manipulation and Orthographic Failures

A thoroughly documented failure mode across leading language models is their inability to reliably perform character-level manipulation or rigorous orthographic analysis. This representational deficiency is popularly characterized by the "strawberry problem," a pervasive phenomenon wherein highly capable models consistently miscalculate the number of 'r' characters in the word "strawberry" [cite: 6, 29]. 

This specific error arises because the model does not ingest the literal string sequence of characters. Depending on the exact Byte-Pair Encoding vocabulary utilized, the word is segmented into discrete tokens. OpenAI's implementations may split the word into `["straw", "berry"]` or further into `["Str", "aw", "berry"]` [cite: 6, 28, 29]. Within the high-dimensional embedding space, these individual tokens represent broad semantic concepts such as fruit, sweetness, and the color red, rather than phonetic boundaries or orthographic character clusters [cite: 6, 28]. Consequently, when tasked with counting characters, the model must deduce the internal composition of a token based entirely on statistical associations absorbed during pretraining, rather than programmatically inspecting the string array [cite: 28, 29]. 

This representational gap causes significant brittleness in tasks demanding precise sub-lexical accuracy. Determining whether a word contains double letters, generating exact acronyms, reversing strings, and spelling complex words backwards consistently fall outside the natural scope of subword embeddings [cite: 28, 29]. Furthermore, the probabilistic text generation mechanisms of transformers inherently prioritize predicting the most statistically likely semantic continuation. This leads models to frequently sacrifice strict orthographic and formatting accuracy in favor of contextual plausibility, resulting in hallucinations of spelling when forced into constrained formatting requirements [cite: 6, 30]. Subword regularization techniques, such as BPE-dropout during training, can marginally improve resilience to typographical variation, but the fundamental blindness to character structures remains a persistent structural limitation [cite: 11, 30].

### Deficiencies in Mathematical and Arithmetic Reasoning

Language models notoriously struggle with mathematical calculations and multi-step quantitative reasoning, not because their underlying neural architectures inherently lack the capacity for logic, but because their input mechanism shatters numerical data unpredictably [cite: 5]. Tokenizers are engineered to compress human language by grouping highly frequent character sequences, but numerical strings do not follow the morphological or syntactic rules of natural language text [cite: 5].

When processing numbers, a subword tokenizer parses the input arbitrarily based on training data frequency rather than mathematical significance [cite: 5]. A highly frequent number like "100" or "2024" likely exists as a single, distinct token within the vocabulary, allowing the model to process it cohesively. Conversely, an arbitrary large integer, such as "87439," might be fragmented into `["874", "39"]`, or even reduced to individual character tokens [cite: 5]. 

This unstructured numerical tokenization permanently destroys positional notation. The tokenizer's assignment of abstract integer IDs to these numerical fragments strips away any inherent quantitative semantics or scale [cite: 5]. To the model, the token representing the string "874" is an abstract mathematical entity devoid of inherent magnitude, functionally indistinguishable in the embedding layer from the tokens corresponding to words like "apple" or "cloud" [cite: 5]. As a result, the model must attempt to execute complex arithmetic by mimicking statistical text patterns observed in the training corpus rather than by applying the deterministic, axiomatic rules of addition or multiplication [cite: 5, 31]. Rare numbers trigger unfamiliar token combinations, resulting in cascading logical failures, misintegration of problem constraints, and the hallucination of non-existent mathematical features during complex reasoning chains [cite: 5, 32]. 

### Constraints on Phonological Processing and Rhyming

Subword tokenization is exclusively optimized for the efficient statistical modeling of written graphemes within a massive text corpus; it rarely aligns with the linguistically meaningful phonological or syllabic units of spoken language [cite: 33, 34]. Because Byte-Pair Encoding greedily merges characters based strictly on textual frequency, the resulting tokens arbitrarily cross phonetic syllable boundaries and strip away the underlying acoustic properties of the word [cite: 33, 34].

Extensive evaluations utilizing phonological benchmarks demonstrate that language models perform sub-optimally compared to human baselines on tasks requiring direct phonemic awareness, including grapheme-to-phoneme conversion, precise syllable counting, and complex rhyme generation [cite: 33, 34, 35]. Research indicates a substantial gap in capability: models exhibit significantly higher accuracy on phonological tasks when the target vocabulary serendipitously aligns with whole-word tokens, compared to instances where the target words are fragmented into split-word tokens [cite: 33]. The profound disconnection between written subword embeddings and acoustic realities severely impedes the reliable deployment of text-only models in poetry generation, speech-synthesis reasoning tasks, and accurate phonetic transcription applications without targeted architectural interventions [cite: 33, 34].

## The Multilingual Token Tax and Infrastructure Bias

The frequency-based, statistical nature of Byte-Pair Encoding introduces severe technical and economic inequities when models are deployed across global languages. Tokenizer vocabularies are heavily biased by their reference training corpora, which are overwhelmingly composed of English texts and specialized programming code [cite: 26, 36]. As a consequence, tokenizers aggressively optimize subwords for English syntax, allocating the vast majority of their vocabulary slots to English morphological patterns while marginalizing low-resource languages [cite: 27, 37, 38]. 

### Quantifying Disparities Through Token Fertility

The primary metric utilized in computational linguistics for measuring the efficiency of a tokenizer across different languages is "token fertility," which is formally defined as the average number of subword tokens required to accurately represent a single semantic word unit [cite: 12, 26, 38, 39].

[image delta #2, 0 bytes]

 For high-resource languages like English, modern tokenizers generally maintain a highly efficient fertility rate near 1.0 to 1.4, indicating that most common words map cleanly to one or two tokens [cite: 27, 40]. 

However, for languages featuring rich morphology, including agglutinative languages such as Swahili, Zulu, or Turkish, as well as languages utilizing non-Latin scripts like Arabic, Korean, and Hindi, token fertility spikes dramatically [cite: 7, 12, 38, 41]. Complex abugida writing systems are particularly vulnerable; a single semantic word in Malayalam or Telugu may be shattered into six or more distinct, meaningless tokens [cite: 37, 38, 42]. In many cases, these languages break down to individual bytes or erratic fragments purely because their script was critically underrepresented during the tokenizer's initial training phase [cite: 38, 41].

| Linguistic Classification | Language | GPT-4o Token Fertility | Llama 3.1 (8B) Token Fertility | Analytical Notes |
| :--- | :--- | :--- | :--- | :--- |
| **High-Resource Latin** | English | 0.96 | 1.01 | Serves as the base reference; benefits from highly optimized single-token mappings. |
| **High-Resource Latin** | Spanish | ~1.10 | ~1.15 | Closely mirrors English token distribution and compression efficiency. |
| **Cyrillic Script** | Russian | 1.68 | 3.26 | Cyrillic text fragments heavily in Llama 3 compared to the expanded vocabulary of GPT-4o. |
| **Abugida Script** | Hindi | 1.77 | 4.18 | Suffers severe fragmentation due to complex combining marks and codepoint splits. |
| **Connected Script** | Arabic | 1.62 | 3.29 | High fertility driven by complex morphology and specific orthographic constraints. |
| **Low-Resource Indo-Aryan** | Urdu | 1.60 | 3.50+ | Morphologically complex and historically low-resource in massive pretraining corpora. |

Table 2: Comparative Token Fertility Ratios Across Selected Languages and Model Tokenizers [cite: 26, 40, 43, 44]. 



### Computational and Economic Consequences

The phenomenon of high token fertility across non-English texts directly translates into a systemic infrastructure bias widely categorized in computational linguistics as the "Token Tax" [cite: 12, 37, 45]. Because commercial LLM API providers price their services strictly by the total volume of tokens processed for inputs and outputs, native speakers of high-fertility languages are economically penalized [cite: 12, 21, 37]. Translating token inflation directly to economics, the financial cost required to generate the exact same semantic paragraph in languages like Telugu or Amharic can be between three and six times more expensive than generating it in English [cite: 12, 37, 46]. 

Beyond financial penalties, extreme token inflation cripples computational efficiency and hardware utilization. The memory footprint, latency, and overall compute demands of the transformer's self-attention mechanism scale quadratically ($O(n^2)$) relative to the length of the token sequence [cite: 12, 47]. Consequently, a language presenting a token fertility of 2.0 does not merely double the processing time; it theoretically quadruples the required computational resources, memory bandwidth, and associated energy emissions during both the training and inference phases [cite: 12, 47, 48]. 

High fertility also tangibly degrades the effective context window and ultimate model accuracy. A fixed context window of 8,192 tokens can hold extensive software documentation in English, but it may only process a fraction of that exact same material in Burmese or Zulu before catastrophically truncating the input [cite: 24, 37, 41]. Extensive evaluations across massively multilingual benchmarks, such as AfriMMLU, consistently reveal that high token fertility operates as a reliable, negative predictor of downstream LLM accuracy and reasoning capabilities [cite: 12, 45, 47]. Subword fragmentation forces the model to expend substantial attention-head capacity merely attempting to reassemble cohesive words from disparate character pieces, rather than engaging in higher-order semantic reasoning or following complex instructions [cite: 7, 38]. Researchers have observed that reasoning-focused models, which allocate more computation time during generation, can partially mitigate this accuracy gap, but the structural inefficiencies persist [cite: 12, 47].

## Security Vulnerabilities Originating from Tokenization

The structural disconnect between the algorithms utilized to construct a tokenizer vocabulary and the algorithms used to train the language model’s parameters introduces specific, highly exploitable blind spots within the neural network. Because the tokenizer vocabulary is rigorously fixed before the LLM begins its foundational training phase, the model is forced to map deep semantic representations onto tokens it may rarely—or completely never—encounter within its curated training corpus.

### Glitch Tokens and Vocabulary Blind Spots

In 2023, security researchers uncovered a highly disruptive class of anomalous inputs formally designated as "glitch tokens" [cite: 49, 50, 51]. The most canonically cited example of this phenomenon is the token string `SolidGoldMagikarp` [cite: 50, 51, 52, 53]. This string was initially scraped into the vocabulary of the GPT-2 and GPT-3 tokenizers because it frequently appeared as an active username within a specific, highly repetitive Reddit community dedicated to sequential counting [cite: 51, 53]. However, the actual text corpus utilized to train the subsequent language model parameters systematically excluded that specific forum data to prioritize higher-quality text [cite: 51, 53]. 

As a direct result of this dataset discrepancy, the high-dimensional embedding vector associated with the `SolidGoldMagikarp` token ID was initialized but never meaningfully updated through gradient descent during the model's pretraining phase [cite: 51, 53, 54]. The token essentially became a "dead zone" in the model's mathematical representation space [cite: 51]. When users prompted the model to process, repeat, or explain this specific token, the LLM attempted to execute complex linguistic logic over an untrained, noisy vector. This resulted in highly erratic and unpredictable behavior. The model would output hallucinatory gibberish, hurl unexpected insults, generate policy-violating text, or default to completely unrelated concepts (for example, outputting the word "distribute" in place of the input string) [cite: 49, 50, 51, 53, 55]. 

Research subsequent to the initial `SolidGoldMagikarp` discovery has utilized gradient-based discrete optimization frameworks, such as GlitchMiner, to automatically mine models for these anomalies [cite: 49, 56]. Studies have successfully identified thousands of these untrained tokens clustering in the remote embedding spaces of major commercial and open-weights models [cite: 49, 51, 57]. Glitch tokens pose a serious alignment and safety risk because they fundamentally disrupt semantic comprehension, allowing deterministic, erratic failures to be triggered by seemingly innocuous input strings [cite: 49, 53, 55].

### Adversarial Token Boundary Manipulation and Filter Evasion

The rigid and predictable nature of tokenizer boundaries provides a highly effective vector for adversarial exploitation, particularly in the context of bypassing automated safety filters and alignment guardrails [cite: 58, 59, 60]. Standard safety classifiers, keyword blacklists, and internal alignment prompt-injections rely heavily on the assumption that specific harmful concepts will reliably trigger recognized token IDs (e.g., the specific token assigned to "bomb" or "methamphetamine") [cite: 51, 59]. 

Attackers explicitly exploit subword algorithms using token boundary attacks. By strategically inserting spaces, zero-width characters, or homoglyphs (visually identical Unicode characters sourced from different scripts, such as Cyrillic), a threat actor can force the tokenizer to slice a strictly prohibited word into unfamiliar, fragmented tokens [cite: 59, 60, 61]. For example, injecting a specific character combination might break a restricted keyword into benign fragments like `["synt", "he", "size"]` [cite: 59, 60]. 

Because the safety filter evaluates the input at the token ID level rather than analyzing the raw string text, the fragmented IDs successfully evade the blacklist and enter the model [cite: 51, 59]. However, the deeper layers of the transformer, which utilize sophisticated context and self-attention, often piece the semantic meaning of the fragments back together, successfully generating the restricted or harmful content [cite: 59, 60, 61]. Studies measuring character-level adversarial robustness reveal that these techniques reduce refusal rates by up to 22% across modern frontier models [cite: 61]. Furthermore, advanced techniques like Seamless Spurious Token Injection (SSTI) allow attackers to weaponize special control tokens (such as `<SEP>` or end-of-sequence markers) to alter how the LLM determines the boundary between user input and overarching system instructions, effectively hijacking the model’s internal control flow and decision-making logic [cite: 62]. 

## Post-Tokenization Architectures and Future Paradigms

The compounding technical constraints of the multilingual token tax, sublexical processing failures, and token-level security vulnerabilities have spurred intense research into tokenizer-free architectures. Operating directly on the absolute fundamental representation of digital text—raw bytes—eliminates the need for predefined, static vocabularies. This architectural shift inherently resolves the out-of-vocabulary problem, ensures equitable processing across diverse global languages, and provides innate resistance to spelling errors and complex token boundary attacks [cite: 10, 13, 14]. 

### The Challenge of Byte-Level Processing and Dynamic Patching

Historically, training LLMs directly on individual bytes was deemed computationally unfeasible due to the severe sequence length explosion it causes. A single English word requires an average of 4.7 characters, meaning a strictly byte-level transformer would process sequences roughly five times longer than a standard BPE-tokenized model [cite: 13]. Given the quadratic scaling of attention computation relative to sequence length, pure byte-level LLMs were prohibitively expensive to train and deploy at scale [cite: 10, 13, 14].

Recent architectural innovations aim to overcome this fundamental scaling limitation by decoupling the sequence length from the model's overall depth. Instead of relying on static token assignments, researchers are implementing dynamic patching mechanisms. This sophisticated approach feeds raw bytes through a lightweight, localized encoder network that groups consecutive bytes into variable-length "patches" before passing them to the primary, computationally heavy global transformer [cite: 10, 63, 64].

### The Byte Latent Transformer

A prominent and successful example of this paradigm shift is the Byte Latent Transformer (BLT), an architecture developed by researchers at Meta and the University of Washington [cite: 10, 13, 65]. The BLT abandons static tokenization entirely. It models language directly from the raw byte stream and utilizes an advanced entropy-based patching algorithm to dynamically group bytes based on informational density [cite: 63, 65]. 

The BLT system continuously calculates the predictability, or entropy, of the incoming byte sequence. When the sequence is highly predictable (presenting low entropy, such as the middle characters of a common word or a long string of whitespace), the BLT aggressively groups many bytes into a single, long patch [cite: 63, 65]. Conversely, when the input is complex, unpredictable, or highly information-dense (presenting high entropy), the model forms much smaller, granular patches [cite: 63, 65]. 

This dynamic allocation ensures that the model expends intensive computational resources only where the data complexity actually demands it, strictly conserving processing FLOPs on trivial or highly predictable sequences [cite: 10, 13, 65]. Empirical evaluations of the Byte Latent Transformer demonstrate that, when operating under compute-optimal training regimes, these byte-level models can match or even exceed the scaling behavior of state-of-the-art token-based models like Llama 3 [cite: 13, 65, 66]. Tokenizer-free architectures exhibit superior performance on character-level manipulation tasks, enhanced robustness to noisy or heavily misspelled inputs, and significantly improved modeling of the "long tail" of low-resource languages. These results strongly suggest a viable, near-term pathway to eliminating the architectural bottleneck of tokenization entirely, yielding more universally capable foundation models [cite: 10, 13, 14, 65].

## Sources
1. [arxiv.org](https://arxiv.org/html/2410.15052v5)
2. [kith.org](https://www.kith.org/words/2023/12/10/solidgoldmagikarp-and-other-glitch-tokens/)
3. [youtube.com](https://www.youtube.com/shorts/6PRgC3eJR7k)
4. [aclanthology.org](https://aclanthology.org/2024.emnlp-main.649/)
5. [github.com](https://github.com/NiluK/SolidGoldMagikarp)
6. [apxml.com](https://apxml.com/courses/intro-large-language-models/chapter-2-simplified-mechanics-of-llms/tokens-and-embeddings)
7. [medium.com](https://viraajkadam.medium.com/understanding-llms-tokenization-how-models-convert-text-to-numerical-representation-80ba595b166e)
8. [medium.com](https://medium.com/@srirammoorthy11/llm-fundamentals-a-deep-dive-into-tokenization-embeddings-and-vector-spaces-d5c849297c53)
9. [traceloop.com](https://www.traceloop.com/blog/a-comprehensive-guide-to-tokenizing-text-for-llms)
10. [mikexcohen.substack.com](https://mikexcohen.substack.com/p/llm-breakdown-16-tokenization-words)
11. [grokipedia.com](https://grokipedia.com/page/O200k_base_tokenizer)
12. [datacamp.com](https://www.datacamp.com/tutorial/tiktoken-library-python)
13. [developers.openai.com](https://developers.openai.com/cookbook/examples/how_to_count_tokens_with_tiktoken)
14. [github.com](https://github.com/openai/tiktoken)
16. [towardsai.net](https://towardsai.net/p/artificial-intelligence/some-technical-notes-about-llama-3)
21. [emergentmind.com](https://www.emergentmind.com/topics/reasoning-failures-in-llms)
22. [medium.com](https://medium.com/@madasuvishnuraj/why-llms-struggle-with-mathematical-calculation-the-tokenization-problem-83322e9569a2)
23. [secwest.net](https://www.secwest.net/strawberry)
25. [researchgate.net](https://www.researchgate.net/publication/389090646_Large_Language_Models_and_Mathematical_Reasoning_Failures)
26. [aclanthology.org](https://aclanthology.org/2026.africanlp-main.10.pdf)
28. [reddit.com](https://www.reddit.com/r/OpenAI/comments/124v2oi/hindi_8_times_more_expensive_than_english_the/)
29. [researchgate.net](https://www.researchgate.net/publication/395355055_The_Token_Tax_Systematic_Bias_in_Multilingual_Tokenization)
30. [arxiv.org](https://arxiv.org/html/2509.05486v1)
31. [levelup.gitconnected.com](https://levelup.gitconnected.com/how-llm-injection-attacks-are-shaping-the-future-of-ai-security-937d802e6bfe)
32. [arxiv.org](https://arxiv.org/html/2410.15052v4)
34. [toxsec.com](https://www.toxsec.com/p/token-level-ai-security-the-opus)
36. [venturebeat.com](https://venturebeat.com/ai/metas-new-blt-architecture-replaces-tokens-to-make-llms-more-efficient-and-versatile)
37. [towardsdatascience.com](https://towardsdatascience.com/why-your-next-llm-might-not-have-a-tokenizer/)
38. [llmsresearch.com](https://www.llmsresearch.com/p/llms-related-research-papers-published-in-december-2024)
39. [levelup.gitconnected.com](https://levelup.gitconnected.com/tokenization-no-longer-exists-d67ea9eba8b3)
40. [arxiv.org](https://arxiv.org/html/2406.19223v1)
41. [aclanthology.org](https://aclanthology.org/2026.africanlp-main.10.pdf)
43. [emergentmind.com](https://www.emergentmind.com/topics/tokenization-disparities)
45. [emergentmind.com](https://www.emergentmind.com/topics/tokenisation-bias-in-language-models)
49. [vellum.ai](https://www.vellum.ai/blog/llama-3-70b-vs-gpt-4-comparison-analysis)
50. [galileo.ai](https://galileo.ai/blog/is-llama-3-better-than-gpt4)
55. [medium.com](https://medium.com/@madasuvishnuraj/why-llms-struggle-with-mathematical-calculation-the-tokenization-problem-83322e9569a2)
59. [verityai.co](https://verityai.co/blog/technical-exploitation-ai-systems)
60. [ziosec.com](https://ziosec.com/article?slug=ai-jailbreak-techniques-in-2026-a-complete-technical-guide-ziosec)
62. [ijmrset.com](https://www.ijmrset.com/upload/299_Evaluating%20Character-Level%20Adversarial%20Robustness%20of%20Safety-Aligned.pdf)
64. [hyperstack.cloud](https://www.hyperstack.cloud/blog/case-study/the-strawberry-problem-understanding-why-llms-misspell-common-words)
65. [rewire.it](https://rewire.it/blog/beyond-the-subword-morphologically-aware-tokenization/)
66. [runpod.io](https://www.runpod.io/blog/llm-tokenization-limitations)
67. [aclanthology.org](https://aclanthology.org/2024.findings-emnlp.86.pdf)
68. [medium.com](https://medium.com/@puspak.supakar/how-do-llms-understand-the-words-with-typos-spelling-mistakes-f8816d0d2c65)
69. [arxiv.org](https://arxiv.org/html/2506.15138v1)
70. [arxiv.org](https://arxiv.org/html/2504.07989v2)
71. [cureusjournals.com](https://www.cureusjournals.com/articles/13958-beyond-the-tokenization-bottleneck-a-conceptual-framework-for-efficiency-plasticity-trade-offs-in-asean-large-language-model-adaptation)
73. [aclanthology.org](https://aclanthology.org/2025.findings-emnlp.595.pdf)
76. [arxiv.org](https://arxiv.org/html/2406.11214v2)
78. [medium.com](https://medium.com/@vkmauryavk/introducing-llama-3-1-key-points-of-paper-165c29d9c7fd)
79. [arxiv.org](https://arxiv.org/html/2404.02456v2)
80. [researchgate.net](https://www.researchgate.net/publication/384214399_PhonologyBench_Evaluating_Phonological_Skills_of_Large_Language_Models)
81. [aclanthology.org](https://aclanthology.org/2024.knowllm-1.pdf)
83. [pub.towardsai.net](https://pub.towardsai.net/understanding-tokenization-in-large-language-models-25402f51461e)
84. [youtube.com](https://www.youtube.com/watch?v=XQcvWC4Gxrs)
85. [sagarpatil2000.medium.com](https://sagarpatil2000.medium.com/tokenization-in-llms-the-first-step-every-language-model-takes-before-understanding-anything-1d5f2c9c7e50)
88. [verityai.co](https://verityai.co/blog/technical-exploitation-ai-systems)
89. [emergentmind.com](https://www.emergentmind.com/topics/seamless-spurious-token-injection-ssti)
90. [ijmrset.com](https://www.ijmrset.com/upload/299_Evaluating%20Character-Level%20Adversarial%20Robustness%20of%20Safety-Aligned.pdf)
92. [ziosec.com](https://ziosec.com/article?slug=ai-jailbreak-techniques-in-2026-a-complete-technical-guide-ziosec)
94. [arxiv.org](https://arxiv.org/html/2510.09947v1)
95. [occiglot.eu](https://occiglot.eu/posts/eu_tokenizer_perfomance/)
96. [pmc.ncbi.nlm.nih.gov](https://pmc.ncbi.nlm.nih.gov/articles/PMC12380774/)
97. [aclanthology.org](https://aclanthology.org/2025.findings-acl.95.pdf)
99. [researchgate.net](https://www.researchgate.net/publication/395354613_No_Translation_Needed_Forecasting_Quality_from_Fertility_and_Metadata)
104. [gpt.gekko.de](https://gpt.gekko.de/funny-llm-artifacts-from-training-glitches/)
105. [lucalp.dev](https://lucalp.dev/bitter-lesson-tokenization-and-blt/)
106. [fi-le.net](https://fi-le.net/oss/)
107. [grokipedia.com](https://grokipedia.com/page/glitch_token)
109. [lrec-conf.org](http://lrec-conf.org/proceedings/lrec2026/workshops/chipsal/2026.chipsal-1.0.pdf)

**Sources:**
1. [apxml.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQG3uO7rPvHUFNCn-CLnM_tCFNCjG5nr8pOVMXhEG965WhrFN2qKP-TdmExHz5k38RmPdJhpd8R5QE7-o71grEOsyMv1bxyemwhPyoKOeyD2eVHrMeDXYIZfJ0T17yeotmGv7GoQIty1HqRVO3_nCjlu5Bib4aGrzKK5ey0UcnY3W-raNbB1XSC92miCAI5ZcBd4SPyPBBhsX9nTT4IUt8JMYB9QOO8fNtI=)
2. [medium.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQG2kjsCtb2RCHFcbcfuul_-dr2r7P5Pv0HLci7HhAJQJ1AnMcp5uiUS8XHniFEGxtQoZEKOSNU3mC_mthpKJLV75yb0ziHLFKzmsLZBCEb9B32Fu6Sow_o76hEJ-jEx4C5uAeUu_uxyqOizisavisCNkAK55Zb_Ss0JCCM3nfna9P2hUcBxZNGSJ0U2TZso-N8tQZvudwONI2UvUWz9NusdkkS0IwZBWE0JOl9GED04tr0p)
3. [medium.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQHOO7MGuteVjYnQu5uEGDnJyESAvTgHlV7XPVAssu9m3ke8n5Els4goO0V7hD26bXj2SfuXA7x0Mvv7MnlrxKvpGauUGepte03USLzEKjhL1iD6m1e8Lysl6dWjIwwEbta9eg-DeIUeLAFSqULXBRYzwbg8_7Bq4gu-1VqpfWx1FdsPrtoCe6Iyn9waGT4KLYa2QMqOdFfLTCDMBJur5KA1A83Cd25KZrv2uhqsiSzo7jeaSuD-GSuNKqcr_7HYi1W0Bg==)
4. [substack.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQETJntF3J3BmJ5DaxHwTgZhxp592jJO04osxJYKUN6JSxM8JQQsDHHfNVt0kQlHL7SgBjIA0dqx3_C9Ku-mEINmlrgSFNAaeniGUr0NEBd5VDs7xv44Z_tNrqMuwLhQvS2lePcRngPlwQa_ODhBz3rsyyKTiuRivOL3L5k=)
5. [medium.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQERrQOi-81y5H4ddH8WdhSTFb75YSPn-whS6lZwySihFg5XP5cumDGRzGv0ujmdPxtmh2qlJOAqEDcfNn3srCacVqMxceNLIKrPVq4GFMrX-tuDsB0ShN2NpsUoTGWMgwdcsPPrcT-EJmQIsT12YejYayj_5pumqxRpaw4H4o8wXPgP6ZlFyxH_F42BnalImp9K6AeKfJwJiPYuE3TnuIdFiMRRr9XlVblcrVS6d-om)
6. [hyperstack.cloud](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQGRM3KPA6swAO20gKIcrnJY0R0dtPA2hTt5BwYutwdl0NNQieZADlG1E34rgOqo8pXuHEllnpyg30Coz0lAS6F1if01RSWVixDSEYwFGKLrwGzmFSws6Kmscg4LKjaUQRRDuzyoMoC8UYeVybQA9Vctws7WM7Gb8TNW6zQjP10EPwnwwRiZSFfComN5-O6ZrhUbIndOxt501BXjuEQBGuLtO4n9S4pA)
7. [rewire.it](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQELfNA2CIBXp91Bawz4oYADKLhNkqsCGlT2-Sf5uqd3BRlpRHeO2uHOaFYElu-Jw2Ls6C4KUkvUK-0w_Wm4dYh6favWao9deQhB4TC3vEwEmF4PFWr5zjcYHywnGhMpu3xr8w5ONO5PQaJn3fLyYG3a7pSWjlhZyiuBlUM0Sacx4rz8vA==)
8. [medium.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQEmh-PrV3qt2lXcIdOUmedS3euCzRVjm3Ns2hN3P3yiog4Ec1rrv5gJk42zr0-BySUejVYdVskMz0QV6JeE_nNsjGZTEuNBG849vuW_fHl7KaYw-s8r8jizRpxOHiN3MAh0FhaDfSUQ99DaIuC9UuzZQYJ-2DusxCSv92jqd8scSD6m4Pf5ELWmXG-i4QdbUeNlsOvcQ4aKgl8NWkPGqimfHhXzOrCM5pHFTgBnAm-P2GxuGhbT)
9. [traceloop.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQHKgTX60xev_Udf-9kMhBJWcw3GrcYYT4rpvUSGgE_mCajy_TOh1KAuvTISsZxQmtCI01vUACEqYwl7cehUHh92izONYxvvuEibe73fnDe8wTf5MTDY5BydxiiRsODIhjccZPZNZy08nfEB3wXwMIgutW42fkj9Eq-odWW_mqX2xDNn6q050w==)
10. [venturebeat.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQEySK3pMK9DuiD2oGuzQvClhHTSuEYFPjsoJpfX4vTz-rP16ptMXm8DN3pPL3tiRKvVpOLKCn_7IVorU9I_2uujMU16c1hoF_7PbD-STfP7FZnF46WXEk-Xq9OpFprEMaP-NJGKJBfmts5krXEq9gbTBIG-MXoJgskSL3i0v9Ntpmc2Z8dovJjMk1jqK20NBS_1gqE3cF4pZ9cPS2e4IEvU-UK_mMQ=)
11. [aclanthology.org](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQERuk6H69QzBNNcuFRLSyRMXYp1Po_GfKWsxLTxKzfmMKWopccudHwgUIBLib_LL_nHHKdSIBG4wvhGw7WBdN4T9gCfsU8tdYTYvom8E6Gl5TZjim2RdPh77JgaOS3psD7ykTHmCtvj31Y=)
12. [aclanthology.org](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQGlt96TEeipEDS4z9dcL-Zu3XYKO_M_Tv_EjVloWGo9kTAdA7kO4IeQYag52k2x61IhtUtMejTrGW3YqwPSZCSLh_tpLLCvon9qqdQayyNaMgF58tZVnMuKYVdOdlAx2YcWKFDV2EXJC5w=)
13. [towardsdatascience.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQEjyoWprNaRtxwOWUqBAJojZrg6sOj-Rwz-cMze0YPf6GVyuMoOfvk63fy-I7PxY1jVKxmk3FMCnzTeZJiUpTPFWk8Ju3zAoY-DTRCc6hxl0TT3v03_yF8lunwR7GdngBQL1GPoYb3rNnqDox_0mxCCQFXa9lDoenDJ-Ce_mcZk4z6d)
14. [arxiv.org](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQHYzrmowsTA81MunMT4yhlwRbJaxHtwbFH9eqLHfB-DUup5r8Mrb6pv3UCECPA3TdnNmjBOrfj1hn7C1TcKPmaaxIOVgyMjQIj6UkXkZRQizzJUI2K-raZO7Q==)
15. [towardsai.net](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQGhEjKeXdMCNPGZyrTk2sZHVlBo0GEcrvnJkqwRj29GIPR4QeDAifOyAN4cAvxWlz9YboNcDeYctvOyI-HvgF1yxRy0djzhf-yKKagqRoqEEvy716TrEDcRhVs4yl7xKRUJGplNVLmP_GU6qgA1ovo4edNHF6p_u_ZvYLsLOyq4Z7VXwEM2eATSaEq0tzZoTdw=)
16. [youtube.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQEF9DFUpLm0vghBlJM-tu6Cx5LT6U3OHXmi9spyBwS8AaQ-Ip5vzSnKvkGTfCM2Ez2NCsEMeqFjyJqg3Y9QPyTqGWf2j8eE2D7Jxw7Tpj5baFMG0nIvwjc74V2xDwUOHwE=)
17. [youtube.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQEqNv39RrLP-wwsLRczdzw4AlJ7dD8S5JKgYLw1dGqHxw4-c2fqKFvsQg_JBo5sOU1tcXXysC0_JaomBjbP6MaBz6k2KR_5RbP8-buzTn5mcmcsW0Soc49b3r5ji8u7mkLT)
18. [grokipedia.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQG8JaA-LapwhB3s5EOVS8td0n0ceAgG80PNxnEVAdiOJaV9p1U8NnuSXVobVk9d3doEBd2imSOwr09LIuSTkXnMNIT1iZyjha-Jny0Ww0b9_miK-wpbC360UxYkwwH9GNx2W5XAFp4=)
19. [github.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQF_o-0ikUvHX84Rk1-ucbXSnNxjJlDttef7LIvEIC_2h0zT6pxudw19DFdjNt3Nk5hIBg8-bW576MLbVQok38VoP7evQLt--vHYfx6c-y0Sc2uRyxDJNil6)
20. [datacamp.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQFlqIwrwUInhbEFgaN9f-QFyD9NCMz7dIbJZeV27rBpOHqXBtcWchZwTXdOp6ZWa_BkT30bTjPFkn74K1Ic4c0kV51ZFZZb964v0MpZLmZeHp2BH_6xKZmtYqCXTOlIDEz4ZOWswUeUj9ZlBN9IW-A=)
21. [openai.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQG9FYs7wcawX7cW9v1XiB8ZJPgDFriXkHHvLaXZwqQCUJljvhuRdHCQTKQZdfByvxzs53JZrgS-eI9vjgE_ub5rmCgPZDEstdifYgq4OuU8mtkKU54G4Ot3llUz3lksIPW3bZJA3hwCLJxkPlWjWZVeUWSL7FnPobFVfTK2XVVViBb7EE3lSc4=)
22. [arxiv.org](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQEwZK37QTm5tvRufUvI7LkdBfvbsYdfY4kD3mqkpAbk3TC1nQv1BUxarFNnINUS5T_5bE-bU2hJVNFWhiFH9IBWqcuMw0bG6GexTXlAukm-UzevS-10O8pBsQ==)
23. [towardsai.net](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQG0guU-I7jyReiUarpLyZBj4t80iNr06fYlp9aHJECury-znoHqqkCHwgVUWdBqi0ofDyFLlOOgmjkwtQv9zIIUOO6DG0zKHrwSBy0ViZLLYQvwmfCVCDBN4bG__r-P9UfLYyUYkZxeskFEj4TTODnVQbjLFjw4nfUcQQ9SD6J0-4jlUdLp8oD4)
24. [galileo.ai](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQGig0cuy7QCIr7yi6IaK-Gip0fV6SnfEuPZNhWkUA4P6Qp9s_fzvhgZLo6XUR9pUMdcYkfQAZ9zrKOYKjZz7m_H5NcvlwMSfH1U87ig_JinHz_o9ZKIfvDz1EqCk83AcExDftfqypBbQfI=)
25. [medium.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQFUd_i2A2z-cV3qGEgouAdAxLpgXo_gCted3onFzvApspUSu6LepFRIGxWeZvCOSe6kzo0WQmIuMBusAHW5zIh3tgE_nhPVIL7XMxwWYamVusD8IB4pViuOXVf8IyJLORjgMosMREJOLU3wq5bsq-2XzjeNQcpbiZw8ny-QIVhjxo-yA_XcZvQ4ebHC)
26. [nih.gov](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQHNFO7Zpn_06yZxg6LCafsUDQG32ES7gB2P4i9_Dq3WnRG8TB_G-fq3i_TKri7kaMm5ZHSb1Tf7GlWuD0tzl6zU86oDUj1WJftETyHD-5EgYGhMCGDtYnebEf1som5MvFfZu8YB8GD4Pw==)
27. [arxiv.org](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQHtrFFKeRS1k5G0dLVfzaUoct9xGbt8DTR1FTcTZBlsZENfHLqzJzFRwth8OKAzgSGz8d36Cy0WXKh7X0uIQ5t16Wk5zBY0AYdGyXwopaNwMBS3KGXciDHzXw==)
28. [runpod.io](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQHZuejLkSMYK5ll1xTXyF-z95Y2BsKLLfM0xRql5FZkXzAyLmcAd7bjyHwfgQyap6DlihTcqOvIYxp4-hoJWJBY-mFw2Etq92kGvUgOnUqLkru5NL-ZeTd8baeQ7FPS1dvnOTmHsJAwjNHEDQkn)
29. [secwest.net](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQGhvXO0_dqC2LjVBdgYjKw1wv3LJ0zl30WYV6j7RfhZiQ7er9ubjacNNjcWXGGy2hEmoZ9WkaUmqqdlWzfrpe-De7Ew6qJs3rvU_x085NSXh5pHhKAUCY0M)
30. [medium.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQHfZgluRjN8-05MR2r0lpbb4s0ozSxi6cuu-O5AA3Hgt7qMVyj99kjvffI84JScSItaMu1EyETlRneGA_pha1ALQjoAoczJpX-v2J_BayCYfySyGCZJxUuKnRWKrLV16DuBEI6zOoL8Kt3ATYmhbYVdwMcTNpmV-2ODyLaSHQl7VgFNhcXqbmyntFwtsZrZzQfenqSibZnSAGQnd0oUizL2JnTT)
31. [researchgate.net](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQGFHcRgWK139o6WYk_6ig3hbnAAeiv7NEJVqUEviH4Jlm_l372jFtzTpEY_hv_K7-jZqUiUVrLAuob-WIdYbjfdnmRT3hmjw6wFkYzGH0Tkll_NjVN4RFbhLj_wekEdpdNxEQ23j6ZE70_ekE2lgO9A7pC5D3I_B3n7D3xK1whgnw9chnoGRmgHgrR3T08IO99DJMMGNHFQ-s4PJrIdeojvdIM=)
32. [emergentmind.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQHbLzGmeRtCZNUGe95B6K9JAfxosB8idHAWQfUg85x3FUJZKBr9w1QvaN0G1CYDp3KhIyK3f-dH6AsFkM5CQcMGHCuanabWvKThT32fXejOaghqgXdGMIRTU6U9LGaUH3MDh79_3hEh1k7Py7g1X_N0jC7STQ==)
33. [arxiv.org](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQFWI3JOaf3aUXl-XBzu0f0iqIvivycaffj4QYopnx0w-sqg41Nd_0qRzANhZvKxMSmmjGy2Itex8lowpXL71PnoRqTPwpJg_iK0d3aat7UC58jeKktbFyJyuA==)
34. [researchgate.net](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQHOhEzRmYFbPYLMCJQWHtodTTaDemHipZRV_KvWJTWDrWD7UCuJ88wu6XQRl1UeTn3mfwmogswy8aMVzh4n-o_dsr1dC2YpiUFQ1goxIchfoXNREmW_7wTt-BZnPsSDDDnoPhwAMq7XIB-CDSVSKKY4vmpSuNhmTZToyolxzslS-eWbK8hjV2CHFdDNb1cp3mjdB8cY8dz9wzWF5x80N5Lx3rL3bagcRT8tooumUANb)
35. [aclanthology.org](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQE7mHQDFmPInE7eaQgmYN3ltYqe0TDSTUi4fLN1HpRmNmrDyjZ09Pn4ytgnkShmzm0exMYMa36eqP-0LTQfrodxARr_MTAxjvFUDoC31FXbIPphal0c47vtXcFRmrgjB4DB)
36. [arxiv.org](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQEYo7jsfCE8aERkYrUzhMMF9ovZyGurBcKNQ9Uyytn2rsBJ7wQRNuoZwhGHHK3K2Q2-uCndo8S8VoV7VDgpw8HjKinboJwui5WB-ZFxmYF6pVjJt3-W2oX5Mw==)
37. [emergentmind.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQH0zAa_d0pilKRg1rNGELF_AVsxHzviJWJ-Y7ZEhLSekBqI4uFzshBSVcFRzrH6AndDHaX3dZi1fICtpeyDFAZbw2WKBrXZCQSq_k_TXEIJBY7RuZ1E8j9eO2nblMBZfhaKKsA_pB7OWiGYk4og22iAgpY=)
38. [cureusjournals.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQEItCr4uWCZIwZLB8ZF7youclqFqVJUijCGoxXHYdslt7Lpgi5xltSajmx9ZmTer-XTl5moKr1i7lVcg2byG61WEv4ZHmyKqN7QTQyzfz-Yi9WBmBhA_x-ndk9CoVjfZQnqmXuk1hJUUfc3_szsn39F623UbWqlEbJaJHpJ5fSyHGsZqg0i7zvt4TkBHAgEvBb3L5crr9g-XDcfeVQGGIPRKwA-CZafa24uL82yW1ciUV1Ji3POsKBcplCsdu3_tBmtnZowwVzS6hZFJtP2m7MWTuc-AbLMlz9JZ0DY99Hwh8-UjMfZ991_dYFzVA==)
39. [arxiv.org](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQFiunw-nV0jpIKqNTFUMDubBMZQumYpj0FAOFUQ_z_o8K-h6gD7n3MR8AoD6NNmpWwdsfIedQEjMQ_T9ddC7dDexJhpjvmpM-bnxvkiSrfeIXoe3BeY1a3m0A==)
40. [occiglot.eu](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQFxf3aVcfpK84NRRYku5sKmQ5Ut3pEj4d5cV2d1PWFezHZyHDv5aJH7nmt0_KUwpe0l3-INXZErOomVJlPJRAXLVeHXULaEe4kOekc0dmSg8m2geN9CnowNJKq68IBfoQs7y5BFNT1gMQ==)
41. [emergentmind.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQEcXhn6m3kTB3YR_lzw8FR76pelWoDZnXM2lYJoqe3AWIcqpGJKbU9M-UQ7m-gIJb938RvNqW4qpfphdH-tfl4PMANDcM0_3MN98ViBQid8WkQKpgIf2PEk0VcseEFQtO-6euVESckcTs8DAcwmWpBP7_mhiJwh5KO--JQ-R_I=)
42. [reddit.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQEl6_U2KY41aCMmUuHCtjEyUga9-KE4_vbwzfmVdpK0TB-Fd9ZRzCyAb3QXPEo5zRUI8962et4tWVZtl8CUcL0gN27G1RbqD53vdfGV62PMBcvwrv_uhHx_S7v4jttLGwyHc7KPXYULP3CeMmtLYjWsjBKkU1miIt65auFR19DqzxBEOr0qN8YU9oNWW0C-jSQZzXxVbg==)
43. [aclanthology.org](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQHCpCuKsV7nrlQmkGcs-tQpNROZJEJk38TZBfbetr7OIav5746C1dIsyLLpTq7LxIlk6wdvUf8s6qgDQGxJwEXd2ncfo6kW-U_PNP8pFCXzgVqEE4fwmYGQlP4QXiqodbDintB-d_5e)
44. [lrec-conf.org](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQElVEvKtskhpBHuC2gRVFzV1sog5RLgXS_5mkG5DV1-j0bn1iUqPdor0ujlZgJQamhCHu8sPaTxzDzB4MA_HJknqHP_-HAYJuCNYHZb-ttZLE-HycD1HvRchpsCV4ee9he_hv0AOBsfBqhrLf-JNYd7ST0tppeebTbclBI-UIQ7ckFPFoM31w==)
45. [researchgate.net](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQFtyflhC_O6zLb1zJNhJIa0YK7C-8PVxqEBtgUiVcIIDrTOLrd_kp_l0eJJigzzcXcKcUqBPBJp9KDqPKCHlq1B3wna7aW-rlnVPwYBpW9ivSLiI7krigFiuIE9DSjaxrcG_c05nMWB552u9BsKYFcYUTH0PZB6vXbLl6RDcsCNx8ZfHSPvvXNyZwX5wMZDg9XUfbqGjoU05BIoNgSLttMELMXN_NXSzQMIQZ08iDs=)
46. [vellum.ai](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQF0Iw4Hrh2olQ2k0WwMkvP9Bmeo-940JH44zd4DMoXKEBYFM-EMTDYIqZGqjllOQSpRYtIfHm3fVbiJa50Va0wsvi8aHWqFXLrFhyvXXf4O7fhelUlZALc3vLBBC1NTgnEF5WVCwKvaYLzAbhOqE-Ivq7XC-0uYzYv4)
47. [arxiv.org](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQHWrau3XIX1mSfNuhHE-vVEYbor5EzIPpvhiSJTSE6vfTGP5owGZ0NXyxFUldWWaIeKvXjKK920H9yhyT3d8t4DnvGFKbGFnqgIgnSHpED3gVeqRvHwqU-tAw==)
48. [researchgate.net](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQGW9Imqg99Au40p2026ZdwlCfZikrwar9DYiH5y62yD4TF5O8SaTJ947VQWdrmPiLnAdNw88dTfLElw32kyRXQA6_zvz-9fBCHwmgQPx9GD81mkm_nZwHM8L94KZPS6qNz6ZD_rEF2STIGr4x2HhmXS4Cxt1WQOKqnuO-KtDx3pB3zredod94wzFWlEw3fz7lOsFMCW8P-oyz6Jqos8xdlJDqRX)
49. [arxiv.org](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQFZzt1d7btPFdkXTNI_2AVt-nT_ION7LE_h3-KwSCkQHIwxhUJMAPl79RKuGcuwCooF-PFPspJlo196j8S8V-Z4B1JKv5GGGhoGiedbTOaVrZBMPuF4_97qOw==)
50. [kith.org](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQHMMhi9KrwVY-0FzzlkP3AIJeXTfMn6dJ_Yj2Xk0V4cp5gQbOWzTVYjsELGhatiNvdl29eCw9Y3R8ctqxxcrv_hgCUw_sg5GYAYZX_tutIqmCvceVQxO2yP2s1x6fCwoWnO9fq8a7D-qISiqR4bcjuMvAIgQZBQvkhynGKbgF1v7V_V2fP1Qw==)
51. [toxsec.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQE-oGCmlZwsLiDYLyASuK0dSCsGxr8s4oclmIoGAWtb0C18p4Y3gcGmV4zrSEm9wdhUViUHAIw4wCg1_Bw9_GRIKBHEp9y24hWb-wlAv6HrcOI5lV_ywGDLz8EzdVY1m75WZ4jxuteIxojS3vd2t_A=)
52. [aclanthology.org](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQEnhMUDHUbqDOJhbgsYmnXjXBj4JNckeemk0o1ov1kbuV8cGcRsu7ljUY_Fo_mi5s4xGVguocrRohCNyb0gEWqUNMx1oEvfLx2FaJ7_H80W7fEhzIb8SYBjCYMmx9dmBrVgoSY=)
53. [gekko.de](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQHbGcMRw22kH-3I-lSgV4IKHLfl5QDymnZ9j4WcrLkr2PAR2f9mGBvmZhRyrrEHQBiZPsUR2SFIavsodCO8Z5E1Fp4cztMuWeYWoJVezCtilMOBNoFSR6NrX8DB9ut3bItf4RfYptRCb6z7f9OLp4f6xL6BOcaS)
54. [github.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQFBv07VK7RfAc4xvofh-ieww8mYyDiFF2EgzxoEWlPIIUU2tVngOGn44WfHNh4SUuasQ8xCvg8TOZE1sed9cqZSN28fK98TS_-1LQXVZpnViRwUHJKLalU2bZc4I4XiDoc=)
55. [grokipedia.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQGLb2JqTAcb3b1kfXmoHFZyxLLYmaTfmaLl5HE43jOLrebCzbLGmhiWHwuQNVKIUE5nE805C4cxAnKLwuAVlRumWIKAggWBabzq4ODvB0n4dw64lr6BFv4L-HCb-txU)
56. [arxiv.org](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQGhziElbGEZWB0kj-PHXtlITJigKxW6oepfBLz_pZsV7tRm8SIa53zp7b8RZCbnevNnwLcOgOxyBxW8xbm0J9odBXsZrRPZYMCgsQKG9AXykRIskNQ2n7YfLg==)
57. [fi-le.net](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQFv7nVjRZi_RRuBxxBILfZw7QpUciyaztfVV1t7AiuqLr-n_BpgBXaUEdBraec77QXWeOelWmrS3QBL-bda_oi9_ln-Pfmksq_P8dVB)
58. [gitconnected.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQHFJyQO6SW1KQZjyAIiG08CBDtpkEkTAk6fkv0lSXmNxlp9mlsRcSns24w23JEk0dc4nHk7HU9p-sNJJ3PGfKyDzdtov0JTwln4ieC9DRpZFkd4Wgn7KRxRDvWAXYC6FIaIz88zdLJQ4m45BpMG5Ih-s-TX2SpMdECJDxVZQPM5d2DolwSIxYzMwm2sMqRc92S0944_p0oZB0oPaO-Qec5vSVT3)
59. [verityai.co](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQFt0O-muX9oyKq4RzCxqKGayWKAdK8wov6zWH4KSwWUyUzn6wTu7NRnwg42ppmmK4YMpC2oDG52nUtLBxg1jMWa-wVsvoAHjltGtva-pdk25J8A407JcDQ6bU1UUJekSL-Sn7WCkV0tn7G9L6EWNBsL)
60. [ziosec.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQFSohzRfRU1i5nqZWj93PH4SYfaEHE9sM9u8nOaXXu1UGo_psbQ_L8TEfJA9Ipqixe-uCQ16XB2v6-MfC4XWP1FPbJecHBQJGT0Qp2wbB8r-B0ShjGXzQANqVE2m0qAt_ckim4UeQgmzU8X9rEYsWjddAFnA_YG2qntbPNy-BuvoURUvmQ_dlOY79Cab93QI-oIEKmMqffe)
61. [ijmrset.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQHMunlkm-5-eHlmqlxwjNU6hfaDedRv5w6zzATXEhpbNa7R88PXxE1KYeg3aW6kPmSWZeU4NC4_9JT5qm8XlDTReBX7vridRfwzR1TLryKmf_AntWaG6hq5U37_WYmkCczpfJIjpNY5p1jmtyKDw-qWYUjJk-a0aMQe2VR_3IFaE_m2V2XSeUa4pYuI2PLPbAbQFGKc6a2nQsjQ9QatqeWHqCUK1sUEoKmjmg==)
62. [emergentmind.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQFzpcXA4SWMgY0nmPzM79xP7wMsYf-rxZ_7ydl52fRENJpnneTCxgnEhZsrxem70uI1jMhGBYVlby8WpIZFa5dx0UVQiBX29Zraj94Btk2TRabi0MAOo9AUtSV9ZaYKI9nQC-omBHWjOZSD0eWgYnSDJR5bFoWmK7i6aGA6A0eINQ==)
63. [llmsresearch.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQF5XduM3BcEbInsJ7dQar9JphJVDQzTSCoY9oODSF-o0GTllfZzNVwPas7KrrQjh1JYVr3xcFWPIhnwpT9L_w90DGROScBdsH-PwzboA_IJ3bDOoiBQ1Zg2cIvSwr4irWLWiVOVkJUBC35PwyAIekTVG6kHI6bTBW2yTLwA3Tulvx_uEBaUrMHrR5ZAfA==)
64. [aclanthology.org](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQFwUFjY-gN42SrIjnrfk1PelSHJrodgwknM-7V-uUalR4IQPtozfFupSC-m8-C3prVT5XBB9iY49DsKv-AKlQB9k4fC7UB_CbmfzSNCSWmdE8iJ-3M5CYM3GeVgNLzxTV8DRwRi9YwsaI1_)
65. [gitconnected.com](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQGqPwwOUlc-nWFodeFvMNyjY0m0OCP8PBC687IhA81Z-99z_MSStYc67VV2ualGXQRr0cj3GQHN9mVFUtwJ2iRdni-zq6Ic-H05bVS_RxgYW61K7h7YqdcvlaqkN1-CxbMZuxsewK6uMxfPJO3kP98jR6XxWpCbVlID-pL4j8kIl_A=)
66. [lucalp.dev](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQH0NWWMbaD9vTcE3vNu_fYTr-XN56V0NNPv13cZMcnae980j1agkVk9pjp-KE2cW0qviU3eBqjXnSRCco8485-7gr2kSDdi8CWHEL3iG3sJb8-bN7K5iVKKFReIeWFjp6ynG_Z3XanhYNWgpNs=)