← Home

@dqbd/tiktoken

JS/WASM bindings for tiktoken

22
Versions
MIT
License
No
Install Scripts
Missing
Provenance

Supply chain provenance

Status for the latest visible version.

No SLSA provenance npm registry signatures gitHead linked

Without SLSA provenance there is no cryptographic link between this tarball and the public source — the axios compromise (March 2026) relied on exactly this gap.

Maintainers

davidduong

Accepted risks

Findings the reviewer chose to accept rather than block on.

SourceRuleReasonAccepted byWhen
source-diff source-size-tripled AI (source-diff): Size increase is from bundling BPE encoder data files inline instead of fetching at runtime; expected for this package. ai
source-diff obfuscated-file:encoders/p50k_edit.js AI (source-diff): ESM variant of BPE rank data file; same base64 token data, not obfuscation. ai
source-diff obfuscated-file:encoders/r50k_base.js AI (source-diff): ESM variant of BPE rank data file; same base64 token data, not obfuscation. ai
source-diff obfuscated-file:encoders/cl100k_base.cjs AI (source-diff): Pure JSON data file containing BPE rank tables for tiktoken tokenizer; base64 tokens cause long lines, not obfuscation. ai
source-diff obfuscated-file:encoders/gpt2.cjs AI (source-diff): Pure JSON data file containing BPE rank tables for tiktoken tokenizer; base64 tokens cause long lines, not obfuscation. ai
source-diff obfuscated-file:encoders/p50k_base.cjs AI (source-diff): Pure JSON data file containing BPE rank tables for tiktoken tokenizer; base64 tokens cause long lines, not obfuscation. ai
source-diff obfuscated-file:encoders/p50k_edit.cjs AI (source-diff): Pure JSON data file containing BPE rank tables for tiktoken tokenizer; base64 tokens cause long lines, not obfuscation. ai
source-diff obfuscated-file:encoders/r50k_base.cjs AI (source-diff): Pure JSON data file containing BPE rank tables for tiktoken tokenizer; base64 tokens cause long lines, not obfuscation. ai
source-diff obfuscated-file:encoders/cl100k_base.js AI (source-diff): ESM variant of BPE rank data file; same base64 token data, not obfuscation. ai
source-diff obfuscated-file:encoders/gpt2.js AI (source-diff): ESM variant of BPE rank data file; same base64 token data, not obfuscation. ai
source-diff obfuscated-file:encoders/p50k_base.js AI (source-diff): ESM variant of BPE rank data file; same base64 token data, not obfuscation. ai
source-diff obfuscated-file:encoders/o200k_base.js AI (source-diff): ESM variant of the same BPE encoder data file. Same format as existing encoder files; long lines are base64-encoded token rank data, not obfuscated code. ai
source-diff obfuscated-file:encoders/o200k_base.cjs AI (source-diff): This is BPE encoder data (base64-encoded token ranks) in the same format as all other encoder files in this package. Long lines are data, not obfuscated code. ai

Versions (showing 22 of 22)

Version Deps Published
1.0.22 0 / 0
1.0.21 0 / 0
1.0.20 0 / 0
1.0.19 0 / 0
1.0.18 0 / 0
1.0.17 0 / 0
1.0.16 0 / 0
1.0.15 0 / 0
1.0.14 0 / 0
1.0.13 0 / 0
1.0.12 0 / 0
1.0.7 0 / 0
1.0.6 0 / 0
1.0.5 0 / 0
1.0.4 0 / 0
1.0.3 0 / 0
1.0.2 0 / 0
1.0.1 0 / 0
1.0.0 0 / 0
0.4.0 0 / 4
0.2.1 0 / 0
0.2.0 0 / 0

v1.0.22

1 finding
LOW No provenance attestation provenance

Package was published without Sigstore provenance. Consider requesting the maintainer enable provenance via CI/CD.

v1.0.21

1 finding
LOW No provenance attestation provenance

Package was published without Sigstore provenance. Consider requesting the maintainer enable provenance via CI/CD.

v1.0.20

1 finding
LOW No provenance attestation provenance

Package was published without Sigstore provenance. Consider requesting the maintainer enable provenance via CI/CD.

v1.0.19

1 finding
LOW No provenance attestation provenance

Package was published without Sigstore provenance. Consider requesting the maintainer enable provenance via CI/CD.

v1.0.18

1 finding
LOW No provenance attestation provenance

Package was published without Sigstore provenance. Consider requesting the maintainer enable provenance via CI/CD.

v1.0.17

1 finding
LOW No provenance attestation provenance

Package was published without Sigstore provenance. Consider requesting the maintainer enable provenance via CI/CD.

v1.0.16

3 findings
HIGH New obfuscated file: encoders/o200k_base.cjs source-diff

Newly added source file contains lines over 3000 chars, suggesting minified or obfuscated code. New obfuscated files are a strong attack indicator.

HIGH New obfuscated file: encoders/o200k_base.js source-diff

Newly added source file contains lines over 3000 chars, suggesting minified or obfuscated code. New obfuscated files are a strong attack indicator.

LOW No provenance attestation provenance

Package was published without Sigstore provenance. Consider requesting the maintainer enable provenance via CI/CD.

v1.0.15

3 findings
HIGH New obfuscated file: encoders/o200k_base.cjs source-diff

Newly added source file contains lines over 3000 chars, suggesting minified or obfuscated code. New obfuscated files are a strong attack indicator.

HIGH New obfuscated file: encoders/o200k_base.js source-diff

Newly added source file contains lines over 3000 chars, suggesting minified or obfuscated code. New obfuscated files are a strong attack indicator.

LOW No provenance attestation provenance

Package was published without Sigstore provenance. Consider requesting the maintainer enable provenance via CI/CD.

v1.0.14

1 finding
LOW No provenance attestation provenance

Package was published without Sigstore provenance. Consider requesting the maintainer enable provenance via CI/CD.

v1.0.13

11 findings
HIGH New obfuscated file: encoders/cl100k_base.cjs source-diff

Newly added source file contains lines over 3000 chars, suggesting minified or obfuscated code. New obfuscated files are a strong attack indicator.

HIGH New obfuscated file: encoders/gpt2.cjs source-diff

Newly added source file contains lines over 3000 chars, suggesting minified or obfuscated code. New obfuscated files are a strong attack indicator.

HIGH New obfuscated file: encoders/p50k_base.cjs source-diff

Newly added source file contains lines over 3000 chars, suggesting minified or obfuscated code. New obfuscated files are a strong attack indicator.

HIGH New obfuscated file: encoders/p50k_edit.cjs source-diff

Newly added source file contains lines over 3000 chars, suggesting minified or obfuscated code. New obfuscated files are a strong attack indicator.

HIGH New obfuscated file: encoders/r50k_base.cjs source-diff

Newly added source file contains lines over 3000 chars, suggesting minified or obfuscated code. New obfuscated files are a strong attack indicator.

HIGH New obfuscated file: encoders/cl100k_base.js source-diff

Newly added source file contains lines over 3000 chars, suggesting minified or obfuscated code. New obfuscated files are a strong attack indicator.

HIGH New obfuscated file: encoders/gpt2.js source-diff

Newly added source file contains lines over 3000 chars, suggesting minified or obfuscated code. New obfuscated files are a strong attack indicator.

HIGH New obfuscated file: encoders/p50k_base.js source-diff

Newly added source file contains lines over 3000 chars, suggesting minified or obfuscated code. New obfuscated files are a strong attack indicator.

HIGH New obfuscated file: encoders/p50k_edit.js source-diff

Newly added source file contains lines over 3000 chars, suggesting minified or obfuscated code. New obfuscated files are a strong attack indicator.

HIGH New obfuscated file: encoders/r50k_base.js source-diff

Newly added source file contains lines over 3000 chars, suggesting minified or obfuscated code. New obfuscated files are a strong attack indicator.

LOW No provenance attestation provenance

Package was published without Sigstore provenance. Consider requesting the maintainer enable provenance via CI/CD.

v1.0.12

11 findings
HIGH New obfuscated file: encoders/cl100k_base.cjs source-diff

Newly added source file contains lines over 3000 chars, suggesting minified or obfuscated code. New obfuscated files are a strong attack indicator.

HIGH New obfuscated file: encoders/gpt2.cjs source-diff

Newly added source file contains lines over 3000 chars, suggesting minified or obfuscated code. New obfuscated files are a strong attack indicator.

HIGH New obfuscated file: encoders/p50k_base.cjs source-diff

Newly added source file contains lines over 3000 chars, suggesting minified or obfuscated code. New obfuscated files are a strong attack indicator.

HIGH New obfuscated file: encoders/p50k_edit.cjs source-diff

Newly added source file contains lines over 3000 chars, suggesting minified or obfuscated code. New obfuscated files are a strong attack indicator.

HIGH New obfuscated file: encoders/r50k_base.cjs source-diff

Newly added source file contains lines over 3000 chars, suggesting minified or obfuscated code. New obfuscated files are a strong attack indicator.

HIGH New obfuscated file: encoders/cl100k_base.js source-diff

Newly added source file contains lines over 3000 chars, suggesting minified or obfuscated code. New obfuscated files are a strong attack indicator.

HIGH New obfuscated file: encoders/gpt2.js source-diff

Newly added source file contains lines over 3000 chars, suggesting minified or obfuscated code. New obfuscated files are a strong attack indicator.

HIGH New obfuscated file: encoders/p50k_base.js source-diff

Newly added source file contains lines over 3000 chars, suggesting minified or obfuscated code. New obfuscated files are a strong attack indicator.

HIGH New obfuscated file: encoders/p50k_edit.js source-diff

Newly added source file contains lines over 3000 chars, suggesting minified or obfuscated code. New obfuscated files are a strong attack indicator.

HIGH New obfuscated file: encoders/r50k_base.js source-diff

Newly added source file contains lines over 3000 chars, suggesting minified or obfuscated code. New obfuscated files are a strong attack indicator.

LOW No provenance attestation provenance

Package was published without Sigstore provenance. Consider requesting the maintainer enable provenance via CI/CD.

v1.0.7

1 finding
LOW No provenance attestation provenance

Package was published without Sigstore provenance. Consider requesting the maintainer enable provenance via CI/CD.

v1.0.6

1 finding
LOW No provenance attestation provenance

Package was published without Sigstore provenance. Consider requesting the maintainer enable provenance via CI/CD.

v1.0.5

1 finding
LOW No provenance attestation provenance

Package was published without Sigstore provenance. Consider requesting the maintainer enable provenance via CI/CD.

v1.0.4

1 finding
LOW No provenance attestation provenance

Package was published without Sigstore provenance. Consider requesting the maintainer enable provenance via CI/CD.

v1.0.3

1 finding
LOW No provenance attestation provenance

Package was published without Sigstore provenance. Consider requesting the maintainer enable provenance via CI/CD.

v1.0.2

1 finding
LOW No provenance attestation provenance

Package was published without Sigstore provenance. Consider requesting the maintainer enable provenance via CI/CD.

v1.0.1

1 finding
LOW No provenance attestation provenance

Package was published without Sigstore provenance. Consider requesting the maintainer enable provenance via CI/CD.

v1.0.0

1 finding
LOW No provenance attestation provenance

Package was published without Sigstore provenance. Consider requesting the maintainer enable provenance via CI/CD.

v0.4.0

2 findings
HIGH Missing gitHead — previous versions had it provenance

This version has no gitHead field linking it to a source commit, but previous versions did. This suggests the publish environment changed. Published by: davidduong.

LOW No provenance attestation provenance

Package was published without Sigstore provenance. Consider requesting the maintainer enable provenance via CI/CD.

v0.2.1

2 findings
HIGH Missing gitHead — previous versions had it provenance

This version has no gitHead field linking it to a source commit, but previous versions did. This suggests the publish environment changed. Published by: davidduong.

LOW No provenance attestation provenance

Package was published without Sigstore provenance. Consider requesting the maintainer enable provenance via CI/CD.

v0.2.0

1 finding
LOW No provenance attestation provenance

Package was published without Sigstore provenance. Only ~12% of npm packages have provenance, so this is common but not ideal.