baby636/tokenizers

master

tokenizers

Grade	Name	Complexity	Churn	Issues
F	bindings\python\tests\bindings\test_pre_tokenizers.py	23	5	14
F	bindings\node\lib\bindings\raw-encoding.test.ts	43	1	105
F	bindings\node\lib\bindings\utils.test.ts	44	1	63
F	bindings\node\lib\bindings\post-processors.test.ts	20	1	29
F	bindings\node\lib\bindings\tokenizer.test.ts	54	1	131
F	bindings\node\lib\bindings\models.test.ts	40	1	38
D	bindings\node\lib\bindings\decoders.test.ts	12	2	20
D	bindings\node\lib\implementations\encoding.test.ts	14	1	22
D	bindings\node\lib\bindings\pre-tokenizers.test.ts	13	2	22
C	bindings\python\tests\bindings\test_decoders.py		5	5
C	bindings\node\lib\implementations\tokenizers\base.tokenizer.test.ts	10	1	16
C	docs\source\_static\css\huggingface.css		1	6
C	bindings\python\tests\bindings\test_tokenizer.py	26	1	6
C	bindings\node\lib\bindings\normalizers.test.ts	9	1	16
B	bindings\python\tests\bindings\test_processors.py		2	4
B	bindings\python\tests\bindings\test_normalizers.py	19	3	4
B	bindings\python\tests\bindings\test_models.py	16	4	4
B	bindings\python\py_src\tokenizers\tools\visualizer.py	43	1	4
B	bindings\node\lib\implementations\tokenizers\bert-wordpiece.tokenizer.test.ts	6	1	9
A	bindings\node\jest.config.js		1	-