Revert "Replace the tokenizer with a flex-based scanner (#3846)"

This reverts commit 99eaf5faf9.
This commit is contained in:
Ashe Connor
2017-11-10 10:22:47 +11:00
committed by Ashe Connor
parent 0f4955e5d5
commit 0698b0f36e
15 changed files with 202 additions and 8914 deletions

3
.gitignore vendored
View File

@@ -8,6 +8,3 @@ lib/linguist/samples.json
/node_modules
test/fixtures/ace_modes.json
/vendor/gems/
/tmp
*.bundle
*.so