Joshua Peek
d92d208a45
Fix tests for pygments.rb 0.3.x
2012-10-07 15:39:02 -05:00
Joshua Peek
ebd6077cd7
Add wrap flag to text languages
2012-10-07 15:34:13 -05:00
Joshua Peek
10e875e899
Print out samples db diffs
2012-09-24 10:07:08 -05:00
Ryan Tomayko
bda895eaae
Test Mac Format detection and line splitting
2012-09-10 01:52:30 -07:00
Scott J. Goldman
e415a1351b
When testing if a blob is indexable, check size first
...
Otherwise, charlock_holmes will allocate another large binary
buffer for testing the encoding, which is a problem if the binary
blob is many hundreds of MB large. It'll just fail and crash ruby.
2012-08-31 22:47:19 -07:00
Joshua Peek
b85aeaad3e
Inline mime type lookup into blob helper
2012-08-20 11:33:16 -05:00
Joshua Peek
64f3509222
Remove other mime type hacks
2012-08-20 11:29:22 -05:00
Joshua Peek
630dca515a
Trim down mime type overrides that are old or now pushed upstream
...
Related #206
2012-08-20 11:11:42 -05:00
Joshua Peek
220108857c
Skip emiting comment tokens
2012-08-20 10:34:07 -05:00
Joshua Peek
96267e8696
Sort test assertion
2012-08-03 15:11:30 -05:00
Joshua Peek
16a67cb852
Move shebang detection into classifier
...
Fixes #203
2012-08-03 15:07:36 -05:00
Joshua Peek
fbbaff09cd
Stop treating text as a language
2012-08-03 13:55:51 -05:00
Joshua Peek
6014bd015e
Change find_by_filename api to return all matching languages
2012-08-03 13:53:12 -05:00
Joshua Peek
e5d302459f
Fix tokenzing empty strings
2012-07-24 11:49:29 -05:00
Joshua Peek
53300ca581
Add brackets to tokens
2012-07-24 11:28:27 -05:00
Joshua Peek
f5705eaf38
Parse float tokens
2012-07-24 11:23:06 -05:00
Joshua Peek
059f661eb6
Rename Max/MSP to Max
2012-07-24 11:03:09 -05:00
Joshua Peek
6ac9138aed
Remove pathname
...
Closes #207
2012-07-23 16:50:30 -05:00
Joshua Peek
fca6a9a9e1
Fix md5 nesting
2012-07-23 16:19:56 -05:00
Joshua Peek
7b6caa0f6c
Rename samples subdirectories
2012-07-23 15:52:49 -05:00
Joshua Peek
5cda67530d
Seperate test file for samples
2012-07-23 13:55:30 -05:00
Joshua Peek
6842044b52
Store md5 in samples db
2012-07-23 13:50:35 -05:00
Joshua Peek
bf944f6d1a
Make classify a function on the Classifier
2012-07-23 13:47:15 -05:00
Joshua Peek
b9779e805e
Move outdated check to samples
2012-07-23 13:21:30 -05:00
Joshua Peek
80e8ee7ce6
Rename Sample -> Samples
2012-07-23 13:15:27 -05:00
Joshua Peek
0c9a947f39
Load classifer db into sample data hash
2012-07-23 13:13:52 -05:00
Joshua Peek
3172bf5b46
Remove gc for now
2012-07-23 12:23:20 -05:00
Joshua Peek
5b28336d56
Move db verification into tests
2012-07-23 12:21:26 -05:00
Joshua Peek
b7f58d96cb
Compare md5s of dbs
2012-07-23 12:17:32 -05:00
Joshua Peek
d6fb95b06f
Add nested md5 digest
2012-07-23 12:13:08 -05:00
Joshua Peek
36da52e383
Move specific filename samples into their own dir
2012-07-20 17:20:45 -05:00
Joshua Peek
144655f2c5
Require explicit primary_extension
2012-07-20 16:54:49 -05:00
Joshua Peek
eb2c07e511
Sample returns simple hash objects
2012-07-20 16:17:37 -05:00
Joshua Peek
7292bdc180
Change Classifier to accept language name Strings
2012-07-20 15:52:27 -05:00
Joshua Peek
bbc5225086
Pending samples work now
2012-07-20 15:36:48 -05:00
Joshua Peek
2637d8dc55
Add tokenize helper to Tokenize class
2012-07-20 15:14:58 -05:00
Joshua Peek
175d4244c2
Extract single and multi line comment parser
2012-07-20 15:06:21 -05:00
Joshua Peek
d063089430
Add coq comments
2012-07-20 14:45:19 -05:00
Joshua Peek
5521dd08a0
Move test fixtures to samples/
2012-06-22 10:09:24 -05:00
Joshua Peek
2b712dc790
Guard against classify nil data
2012-06-21 11:47:32 -05:00
Joshua Peek
540f2a0941
More matlab samples
2012-06-21 10:44:31 -05:00
Joshua Peek
497da86262
Strip tex and matlab leading inline comments
2012-06-21 10:38:28 -05:00
Joshua Peek
4b9b8a5058
Remove matlab file with bogus keywords
2012-06-21 10:25:30 -05:00
Joshua Peek
5cdd5e206c
Improve operator tokenizing
2012-06-20 17:16:53 -05:00
Joshua Peek
516a220d9f
Verify classifer counts
2012-06-20 15:48:46 -05:00
Joshua Peek
f68e94f181
Skip number literals
2012-06-20 11:26:14 -05:00
Joshua Peek
e9eae4e008
Skip pending tests
2012-06-20 11:19:02 -05:00
Joshua Peek
e33d8f3685
Merge branch 'master' into bayesian
2012-06-20 11:18:47 -05:00
Joshua Peek
a10e52a3c2
Revert removing some fixtures
2012-06-20 11:18:16 -05:00
Joshua Peek
645a87d02b
Remove dead fixture test
2012-06-19 16:34:13 -05:00