Commit Graph

328 Commits

Author SHA1 Message Date
Joshua Peek
d92d208a45 Fix tests for pygments.rb 0.3.x 2012-10-07 15:39:02 -05:00
Joshua Peek
ebd6077cd7 Add wrap flag to text languages 2012-10-07 15:34:13 -05:00
Joshua Peek
10e875e899 Print out samples db diffs 2012-09-24 10:07:08 -05:00
Ryan Tomayko
bda895eaae Test Mac Format detection and line splitting 2012-09-10 01:52:30 -07:00
Scott J. Goldman
e415a1351b When testing if a blob is indexable, check size first
Otherwise, charlock_holmes will allocate another large binary
buffer for testing the encoding, which is a problem if the binary
blob is many hundreds of MB large. It'll just fail and crash ruby.
2012-08-31 22:47:19 -07:00
Joshua Peek
b85aeaad3e Inline mime type lookup into blob helper 2012-08-20 11:33:16 -05:00
Joshua Peek
64f3509222 Remove other mime type hacks 2012-08-20 11:29:22 -05:00
Joshua Peek
630dca515a Trim down mime type overrides that are old or now pushed upstream
Related #206
2012-08-20 11:11:42 -05:00
Joshua Peek
220108857c Skip emiting comment tokens 2012-08-20 10:34:07 -05:00
Joshua Peek
96267e8696 Sort test assertion 2012-08-03 15:11:30 -05:00
Joshua Peek
16a67cb852 Move shebang detection into classifier
Fixes #203
2012-08-03 15:07:36 -05:00
Joshua Peek
fbbaff09cd Stop treating text as a language 2012-08-03 13:55:51 -05:00
Joshua Peek
6014bd015e Change find_by_filename api to return all matching languages 2012-08-03 13:53:12 -05:00
Joshua Peek
e5d302459f Fix tokenzing empty strings 2012-07-24 11:49:29 -05:00
Joshua Peek
53300ca581 Add brackets to tokens 2012-07-24 11:28:27 -05:00
Joshua Peek
f5705eaf38 Parse float tokens 2012-07-24 11:23:06 -05:00
Joshua Peek
059f661eb6 Rename Max/MSP to Max 2012-07-24 11:03:09 -05:00
Joshua Peek
6ac9138aed Remove pathname
Closes #207
2012-07-23 16:50:30 -05:00
Joshua Peek
fca6a9a9e1 Fix md5 nesting 2012-07-23 16:19:56 -05:00
Joshua Peek
7b6caa0f6c Rename samples subdirectories 2012-07-23 15:52:49 -05:00
Joshua Peek
5cda67530d Seperate test file for samples 2012-07-23 13:55:30 -05:00
Joshua Peek
6842044b52 Store md5 in samples db 2012-07-23 13:50:35 -05:00
Joshua Peek
bf944f6d1a Make classify a function on the Classifier 2012-07-23 13:47:15 -05:00
Joshua Peek
b9779e805e Move outdated check to samples 2012-07-23 13:21:30 -05:00
Joshua Peek
80e8ee7ce6 Rename Sample -> Samples 2012-07-23 13:15:27 -05:00
Joshua Peek
0c9a947f39 Load classifer db into sample data hash 2012-07-23 13:13:52 -05:00
Joshua Peek
3172bf5b46 Remove gc for now 2012-07-23 12:23:20 -05:00
Joshua Peek
5b28336d56 Move db verification into tests 2012-07-23 12:21:26 -05:00
Joshua Peek
b7f58d96cb Compare md5s of dbs 2012-07-23 12:17:32 -05:00
Joshua Peek
d6fb95b06f Add nested md5 digest 2012-07-23 12:13:08 -05:00
Joshua Peek
36da52e383 Move specific filename samples into their own dir 2012-07-20 17:20:45 -05:00
Joshua Peek
144655f2c5 Require explicit primary_extension 2012-07-20 16:54:49 -05:00
Joshua Peek
eb2c07e511 Sample returns simple hash objects 2012-07-20 16:17:37 -05:00
Joshua Peek
7292bdc180 Change Classifier to accept language name Strings 2012-07-20 15:52:27 -05:00
Joshua Peek
bbc5225086 Pending samples work now 2012-07-20 15:36:48 -05:00
Joshua Peek
2637d8dc55 Add tokenize helper to Tokenize class 2012-07-20 15:14:58 -05:00
Joshua Peek
175d4244c2 Extract single and multi line comment parser 2012-07-20 15:06:21 -05:00
Joshua Peek
d063089430 Add coq comments 2012-07-20 14:45:19 -05:00
Joshua Peek
5521dd08a0 Move test fixtures to samples/ 2012-06-22 10:09:24 -05:00
Joshua Peek
2b712dc790 Guard against classify nil data 2012-06-21 11:47:32 -05:00
Joshua Peek
540f2a0941 More matlab samples 2012-06-21 10:44:31 -05:00
Joshua Peek
497da86262 Strip tex and matlab leading inline comments 2012-06-21 10:38:28 -05:00
Joshua Peek
4b9b8a5058 Remove matlab file with bogus keywords 2012-06-21 10:25:30 -05:00
Joshua Peek
5cdd5e206c Improve operator tokenizing 2012-06-20 17:16:53 -05:00
Joshua Peek
516a220d9f Verify classifer counts 2012-06-20 15:48:46 -05:00
Joshua Peek
f68e94f181 Skip number literals 2012-06-20 11:26:14 -05:00
Joshua Peek
e9eae4e008 Skip pending tests 2012-06-20 11:19:02 -05:00
Joshua Peek
e33d8f3685 Merge branch 'master' into bayesian 2012-06-20 11:18:47 -05:00
Joshua Peek
a10e52a3c2 Revert removing some fixtures 2012-06-20 11:18:16 -05:00
Joshua Peek
645a87d02b Remove dead fixture test 2012-06-19 16:34:13 -05:00