Make colorize safer:

- don't try to colorize blobs that have a high ratio of
    long lines -- these are most likely minified files or something else
    strange that will blow up Pygments.rb
  - re github/github#3938
This commit is contained in:
Rob Sanheim
2012-05-21 11:31:21 -05:00
parent 7a84cfc24e
commit 1c7b8ebe71
3 changed files with 55 additions and 3 deletions

View File

@@ -160,6 +160,29 @@ module Linguist
size.to_i > MEGABYTE size.to_i > MEGABYTE
end end
# Public: Is the blob safe to colorize?
#
# We use Pygments.rb for syntax highlighting blobs, which
# has some quirks and also is essentially 'un-killable' via
# normal timeout. To workaround this we try to
# carefully handling Pygments.rb anything it can't handle.
#
# Return true or false
def safe_to_colorize?
text? && !large? && !high_ratio_of_long_lines?
end
# Internal: Does the blob have a ratio of long lines?
#
# These types of files are usually going to make Pygments.rb
# angry if we try to colorize them.
#
# Return true or false
def high_ratio_of_long_lines?
return false if loc == 0
size / loc > 5000
end
# Public: Is the blob viewable? # Public: Is the blob viewable?
# #
# Non-viewable blobs will just show a "View Raw" link # Non-viewable blobs will just show a "View Raw" link
@@ -642,7 +665,7 @@ module Linguist
# #
# Returns html String # Returns html String
def colorize(options = {}) def colorize(options = {})
return if !text? || large? return unless safe_to_colorize?
options[:options] ||= {} options[:options] ||= {}
options[:options][:encoding] ||= encoding options[:options][:encoding] ||= encoding
lexer.highlight(data, options) lexer.highlight(data, options)

24
test/fixtures/steelseries-min.js vendored Normal file

File diff suppressed because one or more lines are too long

View File

@@ -513,7 +513,12 @@ class TestBlob < Test::Unit::TestCase
HTML HTML
end end
def test_colorize_doesnt_skip_minified_files def test_colorize_does_skip_minified_files
assert blob("jquery-1.6.1.min.js").colorize assert_nil blob("jquery-1.6.1.min.js").colorize
end
# Pygments.rb was taking exceeding long on this particular file
def test_colorize_doesnt_blow_up_with_files_with_high_ratio_of_long_lines
assert_nil blob("steelseries-min.js").colorize
end end
end end