From 4b3b1a80f663851e625a7ddbf142a972c9370e76 Mon Sep 17 00:00:00 2001 From: Brandon Keepers Date: Fri, 16 Jan 2015 09:02:51 -0500 Subject: [PATCH 01/55] Rearrange docs --- CONTRIBUTING.md | 75 ++++++++++++++------ README.md | 184 ++++-------------------------------------------- 2 files changed, 68 insertions(+), 191 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 697f2b2f..4afdd473 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,33 +1,68 @@ -## Contributing +# Contributing -The majority of contributions won't need to touch any Ruby code at all. The [master language list][languages] is just a YAML configuration file. +The majority of contributions won't need to touch any Ruby code at all. Linguist defines a list of all languages known to GitHub in [lib/linguist/languages.yml](https://github.com/github/linguist/blob/master/lib/linguist/languages.yml). -Almost all bug fixes or new language additions should come with some additional code samples. Just drop them under [`samples/`][samples] in the correct subdirectory and our test suite will automatically test them. In most cases you shouldn't need to add any new assertions. +Most languages are detected by their file extension. For disambiguating between files with common extensions, we first apply some common-sense heuristics to pick out obvious languages. After that, we use a [statistical classifier](https://github.com/github/linguist/blob/master/lib/linguist/classifier.rb). This process can help us tell the difference between, for example, `.h` files which could be either C, C++, or Obj-C. -### My code is detected as the wrong language +## Adding a language -This can usually be solved either by adding a new filename or file name extension to the language's entry in [`languages.yml`][languages] or adding more [samples][samples] for your language to the repository to make Linguist's classifier smarter. +We try only to add languages once they have some usage on GitHub. In most cases we prefer that languages be in use in hundreds of repositories before supporting them in Linguist. -### Syntax highlighting looks wrong - -Assuming your code is being detected as the right language (see above), in most cases this is due to a bug in the language grammar rather than a bug in Linguist. [`grammars.yml`][grammars] lists all the grammars we use for syntax highlighting on github.com. Find the one corresponding to your code's programming language and submit a bug report upstream. If you can, try to reproduce the highlighting problem in the text editor that the grammar is designed for (TextMate, Sublime Text, or Atom) and include that information in your bug report. - -You can also try to fix the bug yourself and submit a Pull Request. [This piece from TextMate's documentation](http://manual.macromates.com/en/language_grammars) offers a good introduction on how to work with TextMate-compatible grammars. You can test grammars using [Lightshow](https://lightshow.githubapp.com). - -Once the bug has been fixed upstream, please let us know and we'll pick it up for GitHub. - -### I want to add support for the `X` programming language - -Great! You'll need to: +To add support for a new language: 0. Add an entry for your language to [`languages.yml`][languages]. 0. Add a grammar for your language. Please only add grammars that have a license that permits redistribution. - 0. Add your grammar as a submodule: `git submodule add https://github.com/JaneSmith/MyGrammar vendor/grammars/MyGrammar`. - 0. Add your grammar to [`grammars.yml`][grammars] by running `script/download-grammars --add vendor/grammars/MyGrammar`. -0. Add samples for your language to the [samples directory][samples]. + 0. Add your grammar as a submodule: `git submodule add https://github.com/JaneSmith/MyGrammar vendor/grammars/MyGrammar`. + 0. Add your grammar to [`grammars.yml`][grammars] by running `script/download-grammars --add vendor/grammars/MyGrammar`. +0. Add samples for your language to the [samples directory][samples] in the correct subdirectory. +0. Open a pull request, linking to a [GitHub search result](https://github.com/search?utf8=%E2%9C%93&q=extension%3Aboot+NOT+nothack&type=Code&ref=searchresults) showing in-the-wild usage. -We try only to add languages once they have some usage on GitHub, so please note in-the-wild usage examples in your pull request. In most cases we prefer that languages already be in use in hundreds of repositories before supporting them in Linguist. +## Fixing syntax highlighting + +Syntax highlighting in GitHub is performed using TextMate-compatible grammars. These are the same grammars that TextMate, Sublime Text and Atom use. Every language in `languages.yml` is mapped to its corresponding TM `scope`. This scope will be used when picking up a grammar for highlighting. + +Assuming your code is being detected as the right language (see [Language Detection](#language-detection) above), in most cases this is due to a bug in the language grammar rather than a bug in Linguist. [`grammars.yml`][grammars] lists all the grammars we use for syntax highlighting on github.com. Find the one corresponding to your code's programming language and submit a bug report upstream. If you can, try to reproduce the highlighting problem in the text editor that the grammar is designed for (TextMate, Sublime Text, or Atom) and include that information in your bug report. + +You can also try to fix the bug yourself and submit a Pull Request. [TextMate's documentation](http://manual.macromates.com/en/language_grammars) offers a good introduction on how to work with TextMate-compatible grammars. You can test grammars using [Lightshow](https://lightshow.githubapp.com). + +Once the bug has been fixed upstream, please let us know and we'll pick it up for GitHub. + +## Testing + +For development you are going to want to checkout out the source. To get it, clone the repo and run [Bundler](http://gembundler.com/) to install its dependencies. + + git clone https://github.com/github/linguist.git + cd linguist/ + script/bootstrap + +To run the tests: + + bundle exec rake test + +Sometimes getting the tests running can be too much work, especially if you don't have much Ruby experience. It's okay: be lazy and let our build bot [Travis](http://travis-ci.org/#!/github/linguist) run the tests for you. Just open a pull request and the bot will start cranking away. + +Here's our current build status: [![Build Status](https://secure.travis-ci.org/github/linguist.png?branch=master)](http://travis-ci.org/github/linguist) + +## Releasing + +If you are the current maintainer of this gem: + +0. Create a branch for the release: `git checkout -b cut-release-vxx.xx.xx` +0. Make sure your local dependencies are up to date: `script/bootstrap` +0. Ensure that samples are updated: `bundle exec rake samples` +0. Ensure that tests are green: `bundle exec rake test` +0. Bump gem version in `lib/linguist/version.rb`, [like this](https://github.com/github/linguist/commit/8d2ea90a5ba3b2fe6e1508b7155aa4632eea2985). +0. Make a PR to github/linguist, [like this](https://github.com/github/linguist/pull/1238). +0. Build a local gem: `bundle exec rake build_gem` +0. Test the gem: + 0. Bump the Gemfile and Gemfile.lock versions for an app which relies on this gem + 0. Install the new gem locally + 0. Test behavior locally, branch deploy, whatever needs to happen +0. Merge github/linguist PR +0. Tag and push: `git tag vx.xx.xx; git push --tags` +0. Push to rubygems.org -- `gem push github-linguist-3.0.0.gem` [grammars]: /grammars.yml [languages]: /lib/linguist/languages.yml [samples]: /samples +[new-issue]: https://github.com/github/linguist/issues/new diff --git a/README.md b/README.md index c2c803a8..9fded07d 100644 --- a/README.md +++ b/README.md @@ -1,196 +1,38 @@ # Linguist -We use this library at GitHub to detect blob languages, ignore binary files, suppress generated files in diffs, and generate language breakdown graphs. +This library is used on GitHub.com to detect blob languages, ignore binary or vendored files, suppress generated files in diffs, and generate language breakdown graphs. -Tips for filing issues and creating pull requests can be found in [`CONTRIBUTING.md`](/CONTRIBUTING.md). +See [Troubleshooting](#troubleshooting) and [`CONTRIBUTING.md`](/CONTRIBUTING.md) before filing an issue or creating a pull request. -## Features +## Troubleshooting -### Language detection +### My repository is detected as the wrong language -Linguist defines a list of all languages known to GitHub in a [yaml file](https://github.com/github/linguist/blob/master/lib/linguist/languages.yml). +![language stats bar](https://cloud.githubusercontent.com/assets/173/5562290/48e24654-8ddf-11e4-8fe7-735b0ce3a0d3.png) -Most languages are detected by their file extension. For disambiguating between files with common extensions, we first apply some common-sense heuristics to pick out obvious languages. After that, we use a -[statistical -classifier](https://github.com/github/linguist/blob/master/lib/linguist/classifier.rb). -This process can help us tell the difference between, for example, `.h` files which could be either C, C++, or Obj-C. +The Language stats bar is built by aggregating the languages of each file in that repository. If it is reporting a language that you don't expect, click on the name of the language and to see a list of the files that are identified as that language. If there are files in your repository -```ruby +consider moving the files into one of the -Linguist::FileBlob.new("lib/linguist.rb").language.name #=> "Ruby" - -Linguist::FileBlob.new("bin/linguist").language.name #=> "Ruby" -``` - -See [lib/linguist/language.rb](https://github.com/github/linguist/blob/master/lib/linguist/language.rb) and [lib/linguist/languages.yml](https://github.com/github/linguist/blob/master/lib/linguist/languages.yml). - -### Syntax Highlighting - -Syntax highlighting in GitHub is performed using TextMate-compatible grammars. These are the same grammars that TextMate, Sublime Text and Atom use. - -Every language in `languages.yml` is mapped to its corresponding TM `scope`. This scope will be used when picking up a grammar for highlighting. **When adding a new language to Linguist, please add its corresponding scope too (assuming there's an existing TextMate bundle, Sublime Text package, or Atom package) so syntax highlighting works for it**. - -### Stats - -The Language stats bar that you see on every repository is built by aggregating the languages of each file in that repository. The top language in the graph determines the project's primary language. - -The repository stats API, accessed through `#languages`, can be used on a directory: - -***API UPDATE*** - -Since [Version 3.0.0](https://github.com/github/linguist/releases/tag/v3.0.0) Linguist expects a git repository (in the form of a [Rugged::Repository](https://github.com/libgit2/rugged#repositories)) to be passed when initializing `Linguist::Repository`. - - -```ruby -require 'rugged' -require 'linguist' - -repo = Rugged::Repository.new('.') -project = Linguist::Repository.new(repo, repo.head.target_id) -project.language #=> "Ruby" -project.languages #=> { "Ruby" => 119387 } -``` - -These stats are also printed out by the `linguist` binary. You can use the -`--breakdown` flag, and the binary will also output the breakdown of files by language. - -You can try running `linguist` on the root directory in this repository itself: - - $ bundle exec linguist --breakdown - - 100.00% Ruby - - Ruby: - Gemfile - Rakefile - bin/linguist - github-linguist.gemspec - lib/linguist.rb - lib/linguist/blob_helper.rb - lib/linguist/classifier.rb - lib/linguist/file_blob.rb - lib/linguist/generated.rb - lib/linguist/heuristics.rb - lib/linguist/language.rb - lib/linguist/lazy_blob.rb - lib/linguist/md5.rb - lib/linguist/repository.rb - lib/linguist/samples.rb - lib/linguist/tokenizer.rb - lib/linguist/version.rb - test/test_blob.rb - test/test_classifier.rb - test/test_heuristics.rb - test/test_language.rb - test/test_md5.rb - test/test_pedantic.rb - test/test_repository.rb - test/test_samples.rb - test/test_tokenizer.rb - -#### Ignore vendored files - -Checking other code into your git repo is a common practice. But this often inflates your project's language stats and may even cause your project to be labeled as another language. We are able to identify some of these files and directories and exclude them. - -```ruby -Linguist::FileBlob.new("vendor/plugins/foo.rb").vendored? # => true -``` - -See [Linguist::BlobHelper#vendored?](https://github.com/github/linguist/blob/master/lib/linguist/blob_helper.rb) and [lib/linguist/vendor.yml](https://github.com/github/linguist/blob/master/lib/linguist/vendor.yml). - -#### Generated file detection - -Not all plain text files are true source files. Generated files like minified js and compiled CoffeeScript can be detected and excluded from language stats. As an extra bonus, these files are suppressed in diffs. - -```ruby -Linguist::FileBlob.new("underscore.min.js").generated? # => true -``` - -See [Linguist::Generated#generated?](https://github.com/github/linguist/blob/master/lib/linguist/generated.rb). +See [Overrides](#overrides) ## Overrides -Linguist supports custom overrides for language definitions and vendored paths. Add a `.gitattributes` file to your project using the keys `linguist-language` and `linguist-vendored` with the standard git-style path matchers for the files you want to override. +Linguist supports custom overrides for language definitions and vendored paths. Please note that the overrides currently only affect the language statistics for a repository and not the syntax-highlighting of files. -Please note that the overrides currently only affect the language statistics for a repository and not the syntax-highlighting of files. +Commit a `.gitattributes` file to your project and use standard git-style path matchers for the files you want to override to set `linguist-language` and `linguist-vendored`. ``` $ cat .gitattributes *.rb linguist-language=Java - -$ linguist --breakdown -100.00% Java - -Java: -ruby_file.rb ``` -By default, Linguist treats all of the paths defined in [lib/linguist/vendor.yml](https://github.com/github/linguist/blob/master/lib/linguist/vendor.yml) as vendored and therefore doesn't include them in the language statistics for a repository. Use the `linguist-vendored` attribute to vendor or un-vendor paths. +Checking code you didn't write, such as JavaScript libraries, into your git repo is a common practice, but this often inflates your project's language stats and may even cause your project to be labeled as another language. By default, Linguist treats all of the paths defined in [lib/linguist/vendor.yml](https://github.com/github/linguist/blob/master/lib/linguist/vendor.yml) as vendored and therefore doesn't include them in the language statistics for a repository. + +Use the `linguist-vendored` attribute to vendor or un-vendor paths. ``` $ cat .gitattributes special-vendored-path/* linguist-vendored jquery.js linguist-vendored=false ``` - -## Installation - -Github.com is usually running the latest version of the `github-linguist` gem that is released on [RubyGems.org](http://rubygems.org/gems/github-linguist). - -But for development you are going to want to checkout out the source. To get it, clone the repo and run [Bundler](http://gembundler.com/) to install its dependencies. - - git clone https://github.com/github/linguist.git - cd linguist/ - script/bootstrap - -To run the tests: - - bundle exec rake test - -### A note on language extensions - -Linguist has a number of methods available to it for identifying the language of a particular file. The initial lookup is based upon the extension of the file, possible file extensions are defined in an array called `extensions`. Take a look at this example for example for `Perl`: - -``` -Perl: - type: programming - ace_mode: perl - color: "#0298c3" - extensions: - - .pl - - .PL - - .perl - - .ph - - .plx - - .pm - - .pod - - .psgi - interpreters: - - perl -``` -Any of the extensions defined are valid but the first in this array should be the most popular. - -### Testing - -Sometimes getting the tests running can be too much work, especially if you don't have much Ruby experience. It's okay: be lazy and let our build bot [Travis](http://travis-ci.org/#!/github/linguist) run the tests for you. Just open a pull request and the bot will start cranking away. - -Here's our current build status, which is hopefully green: [![Build Status](https://secure.travis-ci.org/github/linguist.png?branch=master)](http://travis-ci.org/github/linguist) - -### Releasing - -If you are the current maintainer of this gem: - - 0. Create a branch for the release: `git checkout -b cut-release-vxx.xx.xx` - 0. Make sure your local dependencies are up to date: `script/bootstrap` - 0. Ensure that samples are updated: `bundle exec rake samples` - 0. Ensure that tests are green: `bundle exec rake test` - 0. Bump gem version in `lib/linguist/version.rb`. For example, [like this](https://github.com/github/linguist/commit/8d2ea90a5ba3b2fe6e1508b7155aa4632eea2985). - 0. Make a PR to github/linguist. For example, [#1238](https://github.com/github/linguist/pull/1238). - 0. Build a local gem: `bundle exec rake build_gem` - 0. Testing: - 0. Bump the Gemfile and Gemfile.lock versions for an app which relies on this gem - 0. Install the new gem locally - 0. Test behavior locally, branch deploy, whatever needs to happen - 0. Merge github/linguist PR - 0. Tag and push: `git tag vx.xx.xx; git push --tags` - 0. Push to rubygems.org -- `gem push github-linguist-3.0.0.gem` From b56671c20dc194d7544d3dc6d8c26125eadc93e1 Mon Sep 17 00:00:00 2001 From: Brandon Keepers Date: Fri, 16 Jan 2015 10:15:47 -0500 Subject: [PATCH 02/55] Lists are easier to follow than walls of text --- README.md | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 9fded07d..e711c659 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,8 @@ # Linguist +[issues]: https://github.com/github/linguist/issues +[new-issue]: https://github.com/github/linguist/issues/new + This library is used on GitHub.com to detect blob languages, ignore binary or vendored files, suppress generated files in diffs, and generate language breakdown graphs. See [Troubleshooting](#troubleshooting) and [`CONTRIBUTING.md`](/CONTRIBUTING.md) before filing an issue or creating a pull request. @@ -10,11 +13,12 @@ See [Troubleshooting](#troubleshooting) and [`CONTRIBUTING.md`](/CONTRIBUTING.md ![language stats bar](https://cloud.githubusercontent.com/assets/173/5562290/48e24654-8ddf-11e4-8fe7-735b0ce3a0d3.png) -The Language stats bar is built by aggregating the languages of each file in that repository. If it is reporting a language that you don't expect, click on the name of the language and to see a list of the files that are identified as that language. If there are files in your repository +The Language stats bar is built by aggregating the languages of each file in that repository. If it is reporting a language that you don't expect: -consider moving the files into one of the - -See [Overrides](#overrides) +0. Click on the name of the language in the stats bar to see a list of the files that are identified as that language. +0. If you see files that you didn't write, consider moving the files into one of the [paths for vendored code](https://github.com/github/linguist/blob/master/lib/linguist/vendor.yml), or use the [manual overrides](#overrides) feature to ignore them. +0. If the files are being misclassified, search for [open issues][issues] to see if anyone else has already reported the issue. Any information you an add, especially links to public repositories, is helpful. +0. If there are no reported issues of this misclassification, [open an issue][new-issue] and include a link to the repository or a sample of the code that is being misclassified. ## Overrides @@ -27,7 +31,7 @@ $ cat .gitattributes *.rb linguist-language=Java ``` -Checking code you didn't write, such as JavaScript libraries, into your git repo is a common practice, but this often inflates your project's language stats and may even cause your project to be labeled as another language. By default, Linguist treats all of the paths defined in [lib/linguist/vendor.yml](https://github.com/github/linguist/blob/master/lib/linguist/vendor.yml) as vendored and therefore doesn't include them in the language statistics for a repository. +Checking code you didn't write, such as JavaScript libraries, into your git repo is a common practice, but this often inflates your project's language stats and may even cause your project to be labeled as another language. By default, Linguist treats all of the paths defined in [lib/linguist/vendor.yml](https://github.com/github/linguist/blob/master/lib/linguist/vendor.yml) as vendored and therefore doesn't include them in the language statistics for a repository. Use the `linguist-vendored` attribute to vendor or un-vendor paths. From 3886c406abe7eaf56e451c3eca1f8358bb9b2cc9 Mon Sep 17 00:00:00 2001 From: Brandon Keepers Date: Fri, 16 Jan 2015 10:28:09 -0500 Subject: [PATCH 03/55] Add section on fixing misclassifications --- CONTRIBUTING.md | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index e7256579..ade8bffd 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,8 +1,6 @@ # Contributing -The majority of contributions won't need to touch any Ruby code at all. Linguist defines a list of all languages known to GitHub in [lib/linguist/languages.yml](https://github.com/github/linguist/blob/master/lib/linguist/languages.yml). - -Most languages are detected by their file extension. For disambiguating between files with common extensions, we first apply some common-sense heuristics to pick out obvious languages. After that, we use a [statistical classifier](https://github.com/github/linguist/blob/master/lib/linguist/classifier.rb). This process can help us tell the difference between, for example, `.h` files which could be either C, C++, or Obj-C. +The majority of contributions won't need to touch any Ruby code at all. ## Adding a language @@ -25,9 +23,15 @@ In addition, if your new language defines an extension that's already listed in Remember, the goal here is to try and avoid false positives! +## Fixing a misclassified language + +Most languages are detected by their file extension defined in [languages.yml][languages]. For disambiguating between files with common extensions, linguist applies some [heuristics](/lib/linguist/heuristics.rb) and a [statistical classifier](lib/linguist/classifier.rb). This process can help differentiate between, for example, `.h` files which could be either C, C++, or Obj-C. + +Misclassifications can often be solved by either adding a new filename or extension for the language or adding more [samples][samples] to make the classifier smarter. + ## Fixing syntax highlighting -Syntax highlighting in GitHub is performed using TextMate-compatible grammars. These are the same grammars that TextMate, Sublime Text and Atom use. Every language in `languages.yml` is mapped to its corresponding TM `scope`. This scope will be used when picking up a grammar for highlighting. +Syntax highlighting in GitHub is performed using TextMate-compatible grammars. These are the same grammars that TextMate, Sublime Text and Atom use. Every language in [languages.yml][languages] is mapped to its corresponding TM `scope`. This scope will be used when picking up a grammar for highlighting. Assuming your code is being detected as the right language (see [Language Detection](#language-detection) above), in most cases this is due to a bug in the language grammar rather than a bug in Linguist. [`grammars.yml`][grammars] lists all the grammars we use for syntax highlighting on github.com. Find the one corresponding to your code's programming language and submit a bug report upstream. If you can, try to reproduce the highlighting problem in the text editor that the grammar is designed for (TextMate, Sublime Text, or Atom) and include that information in your bug report. From 034e510ba520050b92ef63a68bcd8ffe93189646 Mon Sep 17 00:00:00 2001 From: Paul Chaignon Date: Mon, 2 Feb 2015 21:24:22 +0100 Subject: [PATCH 04/55] Grammar for TXL from Sublime Text package --- .gitmodules | 3 +++ grammars.yml | 2 ++ lib/linguist/languages.yml | 2 +- vendor/grammars/TXL | 1 + 4 files changed, 7 insertions(+), 1 deletion(-) create mode 160000 vendor/grammars/TXL diff --git a/.gitmodules b/.gitmodules index ee7dad93..77fd2474 100644 --- a/.gitmodules +++ b/.gitmodules @@ -579,3 +579,6 @@ [submodule "vendor/grammars/JSyntax"] path = vendor/grammars/JSyntax url = https://github.com/bcj/JSyntax +[submodule "vendor/grammars/TXL"] + path = vendor/grammars/TXL + url = https://github.com/MikeHoffert/Sublime-Text-TXL-syntax diff --git a/grammars.yml b/grammars.yml index e9eddd79..0e8b5e01 100644 --- a/grammars.yml +++ b/grammars.yml @@ -106,6 +106,8 @@ vendor/grammars/SublimeBrainfuck: - source.bf vendor/grammars/SublimeXtend: - source.xtend +vendor/grammars/TXL/: +- source.txl vendor/grammars/Textmate-Gosu-Bundle: - source.gosu.2 vendor/grammars/VBDotNetSyntax: diff --git a/lib/linguist/languages.yml b/lib/linguist/languages.yml index c492ed36..6598563e 100644 --- a/lib/linguist/languages.yml +++ b/lib/linguist/languages.yml @@ -2921,7 +2921,7 @@ TXL: type: programming extensions: - .txl - tm_scope: none + tm_scope: source.txl ace_mode: text Tcl: diff --git a/vendor/grammars/TXL b/vendor/grammars/TXL new file mode 160000 index 00000000..c1c98dfa --- /dev/null +++ b/vendor/grammars/TXL @@ -0,0 +1 @@ +Subproject commit c1c98dfa86a8510532aee3df99181f9e0487fee8 From 8fe9ec05213ac177ef3e7a4e81ba2f80497cefa5 Mon Sep 17 00:00:00 2001 From: Paul Chaignon Date: Mon, 2 Feb 2015 21:37:24 +0100 Subject: [PATCH 05/55] Grammar for G-Code from Sublime Text package --- .gitmodules | 3 +++ grammars.yml | 6 ++++++ lib/linguist/languages.yml | 2 +- vendor/grammars/G-Code | 1 + 4 files changed, 11 insertions(+), 1 deletion(-) create mode 160000 vendor/grammars/G-Code diff --git a/.gitmodules b/.gitmodules index 77fd2474..6857325f 100644 --- a/.gitmodules +++ b/.gitmodules @@ -582,3 +582,6 @@ [submodule "vendor/grammars/TXL"] path = vendor/grammars/TXL url = https://github.com/MikeHoffert/Sublime-Text-TXL-syntax +[submodule "vendor/grammars/G-Code"] + path = vendor/grammars/G-Code + url = https://github.com/robotmaster/sublime-text-syntax-highlighting diff --git a/grammars.yml b/grammars.yml index 0e8b5e01..2b1d2c72 100644 --- a/grammars.yml +++ b/grammars.yml @@ -39,6 +39,12 @@ vendor/grammars/Docker.tmbundle: - source.dockerfile vendor/grammars/Elm.tmLanguage: - source.elm +vendor/grammars/G-Code/: +- source.LS +- source.MCPOST +- source.MOD +- source.apt +- source.gcode vendor/grammars/GDScript-sublime/: - source.gdscript vendor/grammars/Handlebars: diff --git a/lib/linguist/languages.yml b/lib/linguist/languages.yml index 6598563e..2be249ba 100644 --- a/lib/linguist/languages.yml +++ b/lib/linguist/languages.yml @@ -942,7 +942,7 @@ G-code: - .g - .gco - .gcode - tm_scope: none + tm_scope: source.gcode ace_mode: gcode GAMS: diff --git a/vendor/grammars/G-Code b/vendor/grammars/G-Code new file mode 160000 index 00000000..81e8b03e --- /dev/null +++ b/vendor/grammars/G-Code @@ -0,0 +1 @@ +Subproject commit 81e8b03e3dc71f2c8eddfb389dbda21a320f45c1 From 2044e191a2f42edbf3848a4f069b4d913a8f4b32 Mon Sep 17 00:00:00 2001 From: Paul Chaignon Date: Mon, 2 Feb 2015 21:48:27 +0100 Subject: [PATCH 06/55] Grammar for Grace from TextMate bundle --- .gitmodules | 3 +++ grammars.yml | 2 ++ lib/linguist/languages.yml | 2 +- vendor/grammars/grace-tmbundle | 1 + 4 files changed, 7 insertions(+), 1 deletion(-) create mode 160000 vendor/grammars/grace-tmbundle diff --git a/.gitmodules b/.gitmodules index 6857325f..5980861a 100644 --- a/.gitmodules +++ b/.gitmodules @@ -585,3 +585,6 @@ [submodule "vendor/grammars/G-Code"] path = vendor/grammars/G-Code url = https://github.com/robotmaster/sublime-text-syntax-highlighting +[submodule "vendor/grammars/grace-tmbundle"] + path = vendor/grammars/grace-tmbundle + url = https://github.com/zmthy/grace-tmbundle diff --git a/grammars.yml b/grammars.yml index 2b1d2c72..ee83c6b0 100644 --- a/grammars.yml +++ b/grammars.yml @@ -227,6 +227,8 @@ vendor/grammars/gnuplot-tmbundle: - source.gnuplot vendor/grammars/go-tmbundle: - source.go +vendor/grammars/grace-tmbundle/: +- source.grace vendor/grammars/gradle.tmbundle: - source.groovy.gradle vendor/grammars/graphviz.tmbundle: diff --git a/lib/linguist/languages.yml b/lib/linguist/languages.yml index 2be249ba..dddded97 100644 --- a/lib/linguist/languages.yml +++ b/lib/linguist/languages.yml @@ -1092,7 +1092,7 @@ Grace: type: programming extensions: - .grace - tm_scope: none + tm_scope: source.grace ace_mode: text Gradle: diff --git a/vendor/grammars/grace-tmbundle b/vendor/grammars/grace-tmbundle new file mode 160000 index 00000000..c342d35c --- /dev/null +++ b/vendor/grammars/grace-tmbundle @@ -0,0 +1 @@ +Subproject commit c342d35c76d6a7dd5cd91157ca5b39481ef59e96 From 8f70604466c9fd8890ddc23e5bf0b6958df78e03 Mon Sep 17 00:00:00 2001 From: Paul Chaignon Date: Mon, 2 Feb 2015 21:50:49 +0100 Subject: [PATCH 07/55] Grammar for Ox from Sublime Text package --- .gitmodules | 3 +++ grammars.yml | 2 ++ lib/linguist/languages.yml | 2 +- vendor/grammars/sublime-text-ox | 1 + 4 files changed, 7 insertions(+), 1 deletion(-) create mode 160000 vendor/grammars/sublime-text-ox diff --git a/.gitmodules b/.gitmodules index 5980861a..6aa15594 100644 --- a/.gitmodules +++ b/.gitmodules @@ -588,3 +588,6 @@ [submodule "vendor/grammars/grace-tmbundle"] path = vendor/grammars/grace-tmbundle url = https://github.com/zmthy/grace-tmbundle +[submodule "vendor/grammars/sublime-text-ox"] + path = vendor/grammars/sublime-text-ox + url = https://github.com/andreashetland/sublime-text-ox diff --git a/grammars.yml b/grammars.yml index ee83c6b0..d4b76122 100644 --- a/grammars.yml +++ b/grammars.yml @@ -457,6 +457,8 @@ vendor/grammars/sublime-sourcepawn: - source.sp vendor/grammars/sublime-tea: - source.tea +vendor/grammars/sublime-text-ox/: +- source.ox vendor/grammars/sublime_cobol: - source.acucobol - source.cobol diff --git a/lib/linguist/languages.yml b/lib/linguist/languages.yml index dddded97..8f15c38f 100644 --- a/lib/linguist/languages.yml +++ b/lib/linguist/languages.yml @@ -2096,7 +2096,7 @@ Ox: - .ox - .oxh - .oxo - tm_scope: none + tm_scope: source.ox ace_mode: text Oxygene: diff --git a/vendor/grammars/sublime-text-ox b/vendor/grammars/sublime-text-ox new file mode 160000 index 00000000..bdd03e09 --- /dev/null +++ b/vendor/grammars/sublime-text-ox @@ -0,0 +1 @@ +Subproject commit bdd03e09fabc0b54567136709cdd33d7641b0e19 From 3c34da8bd39466b77268cebbb57ca13cd467d768 Mon Sep 17 00:00:00 2001 From: Paul Chaignon Date: Tue, 3 Feb 2015 12:52:06 +0100 Subject: [PATCH 08/55] Sample for eC --- samples/eC/Designer.ec | 337 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 337 insertions(+) create mode 100644 samples/eC/Designer.ec diff --git a/samples/eC/Designer.ec b/samples/eC/Designer.ec new file mode 100644 index 00000000..38ca035f --- /dev/null +++ b/samples/eC/Designer.ec @@ -0,0 +1,337 @@ +import "ide" + +class Designer : DesignerBase +{ + ~Designer() + { + if(GetActiveDesigner() == this) + { + SetActiveDesigner(null); + } + if(classDesigner) + delete classDesigner; + } + + // *** DesignerBase Implementation *** + + void ModifyCode() + { + codeEditor.ModifyCode(); + } + + void UpdateProperties() + { + codeEditor.DesignerModifiedObject(); + } + + void CodeAddObject(Instance instance, ObjectInfo * object) + { + codeEditor.AddObject(instance, object); + } + + void SheetAddObject(ObjectInfo object) + { + codeEditor.sheet.AddObject(object, object.name, typeData, true); //className, true); + } + + void AddToolBoxClass(Class _class) + { + ((IDEWorkSpace)master).toolBox.AddControl(_class); + } + + void AddDefaultMethod(Instance instance, Instance classInstance) + { + Class _class = instance._class; + Method defaultMethod = null; + + for( ; _class; _class = _class.base) + { + Method method; + int minID = MAXINT; + for(method = (Method)_class.methods.first; method; method = (Method)((BTNode)method).next) + { + if(method.type == virtualMethod) + { + if(!method.dataType) + method.dataType = ProcessTypeString(method.dataTypeString, false); + if(method.vid < minID && (instance == classInstance || (method.dataType.thisClass && eClass_IsDerived(classInstance._class, method.dataType.thisClass.registered)))) + { + defaultMethod = method; + minID = method.vid; + } + } + } + if(defaultMethod) + break; + } + codeEditor.AddMethod(defaultMethod); + } + + bool ObjectContainsCode(ObjectInfo object) + { + // Confirmation if control contains code + if(object.instCode) + { + MembersInit members; + if(object.instCode.members) + { + for(members = object.instCode.members->first; members; members = members.next) + { + if(members.type == methodMembersInit) + { + //if(!Code_IsFunctionEmpty(members.function)) + { + return true; + } + } + } + } + } + return false; + } + + void DeleteObject(ObjectInfo object) + { + if(codeEditor) + codeEditor.DeleteObject(object); + } + + void RenameObject(ObjectInfo object, const char * name) + { + if(object && (name || !object.classDefinition)) + codeEditor.RenameObject(object, name); + } + + bool FindObject(Instance * object, const char * string) + { + ObjectInfo classObject; + for(classObject = codeEditor.classes.first; classObject; classObject = classObject.next) + { + ObjectInfo check; + if(classObject.name && !strcmp(string, classObject.name)) + { + *object = classObject.instance; + break; + } + for(check = classObject.instances.first; check; check = check.next) + { + if(check.name && !strcmp(string, check.name)) + { + *object = check.instance; + break; + } + } + if(check) + return true; + } + return false; + } + + void SelectObjectFromDesigner(ObjectInfo object) + { + codeEditor.SelectObjectFromDesigner(object); + } + + borderStyle = sizable; + isActiveClient = true; + hasVertScroll = true; + hasHorzScroll = true; + hasClose = true; + hasMaximize = true; + hasMinimize = true; + text = $"Designer"; + menu = Menu { }; + anchor = Anchor { left = 300, right = 150, top = 0, bottom = 0 }; + + ToolBox toolBox; + CodeEditor codeEditor; + + Menu fileMenu { menu, $"File", f }; + MenuItem fileSaveItem + { + fileMenu, $"Save", s, ctrlS; + bool NotifySelect(MenuItem selection, Modifiers mods) + { + return codeEditor.MenuFileSave(selection, mods); + } + }; + MenuItem fileSaveAsItem + { + fileMenu, $"Save As...", a; + bool NotifySelect(MenuItem selection, Modifiers mods) + { + return codeEditor.MenuFileSaveAs(selection, mods); + } + }; + bool debugClosing; + + bool OnClose(bool parentClosing) + { + if(!parentClosing) + { + if(codeEditor && codeEditor.inUseDebug && !debugClosing) + { + debugClosing = true; + closing = false; + if(CloseConfirmation(false)) + { + visible = false; + if(modifiedDocument) + OnFileModified({ modified = true }, null); + } + debugClosing = false; + return false; + } + if(codeEditor && !codeEditor.closing && !debugClosing) + { + if(!codeEditor.visible) + { + if(!codeEditor.Destroy(0)) + return false; + else + codeEditor = null; + } + else + { + visible = false; + return false; + } + } + } + return true; + } + + bool OnActivate(bool active, Window previous, bool * goOnWithActivation, bool direct) + { + if(active) + { + codeEditor.EnsureUpToDate(); + codeEditor.fixCaret = true; + /* + if(classDesigner) + classDesigner.Activate(); + */ + } + return true; + } + + bool OnKeyHit(Key key, unichar ch) + { + return codeEditor.sheet.OnKeyHit(key, ch); + } + + watch(modifiedDocument) + { + fileSaveItem.disabled = !modifiedDocument && codeEditor.fileName; + }; + + // *** METHODS ACCESSED FROM PROPERTY SHEET/TOOLBOX/CODE EDITOR *** + void Reset() + { + if(classDesigner) + { + classDesigner.Reset(); + classDesigner.SelectObject(null, null); + classDesigner.Destroy(0); + delete classDesigner; + } + } + + void FillToolBox() + { + if(this && classDesigner) + classDesigner.ListToolBoxClasses(this); + } + + void SelectObject(ObjectInfo object, Instance instance) + { + ClassDesignerBase classDesigner = this.classDesigner; +#ifdef _DEBUG + if(instance && instance._class.module.application != codeEditor.privateModule) + printf("warning: SelectObject: instance._class.module.application != codeEditor.privateModule\n"); +#endif + if(!classDesigner || !instance || classDesigner._class != (Class)eInstance_GetDesigner(instance)) + { + if(classDesigner) + { + classDesigner.SelectObject(null, null); + classDesigner.Destroy(0); + classDesigner = null; + delete this.classDesigner; + } + if(instance) + { + this.classDesigner = classDesigner = eInstance_New(eInstance_GetDesigner(instance)); + incref classDesigner; + //if(!classDesigner.parent) + { + classDesigner.parent = this; + classDesigner.anchor = Anchor { left = 0, right = 0, top = 0, bottom = 0 }; + } + classDesigner.Create(); + } + } + // Call class editor SelectObject + if(classDesigner) + classDesigner.SelectObject(object, instance); + } + + void AddObject() + { + // Call class editor AddObject + if(classDesigner) + classDesigner.AddObject(); + if(visible) + Activate(); + else + codeEditor.Activate(); + } + + void CreateObject(Instance instance, ObjectInfo object, bool isClass, Instance iclass) + { + subclass(ClassDesignerBase) designerClass = eInstance_GetDesigner(instance); + + // Call class editor CreateObject + if(designerClass) + designerClass.CreateObject(this, instance, object, isClass, iclass); + } + + void ::PostCreateObject(Instance instance, ObjectInfo object, bool isClass, Instance iclass) + { + subclass(ClassDesignerBase) designerClass = eInstance_GetDesigner(instance); + + // Call class editor PostCreateObject + if(designerClass) + designerClass.PostCreateObject(instance, object, isClass, iclass); + } + + void ::DroppedObject(Instance instance, ObjectInfo object, bool isClass, Instance iclass) + { + subclass(ClassDesignerBase) designerClass = eInstance_GetDesigner(instance); + + // Call class editor PostCreateObject + if(designerClass) + designerClass.DroppedObject(instance, object, isClass, iclass); + } + + void PrepareTestObject(Instance instance) + { + subclass(ClassDesignerBase) designerClass = eInstance_GetDesigner(instance); + if(designerClass) + designerClass.PrepareTestObject(this, instance); + } + + void ::DestroyObject(Instance instance) + { + subclass(ClassDesignerBase) designerClass = eInstance_GetDesigner(instance); + if(designerClass) + designerClass.DestroyObject(instance); + } + + void ::FixProperty(Property prop, Instance instance) + { + subclass(ClassDesignerBase) designerClass = eInstance_GetDesigner(instance); + if(designerClass) + designerClass.FixProperty(prop, instance); + } +} From 9fe9bf617f162eb2b40e67b6de00c3c28bb6d620 Mon Sep 17 00:00:00 2001 From: Paul Chaignon Date: Tue, 3 Feb 2015 12:52:35 +0100 Subject: [PATCH 09/55] Grammar for eC from TextMate bundle --- .gitmodules | 3 +++ grammars.yml | 2 ++ lib/linguist/languages.yml | 2 +- vendor/grammars/ec.tmbundle | 1 + 4 files changed, 7 insertions(+), 1 deletion(-) create mode 160000 vendor/grammars/ec.tmbundle diff --git a/.gitmodules b/.gitmodules index 6aa15594..37489aa1 100644 --- a/.gitmodules +++ b/.gitmodules @@ -591,3 +591,6 @@ [submodule "vendor/grammars/sublime-text-ox"] path = vendor/grammars/sublime-text-ox url = https://github.com/andreashetland/sublime-text-ox +[submodule "vendor/grammars/ec.tmbundle"] + path = vendor/grammars/ec.tmbundle + url = https://github.com/ecere/ec.tmbundle diff --git a/grammars.yml b/grammars.yml index d4b76122..9b757868 100644 --- a/grammars.yml +++ b/grammars.yml @@ -200,6 +200,8 @@ vendor/grammars/dylan.tmbundle: - source.makegen vendor/grammars/ebundles/Bundles/MSDOS batch file.tmbundle: - source.dosbatch +vendor/grammars/ec.tmbundle/: +- source.c.ec vendor/grammars/eiffel.tmbundle: - source.eiffel vendor/grammars/elixir-tmbundle: diff --git a/lib/linguist/languages.yml b/lib/linguist/languages.yml index 8f15c38f..c5af5967 100644 --- a/lib/linguist/languages.yml +++ b/lib/linguist/languages.yml @@ -3331,7 +3331,7 @@ eC: extensions: - .ec - .eh - tm_scope: none + tm_scope: source.c.ec ace_mode: text edn: diff --git a/vendor/grammars/ec.tmbundle b/vendor/grammars/ec.tmbundle new file mode 160000 index 00000000..b8ec2d32 --- /dev/null +++ b/vendor/grammars/ec.tmbundle @@ -0,0 +1 @@ +Subproject commit b8ec2d32afcaaef5ed7471d80aa4e0cd8717944d From 3a1b17f1f9a652b2d318ecc35260a5e16caf3bbd Mon Sep 17 00:00:00 2001 From: Lars Brinkhoff Date: Tue, 3 Feb 2015 13:04:06 +0100 Subject: [PATCH 10/55] Add .4TH Forth extension. --- lib/linguist/languages.yml | 1 + samples/Forth/tools.4TH | 133 +++++++++++++++++++++++++++++++++++++ 2 files changed, 134 insertions(+) create mode 100644 samples/Forth/tools.4TH diff --git a/lib/linguist/languages.yml b/lib/linguist/languages.yml index c492ed36..660fe5bc 100644 --- a/lib/linguist/languages.yml +++ b/lib/linguist/languages.yml @@ -918,6 +918,7 @@ Forth: color: "#341708" extensions: - .fth + - .4TH - .4th - .F - .f diff --git a/samples/Forth/tools.4TH b/samples/Forth/tools.4TH new file mode 100644 index 00000000..b08a29fe --- /dev/null +++ b/samples/Forth/tools.4TH @@ -0,0 +1,133 @@ +\ -*- forth -*- Copyright 2004, 2013 Lars Brinkhoff + +( Tools words. ) + +: .s ( -- ) + [char] < emit depth (.) ." > " + 'SP @ >r r@ depth 1- cells + + begin + dup r@ <> + while + dup @ . + /cell - + repeat r> 2drop ; + +: ? @ . ; + +: c? c@ . ; + +: dump bounds do i ? /cell +loop cr ; + +: cdump bounds do i c? loop cr ; + +: again postpone branch , ; immediate + +: see-find ( caddr -- end xt ) + >r here lastxt @ + begin + dup 0= abort" Undefined word" + dup r@ word= if r> drop exit then + nip dup >nextxt + again ; + +: cabs ( char -- |char| ) dup 127 > if 256 swap - then ; + +: xt. ( xt -- ) + ( >name ) count cabs type ; + +: xt? ( xt -- flag ) + >r lastxt @ begin + ?dup + while + dup r@ = if r> 2drop -1 exit then + >nextxt + repeat r> drop 0 ; + +: disassemble ( x -- ) + dup xt? if + ( >name ) count + dup 127 > if ." postpone " then + cabs type + else + . + then ; + +: .addr dup . ; + +: see-line ( addr -- ) + cr ." ( " .addr ." ) " @ disassemble ; + +: see-word ( end xt -- ) + >r ." : " r@ xt. + r@ >body do i see-line /cell +loop + ." ;" r> c@ 127 > if ." immediate" then ; + +: see bl word see-find see-word cr ; + +: #body bl word see-find >body - ; + +: type-word ( end xt -- flag ) + xt. space drop 0 ; + +: traverse-dictionary ( in.. xt -- out.. ) + \ xt execution: ( in.. end xt2 -- in.. 0 | in.. end xt2 -- out.. true ) + >r here lastxt @ begin + ?dup + while + r> 2dup >r >r execute + if r> r> 2drop exit then + r> dup >nextxt + repeat r> 2drop ; + +: words ( -- ) + ['] type-word traverse-dictionary cr ; + +\ ---------------------------------------------------------------------- + +( Tools extension words. ) + +\ ;code + +\ assembler + +\ in kernel: bye + +\ code + +\ cs-pick + +\ cs-roll + +\ editor + +: forget ' dup >nextxt lastxt ! 'here ! reveal ; + +\ Kernel: state + +\ [else] + +\ [if] + +\ [then] + +\ ---------------------------------------------------------------------- + +( Forth2012 tools extension words. ) + +\ TODO: n>r + +\ TODO: nr> + +\ TODO: synonym + +: [undefined] bl-word find nip 0= ; immediate + +: [defined] postpone [undefined] invert ; immediate + +\ ---------------------------------------------------------------------- + +: @+ ( addr -- addr+/cell x ) dup cell+ swap @ ; + +: !+ ( x addr -- addr+/cell ) tuck ! cell+ ; + +: -rot swap >r swap r> ; From 2077fa3837b040c943b2e738a234b61931a1292f Mon Sep 17 00:00:00 2001 From: Lars Brinkhoff Date: Sun, 1 Feb 2015 13:10:39 +0100 Subject: [PATCH 11/55] 'Text' doesn't qualify as a valid modeline language. --- lib/linguist/strategy/modeline.rb | 2 +- test/fixtures/Data/Modelines/example_smalltalk.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/linguist/strategy/modeline.rb b/lib/linguist/strategy/modeline.rb index 6a07fe3f..f8d8d52a 100644 --- a/lib/linguist/strategy/modeline.rb +++ b/lib/linguist/strategy/modeline.rb @@ -1,7 +1,7 @@ module Linguist module Strategy class Modeline - EmacsModeline = /-\*-\s*(?:mode:)?\s*(\w+);?\s*-\*-/ + EmacsModeline = /-\*-\s*mode:\s*(\w+);?\s*-\*-/ VimModeline = /\/\*\s*vim:\s*set\s*(?:ft|filetype)=(\w+):\s*\*\// # Public: Detects language based on Vim and Emacs modelines diff --git a/test/fixtures/Data/Modelines/example_smalltalk.md b/test/fixtures/Data/Modelines/example_smalltalk.md index 6a927e40..edf4bc13 100644 --- a/test/fixtures/Data/Modelines/example_smalltalk.md +++ b/test/fixtures/Data/Modelines/example_smalltalk.md @@ -1 +1 @@ -; -*-Smalltalk-*- +; -*-mode:Smalltalk-*- From d496aaae5587ec58b2f27e2b68108410de95f905 Mon Sep 17 00:00:00 2001 From: Arfon Smith Date: Wed, 4 Feb 2015 08:28:55 -0600 Subject: [PATCH 12/55] Grammar update --- vendor/grammars/AutoHotkey | 2 +- vendor/grammars/NimLime | 2 +- vendor/grammars/ats.sublime | 2 +- vendor/grammars/ceylon-sublimetext | 2 +- vendor/grammars/factor | 2 +- vendor/grammars/fsharpbinding | 2 +- vendor/grammars/haxe-sublime-bundle | 2 +- vendor/grammars/latex.tmbundle | 2 +- vendor/grammars/mercury-tmlanguage | 2 +- vendor/grammars/objective-c.tmbundle | 2 +- 10 files changed, 10 insertions(+), 10 deletions(-) diff --git a/vendor/grammars/AutoHotkey b/vendor/grammars/AutoHotkey index 371f9b23..9b42c86e 160000 --- a/vendor/grammars/AutoHotkey +++ b/vendor/grammars/AutoHotkey @@ -1 +1 @@ -Subproject commit 371f9b23071bf4f8df74bf162b5f4441ec6f0bb0 +Subproject commit 9b42c86e75a78e0f3c37d87476c1d943803fa76e diff --git a/vendor/grammars/NimLime b/vendor/grammars/NimLime index 1be33dac..75811539 160000 --- a/vendor/grammars/NimLime +++ b/vendor/grammars/NimLime @@ -1 +1 @@ -Subproject commit 1be33dac08c34595584bb60115d9b31402e3a843 +Subproject commit 75811539ec1a32e72821a540e1ef73fce0febd43 diff --git a/vendor/grammars/ats.sublime b/vendor/grammars/ats.sublime index 1e49e0b7..d954ef6b 160000 --- a/vendor/grammars/ats.sublime +++ b/vendor/grammars/ats.sublime @@ -1 +1 @@ -Subproject commit 1e49e0b7e00b7be2b971c4ac3b247d7255e09603 +Subproject commit d954ef6b27dbdbed6276c7b9b63d3a39d9f69a2c diff --git a/vendor/grammars/ceylon-sublimetext b/vendor/grammars/ceylon-sublimetext index a81ad702..07029801 160000 --- a/vendor/grammars/ceylon-sublimetext +++ b/vendor/grammars/ceylon-sublimetext @@ -1 +1 @@ -Subproject commit a81ad702b450a440c9d366fd2e128ea6a895dabd +Subproject commit 070298013e732cc7233c5169181a4a65c0ad6ef9 diff --git a/vendor/grammars/factor b/vendor/grammars/factor index 14b3261b..4ada3288 160000 --- a/vendor/grammars/factor +++ b/vendor/grammars/factor @@ -1 +1 @@ -Subproject commit 14b3261befc8a2b98fda8bd3203af7845d4cdcb1 +Subproject commit 4ada3288806d32beaf68e84c079653106e366d5f diff --git a/vendor/grammars/fsharpbinding b/vendor/grammars/fsharpbinding index f8dd50c3..92d969b1 160000 --- a/vendor/grammars/fsharpbinding +++ b/vendor/grammars/fsharpbinding @@ -1 +1 @@ -Subproject commit f8dd50c35cb775cf2db0bf10b77c54093004eec2 +Subproject commit 92d969b19715192928d0c865c87a1aac3ba98dac diff --git a/vendor/grammars/haxe-sublime-bundle b/vendor/grammars/haxe-sublime-bundle index 6359431d..50c5aa0e 160000 --- a/vendor/grammars/haxe-sublime-bundle +++ b/vendor/grammars/haxe-sublime-bundle @@ -1 +1 @@ -Subproject commit 6359431d8837ea90f245b29def9e2827ff3f040b +Subproject commit 50c5aa0e10f277f83dfe3d0e269a1043271be4df diff --git a/vendor/grammars/latex.tmbundle b/vendor/grammars/latex.tmbundle index 2e8d7c93..16154cef 160000 --- a/vendor/grammars/latex.tmbundle +++ b/vendor/grammars/latex.tmbundle @@ -1 +1 @@ -Subproject commit 2e8d7c939700548144a2e6f6ef299aa0e3215f76 +Subproject commit 16154cef75708eabc5ebd83c61ad49e8830d835d diff --git a/vendor/grammars/mercury-tmlanguage b/vendor/grammars/mercury-tmlanguage index eaef0b06..1cb8e949 160000 --- a/vendor/grammars/mercury-tmlanguage +++ b/vendor/grammars/mercury-tmlanguage @@ -1 +1 @@ -Subproject commit eaef0b0643b2cea0d7d26056f2dd264c5a652be9 +Subproject commit 1cb8e94922803658040bc29aa732c1671e2afe5b diff --git a/vendor/grammars/objective-c.tmbundle b/vendor/grammars/objective-c.tmbundle index 33c6be40..8387be89 160000 --- a/vendor/grammars/objective-c.tmbundle +++ b/vendor/grammars/objective-c.tmbundle @@ -1 +1 @@ -Subproject commit 33c6be4028e8c52191d84b23773021c17db3de99 +Subproject commit 8387be8975e47cd8e4314cb197e4fc2834159807 From b8f3078966e599f98a469f4c4b401e8db5be2305 Mon Sep 17 00:00:00 2001 From: Arfon Smith Date: Wed, 4 Feb 2015 08:31:36 -0600 Subject: [PATCH 13/55] Bumping version to v4.3.1 --- lib/linguist/version.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/linguist/version.rb b/lib/linguist/version.rb index cffac61e..e8c4f98b 100644 --- a/lib/linguist/version.rb +++ b/lib/linguist/version.rb @@ -1,3 +1,3 @@ module Linguist - VERSION = "4.3.0" + VERSION = "4.3.1" end From d7b97915145d01f73cacf3d9527e84ccc00c23e8 Mon Sep 17 00:00:00 2001 From: Paul Chaignon Date: Wed, 4 Feb 2015 19:47:23 +0100 Subject: [PATCH 14/55] Remove grammar for AutoHotkey --- .gitmodules | 3 --- grammars.yml | 2 -- vendor/grammars/AutoHotkey | 1 - 3 files changed, 6 deletions(-) delete mode 160000 vendor/grammars/AutoHotkey diff --git a/.gitmodules b/.gitmodules index 6aa15594..425a486e 100644 --- a/.gitmodules +++ b/.gitmodules @@ -552,9 +552,6 @@ [submodule "vendor/grammars/liquid.tmbundle"] path = vendor/grammars/liquid.tmbundle url = https://github.com/bastilian/validcode-textmate-bundles -[submodule "vendor/grammars/AutoHotkey"] - path = vendor/grammars/AutoHotkey - url = https://github.com/ahkscript/AutoHotkey [submodule "vendor/grammars/ats.sublime"] path = vendor/grammars/ats.sublime url = https://github.com/steinwaywhw/ats-mode-sublimetext diff --git a/grammars.yml b/grammars.yml index d4b76122..2f539728 100644 --- a/grammars.yml +++ b/grammars.yml @@ -24,8 +24,6 @@ vendor/grammars/Agda.tmbundle: - source.agda vendor/grammars/Alloy.tmbundle: - source.alloy -vendor/grammars/AutoHotkey: -- source.ahk vendor/grammars/CLIPS-sublime: - source.clips vendor/grammars/ColdFusion: diff --git a/vendor/grammars/AutoHotkey b/vendor/grammars/AutoHotkey deleted file mode 160000 index 371f9b23..00000000 --- a/vendor/grammars/AutoHotkey +++ /dev/null @@ -1 +0,0 @@ -Subproject commit 371f9b23071bf4f8df74bf162b5f4441ec6f0bb0 From 0479a8998226a7d30db43803b3a6a76b9155cbe6 Mon Sep 17 00:00:00 2001 From: Paul Chaignon Date: Wed, 4 Feb 2015 19:57:48 +0100 Subject: [PATCH 15/55] Update URL for AutoHotkey grammar --- .gitmodules | 3 +++ grammars.yml | 2 ++ vendor/grammars/AutoHotkey | 1 + 3 files changed, 6 insertions(+) create mode 160000 vendor/grammars/AutoHotkey diff --git a/.gitmodules b/.gitmodules index 425a486e..d0afdd65 100644 --- a/.gitmodules +++ b/.gitmodules @@ -588,3 +588,6 @@ [submodule "vendor/grammars/sublime-text-ox"] path = vendor/grammars/sublime-text-ox url = https://github.com/andreashetland/sublime-text-ox +[submodule "vendor/grammars/AutoHotkey"] + path = vendor/grammars/AutoHotkey + url = https://github.com/ahkscript/SublimeAutoHotkey diff --git a/grammars.yml b/grammars.yml index 2f539728..1c82a254 100644 --- a/grammars.yml +++ b/grammars.yml @@ -24,6 +24,8 @@ vendor/grammars/Agda.tmbundle: - source.agda vendor/grammars/Alloy.tmbundle: - source.alloy +vendor/grammars/AutoHotkey/: +- source.ahk vendor/grammars/CLIPS-sublime: - source.clips vendor/grammars/ColdFusion: diff --git a/vendor/grammars/AutoHotkey b/vendor/grammars/AutoHotkey new file mode 160000 index 00000000..9b42c86e --- /dev/null +++ b/vendor/grammars/AutoHotkey @@ -0,0 +1 @@ +Subproject commit 9b42c86e75a78e0f3c37d87476c1d943803fa76e From bb9f6ff082409cd44d434873edb89aedca4da8ae Mon Sep 17 00:00:00 2001 From: Arfon Smith Date: Wed, 4 Feb 2015 13:51:25 -0600 Subject: [PATCH 16/55] Update README.md --- README.md | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 2bebfafe..0234e2f3 100644 --- a/README.md +++ b/README.md @@ -110,7 +110,11 @@ See [Linguist::Generated#generated?](https://github.com/github/linguist/blob/mas ## Overrides -Linguist supports custom overrides for language definitions and vendored paths. Add a `.gitattributes` file to your project using the keys `linguist-language` and `linguist-vendored` with the standard git-style path matchers for the files you want to override. +Linguist supports a number of different custom overrides strategies for language definitions and vendored paths. + +### Using gitattributes + +Add a `.gitattributes` file to your project using the keys `linguist-language` and `linguist-vendored` with the standard git-style path matchers for the files you want to override. Please note that the overrides currently only affect the language statistics for a repository and not the syntax-highlighting of files. @@ -133,6 +137,20 @@ special-vendored-path/* linguist-vendored jquery.js linguist-vendored=false ``` +### Using Emacs and Vim modelines + +Alternatively, you can use Vim and Emacs style modelines to set the language for a single file. Modelines can be placed anywhere within a file and are respected when determining how to syntax-highlight a file on GitHub.com + +``` +Vim-style +vim: set filetype=prolog: +vim: set ft=cpp: + +Emacs-style +-*- mode: php;-*- +``` + + ## Installation Github.com is usually running the latest version of the `github-linguist` gem that is released on [RubyGems.org](http://rubygems.org/gems/github-linguist). From 67fae52b32f7e57749dceea1d039324b8425ddfc Mon Sep 17 00:00:00 2001 From: Arfon Smith Date: Wed, 4 Feb 2015 13:52:08 -0600 Subject: [PATCH 17/55] Vim and Emacs modelines --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 0234e2f3..cdc71098 100644 --- a/README.md +++ b/README.md @@ -142,11 +142,11 @@ jquery.js linguist-vendored=false Alternatively, you can use Vim and Emacs style modelines to set the language for a single file. Modelines can be placed anywhere within a file and are respected when determining how to syntax-highlight a file on GitHub.com ``` -Vim-style +Vim vim: set filetype=prolog: vim: set ft=cpp: -Emacs-style +Emacs -*- mode: php;-*- ``` From a664b9dd0cb2980e2545c11e5ab428bb7ffb13b6 Mon Sep 17 00:00:00 2001 From: Paul Chaignon Date: Wed, 4 Feb 2015 21:26:42 +0100 Subject: [PATCH 18/55] Sample for Inno Setup --- samples/Inno Setup/expat.iss | 69 ++++++++++++++++++++++++++++++++++++ 1 file changed, 69 insertions(+) create mode 100644 samples/Inno Setup/expat.iss diff --git a/samples/Inno Setup/expat.iss b/samples/Inno Setup/expat.iss new file mode 100644 index 00000000..4ccdbf2c --- /dev/null +++ b/samples/Inno Setup/expat.iss @@ -0,0 +1,69 @@ +; Basic setup script for the Inno Setup installer builder. For more +; information on the free installer builder, see www.jrsoftware.org. +; +; This script was contributed by Tim Peters. +; It was designed for Inno Setup 2.0.19 but works with later versions as well. + +[Setup] +AppName=Expat +AppId=expat +AppVersion=2.1.0 +AppVerName=Expat 2.1.0 +AppCopyright=Copyright 1998-2012 Thai Open Source Software Center, Clark Cooper, and the Expat maintainers +AppPublisher=The Expat Developers +AppPublisherURL=http://www.libexpat.org/ +AppSupportURL=http://www.libexpat.org/ +AppUpdatesURL=http://www.libexpat.org/ +UninstallDisplayName=Expat XML Parser 2.1.0 +VersionInfoVersion=2.1.0 + +DefaultDirName={pf}\Expat 2.1.0 +UninstallFilesDir={app}\Uninstall + +Compression=lzma +SolidCompression=yes +SourceDir=.. +OutputDir=win32 +DisableStartupPrompt=yes +AllowNoIcons=yes +DisableProgramGroupPage=yes +DisableReadyPage=yes + +[Files] +Flags: ignoreversion; Source: win32\bin\Release\xmlwf.exe; DestDir: "{app}\Bin" +Flags: ignoreversion; Source: win32\MANIFEST.txt; DestDir: "{app}" +Flags: ignoreversion; Source: Changes; DestDir: "{app}"; DestName: Changes.txt +Flags: ignoreversion; Source: COPYING; DestDir: "{app}"; DestName: COPYING.txt +Flags: ignoreversion; Source: README; DestDir: "{app}"; DestName: README.txt +Flags: ignoreversion; Source: doc\*.html; DestDir: "{app}\Doc" +Flags: ignoreversion; Source: doc\*.css; DestDir: "{app}\Doc" +Flags: ignoreversion; Source: doc\*.png; DestDir: "{app}\Doc" +Flags: ignoreversion; Source: win32\bin\Release\*.dll; DestDir: "{app}\Bin" +Flags: ignoreversion; Source: win32\bin\Release\*.lib; DestDir: "{app}\Bin" +Flags: ignoreversion; Source: expat.dsw; DestDir: "{app}\Source" +Flags: ignoreversion; Source: win32\README.txt; DestDir: "{app}\Source" +Flags: ignoreversion; Source: bcb5\*.bp*; DestDir: "{app}\Source\bcb5" +Flags: ignoreversion; Source: bcb5\*.mak; DestDir: "{app}\Source\bcb5" +Flags: ignoreversion; Source: bcb5\*.def; DestDir: "{app}\Source\bcb5" +Flags: ignoreversion; Source: bcb5\*.txt; DestDir: "{app}\Source\bcb5" +Flags: ignoreversion; Source: bcb5\*.bat; DestDir: "{app}\Source\bcb5" +Flags: ignoreversion; Source: lib\*.c; DestDir: "{app}\Source\lib" +Flags: ignoreversion; Source: lib\*.h; DestDir: "{app}\Source\lib" +Flags: ignoreversion; Source: lib\*.def; DestDir: "{app}\Source\lib" +Flags: ignoreversion; Source: lib\*.dsp; DestDir: "{app}\Source\lib" +Flags: ignoreversion; Source: examples\*.c; DestDir: "{app}\Source\examples" +Flags: ignoreversion; Source: examples\*.dsp; DestDir: "{app}\Source\examples" +Flags: ignoreversion; Source: tests\*.c; DestDir: "{app}\Source\tests" +Flags: ignoreversion; Source: tests\*.cpp; DestDir: "{app}\Source\tests" +Flags: ignoreversion; Source: tests\*.h; DestDir: "{app}\Source\tests" +Flags: ignoreversion; Source: tests\README.txt; DestDir: "{app}\Source\tests" +Flags: ignoreversion; Source: tests\benchmark\*.c; DestDir: "{app}\Source\tests\benchmark" +Flags: ignoreversion; Source: tests\benchmark\*.ds*; DestDir: "{app}\Source\tests\benchmark" +Flags: ignoreversion; Source: tests\benchmark\README.txt; DestDir: "{app}\Source\tests\benchmark" +Flags: ignoreversion; Source: xmlwf\*.c*; DestDir: "{app}\Source\xmlwf" +Flags: ignoreversion; Source: xmlwf\*.h; DestDir: "{app}\Source\xmlwf" +Flags: ignoreversion; Source: xmlwf\*.dsp; DestDir: "{app}\Source\xmlwf" + +[Messages] +WelcomeLabel1=Welcome to the Expat XML Parser Setup Wizard +WelcomeLabel2=This will install [name/ver] on your computer.%n%nExpat is an XML parser with a C-language API, and is primarily made available to allow developers to build applications which use XML using a portable API and fast implementation.%n%nIt is strongly recommended that you close all other applications you have running before continuing. This will help prevent any conflicts during the installation process. From 2e76ce740e7d64be7d19b07fbc23b35bbaa155c5 Mon Sep 17 00:00:00 2001 From: Paul Chaignon Date: Wed, 4 Feb 2015 21:28:36 +0100 Subject: [PATCH 19/55] Grammar for Inno Setup from Sublime Text package --- .gitmodules | 3 +++ grammars.yml | 2 ++ lib/linguist/languages.yml | 2 +- vendor/grammars/InnoSetup | 1 + 4 files changed, 7 insertions(+), 1 deletion(-) create mode 160000 vendor/grammars/InnoSetup diff --git a/.gitmodules b/.gitmodules index 4e72920d..bb4871bd 100644 --- a/.gitmodules +++ b/.gitmodules @@ -594,3 +594,6 @@ [submodule "vendor/grammars/ec.tmbundle"] path = vendor/grammars/ec.tmbundle url = https://github.com/ecere/ec.tmbundle +[submodule "vendor/grammars/InnoSetup"] + path = vendor/grammars/InnoSetup + url = https://github.com/idleberg/InnoSetup-Sublime-Text diff --git a/grammars.yml b/grammars.yml index 00f2c9b1..d7dd24de 100644 --- a/grammars.yml +++ b/grammars.yml @@ -51,6 +51,8 @@ vendor/grammars/Handlebars: - text.html.handlebars vendor/grammars/IDL-Syntax: - source.webidl +vendor/grammars/InnoSetup/: +- source.inno vendor/grammars/Isabelle.tmbundle: - source.isabelle.root - source.isabelle.theory diff --git a/lib/linguist/languages.yml b/lib/linguist/languages.yml index e899046e..28fdae5a 100644 --- a/lib/linguist/languages.yml +++ b/lib/linguist/languages.yml @@ -1346,7 +1346,7 @@ Inform 7: Inno Setup: extensions: - .iss - tm_scope: none + tm_scope: source.inno ace_mode: text Io: diff --git a/vendor/grammars/InnoSetup b/vendor/grammars/InnoSetup new file mode 160000 index 00000000..875ba96c --- /dev/null +++ b/vendor/grammars/InnoSetup @@ -0,0 +1 @@ +Subproject commit 875ba96c32df269f5be92bd366d20260f811d844 From 6ca149de1ddebab4eb64939bfd073d840fd523c4 Mon Sep 17 00:00:00 2001 From: Park Hyunwoo Date: Thu, 5 Feb 2015 10:27:14 +0900 Subject: [PATCH 20/55] Missing whitespace --- lib/linguist/vendor.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/linguist/vendor.yml b/lib/linguist/vendor.yml index 4ba241ca..9f6b401b 100644 --- a/lib/linguist/vendor.yml +++ b/lib/linguist/vendor.yml @@ -40,7 +40,7 @@ # Minified JavaScript and CSS - (\.|-)min\.(js|css)$ -#Stylesheets imported from packages +# Stylesheets imported from packages - ([^\s]*)import\.(css|less|scss|styl)$ # Bootstrap css and js From 201b0ba53c2f7744c12ffc8e9bc78c9a41716ced Mon Sep 17 00:00:00 2001 From: Arfon Smith Date: Fri, 6 Feb 2015 08:41:37 -0600 Subject: [PATCH 21/55] Making modelines case-insensitive --- lib/linguist/strategy/modeline.rb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/linguist/strategy/modeline.rb b/lib/linguist/strategy/modeline.rb index f8d8d52a..e77597f4 100644 --- a/lib/linguist/strategy/modeline.rb +++ b/lib/linguist/strategy/modeline.rb @@ -1,8 +1,8 @@ module Linguist module Strategy class Modeline - EmacsModeline = /-\*-\s*mode:\s*(\w+);?\s*-\*-/ - VimModeline = /\/\*\s*vim:\s*set\s*(?:ft|filetype)=(\w+):\s*\*\// + EmacsModeline = /-\*-\s*mode:\s*(\w+);?\s*-\*-/i + VimModeline = /\/\*\s*vim:\s*set\s*(?:ft|filetype)=(\w+):\s*\*\//i # Public: Detects language based on Vim and Emacs modelines # From 0db1d1c8ca2267f0e30c26ae31af773044056897 Mon Sep 17 00:00:00 2001 From: Arfon Smith Date: Fri, 6 Feb 2015 08:48:59 -0600 Subject: [PATCH 22/55] Modifying some modeline fixtures to test case InSeNsItivitY --- test/fixtures/Data/Modelines/iamphp.inc | 2 +- test/fixtures/Data/Modelines/not_perl.pl | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/test/fixtures/Data/Modelines/iamphp.inc b/test/fixtures/Data/Modelines/iamphp.inc index 14370309..9e67098e 100644 --- a/test/fixtures/Data/Modelines/iamphp.inc +++ b/test/fixtures/Data/Modelines/iamphp.inc @@ -1 +1 @@ -; -*- mode: php;-*- +; -*- MoDe: PhP;-*- diff --git a/test/fixtures/Data/Modelines/not_perl.pl b/test/fixtures/Data/Modelines/not_perl.pl index df8e9fc5..39e38958 100644 --- a/test/fixtures/Data/Modelines/not_perl.pl +++ b/test/fixtures/Data/Modelines/not_perl.pl @@ -1,3 +1,3 @@ -/* vim: set filetype=prolog: */ +/* vim: set filEtype=pRoloG: */ # I am Prolog From 462a570d3c683557b12e3473e93be3fda072ab76 Mon Sep 17 00:00:00 2001 From: Brandon Keepers Date: Fri, 6 Feb 2015 16:04:32 -0500 Subject: [PATCH 23/55] Friendly welcome --- CONTRIBUTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index ade8bffd..31021753 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,6 +1,6 @@ # Contributing -The majority of contributions won't need to touch any Ruby code at all. +Hi there! We're thrilled that you'd like to contribute to this project. Your help is essential for keeping it great. The majority of contributions won't need to touch any Ruby code at all. ## Adding a language From 5c29ce06959695ceeda3597b51ed5e7fd12583b2 Mon Sep 17 00:00:00 2001 From: Brandon Keepers Date: Fri, 6 Feb 2015 16:26:55 -0500 Subject: [PATCH 24/55] Add usage and link to contributing --- README.md | 41 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) diff --git a/README.md b/README.md index 215aaf52..84b53b62 100644 --- a/README.md +++ b/README.md @@ -55,3 +55,44 @@ vim: set ft=cpp: Emacs -*- mode: php;-*- ``` + +## Usage + +Install the gem: + +``` +$ gem install github-linguist +``` + +Then use it in your application: + +```ruby +require 'rugged' +require 'linguist' + +repo = Rugged::Repository.new('.') +project = Linguist::Repository.new(repo, repo.head.target_id) +project.language #=> "Ruby" +project.languages #=> { "Ruby" => 119387 } +``` + +These stats are also printed out by the `linguist` executable. You can use the +`--breakdown` flag, and the binary will also output the breakdown of files by language. + +You can try running `linguist` on the root directory in this repository itself: + +$ bundle exec linguist --breakdown + +100.00% Ruby + +Ruby: +Gemfile +Rakefile +bin/linguist +github-linguist.gemspec +lib/linguist.rb +… + +## Contributing + +Please check out our [contributing guidelines](CONTRIBUTING.md). From 107a27aa2560f5a8114b19e9fd3ec0d88553aef4 Mon Sep 17 00:00:00 2001 From: Brandon Keepers Date: Fri, 6 Feb 2015 16:41:57 -0500 Subject: [PATCH 25/55] Fix formatting of example --- README.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/README.md b/README.md index 84b53b62..4a7ea415 100644 --- a/README.md +++ b/README.md @@ -81,6 +81,7 @@ These stats are also printed out by the `linguist` executable. You can use the You can try running `linguist` on the root directory in this repository itself: +``` $ bundle exec linguist --breakdown 100.00% Ruby @@ -92,7 +93,10 @@ bin/linguist github-linguist.gemspec lib/linguist.rb … +``` ## Contributing Please check out our [contributing guidelines](CONTRIBUTING.md). + +## From 4543c7a0b307a648c7342e23cb970f56c48f3e80 Mon Sep 17 00:00:00 2001 From: Arfon Smith Date: Sat, 7 Feb 2015 08:47:17 -0600 Subject: [PATCH 26/55] Use the shebang strategy first --- lib/linguist/language.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/linguist/language.rb b/lib/linguist/language.rb index 7fbf8a96..2b6e597f 100644 --- a/lib/linguist/language.rb +++ b/lib/linguist/language.rb @@ -96,8 +96,8 @@ module Linguist STRATEGIES = [ Linguist::Strategy::Modeline, - Linguist::Strategy::Filename, Linguist::Shebang, + Linguist::Strategy::Filename, Linguist::Heuristics, Linguist::Classifier ] From 88e79cd3a8bc7749783693a35ea4409a4709303a Mon Sep 17 00:00:00 2001 From: Arfon Smith Date: Sat, 7 Feb 2015 10:24:03 -0600 Subject: [PATCH 27/55] Adding fixtures to test shebang strategy ordering --- test/fixtures/Shell/crossbuild_liblua5.1 | 57 +++++++++++++ test/fixtures/Shell/graylog2-server.init.d | 99 ++++++++++++++++++++++ 2 files changed, 156 insertions(+) create mode 100644 test/fixtures/Shell/crossbuild_liblua5.1 create mode 100755 test/fixtures/Shell/graylog2-server.init.d diff --git a/test/fixtures/Shell/crossbuild_liblua5.1 b/test/fixtures/Shell/crossbuild_liblua5.1 new file mode 100644 index 00000000..d4d14462 --- /dev/null +++ b/test/fixtures/Shell/crossbuild_liblua5.1 @@ -0,0 +1,57 @@ +#! /bin/sh +# +# Builds and installs liblua5.1 for the cross toolchain. +# Executed by build-uqm-dependencies.chroot + +# Include our common functions +. /usr/lib/crossbuild/crossbuild.subr + +# envvar LIBLUA51_URL +# +# Specifies the URL of the liblua5.1 source tarball you want to use. +export LIBLUA51_URL="http://www.lua.org/ftp/lua-5.1.5.tar.gz" + +# envvar INSTALL_TOP +# +# This determines where lua's makefiles install everything (we don't want to use +# /usr/local!). +export INSTALL_TOP="/usr/${HOST_TRIPLET}" + +# envvar TO_BIN +# +# Names of the binary files to install (that's right, lua's makefiles don't +# determine this automatically, and since we end up with files named according +# to Windows conventions the install chokes without these) +export TO_BIN="lua.exe luac.exe" + +# envvar TO_LIB +# +# Names of the libraries to install, see TO_BIN +export TO_LIB="liblua.a lua51.dll" + + +# liblua5.1 uses custom makefiles and does not natively support cross-building. +# However, with our cross toolchain in its PATH it successfully builds the mingw +# target. +export PATH=/usr/${HOST_TRIPLET}/bin:${PATH} + +echo "*************************************************************************" +echo "--- BEGIN: crossbuild_liblua5.1 ---" + +get_tarball "liblua5.1" "${LIBLUA51_URL}" gz + +cd ${SRC_ROOT_DIR}/liblua5.1/* + +if [ -f Makefile ]; then + make clean + make --environment-overrides mingw install + + +else + echo "crossbuild_liblua5.1 failed: Could not find Makefile" + echo "(is the liblua5.1 source tarball sane?)" + exit 1 +fi + +echo "--- END: crossbuild_liblua5.1 ---" +echo "*************************************************************************" diff --git a/test/fixtures/Shell/graylog2-server.init.d b/test/fixtures/Shell/graylog2-server.init.d new file mode 100755 index 00000000..29e513ad --- /dev/null +++ b/test/fixtures/Shell/graylog2-server.init.d @@ -0,0 +1,99 @@ +#!/bin/bash + +### BEGIN INIT INFO +# Provides: graylog2-server +# Required-Start: $network +# Required-Stop: $network +# Default-Start: 2 3 4 5 +# Default-Stop: 1 +# Short-Description: Start Graylog2 server +### END INIT INFO + +# Written by Lital Natan + +PREFIX=/usr +SHAREDIR=$PREFIX/share/graylog2-server +SERVER_JAR=$SHAREDIR/graylog2-server.jar +SYSLOG4J_JAR=$SHAREDIR/syslog4j-0.9.46-bin.jar +SVCNAME="graylog2-server" + +CONFIG="/etc/graylog2.conf" +LOGFILE="/var/log/graylog2.log" +PIDFILE="/var/run/graylog2.pid" + +start() { + if [ ! -e $CONFIG ]; then + echo "Config file $CONFIG does not exist" + return 1 + fi + + echo "Starting ${SVCNAME}" + nohup `which java` -cp $SERVER_JAR:$SYSLOG4J_JAR org.graylog2.Main \ + -p ${PIDFILE} -f ${CONFIG} > $LOGFILE 2>&1 & + + # Sleep before testing the service + sleep 2 + + graylog2_test || return 1 +} + +stop() { + pid=`< $PIDFILE` + kill $pid + rm -f ${PIDFILE} # just in case +} + +graylog2_test() { + # Graylog2 only deletes its PID file if it hits a config error + if [ ! -e ${PIDFILE} ]; then + echo "Configuration error, check ${CONFIG}" + return 1 + fi + + local pid=`cat ${PIDFILE}` + + # Graylog2 isn't running, so that means there was a problem + if [ ! -e /proc/$pid ]; then + echo "Something went wrong, check ${LOGFILE}" + rm -f ${PIDFILE} + return 1 + else + return 0 + fi +} + + +status() { + graylog2_test > /dev/null 2>&1 + if [ "$?" == "0" ]; then + echo "Graylog2 server is up" + return 0 + else + echo "Graylog2 server is down" + return 1 + fi +} + + +restart() { + stop + start +} + +case "$1" in + start) + start + ;; + stop) + stop + ;; + status) + status + ;; + restart) + restart + ;; + *) + echo "Usage $0 {start|stop|restart|status}" + RETVAL=1 +esac From a08f0da30d4147e1625130fe02ab9ad813704328 Mon Sep 17 00:00:00 2001 From: Paul Chaignon Date: Sat, 7 Feb 2015 19:58:49 +0100 Subject: [PATCH 28/55] Grammar for Mediawiki from TextMate bundle --- .gitmodules | 3 +++ grammars.yml | 2 ++ lib/linguist/languages.yml | 2 +- vendor/grammars/mediawiki.tmbundle | 1 + 4 files changed, 7 insertions(+), 1 deletion(-) create mode 160000 vendor/grammars/mediawiki.tmbundle diff --git a/.gitmodules b/.gitmodules index bb4871bd..73029e34 100644 --- a/.gitmodules +++ b/.gitmodules @@ -597,3 +597,6 @@ [submodule "vendor/grammars/InnoSetup"] path = vendor/grammars/InnoSetup url = https://github.com/idleberg/InnoSetup-Sublime-Text +[submodule "vendor/grammars/mediawiki.tmbundle"] + path = vendor/grammars/mediawiki.tmbundle + url = https://github.com/textmate/mediawiki.tmbundle diff --git a/grammars.yml b/grammars.yml index d7dd24de..b4a9c6e7 100644 --- a/grammars.yml +++ b/grammars.yml @@ -338,6 +338,8 @@ vendor/grammars/matlab.tmbundle: - source.octave vendor/grammars/maven.tmbundle: - text.xml.pom +vendor/grammars/mediawiki.tmbundle/: +- text.html.mediawiki vendor/grammars/mercury-tmlanguage: - source.mercury vendor/grammars/monkey.tmbundle: diff --git a/lib/linguist/languages.yml b/lib/linguist/languages.yml index 28fdae5a..9b197fd3 100644 --- a/lib/linguist/languages.yml +++ b/lib/linguist/languages.yml @@ -1807,7 +1807,7 @@ MediaWiki: wrap: true extensions: - .mediawiki - tm_scope: none + tm_scope: text.html.mediawiki ace_mode: text Mercury: diff --git a/vendor/grammars/mediawiki.tmbundle b/vendor/grammars/mediawiki.tmbundle new file mode 160000 index 00000000..f8dead50 --- /dev/null +++ b/vendor/grammars/mediawiki.tmbundle @@ -0,0 +1 @@ +Subproject commit f8dead507a1aed539376b9fcfde877a855842e8e From 4c037c644f26bb22152edb09b72b9cd145e6e47c Mon Sep 17 00:00:00 2001 From: Paul Chaignon Date: Sun, 8 Feb 2015 10:43:45 +0100 Subject: [PATCH 29/55] Grammar for GAP from TextMate bundle --- .gitmodules | 3 +++ grammars.yml | 2 ++ lib/linguist/languages.yml | 2 +- vendor/grammars/gap-tmbundle | 1 + 4 files changed, 7 insertions(+), 1 deletion(-) create mode 160000 vendor/grammars/gap-tmbundle diff --git a/.gitmodules b/.gitmodules index bb4871bd..a4462410 100644 --- a/.gitmodules +++ b/.gitmodules @@ -597,3 +597,6 @@ [submodule "vendor/grammars/InnoSetup"] path = vendor/grammars/InnoSetup url = https://github.com/idleberg/InnoSetup-Sublime-Text +[submodule "vendor/grammars/gap-tmbundle"] + path = vendor/grammars/gap-tmbundle + url = https://github.com/dhowden/gap-tmbundle diff --git a/grammars.yml b/grammars.yml index d7dd24de..fc6a808e 100644 --- a/grammars.yml +++ b/grammars.yml @@ -225,6 +225,8 @@ vendor/grammars/fortran.tmbundle: - source.fortran.modern vendor/grammars/fsharpbinding: - source.fsharp +vendor/grammars/gap-tmbundle/: +- source.gap vendor/grammars/gettext.tmbundle: - source.po vendor/grammars/gnuplot-tmbundle: diff --git a/lib/linguist/languages.yml b/lib/linguist/languages.yml index 28fdae5a..c8938202 100644 --- a/lib/linguist/languages.yml +++ b/lib/linguist/languages.yml @@ -961,7 +961,7 @@ GAP: - .gd - .gi - .tst - tm_scope: none + tm_scope: source.gap ace_mode: text GAS: diff --git a/vendor/grammars/gap-tmbundle b/vendor/grammars/gap-tmbundle new file mode 160000 index 00000000..291a0469 --- /dev/null +++ b/vendor/grammars/gap-tmbundle @@ -0,0 +1 @@ +Subproject commit 291a0469dd4eb7082b08e828edc8bf0cfbb9b599 From d17f5dfd9e82e189dc56477eef24e63102eae31a Mon Sep 17 00:00:00 2001 From: Paul Chaignon Date: Sun, 8 Feb 2015 11:14:18 +0100 Subject: [PATCH 30/55] Papyrus grammar from Sublime Text package --- .gitmodules | 3 +++ grammars.yml | 4 ++++ lib/linguist/languages.yml | 2 +- vendor/grammars/SublimePapyrus | 1 + 4 files changed, 9 insertions(+), 1 deletion(-) create mode 160000 vendor/grammars/SublimePapyrus diff --git a/.gitmodules b/.gitmodules index a4462410..01ef7c5f 100644 --- a/.gitmodules +++ b/.gitmodules @@ -600,3 +600,6 @@ [submodule "vendor/grammars/gap-tmbundle"] path = vendor/grammars/gap-tmbundle url = https://github.com/dhowden/gap-tmbundle +[submodule "vendor/grammars/SublimePapyrus"] + path = vendor/grammars/SublimePapyrus + url = https://github.com/Kapiainen/SublimePapyrus diff --git a/grammars.yml b/grammars.yml index fc6a808e..66ae4138 100644 --- a/grammars.yml +++ b/grammars.yml @@ -112,6 +112,10 @@ vendor/grammars/Sublime-VimL: - source.viml vendor/grammars/SublimeBrainfuck: - source.bf +vendor/grammars/SublimePapyrus/: +- source.compiled-papyrus +- source.papyrus +- source.papyrus-assembly vendor/grammars/SublimeXtend: - source.xtend vendor/grammars/TXL/: diff --git a/lib/linguist/languages.yml b/lib/linguist/languages.yml index c8938202..ac54b94a 100644 --- a/lib/linguist/languages.yml +++ b/lib/linguist/languages.yml @@ -2158,7 +2158,7 @@ Papyrus: color: "#6600cc" extensions: - .psc - tm_scope: none + tm_scope: source.papyrus ace_mode: text Parrot: diff --git a/vendor/grammars/SublimePapyrus b/vendor/grammars/SublimePapyrus new file mode 160000 index 00000000..5a7b3e7b --- /dev/null +++ b/vendor/grammars/SublimePapyrus @@ -0,0 +1 @@ +Subproject commit 5a7b3e7b29fdee835d0198d1f20e21c0ff874154 From cc5881dca22f914a186be3d23a7eecaae55f4a7e Mon Sep 17 00:00:00 2001 From: Paul Chaignon Date: Sun, 8 Feb 2015 11:20:40 +0100 Subject: [PATCH 31/55] Grammar for Propeller Spin from Sublime Text package --- .gitmodules | 3 +++ grammars.yml | 3 +++ lib/linguist/languages.yml | 2 +- vendor/grammars/sublime-spintools | 1 + 4 files changed, 8 insertions(+), 1 deletion(-) create mode 160000 vendor/grammars/sublime-spintools diff --git a/.gitmodules b/.gitmodules index 01ef7c5f..e7bcdac5 100644 --- a/.gitmodules +++ b/.gitmodules @@ -603,3 +603,6 @@ [submodule "vendor/grammars/SublimePapyrus"] path = vendor/grammars/SublimePapyrus url = https://github.com/Kapiainen/SublimePapyrus +[submodule "vendor/grammars/sublime-spintools"] + path = vendor/grammars/sublime-spintools + url = https://github.com/bitbased/sublime-spintools diff --git a/grammars.yml b/grammars.yml index 66ae4138..ded5e7cb 100644 --- a/grammars.yml +++ b/grammars.yml @@ -465,6 +465,9 @@ vendor/grammars/sublime-rust: - source.rust vendor/grammars/sublime-sourcepawn: - source.sp +vendor/grammars/sublime-spintools/: +- source.regexp.spin +- source.spin vendor/grammars/sublime-tea: - source.tea vendor/grammars/sublime-text-ox/: diff --git a/lib/linguist/languages.yml b/lib/linguist/languages.yml index ac54b94a..b02b4148 100644 --- a/lib/linguist/languages.yml +++ b/lib/linguist/languages.yml @@ -2323,7 +2323,7 @@ Propeller Spin: color: "#2b446d" extensions: - .spin - tm_scope: none + tm_scope: source.spin ace_mode: text Protocol Buffer: diff --git a/vendor/grammars/sublime-spintools b/vendor/grammars/sublime-spintools new file mode 160000 index 00000000..56d326a4 --- /dev/null +++ b/vendor/grammars/sublime-spintools @@ -0,0 +1 @@ +Subproject commit 56d326a44ec0be0f301d5fb1d9e1ada9944554bb From 30d2883436cba5733031536200b148ae6361833a Mon Sep 17 00:00:00 2001 From: Paul Chaignon Date: Sun, 8 Feb 2015 11:44:11 +0100 Subject: [PATCH 32/55] Grammar for PogoScript from TextMate bundle --- .gitmodules | 3 +++ grammars.yml | 2 ++ lib/linguist/languages.yml | 2 +- vendor/grammars/PogoScript.tmbundle | 1 + 4 files changed, 7 insertions(+), 1 deletion(-) create mode 160000 vendor/grammars/PogoScript.tmbundle diff --git a/.gitmodules b/.gitmodules index e7bcdac5..016d28e9 100644 --- a/.gitmodules +++ b/.gitmodules @@ -606,3 +606,6 @@ [submodule "vendor/grammars/sublime-spintools"] path = vendor/grammars/sublime-spintools url = https://github.com/bitbased/sublime-spintools +[submodule "vendor/grammars/PogoScript.tmbundle"] + path = vendor/grammars/PogoScript.tmbundle + url = https://github.com/featurist/PogoScript.tmbundle diff --git a/grammars.yml b/grammars.yml index ded5e7cb..cfaa8bfc 100644 --- a/grammars.yml +++ b/grammars.yml @@ -72,6 +72,8 @@ vendor/grammars/NimLime: - source.nimcfg vendor/grammars/PHP-Twig.tmbundle: - text.html.twig +vendor/grammars/PogoScript.tmbundle/: +- source.pogoscript vendor/grammars/RDoc.tmbundle: - text.rdoc vendor/grammars/Racket: diff --git a/lib/linguist/languages.yml b/lib/linguist/languages.yml index b02b4148..cb274d59 100644 --- a/lib/linguist/languages.yml +++ b/lib/linguist/languages.yml @@ -2276,7 +2276,7 @@ PogoScript: color: "#d80074" extensions: - .pogo - tm_scope: none + tm_scope: source.pogoscript ace_mode: text PostScript: diff --git a/vendor/grammars/PogoScript.tmbundle b/vendor/grammars/PogoScript.tmbundle new file mode 160000 index 00000000..2255586f --- /dev/null +++ b/vendor/grammars/PogoScript.tmbundle @@ -0,0 +1 @@ +Subproject commit 2255586f9ec69bab052b2250615db9b824774178 From 0019abe5e7f8f63fb9e23a1d499adc119b8b6dcc Mon Sep 17 00:00:00 2001 From: Paul Chaignon Date: Sun, 8 Feb 2015 11:45:54 +0100 Subject: [PATCH 33/55] Opal grammar from Sublime Text package --- .gitmodules | 3 +++ grammars.yml | 3 +++ lib/linguist/languages.yml | 2 +- vendor/grammars/sublime-opal | 1 + 4 files changed, 8 insertions(+), 1 deletion(-) create mode 160000 vendor/grammars/sublime-opal diff --git a/.gitmodules b/.gitmodules index 016d28e9..65e6843d 100644 --- a/.gitmodules +++ b/.gitmodules @@ -609,3 +609,6 @@ [submodule "vendor/grammars/PogoScript.tmbundle"] path = vendor/grammars/PogoScript.tmbundle url = https://github.com/featurist/PogoScript.tmbundle +[submodule "vendor/grammars/sublime-opal"] + path = vendor/grammars/sublime-opal + url = https://github.com/artifactz/sublime-opal diff --git a/grammars.yml b/grammars.yml index cfaa8bfc..8774c5ea 100644 --- a/grammars.yml +++ b/grammars.yml @@ -461,6 +461,9 @@ vendor/grammars/sublime-nginx: - source.nginx vendor/grammars/sublime-nix: - source.nix +vendor/grammars/sublime-opal/: +- source.opal +- source.opalsysdefs vendor/grammars/sublime-robot-plugin: - text.robot vendor/grammars/sublime-rust: diff --git a/lib/linguist/languages.yml b/lib/linguist/languages.yml index cb274d59..5321c940 100644 --- a/lib/linguist/languages.yml +++ b/lib/linguist/languages.yml @@ -2052,7 +2052,7 @@ Opal: color: "#f7ede0" extensions: - .opal - tm_scope: none + tm_scope: source.opal ace_mode: text OpenCL: diff --git a/vendor/grammars/sublime-opal b/vendor/grammars/sublime-opal new file mode 160000 index 00000000..55ae90b9 --- /dev/null +++ b/vendor/grammars/sublime-opal @@ -0,0 +1 @@ +Subproject commit 55ae90b910933889d01c55766424db5b3e10332e From 74cd03de0b4a0abd0b6a0d544f35b6b1a2c3dfe0 Mon Sep 17 00:00:00 2001 From: Brandon Keepers Date: Mon, 9 Feb 2015 09:57:24 -0500 Subject: [PATCH 34/55] Remove old link in docs --- CONTRIBUTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 31021753..026de664 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -33,7 +33,7 @@ Misclassifications can often be solved by either adding a new filename or extens Syntax highlighting in GitHub is performed using TextMate-compatible grammars. These are the same grammars that TextMate, Sublime Text and Atom use. Every language in [languages.yml][languages] is mapped to its corresponding TM `scope`. This scope will be used when picking up a grammar for highlighting. -Assuming your code is being detected as the right language (see [Language Detection](#language-detection) above), in most cases this is due to a bug in the language grammar rather than a bug in Linguist. [`grammars.yml`][grammars] lists all the grammars we use for syntax highlighting on github.com. Find the one corresponding to your code's programming language and submit a bug report upstream. If you can, try to reproduce the highlighting problem in the text editor that the grammar is designed for (TextMate, Sublime Text, or Atom) and include that information in your bug report. +Assuming your code is being detected as the right language, in most cases this is due to a bug in the language grammar rather than a bug in Linguist. [`grammars.yml`][grammars] lists all the grammars we use for syntax highlighting on github.com. Find the one corresponding to your code's programming language and submit a bug report upstream. If you can, try to reproduce the highlighting problem in the text editor that the grammar is designed for (TextMate, Sublime Text, or Atom) and include that information in your bug report. You can also try to fix the bug yourself and submit a Pull Request. [TextMate's documentation](http://manual.macromates.com/en/language_grammars) offers a good introduction on how to work with TextMate-compatible grammars. You can test grammars using [Lightshow](https://lightshow.githubapp.com). From bdc724d5485ea71ab81309f8732fca1416fe37b9 Mon Sep 17 00:00:00 2001 From: Brandon Keepers Date: Mon, 9 Feb 2015 10:19:36 -0500 Subject: [PATCH 35/55] Detect "use 5.xxx" as Perl --- lib/linguist/heuristics.rb | 2 +- samples/Perl/use5.pl | 3 +++ test/test_heuristics.rb | 2 +- 3 files changed, 5 insertions(+), 2 deletions(-) create mode 100644 samples/Perl/use5.pl diff --git a/lib/linguist/heuristics.rb b/lib/linguist/heuristics.rb index ee1a3625..a28b054a 100644 --- a/lib/linguist/heuristics.rb +++ b/lib/linguist/heuristics.rb @@ -92,7 +92,7 @@ module Linguist disambiguate "Perl", "Perl6", "Prolog" do |data| if data.include?("use v6") Language["Perl6"] - elsif data.include?("use strict") + elsif data.match(/use strict|use\s+v?5\./) Language["Perl"] elsif data.include?(":-") Language["Prolog"] diff --git a/samples/Perl/use5.pl b/samples/Perl/use5.pl new file mode 100644 index 00000000..de7d248c --- /dev/null +++ b/samples/Perl/use5.pl @@ -0,0 +1,3 @@ +use Mojolicious::Lite; +use 5.20.0; +use experimental 'signatures'; diff --git a/test/test_heuristics.rb b/test/test_heuristics.rb index 3ea4af78..02671b78 100644 --- a/test/test_heuristics.rb +++ b/test/test_heuristics.rb @@ -48,7 +48,7 @@ class TestHeuristcs < Minitest::Test def test_pl_prolog_perl_by_heuristics assert_heuristics({ "Prolog" => "Prolog/turing.pl", - "Perl" => "Perl/perl-test.t", + "Perl" => ["Perl/perl-test.t", "Perl/use5.pl"] }) end From 72acbc567bc2fc09b03efaa6240e16a8f95bc540 Mon Sep 17 00:00:00 2001 From: Adam Roben Date: Mon, 9 Feb 2015 13:37:54 -0500 Subject: [PATCH 36/55] Update Lightshow's URL This is the public URL going forward. The old URL will redirect to the new one. --- CONTRIBUTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 026de664..b1c8a09c 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -35,7 +35,7 @@ Syntax highlighting in GitHub is performed using TextMate-compatible grammars. T Assuming your code is being detected as the right language, in most cases this is due to a bug in the language grammar rather than a bug in Linguist. [`grammars.yml`][grammars] lists all the grammars we use for syntax highlighting on github.com. Find the one corresponding to your code's programming language and submit a bug report upstream. If you can, try to reproduce the highlighting problem in the text editor that the grammar is designed for (TextMate, Sublime Text, or Atom) and include that information in your bug report. -You can also try to fix the bug yourself and submit a Pull Request. [TextMate's documentation](http://manual.macromates.com/en/language_grammars) offers a good introduction on how to work with TextMate-compatible grammars. You can test grammars using [Lightshow](https://lightshow.githubapp.com). +You can also try to fix the bug yourself and submit a Pull Request. [TextMate's documentation](http://manual.macromates.com/en/language_grammars) offers a good introduction on how to work with TextMate-compatible grammars. You can test grammars using [Lightshow](https://github-lightshow.herokuapp.com). Once the bug has been fixed upstream, please let us know and we'll pick it up for GitHub. From 11f7e6e1b4e53e4bdd47580353384d6eb6bbec3a Mon Sep 17 00:00:00 2001 From: Paul Chaignon Date: Mon, 9 Feb 2015 21:29:53 +0100 Subject: [PATCH 37/55] Add Papyrus and GAP grammars to whitelist --- test/test_grammars.rb | 24 ++++++++++++++---------- 1 file changed, 14 insertions(+), 10 deletions(-) diff --git a/test/test_grammars.rb b/test/test_grammars.rb index 188bc187..f30289a6 100644 --- a/test/test_grammars.rb +++ b/test/test_grammars.rb @@ -3,12 +3,16 @@ require_relative "./helper" class TestGrammars < Minitest::Test ROOT = File.expand_path("../..", __FILE__) - # These grammars have no license but have been grandfathered in. New grammars - # must have a license that allows redistribution. - UNLICENSED_GRAMMARS_WHITELIST = %w[ - vendor/grammars/Sublime-Lasso - vendor/grammars/Sublime-REBOL - vendor/grammars/x86-assembly-textmate-bundle + LICENSE_WHITELIST = [ + # This grammar's MIT license is inside a subdirectory. + "vendor/grammars/SublimePapyrus", + "vendor/grammars/gap-tmbundle", + + # These grammars have no license but have been grandfathered in. New grammars + # must have a license that allows redistribution. + "vendor/grammars/Sublime-Lasso", + "vendor/grammars/Sublime-REBOL", + "vendor/grammars/x86-assembly-textmate-bundle", ].freeze def setup @@ -77,9 +81,9 @@ class TestGrammars < Minitest::Test unlicensed = categories[:unlicensed] || [] unrecognized = categories[:unrecognized] || [] - disallowed_unlicensed = unlicensed - UNLICENSED_GRAMMARS_WHITELIST - disallowed_unrecognized = unrecognized - UNLICENSED_GRAMMARS_WHITELIST - extra_whitelist_entries = UNLICENSED_GRAMMARS_WHITELIST - (unlicensed | unrecognized) + disallowed_unlicensed = unlicensed - LICENSE_WHITELIST + disallowed_unrecognized = unrecognized - LICENSE_WHITELIST + extra_whitelist_entries = LICENSE_WHITELIST - (unlicensed | unrecognized) message = "" if disallowed_unlicensed.any? @@ -93,7 +97,7 @@ class TestGrammars < Minitest::Test end if extra_whitelist_entries.any? message << "\n\n" unless message.empty? - message << "The following grammar submodules are listed in UNLICENSED_GRAMMARS_WHITELIST but either have a license (yay!)\n" + message << "The following grammar submodules are listed in LICENSE_WHITELIST but either have a license (yay!)\n" message << "or have been removed from the repository. Please remove them from the whitelist.\n" message << extra_whitelist_entries.sort.join("\n") end From d11b2f05bb1bc106d5617f6d3fbd2b391a95c005 Mon Sep 17 00:00:00 2001 From: Paul Chaignon Date: Mon, 9 Feb 2015 22:57:19 +0100 Subject: [PATCH 38/55] Detection of zlib license --- test/test_grammars.rb | 2 ++ 1 file changed, 2 insertions(+) diff --git a/test/test_grammars.rb b/test/test_grammars.rb index f30289a6..f71446d8 100644 --- a/test/test_grammars.rb +++ b/test/test_grammars.rb @@ -135,6 +135,8 @@ class TestGrammars < Minitest::Test "unlicense" elsif content.include?("http://www.wtfpl.net/txt/copying/") "WTFPL" + elsif content.include?("zlib") && content.include?("license") && content.include?("2. Altered source versions must be plainly marked as such") + "zlib" end end end From 4eb8903bed2c509eaedf80aee4840072179d41db Mon Sep 17 00:00:00 2001 From: Paul Chaignon Date: Mon, 9 Feb 2015 23:01:24 +0100 Subject: [PATCH 39/55] Comment for GAP grammar in whitelist --- test/test_grammars.rb | 2 ++ 1 file changed, 2 insertions(+) diff --git a/test/test_grammars.rb b/test/test_grammars.rb index f71446d8..18cbce28 100644 --- a/test/test_grammars.rb +++ b/test/test_grammars.rb @@ -6,6 +6,8 @@ class TestGrammars < Minitest::Test LICENSE_WHITELIST = [ # This grammar's MIT license is inside a subdirectory. "vendor/grammars/SublimePapyrus", + + # This grammar has a nonstandard but acceptable license. "vendor/grammars/gap-tmbundle", # These grammars have no license but have been grandfathered in. New grammars From b2ee2cc7b8b72953025fe27116e370ab8df30265 Mon Sep 17 00:00:00 2001 From: Adam Roben Date: Tue, 10 Feb 2015 13:31:11 -0500 Subject: [PATCH 40/55] Detect all markup languages when computing language statistics Originally, only "programming" languages were included in repository language statistics. In 33ebee0f6a1e097550df0c7a69c5ffe57223f558 we started detecting a few selected "markup" languages as well. We didn't include all "markup" languages because at the time formats like Markdown and AsciiDoc were labeled as "markup" languages, and we thought that including those prose (i.e., non-code) languages in repository statistics on github.com was misleading for repositories that are largely about code but also contain a lot of documentation (e.g., rails/rails). This hand-picked set of whitelisted "markup" languages can cause strange categorization for some repositories. For example, it includes CSS (and some variants) but not HTML. This results in repositories that contain the source code for a static website being classified as either a JavaScript (programming) or CSS (markup) repository, with no mention of HTML anywhere. Fast-forward to today, and prose languages are no longer "markup" languages; they're now "prose" languages. So now we can include all "markup" languages in repository language statistics without worrying about undesirable effects for documentation-heavy repositories. --- lib/linguist/language.rb | 7 ------- lib/linguist/repository.rb | 5 +++-- 2 files changed, 3 insertions(+), 9 deletions(-) diff --git a/lib/linguist/language.rb b/lib/linguist/language.rb index 7fbf8a96..35c58c30 100644 --- a/lib/linguist/language.rb +++ b/lib/linguist/language.rb @@ -32,13 +32,6 @@ module Linguist # Valid Languages types TYPES = [:data, :markup, :programming, :prose] - # Names of non-programming languages that we will still detect - # - # Returns an array - def self.detectable_markup - ["CSS", "Less", "Sass", "SCSS", "Stylus", "TeX"] - end - # Detect languages by a specific type # # type - A symbol that exists within TYPES diff --git a/lib/linguist/repository.rb b/lib/linguist/repository.rb index 41e829c5..3c197fad 100644 --- a/lib/linguist/repository.rb +++ b/lib/linguist/repository.rb @@ -8,6 +8,8 @@ module Linguist # Its primary purpose is for gathering language statistics across # the entire project. class Repository + DETECTABLE_TYPES = [:programming, :markup].freeze + attr_reader :repository # Public: Create a new Repository based on the stats of @@ -159,8 +161,7 @@ module Linguist # Skip vendored or generated blobs next if blob.vendored? || blob.generated? || blob.language.nil? - # Only include programming languages and acceptable markup languages - if blob.language.type == :programming || Language.detectable_markup.include?(blob.language.name) + if DETECTABLE_TYPES.include?(blob.language.type) file_map[new] = [blob.language.group.name, blob.size] end end From 9a2f2c1fb24ce9aa3eb7fb770d16218544a9d962 Mon Sep 17 00:00:00 2001 From: Lars Brinkhoff Date: Tue, 10 Feb 2015 20:06:00 +0100 Subject: [PATCH 41/55] Tweak heuristic for free-form Fortran. --- lib/linguist/heuristics.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/linguist/heuristics.rb b/lib/linguist/heuristics.rb index a28b054a..d4b99f4e 100644 --- a/lib/linguist/heuristics.rb +++ b/lib/linguist/heuristics.rb @@ -157,7 +157,7 @@ module Linguist disambiguate "FORTRAN", "Forth" do |data| if /^: /.match(data) Language["Forth"] - elsif /^([c*][^a-z]| (subroutine|program)\s|!)/i.match(data) + elsif /^([c*][^a-z]| (subroutine|program)\s|\s*!)/i.match(data) Language["FORTRAN"] end end From 4e4a18c71a56e0a45dcca3673a49fe0c1190b3d9 Mon Sep 17 00:00:00 2001 From: Adam Roben Date: Tue, 10 Feb 2015 14:08:58 -0500 Subject: [PATCH 42/55] Classify CSS as a markup language It was the only language in Language.detectable_markup that wasn't actually classified as "markup". --- lib/linguist/languages.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/linguist/languages.yml b/lib/linguist/languages.yml index c02ebfea..75d30673 100644 --- a/lib/linguist/languages.yml +++ b/lib/linguist/languages.yml @@ -438,6 +438,7 @@ COBOL: ace_mode: cobol CSS: + type: markup ace_mode: css color: "#563d7c" extensions: From 3475aefd042e97240b0cff509fc6a1db20e3fc24 Mon Sep 17 00:00:00 2001 From: Jan Brecka Date: Tue, 10 Feb 2015 21:15:45 +0100 Subject: [PATCH 43/55] change submodule url --- .gitmodules | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.gitmodules b/.gitmodules index d4ae5db2..82559a75 100644 --- a/.gitmodules +++ b/.gitmodules @@ -249,7 +249,7 @@ url = https://github.com/shellderp/sublime-robot-plugin [submodule "vendor/grammars/actionscript3-tmbundle"] path = vendor/grammars/actionscript3-tmbundle - url = https://github.com/simongregory/actionscript3-tmbundle + url = https://github.com/honzabrecka/actionscript3-tmbundle [submodule "vendor/grammars/Sublime-QML"] path = vendor/grammars/Sublime-QML url = https://github.com/skozlovf/Sublime-QML From e0c1107a253405c41763a9064dbf06ce0c8863fb Mon Sep 17 00:00:00 2001 From: Adam Roben Date: Tue, 10 Feb 2015 15:33:13 -0500 Subject: [PATCH 44/55] Classify SVG files as data This keeps SVG files (which are really just images, even though they happen to be represented as text) out of repository language statistics. --- lib/linguist/languages.yml | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/lib/linguist/languages.yml b/lib/linguist/languages.yml index 75d30673..972a9b67 100644 --- a/lib/linguist/languages.yml +++ b/lib/linguist/languages.yml @@ -2684,6 +2684,13 @@ STON: tm_scope: source.smalltalk ace_mode: text +SVG: + type: data + extensions: + - .svg + tm_scope: source.xml + ace_mode: xml + Sage: type: programming group: Python @@ -3199,7 +3206,6 @@ XML: - .srdf - .stTheme - .sublime-snippet - - .svg - .targets - .tmCommand - .tmLanguage From d673010420066179e21fd9501a512b46e268f22e Mon Sep 17 00:00:00 2001 From: Lars Brinkhoff Date: Wed, 4 Feb 2015 08:04:51 +0100 Subject: [PATCH 45/55] Add .nasm extension for Assembly files. Sample file by Chris Hinsley. --- lib/linguist/languages.yml | 1 + samples/Assembly/forth.nasm | 2841 +++++++++++++++++++++++++++++++++++ 2 files changed, 2842 insertions(+) create mode 100644 samples/Assembly/forth.nasm diff --git a/lib/linguist/languages.yml b/lib/linguist/languages.yml index c02ebfea..33a22975 100644 --- a/lib/linguist/languages.yml +++ b/lib/linguist/languages.yml @@ -206,6 +206,7 @@ Assembly: - .asm - .ASM - .a51 + - .nasm tm_scope: source.asm.x86 ace_mode: assembly_x86 diff --git a/samples/Assembly/forth.nasm b/samples/Assembly/forth.nasm new file mode 100644 index 00000000..c411c7ff --- /dev/null +++ b/samples/Assembly/forth.nasm @@ -0,0 +1,2841 @@ +;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; +;; A Forth by Chris Hinsley +;; nasm -f macho forth.nasm +;; ld -o forth -e _main forth.o +;; ./forth +;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; + + %define VERSION_NUM 30 + + ; various buffer area sizes + %define DATA_STACK_SIZE 1024 + %define USER_DEFS_SIZE (64*1024) + %define NUM_HASH_CHAINS 64 + %define MAX_LINE_SIZE 128 + + %define SYS_exit 1 + %define SYS_read 3 + %define SYS_write 4 + %define SYS_open 5 + %define SYS_close 6 + %define SYS_unlink 10 + %define SYS_mprotect 74 + %define SYS_fsync 95 + %define SYS_rename 128 + %define SYS_stat 188 + %define SYS_lseek 199 + %define SYS_fstat 189 + %define SYS_ftruncate 201 + + %define PROT_READ 0x01 ;pages can be read + %define PROT_WRITE 0x02 ;pages can be written + %define PROT_EXEC 0x04 ;pages can be executed + %define PROT_ALL (PROT_READ | PROT_WRITE | PROT_EXEC) + %define PAGE_SIZE 4096 + +;;;;;;;;;;;;;;;;;;;;;;;;;; +; some NASM codeing macros +;;;;;;;;;;;;;;;;;;;;;;;;;; + + %macro loopstart 0 + %push loopstart + %$loop_start: + %endmacro + + %macro break 0 + jmp %$loop_exit + %endmacro + + %macro breakif 1 + j%+1 %$loop_exit + %endmacro + + %macro loopend 0 + jmp %$loop_start + %$loop_exit: + %pop + %endmacro + + %macro repeat 0 + %push repeat + %$loop_start: + %endmacro + + %macro until 1 + j%-1 %$loop_start + %$loop_exit: + %pop + %endmacro + + %macro if 1 + %push if + j%-1 %$ifnot + %endmacro + + %macro else 0 + %ifctx if + %repl else + jmp %$ifend + %$ifnot: + %else + %error "expected `if' before `else'" + %endif + %endmacro + + %macro endif 0 + %ifctx if + %$ifnot: + %pop + %elifctx else + %$ifend: + %pop + %else + %error "expected `if' or `else' before `endif'" + %endif + %endmacro + +;;;;;;;;;;;;;;;; +; base VM macros +;;;;;;;;;;;;;;;; + + ; eip Forths IP + ; esp Forths R + ; ebp Forths S + ; ebx Forths TOS + + ; push on to return stack + %macro PUSHRSP 1 + push %1 + %endm + + ; pop top of return stack + %macro POPRSP 1 + pop %1 + %endm + + ; save into return stack + %macro PUTRSP 2 + %if (%2 = 0) + mov [esp], %1 + %elif ((%2 >= -128) && (%2 < 128)) + mov [byte esp + %2], %1 + %else + mov [long esp + %2], %1 + %endif + %endm + + ; load from return stack + %macro PICKRSP 2 + %if (%2 = 0) + mov %1, [esp] + %elif ((%2 >= -128) && (%2 < 128)) + mov %1, [byte esp + %2] + %else + mov %1, [long esp + %2] + %endif + %endm + + ; set return stack + %macro SETRSP 1 + mov esp, %1 + %endm + + ; get return stack + %macro GETRSP 1 + mov %1, esp + %endm + + ; adjust return stack + %macro ADDRSP 1 + %if ((%1 >= -128) && (%1 < 128)) + add esp, byte %1 + %else + add esp, %1 + %endif + %endm + + ; push on to data stack + %macro PUSHDSP 1 + sub ebp, byte 4 + mov [ebp], %1 + %endm + + ; pop top of data stack + %macro POPDSP 1 + mov %1, [ebp] + add ebp, byte 4 + %endm + + ; save into data stack + %macro PUTDSP 2 + %if (%2 = 0) + mov [ebp], %1 + %elif ((%2 >= -128) && (%2 < 128)) + mov [byte ebp + %2], %1 + %else + mov [long ebp + %2], %1 + %endif + %endm + + ; load from data stack + %macro PICKDSP 2 + %if (%2 = 0) + mov %1, [ebp] + %elif ((%2 >= -128) && (%2 < 128)) + mov %1, [byte ebp + %2] + %else + mov %1, [long ebp + %2] + %endif + %endm + + ; set data stack + %macro SETDSP 1 + mov ebp, %1 + %endm + + ; get data stack + %macro GETDSP 1 + mov %1, ebp + %endm + + ; adjust data stack + %macro ADDDSP 1 + %if ((%1 >= -128) && (%1 < 128)) + add ebp, byte %1 + %else + add ebp, %1 + %endif + %endm + + ; load value onto data stack + %macro LOADTOS 1 + PUSHDSP ebx + mov ebx, %1 + %endm + + ; move from data to return stack + %macro TORSP 0 + PUSHRSP ebx + POPDSP ebx + %endm + + ; move from return to data stack + %macro FROMRSP 0 + PUSHDSP ebx + POPRSP ebx + %endm + + ; copy from return to data stack + %macro FETCHRSP 0 + PUSHDSP ebx + PICKRSP ebx, 0 + %endm + + ; align reg + %define DP_ALIGN 3 + %macro ALIGNREG 1 + add %1, byte DP_ALIGN + and %1, byte ~DP_ALIGN + %endm + +;;;;;;;;;;;;;;;;;;;;;;;;;;;; +; dictionary building macros +;;;;;;;;;;;;;;;;;;;;;;;;;;;; + + ; format of dictionary entry flag byte + %define F_IMMED 0x80 + %define F_HIDDEN 0x20 + %define F_LENMASK 0x1f + + %define NULL 0 + %define H_LLINK 0 + %define H_HLINK 4 + %define H_NSIZE 8 + %define H_NAME 9 + + %define XT_BODY -12 + %define XT_LENGTH -8 + %define XT_COMPILE -4 + %define XT_SIZE 12 + + %macro defword 4 + %push newword + %strlen len %1 + align 4 + dic_%3: + dd NULL ; LATEST list link + dd NULL ; hash chain link + db len + %2 ; flags + length byte + db %1 ; the name + dd %3 ; body pointer + dd %$code_end - %3 ; code length + dd %4 ; compile action word + %3: + %endm ; assembler code follows + + %macro defword_end 0 + %$code_end: + %pop + %endm + + %macro defvar 4 + defword %1, %2, %3, WORD_INLINE_COMMA + LOADTOS var_%3 + ret + defword_end + align 4 + var_%3: + dd %4 + %endm + + %macro defvar2 5 + defword %1, %2, %3, WORD_INLINE_COMMA + LOADTOS var_%3 + ret + defword_end + align 4 + var_%3: + dd %4 + dd %5 + %endm + + %macro defconst 4 + defword %1, %2, %3, WORD_INLINE_COMMA + LOADTOS %4 + ret + defword_end + %endm + +;;;;;;;;;;;;;;;;;;;;;;;;;; +; entry point +;;;;;;;;;;;;;;;;;;;;;;;;;; + + SECTION .text + global _main +_main: + ; use mprotect to allow read/write/execute of the data section + mov edx, forth_start + and edx, -PAGE_SIZE ;start address + mov ecx, forth_end + sub ecx, edx ;length + mov ebx, PROT_ALL ;flags + push ebx + push ecx + push edx + push 0 ;padding + mov eax, SYS_mprotect + int 0x80 + add esp, 16 + jmp forth_start + + SECTION .data +forth_start: + ; init data and return stacks, saving initial positions + ; in Forth vars R0 and S0 + cld + GETRSP [var_WORD_SZ] + SETDSP [var_WORD_SZ] + ADDRSP -DATA_STACK_SIZE + GETRSP [var_WORD_RZ] + + ; link built in dictionary + mov esi, dictionary_start + xor edi, edi + repeat + lodsd + mov [eax + H_LLINK], edi + mov edi, eax + push esi + mov cl, [eax + H_NSIZE] + and ecx, F_LENMASK + lea esi, [eax + H_NAME] + call strhashi + and ebx, NUM_HASH_CHAINS-1 + mov esi, hash_buckets + mov eax, [esi + (ebx * 4)] + mov [esi + (ebx * 4)], edi + mov [edi + H_HLINK], eax + pop esi + cmp esi, dictionary_end + until z + mov [var_WORD_LATEST], edi + + ; run temp interpreter loop till we can get into the real QUIT word + call WORD_LBRAC ; interpret state + LOADTOS 666q ; octal ! + TORSP + LOADTOS 0 + TORSP + LOADTOS bootfile + TORSP + call WORD_SYS_OPEN + call WORD_SYSCALL + ADDRSP 12 + TORSP ; ( fd ) of "forth.f" + loopstart + LOADTOS tib_buffer + LOADTOS MAX_LINE_SIZE + FETCHRSP ; ( c-addr len fd ) + call WORD_READLINE ; ( num flag flag ) + call WORD_DROP2 + LOADTOS tib_buffer + call WORD_SWAP + call WORD_INHASH + call WORD_STORE2 + LOADTOS 0 + call WORD_TOIN + call WORD_STORE + call WORD_INTERPRET + loopend ; and loop till QUIT takes over + +;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; +; a few case insensative string operations +;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; + + %macro to_lower 1 + ; lower case check + cmp %1, 'A' + if ge + cmp %1, 'Z' + if le + ; make it lower case + add %1, byte 'a' - 'A' + endif + endif + %endm + +strcpyi: + test ecx, ecx + if nz + strcpyi_l1: + lodsb + to_lower al + stosb + loop strcpyi_l1 + endif + ret + +strcmpi: + test ecx, ecx + if nz + strcmpi_l1: + lodsb + mov bl, [edi] + lea edi, [edi + 1] + to_lower al + to_lower bl + cmp bl, al + if z + loop strcmpi_l1 + endif + endif + ret + +;;;;;;;;;;;;;;; +; hash function +;;;;;;;;;;;;;;; + +strhashi: + mov ebx, 5381 + test ecx, ecx + if nz + mov edx, 33 + strhashi_l1: + lodsb + movzx eax, al + to_lower eax + imul ebx, edx + add ebx, eax + loop strhashi_l1 + endif + ret + +;;;;;;;;;;;;;;;;;;; +; syscall functions +;;;;;;;;;;;;;;;;;;; + +_syscall: + int 0x80 + if c + neg eax + endif + ret + +_lsyscall: + int 0x80 + if c + not eax + not edx + add eax, 1 + adc edx, 0 + endif + ret + +;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; +; built in variables +; STATE Is the interpreter executing code (0) or compiling a word (non-zero)? +; LATEST Points to the latest (most recently defined) word in the dictionary. +; DP Points to the next free byte of memory. When compiling, compiled words go here. +; S0 Stores the address of the top of the parameter stack. +; R0 The address of the top of the return stack. +; BASE The current base for printing and reading numbers. +; #IN The current input buffer descriptor. +; >IN The current input offset. +; SOURCEFD The current input source file descriptor. +; BLK The current block number. +; CHARBUF Single char buffer. +;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; + + defvar "state", 0, WORD_STATE, 0 + defvar "dp", 0, WORD_DP, dictionary_start + defvar "latest", 0, WORD_LATEST, 0 + defvar "s0", 0, WORD_SZ, 0 + defvar "r0", 0, WORD_RZ, 0 + defvar "base", 0, WORD_BASE, 10 + defvar2 "#IN", 0, WORD_INHASH, 0, 0 + defvar ">in", 0, WORD_TOIN, 0 + defvar "sourcefd", 0, WORD_SOURCEFD, 0 + defvar "blk", 0, WORD_BLK, 0 + defvar "charbuf", 0, WORD_CHARBUF, 0 + +;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; +; built in constants +; VERSION The current version of this FORTH. +; WORDBUF The address of the buffer WORD uses. +; LINESIZE The line buffer size. +; F_IMMED The IMMEDIATE flag's actual value. +; F_HIDDEN The HIDDEN flag's actual value. +; F_LENMASK The length mask in the flags/len byte. +; H_NSIZE The flags/len field offset. +; H_NAME The name field offset. +; XT_BODY The xt body pointer. +; XT_LENGTH The xt length field offset. +; XT_COMPILE The xt compile field offset. +; XT_SIZE The xt size offset. +; SYS_* The numeric codes of various syscalls. +; O_* Various sycall flags/modes. +;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; + + defconst "version", 0, WORD_VERSION, VERSION_NUM + defconst "wordbuf", 0, WORD_WORDBUF, word_buf + defconst "linesize", 0, WORD_LINESIZE, MAX_LINE_SIZE + defconst "f_immed", 0, WORD__F_IMMED, F_IMMED + defconst "f_hidden", 0, WORD__F_HIDDEN, F_HIDDEN + defconst "f_lenmask", 0, WORD__F_LENMASK, F_LENMASK + defconst "h_nsize", 0, WORD__H_NSIZE, H_NSIZE + defconst "h_name", 0, WORD__H_NAME, H_NAME + defconst "xt_body", 0, WORD__XT_BODY, XT_BODY + defconst "xt_length", 0, WORD__XT_LENGTH, XT_LENGTH + defconst "xt_compile", 0, WORD__XT_COMPILE, XT_COMPILE + defconst "xt_size", 0, WORD__XT_SIZE, XT_SIZE + + defconst "sys_exit", 0, WORD_SYS_EXIT, SYS_exit + defconst "sys_open", 0, WORD_SYS_OPEN, SYS_open + defconst "sys_close", 0, WORD_SYS_CLOSE, SYS_close + defconst "sys_read", 0, WORD_SYS_READ, SYS_read + defconst "sys_write", 0, WORD_SYS_WRITE, SYS_write + defconst "sys_unlink", 0, WORD_SYS_UNLINK, SYS_unlink + defconst "sys_rename", 0, WORD_SYS_RENAME, SYS_rename + defconst "sys_ftruncate", 0, WORD_SYS_FTRUNCATE, SYS_ftruncate + defconst "sys_fsync", 0, WORD_SYS_FSYNC, SYS_fsync + defconst "sys_lseek", 0, WORD_SYS_LSEEK, SYS_lseek + defconst "sys_fstat", 0, WORD_SYS_FSTAT, SYS_fstat + defconst "sys_stat", 0, WORD_SYS_STAT, SYS_stat + + defconst "o_rdonly", 0, WORD_O_RDONLY, 0x0 + defconst "o_wronly", 0, WORD_O_WRONLY, 0x1 + defconst "o_rdwr", 0, WORD_O_RDWR, 0x2 + defconst "o_creat", 0, WORD_O_CREAT, 0x100 + defconst "o_excl", 0, WORD_O_EXCL, 0x200 + defconst "o_trunc", 0, WORD_O_TRUNC, 0x1000 + defconst "o_append", 0, WORD_O_APPEND, 0x2000 + defconst "o_nonblock", 0, WORD_O_NONBLOCK, 0x4000 + +;;;;;;;;;;;;;;;;;;;;;;;;;;; +; data stack ordering words +;;;;;;;;;;;;;;;;;;;;;;;;;;; + + defword "dsp@", 0, WORD_DSPFETCH, WORD_INLINE_COMMA + PUSHDSP ebx + GETDSP ebx + ret + defword_end + + defword "dsp!", 0, WORD_DSPSTORE, WORD_INLINE_COMMA + SETDSP ebx + POPDSP ebx + ret + defword_end + + defword "drop", 0, WORD_DROP, WORD_INLINE_COMMA + POPDSP ebx + ret + defword_end + + defword "swap", 0, WORD_SWAP, WORD_INLINE_COMMA + xchg ebx, [ebp] + ret + defword_end + + defword "dup", 0, WORD_DUP, WORD_INLINE_COMMA + PUSHDSP ebx + ret + defword_end + + defword "over", 0, WORD_OVER, WORD_INLINE_COMMA + PUSHDSP ebx + PICKDSP ebx, 4 + ret + defword_end + + defword "rot", 0, WORD_ROT, WORD_INLINE_COMMA + mov eax, ebx + PICKDSP ecx, 0 + PICKDSP ebx, 4 + PUTDSP eax, 0 + PUTDSP ecx, 4 + ret + defword_end + + defword "-rot", 0, WORD_NROT, WORD_INLINE_COMMA + mov eax, ebx + PICKDSP ebx, 0 + PICKDSP ecx, 4 + PUTDSP ecx, 0 + PUTDSP eax, 4 + ret + defword_end + + defword "2drop", 0, WORD_DROP2, WORD_INLINE_COMMA + PICKDSP ebx, 4 + ADDDSP 8 + ret + defword_end + + defword "2dup", 0, WORD_DUP2, WORD_INLINE_COMMA + PICKDSP eax, 0 + ADDDSP -8 + PUTDSP eax, 0 + PUTDSP ebx, 4 + ret + defword_end + + defword "2swap", 0, WORD_SWAP2, WORD_INLINE_COMMA + mov eax, ebx + PICKDSP ecx, 0 + PICKDSP ebx, 4 + PICKDSP edx, 8 + PUTDSP edx, 0 + PUTDSP eax, 4 + PUTDSP ecx, 8 + ret + defword_end + + defword "2rot", 0, WORD_ROT2, WORD_INLINE_COMMA + mov eax, ebx + PICKDSP ecx, 16 + PICKDSP ebx, 12 + PICKDSP edx, 8 + PICKDSP edi, 4 + PICKDSP esi, 0 + PUTDSP edx, 16 + PUTDSP edi, 12 + PUTDSP esi, 8 + PUTDSP eax, 4 + PUTDSP ecx, 0 + ret + defword_end + + defword "?dup", 0, WORD_QDUP, WORD_INLINE_COMMA + test ebx, ebx + if nz + PUSHDSP ebx + endif + ret + defword_end + + defword "!?dup", 0, WORD_NQDUP, WORD_INLINE_COMMA + test ebx, ebx + if z + PUSHDSP ebx + endif + ret + defword_end + + defword "nip", 0, WORD_NIP, WORD_INLINE_COMMA + ADDDSP 4 + ret + defword_end + + defword "tuck", 0, WORD_TUCK, WORD_INLINE_COMMA + PICKDSP eax, 0 + PUTDSP ebx, 0 + PUSHDSP eax + ret + defword_end + + defword "pick", 0, WORD_PICK, WORD_INLINE_COMMA + mov ebx, [ebp + (ebx * 4)] + ret + defword_end + + defword "2tuck", 0, WORD_TUCK2, WORD_INLINE_COMMA + PICKDSP eax, 0 + PICKDSP ecx, 4 + PICKDSP edx, 8 + ADDDSP -8 + PUTDSP eax, 0 + PUTDSP ecx, 4 + PUTDSP edx, 8 + PUTDSP ebx, 12 + PUTDSP eax, 16 + ret + defword_end + + defword "2nip", 0, WORD_NIP2, WORD_INLINE_COMMA + PICKDSP eax, 0 + ADDDSP 8 + PUTDSP eax, 0 + ret + defword_end + + defword "2over", 0, WORD_OVER2, WORD_INLINE_COMMA + ADDDSP -8 + PUTDSP ebx, 4 + PICKDSP ebx, 16 + PUTDSP ebx, 0 + PICKDSP ebx, 12 + ret + defword_end + +;;;;;;;;;;;;;;;;;;;;;;;;;;;;; +; return stack ordering words +;;;;;;;;;;;;;;;;;;;;;;;;;;;;; + + defword ">r", 0, WORD_TOR, WORD_INLINE_COMMA + TORSP + ret + defword_end + + defword "r>", 0, WORD_FROMR, WORD_INLINE_COMMA + FROMRSP + ret + defword_end + + defword "2>r", 0, WORD_TOR2, WORD_INLINE_COMMA + ADDRSP -8 + PICKDSP ecx, 0 + PUTRSP ebx, 0 + PUTRSP ecx, 4 + PICKDSP ebx, 4 + ADDDSP 8 + ret + defword_end + + defword "2r>", 0, WORD_FROMR2, WORD_INLINE_COMMA + ADDDSP -8 + PUTDSP ebx, 4 + PICKRSP ebx, 0 + PICKRSP ecx, 4 + PUTDSP ecx, 0 + ADDRSP 8 + ret + defword_end + + defword "rsp@", 0, WORD_RSPFETCH, WORD_INLINE_COMMA + PUSHDSP ebx + GETRSP ebx + ret + defword_end + + defword "r@", 0, WORD_RFETCH, WORD_INLINE_COMMA + PUSHDSP ebx + PICKRSP ebx, 0 + ret + defword_end + + defword "r!", 0, WORD_RSTORE, WORD_INLINE_COMMA + PUTRSP ebx, 0 + POPDSP ebx + ret + defword_end + + defword "2r@", 0, WORD_RFETCH2, WORD_INLINE_COMMA + ADDDSP -8 + PUTDSP ebx, 4 + PICKRSP ebx, 4 + PICKRSP ecx, 0 + PUTDSP ecx, 0 + ret + defword_end + + defword "rsp!", 0, WORD_RSPSTORE, WORD_INLINE_COMMA + SETRSP ebx + POPDSP ebx + ret + defword_end + + defword "rdrop", 0, WORD_RDROP, WORD_INLINE_COMMA + ADDRSP 4 + ret + defword_end + + defword "2rdrop", 0, WORD_RDROP2, WORD_INLINE_COMMA + ADDRSP 8 + ret + defword_end + + defword "n>r", 0, WORD_NTOR, WORD_CALL_COMMA + PUSHDSP ebx + PICKRSP eax, 0 + mov ecx, ebx + inc ecx + neg ebx + lea esp, [esp + (ebx * 4)] + mov esi, ebp + mov edi, esp + rep movsd + mov ebp, esi + POPDSP ebx + jmp eax + defword_end + + defword "nr>", 0, WORD_NFROMR, WORD_CALL_COMMA + PUSHDSP ebx + POPRSP eax + PICKRSP ebx, 0 + inc ebx + mov ecx, ebx + neg ebx + lea ebp, [ebp + (ebx * 4)] + mov esi, esp + mov edi, ebp + rep movsd + mov esp, esi + POPDSP ebx + jmp eax + defword_end + +;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; +; memory fetch and store words +;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; + + defword "!", 0, WORD_STORE, WORD_INLINE_COMMA + PICKDSP eax, 0 + mov [ebx], eax + PICKDSP ebx, 4 + ADDDSP 8 + ret + defword_end + + defword "@", 0, WORD_FETCH, WORD_INLINE_COMMA + mov ebx, [ebx] + ret + defword_end + + defword "+!", 0, WORD_ADDSTORE, WORD_INLINE_COMMA + PICKDSP eax, 0 + add [ebx], eax + PICKDSP ebx, 4 + ADDDSP 8 + ret + defword_end + + defword "-!", 0, WORD_SUBSTORE, WORD_INLINE_COMMA + PICKDSP eax, 0 + sub [ebx], eax + PICKDSP ebx, 4 + ADDDSP 8 + ret + defword_end + + defword "c!", 0, WORD_STOREBYTE, WORD_INLINE_COMMA + PICKDSP eax, 0 + mov [ebx], al + PICKDSP ebx, 4 + ADDDSP 8 + ret + defword_end + + defword "c+!", 0, WORD_ADDBYTE, WORD_INLINE_COMMA + PICKDSP eax, 0 + add [ebx], al + PICKDSP ebx, 4 + ADDDSP 8 + ret + defword_end + + defword "c@", 0, WORD_FETCHBYTE, WORD_INLINE_COMMA + mov eax, ebx + xor ebx, ebx + mov bl, [eax] + ret + defword_end + + defword "w!", 0, WORD_STORESHORT, WORD_INLINE_COMMA + PICKDSP eax, 0 + mov [ebx], ax + PICKDSP ebx, 4 + ADDDSP 8 + ret + defword_end + + defword "w@", 0, WORD_FETCHSHORT, WORD_INLINE_COMMA + mov eax, ebx + xor ebx, ebx + mov bx, [eax] + ret + defword_end + + defword "2!", 0, WORD_STORE2, WORD_INLINE_COMMA + PICKDSP ecx, 4 + PICKDSP edx, 0 + mov [ebx + 4], ecx + mov [ebx], edx + PICKDSP ebx, 8 + ADDDSP 12 + ret + defword_end + + defword "2@", 0, WORD_FETCH2, WORD_INLINE_COMMA + ADDDSP -4 + mov ecx, [ebx +4] + mov ebx, [ebx] + PUTDSP ecx, 0 + ret + defword_end + + defword "blank", 0, WORD_BLANK, WORD_CALL_COMMA + mov ecx, ebx + PICKDSP ebx, 4 + PICKDSP edi, 0 + ADDDSP 8 + mov eax, 0x20 + rep stosb + ret + defword_end + + defword "erase", 0, WORD_ERASE, WORD_CALL_COMMA + mov ecx, ebx + PICKDSP ebx, 4 + PICKDSP edi, 0 + ADDDSP 8 + xor eax, eax + rep stosb + ret + defword_end + + defword "fill", 0, WORD_FILL, WORD_CALL_COMMA + mov eax, ebx + PICKDSP ebx, 8 + PICKDSP edi, 4 + PICKDSP ecx, 0 + ADDDSP 12 + rep stosb + ret + defword_end + + defword "cmove>", 0, WORD_CMOVEB, WORD_CALL_COMMA + mov ecx, ebx + PICKDSP ebx, 8 + PICKDSP esi, 4 + PICKDSP edi, 0 + ADDDSP 12 + lea esi, [esi + ecx - 1] + lea edi, [edi + ecx - 1] + std + rep movsb + cld + ret + defword_end + + defword "cmove", 0, WORD_CMOVE, WORD_CALL_COMMA + mov ecx, ebx + PICKDSP ebx, 8 + PICKDSP esi, 4 + PICKDSP edi, 0 + ADDDSP 12 + rep movsb + ret + defword_end + + defword "move", 0, WORD_MOVE, WORD_CALL_COMMA + mov ecx, ebx + PICKDSP ebx, 8 + PICKDSP esi, 4 + PICKDSP edi, 0 + ADDDSP 12 + cmp esi, edi + if a + rep movsb + else + lea esi, [esi + ecx -1] + lea edi, [edi + ecx -1] + std + rep movsb + cld + endif + ret + defword_end + +;;;;;;;;;;;;;;;;;;;;;;;;;;;; +; single precision alu words +;;;;;;;;;;;;;;;;;;;;;;;;;;;; + + defword "+", 0, WORD_ADD, WORD_INLINE_COMMA + add ebx, [ebp] + ADDDSP 4 + ret + defword_end + + defword "-", 0, WORD_SUB, WORD_INLINE_COMMA + mov eax, ebx + POPDSP ebx + sub ebx, eax + ret + defword_end + + defword "*", 0, WORD_MULL, WORD_INLINE_COMMA + imul ebx, [ebp] + ADDDSP 4 + ret + defword_end + + defword "/", 0, WORD_DIV, WORD_INLINE_COMMA + POPDSP eax + cdq + idiv ebx + mov ebx, eax + ret + defword_end + + defword "mod", 0, WORD_MOD, WORD_INLINE_COMMA + POPDSP eax + cdq + idiv ebx + mov ebx, edx + ret + defword_end + + defword "1+", 0, WORD_INCR, WORD_INLINE_COMMA + add ebx, byte 1 + ret + defword_end + + defword "1-", 0, WORD_DECR, WORD_INLINE_COMMA + sub ebx, byte 1 + ret + defword_end + + defword "4+", 0, WORD_INCR4, WORD_INLINE_COMMA + add ebx, byte 4 + ret + defword_end + + defword "4-", 0, WORD_DECR4, WORD_INLINE_COMMA + sub ebx, byte 4 + ret + defword_end + + defword "2+", 0, WORD_INCR2, WORD_INLINE_COMMA + add ebx, byte 2 + ret + defword_end + + defword "2-", 0, WORD_DECR2, WORD_INLINE_COMMA + sub ebx, byte 2 + ret + defword_end + + defword "2*", 0, WORD_TWOMUL, WORD_INLINE_COMMA + shl ebx, byte 1 + ret + defword_end + + defword "2/", 0, WORD_TWODIV, WORD_INLINE_COMMA + sar ebx, byte 1 + ret + defword_end + + defword "abs", 0, WORD_ABS, WORD_INLINE_COMMA + mov eax, ebx + sar eax, byte 31 + add ebx, eax + xor ebx, eax + ret + defword_end + + defword "min", 0, WORD_MIN, WORD_INLINE_COMMA + POPDSP eax + cmp ebx, eax + if g + mov ebx, eax + endif + ret + defword_end + + defword "max", 0, WORD_MAX, WORD_INLINE_COMMA + POPDSP eax + cmp ebx, eax + if l + mov ebx, eax + endif + ret + defword_end + + defword "lshift", 0, WORD_LSHIFT, WORD_INLINE_COMMA + mov ecx, ebx + POPDSP ebx + shl ebx, cl + ret + defword_end + + defword "rshift", 0, WORD_RSHIFT, WORD_INLINE_COMMA + mov ecx, ebx + POPDSP ebx + shr ebx, cl + ret + defword_end + + defword "and", 0, WORD_AND, WORD_INLINE_COMMA + and ebx, [ebp] + ADDDSP 4 + ret + defword_end + + defword "or", 0, WORD_OR, WORD_INLINE_COMMA + or ebx, [ebp] + ADDDSP 4 + ret + defword_end + + defword "xor", 0, WORD_XOR, WORD_INLINE_COMMA + xor ebx, [ebp] + ADDDSP 4 + ret + defword_end + + defword "negate", 0, WORD_NEGATE, WORD_INLINE_COMMA + neg ebx + ret + defword_end + + defword "invert", 0, WORD_INVERT, WORD_INLINE_COMMA + not ebx + ret + defword_end + +;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; +; single precision comparision words +;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; + + defword "=", 0, WORD_EQ, WORD_INLINE_COMMA + cmp [ebp], ebx + sete bl + ADDDSP 4 + movzx ebx, bl + neg ebx + ret + defword_end + + defword "<>", 0, WORD_NE, WORD_INLINE_COMMA + cmp [ebp], ebx + setne bl + ADDDSP 4 + movzx ebx, bl + neg ebx + ret + defword_end + + defword "<", 0, WORD_LT, WORD_INLINE_COMMA + cmp [ebp], ebx + setl bl + ADDDSP 4 + movzx ebx, bl + neg ebx + ret + defword_end + + defword ">", 0, WORD_GT, WORD_INLINE_COMMA + cmp [ebp], ebx + setg bl + ADDDSP 4 + movzx ebx, bl + neg ebx + ret + defword_end + + defword "u<", 0, WORD_ULT, WORD_INLINE_COMMA + cmp [ebp], ebx + setb bl + ADDDSP 4 + movzx ebx, bl + neg ebx + ret + defword_end + + defword "u>", 0, WORD_UGT, WORD_INLINE_COMMA + cmp [ebp], ebx + seta bl + ADDDSP 4 + movzx ebx, bl + neg ebx + ret + defword_end + + defword "u<=", 0, WORD_ULTEQ, WORD_INLINE_COMMA + cmp [ebp], ebx + setbe bl + ADDDSP 4 + movzx ebx, bl + neg ebx + ret + defword_end + + defword "u>=", 0, WORD_UGTEQ, WORD_INLINE_COMMA + cmp [ebp], ebx + setae bl + ADDDSP 4 + movzx ebx, bl + neg ebx + ret + defword_end + + defword "<=", 0, WORD_LTEQ, WORD_INLINE_COMMA + cmp [ebp], ebx + setle bl + ADDDSP 4 + movzx ebx, bl + neg ebx + ret + defword_end + + defword ">=", 0, WORD_GTEQ, WORD_INLINE_COMMA + cmp [ebp], ebx + setge bl + ADDDSP 4 + movzx ebx, bl + neg ebx + ret + defword_end + + defword "0=", 0, WORD_ZEQ, WORD_INLINE_COMMA + test ebx, ebx + setz bl + movzx ebx, bl + neg ebx + ret + defword_end + + defword "0<>", 0, WORD_ZNE, WORD_INLINE_COMMA + test ebx, ebx + setnz bl + movzx ebx, bl + neg ebx + ret + defword_end + + defword "0<", 0, WORD_ZLT, WORD_INLINE_COMMA + test ebx, ebx + setl bl + movzx ebx, bl + neg ebx + ret + defword_end + + defword "0>", 0, WORD_ZGT, WORD_INLINE_COMMA + test ebx, ebx + setg bl + movzx ebx, bl + neg ebx + ret + defword_end + + defword "0<=", 0, WORD_ZLTEQ, WORD_INLINE_COMMA + test ebx, ebx + setle bl + movzx ebx, bl + neg ebx + ret + defword_end + + defword "0>=", 0, WORD_ZGTEQ, WORD_INLINE_COMMA + test ebx, ebx + setge bl + movzx ebx, bl + neg ebx + ret + defword_end + +;;;;;;;;;;;;;;;;;;;;;;;;;;;; +; double precision ALU words +;;;;;;;;;;;;;;;;;;;;;;;;;;;; + + defword "s>d", 0, WORD_STOD, WORD_INLINE_COMMA + mov eax, ebx + cdq + PUSHDSP eax + mov ebx, edx + ret + defword_end + + defword "d>s", 0, WORD_DTOS, WORD_INLINE_COMMA + POPDSP ebx + ret + defword_end + + defword "d+", 0, WORD_DPLUS, WORD_INLINE_COMMA + PICKDSP ecx, 8 + PICKDSP edx, 4 + PICKDSP eax, 0 + add eax, ecx + adc ebx, edx + PUTDSP eax, 8 + ADDDSP 8 + ret + defword_end + + defword "d-", 0, WORD_DMINUS, WORD_INLINE_COMMA + PICKDSP ecx, 8 + PICKDSP edx, 4 + PICKDSP eax, 0 + sub ecx, eax + sbb edx, ebx + PUTDSP ecx, 8 + mov ebx, edx + ADDDSP 8 + ret + defword_end + + defword "d2*", 0, WORD_D2STAR, WORD_INLINE_COMMA + PICKDSP eax, 0 + shl eax, 1 + rcl ebx, 1 + PUTDSP eax, 0 + ret + defword_end + + defword "d2/", 0, WORD_D2SLASH, WORD_INLINE_COMMA + PICKDSP eax, 0 + sar ebx, 1 + rcr eax, 1 + PUTDSP eax, 0 + ret + defword_end + + defword "*/", 0, WORD_MULDIV, WORD_INLINE_COMMA + PICKDSP edx, 4 + PICKDSP eax, 0 + imul edx + idiv ebx + mov ebx, eax + ADDDSP 8 + ret + defword_end + + defword "*/mod", 0, WORD_STARSMOD, WORD_INLINE_COMMA + PICKDSP edx, 4 + PICKDSP eax, 0 + imul edx + idiv ebx + PUTDSP edx, 4 + ADDDSP 4 + mov ebx, eax + ret + defword_end + + defword "/mod", 0, WORD_DIVMOD, WORD_INLINE_COMMA + PICKDSP eax, 0 + cdq + idiv ebx + PUTDSP edx, 0 + mov ebx, eax + ret + defword_end + + defword "dnegate", 0, WORD_DNEGATE, WORD_INLINE_COMMA + PICKDSP eax, 0 + not eax + not ebx + add eax, 1 + adc ebx, 0 + PUTDSP eax, 0 + ret + defword_end + + defword "dabs", 0, WORD_DABS, WORD_INLINE_COMMA + test ebx, ebx + if l + PICKDSP eax, 0 + not eax + not ebx + add eax, 1 + adc ebx, 0 + PUTDSP eax, 0 + endif + ret + defword_end + + defword "dmax", 0, WORD_DMAX, WORD_INLINE_COMMA + PICKDSP ecx, 8 + PICKDSP edx, 4 + PICKDSP eax, 0 + ADDDSP 8 + mov esi, ecx + mov edi, edx + sub esi, eax + sbb edi, ebx + if l + PUTDSP eax, 0 + else + mov ebx, edx + endif + ret + defword_end + + defword "dmin", 0, WORD_DMIN, WORD_INLINE_COMMA + PICKDSP ecx, 8 + PICKDSP edx, 4 + PICKDSP eax, 0 + ADDDSP 8 + mov esi, ecx + mov edi, edx + sub esi, eax + sbb edi, ebx + if ge + PUTDSP eax, 0 + else + mov ebx, edx + endif + ret + defword_end + +;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; +; double precision comparision words +;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; + + defword "d0=", 0, WORD_DZEQ, WORD_INLINE_COMMA + or ebx, [ebp] + setz bl + ADDDSP 4 + movzx ebx, bl + neg ebx + ret + defword_end + + defword "d0<>", 0, WORD_DZNEQ, WORD_INLINE_COMMA + or ebx, [ebp] + setnz bl + ADDDSP 4 + movzx ebx, bl + neg ebx + ret + defword_end + + defword "d0<", 0, WORD_DZLT, WORD_INLINE_COMMA + test ebx, ebx + setl bl + ADDDSP 4 + movzx ebx, bl + neg ebx + ret + defword_end + + defword "d=", 0, WORD_DEQ, WORD_INLINE_COMMA + PICKDSP ecx, 8 + PICKDSP eax, 4 + sub ecx, [ebp] + sbb eax, ebx + setz bl + ADDDSP 12 + movzx ebx, bl + neg ebx + ret + defword_end + + defword "d<>", 0, WORD_DNEQ, WORD_INLINE_COMMA + PICKDSP ecx, 8 + PICKDSP eax, 4 + sub ecx, [ebp] + sbb eax, ebx + setnz bl + ADDDSP 12 + movzx ebx, bl + neg ebx + ret + defword_end + + defword "d<", 0, WORD_DLT, WORD_INLINE_COMMA + PICKDSP ecx, 8 + PICKDSP eax, 4 + sub ecx, [ebp] + sbb eax, ebx + setl bl + ADDDSP 12 + movzx ebx, bl + neg ebx + ret + defword_end + + defword "du<", 0, WORD_DULT, WORD_INLINE_COMMA + PICKDSP ecx, 8 + PICKDSP eax, 4 + sub ecx, [ebp] + sbb eax, ebx + setb bl + ADDDSP 12 + movzx ebx, bl + neg ebx + ret + defword_end + +;;;;;;;;;;;;;;;;;;;;;;; +; mixed precision words +;;;;;;;;;;;;;;;;;;;;;;; + + defword "m+", 0, WORD_MPLUS, WORD_INLINE_COMMA + PICKDSP eax, 4 + PICKDSP edx, 0 + add eax, ebx + adc edx, 0 + PUTDSP eax, 4 + mov ebx, edx + ADDDSP 4 + ret + defword_end + + defword "m-", 0, WORD_MMINUS, WORD_INLINE_COMMA + PICKDSP eax, 4 + PICKDSP edx, 0 + sub eax, ebx + sbb edx, 0 + PUTDSP eax, 4 + mov ebx, edx + ADDDSP 4 + ret + defword_end + + defword "m*", 0, WORD_MULSTAR, WORD_INLINE_COMMA + PICKDSP eax, 0 + imul ebx + PUTDSP eax, 0 + mov ebx, edx + ret + defword_end + + defword "m/", 0, WORD_MSLASH, WORD_INLINE_COMMA + PICKDSP eax, 4 + PICKDSP edx, 0 + idiv ebx + mov ebx, eax + ADDDSP 8 + ret + defword_end + + defword "um*", 0, WORD_UMULSTAR, WORD_INLINE_COMMA + PICKDSP eax, 0 + mul ebx + PUTDSP eax, 0 + mov ebx, edx + ret + defword_end + + defword "um/mod", 0, WORD_UMDIVMOD, WORD_INLINE_COMMA + PICKDSP eax, 4 + PICKDSP edx, 0 + div ebx + PUTDSP edx, 4 + mov ebx, eax + ADDDSP 4 + ret + defword_end + + defword "fm/mod", 0, WORD_FMDIVMOD, WORD_INLINE_COMMA + PICKDSP edx, 0 + PICKDSP eax, 4 + mov ecx, ebx + ADDDSP 4 + xor ecx, edx + idiv ebx + test ecx, ecx + if s + test edx, edx + if nz + dec eax + add edx, ebx + endif + endif + PUTDSP edx, 0 + mov ebx, eax + ret + defword_end + + defword "sm/rem", 0, WORD_SMDIVREM, WORD_INLINE_COMMA + PICKDSP eax, 4 + PICKDSP edx, 0 + idiv ebx + PUTDSP edx, 4 + mov ebx, eax + ADDDSP 4 + ret + defword_end + + defword "u/mod", 0, WORD_UDIVMOD, WORD_INLINE_COMMA + xor edx, edx + PICKDSP eax, 0 + div ebx + PUTDSP edx, 0 + mov ebx, eax + ret + defword_end + + defword "dm*", 0, WORD_DMULSTAR, WORD_CALL_COMMA + call WORD_TUCK + call WORD_MULL + TORSP + call WORD_UMULSTAR + FROMRSP + call WORD_ADD + ret + defword_end + +;;;;;;;;;;;;;;;;;;;; +; control flow words +;;;;;;;;;;;;;;;;;;;; + + defword "branch", 0, WORD_BRANCH, WORD_INLINE_COMMA +i_jmp: + jmp strict near i_ret + ret + defword_end + + defword "0branch", 0, WORD_ZBRANCH, WORD_INLINE_COMMA + mov eax, ebx + POPDSP ebx + test eax, eax + jz strict near i_jmp + ret + defword_end + + defword "exit", 0, WORD_EXIT, WORD_EXIT_COMMA +i_ret: + ret + defword_end + + defword "exit,", 0, WORD_EXIT_COMMA, WORD_CALL_COMMA + mov edi, [var_WORD_DP] + sub edi, 5 + cmp edi, [lastcall] ; are we just after a call instruction ? + if z + mov al, [i_jmp] + mov [edi], al ; change it to a jmp + endif + mov edi, [var_WORD_DP] + mov al, [i_ret] + stosb + mov [var_WORD_DP], edi + POPDSP ebx + ret + defword_end + + defword "execute", 0, WORD_EXECUTE, WORD_CALL_COMMA + mov eax, ebx ; Get xt into eax + POPDSP ebx ; After xt runs its ret will continue executing the current word. + jmp eax ; and jump to it. + defword_end + +;;;;;;;;;;;;;;;;;;;;;;;;;; +; terminal input words +;;;;;;;;;;;;;;;;;;;;;;;;;; + + defword "read-char", 0, WORD_READCHAR, WORD_CALL_COMMA + mov ecx, var_WORD_CHARBUF ; 2nd param: buffer + mov edx, 1 ; 3rd param: max length + push edx + push ecx + push ebx + mov eax, SYS_read ; syscall: read + call _syscall + add esp, 12 + xor ebx, ebx + test eax, eax + if be + mov ebx, -1 + endif + ret + defword_end + + defword "read-line", 0, WORD_READLINE, WORD_CALL_COMMA + call WORD_NROT + call WORD_OVER + call WORD_ADD + call WORD_OVER ; ( fd start end cur ) +readline_l1: + PICKDSP eax, 0 + cmp ebx, eax + jz readline_l4 + PUSHDSP ebx + PICKDSP ebx, 12 + call WORD_READCHAR + test ebx, ebx + jz readline_l2 + call WORD_DROP + call WORD_DROP2 + call WORD_DROP2 + LOADTOS 0 + LOADTOS 0 + LOADTOS -1 + jmp readline_l5 +readline_l2: + mov ebx, [var_WORD_CHARBUF] + cmp ebx, 10 ; LF + jz readline_l3 + call WORD_OVER + call WORD_STOREBYTE + call WORD_INCR + jmp readline_l1 +readline_l3: + call WORD_DROP +readline_l4: + call WORD_NIP + call WORD_SWAP + call WORD_SUB + call WORD_NIP + LOADTOS -1 + LOADTOS 0 +readline_l5: + ret + defword_end + + defword "key", 0, WORD_KEY, WORD_CALL_COMMA + PUSHDSP ebx + xor ebx, ebx ; stdin + call WORD_READCHAR + mov ebx, [var_WORD_CHARBUF] + ret + defword_end + + defword "accept", 0, WORD_ACCEPT, WORD_CALL_COMMA + call WORD_OVER + call WORD_ADD + call WORD_OVER ; ( start end cur ) +accept_l1: + call WORD_KEY + cmp ebx, 127 ; BS + jz accept_l2 + cmp ebx, 10 ; LF + jz accept_l3 + call WORD_OVER ; ( start end cur key cur ) + call WORD_STOREBYTE + call WORD_INCR ; ( start end cur' ) + PICKDSP eax, 0 + cmp ebx, eax + jz accept_l4 + jmp accept_l1 +accept_l2: + PICKDSP eax, 4 ; ( start end cur' ) + cmp ebx, eax + jz accept_l1 + call WORD_DECR + jmp accept_l1 +accept_l3: + call WORD_DROP ; ( start end cur' ) +accept_l4: + call WORD_NIP + call WORD_SWAP + call WORD_SUB + ret + defword_end + + defword "tabs>spaces", 0, WORD_TABSTOSPACES, WORD_CALL_COMMA + mov ecx, ebx + POPDSP esi + test ecx, ecx + if nz + repeat + lodsb + cmp al, 9 ;TAB + if z + mov byte [esi - 1], ' ' + endif + dec ecx + until z + endif + POPDSP ebx + ret + defword_end + +;;;;;;;;;;;;;;;;;;;;;;; +; terminal output words +;;;;;;;;;;;;;;;;;;;;;;; + + defword "type-fd", 0, WORD_TYPE_FD, WORD_CALL_COMMA + PICKDSP edx, 0 ; 3rd param: length of string + PICKDSP ecx, 4 ; 2nd param: address of string + ADDDSP 8 ; 1st param: FD in ebx + mov eax, SYS_write ; write syscall + push edx + push ecx + push ebx + call _syscall + add esp, 12 + POPDSP ebx + ret + defword_end + + defword "type", 0, WORD_TYPE, WORD_CALL_COMMA + LOADTOS 1 ; stdout + call WORD_TYPE_FD + ret + defword_end + + defword "emit", 0, WORD_EMIT, WORD_CALL_COMMA + mov [emit_scratch], bl ; write needs the address of the byte to write + mov ebx, emit_scratch + LOADTOS 1 + call WORD_TYPE + ret + defword_end + +;;;;;;;;;;;;;;;;;;; +; system call words +;;;;;;;;;;;;;;;;;;; + + defword "syscall", 0, WORD_SYSCALL, WORD_CALL_COMMA + pop eax + mov [syscallret], eax ; save return address + mov eax, ebx ; System call number (see ) + call _syscall + mov ebx, eax ; Result (negative for -errno) + jmp [syscallret] ; return to caller + defword_end + + defword "lsyscall", 0, WORD_LSYSCALL, WORD_CALL_COMMA + pop eax + mov [syscallret], eax ; save return address + mov eax, ebx ; System call number (see ) + call _lsyscall + PUSHDSP eax + mov ebx, edx ; Result (negative for -errno) + jmp [syscallret] ; return to caller + defword_end + +;;;;;;;;;;;;;; +; string words +;;;;;;;;;;;;;; + + defword "count", 0, WORD_COUNT, WORD_CALL_COMMA + xor eax, eax + mov al, [ebx] + inc ebx + LOADTOS eax + ret + defword_end + + defword "-trailing", 0, WORD_TRAILING, WORD_CALL_COMMA + test ebx, ebx + if nz + PICKDSP esi, 0 + mov ecx, ebx + add esi, ebx + dec esi + std + trailing_l1: + lodsb + cmp al, ' ' + if be + loop trailing_l1 + endif + mov ebx, ecx + cld + endif + ret + defword_end + + defword "/string", 0, WORD_SSTRING, WORD_CALL_COMMA + mov eax, ebx + POPDSP ebx + PICKDSP ecx, 0 + sub ebx, eax + add ecx, eax + PUTDSP ecx, 0 + ret + defword_end + + defword "compare", 0, WORD_COMPARE, WORD_CALL_COMMA + PICKDSP esi, 8 + PICKDSP edx, 4 + PICKDSP edi, 0 + ADDDSP 12 + mov ecx, ebx + cmp edx, ebx + if be + mov ecx, edx + endif + test ecx, ecx ; ecx lowest length + jnz compare_l2 + cmp edx, ebx + jz compare_l3 ; both are 0 length + jmp compare_l4 ; otherwise the longest wins +compare_l2: + cmpsb + jnz compare_l4 ; chars not same + loop compare_l2 + cmp edx, ebx ; all chars same + jnz compare_l4 ; strings not same size +compare_l3: + xor ebx, ebx ; same + jmp compare_l7 +compare_l4: + ja compare_l6 +compare_l5: + mov ebx, -1 + jmp compare_l7 +compare_l6: + mov ebx, 1 +compare_l7: + ret + defword_end + + defword "icompare", 0, WORD_COMPAREI, WORD_CALL_COMMA + PICKDSP esi, 8 + PICKDSP edx, 4 + PICKDSP edi, 0 + ADDDSP 12 + mov ecx, ebx + cmp edx, ebx + if be + mov ecx, edx + endif + test ecx, ecx ; ecx lowest length + jnz comparei_l2 + cmp edx, ebx + jz comparei_l3 ; both are 0 length + jmp comparei_l4 ; otherwise the longest wins +comparei_l2: + mov al, [esi] + mov ah, [edi] + to_lower al + to_lower ah + cmp ah, al + jnz comparei_l4 ; chars not same + inc edi + inc esi + loop comparei_l2 + cmp edx, ebx ; all chars same + jnz comparei_l4 ; strings not same size +comparei_l3: + xor ebx, ebx ; same + jmp comparei_l7 +comparei_l4: + ja comparei_l6 +comparei_l5: + mov ebx, -1 + jmp comparei_l7 +comparei_l6: + mov ebx, 1 +comparei_l7: + ret + defword_end + +;;;;;;;;;;;;;;;;;;;;;;;;;;;; +; dictionary searching words +;;;;;;;;;;;;;;;;;;;;;;;;;;;; + + defword "find", 0, WORD_FIND, WORD_CALL_COMMA + call WORD_DUP + call WORD_COUNT + call WORD_FIND_DICT + test ebx, ebx + if nz + mov dl, [ebx + H_NSIZE] + call WORD_TCFA + LOADTOS 1 + and edx, F_IMMED + if z + neg ebx + endif + call WORD_ROT + call WORD_DROP + endif + ret + defword_end + + defword "(find)", 0, WORD_FIND_DICT, WORD_CALL_COMMA + mov ecx, ebx ; ecx = length + POPDSP edi ; edi = address + PUSHRSP ecx + mov esi, edi + call strhashi + and ebx, NUM_HASH_CHAINS-1 + mov esi, hash_buckets + mov edx, [esi + (ebx * 4)] + POPRSP ecx ; edx can now scan back through this hash chain +findd_l1: + test edx, edx ; NULL pointer? (end of the linked list) + je findd_l4 + xor eax, eax + mov al, [edx + H_NSIZE] ; al = flags+length field + and al, (F_HIDDEN|F_LENMASK) ; al = name length + cmp al, cl ; Length is the same? + jne findd_l2 + PUSHRSP ecx ; Save the length + PUSHRSP edi ; Save the address (repe cmpsb will move this pointer) + lea esi, [edx + H_NAME] ; Dictionary string we are checking against. + call strcmpi + POPRSP edi + POPRSP ecx + jne findd_l2 ; Not the same. + mov ebx, edx + ret +findd_l2: + mov edx, [edx + H_HLINK] ; Move back through the link field to the previous word + jmp findd_l1 ; .. and loop. +findd_l4: + xor ebx, ebx ; Return zero to indicate not found. + ret + defword_end + + defword ">cfa", 0, WORD_TCFA, WORD_CALL_COMMA + add ebx, H_NSIZE + mov al, [ebx] ; Load flags+len into al. + inc ebx ; skip flags+len byte. + and eax, F_LENMASK ; Just the length, not the flags. + add ebx, eax ; skip the name + add ebx, XT_SIZE ; skip to the xt + ret + defword_end + + defword "(bucket)", 0, WORD_BUCKET, WORD_CALL_COMMA + mov ecx, ebx ; ecx = length + POPDSP ebx ; ebx = address of name + PUSHRSP esi + mov esi, ebx + call strhashi + and ebx, NUM_HASH_CHAINS-1 + mov esi, hash_buckets + lea ebx, [esi + (ebx * 4)] + POPRSP esi + ret + defword_end + + defword "unused", 0, WORD_UNUSED, WORD_CALL_COMMA + LOADTOS forth_end + LOADTOS [var_WORD_DP] + call WORD_SUB + ret + defword_end + +;;;;;;;;;;;;;;;;;;;;;;;;;;; +; dictionary building words +;;;;;;;;;;;;;;;;;;;;;;;;;;; + + defword "align", 0, WORD_ALIGNDP, WORD_CALL_COMMA + mov eax, [var_WORD_DP] + ALIGNREG eax + mov [var_WORD_DP], eax + ret + defword_end + + defword "header,", 0, WORD_HEADER_COMMA, WORD_CALL_COMMA + mov ecx, ebx ; ecx = length + POPDSP ebx ; ebx = address of name + call WORD_ALIGNDP ; align header + mov edi, [var_WORD_DP] ; edi is the address of the header + mov eax, [var_WORD_LATEST] ; Get link pointer + mov [edi + H_LLINK], eax ; and store it in the header. + mov [var_WORD_LATEST], edi + PUSHRSP ebx ; hash chain + PUSHRSP ecx + mov esi, ebx + call strhashi + and ebx, NUM_HASH_CHAINS-1 + mov esi, hash_buckets + mov eax, [esi + (ebx * 4)] + mov [esi + (ebx * 4)], edi + mov [edi + H_HLINK], eax ; and store it in the header. + POPRSP ecx + POPRSP esi + mov [edi + H_NSIZE], cl ; Store the length/flags byte. + add edi, H_NAME + call strcpyi + mov ecx, XT_SIZE + xor eax, eax + rep stosb ; clear the gap till the xt + mov [var_WORD_DP], edi + mov long [edi + XT_COMPILE], WORD_CALL_COMMA ;compile action + POPDSP ebx + ret + defword_end + + defword "lit,", 0, WORD_LIT_COMMA, WORD_CALL_COMMA + mov esi, litc_l1 + mov edi, [var_WORD_DP] + mov ecx, litc_l2 - litc_l1 - 4 + rep movsb + mov [var_WORD_DP], edi + ret + defword_end +litc_l1: + LOADTOS 0xBAADF00D +litc_l2: + + defword "slits", 0, WORD_SLITS, WORD_CALL_COMMA + PUSHDSP ebx + POPRSP esi + xor eax, eax + lodsb ; get the length of the string + PUSHDSP esi ; push the address of the start of the string + mov ebx, eax ; push length on the stack + add esi, eax ; skip past the string + jmp esi + defword_end + + defword "clits", 0, WORD_CLITS, WORD_CALL_COMMA + FROMRSP + xor eax, eax + mov al, [ebx] + lea eax, [ebx + eax + 1] + jmp eax + defword_end + + defword ",", 0, WORD_COMMA, WORD_CALL_COMMA + mov edi, [var_WORD_DP] ; DP + mov eax, ebx + stosd ; Store it. + mov [var_WORD_DP], edi ; Update DP (incremented) + POPDSP ebx + ret + defword_end + + defword "c,", 0, WORD_CHAR_COMMA, WORD_CALL_COMMA + mov eax, ebx + mov edi, [var_WORD_DP] ; DP + stosb ; Store it. + mov [var_WORD_DP], edi ; Update DP (incremented) + POPDSP ebx + ret + defword_end + + defword ":", 0, WORD_COLON, WORD_CALL_COMMA + call WORD_PARSENAME + call WORD_HEADER_COMMA ; Create the dictionary entry / header + mov eax, [var_WORD_DP] + mov [eax + XT_BODY], eax + call WORD_LATEST + call WORD_FETCH + call WORD_HIDDEN ; Make the word hidden + call WORD_RBRAC ; Go into compile mode. + ret + defword_end + + defword "create", 0, WORD_CREATE, WORD_CALL_COMMA + call WORD_PARSENAME + call WORD_HEADER_COMMA + mov esi, create_l1 + mov edi, [var_WORD_DP] + PUSHRSP edi + mov ecx, create_l4 - create_l1 + rep movsb + mov [var_WORD_DP], edi + mov edx, edi + call WORD_ALIGNDP + POPRSP eax + mov edi, [var_WORD_DP] + sub edx, eax + mov [eax + create_l2 - create_l1 - 4], edi + mov [eax + XT_BODY], edi + mov [eax + XT_LENGTH], edx + ret + defword_end +create_l1: + LOADTOS 0xBAADF00D +create_l2: + call strict near create_l3 +create_l3: + ret +create_l4: + + defword "dodoes", 0, WORD_DODOES, WORD_CALL_COMMA + call WORD_LATEST + call WORD_FETCH + call WORD_TCFA + add ebx, create_l3 - create_l1 - 4 + POPDSP eax + sub eax, ebx + sub eax, 4 + mov [ebx], eax + POPDSP ebx + ret + defword_end + + defword "does>", F_IMMED, WORD_DOES, WORD_CALL_COMMA + call WORD_LIT_COMMA + LOADTOS [var_WORD_DP] + add ebx, 10 + call WORD_COMMA + LOADTOS WORD_DODOES + call WORD_COMPILE_COMMA + LOADTOS 0 + mov bl, [does_l1] + call WORD_CHAR_COMMA +does_l1: + ret + defword_end + + defword "postpone", F_IMMED, WORD_POSTPONE, WORD_CALL_COMMA + call WORD_PARSENAME + call WORD_FIND_DICT + mov dl, [ebx + H_NSIZE] + call WORD_TCFA + and edx, F_IMMED + if z + call WORD_LIT_COMMA + call WORD_COMMA + LOADTOS WORD_COMPILE_COMMA + endif + jmp WORD_COMPILE_COMMA + ret + defword_end + + defword "call,", 0, WORD_CALL_COMMA, WORD_CALL_COMMA + mov edi, [var_WORD_DP] + mov [lastcall], edi ; record last location of last call + mov esi, i_call + movsb + mov eax, ebx + sub eax, 4 + sub eax, edi + stosd + mov [var_WORD_DP], edi + POPDSP ebx + ret + defword_end + + defword "inline,", 0, WORD_INLINE_COMMA, WORD_CALL_COMMA + mov ecx, [ebx + XT_LENGTH] + dec ecx ; actual code length minus ret + mov esi, ebx + mov edi, [var_WORD_DP] + rep movsb ; inline copy the code + mov [var_WORD_DP], edi ; update DP + POPDSP ebx + ret + defword_end + + defword "compile,", 0, WORD_COMPILE_COMMA, WORD_INLINE_COMMA + call [ebx + XT_COMPILE] + ret + defword_end + + defword ";", F_IMMED, WORD_SEMICOLON, WORD_CALL_COMMA + LOADTOS WORD_EXIT +i_call: + call strict near WORD_COMPILE_COMMA + call WORD_LATEST + call WORD_FETCH + call WORD_HIDDEN ; toggle hidden flag -- unhide the word (see below for definition). + call WORD_LBRAC ; go back to IMMEDIATE mode. + mov edx, ebx + mov ebx, [var_WORD_LATEST] + call WORD_TCFA + mov ecx, [var_WORD_DP] + sub ecx, ebx + mov [ebx + XT_LENGTH], ecx ; set code size of word + mov ebx, edx + ret + defword_end + + defword "immediate", 0, WORD_IMMEDIATE, WORD_CALL_COMMA + mov edi, [var_WORD_LATEST] ; LATEST word. + add edi, H_NSIZE ; Point to name/flags byte. + xor byte [edi], F_IMMED ; Toggle the IMMED bit. + ret + defword_end + + defword "hidden", 0, WORD_HIDDEN, WORD_CALL_COMMA + add ebx, H_NSIZE ; Point to name/flags byte. + xor byte [ebx], F_HIDDEN ; Toggle the HIDDEN bit. + POPDSP ebx + ret + defword_end + + defword "[", F_IMMED, WORD_LBRAC, WORD_CALL_COMMA + mov long [var_WORD_STATE], 0 ; Set STATE to 0. + ret + defword_end + + defword "]", 0, WORD_RBRAC, WORD_CALL_COMMA + mov long [var_WORD_STATE], 1 ; Set STATE to 1. + ret + defword_end + +;;;;;;;;;;;;;;;;;;;;; +; source buffer words +;;;;;;;;;;;;;;;;;;;;; + + defword "source", 0, WORD_SOURCE, WORD_CALL_COMMA + call WORD_INHASH + call WORD_FETCH2 + ret + defword_end + + defword "refill", 0, WORD_REFILL, WORD_CALL_COMMA + LOADTOS tib_buffer + call WORD_LINESIZE ; ( tib len ) + call WORD_OVER + call WORD_SWAP ; ( tib tib len ) + call WORD_ACCEPT ; read line into TIB + call WORD_DUP2 + call WORD_TABSTOSPACES + call WORD_INHASH + call WORD_STORE2 ; set as current WORD_SOURCE + LOADTOS 0 + call WORD_TOIN + call WORD_STORE ; set to start of buffer + LOADTOS -1 + ret + defword_end + + defword "isspace?", 0, WORD_ISSPACE, WORD_CALL_COMMA + LOADTOS ' ' + call WORD_ULTEQ + ret + defword_end + + defword "isnotspace?", 0, WORD_ISNOTSPACE, WORD_CALL_COMMA + call WORD_ISSPACE + call WORD_ZEQ + ret + defword_end + + defword "xt-skip", 0, WORD_XTSKIP, WORD_CALL_COMMA + TORSP +xtskip_l1: + test ebx, ebx + jz xtskip_l3 + call WORD_OVER + call WORD_FETCHBYTE + FETCHRSP + call WORD_EXECUTE + test ebx, ebx + jz xtskip_l2 + mov ebx, 1 + call WORD_SSTRING + jmp xtskip_l1 +xtskip_l2: + call WORD_DROP +xtskip_l3: + ADDRSP 4 + ret + defword_end + +;;;;;;;;;;;;;;;;;;;;;; +; input parseing words +;;;;;;;;;;;;;;;;;;;;;; + + defword "parse-name", 0, WORD_PARSENAME, WORD_CALL_COMMA + call WORD_SOURCE + call WORD_TOIN + call WORD_FETCH + call WORD_SSTRING + LOADTOS WORD_ISSPACE + call WORD_XTSKIP + call WORD_OVER + TORSP + LOADTOS WORD_ISNOTSPACE + call WORD_XTSKIP + call WORD_DUP2 + LOADTOS 1 + call WORD_MIN + call WORD_ADD + call WORD_SOURCE + call WORD_DROP + call WORD_SUB + call WORD_TOIN + call WORD_STORE + call WORD_DROP + FROMRSP + call WORD_TUCK + call WORD_SUB +; code to print out "P CR" +;LOADTOS 80 +;call WORD_EMIT +;LOADTOS 32 +;call WORD_EMIT +;call WORD_DUP2 +;call WORD_TYPE +;LOADTOS 10 +;call WORD_EMIT + ret + defword_end + + defword "word-name", 0, WORD_WORDNAME, WORD_CALL_COMMA + call WORD_PARSENAME ; ( start len ) + LOADTOS word_buf ; ( string size buf ) + call WORD_DUP2 ; ( string size buf size buf ) + call WORD_STOREBYTE ; ( string size buf ) + call WORD_INCR ; ( string size buf+1 ) + call WORD_SWAP ; ( string buf+1 size ) + call WORD_CMOVE + LOADTOS word_buf ; ( cstring ) +; debug code to print out "N CR" +;LOADTOS 78 +;call WORD_EMIT +;LOADTOS 32 +;call WORD_EMIT +;call WORD_DUP2 +;call WORD_TYPE +;LOADTOS 10 +;call WORD_EMIT + ret + defword_end + + defword "interp-name", 0, WORD_INTERPNAME, WORD_CALL_COMMA + call WORD_PARSENAME ; ( start len ) + LOADTOS intep_name_buf ; ( string size buf ) + call WORD_DUP2 ; ( string size buf size buf ) + call WORD_STOREBYTE ; ( string size buf ) + call WORD_INCR ; ( string size buf+1 ) + call WORD_SWAP ; ( string buf+1 size ) + call WORD_CMOVE + LOADTOS intep_name_buf ;( cstring ) + ret + defword_end + + defword "interpret", 0, WORD_INTERPRET, WORD_CALL_COMMA + loopstart + call WORD_INTERPNAME + mov al, [ebx] + test al, al + breakif z + ; debug code to print out "I CR" + ;LOADTOS 73 + ;call WORD_EMIT + ;LOADTOS 32 + ;call WORD_EMIT + ;call WORD_DUP + ;call WORD_COUNT + ;call WORD_TYPE + ;LOADTOS 10 + ;call WORD_EMIT + call WORD_INTERP + loopend + call WORD_DROP + ret + defword_end + + defword "interp", 0, WORD_INTERP, WORD_CALL_COMMA + call WORD_FIND ; ( cstring 0 | xt 1 | xt | -1 ) + mov eax, ebx + POPDSP ebx + test eax, eax + jz tryasnumber + jle nonimediate +executeword: + mov eax, ebx + POPDSP ebx + jmp eax +nonimediate: + mov eax, [var_WORD_STATE] + test eax, eax ; are we in imedeate mode ? + jz executeword + jmp WORD_COMPILE_COMMA ; compile xt +tryasnumber: + call WORD_COUNT ; ( adr len ) + LOADTOS 0 + LOADTOS 0 + call WORD_SWAP2 ; ( 0d addr len ) + call WORD_TOSNUMBER ; ( d addr len ) + test ebx, ebx + jnz parseproblem + call WORD_DROP2 + call WORD_DROP ; ( num ) + mov eax, [var_WORD_STATE] + test eax, eax + if nz + call WORD_LIT_COMMA ; compile LIT + call WORD_COMMA ; compile value + endif + ret +parseproblem: + LOADTOS errmsg + LOADTOS errmsgend - errmsg + LOADTOS 2 + call WORD_TYPE_FD + LOADTOS errmsgnl + LOADTOS 1 + LOADTOS 2 + call WORD_TYPE_FD + LOADTOS tib_buffer + LOADTOS [var_WORD_TOIN] + LOADTOS 2 + call WORD_TYPE_FD + LOADTOS errmsgnl + LOADTOS 1 + LOADTOS 2 + call WORD_TYPE_FD + call WORD_DROP2 + call WORD_DROP2 + ret + defword_end + + defword ">number", 0, WORD_TONUMBER, WORD_CALL_COMMA + call WORD_OVER + call WORD_ADD + call WORD_SWAP ; ( ud end cur ) +tonumber_l1: + PICKDSP eax, 0 + cmp ebx, eax + jz near tonumber_l4 + call WORD_DUP + call WORD_FETCHBYTE ; ( ud end cur char ) + to_lower ebx + sub ebx, byte '0' + jb tonumber_l3 ; < '0'? + cmp ebx, byte 10 + jb tonumber_l2 ; <= '9' ? + sub ebx, byte 'a' - '0' + jb tonumber_l3 ; < 'a' ? + add ebx, byte 10 +tonumber_l2: + cmp ebx, [var_WORD_BASE] + jge tonumber_l3 ; >= WORD_BASE ? + TORSP + call WORD_SWAP2 ; ( end cur ud ) + LOADTOS [var_WORD_BASE] + call WORD_DMULSTAR + FROMRSP + call WORD_MPLUS ; ( end cur ud' ) + call WORD_SWAP2 + call WORD_INCR ; ( ud' end cur' ) + jmp tonumber_l1 +tonumber_l3: + call WORD_DROP +tonumber_l4: + call WORD_SWAP + call WORD_OVER + call WORD_SUB ; ( ud' c-addr u2 ) + ret + defword_end + + defword ">snumber", 0, WORD_TOSNUMBER, WORD_CALL_COMMA + test ebx, ebx + if nz + PICKDSP eax, 0 + mov cl, [eax] + cmp cl, '-' + jnz WORD_TONUMBER ; not '-' + inc eax + PUTDSP eax, 0 + dec ebx + call WORD_TONUMBER + call WORD_SWAP2 + call WORD_DNEGATE + call WORD_SWAP2 + endif + ret + defword_end + +;;;;;;;;;;; +; tick word +;;;;;;;;;;; + + defword "ticks", 0, WORD_TICKS, WORD_CALL_COMMA + sub ebp, byte 8 + rdtsc + mov [byte ebp -4], ebx + mov [ebp], eax + mov ebx, edx + ret + defword_end + +;;;;;;;;;;; +; test word +;;;;;;;;;;; + + defword "test", 0, WORD_TEST, WORD_CALL_COMMA + ret + defword_end + +;;;;;;;;;;;;;;;;; +; read/write data +;;;;;;;;;;;;;;;;; + + align 4 +syscallret: + ; return address saved by syscall + dd 0 +lastcall: + ; last call layed down by compiler + dd 0 + +tib_buffer: + ; keyboard input buffer + times MAX_LINE_SIZE db 0 +word_buf: + ; static buffer where WORD returns. Subsequent calls + ; overwrite this buffer. + times MAX_LINE_SIZE db 0 +intep_name_buf: + ; static buffer where INTERPNAME returns. Subsequent calls + ; overwrite this buffer. + times MAX_LINE_SIZE db 0 +emit_scratch: + ; scratch used by EMIT + db 0 +errmsg: + db "PARSE ERROR:" +errmsgend: +errmsgnl: + db 10 +bootfile: + db "forth.f" + db 0 + +;;;;;;;;;;;;;;;;;;;;;;;;;;;; +; dictionary hash table (64) +;;;;;;;;;;;;;;;;;;;;;;;;;;;; + + align 4 +hash_buckets: + dd 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 + dd 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 + dd 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 + dd 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 + +;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; +; addresses of all built in dictionary words. +; this ends up as part of the user space after booting ! +;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; + + align 4 +dictionary_start: + dd dic_WORD_ABS + dd dic_WORD_ACCEPT + dd dic_WORD_ADD + dd dic_WORD_ADDBYTE + dd dic_WORD_ADDSTORE + dd dic_WORD_ALIGNDP + dd dic_WORD_AND + dd dic_WORD_BASE + dd dic_WORD_BLANK + dd dic_WORD_BLK + dd dic_WORD_BRANCH + dd dic_WORD_BUCKET + dd dic_WORD_CALL_COMMA + dd dic_WORD_CHARBUF + dd dic_WORD_CHAR_COMMA + dd dic_WORD_CLITS + dd dic_WORD_CMOVE + dd dic_WORD_CMOVEB + dd dic_WORD_COLON + dd dic_WORD_COMMA + dd dic_WORD_COMPARE + dd dic_WORD_COMPAREI + dd dic_WORD_COMPILE_COMMA + dd dic_WORD_COUNT + dd dic_WORD_CREATE + dd dic_WORD_D2SLASH + dd dic_WORD_D2STAR + dd dic_WORD_DABS + dd dic_WORD_DECR + dd dic_WORD_DECR2 + dd dic_WORD_DECR4 + dd dic_WORD_DEQ + dd dic_WORD_DIV + dd dic_WORD_DIVMOD + dd dic_WORD_DLT + dd dic_WORD_DMAX + dd dic_WORD_DMIN + dd dic_WORD_DMINUS + dd dic_WORD_DMULSTAR + dd dic_WORD_DNEGATE + dd dic_WORD_DNEQ + dd dic_WORD_DODOES + dd dic_WORD_DOES + dd dic_WORD_DP + dd dic_WORD_DPLUS + dd dic_WORD_DROP + dd dic_WORD_DROP2 + dd dic_WORD_DSPFETCH + dd dic_WORD_DSPSTORE + dd dic_WORD_DTOS + dd dic_WORD_DULT + dd dic_WORD_DUP + dd dic_WORD_DUP2 + dd dic_WORD_DZEQ + dd dic_WORD_DZLT + dd dic_WORD_DZNEQ + dd dic_WORD_EMIT + dd dic_WORD_EQ + dd dic_WORD_ERASE + dd dic_WORD_EXECUTE + dd dic_WORD_EXIT + dd dic_WORD_FETCH + dd dic_WORD_FETCH2 + dd dic_WORD_FETCHBYTE + dd dic_WORD_FETCHSHORT + dd dic_WORD_FILL + dd dic_WORD_FIND + dd dic_WORD_FIND_DICT + dd dic_WORD_FMDIVMOD + dd dic_WORD_FROMR + dd dic_WORD_FROMR2 + dd dic_WORD_GT + dd dic_WORD_GTEQ + dd dic_WORD_HEADER_COMMA + dd dic_WORD_HIDDEN + dd dic_WORD_IMMEDIATE + dd dic_WORD_INCR + dd dic_WORD_INCR2 + dd dic_WORD_INCR4 + dd dic_WORD_INHASH + dd dic_WORD_INLINE_COMMA + dd dic_WORD_INTERP + dd dic_WORD_INTERPNAME + dd dic_WORD_INTERPRET + dd dic_WORD_INVERT + dd dic_WORD_ISNOTSPACE + dd dic_WORD_ISSPACE + dd dic_WORD_KEY + dd dic_WORD_LATEST + dd dic_WORD_LBRAC + dd dic_WORD_LINESIZE + dd dic_WORD_LIT_COMMA + dd dic_WORD_LSHIFT + dd dic_WORD_LSYSCALL + dd dic_WORD_LT + dd dic_WORD_LTEQ + dd dic_WORD_MAX + dd dic_WORD_MIN + dd dic_WORD_MMINUS + dd dic_WORD_MOD + dd dic_WORD_MOVE + dd dic_WORD_MPLUS + dd dic_WORD_MSLASH + dd dic_WORD_MULDIV + dd dic_WORD_MULL + dd dic_WORD_MULSTAR + dd dic_WORD_NE + dd dic_WORD_NEGATE + dd dic_WORD_NFROMR + dd dic_WORD_NIP + dd dic_WORD_NIP2 + dd dic_WORD_NQDUP + dd dic_WORD_NROT + dd dic_WORD_NTOR + dd dic_WORD_OR + dd dic_WORD_OVER + dd dic_WORD_OVER2 + dd dic_WORD_O_APPEND + dd dic_WORD_O_CREAT + dd dic_WORD_O_EXCL + dd dic_WORD_O_NONBLOCK + dd dic_WORD_O_RDONLY + dd dic_WORD_O_RDWR + dd dic_WORD_O_TRUNC + dd dic_WORD_O_WRONLY + dd dic_WORD_PARSENAME + dd dic_WORD_PICK + dd dic_WORD_POSTPONE + dd dic_WORD_QDUP + dd dic_WORD_RBRAC + dd dic_WORD_RDROP + dd dic_WORD_RDROP2 + dd dic_WORD_READCHAR + dd dic_WORD_READLINE + dd dic_WORD_REFILL + dd dic_WORD_RFETCH + dd dic_WORD_RFETCH2 + dd dic_WORD_ROT + dd dic_WORD_ROT2 + dd dic_WORD_RSHIFT + dd dic_WORD_RSPFETCH + dd dic_WORD_RSPSTORE + dd dic_WORD_RSTORE + dd dic_WORD_RZ + dd dic_WORD_SEMICOLON + dd dic_WORD_SLITS + dd dic_WORD_SMDIVREM + dd dic_WORD_SOURCE + dd dic_WORD_SOURCEFD + dd dic_WORD_SSTRING + dd dic_WORD_STARSMOD + dd dic_WORD_STATE + dd dic_WORD_STOD + dd dic_WORD_STORE + dd dic_WORD_STORE2 + dd dic_WORD_STOREBYTE + dd dic_WORD_STORESHORT + dd dic_WORD_SUB + dd dic_WORD_SUBSTORE + dd dic_WORD_SWAP + dd dic_WORD_SWAP2 + dd dic_WORD_SYSCALL + dd dic_WORD_SYS_CLOSE + dd dic_WORD_SYS_EXIT + dd dic_WORD_SYS_FSTAT + dd dic_WORD_SYS_FSYNC + dd dic_WORD_SYS_FTRUNCATE + dd dic_WORD_SYS_LSEEK + dd dic_WORD_SYS_OPEN + dd dic_WORD_SYS_READ + dd dic_WORD_SYS_RENAME + dd dic_WORD_SYS_STAT + dd dic_WORD_SYS_UNLINK + dd dic_WORD_SYS_WRITE + dd dic_WORD_SZ + dd dic_WORD_TABSTOSPACES + dd dic_WORD_TCFA + dd dic_WORD_TICKS + dd dic_WORD_TOIN + dd dic_WORD_TONUMBER + dd dic_WORD_TOR + dd dic_WORD_TOR2 + dd dic_WORD_TOSNUMBER + dd dic_WORD_TRAILING + dd dic_WORD_TUCK + dd dic_WORD_TUCK2 + dd dic_WORD_TWODIV + dd dic_WORD_TWOMUL + dd dic_WORD_TYPE + dd dic_WORD_TYPE_FD + dd dic_WORD_UDIVMOD + dd dic_WORD_UGT + dd dic_WORD_UGTEQ + dd dic_WORD_ULT + dd dic_WORD_ULTEQ + dd dic_WORD_UMDIVMOD + dd dic_WORD_UMULSTAR + dd dic_WORD_UNUSED + dd dic_WORD_VERSION + dd dic_WORD_WORDBUF + dd dic_WORD_WORDNAME + dd dic_WORD_XOR + dd dic_WORD_XTSKIP + dd dic_WORD_ZBRANCH + dd dic_WORD_ZEQ + dd dic_WORD_ZGT + dd dic_WORD_ZGTEQ + dd dic_WORD_ZLT + dd dic_WORD_ZLTEQ + dd dic_WORD_ZNE + dd dic_WORD__F_HIDDEN + dd dic_WORD__F_IMMED + dd dic_WORD__F_LENMASK + dd dic_WORD__H_NAME + dd dic_WORD__H_NSIZE + dd dic_WORD__XT_BODY + dd dic_WORD__XT_COMPILE + dd dic_WORD__XT_LENGTH + dd dic_WORD__XT_SIZE + dd dic_WORD_TEST +dictionary_end: + +;;;;;;;;;;;;;;;;;;;;;;;;;; +; room for user dictionary +;;;;;;;;;;;;;;;;;;;;;;;;;; + + times USER_DEFS_SIZE db 0 +forth_end: From 066052ddd20e00ac7133761173898e5155f83255 Mon Sep 17 00:00:00 2001 From: Adam Roben Date: Thu, 12 Feb 2015 10:04:44 -0500 Subject: [PATCH 46/55] Exclude documentation files from language statistics Documentation is an important part of a software project but is not generally thought of as part of the code for that project. Repository language statistics are used to quantify the project's code, so it makes sense to exclude documentation from those computations. Documentation files are recognized similarly to vendored files. lib/linguist/documentation.yml contains regular expressions to match common names for documentation files. A new linguist-documentation Git attribute can be used to override those conventions. --- README.md | 12 +++++++++++- lib/linguist/blob_helper.rb | 15 +++++++++++++++ lib/linguist/documentation.yml | 18 ++++++++++++++++++ lib/linguist/lazy_blob.rb | 10 +++++++++- lib/linguist/repository.rb | 2 +- test/test_blob.rb | 32 ++++++++++++++++++++++++++++++++ test/test_repository.rb | 12 ++++++++++++ 7 files changed, 98 insertions(+), 3 deletions(-) create mode 100644 lib/linguist/documentation.yml diff --git a/README.md b/README.md index 4a7ea415..6f8a3993 100644 --- a/README.md +++ b/README.md @@ -26,7 +26,7 @@ Linguist supports a number of different custom overrides strategies for language ### Using gitattributes -Add a `.gitattributes` file to your project and use standard git-style path matchers for the files you want to override to set `linguist-language` and `linguist-vendored`. +Add a `.gitattributes` file to your project and use standard git-style path matchers for the files you want to override to set `linguist-documentation`, `linguist-language`, and `linguist-vendored`. ``` $ cat .gitattributes @@ -43,6 +43,16 @@ special-vendored-path/* linguist-vendored jquery.js linguist-vendored=false ``` +Similar to vendored files, Linguist excludes documentation files from your project's language stats. [lib/linguist/documentation.yml](lib/linguist/documentation.yml) lists common documentation paths and excludes them from the language statistics for your repository. + +Use the `linguist-documentation` attribute to mark or unmark paths as documentation. + +``` +$ cat .gitattributes +project-docs/* linguist-documentation +docs/formatter.rb linguist-documentation=false +``` + ### Using Emacs and Vim modelines Alternatively, you can use Vim and Emacs style modelines to set the language for a single file. Modelines can be placed anywhere within a file and are respected when determining how to syntax-highlight a file on GitHub.com diff --git a/lib/linguist/blob_helper.rb b/lib/linguist/blob_helper.rb index c368b4d0..ff11aefd 100644 --- a/lib/linguist/blob_helper.rb +++ b/lib/linguist/blob_helper.rb @@ -236,6 +236,21 @@ module Linguist name =~ VendoredRegexp ? true : false end + documentation_paths = YAML.load_file(File.expand_path("../documentation.yml", __FILE__)) + DocumentationRegexp = Regexp.new(documentation_paths.join('|')) + + # Public: Is the blob in a documentation directory? + # + # Documentation files are ignored by language statistics. + # + # See "documentation.yml" for a list of documentation conventions that match + # this pattern. + # + # Return true or false + def documentation? + name =~ DocumentationRegexp ? true : false + end + # Public: Get each line of data # # Requires Blob#data diff --git a/lib/linguist/documentation.yml b/lib/linguist/documentation.yml new file mode 100644 index 00000000..57fc6151 --- /dev/null +++ b/lib/linguist/documentation.yml @@ -0,0 +1,18 @@ +# Documentation files and directories are excluded from language +# statistics. +# +# Lines in this file are Regexps that are matched against the file +# pathname. +# +# Please add additional test coverage to +# `test/test_blob.rb#test_documentation` if you make any changes. + +## Documentation Conventions ## + +- ^docs?/ +- ^Documentation/ + +- (^|/)CONTRIBUTING(\.|$) +- (^|/)COPYING(\.|$) +- (^|/)LICEN[CS]E(\.|$) +- (^|/)README(\.|$) diff --git a/lib/linguist/lazy_blob.rb b/lib/linguist/lazy_blob.rb index 9691bca5..5465a71f 100644 --- a/lib/linguist/lazy_blob.rb +++ b/lib/linguist/lazy_blob.rb @@ -4,7 +4,7 @@ require 'rugged' module Linguist class LazyBlob - GIT_ATTR = ['linguist-language', 'linguist-vendored'] + GIT_ATTR = ['linguist-documentation', 'linguist-language', 'linguist-vendored'] GIT_ATTR_OPTS = { :priority => [:index], :skip_system => true } GIT_ATTR_FLAGS = Rugged::Repository::Attributes.parse_opts(GIT_ATTR_OPTS) @@ -37,6 +37,14 @@ module Linguist end end + def documentation? + if attr = git_attributes['linguist-documentation'] + boolean_attribute(attr) + else + super + end + end + def language return @language if defined?(@language) diff --git a/lib/linguist/repository.rb b/lib/linguist/repository.rb index 3c197fad..3837977f 100644 --- a/lib/linguist/repository.rb +++ b/lib/linguist/repository.rb @@ -159,7 +159,7 @@ module Linguist blob = Linguist::LazyBlob.new(repository, delta.new_file[:oid], new, mode.to_s(8)) # Skip vendored or generated blobs - next if blob.vendored? || blob.generated? || blob.language.nil? + next if blob.vendored? || blob.documentation? || blob.generated? || blob.language.nil? if DETECTABLE_TYPES.include?(blob.language.type) file_map[new] = [blob.language.group.name, blob.size] diff --git a/test/test_blob.rb b/test/test_blob.rb index ceb54bb3..fabd5a74 100644 --- a/test/test_blob.rb +++ b/test/test_blob.rb @@ -441,6 +441,38 @@ class TestBlob < Minitest::Test assert sample_blob("subproject/activator.bat").vendored? end + def test_documentation + assert_predicate fixture_blob("doc/foo.md"), :documentation? + assert_predicate fixture_blob("docs/foo.md"), :documentation? + refute_predicate fixture_blob("project/doc/foo.md"), :documentation? + refute_predicate fixture_blob("project/docs/foo.md"), :documentation? + + assert_predicate fixture_blob("Documentation/foo.md"), :documentation? + refute_predicate fixture_blob("project/Documentation/foo.md"), :documentation? + + assert_predicate fixture_blob("README"), :documentation? + assert_predicate fixture_blob("README.md"), :documentation? + assert_predicate fixture_blob("README.txt"), :documentation? + assert_predicate fixture_blob("foo/README"), :documentation? + + assert_predicate fixture_blob("CONTRIBUTING"), :documentation? + assert_predicate fixture_blob("CONTRIBUTING.md"), :documentation? + assert_predicate fixture_blob("CONTRIBUTING.txt"), :documentation? + assert_predicate fixture_blob("foo/CONTRIBUTING"), :documentation? + + assert_predicate fixture_blob("LICENSE"), :documentation? + assert_predicate fixture_blob("LICENCE.md"), :documentation? + assert_predicate fixture_blob("LICENSE.txt"), :documentation? + assert_predicate fixture_blob("foo/LICENSE"), :documentation? + + assert_predicate fixture_blob("COPYING"), :documentation? + assert_predicate fixture_blob("COPYING.md"), :documentation? + assert_predicate fixture_blob("COPYING.txt"), :documentation? + assert_predicate fixture_blob("foo/COPYING"), :documentation? + + refute_predicate fixture_blob("foo.md"), :documentation? + end + def test_language Samples.each do |sample| blob = sample_blob(sample[:path]) diff --git a/test/test_repository.rb b/test/test_repository.rb index b661668d..fcdd4f0c 100644 --- a/test/test_repository.rb +++ b/test/test_repository.rb @@ -99,4 +99,16 @@ class TestRepository < Minitest::Test # overridden .gitattributes assert !override_unvendored.vendored? end + + def test_linguist_override_documentation? + attr_commit = "d4c8fb8a28e91f97a7e53428a365c0abbac36d3d" + repo = linguist_repo(attr_commit).read_index + + readme = Linguist::LazyBlob.new(rugged_repository, attr_commit, "README.md") + arduino = Linguist::LazyBlob.new(rugged_repository, attr_commit, "samples/Arduino/hello.ino") + + # overridden by .gitattributes + refute_predicate readme, :documentation? + assert_predicate arduino, :documentation? + end end From a817d95d6cd9b148be1b6e2b34f8cc20fbd20769 Mon Sep 17 00:00:00 2001 From: Adam Roben Date: Fri, 13 Feb 2015 11:57:21 -0500 Subject: [PATCH 47/55] Fix CSS/SVG grammar names --- lib/linguist/languages.yml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/lib/linguist/languages.yml b/lib/linguist/languages.yml index 972a9b67..4df91c25 100644 --- a/lib/linguist/languages.yml +++ b/lib/linguist/languages.yml @@ -439,6 +439,7 @@ COBOL: CSS: type: markup + tm_scope: source.css ace_mode: css color: "#563d7c" extensions: @@ -2688,7 +2689,7 @@ SVG: type: data extensions: - .svg - tm_scope: source.xml + tm_scope: text.xml ace_mode: xml Sage: From 6a86e8ea97a25c90f2a994efd1eab45b3edc9330 Mon Sep 17 00:00:00 2001 From: Adam Roben Date: Fri, 13 Feb 2015 14:26:36 -0500 Subject: [PATCH 48/55] Add BlobHelper#include_in_language_stats? This just extracts some logic from Repository#compute_stats and makes it testable. --- lib/linguist/blob_helper.rb | 10 ++++++++++ lib/linguist/repository.rb | 10 ++-------- test/test_blob.rb | 25 +++++++++++++++++++++++++ 3 files changed, 37 insertions(+), 8 deletions(-) diff --git a/lib/linguist/blob_helper.rb b/lib/linguist/blob_helper.rb index ff11aefd..56c15f02 100644 --- a/lib/linguist/blob_helper.rb +++ b/lib/linguist/blob_helper.rb @@ -332,5 +332,15 @@ module Linguist def tm_scope language && language.tm_scope end + + DETECTABLE_TYPES = [:programming, :markup].freeze + + # Internal: Should this blob be included in repository language statistics? + def include_in_language_stats? + !vendored? && + !documentation? && + !generated? && + language && DETECTABLE_TYPES.include?(language.type) + end end end diff --git a/lib/linguist/repository.rb b/lib/linguist/repository.rb index 3837977f..895a3754 100644 --- a/lib/linguist/repository.rb +++ b/lib/linguist/repository.rb @@ -8,8 +8,6 @@ module Linguist # Its primary purpose is for gathering language statistics across # the entire project. class Repository - DETECTABLE_TYPES = [:programming, :markup].freeze - attr_reader :repository # Public: Create a new Repository based on the stats of @@ -158,12 +156,8 @@ module Linguist blob = Linguist::LazyBlob.new(repository, delta.new_file[:oid], new, mode.to_s(8)) - # Skip vendored or generated blobs - next if blob.vendored? || blob.documentation? || blob.generated? || blob.language.nil? - - if DETECTABLE_TYPES.include?(blob.language.type) - file_map[new] = [blob.language.group.name, blob.size] - end + next unless blob.include_in_language_stats? + file_map[new] = [blob.language.group.name, blob.size] end end diff --git a/test/test_blob.rb b/test/test_blob.rb index fabd5a74..24eaff83 100644 --- a/test/test_blob.rb +++ b/test/test_blob.rb @@ -517,4 +517,29 @@ class TestBlob < Minitest::Test refute blob.new(" ").empty? refute blob.new("nope").empty? end + + def test_include_in_language_stats + vendored = sample_blob("bower_components/custom/custom.js") + assert_predicate vendored, :vendored? + refute_predicate vendored, :include_in_language_stats? + + documentation = fixture_blob("README") + assert_predicate documentation, :documentation? + refute_predicate documentation, :include_in_language_stats? + + generated = sample_blob("CSS/bootstrap.min.css") + assert_predicate generated, :generated? + refute_predicate generated, :include_in_language_stats? + + data = sample_blob("Ant Build System/filenames/ant.xml") + assert_equal :data, data.language.type + refute_predicate data, :include_in_language_stats? + + prose = sample_blob("Markdown/tender.md") + assert_equal :prose, prose.language.type + refute_predicate prose, :include_in_language_stats? + + included = sample_blob("HTML/pages.html") + assert_predicate included, :include_in_language_stats? + end end From 8d2dd55c9469e912b046a27a9bd4414285a1bf07 Mon Sep 17 00:00:00 2001 From: Adam Roben Date: Fri, 13 Feb 2015 14:30:44 -0500 Subject: [PATCH 49/55] Categorize INSTALL files as documentation too --- lib/linguist/documentation.yml | 1 + test/test_blob.rb | 5 +++++ 2 files changed, 6 insertions(+) diff --git a/lib/linguist/documentation.yml b/lib/linguist/documentation.yml index 57fc6151..b884cd35 100644 --- a/lib/linguist/documentation.yml +++ b/lib/linguist/documentation.yml @@ -14,5 +14,6 @@ - (^|/)CONTRIBUTING(\.|$) - (^|/)COPYING(\.|$) +- (^|/)INSTALL(\.|$) - (^|/)LICEN[CS]E(\.|$) - (^|/)README(\.|$) diff --git a/test/test_blob.rb b/test/test_blob.rb index 24eaff83..149588e9 100644 --- a/test/test_blob.rb +++ b/test/test_blob.rb @@ -470,6 +470,11 @@ class TestBlob < Minitest::Test assert_predicate fixture_blob("COPYING.txt"), :documentation? assert_predicate fixture_blob("foo/COPYING"), :documentation? + assert_predicate fixture_blob("INSTALL"), :documentation? + assert_predicate fixture_blob("INSTALL.md"), :documentation? + assert_predicate fixture_blob("INSTALL.txt"), :documentation? + assert_predicate fixture_blob("foo/INSTALL"), :documentation? + refute_predicate fixture_blob("foo.md"), :documentation? end From 1bf91d4281932030bcbbe9ace852fe71d2821cd9 Mon Sep 17 00:00:00 2001 From: Adam Roben Date: Fri, 13 Feb 2015 14:31:40 -0500 Subject: [PATCH 50/55] Use .html files in some #documentation? tests This is what we were trying to classify in the twbs/bootstrap repository, for example. --- test/test_blob.rb | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/test/test_blob.rb b/test/test_blob.rb index 149588e9..d59e6794 100644 --- a/test/test_blob.rb +++ b/test/test_blob.rb @@ -442,10 +442,10 @@ class TestBlob < Minitest::Test end def test_documentation - assert_predicate fixture_blob("doc/foo.md"), :documentation? - assert_predicate fixture_blob("docs/foo.md"), :documentation? - refute_predicate fixture_blob("project/doc/foo.md"), :documentation? - refute_predicate fixture_blob("project/docs/foo.md"), :documentation? + assert_predicate fixture_blob("doc/foo.html"), :documentation? + assert_predicate fixture_blob("docs/foo.html"), :documentation? + refute_predicate fixture_blob("project/doc/foo.html"), :documentation? + refute_predicate fixture_blob("project/docs/foo.html"), :documentation? assert_predicate fixture_blob("Documentation/foo.md"), :documentation? refute_predicate fixture_blob("project/Documentation/foo.md"), :documentation? From 6f5c935837e1ea206295444f9f94e7985725a523 Mon Sep 17 00:00:00 2001 From: Adam Roben Date: Fri, 13 Feb 2015 14:52:30 -0500 Subject: [PATCH 51/55] Clarify the difference between vendored and documentation files --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 6f8a3993..6a2447a9 100644 --- a/README.md +++ b/README.md @@ -33,7 +33,7 @@ $ cat .gitattributes *.rb linguist-language=Java ``` -Checking code you didn't write, such as JavaScript libraries, into your git repo is a common practice, but this often inflates your project's language stats and may even cause your project to be labeled as another language. By default, Linguist treats all of the paths defined in [lib/linguist/vendor.yml](https://github.com/github/linguist/blob/master/lib/linguist/vendor.yml) as vendored and therefore doesn't include them in the language statistics for a repository. +Checking code you didn't write, such as JavaScript libraries, into your git repo is a common practice, but this often inflates your project's language stats and may even cause your project to be labeled as another language. By default, Linguist treats all of the paths defined in [lib/linguist/vendor.yml](https://github.com/github/linguist/blob/master/lib/linguist/vendor.yml) as vendored and therefore doesn't include them in the language statistics for a repository. Vendored files are also hidden by default in diffs on github.com. Use the `linguist-vendored` attribute to vendor or un-vendor paths. @@ -43,7 +43,7 @@ special-vendored-path/* linguist-vendored jquery.js linguist-vendored=false ``` -Similar to vendored files, Linguist excludes documentation files from your project's language stats. [lib/linguist/documentation.yml](lib/linguist/documentation.yml) lists common documentation paths and excludes them from the language statistics for your repository. +Similar to vendored files, Linguist excludes documentation files from your project's language stats. (Unlike vendored files, documentation files are displayed in diffs on github.com.) [lib/linguist/documentation.yml](lib/linguist/documentation.yml) lists common documentation paths and excludes them from the language statistics for your repository. Use the `linguist-documentation` attribute to mark or unmark paths as documentation. From 353479fe7251391222e017c4be82b820e1069969 Mon Sep 17 00:00:00 2001 From: Adam Roben Date: Fri, 13 Feb 2015 16:20:14 -0500 Subject: [PATCH 52/55] Add a color for HTML Courtesy of @fabianperez and the HTML5 logo. --- lib/linguist/languages.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/linguist/languages.yml b/lib/linguist/languages.yml index 4df91c25..48160604 100644 --- a/lib/linguist/languages.yml +++ b/lib/linguist/languages.yml @@ -1174,6 +1174,7 @@ HTML: type: markup tm_scope: text.html.basic ace_mode: html + color: "#e44b23" aliases: - xhtml extensions: From 5ad6add91e16139477aec0ce10bf8cb1da025e96 Mon Sep 17 00:00:00 2001 From: Adam Roben Date: Fri, 13 Feb 2015 16:44:10 -0500 Subject: [PATCH 53/55] Bump version to 4.4.0 --- lib/linguist/version.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/linguist/version.rb b/lib/linguist/version.rb index e8c4f98b..674a61b9 100644 --- a/lib/linguist/version.rb +++ b/lib/linguist/version.rb @@ -1,3 +1,3 @@ module Linguist - VERSION = "4.3.1" + VERSION = "4.4.0" end From 3a4ab156b0597755f9644749748642d0417ad11b Mon Sep 17 00:00:00 2001 From: Adam Roben Date: Mon, 16 Feb 2015 09:45:24 -0500 Subject: [PATCH 54/55] Update all grammars MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * vendor/grammars/GDScript-sublime 99a0d51...44ac5c4 (3): > Merge pull request #7 from erbridge/null-keyword > Merge pull request #6 from erbridge/assert-keyword > Merge pull request #5 from erbridge/update-yaml * vendor/grammars/Modelica d7e50e3...f2b1242 (8): > Update README.md > Update README.md > Update README.md > Update README.md > Update README.md > Update README.md > Delete .gitattributes > Create .gitattributes * vendor/grammars/NimLime 7581153...58a1e0c (5): > Merge pull request #13 from fenekku/master > Refactored a bit nimsuggest service hosting a bit more robust > Print error if nimsuggest does not respond > Close test sock Turn off autoreload > Merge pull request #14 from Varriount/update-to-sockets * vendor/grammars/Sublime-SQF-Language 708c78a...0313fbe (1): > Merge pull request #12 from DenVdmj/master * vendor/grammars/SublimePapyrus 5a7b3e7...152c7b7 (1): > Added setting * vendor/grammars/ats.sublime d954ef6...2565468 (2): > fix record/tuple/then/else with { > fix bugs for single quote char * vendor/grammars/c.tmbundle f825425...f6048af (1): > Add syntax highlight for CoreGraphics types * vendor/grammars/dart-sublime-bundle fecdbc5...c1afc62 (4): > Merge pull request #450 from guillermooo-forks/prep-release > Merge pull request #449 from guillermooo-forks/fix-analysis-server-init > Merge pull request #448 from guillermooo-forks/prep-release > Merge pull request #446 from guillermooo-forks/new-analysis-server-integration-linting * vendor/grammars/elixir-tmbundle dcf1fc1...9c63ff0 (1): > Merge pull request #34 from bharendt/master * vendor/grammars/factor 4ada328...ec896cd (17): > images.tiff: Fix partial refactor rollback and offeset typo. > images.tiff: Fix infinite loop bug exposed by AFL test suite. IDFs are found at certain offsets, then the next offset is after the IDF. If a next-offset is its own previous offset, then it would loop. Now we record the offset and stop if it would loop. > alien.libraries: Cannot include alien.libraries.finder here for docs. > Explicitly reference find-library in alien docs > Fix typo in hash-set documentation > opencl: remove (use size_t instead). > forestdb.ffi: Update ffi. > forestdb: Embrace the forestdb handle vs kvs handle idiom. Update tests. > stack-checker.state: Fix error in docs. Tweak doc grammar. > bootstrap.image.upload: Use pscp on Windows for uploading boot images since c:\ paths are treated as hostnames and the command fails. > stack-checker: minor help-lint fixes. > compiler.tree.*: more compiler tree docs > stack-checker.*: misc doc additions > stack-checker.known-words: docs and tests > stack-checker.values: docs and tests > stack-checker.state: docs and tests > stack-checker.backend: docs and tests * vendor/grammars/fsharpbinding 92d969b...0cd6439 (2): > revert typo, typo is actually upstream in monodevelop > bump version number * vendor/grammars/java.tmbundle a74cb83...ccdebdf (1): > Highlight volatile like synchronized * vendor/grammars/javadoc.tmbundle 484d468...5276d7a (1): > Grammar: Be strict on what can start an HTML tag * vendor/grammars/language-clojure bae6eee...cfc8a5c (6): > Prepare 0.12.0 release > Merge pull request #14 from joelash/master > Prepare 0.11.0 release > Merge pull request #13 from hanjos/proposed > Add initial spec > Add initial Travis CI config * vendor/grammars/language-csharp fba368a...d07ba8f (4): > Prepare 0.5.0 release > Merge pull request #19 from atom/ks-remove-deprecations > Prepare 0.4.0 release > Merge pull request #16 from damieng/master * vendor/grammars/language-gfm 18400b2...5f5df30 (3): > Prepare 0.64.0 release > Add heading spec with no space > Merge pull request #77 from jonathandelgado/master * vendor/grammars/language-python 0141d44...8daa100 (4): > Prepare 0.32.0 release > Merge pull request #54 from nicktimko/binary-literals > Prepare 0.31.0 release > Merge pull request #53 from nicktimko/python-3-octals * vendor/grammars/latex.tmbundle 16154ce...0441781 (16): > Remove unnecessary code from tests > Try to read tex file using different encodings > Recognize errors in files containing spaces > Prepend `lib` to path in Python scripts > Fix minor style issues reported by `flake8` > Tell `flake8` to ignore “relative” imports > Use `format` instead of `sprintf` > Use encoding `latin_1` to open log files > Add “LaTeX Tidy” to the bundle again > Add `YAML::Tiny` library to the bundle > Use `texdoc` to show documentation for single word > Determine `TEXMFMAIN` only once in `texdoc` > Autoupdate cache for “Documentation for Package” > Update “Documentation for Package” > Use direct imports in `texdoc` > Add folding markers for sections * vendor/grammars/objective-c.tmbundle 8387be8...1bade8a (1): > Add syntax highlight for the typedefs in NSObjCRuntime.h * vendor/grammars/php.tmbundle 1ae104d...7178a10 (4): > Doc Snippet: Allow for static keyword in declarations > Further work on escapes in single-quoted regular expressions > Grammar: Correct improper '\' escaping > Improve/simplify escaping rules for single-quoted regexps * vendor/grammars/sublime-nix 412f7e1...217ffe5 (1): > Fix comma-first --- vendor/grammars/GDScript-sublime | 2 +- vendor/grammars/Modelica | 2 +- vendor/grammars/NimLime | 2 +- vendor/grammars/Sublime-SQF-Language | 2 +- vendor/grammars/SublimePapyrus | 2 +- vendor/grammars/ats.sublime | 2 +- vendor/grammars/c.tmbundle | 2 +- vendor/grammars/dart-sublime-bundle | 2 +- vendor/grammars/elixir-tmbundle | 2 +- vendor/grammars/factor | 2 +- vendor/grammars/fsharpbinding | 2 +- vendor/grammars/java.tmbundle | 2 +- vendor/grammars/javadoc.tmbundle | 2 +- vendor/grammars/language-clojure | 2 +- vendor/grammars/language-csharp | 2 +- vendor/grammars/language-gfm | 2 +- vendor/grammars/language-python | 2 +- vendor/grammars/latex.tmbundle | 2 +- vendor/grammars/objective-c.tmbundle | 2 +- vendor/grammars/php.tmbundle | 2 +- vendor/grammars/sublime-nix | 2 +- 21 files changed, 21 insertions(+), 21 deletions(-) diff --git a/vendor/grammars/GDScript-sublime b/vendor/grammars/GDScript-sublime index 99a0d512..44ac5c4a 160000 --- a/vendor/grammars/GDScript-sublime +++ b/vendor/grammars/GDScript-sublime @@ -1 +1 @@ -Subproject commit 99a0d512248fb85741b00d9d702cf97797de9a4c +Subproject commit 44ac5c4af2e7b28a7abe0a5c56855b22d6ef3b19 diff --git a/vendor/grammars/Modelica b/vendor/grammars/Modelica index d7e50e39..f2b1242b 160000 --- a/vendor/grammars/Modelica +++ b/vendor/grammars/Modelica @@ -1 +1 @@ -Subproject commit d7e50e39c14a49153d3c17dfbd623258cf1a5d69 +Subproject commit f2b1242b93c9958aec0dbee2e02b35b12753e506 diff --git a/vendor/grammars/NimLime b/vendor/grammars/NimLime index 75811539..58a1e0c0 160000 --- a/vendor/grammars/NimLime +++ b/vendor/grammars/NimLime @@ -1 +1 @@ -Subproject commit 75811539ec1a32e72821a540e1ef73fce0febd43 +Subproject commit 58a1e0c0c12b34f4a57af66e119f6c09bfce4a73 diff --git a/vendor/grammars/Sublime-SQF-Language b/vendor/grammars/Sublime-SQF-Language index 708c78a0..0313fbe6 160000 --- a/vendor/grammars/Sublime-SQF-Language +++ b/vendor/grammars/Sublime-SQF-Language @@ -1 +1 @@ -Subproject commit 708c78a0ba5e9449c3a913243ce9dfdeb458e68c +Subproject commit 0313fbe6fb962a21c337a327dcb72235af913536 diff --git a/vendor/grammars/SublimePapyrus b/vendor/grammars/SublimePapyrus index 5a7b3e7b..152c7b79 160000 --- a/vendor/grammars/SublimePapyrus +++ b/vendor/grammars/SublimePapyrus @@ -1 +1 @@ -Subproject commit 5a7b3e7b29fdee835d0198d1f20e21c0ff874154 +Subproject commit 152c7b79ffe3bfcef877f82d845fda0bc2608b08 diff --git a/vendor/grammars/ats.sublime b/vendor/grammars/ats.sublime index d954ef6b..2565468f 160000 --- a/vendor/grammars/ats.sublime +++ b/vendor/grammars/ats.sublime @@ -1 +1 @@ -Subproject commit d954ef6b27dbdbed6276c7b9b63d3a39d9f69a2c +Subproject commit 2565468fd437fbe436bc21509a72ba8bad35fc7c diff --git a/vendor/grammars/c.tmbundle b/vendor/grammars/c.tmbundle index f8254252..f6048afe 160000 --- a/vendor/grammars/c.tmbundle +++ b/vendor/grammars/c.tmbundle @@ -1 +1 @@ -Subproject commit f8254252620b24c50d23ec804344915d498ecd7f +Subproject commit f6048afe693e50adf47d46aba791f95c9138823e diff --git a/vendor/grammars/dart-sublime-bundle b/vendor/grammars/dart-sublime-bundle index fecdbc5f..c1afc623 160000 --- a/vendor/grammars/dart-sublime-bundle +++ b/vendor/grammars/dart-sublime-bundle @@ -1 +1 @@ -Subproject commit fecdbc5f246571025bf3d22feb9fd9a260719e02 +Subproject commit c1afc623bcff9325edc7a50583575f6a9024dd65 diff --git a/vendor/grammars/elixir-tmbundle b/vendor/grammars/elixir-tmbundle index dcf1fc12..9c63ff09 160000 --- a/vendor/grammars/elixir-tmbundle +++ b/vendor/grammars/elixir-tmbundle @@ -1 +1 @@ -Subproject commit dcf1fc125c81c904fae8dc91d483905bdb5d5f5d +Subproject commit 9c63ff09bd411d3e5c5a737145fc33d288804c9c diff --git a/vendor/grammars/factor b/vendor/grammars/factor index 4ada3288..ec896cd5 160000 --- a/vendor/grammars/factor +++ b/vendor/grammars/factor @@ -1 +1 @@ -Subproject commit 4ada3288806d32beaf68e84c079653106e366d5f +Subproject commit ec896cd5ad05d4ae9b837fa5a64cd9431d2a14d6 diff --git a/vendor/grammars/fsharpbinding b/vendor/grammars/fsharpbinding index 92d969b1..0cd6439b 160000 --- a/vendor/grammars/fsharpbinding +++ b/vendor/grammars/fsharpbinding @@ -1 +1 @@ -Subproject commit 92d969b19715192928d0c865c87a1aac3ba98dac +Subproject commit 0cd6439b513631c469d69b0e02a26804b068674b diff --git a/vendor/grammars/java.tmbundle b/vendor/grammars/java.tmbundle index a74cb835..ccdebdf8 160000 --- a/vendor/grammars/java.tmbundle +++ b/vendor/grammars/java.tmbundle @@ -1 +1 @@ -Subproject commit a74cb835b821e9a3498d1810d546cb49881df73e +Subproject commit ccdebdf888400c492ea55884e03a8f9d1b61de8b diff --git a/vendor/grammars/javadoc.tmbundle b/vendor/grammars/javadoc.tmbundle index 484d468f..5276d7a9 160000 --- a/vendor/grammars/javadoc.tmbundle +++ b/vendor/grammars/javadoc.tmbundle @@ -1 +1 @@ -Subproject commit 484d468f47f6a17c98f73b369fa00a6bbc2a22fd +Subproject commit 5276d7a93f0cf53b7d425c39c6968b09ea9f2d40 diff --git a/vendor/grammars/language-clojure b/vendor/grammars/language-clojure index bae6eee8..cfc8a5ce 160000 --- a/vendor/grammars/language-clojure +++ b/vendor/grammars/language-clojure @@ -1 +1 @@ -Subproject commit bae6eee8557c2158592ac485a7168ccd10fc6dfb +Subproject commit cfc8a5ce603efdeb4018dd7f04846815cc3de9b1 diff --git a/vendor/grammars/language-csharp b/vendor/grammars/language-csharp index fba368a8..d07ba8f1 160000 --- a/vendor/grammars/language-csharp +++ b/vendor/grammars/language-csharp @@ -1 +1 @@ -Subproject commit fba368a8397e298fa78679b4ee0a542b2474c150 +Subproject commit d07ba8f16818d95ff663929f8104f7c101cc3ae6 diff --git a/vendor/grammars/language-gfm b/vendor/grammars/language-gfm index 18400b22..5f5df306 160000 --- a/vendor/grammars/language-gfm +++ b/vendor/grammars/language-gfm @@ -1 +1 @@ -Subproject commit 18400b22cd7968afee982b4e0aa57ef7813ce991 +Subproject commit 5f5df3064c9dccbfd8dac8d260b0d4a843ac88a5 diff --git a/vendor/grammars/language-python b/vendor/grammars/language-python index 0141d449..8daa1008 160000 --- a/vendor/grammars/language-python +++ b/vendor/grammars/language-python @@ -1 +1 @@ -Subproject commit 0141d44946d55ae06ce4dba90b1b0e08db1e437e +Subproject commit 8daa10089a14bacf2346bb16f31d1189489696e0 diff --git a/vendor/grammars/latex.tmbundle b/vendor/grammars/latex.tmbundle index 16154cef..04417819 160000 --- a/vendor/grammars/latex.tmbundle +++ b/vendor/grammars/latex.tmbundle @@ -1 +1 @@ -Subproject commit 16154cef75708eabc5ebd83c61ad49e8830d835d +Subproject commit 04417819498f3a8965e49b86cee3bdd1ad475b19 diff --git a/vendor/grammars/objective-c.tmbundle b/vendor/grammars/objective-c.tmbundle index 8387be89..1bade8a1 160000 --- a/vendor/grammars/objective-c.tmbundle +++ b/vendor/grammars/objective-c.tmbundle @@ -1 +1 @@ -Subproject commit 8387be8975e47cd8e4314cb197e4fc2834159807 +Subproject commit 1bade8a1c919c358fc4a6d83ba93e98e419ffede diff --git a/vendor/grammars/php.tmbundle b/vendor/grammars/php.tmbundle index 1ae104d8..7178a102 160000 --- a/vendor/grammars/php.tmbundle +++ b/vendor/grammars/php.tmbundle @@ -1 +1 @@ -Subproject commit 1ae104d86b303a7cf8460a4abca2125331b7bab6 +Subproject commit 7178a102ce62352d3512301f62ddec8157cd0d8a diff --git a/vendor/grammars/sublime-nix b/vendor/grammars/sublime-nix index 412f7e1d..217ffe59 160000 --- a/vendor/grammars/sublime-nix +++ b/vendor/grammars/sublime-nix @@ -1 +1 @@ -Subproject commit 412f7e1da57bd1be9ce185f27c0f36a95639ca25 +Subproject commit 217ffe591fca71624f63099d8fb64b1e3001939a From 10ffd870e2ea21d4d73048bb0e07410bd56e5bec Mon Sep 17 00:00:00 2001 From: Adam Roben Date: Mon, 16 Feb 2015 09:50:04 -0500 Subject: [PATCH 55/55] Update to latest version of season This is needed to parse the Clojure grammar. --- package.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/package.json b/package.json index acd7cd9e..025b6c1e 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "repository": "https://github.com/github/linguist", "dependencies": { - "season": "~>3.0" + "season": "~>5.0" } }