Commit Graph

50 Commits

Author SHA1 Message Date
Arfon Smith
e0d890240b Adding basic script for adding language_id fields to languages.yml 2016-09-12 21:35:10 -07:00
Paul Chaignon
adaf4011bc Support for .YAML-tmLanguage grammar files 2016-06-26 20:57:50 +02:00
Paul Chaignon
4a031107ac Support for Sublime Text YAML syntax definitions
Sublime Text YAML syntax definitions use the .sublime-syntax file extension

Most syntax files declare a YAML 1.2 syntax although they are YAML 1.1 compatible
Thus, the YAML version header is stripped off before parsing
Displays a warning if parsing fails

In .sublime-syntax files, the scope is under the 'scope' key -- as opposed to the usual 'scopeName' key
2016-06-26 19:47:34 +02:00
Brandon Keepers
12228fb525 Proper exit status for script/licensed 2016-04-16 10:25:34 -04:00
Arfon Smith
065c809dd5 Updating grammars based on script/licensed 2016-03-18 16:52:49 -06:00
Arfon Smith
b5091e88ad Updating licensor -> licensed 2016-03-18 12:32:52 -06:00
Arfon Smith
afcf1c6c22 Merge branch 'master' into licensor 2016-03-17 19:57:29 -06:00
Dan Buch
550b67215c Fetch libicu via Travis, switch back to container-based
This is a test.  I don't understand the details of how the `bundle config
build.charlock_holmes` bit works/fails, so the intent here is to get a PR posted
for purposes of getting feedback via the full Travis machinery.
2015-11-08 06:41:22 -05:00
Brandon Keepers
0d469e2966 Add licensor 2015-10-20 17:44:31 -04:00
Adam Roben
57429f3194 Don't warn if a package contains multiple copies of a grammar
This is pretty common (e.g., if the package contains the grammar in both
XML and JSON).
2015-05-19 09:27:13 -04:00
Adam Roben
9bdf890fd8 Fix nil NoMethodError when converting a duplicated grammar
We were sticking a nil value into the grammars Array when we encountered
a duplicate scope. Now we strip out nils before returning the Array.
2015-05-19 09:27:13 -04:00
Paul Chaignon
ca12a99970 Handle SSH links to submodules 2015-05-01 21:03:02 +02:00
Brandon Keepers
8da458e1a8 apt-get update before downloading deps 2015-03-08 16:24:05 -07:00
Adam Roben
6fef6b578a Fix script/convert-grammars --add 2015-01-08 10:49:01 -05:00
Florian M. Schäfer
417171cfe1 Don't pass JSON-files through csonc 2015-01-07 15:27:40 +01:00
Florian Schäfer
6843aa7cc8 Enable file extension .json for grammar files 2015-01-07 15:27:40 +01:00
Florian M. Schäfer
ab1b603c78 Require thread-gem and solve problem with visibility of variable p in convert-grammars
* The `thread`-gem is required for the script, because otherwise Ruby throws an error that `Queue` in line 259 is an uninitialized constant
* The variable `p` was previously in the same method with the status message saying `OK ‹path› ‹converted scopes›`. But `p` is now defined in load_grammars() and thus not visible when showing the OK-message in install_grammars(). This was solved by adding a path-parameter to install_grammars().
2015-01-06 22:06:55 +01:00
Adam Roben
7edddec920 Make convert-grammars a little quieter
No need to say "Done" at the end. We're obviously done; the script is
exiting.
2015-01-06 13:39:18 -05:00
Adam Roben
917bc48348 Use File.directory? to test for local directories
Some of our submodules end in ".tmLanguage", which we used to think
meant it was a file rather than a directory.
2015-01-06 13:31:26 -05:00
Adam Roben
581dc36c47 Ruby 1.9 compatibility for convert-grammars 2015-01-06 13:28:26 -05:00
Adam Roben
ada6f6882a Test that grammars.yml lists the right scopes for each submodule
convert-grammars now supports a few flags that we can use to make it
dump out the YAML just for the local grammar submodules. We can then
compare this to the YAML that's actually in grammars.yml to check that
they're the same. If they aren't, grammars.yml needs to be updated.
This will help catch mistakes like using the wrong scope name.
2015-01-06 13:28:25 -05:00
Adam Roben
78a0030d46 download-grammars -> convert-grammars
Downloading is only a small part of what this script does. The main
thing it does is convert grammars to JSON.
2015-01-06 13:28:25 -05:00
Adam Roben
188e579df7 Use a single git fetch invocation in CI 2015-01-06 12:56:08 -05:00
Adam Roben
d82e4801ff Use --with-icu-dir everywhere but Ruby 2.0
--with-icu-dir is broken on Ruby 2.0 due to
https://bugs.ruby-lang.org/issues/8074. Specifying
--with-icu-include/--with-icu-lib is broken on Ruby 2.1 and newer due to
https://github.com/bundler/bundler/pull/3338. So now we use
--with-icu-dir everywhere but Ruby 2.0.
2015-01-06 12:44:39 -05:00
Adam Roben
d258d146b8 Move before_install steps to a separate script 2015-01-06 12:44:14 -05:00
Adam Roben
84235478ef Download and install libicu-dev without using sudo
We tell apt-get to download it to vendor/apt and then install it into a
vendor/icu directory.

We should be able to just specify --with-icu-dir, but apparently Ruby
2.0 (but not 1.9 or 2.1 or 2.2) has a bug that requires us to use
--with-icu-include/--with-icu-lib instead. Otherwise it can't find the
ICU libraries.
2014-12-31 15:42:34 -05:00
Adam Roben
33afa30c30 Don't use question marks in struct fields
Apparently it doesn't work in Ruby 1.9.
2014-12-22 17:06:49 -05:00
Brandon Keepers
8369d253f8 Move grammar sources to vendor/grammars 2014-12-22 16:28:58 -05:00
Adam Roben
47b785a8fd Initialize submodules before updating them
Initialization cannot happen in parallel because every submodule has to
get written to .git/config.
2014-12-19 16:05:17 -05:00
Adam Roben
6a2bf3fd2f Update submodules in parallel to speed up bootstrap
This runs 8 `git submodule update` processes in parallel, speeding up
bootstrap from 2 minutes to 30 seconds for me. (Obviously this is
dependent on bandwidth.)
2014-12-19 16:05:17 -05:00
Adam Roben
f2ab426d38 Move all grammars that use Git repos to submodules
This makes it so we don't have to redownload all the grammars every time
we build the grammars gem. It will also let us verify that grammars.yml
is accurate in the future by checking it against the submodules on disk.

script/bootstrap now updates the submodules.
2014-12-19 15:39:02 -05:00
Brandon Keepers
16a6dda3dd Remove cached gems 2014-12-19 10:52:29 -05:00
Arfon Smith
15191b068d Merge pull request #1890 from github/cut-release-v4.2.4
Updating Linguist version for v4.2.4
2014-12-18 14:31:21 -08:00
Arfon Smith
ee9cc24e52 Longer timeout 2014-12-18 14:28:33 -08:00
Adam Roben
b9eacfefc8 Add script/bootstrap
This just runs Bundler with the right options. Maybe it will do more in
the future, like update submodules.
2014-12-18 12:13:32 -05:00
Brandon Keepers
70b1ec97db Clean before running janky build 2014-12-03 10:07:59 -08:00
Adam Roben
879e4977e4 Handle includes like source.c#block 2014-11-13 13:45:02 -05:00
Adam Roben
613b71719f Add back some accidentally pruned grammars
A bug in the prune-grammars script caused these to be removed.
2014-11-13 13:42:36 -05:00
Adam Roben
2870f6d038 Prune unused grammars
script/prune-grammars will remove any grammars that aren't needed from
grammars.yml.
2014-11-13 13:16:24 -05:00
Adam Roben
046fb18980 Add github-linguist-grammars gem
The purpose of this gem is to package up the language grammars that are
used for syntax highlighting on github.com. The grammars are TextMate,
Sublime Text, or Atom language grammars, converted to JSON and given the
filename SCOPE.json, where SCOPE is the language scope that the grammar
defines.

The github-linguist-grammars gem packages up all the grammars, and also
exports a Linguist::Grammars.path method to locate the directory
containing the grammars.

To build the gem, simply run `rake build_grammars_gem`. The grammars.yml
file lists all the repositories we download grammars from, as well as
which scopes are defined by each repository. The
script/download-grammars script takes that list and downloads and
processes the grammars into the format expected by the gem.
2014-11-13 11:03:53 -05:00
Arfon Smith
a7b8e38bf3 CI step for samples 2014-10-23 12:16:02 +01:00
Arfon Smith
2a66b754c2 CI config 2014-10-21 18:41:25 -04:00
Arfon Smith
6d07302963 Update cibuild 2014-10-21 16:49:13 -05:00
Arfon Smith
d831205f6a Update cibuild 2014-10-21 16:41:55 -05:00
Arfon Smith
a9b9e6216b Update cibuild 2014-10-21 16:10:05 -05:00
Arfon Smith
3ba090de7e Update cibuild 2014-10-21 16:07:49 -05:00
Arfon Smith
c105208481 Update cibuild 2014-10-21 14:06:43 -05:00
Arfon Smith
0c9e14eeff Update cibuild 2014-10-21 14:06:13 -05:00
Arfon Smith
2a8a5cdca9 +x 2014-10-21 14:04:52 -05:00
Arfon Smith
1f91acbd9d Janky CI 2014-10-21 14:03:30 -05:00