Merge pull request #1821 from github/cut-release-v4.2.3

v4.2.3

.gitignore

@@ -1,3 +1,6 @@
-Gemfile.lock
+/Gemfile.lock
 .bundle/
-vendor/
+benchmark/
+lib/linguist/samples.json
+/grammars
+/node_modules

.travis.yml

@@ -1,9 +1,13 @@
-before_install: 
+before_install:
+  - git fetch origin master:master
+  - git fetch origin v2.0.0:v2.0.0
+  - git fetch origin test/attributes:test/attributes
+  - git fetch origin test/master:test/master
   - sudo apt-get install libicu-dev -y
-  - gem update --system 2.1.11
 rvm:
   - 1.9.3
   - 2.0.0
-  - 2.1.1
+  - 2.1
+  - 2.2
 notifications:
   disabled: true

CONTRIBUTING.md

@@ -0,0 +1,31 @@
+## Contributing
+
+The majority of contributions won't need to touch any Ruby code at all. The [master language list][languages] is just a YAML configuration file.
+
+Almost all bug fixes or new language additions should come with some additional code samples. Just drop them under [`samples/`][samples] in the correct subdirectory and our test suite will automatically test them. In most cases you shouldn't need to add any new assertions.
+
+### My code is detected as the wrong language
+
+This can usually be solved either by adding a new filename or file name extension to the language's entry in [`languages.yml`][languages] or adding more [samples][samples] for your language to the repository to make Linguist's classifier smarter.
+
+### Syntax highlighting looks wrong
+
+Assuming your code is being detected as the right language (see above), in most cases this is due to a bug in the language grammar rather than a bug in Linguist. [`grammars.yml`][grammars] lists all the grammars we use for syntax highlighting on github.com. Find the one corresponding to your code's programming language and submit a bug report upstream.
+
+You can also try to fix the bug yourself and submit a Pull Request. [This piece from TextMate's documentation](http://manual.macromates.com/en/language_grammars) offers a good introduction on how to work with TextMate-compatible grammars. You can test grammars using [Lightshow](https://lightshow.githubapp.com).
+
+Once the bug has been fixed upstream, please let us know and we'll pick it up for GitHub.
+
+### I want to add support for the `X` programming language
+
+Great! You'll need to:
+
+0. Add an entry for your language to [`languages.yml`][languages].
+0. Add a grammar for your language to [`grammars.yml`][grammars] by running `script/download-grammars --add URL`. Please only add grammars that have a license that permits redistribution.
+0. Add samples for your language to the [samples directory][samples].
+
+We try only to add languages once they have some usage on GitHub, so please note in-the-wild usage examples in your pull request. In most cases we prefer that languages already be in use in hundreds of repositories before supporting them in Linguist.
+
+[grammars]: /grammars.yml
+[languages]: /lib/linguist/languages.yml
+[samples]: /samples

Gemfile

@@ -1,2 +1,5 @@
 source 'https://rubygems.org'
-gemspec
+gemspec :name => "github-linguist"
+gemspec :name => "github-linguist-grammars"
+gem 'test-unit', require: false if RUBY_VERSION >= '2.2'
+gem 'byebug' if RUBY_VERSION >= '2.0'

README.md

@@ -1,12 +1,14 @@
 # Linguist
 
-We use this library at GitHub to detect blob languages, highlight code, ignore binary files, suppress generated files in diffs, and generate language breakdown graphs.
+We use this library at GitHub to detect blob languages, ignore binary files, suppress generated files in diffs, and generate language breakdown graphs.
+
+Tips for filing issues and creating pull requests can be found in [`CONTRIBUTING.md`](/CONTRIBUTING.md).
 
 ## Features
 
 ### Language detection
 
-Linguist defines a list of all languages known to GitHub in a [yaml file](https://github.com/github/linguist/blob/master/lib/linguist/languages.yml). In order for a file to be highlighted, a language and a lexer must be defined there.
+Linguist defines a list of all languages known to GitHub in a [yaml file](https://github.com/github/linguist/blob/master/lib/linguist/languages.yml).
 
 Most languages are detected by their file extension. For disambiguating between files with common extensions, we first apply some common-sense heuristics to pick out obvious languages. After that, we use a
 [statistical
@@ -24,7 +26,9 @@ See [lib/linguist/language.rb](https://github.com/github/linguist/blob/master/li
 
 ### Syntax Highlighting
 
-The actual syntax highlighting is handled by our Pygments wrapper, [pygments.rb](https://github.com/tmm1/pygments.rb). It also provides a [Lexer abstraction](https://github.com/tmm1/pygments.rb/blob/master/lib/pygments/lexer.rb) that determines which highlighter should be used on a file.
+Syntax highlighting in GitHub is performed using TextMate-compatible grammars. These are the same grammars that TextMate, Sublime Text and Atom use.
+
+Every language in `languages.yml` is mapped to its corresponding TM `scope`. This scope will be used when picking up a grammar for highlighting. **When adding a new language to Linguist, please add its corresponding scope too (assuming there's an existing TextMate bundle, Sublime Text package, or Atom package) so syntax highlighting works for it**.
 
 ### Stats
 
@@ -32,33 +36,57 @@ The Language stats bar that you see on every repository is built by aggregating
 
 The repository stats API, accessed through `#languages`, can be used on a directory:
 
+***API UPDATE***
+
+Since [Version 3.0.0](https://github.com/github/linguist/releases/tag/v3.0.0) Linguist expects a git repository (in the form of a [Rugged::Repository](https://github.com/libgit2/rugged#repositories)) to be passed when initializing `Linguist::Repository`.
+
+
 ```ruby
-project = Linguist::Repository.from_directory(".")
-project.language.name  #=> "Ruby"
-project.languages      #=> { "Ruby" => 0.98, "Shell" => 0.02 }
+require 'rugged'
+require 'linguist'
+
+repo = Rugged::Repository.new('.')
+project = Linguist::Repository.new(repo, repo.head.target_id)
+project.language       #=> "Ruby"
+project.languages      #=> { "Ruby" => 119387 }
 ```
 
 These stats are also printed out by the `linguist` binary. You can use the
 `--breakdown` flag, and the binary will also output the breakdown of files by language.
 
-You can try running `linguist` on the `lib/` directory in this repository itself:
+You can try running `linguist` on the root directory in this repository itself:
 
-    $ bundle exec linguist lib/ --breakdown
+    $ bundle exec linguist --breakdown
 
     100.00% Ruby
 
     Ruby:
-    linguist/blob_helper.rb
-    linguist/classifier.rb
-    linguist/file_blob.rb
-    linguist/generated.rb
-    linguist/heuristics.rb
-    linguist/language.rb
-    linguist/md5.rb
-    linguist/repository.rb
-    linguist/samples.rb
-    linguist/tokenizer.rb
-    linguist.rb
+    Gemfile
+    Rakefile
+    bin/linguist
+    github-linguist.gemspec
+    lib/linguist.rb
+    lib/linguist/blob_helper.rb
+    lib/linguist/classifier.rb
+    lib/linguist/file_blob.rb
+    lib/linguist/generated.rb
+    lib/linguist/heuristics.rb
+    lib/linguist/language.rb
+    lib/linguist/lazy_blob.rb
+    lib/linguist/md5.rb
+    lib/linguist/repository.rb
+    lib/linguist/samples.rb
+    lib/linguist/tokenizer.rb
+    lib/linguist/version.rb
+    test/test_blob.rb
+    test/test_classifier.rb
+    test/test_heuristics.rb
+    test/test_language.rb
+    test/test_md5.rb
+    test/test_pedantic.rb
+    test/test_repository.rb
+    test/test_samples.rb
+    test/test_tokenizer.rb
 
 #### Ignore vendored files
 
@@ -80,9 +108,34 @@ Linguist::FileBlob.new("underscore.min.js").generated? # => true
 
 See [Linguist::Generated#generated?](https://github.com/github/linguist/blob/master/lib/linguist/generated.rb).
 
+## Overrides
+
+Linguist supports custom overrides for language definitions and vendored paths. Add a `.gitattributes` file to your project using the keys `linguist-language` and `linguist-vendored` with the standard git-style path matchers for the files you want to override.
+
+Please note that the overrides currently only affect the language statistics for a repository and not the syntax-highlighting of files.
+
+```
+$ cat .gitattributes
+*.rb linguist-language=Java
+
+$ linguist --breakdown
+100.00% Java
+
+Java:
+ruby_file.rb
+```
+
+By default, Linguist treats all of the paths defined in [lib/linguist/vendor.yml](https://github.com/github/linguist/blob/master/lib/linguist/vendor.yml) as vendored and therefore doesn't include them in the language statistics for a repository. Use the `linguist-vendored` attribute to vendor or un-vendor paths.
+
+```
+$ cat .gitattributes
+special-vendored-path/* linguist-vendored
+jquery.js linguist-vendored=false
+```
+
 ## Installation
 
-github.com is usually running the latest version of the `github-linguist` gem that is released on [RubyGems.org](http://rubygems.org/gems/github-linguist).
+Github.com is usually running the latest version of the `github-linguist` gem that is released on [RubyGems.org](http://rubygems.org/gems/github-linguist).
 
 But for development you are going to want to checkout out the source. To get it, clone the repo and run [Bundler](http://gembundler.com/) to install its dependencies.
 
@@ -94,18 +147,6 @@ To run the tests:
 
     bundle exec rake test
 
-## Contributing
-
-The majority of contributions won't need to touch any Ruby code at all. The [master language list](https://github.com/github/linguist/blob/master/lib/linguist/languages.yml) is just a YAML configuration file.
-
-We try to only add languages once they have some usage on GitHub, so please note in-the-wild usage examples in your pull request.
-
-Almost all bug fixes or new language additions should come with some additional code samples. Just drop them under [`samples/`](https://github.com/github/linguist/tree/master/samples) in the correct subdirectory and our test suite will automatically test them. In most cases you shouldn't need to add any new assertions.
-
-To update the `samples.json` after adding new files to [`samples/`](https://github.com/github/linguist/tree/master/samples):
-
-    bundle exec rake samples
-
 ### A note on language extensions
 
 Linguist has a number of methods available to it for identifying the language of a particular file. The initial lookup is based upon the extension of the file, possible file extensions are defined in an array called `extensions`. Take a look at this example for example for `Perl`:
@@ -143,13 +184,13 @@ If you are the current maintainer of this gem:
  0. Make sure your local dependencies are up to date: `bundle install`
  0. Ensure that samples are updated: `bundle exec rake samples`
  0. Ensure that tests are green: `bundle exec rake test`
- 0. Bump gem version in github-linguist.gemspec.  For example, [like this](https://github.com/github/linguist/commit/97908204a385940e47251af9ecb689e8f6515c48).
- 0. Make a PR to github/linguist.  For example, [#1075](https://github.com/github/linguist/pull/1075).
- 0. Build a local gem: `gem build github-linguist.gemspec`
+ 0. Bump gem version in `lib/linguist/version.rb`.  For example, [like this](https://github.com/github/linguist/commit/8d2ea90a5ba3b2fe6e1508b7155aa4632eea2985).
+ 0. Make a PR to github/linguist.  For example, [#1238](https://github.com/github/linguist/pull/1238).
+ 0. Build a local gem: `bundle exec rake build_gem`
  0. Testing:
    0. Bump the Gemfile and Gemfile.lock versions for an app which relies on this gem
    0. Install the new gem locally
    0. Test behavior locally, branch deploy, whatever needs to happen
  0. Merge github/linguist PR
  0. Tag and push: `git tag vx.xx.xx; git push --tags`
- 0. Push to rubygems.org -- `gem push github-linguist-2.10.12.gem`
+ 0. Push to rubygems.org -- `gem push github-linguist-3.0.0.gem`

Rakefile

@@ -1,27 +1,103 @@
-require 'json'
+require 'bundler/setup'
 require 'rake/clean'
 require 'rake/testtask'
 require 'yaml'
+require 'yajl'
 
 task :default => :test
 
 Rake::TestTask.new
 
-task :samples do
-  require 'linguist/samples'
-  require 'yajl'
-  data = Linguist::Samples.data
-  json = Yajl::Encoder.encode(data, :pretty => true)
-  File.open('lib/linguist/samples.json', 'w') { |io| io.write json }
+# Extend test task to check for samples
+task :test => :check_samples
+
+desc "Check that we have samples.json generated"
+task :check_samples do
+  unless File.exist?('lib/linguist/samples.json')
+    Rake::Task[:samples].invoke
+  end
 end
 
-task :build_gem do
+task :samples do
+  require 'linguist/samples'
+  json = Yajl.dump(Linguist::Samples.data, :pretty => true)
+  File.write 'lib/linguist/samples.json', json
+end
+
+task :build_gem => :samples do
   languages = YAML.load_file("lib/linguist/languages.yml")
-  File.write("lib/linguist/languages.json", JSON.dump(languages))
+  File.write("lib/linguist/languages.json", Yajl.dump(languages))
   `gem build github-linguist.gemspec`
   File.delete("lib/linguist/languages.json")
 end
 
+task :build_grammars_gem do
+  rm_rf "grammars"
+  sh "script/download-grammars"
+  sh "gem", "build", "github-linguist-grammars.gemspec"
+end
+
+namespace :benchmark do
+  benchmark_path = "benchmark/results"
+
+  # $ bundle exec rake benchmark:generate CORPUS=path/to/samples
+  desc "Generate results for"
+  task :generate do
+    ref = `git rev-parse HEAD`.strip[0,8]
+
+    corpus = File.expand_path(ENV["CORPUS"] || "samples")
+
+    require 'linguist/language'
+
+    results = Hash.new
+    Dir.glob("#{corpus}/**/*").each do |file|
+      next unless File.file?(file)
+      filename = file.gsub("#{corpus}/", "")
+      results[filename] = Linguist::FileBlob.new(file).language
+    end
+
+    # Ensure results directory exists
+    FileUtils.mkdir_p("benchmark/results")
+
+    # Write results
+    if `git status`.include?('working directory clean')
+      result_filename = "benchmark/results/#{File.basename(corpus)}-#{ref}.json"
+    else
+      result_filename = "benchmark/results/#{File.basename(corpus)}-#{ref}-unstaged.json"
+    end
+
+    File.write(result_filename, results.to_json)
+    puts "wrote #{result_filename}"
+  end
+
+  # $ bundle exec rake benchmark:compare REFERENCE=path/to/reference.json CANDIDATE=path/to/candidate.json
+  desc "Compare results"
+  task :compare do
+    reference_file = ENV["REFERENCE"]
+    candidate_file = ENV["CANDIDATE"]
+
+    reference = Yajl.load(File.read(reference_file))
+    reference_counts = Hash.new(0)
+    reference.each { |filename, language| reference_counts[language] += 1 }
+
+    candidate = Yajl.load(File.read(candidate_file))
+    candidate_counts = Hash.new(0)
+    candidate.each { |filename, language| candidate_counts[language] += 1 }
+
+    changes = diff(reference_counts, candidate_counts)
+
+    if changes.any?
+      changes.each do |language, (before, after)|
+        before_percent = 100 * before / reference.size.to_f
+        after_percent = 100 * after / candidate.size.to_f
+        puts "%s changed from %.1f%% to %.1f%%" % [language || 'unknown', before_percent, after_percent]
+      end
+    else
+      puts "No changes"
+    end
+  end
+end
+
 namespace :classifier do
   LIMIT = 1_000
 
@@ -37,7 +113,7 @@ namespace :classifier do
       next if file_language.nil? || file_language == 'Text'
       begin
         data = open(file_url).read
-        guessed_language, score = Linguist::Classifier.classify(Linguist::Samples::DATA, data).first
+        guessed_language, score = Linguist::Classifier.classify(Linguist::Samples.cache, data).first
 
         total += 1
         guessed_language == file_language ? correct += 1 : incorrect += 1
@@ -54,14 +130,12 @@ namespace :classifier do
 
   def each_public_gist
     require 'open-uri'
-    require 'json'
-
     url = "https://api.github.com/gists/public"
 
     loop do
       resp = open(url)
       url = resp.meta['link'][/<([^>]+)>; rel="next"/, 1]
-      gists = JSON.parse(resp.read)
+      gists = Yajl.load(resp.read)
 
       for gist in gists
         for filename, attrs in gist['files']
@@ -71,3 +145,10 @@ namespace :classifier do
     end
   end
 end
+
+
+def diff(a, b)
+  (a.keys | b.keys).each_with_object({}) do |key, diff|
+    diff[key] = [a[key], b[key]] unless a[key] == b[key]
+  end
+end

bin/linguist

@@ -4,7 +4,9 @@
 #     usage: linguist <path> [<--breakdown>]
 
 require 'linguist/file_blob'
+require 'linguist/language'
 require 'linguist/repository'
+require 'rugged'
 
 path = ARGV[0] || Dir.pwd
 
@@ -18,7 +20,8 @@ ARGV.shift
 breakdown = true if ARGV[0] == "--breakdown"
 
 if File.directory?(path)
-  repo = Linguist::Repository.from_directory(path)
+  rugged = Rugged::Repository.new(path)
+  repo = Linguist::Repository.new(rugged, rugged.head.target_id)
   repo.languages.sort_by { |_, size| size }.reverse.each do |language, size|
     percentage = ((size / repo.size.to_f) * 100)
     percentage = sprintf '%.2f' % percentage
@@ -28,7 +31,7 @@ if File.directory?(path)
     puts
     file_breakdown = repo.breakdown_by_file
     file_breakdown.each do |lang, files|
-      puts "#{lang}:" 
+      puts "#{lang}:"
       files.each do |file|
         puts file
       end

github-linguist-grammars.gemspec

@@ -0,0 +1,14 @@
+require File.expand_path('../lib/linguist/version', __FILE__)
+
+Gem::Specification.new do |s|
+  s.name    = 'github-linguist-grammars'
+  s.version = Linguist::VERSION
+  s.summary = "Language grammars for use with github-linguist"
+
+  s.authors  = "GitHub"
+  s.homepage = "https://github.com/github/linguist"
+
+  s.files = ['lib/linguist/grammars.rb'] + Dir['grammars/*']
+
+  s.add_development_dependency 'plist', '~>3.1'
+end

github-linguist.gemspec

@@ -10,16 +10,16 @@ Gem::Specification.new do |s|
   s.homepage = "https://github.com/github/linguist"
   s.license  = "MIT"
 
-  s.files = Dir['lib/**/*']
+  s.files = Dir['lib/**/*'] - ['lib/linguist/grammars.rb']
   s.executables << 'linguist'
 
-  s.add_dependency 'charlock_holmes', '~> 0.7.1'
+  s.add_dependency 'charlock_holmes', '~> 0.7.3'
   s.add_dependency 'escape_utils',    '~> 1.0.1'
-  s.add_dependency 'mime-types',      '~> 1.19'
-  s.add_dependency 'pygments.rb',     '~> 0.5.4'
+  s.add_dependency 'mime-types',      '>= 1.19'
+  s.add_dependency 'rugged',          '~> 0.22.0b4'
 
-  s.add_development_dependency 'json'
   s.add_development_dependency 'mocha'
+  s.add_development_dependency 'pry'
   s.add_development_dependency 'rake'
   s.add_development_dependency 'yajl-ruby'
 end

grammars.yml

@@ -0,0 +1,422 @@
+---
+http://svn.edgewall.org/repos/genshi/contrib/textmate/Genshi.tmbundle/Syntaxes/Markup%20Template%20%28XML%29.tmLanguage:
+- text.xml.genshi
+http://svn.textmate.org/trunk/Review/Bundles/BlitzMax.tmbundle:
+- source.blitzmax
+http://svn.textmate.org/trunk/Review/Bundles/Cython.tmbundle:
+- source.cython
+http://svn.textmate.org/trunk/Review/Bundles/Forth.tmbundle:
+- source.forth
+http://svn.textmate.org/trunk/Review/Bundles/Parrot.tmbundle:
+- source.parrot.pir
+http://svn.textmate.org/trunk/Review/Bundles/Ruby%20Sass.tmbundle:
+- source.sass
+http://svn.textmate.org/trunk/Review/Bundles/SecondLife%20LSL.tmbundle:
+- source.lsl
+http://svn.textmate.org/trunk/Review/Bundles/VHDL.tmbundle:
+- source.vhdl
+http://svn.textmate.org/trunk/Review/Bundles/XQuery.tmbundle:
+- source.xquery
+https://bitbucket.org/Clams/sublimesystemverilog/get/default.tar.gz:
+- source.systemverilog
+- source.ucfconstraints
+https://bitbucket.org/bitlang/sublime_cobol/raw/b0e9c44ac5f7a2fb553421aa986b35854cbfda4a/COBOL.tmLanguage:
+- source.cobol
+https://fan.googlecode.com/hg-history/Build%201.0.55/adm/tools/textmate/Fan.tmbundle/Syntaxes/Fan.tmLanguage:
+- source.fan
+https://github.com/AlanQuatermain/go-tmbundle:
+- source.go
+https://github.com/Anomareh/PHP-Twig.tmbundle:
+- text.html.twig
+https://github.com/Cirru/sublime-cirru/raw/master/Cirru.tmLanguage:
+- source.cirru
+https://github.com/Cykey/Sublime-Logos:
+- source.logos
+https://github.com/Drako/SublimeBrainfuck/raw/master/Brainfuck.tmLanguage:
+- source.bf
+https://github.com/JohnNilsson/awk-sublime/raw/master/AWK.tmLanguage:
+- source.awk
+https://github.com/JonBons/Sublime-SQF-Language:
+- source.sqf
+https://github.com/MarioRicalde/SCSS.tmbundle:
+- source.scss
+https://github.com/Oldes/Sublime-REBOL:
+- source.rebol
+https://github.com/PogiNate/Sublime-Inform:
+- source.Inform7
+https://github.com/Red-Nova-Technologies/autoitv3-tmbundle:
+- source.autoit.3
+https://github.com/SalGnt/Sublime-VimL:
+- source.viml
+https://github.com/Shammah/boo-sublime/raw/master/Boo.tmLanguage:
+- source.boo
+https://github.com/SublimeText/ColdFusion:
+- source.cfscript
+- source.cfscript.cfc
+- text.cfml.basic
+- text.html.cfm
+https://github.com/SublimeText/NSIS:
+- source.nsis
+https://github.com/Varriount/NimLime:
+- source.nimrod
+- source.nimrod_filter
+- source.nimrodcfg
+https://github.com/alkemist/gradle.tmbundle:
+- source.groovy.gradle
+https://github.com/ambethia/Sublime-Loom:
+- source.loomscript
+https://github.com/angryant0007/VBDotNetSyntax:
+- source.vbnet
+https://github.com/anunayk/cool-tmbundle:
+- source.cool
+https://github.com/aroben/ada.tmbundle/raw/c45eed4d5f98fe3bcbbffbb9e436601ab5bbde4b/Syntaxes/Ada.plist:
+- source.ada
+https://github.com/aroben/ruby.tmbundle@4636a3023153c3034eb6ffc613899ba9cf33b41f:
+- source.ruby
+- text.html.erb
+https://github.com/asbjornenge/Docker.tmbundle:
+- source.dockerfile
+https://github.com/atom/language-clojure:
+- source.clojure
+https://github.com/atom/language-coffee-script:
+- source.coffee
+- source.litcoffee
+https://github.com/atom/language-csharp:
+- source.cs
+- source.csx
+- source.nant-build
+https://github.com/atom/language-javascript:
+- source.js
+- source.js.regexp
+https://github.com/atom/language-python:
+- source.python
+- source.regexp.python
+- text.python.traceback
+https://github.com/atom/language-shellscript:
+- source.shell
+- text.shell-session
+https://github.com/austinwagner/sublime-sourcepawn:
+- source.sp
+https://github.com/bfad/Sublime-Lasso:
+- file.lasso
+https://github.com/bholt/chapel-tmbundle:
+- source.chapel
+https://github.com/brandonwamboldt/sublime-nginx:
+- source.nginx
+https://github.com/bro/bro-sublime:
+- source.bro
+https://github.com/carsonoid/sublime_man_page_support/raw/master/man-groff.tmLanguage:
+- text.groff
+https://github.com/ccreutzig/sublime-MuPAD:
+- source.mupad
+https://github.com/cdwilson/nesC.tmbundle:
+- source.nesc
+https://github.com/christophevg/racket-tmbundle:
+- source.racket
+https://github.com/clemos/haxe-sublime-bundle:
+- source.erazor
+- source.haxe.2
+- source.hss.1
+- source.hxml
+- source.nmml
+https://github.com/cucumber/cucumber-tmbundle:
+- source.ruby.rspec.cucumber.steps
+- text.gherkin.feature
+https://github.com/daaain/Handlebars/raw/master/Handlebars.tmLanguage:
+- text.html.handlebars
+https://github.com/davidpeckham/powershell.tmbundle:
+- source.powershell
+https://github.com/davidrios/jade-tmbundle:
+- source.jade
+- source.pyjade
+https://github.com/elixir-lang/elixir-tmbundle:
+- source.elixir
+- text.elixir
+- text.html.elixir
+https://github.com/ericzou/ebundles/raw/master/Bundles/MSDOS%20batch%20file.tmbundle/Syntaxes/MSDOS%20batch%20file.tmLanguage:
+- source.dosbatch
+https://github.com/euler0/sublime-glsl/raw/master/GLSL.tmLanguage:
+- source.glsl
+https://github.com/fancy-lang/fancy-tmbundle:
+- source.fancy
+https://github.com/fsharp/fsharpbinding:
+- source.fsharp
+https://github.com/gingerbeardman/monkey.tmbundle:
+- source.monkey
+https://github.com/guillermooo/dart-sublime-bundle/raw/master/Dart.tmLanguage:
+- source.dart
+https://github.com/harrism/sublimetext-cuda-cpp/raw/master/cuda-c%2B%2B.tmLanguage:
+- source.cuda-c++
+https://github.com/hww3/pike-textmate:
+- source.pike
+https://github.com/jeancharles-roger/ceylon-sublimetext/raw/master/Ceylon.tmLanguage:
+- source.ceylon
+https://github.com/jfairbank/Sublime-Text-2-OpenEdge-ABL:
+- source.abl
+https://github.com/jhasse/sublime-rust:
+- source.rust
+https://github.com/johanasplund/sublime-befunge/raw/master/Befunge-93.tmLanguage:
+- source.befunge
+https://github.com/joshaven/RDoc.tmbundle:
+- text.rdoc
+https://github.com/jpcamara/Textmate-Gosu-Bundle/raw/master/Gosu.tmbundle/Syntaxes/Gosu.tmLanguage:
+- source.gosu.2
+https://github.com/kswedberg/jquery-tmbundle:
+- source.js.jquery
+https://github.com/laughedelic/sublime-idris/raw/master/Idris.tmLanguage:
+- source.idris
+https://github.com/lavrton/sublime-better-typescript:
+- source.ts
+https://github.com/leafo/moonscript-tmbundle:
+- source.moonscript
+https://github.com/lsf37/Isabelle.tmbundle:
+- source.isabelle.theory
+https://github.com/lunixbochs/x86-assembly-textmate-bundle:
+- source.asm.x86
+https://github.com/macekond/Alloy.tmbundle:
+- source.alloy
+https://github.com/mads379/opa.tmbundle:
+- source.opa
+https://github.com/mads379/scala.tmbundle:
+- source.sbt
+- source.scala
+https://github.com/marconi/mako-tmbundle:
+- text.html.mako
+https://github.com/mattfoster/gnuplot-tmbundle:
+- source.gnuplot
+https://github.com/mgalloy/idl.tmbundle:
+- source.idl
+- source.idl-dlm
+- text.idl-idldoc
+https://github.com/michaeledgar/protobuf-tmbundle:
+- source.protobuf
+https://github.com/mkolosick/Sublime-Coq/raw/master/Coq.tmLanguage:
+- source.coq
+https://github.com/mokus0/Agda.tmbundle:
+- source.agda
+https://github.com/nanoant/Julia.tmbundle:
+- source.julia
+https://github.com/nanoant/assembly.tmbundle/raw/master/Syntaxes/objdump%20C%2B%2B.tmLanguage:
+- objdump.x86asm
+https://github.com/nilium/ooc.tmbundle:
+- source.ooc
+https://github.com/paulmillr/LiveScript.tmbundle:
+- source.livescript
+https://github.com/pferruggiaro/sublime-tea:
+- source.tea
+https://github.com/puppet-textmate-bundle/puppet-textmate-bundle:
+- source.puppet
+https://github.com/pvl/abap.tmbundle:
+- source.abap
+https://github.com/scalate/Scalate.tmbundle:
+- source.scaml
+- text.html.ssp
+https://github.com/shadanan/mathematica-tmbundle:
+- source.mathematica
+https://github.com/shellderp/sublime-robot-plugin:
+- text.robot
+https://github.com/simongregory/actionscript3-tmbundle:
+- source.actionscript.3
+- text.html.asdoc
+- text.xml.flex-config
+https://github.com/skozlovf/Sublime-QML:
+- source.qml
+https://github.com/slash-lang/Slash.tmbundle:
+- text.html.slash
+https://github.com/slavapestov/factor/raw/master/misc/Factor.tmbundle/Syntaxes/Factor.tmLanguage:
+- source.factor
+https://github.com/slim-template/ruby-slim.tmbundle:
+- text.slim
+https://github.com/staltz/SublimeXtend:
+- source.xtend
+https://github.com/statatmbundle/Stata.tmbundle:
+- source.mata
+- source.stata
+https://github.com/technosophos/Vala-TMBundle:
+- source.vala
+https://github.com/textmate/ant.tmbundle:
+- text.xml.ant
+https://github.com/textmate/antlr.tmbundle:
+- source.antlr
+https://github.com/textmate/apache.tmbundle:
+- source.apache-config
+- source.apache-config.mod_perl
+https://github.com/textmate/applescript.tmbundle:
+- source.applescript
+https://github.com/textmate/asp.tmbundle:
+- source.asp
+- text.html.asp
+https://github.com/textmate/bison.tmbundle:
+- source.bison
+https://github.com/textmate/c.tmbundle:
+- source.c
+- source.c++
+- source.c.platform
+https://github.com/textmate/capnproto.tmbundle:
+- source.capnp
+https://github.com/textmate/cmake.tmbundle:
+- source.cache.cmake
+- source.cmake
+https://github.com/textmate/cpp-qt.tmbundle:
+- source.c++.qt
+- source.qmake
+https://github.com/textmate/css.tmbundle:
+- source.css
+https://github.com/textmate/d.tmbundle:
+- source.d
+https://github.com/textmate/diff.tmbundle:
+- source.diff
+https://github.com/textmate/dylan.tmbundle:
+- source.dylan
+- source.lid
+- source.makegen
+https://github.com/textmate/eiffel.tmbundle:
+- source.eiffel
+https://github.com/textmate/erlang.tmbundle:
+- source.erlang
+- text.html.erlang.yaws
+https://github.com/textmate/fortran.tmbundle:
+- source.fortran
+- source.fortran.modern
+https://github.com/textmate/gettext.tmbundle:
+- source.po
+https://github.com/textmate/graphviz.tmbundle:
+- source.dot
+https://github.com/textmate/groovy.tmbundle:
+- source.groovy
+https://github.com/textmate/haskell.tmbundle:
+- source.haskell
+- text.tex.latex.haskell
+https://github.com/textmate/html.tmbundle:
+- text.html.basic
+https://github.com/textmate/ini.tmbundle:
+- source.ini
+https://github.com/textmate/io.tmbundle:
+- source.io
+https://github.com/textmate/java.tmbundle:
+- source.java
+- source.java-properties
+- text.html.jsp
+- text.junit-test-report
+https://github.com/textmate/javadoc.tmbundle:
+- text.html.javadoc
+https://github.com/textmate/javascript-objective-j.tmbundle:
+- source.js.objj
+https://github.com/textmate/json.tmbundle:
+- source.json
+https://github.com/textmate/latex.tmbundle:
+- text.bibtex
+- text.log.latex
+- text.tex
+- text.tex.latex
+- text.tex.latex.beamer
+- text.tex.latex.memoir
+https://github.com/textmate/less.tmbundle:
+- source.css.less
+https://github.com/textmate/lilypond.tmbundle:
+- source.lilypond
+https://github.com/textmate/lisp.tmbundle:
+- source.lisp
+https://github.com/textmate/logtalk.tmbundle:
+- source.logtalk
+https://github.com/textmate/lua.tmbundle:
+- source.lua
+https://github.com/textmate/make.tmbundle:
+- source.makefile
+https://github.com/textmate/markdown.tmbundle:
+- text.html.markdown
+https://github.com/textmate/matlab.tmbundle:
+- source.matlab
+- source.octave
+https://github.com/textmate/maven.tmbundle:
+- text.xml.pom
+https://github.com/textmate/nemerle.tmbundle:
+- source.nemerle
+https://github.com/textmate/ninja.tmbundle:
+- source.ninja
+https://github.com/textmate/objective-c.tmbundle:
+- source.objc
+- source.objc++
+- source.objc.platform
+- source.strings
+https://github.com/textmate/ocaml.tmbundle:
+- source.camlp4.ocaml
+- source.ocaml
+- source.ocamllex
+- source.ocamlyacc
+https://github.com/textmate/pascal.tmbundle:
+- source.pascal
+https://github.com/textmate/perl.tmbundle:
+- source.perl
+https://github.com/textmate/php-smarty.tmbundle:
+- source.smarty
+https://github.com/textmate/php.tmbundle:
+- text.html.php
+https://github.com/textmate/postscript.tmbundle:
+- source.postscript
+https://github.com/textmate/processing.tmbundle:
+- source.processing
+https://github.com/textmate/prolog.tmbundle:
+- source.prolog
+https://github.com/textmate/python-django.tmbundle:
+- source.python.django
+- text.html.django
+https://github.com/textmate/r.tmbundle:
+- source.r
+- text.tex.latex.rd
+https://github.com/textmate/restructuredtext.tmbundle:
+- text.restructuredtext
+https://github.com/textmate/ruby-haml.tmbundle:
+- text.haml
+https://github.com/textmate/ruby-on-rails-tmbundle:
+- source.js.erb.rails
+- source.ruby.rails
+- source.ruby.rails.rjs
+- source.sql.ruby
+- text.html.erb.rails
+https://github.com/textmate/scheme.tmbundle:
+- source.scheme
+https://github.com/textmate/scilab.tmbundle:
+- source.scilab
+https://github.com/textmate/sql.tmbundle:
+- source.sql
+https://github.com/textmate/standard-ml.tmbundle:
+- source.cm
+- source.ml
+https://github.com/textmate/swift.tmbundle:
+- source.swift
+https://github.com/textmate/tcl.tmbundle:
+- source.tcl
+- text.html.tcl
+https://github.com/textmate/text.tmbundle:
+- text.plain
+https://github.com/textmate/textile.tmbundle:
+- text.html.textile
+https://github.com/textmate/textmate.tmbundle:
+- source.regexp.oniguruma
+- source.tm-properties
+https://github.com/textmate/thrift.tmbundle:
+- source.thrift
+https://github.com/textmate/toml.tmbundle:
+- source.toml
+https://github.com/textmate/verilog.tmbundle:
+- source.verilog
+https://github.com/textmate/xml.tmbundle:
+- text.xml
+- text.xml.xsl
+https://github.com/textmate/yaml.tmbundle:
+- source.yaml
+https://github.com/tomas-stefano/smalltalk-tmbundle:
+- source.smalltalk
+https://github.com/vic/ioke-outdated/raw/master/share/TextMate/Ioke.tmbundle/Syntaxes/Ioke.tmLanguage:
+- source.ioke
+https://github.com/vkostyukov/kotlin-sublime-package:
+- source.Kotlin
+https://github.com/vmg/zephir-sublime:
+- source.php.zephir
+https://github.com/whitequark/llvm.tmbundle:
+- source.llvm
+https://github.com/wmertens/sublime-nix:
+- source.nix
+https://raw.githubusercontent.com/eregon/oz-tmbundle/master/Syntaxes/Oz.tmLanguage:
+- source.oz

lib/linguist.rb

@@ -4,4 +4,5 @@ require 'linguist/heuristics'
 require 'linguist/language'
 require 'linguist/repository'
 require 'linguist/samples'
+require 'linguist/shebang'
 require 'linguist/version'

lib/linguist/blob_helper.rb

@@ -1,10 +1,7 @@
 require 'linguist/generated'
-require 'linguist/language'
-
 require 'charlock_holmes'
 require 'escape_utils'
 require 'mime/types'
-require 'pygments'
 require 'yaml'
 
 module Linguist
@@ -112,6 +109,12 @@ module Linguist
       end
     end
 
+    def ruby_encoding
+      if hash = detect_encoding
+        hash[:ruby_encoding]
+      end
+    end
+
     # Try to guess the encoding
     #
     # Returns: a Hash, with :encoding, :confidence, :type
@@ -143,6 +146,13 @@ module Linguist
       end
     end
 
+    # Public: Is the blob empty?
+    #
+    # Return true or false
+    def empty?
+      data.nil? || data == ""
+    end
+
     # Public: Is the blob text?
     #
     # Return true or false
@@ -189,10 +199,6 @@ module Linguist
 
     # Public: Is the blob safe to colorize?
     #
-    # We use Pygments for syntax highlighting blobs. Pygments
-    # can be too slow for very large blobs or for certain
-    # corner-case blobs.
-    #
     # Return true or false
     def safe_to_colorize?
       !large? && text? && !high_ratio_of_long_lines?
@@ -200,9 +206,6 @@ module Linguist
 
     # Internal: Does the blob have a ratio of long lines?
     #
-    # These types of files are usually going to make Pygments.rb
-    # angry if we try to colorize them.
-    #
     # Return true or false
     def high_ratio_of_long_lines?
       return false if loc == 0
@@ -256,10 +259,16 @@ module Linguist
           # without changing the encoding of `data`, and
           # also--importantly--without having to duplicate many (potentially
           # large) strings.
-          encoded_newlines = ["\r\n", "\r", "\n"].
-            map { |nl| nl.encode(encoding).force_encoding(data.encoding) }
+          begin
+            encoded_newlines = ["\r\n", "\r", "\n"].
+              map { |nl| nl.encode(ruby_encoding, "ASCII-8BIT").force_encoding(data.encoding) }
 
-          data.split(Regexp.union(encoded_newlines), -1)
+            data.split(Regexp.union(encoded_newlines), -1)
+          rescue Encoding::ConverterNotFoundError
+            # The data is not splittable in the detected encoding.  Assume it's
+            # one big line.
+            [data]
+          end
         else
           []
         end
@@ -301,34 +310,12 @@ module Linguist
     #
     # Returns a Language or nil if none is detected
     def language
-      return @language if defined? @language
-
-      if defined?(@data) && @data.is_a?(String)
-        data = @data
-      else
-        data = lambda { (binary_mime_type? || binary?) ? "" : self.data }
-      end
-
-      @language = Language.detect(name.to_s, data, mode)
+      @language ||= Language.detect(self)
     end
 
-    # Internal: Get the lexer of the blob.
-    #
-    # Returns a Lexer.
-    def lexer
-      language ? language.lexer : Pygments::Lexer.find_by_name('Text only')
-    end
-
-    # Public: Highlight syntax of blob
-    #
-    # options - A Hash of options (defaults to {})
-    #
-    # Returns html String
-    def colorize(options = {})
-      return unless safe_to_colorize?
-      options[:options] ||= {}
-      options[:options][:encoding] ||= encoding
-      lexer.highlight(data, options)
+    # Internal: Get the TextMate compatible scope for the blob
+    def tm_scope
+      language && language.tm_scope
     end
   end
 end

lib/linguist/classifier.rb

@@ -3,6 +3,25 @@ require 'linguist/tokenizer'
 module Linguist
   # Language bayesian classifier.
   class Classifier
+    # Public: Use the classifier to detect language of the blob.
+    #
+    # blob               - An object that quacks like a blob.
+    # possible_languages - Array of Language objects
+    #
+    # Examples
+    #
+    #   Classifier.call(FileBlob.new("path/to/file"), [
+    #     Language["Ruby"], Language["Python"]
+    #   ])
+    #
+    # Returns an Array of Language objects, most probable first.
+    def self.call(blob, possible_languages)
+      language_names = possible_languages.map(&:name)
+      classify(Samples.cache, blob.data, language_names).map do |name, _|
+        Language[name] # Return the actual Language objects
+      end
+    end
+
     # Public: Train classifier that data is a certain language.
     #
     # db       - Hash classifier database object

lib/linguist/file_blob.rb

@@ -52,5 +52,26 @@ module Linguist
     def size
       File.size(@path)
     end
+
+    # Public: Get file extension.
+    #
+    # Returns a String.
+    def extension
+      extensions.last || ""
+    end
+
+    # Public: Return an array of the file extensions
+    #
+    #     >> Linguist::FileBlob.new("app/views/things/index.html.erb").extensions
+    #     => [".html.erb", ".erb"]
+    #
+    # Returns an Array
+    def extensions
+      basename, *segments = File.basename(name).split(".")
+
+      segments.map.with_index do |segment, index|
+        "." + segments[index..-1].join(".")
+      end
+    end
   end
 end

lib/linguist/generated.rb

@@ -51,29 +51,30 @@ module Linguist
     #
     # Return true or false
     def generated?
-      name == 'Gemfile.lock' ||
-        minified_files? ||
-        compiled_coffeescript? ||
-        xcode_project_file? ||
-        generated_parser? ||
-        generated_net_docfile? ||
-        generated_net_designer_file? ||
-        generated_postscript? ||
-        generated_protocol_buffer? ||
-        generated_jni_header? ||
-        composer_lock? ||
-        node_modules? ||
-        vcr_cassette?
+      minified_files? ||
+      compiled_coffeescript? ||
+      xcode_file? ||
+      generated_parser? ||
+      generated_net_docfile? ||
+      generated_net_designer_file? ||
+      generated_postscript? ||
+      generated_protocol_buffer? ||
+      generated_jni_header? ||
+      composer_lock? ||
+      node_modules? ||
+      godeps? ||
+      vcr_cassette? ||
+      generated_by_zephir?
     end
 
-    # Internal: Is the blob an XCode project file?
+    # Internal: Is the blob an Xcode file?
     #
-    # Generated if the file extension is an XCode project
+    # Generated if the file extension is an Xcode
     # file extension.
     #
     # Returns true of false.
-    def xcode_project_file?
-      ['.xib', '.nib', '.storyboard', '.pbxproj', '.xcworkspacedata', '.xcuserstate'].include?(extname)
+    def xcode_file?
+      ['.nib', '.xcworkspacedata', '.xcuserstate'].include?(extname)
     end
 
     # Internal: Is the blob minified files?
@@ -230,11 +231,26 @@ module Linguist
       !!name.match(/node_modules\//)
     end
 
+    # Internal: Is the blob part of Godeps/,
+    # which are not meant for humans in pull requests.
+    #
+    # Returns true or false.
+    def godeps?
+      !!name.match(/Godeps\//)
+    end
+
     # Internal: Is the blob a generated php composer lock file?
     #
     # Returns true or false.
     def composer_lock?
-      !!name.match(/composer.lock/)
+      !!name.match(/composer\.lock/)
+    end
+
+    # Internal: Is the blob a generated by Zephir
+    #
+    # Returns true or false.
+    def generated_by_zephir?
+      !!name.match(/.\.zep\.(?:c|h|php)$/)
     end
 
     # Is the blob a VCR Cassette file?

lib/linguist/grammars.rb

@@ -0,0 +1,13 @@
+# Note: This file is included in the github-linguist-grammars gem, not the
+# github-linguist gem.
+
+module Linguist
+  module Grammars
+    # Get the path to the directory containing the language grammar JSON files.
+    #
+    # Returns a String.
+    def self.path
+      File.expand_path("../../../grammars", __FILE__)
+    end
+  end
+end

lib/linguist/heuristics.rb

@@ -1,90 +1,160 @@
 module Linguist
   # A collection of simple heuristics that can be used to better analyze languages.
   class Heuristics
-    ACTIVE = false
+    # Public: Use heuristics to detect language of the blob.
+    #
+    # blob               - An object that quacks like a blob.
+    # possible_languages - Array of Language objects
+    #
+    # Examples
+    #
+    #   Heuristics.call(FileBlob.new("path/to/file"), [
+    #     Language["Ruby"], Language["Python"]
+    #   ])
+    #
+    # Returns an Array of languages, or empty if none matched or were inconclusive.
+    def self.call(blob, languages)
+      data = blob.data
 
-    # Public: Given an array of String language names,
-    # apply heuristics against the given data and return an array
-    # of matching languages, or nil.
+      @heuristics.each do |heuristic|
+        return Array(heuristic.call(data)) if heuristic.matches?(languages)
+      end
+
+      [] # No heuristics matched
+    end
+
+    # Internal: Define a new heuristic.
     #
-    # data      - Array of tokens or String data to analyze.
-    # languages - Array of language name Strings to restrict to.
+    # languages - String names of languages to disambiguate.
+    # heuristic - Block which takes data as an argument and returns a Language or nil.
     #
-    # Returns an array of Languages or []
-    def self.find_by_heuristics(data, languages)
-      if active?
-        if languages.all? { |l| ["Objective-C", "C++"].include?(l) }
-          disambiguate_c(data, languages)
-        end
-        if languages.all? { |l| ["Perl", "Prolog"].include?(l) }
-          disambiguate_pl(data, languages)
-        end
-        if languages.all? { |l| ["ECL", "Prolog"].include?(l) }
-          disambiguate_ecl(data, languages)
-        end
-        if languages.all? { |l| ["TypeScript", "XML"].include?(l) }
-          disambiguate_ts(data, languages)
-        end
-        if languages.all? { |l| ["Common Lisp", "OpenCL"].include?(l) }
-          disambiguate_cl(data, languages)
-        end
-        if languages.all? { |l| ["Rebol", "R"].include?(l) }
-          disambiguate_r(data, languages)
-        end
+    # Examples
+    #
+    #     disambiguate "Perl", "Prolog" do |data|
+    #       if data.include?("use strict")
+    #         Language["Perl"]
+    #       elsif data.include?(":-")
+    #         Language["Prolog"]
+    #       end
+    #     end
+    #
+    def self.disambiguate(*languages, &heuristic)
+      @heuristics << new(languages, &heuristic)
+    end
+
+    # Internal: Array of defined heuristics
+    @heuristics = []
+
+    # Internal
+    def initialize(languages, &heuristic)
+      @languages = languages
+      @heuristic = heuristic
+    end
+
+    # Internal: Check if this heuristic matches the candidate languages.
+    def matches?(candidates)
+      candidates.any? && candidates.all? { |l| @languages.include?(l.name) }
+    end
+
+    # Internal: Perform the heuristic
+    def call(data)
+      @heuristic.call(data)
+    end
+
+    disambiguate "Objective-C", "C++", "C" do |data|
+      if (/@(interface|class|protocol|property|end|synchronised|selector|implementation)\b/.match(data))
+        Language["Objective-C"]
+      elsif (/^\s*#\s*include <(cstdint|string|vector|map|list|array|bitset|queue|stack|forward_list|unordered_map|unordered_set|(i|o|io)stream)>/.match(data) ||
+        /^\s*template\s*</.match(data) || /^[^@]class\s+\w+/.match(data) || /^[^@](private|public|protected):$/.match(data) || /std::.+$/.match(data))
+        Language["C++"]
       end
     end
 
-    # .h extensions are ambigious between C, C++, and Objective-C.
-    # We want to shortcut look for Objective-C _and_ now C++ too!
-    #
-    # Returns an array of Languages or []
-    def self.disambiguate_c(data, languages)
-      matches = []
-      matches << Language["Objective-C"] if data.include?("@interface")
-      matches << Language["C++"] if data.include?("#include <cstdint>")
-      matches
+    disambiguate "Perl", "Perl6", "Prolog" do |data|
+      if data.include?("use v6")
+        Language["Perl6"]
+      elsif data.include?("use strict")
+        Language["Perl"]
+      elsif data.include?(":-")
+        Language["Prolog"]
+      end
     end
 
-    def self.disambiguate_pl(data, languages)
-      matches = []
-      matches << Language["Prolog"] if data.include?(":-")
-      matches << Language["Perl"] if data.include?("use strict")
-      matches
+    disambiguate "ECL", "Prolog" do |data|
+      if data.include?(":-")
+        Language["Prolog"]
+      elsif data.include?(":=")
+        Language["ECL"]
+      end
     end
 
-    def self.disambiguate_ecl(data, languages)
-      matches = []
-      matches << Language["Prolog"] if data.include?(":-")
-      matches << Language["ECL"] if data.include?(":=")
-      matches
-    end
-
-    def self.disambiguate_ts(data, languages)
-      matches = []
-      if (data.include?("</translation>"))
-        matches << Language["XML"]
+    disambiguate "IDL", "Prolog" do |data|
+      if data.include?(":-")
+        Language["Prolog"]
       else
-        matches << Language["TypeScript"]
+        Language["IDL"]
       end
-      matches
     end
 
-    def self.disambiguate_cl(data, languages)
-      matches = []
-      matches << Language["Common Lisp"] if data.include?("(defun ")
-      matches << Language["OpenCL"] if /\/\* |\/\/ |^\}/.match(data)
-      matches
+    disambiguate "Common Lisp", "OpenCL", "Cool" do |data|
+      if data.include?("(defun ")
+        Language["Common Lisp"]
+      elsif /^class/x.match(data)
+        Language["Cool"]
+      elsif /\/\* |\/\/ |^\}/.match(data)
+        Language["OpenCL"]
+      end
     end
 
-    def self.disambiguate_r(data, languages)
-      matches = []
-      matches << Language["Rebol"] if /\bRebol\b/i.match(data)
-      matches << Language["R"] if data.include?("<-")
-      matches
+    disambiguate "Hack", "PHP" do |data|
+      if data.include?("<?hh")
+        Language["Hack"]
+      elsif /<?[^h]/.match(data)
+        Language["PHP"]
+      end
     end
 
-    def self.active?
-      !!ACTIVE
+    disambiguate "Scala", "SuperCollider" do |data|
+      if /\^(this|super)\./.match(data) || /^\s*(\+|\*)\s*\w+\s*{/.match(data) || /^\s*~\w+\s*=\./.match(data)
+        Language["SuperCollider"]
+      elsif /^\s*import (scala|java)\./.match(data) || /^\s*val\s+\w+\s*=/.match(data) || /^\s*class\b/.match(data)
+        Language["Scala"]
+      end
     end
+
+    disambiguate "AsciiDoc", "AGS Script" do |data|
+      Language["AsciiDoc"] if /^=+(\s|\n)/.match(data)
+    end
+
+    disambiguate "FORTRAN", "Forth" do |data|
+      if /^: /.match(data)
+        Language["Forth"]
+      elsif /^([c*][^a-z]|      subroutine\s)/i.match(data)
+        Language["FORTRAN"]
+      end
+    end
+
+    disambiguate "F#", "Forth", "GLSL" do |data|
+      if /^(: |new-device)/.match(data)
+        Language["Forth"]
+      elsif /^\s*(#light|import|let|module|namespace|open|type)/.match(data)
+        Language["F#"]
+      elsif /^\s*(#include|#pragma|precision|uniform|varying|void)/.match(data)
+        Language["GLSL"]
+      end
+    end
+
+    disambiguate "Gosu", "JavaScript" do |data|
+      Language["Gosu"] if /^uses java\./.match(data)
+    end
+
+    disambiguate "LoomScript", "LiveScript" do |data|
+      if /^\s*package\s*[\w\.\/\*\s]*\s*{/.match(data)
+        Language["LoomScript"]
+      else
+        Language["LiveScript"]
+      end
+    end
+
   end
 end

lib/linguist/language.rb

@@ -1,14 +1,17 @@
 require 'escape_utils'
-require 'pygments'
 require 'yaml'
 begin
-  require 'json'
+  require 'yajl'
 rescue LoadError
 end
 
 require 'linguist/classifier'
 require 'linguist/heuristics'
 require 'linguist/samples'
+require 'linguist/file_blob'
+require 'linguist/blob_helper'
+require 'linguist/strategy/filename'
+require 'linguist/shebang'
 
 module Linguist
   # Language names that are recognizable by GitHub. Defined languages
@@ -60,7 +63,7 @@ module Linguist
       end
 
       # Language name index
-      @index[language.name] = @name_index[language.name] = language
+      @index[language.name.downcase] = @name_index[language.name.downcase] = language
 
       language.aliases.each do |name|
         # All Language aliases should be unique. Raise if there is a duplicate.
@@ -68,7 +71,7 @@ module Linguist
           raise ArgumentError, "Duplicate alias: #{name}"
         end
 
-        @index[name] = @alias_index[name] = language
+        @index[name.downcase] = @alias_index[name.downcase] = language
       end
 
       language.extensions.each do |extension|
@@ -90,51 +93,36 @@ module Linguist
       language
     end
 
+    STRATEGIES = [
+      Linguist::Strategy::Filename,
+      Linguist::Shebang,
+      Linguist::Heuristics,
+      Linguist::Classifier
+    ]
+
     # Public: Detects the Language of the blob.
     #
-    # name - String filename
-    # data - String blob data. A block also maybe passed in for lazy
-    #        loading. This behavior is deprecated and you should always
-    #        pass in a String.
-    # mode - Optional String mode (defaults to nil)
+    # blob - an object that includes the Linguist `BlobHelper` interface;
+    #       see Linguist::LazyBlob and Linguist::FileBlob for examples
     #
     # Returns Language or nil.
-    def self.detect(name, data, mode = nil)
-      # A bit of an elegant hack. If the file is executable but extensionless,
-      # append a "magic" extension so it can be classified with other
-      # languages that have shebang scripts.
-      if File.extname(name).empty? && mode && (mode.to_i(8) & 05) == 05
-        name += ".script!"
-      end
+    def self.detect(blob)
+      # Bail early if the blob is binary or empty.
+      return nil if blob.likely_binary? || blob.binary? || blob.empty?
 
-      # First try to find languages that match based on filename.
-      possible_languages = find_by_filename(name)
-
-      # If there is more than one possible language with that extension (or no
-      # extension at all, in the case of extensionless scripts), we need to continue
-      # our detection work
-      if possible_languages.length > 1
-        data = data.call() if data.respond_to?(:call)
-        possible_language_names = possible_languages.map(&:name)
-
-        # Don't bother with emptiness
-        if data.nil? || data == ""
-          nil
-        # Check if there's a shebang line and use that as authoritative
-        elsif (result = find_by_shebang(data)) && !result.empty?
-          result.first
-        # No shebang. Still more work to do. Try to find it with our heuristics.
-        elsif (determined = Heuristics.find_by_heuristics(data, possible_language_names)) && !determined.empty?
-          determined.first
-        # Lastly, fall back to the probablistic classifier.
-        elsif classified = Classifier.classify(Samples::DATA, data, possible_language_names ).first
-          # Return the actual Language object based of the string language name (i.e., first element of `#classify`)
-          Language[classified[0]]
+      # Call each strategy until one candidate is returned.
+      STRATEGIES.reduce([]) do |languages, strategy|
+        candidates = strategy.call(blob, languages)
+        if candidates.size == 1
+          return candidates.first
+        elsif candidates.size > 1
+          # More than one candidate was found, pass them to the next strategy.
+          candidates
+        else
+          # No candiates were found, pass on languages from the previous strategy.
+          languages
         end
-      else
-        # Simplest and most common case, we can just return the one match based on extension
-        possible_languages.first
-      end
+      end.first
     end
 
     # Public: Get all Languages
@@ -155,7 +143,7 @@ module Linguist
     #
     # Returns the Language or nil if none was found.
     def self.find_by_name(name)
-      @name_index[name]
+      name && @name_index[name.downcase]
     end
 
     # Public: Look up Language by one of its aliases.
@@ -169,7 +157,7 @@ module Linguist
     #
     # Returns the Lexer or nil if none was found.
     def self.find_by_alias(name)
-      @alias_index[name]
+      name && @alias_index[name.downcase]
     end
 
     # Public: Look up Languages by filename.
@@ -183,26 +171,54 @@ module Linguist
     #
     # Returns all matching Languages or [] if none were found.
     def self.find_by_filename(filename)
-      basename, extname = File.basename(filename), File.extname(filename)
-      langs = @filename_index[basename] +
-              @extension_index[extname]
-      langs.compact.uniq
+      basename = File.basename(filename)
+
+      # find the first extension with language definitions
+      extname = FileBlob.new(filename).extensions.detect do |e|
+        !@extension_index[e].empty?
+      end
+
+      (@filename_index[basename] + @extension_index[extname]).compact.uniq
     end
 
-    # Public: Look up Languages by shebang line.
+    # Public: Look up Languages by file extension.
     #
-    # data - Array of tokens or String data to analyze.
+    # extname - The extension String.
     #
     # Examples
     #
-    #   Language.find_by_shebang("#!/bin/bash\ndate;")
+    #   Language.find_by_extension('.rb')
+    #   # => [#<Language name="Ruby">]
+    #
+    #   Language.find_by_extension('rb')
+    #   # => [#<Language name="Ruby">]
+    #
+    # Returns all matching Languages or [] if none were found.
+    def self.find_by_extension(extname)
+      extname = ".#{extname}" unless extname.start_with?(".")
+      @extension_index[extname]
+    end
+
+    # DEPRECATED
+    def self.find_by_shebang(data)
+      @interpreter_index[Shebang.interpreter(data)]
+    end
+
+    # Public: Look up Languages by interpreter.
+    #
+    # interpreter - String of interpreter name
+    #
+    # Examples
+    #
+    #   Language.find_by_interpreter("bash")
     #   # => [#<Language name="Bash">]
     #
     # Returns the matching Language
-    def self.find_by_shebang(data)
-      @interpreter_index[Linguist.interpreter_from_shebang(data)]
+    def self.find_by_interpreter(interpreter)
+      @interpreter_index[interpreter]
     end
 
+
     # Public: Look up Language by its name or lexer.
     #
     # name - The String name of the Language
@@ -217,7 +233,7 @@ module Linguist
     #
     # Returns the Language or nil if none was found.
     def self.[](name)
-      @index[name]
+      name && @index[name.downcase]
     end
 
     # Public: A List of popular languages
@@ -276,9 +292,16 @@ module Linguist
       # Set aliases
       @aliases = [default_alias_name] + (attributes[:aliases] || [])
 
-      # Lookup Lexer object
-      @lexer = Pygments::Lexer.find_by_name(attributes[:lexer] || name) ||
-        raise(ArgumentError, "#{@name} is missing lexer")
+      # Load the TextMate scope name or try to guess one
+      @tm_scope = attributes[:tm_scope] || begin
+        context = case @type
+                  when :data, :markup, :prose
+                    'text'
+                  when :programming, nil
+                    'source'
+                  end
+        "#{context}.#{@name.downcase}"
+      end
 
       @ace_mode = attributes[:ace_mode]
       @wrap = attributes[:wrap] || false
@@ -353,6 +376,11 @@ module Linguist
     # Returns the Lexer
     attr_reader :lexer
 
+    # Public: Get the name of a TextMate-compatible scope
+    #
+    # Returns the scope
+    attr_reader :tm_scope
+
     # Public: Get Ace mode
     #
     # Examples
@@ -395,11 +423,6 @@ module Linguist
     #
     # Returns the extensions Array
     attr_reader :filenames
-    
-    # Public: Return all possible extensions for language
-    def all_extensions
-      (extensions + [primary_extension]).uniq
-    end
 
     # Deprecated: Get primary extension
     #
@@ -500,16 +523,16 @@ module Linguist
     end
   end
 
-  extensions = Samples::DATA['extnames']
-  interpreters = Samples::DATA['interpreters']
-  filenames = Samples::DATA['filenames']
+  extensions = Samples.cache['extnames']
+  interpreters = Samples.cache['interpreters']
+  filenames = Samples.cache['filenames']
   popular = YAML.load_file(File.expand_path("../popular.yml", __FILE__))
 
   languages_yml = File.expand_path("../languages.yml", __FILE__)
   languages_json = File.expand_path("../languages.json", __FILE__)
 
-  if File.exist?(languages_json) && defined?(JSON)
-    languages = JSON.load(File.read(languages_json))
+  if File.exist?(languages_json) && defined?(Yajl)
+    languages = Yajl.load(File.read(languages_json))
   else
     languages = YAML.load_file(languages_yml)
   end
@@ -522,6 +545,7 @@ module Linguist
     if extnames = extensions[name]
       extnames.each do |extname|
         if !options['extensions'].include?(extname)
+          warn "#{name} has a sample with extension (#{extname}) that isn't explicitly defined in languages.yml" unless extname == '.script!'
           options['extensions'] << extname
         end
       end
@@ -553,12 +577,13 @@ module Linguist
       :type              => options['type'],
       :aliases           => options['aliases'],
       :lexer             => options['lexer'],
+      :tm_scope          => options['tm_scope'],
       :ace_mode          => options['ace_mode'],
       :wrap              => options['wrap'],
       :group_name        => options['group'],
-      :searchable        => options.key?('searchable') ? options['searchable'] : true,
+      :searchable        => options.fetch('searchable', true),
       :search_term       => options['search_term'],
-      :extensions        => [options['extensions'].first] + options['extensions'][1..-1].sort,
+      :extensions        => Array(options['extensions']),
       :interpreters      => options['interpreters'].sort,
       :filenames         => options['filenames'],
       :popular           => popular.include?(name)

lib/linguist/lazy_blob.rb

@@ -0,0 +1,71 @@
+require 'linguist/blob_helper'
+require 'linguist/language'
+require 'rugged'
+
+module Linguist
+  class LazyBlob
+    GIT_ATTR = ['linguist-language', 'linguist-vendored']
+    GIT_ATTR_OPTS = { :priority => [:index], :skip_system => true }
+    GIT_ATTR_FLAGS = Rugged::Repository::Attributes.parse_opts(GIT_ATTR_OPTS)
+
+    include BlobHelper
+
+    MAX_SIZE = 128 * 1024
+
+    attr_reader :repository
+    attr_reader :oid
+    attr_reader :name
+    attr_reader :mode
+
+    def initialize(repo, oid, name, mode = nil)
+      @repository = repo
+      @oid = oid
+      @name = name
+      @mode = mode
+    end
+
+    def git_attributes
+      @git_attributes ||= repository.fetch_attributes(
+        name, GIT_ATTR, GIT_ATTR_FLAGS)
+    end
+
+    def vendored?
+      if attr = git_attributes['linguist-vendored']
+        return boolean_attribute(attr)
+      else
+        return super
+      end
+    end
+
+    def language
+      return @language if defined?(@language)
+
+      @language = if lang = git_attributes['linguist-language']
+        Language.find_by_name(lang)
+      else
+        super
+      end
+    end
+
+    def data
+      load_blob!
+      @data
+    end
+
+    def size
+      load_blob!
+      @size
+    end
+
+    protected
+
+    # Returns true if the attribute is present and not the string "false".
+    def boolean_attribute(attr)
+      attr != "false"
+    end
+
+    def load_blob!
+      @data, @size = Rugged::Blob.to_buffer(repository, oid, MAX_SIZE) if @data.nil?
+    end
+  end
+end

lib/linguist/repository.rb

@@ -1,4 +1,5 @@
-require 'linguist/file_blob'
+require 'linguist/lazy_blob'
+require 'rugged'
 
 module Linguist
   # A Repository is an abstraction of a Grit::Repo or a basic file
@@ -7,100 +8,165 @@ module Linguist
   # Its primary purpose is for gathering language statistics across
   # the entire project.
   class Repository
-    # Public: Initialize a new Repository from a File directory
-    #
-    # base_path - A path String
-    #
-    # Returns a Repository
-    def self.from_directory(base_path)
-      new Dir["#{base_path}/**/*"].
-        select { |f| File.file?(f) }.
-        map { |path| FileBlob.new(path, base_path) }
+    attr_reader :repository
+
+    # Public: Create a new Repository based on the stats of
+    # an existing one
+    def self.incremental(repo, commit_oid, old_commit_oid, old_stats)
+      repo = self.new(repo, commit_oid)
+      repo.load_existing_stats(old_commit_oid, old_stats)
+      repo
     end
 
-    # Public: Initialize a new Repository
+    # Public: Initialize a new Repository to be analyzed for language
+    # data
     #
-    # enum - Enumerator that responds to `each` and
-    #        yields Blob objects
+    # repo - a Rugged::Repository object
+    # commit_oid - the sha1 of the commit that will be analyzed;
+    #              this is usually the master branch
     #
     # Returns a Repository
-    def initialize(enum)
-      @enum = enum
-      @computed_stats = false
-      @language = @size = nil
-      @sizes = Hash.new { 0 }
-      @file_breakdown = Hash.new { |h,k| h[k] = Array.new }
+    def initialize(repo, commit_oid)
+      @repository = repo
+      @commit_oid = commit_oid
+
+      raise TypeError, 'commit_oid must be a commit SHA1' unless commit_oid.is_a?(String)
+    end
+
+    # Public: Load the results of a previous analysis on this repository
+    # to speed up the new scan.
+    #
+    # The new analysis will be performed incrementally as to only take
+    # into account the file changes since the last time the repository
+    # was scanned
+    #
+    # old_commit_oid - the sha1 of the commit that was previously analyzed
+    # old_stats - the result of the previous analysis, obtained by calling
+    #             Repository#cache on the old repository
+    #
+    # Returns nothing
+    def load_existing_stats(old_commit_oid, old_stats)
+      @old_commit_oid = old_commit_oid
+      @old_stats = old_stats
+      nil
     end
 
     # Public: Returns a breakdown of language stats.
     #
     # Examples
     #
-    #   # => { Language['Ruby'] => 46319,
-    #          Language['JavaScript'] => 258 }
+    #   # => { 'Ruby' => 46319,
+    #          'JavaScript' => 258 }
     #
-    # Returns a Hash of Language keys and Integer size values.
+    # Returns a Hash of language names and Integer size values.
     def languages
-      compute_stats
-      @sizes
+      @sizes ||= begin
+        sizes = Hash.new { 0 }
+        cache.each do |_, (language, size)|
+          sizes[language] += size
+        end
+        sizes
+      end
     end
 
     # Public: Get primary Language of repository.
     #
-    # Returns a Language
+    # Returns a language name
     def language
-      compute_stats
-      @language
+      @language ||= begin
+        primary = languages.max_by { |(_, size)| size }
+        primary && primary[0]
+      end
     end
 
     # Public: Get the total size of the repository.
     #
     # Returns a byte size Integer
     def size
-      compute_stats
-      @size
+      @size ||= languages.inject(0) { |s,(_,v)| s + v }
     end
 
     # Public: Return the language breakdown of this repository by file
+    #
+    # Returns a map of language names => [filenames...]
     def breakdown_by_file
-      compute_stats
-      @file_breakdown
+      @file_breakdown ||= begin
+        breakdown = Hash.new { |h,k| h[k] = Array.new }
+        cache.each do |filename, (language, _)|
+          breakdown[language] << filename
+        end
+        breakdown
+      end
     end
 
-    # Internal: Compute language breakdown for each blob in the Repository.
+    # Public: Return the cached results of the analysis
     #
-    # Returns nothing
-    def compute_stats
-      return if @computed_stats
+    # This is a per-file breakdown that can be passed to other instances
+    # of Linguist::Repository to perform incremental scans
+    #
+    # Returns a map of filename => [language, size]
+    def cache
+      @cache ||= begin
+        if @old_commit_oid == @commit_oid
+          @old_stats
+        else
+          compute_stats(@old_commit_oid, @old_stats)
+        end
+      end
+    end
 
-      @enum.each do |blob|
-        # Skip files that are likely binary
-        next if blob.likely_binary?
+    def read_index
+      attr_index = Rugged::Index.new
+      attr_index.read_tree(current_tree)
+      repository.index = attr_index
+    end
 
-        # Skip vendored or generated blobs
-        next if blob.vendored? || blob.generated? || blob.language.nil?
+    def current_tree
+      @tree ||= Rugged::Commit.lookup(repository, @commit_oid).tree
+    end
 
-        # Only include programming languages and acceptable markup languages
-        if blob.language.type == :programming || Language.detectable_markup.include?(blob.language.name)
+    protected
 
-          # Build up the per-file breakdown stats
-          @file_breakdown[blob.language.group.name] << blob.name
+    def compute_stats(old_commit_oid, cache = nil)
+      old_tree = old_commit_oid && Rugged::Commit.lookup(repository, old_commit_oid).tree
 
-          @sizes[blob.language.group] += blob.size
+      read_index
+
+      diff = Rugged::Tree.diff(repository, old_tree, current_tree)
+
+      # Clear file map and fetch full diff if any .gitattributes files are changed
+      if cache && diff.each_delta.any? { |delta| File.basename(delta.new_file[:path]) == ".gitattributes" }
+        diff = Rugged::Tree.diff(repository, old_tree = nil, current_tree)
+        file_map = {}
+      else
+        file_map = cache ? cache.dup : {}
+      end
+
+      diff.each_delta do |delta|
+        old = delta.old_file[:path]
+        new = delta.new_file[:path]
+
+        file_map.delete(old)
+        next if delta.binary
+
+        if [:added, :modified].include? delta.status
+          # Skip submodules
+          mode = delta.new_file[:mode]
+          next if (mode & 040000) != 0
+
+          blob = Linguist::LazyBlob.new(repository, delta.new_file[:oid], new, mode.to_s(8))
+
+          # Skip vendored or generated blobs
+          next if blob.vendored? || blob.generated? || blob.language.nil?
+
+          # Only include programming languages and acceptable markup languages
+          if blob.language.type == :programming || Language.detectable_markup.include?(blob.language.name)
+            file_map[new] = [blob.language.group.name, blob.size]
+          end
         end
       end
 
-      # Compute total size
-      @size = @sizes.inject(0) { |s,(_,v)| s + v }
-
-      # Get primary language
-      if primary = @sizes.max_by { |(_, size)| size }
-        @language = primary[0]
-      end
-
-      @computed_stats = true
-
-      nil
+      file_map
     end
   end
 end

lib/linguist/samples.rb

@@ -1,11 +1,12 @@
 begin
-  require 'json'
+  require 'yajl'
 rescue LoadError
   require 'yaml'
 end
 
 require 'linguist/md5'
 require 'linguist/classifier'
+require 'linguist/shebang'
 
 module Linguist
   # Model for accessing classifier training data.
@@ -17,9 +18,11 @@ module Linguist
     PATH = File.expand_path('../samples.json', __FILE__)
 
     # Hash of serialized samples object
-    if File.exist?(PATH)
-      serializer = defined?(JSON) ? JSON : YAML
-      DATA = serializer.load(File.read(PATH))
+    def self.cache
+      @cache ||= begin
+        serializer = defined?(Yajl) ? Yajl : YAML
+        serializer.load(File.read(PATH))
+      end
     end
 
     # Public: Iterate over each sample.
@@ -28,7 +31,7 @@ module Linguist
     #
     # Returns nothing.
     def self.each(&block)
-      Dir.entries(ROOT).each do |category|
+      Dir.entries(ROOT).sort!.each do |category|
         next if category == '.' || category == '..'
 
         # Skip text and binary for now
@@ -50,14 +53,16 @@ module Linguist
               })
             end
           else
+            path = File.join(dirname, filename)
+
             if File.extname(filename) == ""
-              raise "#{File.join(dirname, filename)} is missing an extension, maybe it belongs in filenames/ subdir"
+              raise "#{path} is missing an extension, maybe it belongs in filenames/ subdir"
             end
 
             yield({
-              :path     => File.join(dirname, filename),
+              :path     => path,
               :language => category,
-              :interpreter => File.exist?(filename) ? Linguist.interpreter_from_shebang(File.read(filename)) : nil,
+              :interpreter => Shebang.interpreter(File.read(path)),
               :extname  => File.extname(filename)
             })
           end
@@ -110,40 +115,4 @@ module Linguist
       db
     end
   end
-
-  # Used to retrieve the interpreter from the shebang line of a file's
-  # data.
-  def self.interpreter_from_shebang(data)
-    lines = data.lines.to_a
-
-    if lines.any? && (match = lines[0].match(/(.+)\n?/)) && (bang = match[0]) =~ /^#!/
-      bang.sub!(/^#! /, '#!')
-      tokens = bang.split(' ')
-      pieces = tokens.first.split('/')
-
-      if pieces.size > 1
-        script = pieces.last
-      else
-        script = pieces.first.sub('#!', '')
-      end
-
-      script = script == 'env' ? tokens[1] : script
-
-      # "python2.6" -> "python"
-      if script =~ /((?:\d+\.?)+)/
-        script.sub! $1, ''
-      end
-
-      # Check for multiline shebang hacks that call `exec`
-      if script == 'sh' &&
-        lines[0...5].any? { |l| l.match(/exec (\w+).+\$0.+\$@/) }
-        script = $1
-      end
-
-      script
-    else
-      nil
-    end
-  end
-
 end

lib/linguist/shebang.rb

@@ -0,0 +1,44 @@
+module Linguist
+  class Shebang
+    # Public: Use shebang to detect language of the blob.
+    #
+    # blob               - An object that quacks like a blob.
+    #
+    # Examples
+    #
+    #   Shebang.call(FileBlob.new("path/to/file"))
+    #
+    # Returns an Array with one Language if the blob has a shebang with a valid
+    # interpreter, or empty if there is no shebang.
+    def self.call(blob, _ = nil)
+      Language.find_by_interpreter interpreter(blob.data)
+    end
+
+    # Public: Get the interpreter from the shebang
+    #
+    # Returns a String or nil
+    def self.interpreter(data)
+      lines = data.lines
+      return unless match = /^#! ?(.+)$/.match(lines.first)
+
+      tokens = match[1].split(' ')
+      script = tokens.first.split('/').last
+
+      script = tokens[1] if script == 'env'
+
+      # If script has an invalid shebang, we might get here
+      return unless script
+
+      # "python2.6" -> "python2"
+      script.sub! $1, '' if script =~ /(\.\d+)$/
+
+      # Check for multiline shebang hacks that call `exec`
+      if script == 'sh' &&
+        lines.first(5).any? { |l| l.match(/exec (\w+).+\$0.+\$@/) }
+        script = $1
+      end
+
+      File.basename(script)
+    end
+  end
+end

lib/linguist/strategy/filename.rb

@@ -0,0 +1,20 @@
+module Linguist
+  module Strategy
+    # Detects language based on filename and/or extension
+    class Filename
+      def self.call(blob, _)
+        name = blob.name.to_s
+
+        # A bit of an elegant hack. If the file is executable but extensionless,
+        # append a "magic" extension so it can be classified with other
+        # languages that have shebang scripts.
+        extensions = FileBlob.new(name).extensions
+        if extensions.empty? && blob.mode && (blob.mode.to_i(8) & 05) == 05
+          name += ".script!"
+        end
+
+        Language.find_by_filename(name)
+      end
+    end
+  end
+end

lib/linguist/vendor.yml

@@ -33,16 +33,36 @@
 # Erlang bundles
 - ^rebar$
 
-# Bootstrap minified css and js
-- (^|/)bootstrap([^.]*)(\.min)?\.(js|css)$
+# Go dependencies
+- Godeps/_workspace/
+
+# Minified JavaScript and CSS
+- (\.|-)min\.(js|css)$
+
+# Bootstrap css and js
+- (^|/)bootstrap([^.]*)\.(js|css)$
+
+# Font Awesome
+- font-awesome.css
 
 # Foundation css
-- foundation.min.css
 - foundation.css
 
+# Normalize.css
+- normalize.css
+
+# Bourbon SCSS
+- (^|/)[Bb]ourbon/.*\.css$
+- (^|/)[Bb]ourbon/.*\.scss$
+
+# Animate.css
+- animate.css
+
 # Vendored dependencies
-- thirdparty/
+- third[-_]?party/
+- 3rd[-_]?party/
 - vendors?/
+- extern(al)?/
 
 # Debian packaging
 - ^debian/
@@ -53,12 +73,12 @@
 ## Commonly Bundled JavaScript frameworks ##
 
 # jQuery
-- (^|/)jquery([^.]*)(\.min)?\.js$
-- (^|/)jquery\-\d\.\d+(\.\d+)?(\.min)?\.js$
+- (^|/)jquery([^.]*)\.js$
+- (^|/)jquery\-\d\.\d+(\.\d+)?\.js$
 
 # jQuery UI
-- (^|/)jquery\-ui(\-\d\.\d+(\.\d+)?)?(\.\w+)?(\.min)?\.(js|css)$
-- (^|/)jquery\.(ui|effects)\.([^.]*)(\.min)?\.(js|css)$
+- (^|/)jquery\-ui(\-\d\.\d+(\.\d+)?)?(\.\w+)?\.(js|css)$
+- (^|/)jquery\.(ui|effects)\.([^.]*)\.(js|css)$
 
 # Prototype
 - (^|/)prototype(.*)\.js$
@@ -90,24 +110,33 @@
 # MathJax
 - (^|/)MathJax/
 
+# Chart.js
+- (^|/)Chart\.js$
+
+# Codemirror
+- (^|/)[Cc]ode[Mm]irror/(lib|mode|theme|addon|keymap)
+
 # SyntaxHighlighter - http://alexgorbatchev.com/
 - (^|/)shBrush([^.]*)\.js$
 - (^|/)shCore\.js$
 - (^|/)shLegacy\.js$
 
 # AngularJS
-- (^|/)angular([^.]*)(\.min)?\.js$
+- (^|/)angular([^.]*)\.js$
 
 # D3.js
-- (^|\/)d3(\.v\d+)?([^.]*)(\.min)?\.js$
+- (^|\/)d3(\.v\d+)?([^.]*)\.js$
 
 # React
-- (^|/)react(-[^.]*)?(\.min)?\.js$
+- (^|/)react(-[^.]*)?\.js$
 
 # Modernizr
-- (^|/)modernizr\-\d\.\d+(\.\d+)?(\.min)?\.js$
+- (^|/)modernizr\-\d\.\d+(\.\d+)?\.js$
 - (^|/)modernizr\.custom\.\d+\.js$
 
+# Knockout
+- (^|/)knockout-(\d+\.){3}(debug\.)?js$
+
 ## Python ##
 
 # django
@@ -124,6 +153,9 @@
 
 ## Obj-C ##
 
+# Cocoapods
+- ^Pods/
+
 # Sparkle
 - (^|/)Sparkle/
 
@@ -141,8 +173,8 @@
 - \.intellisense\.js$
 
 # jQuery validation plugin (MS bundles this with asp.net mvc)
-- (^|/)jquery([^.]*)\.validate(\.unobtrusive)?(\.min)?\.js$
-- (^|/)jquery([^.]*)\.unobtrusive\-ajax(\.min)?\.js$
+- (^|/)jquery([^.]*)\.validate(\.unobtrusive)?\.js$
+- (^|/)jquery([^.]*)\.unobtrusive\-ajax\.js$
 
 # Microsoft Ajax
 - (^|/)[Mm]icrosoft([Mm]vc)?([Aa]jax|[Vv]alidation)(\.debug)?\.js$
@@ -168,6 +200,9 @@
 - (^|/)extjs/src/
 - (^|/)extjs/welcome/
 
+# Html5shiv
+- (^|/)html5shiv\.js$
+
 # Samples folders
 - ^[Ss]amples/
 
@@ -185,8 +220,11 @@
 - ^[Tt]est/fixtures/
 
 # PhoneGap/Cordova
-- (^|/)cordova([^.]*)(\.min)?\.js$
-- (^|/)cordova\-\d\.\d(\.\d)?(\.min)?\.js$
+- (^|/)cordova([^.]*)\.js$
+- (^|/)cordova\-\d\.\d(\.\d)?\.js$
+
+# Foundation js
+- foundation(\..*)?\.js$
 
 # Vagrant
 - ^Vagrantfile$
@@ -194,5 +232,18 @@
 # .DS_Store's
 - .[Dd][Ss]_[Ss]tore$
 
-# Mercury --use-subdirs
-- Mercury/
+# R packages
+- ^vignettes/
+- ^inst/extdata/
+
+# Octicons
+- octicons.css
+- sprockets-octicons.scss
+
+# Typesafe Activator
+- (^|/)activator$
+- (^|/)activator\.bat$
+
+# ProGuard
+- proguard.pro
+- proguard-rules.pro

lib/linguist/version.rb

@@ -1,3 +1,3 @@
 module Linguist
-  VERSION = "2.11.2"
+  VERSION = "4.2.3"
 end

package.json

@@ -0,0 +1,6 @@
+{
+  "repository": "https://github.com/github/linguist",
+  "dependencies": {
+    "season": "~>3.0"
+  }
+}

samples/AGS Script/GlobalScript.asc

@@ -0,0 +1,521 @@
+// main global script file
+
+// A function that initializes a bunch of stuff.
+function initialize_control_panel() { 
+  // Centre the control panel
+  gPanel.Centre(); 
+  // Centre the Restart dialog as well
+  gRestartYN.Centre(); 
+  if (!IsSpeechVoxAvailable()) { 
+    // If there is no speech-vox file, and therefore no speech,
+    // disable all the controls related with speech.
+    lblVoice.Visible = false;  
+    btnVoice.Visible = false;  
+    sldVoice.Visible = false;
+  }
+  else {
+    // If there *is*, then set it to voice and text. It's best to use
+    // both whenever possible, for the player's sake.
+    SetVoiceMode(eSpeechVoiceAndText); 
+    // And reflect this in the control panel.
+    btnVoice.Text = "Voice and Text"; 
+  }
+  if (!System.SupportsGammaControl) {
+    // If we can't change the gamma settings, disable the relevant options.
+    sldGamma.Visible = false; 
+    lblGamma.Visible = false;
+  } 
+  
+  //And now, set all the defaults
+  System.Volume = 100;
+  sldAudio.Value = System.Volume;
+  SetGameSpeed(40);
+  sldSpeed.Value = 40;
+  if (IsSpeechVoxAvailable()) {
+     SetVoiceMode(eSpeechVoiceAndText);
+     btnVoice.Text = "Voice and Text";
+     sldVoice.Value = 255;
+     SetSpeechVolume(255);
+  }
+  if (System.SupportsGammaControl) {
+    System.Gamma = 100;
+    sldGamma.Value = 100;
+  }
+}
+
+// Called when the game starts, before the first room is loaded
+function game_start() {   
+  // Put the code all in a function and then just call the function. 
+  // It saves cluttering up places like game_start.
+  initialize_control_panel(); 
+  // Use the KeyboardMovement module to, per default, replicate the standard
+  // keyboard movement of most Sierra games. See KeyboardMovement.txt for more info
+  KeyboardMovement.SetMode(eKeyboardMovement_Tapping); 
+}
+
+function repeatedly_execute() {
+  
+  // Put here anything you want to happen every game cycle, even when
+  // the game is paused. This will not run when the game is blocked
+  // inside a command like a blocking Walk()
+  
+  if (IsGamePaused() == 1) return;
+
+  // Put here anything you want to happen every game cycle, but not
+  // when the game is paused.
+}
+
+function repeatedly_execute_always() {
+  
+  // Put anything you want to happen every game cycle, even
+  // when the game is blocked inside a command like a
+  // blocking Walk().
+  // You cannot run blocking commands from this function.
+  
+}
+
+function show_inventory_window () 
+{
+  gInventory.Visible = true;
+  // switch to the Use cursor (to select items with)
+  mouse.Mode = eModeInteract;
+  // But, override the appearance to look like the arrow
+  mouse.UseModeGraphic(eModePointer);
+}
+
+function show_save_game_dialog()
+{
+  gSaveGame.Visible = true;
+  // Get the list of save games
+  lstSaveGamesList.FillSaveGameList();
+  if (lstSaveGamesList.ItemCount > 0)
+  {
+    // If there is at least one, set the default text
+    // to be the first game's name
+    txtNewSaveName.Text = lstSaveGamesList.Items[0];
+  }
+  else
+  {
+    // No save games yet, default empty text.
+    txtNewSaveName.Text = "";
+  }
+  mouse.UseModeGraphic(eModePointer);
+  gIconbar.Visible = false;
+}
+
+function show_restore_game_dialog()
+{
+  gRestoreGame.Visible = true;
+  lstRestoreGamesList.FillSaveGameList();
+  mouse.UseModeGraphic(eModePointer);
+  gIconbar.Visible = false;
+}
+
+function close_save_game_dialog()
+{
+  gSaveGame.Visible = false;
+  mouse.UseDefaultGraphic();
+  gIconbar.Visible = true;
+}
+
+function close_restore_game_dialog()
+{
+  gRestoreGame.Visible = false;
+  mouse.UseDefaultGraphic();
+  gIconbar.Visible = true;
+}
+
+// Called when a key is pressed. keycode holds the key's ASCII code
+function on_key_press(eKeyCode keycode) {
+  // The following is called before "if game is paused keycode=0", so
+  // it'll happen even when the game is paused.
+  
+  if ((keycode == eKeyEscape) && gRestartYN.Visible) {
+    //Use ESC to cancel restart.
+    gRestartYN.Visible = false; 
+    gIconbar.Visible = true;
+    // If the panel's not ON, then the player must have gotten here by tapping F9,
+    // therefore his cursor needs restoring. If the panel IS on, then it doesn't,
+    // because it's already a pointer. Get used to thinking like this!!
+    if (!gPanel.Visible) mouse.UseDefaultGraphic(); 
+    return;
+  }
+  if ((keycode == eKeyEscape) && gPanel.Visible) {
+    // Use ESC to turn the panel off.
+    gPanel.Visible = false; 
+    mouse.UseDefaultGraphic();
+    gIconbar.Visible = true;
+    return;
+  }
+  if ((keycode == eKeyEscape) && (gSaveGame.Visible))
+  {
+    // Use ESC to close the save game dialog
+    close_save_game_dialog();
+    return;
+  }
+  if ((keycode == eKeyEscape) && (gRestoreGame.Visible))
+  {
+    // Use ESC to close the restore game dialog
+    close_restore_game_dialog();
+    return;
+  }
+  
+  if (keycode == eKeyReturn) { 
+    // ENTER, in this case merely confirms restart
+    if (gRestartYN.Visible) RestartGame();
+  }
+
+  if (IsGamePaused() || (IsInterfaceEnabled() == 0))
+  {
+    // If the game is paused with a modal GUI on the
+    // screen, or the player interface is disabled in
+    // a cut scene, ignore any keypresses.
+    return;
+  }
+
+  // FUNCTION KEYS AND SYSTEM SHORTCUTS
+  if (keycode == eKeyEscape) {
+    // ESC
+    gPanel.Visible = true; 
+    gIconbar.Visible = false;
+    mouse.UseModeGraphic(eModePointer);
+  }
+  if (keycode == eKeyCtrlQ)  QuitGame(1);   // Ctrl-Q
+  if (keycode == eKeyF5) show_save_game_dialog();   // F5
+  if (keycode == eKeyF7) show_restore_game_dialog();  // F7
+  if (keycode == eKeyF9) {
+    // F9, asks the player to confirm restarting (so much better to always confirm first)
+    gRestartYN.Visible = true;  
+    gIconbar.Visible = false;
+    mouse.UseModeGraphic(eModePointer);
+  }
+  if (keycode == eKeyF12) SaveScreenShot("scrnshot.bmp");  // F12
+  if (keycode == eKeyTab)   show_inventory_window();  // Tab, show inventory
+
+  // GAME COMMAND SHORTCUTS
+  if (keycode == 'W') mouse.Mode=eModeWalkto; //Notice this alternate way to indicate keycodes.
+  if (keycode == 'L') mouse.Mode=eModeLookat; //Note that all we do here is set modes.
+  if (keycode == 'U') mouse.Mode=eModeInteract; //If you want something else to happen, such as GUI buttons highlighting,
+  if (keycode == 'T') mouse.Mode=eModeTalkto; //you'll need some more scripting done.
+  if (keycode == 'I') mouse.Mode=eModeUseinv; //But this will, as-is, give you some standard keyboard shortcuts your players will very much appreciate.
+
+  // For extra cursor modes, such as pick up, feel free to add as you will.
+  // Uncomment the line below if you use the "Pick Up" mode.
+  //if (keycode == 'P' || keycode == 'G') mouse.Mode=eModePickup; 
+
+  // DEBUG FUNCTIONS
+  if (keycode == eKeyCtrlS)  Debug(0,0);  // Ctrl-S, give all inventory
+  if (keycode == eKeyCtrlV)  Debug(1,0);  // Ctrl-V, version
+  if (keycode == eKeyCtrlA)  Debug(2,0);  // Ctrl-A, show walkable areas
+  if (keycode == eKeyCtrlX)  Debug(3,0);  // Ctrl-X, teleport to room
+  if (keycode == eKeyCtrlW && game.debug_mode) 
+    player.PlaceOnWalkableArea(); //Ctrl-W, move to walkable area 
+}
+
+
+function on_mouse_click(MouseButton button) {
+  // called when a mouse button is clicked. button is either LEFT or RIGHT
+  if (IsGamePaused() == 1) {
+    // Game is paused, so do nothing (ie. don't allow mouse click)
+  }
+  else if (button == eMouseLeft) {
+    ProcessClick(mouse.x, mouse.y, mouse.Mode );
+  }
+  else if (button == eMouseRight || button == eMouseWheelSouth){
+    // right-click our mouse-wheel down, so cycle cursor
+    mouse.SelectNextMode();
+  }
+  else if (button == eMouseMiddle) { 
+    // Middle-button-click, default make character walk to clicked area (a little shortcut)
+    // Could have been just "player.Walk(mouse.x,mouse.y)", but it's best to
+    // leave our options open - what if you have a special script triggered
+    // on "walking" mode?
+    ProcessClick(mouse.x, mouse.y, eModeWalkto); 
+  }
+  else if (button == eMouseWheelNorth) { 
+    // Mouse-wheel up, cycle cursors 
+    // If mode isn't WALK, set the previous mode (notice usage of numbers instead
+    // of eNums, when it suits us)...
+    if (mouse.Mode>0) mouse.Mode=mouse.Mode-1; 
+    else 
+    { 
+      // ...but if it is WALK mode...
+      if (player.ActiveInventory!=null) 
+      {
+        //...and the player has a selected inventory item, set mouse mode to UseInv. 
+        mouse.Mode=eModeUseinv; 
+      }
+      else 
+      {
+        // If they don't, however, just set it to mode TALK (change this line if you add more cursor modes)
+        mouse.Mode=eModeTalkto; 
+      }
+    }
+  }
+}
+
+function interface_click(int interface, int button) {
+  // This function is obsolete, from 2.62 and earlier versions.
+}
+
+function btnInvUp_Click(GUIControl *control, MouseButton button) {
+  invCustomInv.ScrollUp();
+}
+
+function btnInvDown_Click(GUIControl *control, MouseButton button) {
+  invCustomInv.ScrollDown();
+}
+
+function btnInvOK_Click(GUIControl *control, MouseButton button) {
+	// They pressed the OK button, close the GUI
+	gInventory.Visible = false;
+	mouse.UseDefaultGraphic();
+}
+
+function btnInvSelect_Click(GUIControl *control, MouseButton button) {
+  
+	// They pressed SELECT, so switch to the Get cursor
+	mouse.Mode = eModeInteract;
+	// But, override the appearance to look like the arrow
+	mouse.UseModeGraphic(eModePointer);
+}
+
+function btnIconInv_Click(GUIControl *control, MouseButton button) {
+  
+  show_inventory_window();
+}
+
+function btnIconCurInv_Click(GUIControl *control, MouseButton button) {
+  
+  if (player.ActiveInventory != null)
+    mouse.Mode = eModeUseinv;
+}
+
+function btnIconSave_Click(GUIControl *control, MouseButton button) 
+{
+  show_save_game_dialog();
+}
+
+function btnIconLoad_Click(GUIControl *control, MouseButton button) 
+{
+  show_restore_game_dialog();
+}
+
+function btnIconExit_Click(GUIControl *control, MouseButton button) {
+  
+  QuitGame(1);
+}
+
+function btnIconAbout_Click(GUIControl *control, MouseButton button) {
+  
+  gPanel.Visible=true;
+  gIconbar.Visible=false;
+  mouse.UseModeGraphic(eModePointer);
+}
+
+function cEgo_Look()
+{
+  Display("Damn, I'm looking good!");
+}
+
+function cEgo_Interact()
+{
+  Display("You rub your hands up and down your clothes.");
+}
+
+function cEgo_Talk()
+{
+  Display("Talking to yourself is a sign of madness!");
+}
+
+//START OF CONTROL PANEL FUNCTIONS
+function btnSave_OnClick(GUIControl *control, MouseButton button)
+{
+  gPanel.Visible = false;
+  mouse.UseDefaultGraphic();
+  gIconbar.Visible = true;
+  Wait(1);
+  btnIconSave_Click(btnIconSave, eMouseLeft);
+}
+
+function gControl_OnClick(GUI *theGui, MouseButton button)
+{
+
+}
+
+function btnAbout_OnClick(GUIControl *control, MouseButton button)
+{
+Display("Adventure Game Studio run-time engine default game.");
+}
+
+function btnQuit_OnClick(GUIControl *control, MouseButton button)
+{
+  gPanel.Visible = false;
+  Wait(1);
+  QuitGame(1);
+  gPanel.Visible = true;
+  gIconbar.Visible = false;
+  mouse.UseModeGraphic(eModePointer);
+}
+
+function btnLoad_OnClick(GUIControl *control, MouseButton button)
+{
+  gPanel.Visible = false;
+  mouse.UseDefaultGraphic();
+  gIconbar.Visible = true;
+  Wait(1);
+  btnIconLoad_Click(btnIconLoad, eMouseLeft);
+}
+
+function btnResume_OnClick(GUIControl *control, MouseButton button)
+{
+  gPanel.Visible = false;
+  mouse.UseDefaultGraphic();
+  gIconbar.Visible = true;
+}
+
+function sldAudio_OnChange(GUIControl *control)
+{
+  System.Volume = sldAudio.Value;
+}
+
+function sldVoice_OnChange(GUIControl *control)
+{
+  // Sets voice volume. Note that we don't check for the existence of speech.vox - 
+  // we did that in game_start, so if it's not there the slider won't even be available.
+  SetSpeechVolume(sldVoice.Value); 
+}
+
+function btnVoice_OnClick(GUIControl *control, MouseButton button)
+{
+  // Note that we don't check for the existence of speech.vox - we did that in game_start,
+  // so if it's not there the button won't even be available.
+  if (btnVoice.Text == "Voice and Text") { 
+    SetVoiceMode(eSpeechVoiceOnly); 
+    btnVoice.Text = "Voice only";
+  }
+  else if (btnVoice.Text == "Voice only") {
+    SetVoiceMode(eSpeechTextOnly);
+    btnVoice.Text = "Text only";
+  }
+  else if (btnVoice.Text == "Text only") {
+    SetVoiceMode(eSpeechVoiceAndText);
+    btnVoice.Text = "Voice and Text";
+  }
+}
+
+function sldGamma_OnChange(GUIControl *control)
+{
+  // Set the gamma. Note there's no need to check for anything else, as we ensured,
+  // in game_start, that the slider won't even appear if it's not possible to do this.
+  System.Gamma = sldGamma.Value; 
+}
+
+function btnDefault_OnClick(GUIControl *control, MouseButton button)
+{
+  // Reset everything to default. You'll have to edit these as well as the sliders
+  // if you'd rather have different default parameters.
+  System.Volume = 100;
+  sldAudio.Value = System.Volume;
+  sldSpeed.Value = 40;
+  SetGameSpeed(40);
+  if (IsSpeechVoxAvailable()) {
+     SetVoiceMode(eSpeechVoiceAndText);
+     btnVoice.Text = "Voice and Text";
+     sldVoice.Value = 255;
+     SetSpeechVolume(255);
+  }
+  if (System.SupportsGammaControl) {
+    System.Gamma = 100;
+    sldGamma.Value = 100;
+  }
+}
+//END OF CONTROL PANEL FUNCTIONS
+
+function dialog_request(int param) 
+{
+  // This is used by the dialog text parser if you need to process
+  // text that the player types in to the parser.
+  // It is not used by default.
+}
+
+function sldSpeed_OnChange(GUIControl *control)
+{
+  SetGameSpeed(sldSpeed.Value);
+}
+
+function btnRestart_OnClick(GUIControl *control, MouseButton button)
+{
+  gRestartYN.Visible=true;
+  gIconbar.Visible=false;
+}
+
+function btnRestartYes_OnClick(GUIControl *control, MouseButton button)
+{
+  RestartGame();
+}
+
+function btnRestartNo_OnClick(GUIControl *control, MouseButton button)
+{
+  gRestartYN.Visible = false;
+  gIconbar.Visible = true;
+  // If the panel's not ON, then the player must have gotten here by tapping F9,
+  // therefore his cursor needs restoring. If the panel IS on, then it doesn't,
+  // because it's already a pointer. Get used to thinking like this!!
+  if (!gPanel.Visible) mouse.UseDefaultGraphic(); 
+}
+
+function btnCancelSave_OnClick(GUIControl *control, MouseButton button)
+{
+  close_save_game_dialog();
+}
+
+function btnSaveGame_OnClick(GUIControl *control, MouseButton button)
+{
+  int gameSlotToSaveInto = lstSaveGamesList.ItemCount + 1;
+  int i = 0;
+  while (i < lstSaveGamesList.ItemCount)
+  {
+    if (lstSaveGamesList.Items[i] == txtNewSaveName.Text)
+    {
+      gameSlotToSaveInto = lstSaveGamesList.SaveGameSlots[i];
+    }
+    i++;
+  }
+  SaveGameSlot(gameSlotToSaveInto, txtNewSaveName.Text);
+  close_save_game_dialog();
+}
+
+function btnCancelRestore_OnClick(GUIControl *control, MouseButton button)
+{
+  close_restore_game_dialog();
+}
+
+function btnRestoreGame_OnClick(GUIControl *control, MouseButton button)
+{
+  if (lstRestoreGamesList.SelectedIndex >= 0)
+  {
+    RestoreGameSlot(lstRestoreGamesList.SaveGameSlots[lstRestoreGamesList.SelectedIndex]);
+  }
+  close_restore_game_dialog();
+}
+
+function lstSaveGamesList_OnSelectionCh(GUIControl *control)
+{
+  txtNewSaveName.Text = lstSaveGamesList.Items[lstSaveGamesList.SelectedIndex];
+}
+
+function txtNewSaveName_OnActivate(GUIControl *control)
+{
+  // Pressing return in the text box simulates clicking the Save button
+  btnSaveGame_OnClick(control, eMouseLeft);
+}
+
+function btnDeleteSave_OnClick(GUIControl *control, MouseButton button)
+{
+  if (lstSaveGamesList.SelectedIndex >= 0)
+  {
+    DeleteSaveSlot(lstSaveGamesList.SaveGameSlots[lstSaveGamesList.SelectedIndex]);
+    lstSaveGamesList.FillSaveGameList();
+  }
+}

samples/AGS Script/GlobalScript.ash

@@ -0,0 +1,4 @@
+// Main header script - this will be included into every script in
+// the game (local and global). Do not place functions here; rather,
+// place import definitions and #define names here to be used by all
+// scripts.

samples/AGS Script/KeyboardMovement_102.asc

@@ -0,0 +1,216 @@
+// Main script for module 'KeyboardMovement'
+
+//****************************************************************************************************
+// DEFINITIONS
+//****************************************************************************************************
+
+#define DISTANCE 10000// distance player walks in Tapping mode before he stops
+
+enum KeyboardMovement_Directions {
+	eKeyboardMovement_Stop, 
+	eKeyboardMovement_DownLeft, 
+	eKeyboardMovement_Down, 
+	eKeyboardMovement_DownRight, 
+	eKeyboardMovement_Left, 
+	eKeyboardMovement_Right, 
+	eKeyboardMovement_UpLeft, 
+	eKeyboardMovement_Up, 
+	eKeyboardMovement_UpRight
+};
+
+//****************************************************************************************************
+// VARIABLES
+//****************************************************************************************************
+
+// keycodes as variables for future key customization functions (static variables?):
+int KeyboardMovement_KeyDown = 380; // down arrow
+int KeyboardMovement_KeyLeft = 375; // left arrow
+int KeyboardMovement_KeyRight = 377; // right arrow
+int KeyboardMovement_KeyUp = 372; // up arrow
+int KeyboardMovement_KeyDownRight = 381; // PgDn (numpad)
+int KeyboardMovement_KeyUpRight = 373; // PgUp (numpad)
+int KeyboardMovement_KeyDownLeft = 379; // End (numpad)
+int KeyboardMovement_KeyUpLeft = 371; // Home (numpad)
+int KeyboardMovement_KeyStop = 376; // 5 (numpad)
+
+KeyboardMovement_Modes KeyboardMovement_Mode = eKeyboardMovement_None; // stores current keyboard control mode (disabled by default)
+
+KeyboardMovement_Directions KeyboardMovement_CurrentDirection = eKeyboardMovement_Stop; // stores current walking direction of player character
+
+//****************************************************************************************************
+// USER FUNCTIONS
+//****************************************************************************************************
+
+//====================================================================================================
+
+static function KeyboardMovement::SetMode(KeyboardMovement_Modes mode) {
+	KeyboardMovement_Mode = mode;
+}
+
+//====================================================================================================
+
+// key customization functions here
+
+//====================================================================================================
+
+//****************************************************************************************************
+// EVENT HANDLER FUNCTIONS
+//****************************************************************************************************
+
+//====================================================================================================
+
+function repeatedly_execute() {
+
+	//--------------------------------------------------
+	// Pressing mode
+	//--------------------------------------------------
+
+	if ((IsGamePaused() == true) || (KeyboardMovement_Mode != eKeyboardMovement_Pressing) || (IsInterfaceEnabled() == false) || (player.on == false)) return 0;
+	  // if game is paused, module or mode disabled, interface disabled or player character hidden, quit function
+
+	KeyboardMovement_Directions newdirection; // declare variable storing new direction
+
+	// get new direction:
+	if ( ((IsKeyPressed(KeyboardMovement_KeyDown)) && (IsKeyPressed(KeyboardMovement_KeyRight))) || (IsKeyPressed(KeyboardMovement_KeyDownRight)) ) newdirection = eKeyboardMovement_DownRight; // if down&right arrows or PgDn (numeric pad) held down, set new direction to Down-Right
+	else if ( ((IsKeyPressed(KeyboardMovement_KeyUp)) && (IsKeyPressed(KeyboardMovement_KeyRight))) || (IsKeyPressed(KeyboardMovement_KeyUpRight)) ) newdirection = eKeyboardMovement_UpRight; // up&right arrows or PgUp (numpad)
+	else if ( ((IsKeyPressed(KeyboardMovement_KeyDown)) && (IsKeyPressed(KeyboardMovement_KeyLeft))) || (IsKeyPressed(KeyboardMovement_KeyDownLeft)) ) newdirection = eKeyboardMovement_DownLeft; // down&left arrows or End (numpad)
+	else if ( ((IsKeyPressed(KeyboardMovement_KeyUp)) && (IsKeyPressed(KeyboardMovement_KeyLeft))) || (IsKeyPressed(KeyboardMovement_KeyUpLeft)) ) newdirection = eKeyboardMovement_UpLeft; // up&left arrows or Home (numpad)
+	else if (IsKeyPressed(KeyboardMovement_KeyDown)) newdirection = eKeyboardMovement_Down; // down arrow
+	else if (IsKeyPressed(KeyboardMovement_KeyLeft)) newdirection = eKeyboardMovement_Left; // left arrow
+	else if (IsKeyPressed(KeyboardMovement_KeyRight)) newdirection = eKeyboardMovement_Right; // right arrow
+	else if (IsKeyPressed(KeyboardMovement_KeyUp)) newdirection = eKeyboardMovement_Up; // up arrow
+	else newdirection = eKeyboardMovement_Stop; // if none of the above held down, set it to stop player character
+
+	if (IsKeyPressed(KeyboardMovement_KeyStop)) newdirection = eKeyboardMovement_Stop; // if 5 (numeric pad) held down, stop player character, regardless of whether some of the above are held down
+
+	if (newdirection != KeyboardMovement_CurrentDirection) { // if new direction is different from current direction
+
+		if (newdirection == eKeyboardMovement_Stop) player.StopMoving(); // if new direction is the Stop command, stop movement of player character
+		else { // if new direction is NOT the Stop command
+
+			int dx, dy; // declare variables storing new walk coordinates
+			if (newdirection == eKeyboardMovement_DownRight) {
+				dx = DISTANCE;
+				dy = DISTANCE;
+			}
+			else if (newdirection == eKeyboardMovement_UpRight) {
+				dx = DISTANCE;
+				dy = -DISTANCE;
+			}
+			else if (newdirection == eKeyboardMovement_DownLeft) {
+				dx = -DISTANCE;
+				dy = DISTANCE;
+			}
+			else if (newdirection == eKeyboardMovement_UpLeft) {
+				dx = -DISTANCE;
+				dy = -DISTANCE;
+			}
+			else if (newdirection == eKeyboardMovement_Down) {
+				dx = 0;
+				dy = DISTANCE;
+			}
+			else if (newdirection == eKeyboardMovement_Left) {
+				dx = -DISTANCE;
+				dy = 0;
+			}
+			else if (newdirection == eKeyboardMovement_Right) {
+				dx = DISTANCE;
+				dy = 0;
+			}
+			else if (newdirection == eKeyboardMovement_Up) {
+				dx = 0;
+				dy = -DISTANCE;
+			}
+
+			player.WalkStraight(player.x + dx, player.y + dy, eNoBlock); // walk player character to the new coordinates
+		}
+		KeyboardMovement_CurrentDirection = newdirection; // update current direction to new direction
+
+	}
+
+}
+
+//====================================================================================================
+
+function on_key_press(int keycode) {
+
+	//--------------------------------------------------
+	// Tapping mode
+	//--------------------------------------------------
+
+	if ((IsGamePaused() == true) || (KeyboardMovement_Mode != eKeyboardMovement_Tapping) || (IsInterfaceEnabled() == false) || (player.on == false)) return 0;
+	  // if game is paused, module or mode disabled, interface disabled or player character hidden, quit function
+
+	KeyboardMovement_Directions newdirection; // declare variable storing new direction
+
+	// get new direction:
+	if (keycode == KeyboardMovement_KeyDownRight) newdirection = eKeyboardMovement_DownRight; // if down-right key pressed, set new direction to Down-Right
+	else if (keycode == KeyboardMovement_KeyUpRight) newdirection = eKeyboardMovement_UpRight;
+	else if (keycode == KeyboardMovement_KeyDownLeft) newdirection = eKeyboardMovement_DownLeft;
+	else if (keycode == KeyboardMovement_KeyUpLeft) newdirection = eKeyboardMovement_UpLeft;
+	else if (keycode == KeyboardMovement_KeyDown) newdirection = eKeyboardMovement_Down;
+	else if (keycode == KeyboardMovement_KeyLeft) newdirection = eKeyboardMovement_Left;
+	else if (keycode == KeyboardMovement_KeyRight) newdirection = eKeyboardMovement_Right;
+	else if (keycode == KeyboardMovement_KeyUp) newdirection = eKeyboardMovement_Up;
+	else if (keycode == KeyboardMovement_KeyStop) newdirection = eKeyboardMovement_Stop; // if stop key pressed, set to stop player character
+
+	if (newdirection != KeyboardMovement_CurrentDirection) { // if new direction is different from current direction
+
+		if (newdirection == eKeyboardMovement_Stop) player.StopMoving(); // if new direction is the Stop command, stop movement of player character
+		else { // if new direction is NOT the Stop command
+
+			int dx, dy; // declare variables storing new walk coordinates
+			if (newdirection == eKeyboardMovement_DownRight) {
+				dx = DISTANCE;
+				dy = DISTANCE;
+			}
+			else if (newdirection == eKeyboardMovement_UpRight) {
+				dx = DISTANCE;
+				dy = -DISTANCE;
+			}
+			else if (newdirection == eKeyboardMovement_DownLeft) {
+				dx = -DISTANCE;
+				dy = DISTANCE;
+			}
+			else if (newdirection == eKeyboardMovement_UpLeft) {
+				dx = -DISTANCE;
+				dy = -DISTANCE;
+			}
+			else if (newdirection == eKeyboardMovement_Down) {
+				dx = 0;
+				dy = DISTANCE;
+			}
+			else if (newdirection == eKeyboardMovement_Left) {
+				dx = -DISTANCE;
+				dy = 0;
+			}
+			else if (newdirection == eKeyboardMovement_Right) {
+				dx = DISTANCE;
+				dy = 0;
+			}
+			else if (newdirection == eKeyboardMovement_Up) {
+				dx = 0;
+				dy = -DISTANCE;
+			}
+
+			player.WalkStraight(player.x + dx, player.y + dy, eNoBlock); // walk player character to the new coordinates
+		}
+		KeyboardMovement_CurrentDirection = newdirection; // update current direction to new direction
+
+	}
+	else { // if new direction is same as current direction
+		player.StopMoving(); // stop player character
+		KeyboardMovement_CurrentDirection = eKeyboardMovement_Stop; // update current direction
+	}
+
+}
+
+//====================================================================================================
+
+function on_event(EventType event, int data) {
+
+	if (event == eEventLeaveRoom) KeyboardMovement_CurrentDirection = eKeyboardMovement_Stop;
+
+}
+
+//====================================================================================================

samples/AGS Script/KeyboardMovement_102.ash

@@ -0,0 +1,13 @@
+// Script header for module 'KeyboardMovement'
+
+#define KeyboardMovement_VERSION 101
+
+enum KeyboardMovement_Modes {
+	eKeyboardMovement_None, 
+	eKeyboardMovement_Tapping, 
+	eKeyboardMovement_Pressing
+};
+
+struct KeyboardMovement {
+	import static function SetMode(KeyboardMovement_Modes mode);
+};

samples/APL/DeepakChopra.apl

@@ -0,0 +1,18 @@
+⍝   You can try this at http://tryapl.org/
+⍝   I can not explain how much I suddenly love this crypto-language
+
+
+
+Starts     ← 'Experiential truth ' 'The physical world ' 'Non-judgment '       'Quantum physics '
+Middles    ← 'nurtures an '        'projects onto '      'imparts reality to ' 'constructs with '
+Qualifiers ← 'abundance of '       'the barrier of '     'self-righteous '     'potential '
+Finishes   ← 'marvel.'             'choices.'            'creativity.'         'actions.'
+
+rf     ← {(?⍴⍵)⊃⍵}
+erf    ← {rf ¨ ⍵}
+
+deepak ← {erf Starts Middles Qualifiers Finishes}
+
+
+
+deepak ⍬

samples/APL/UT.dyalog

@@ -0,0 +1,367 @@
+:NameSpace UT
+
+    sac ← 0
+    expect_orig ← expect ← ⎕NS⍬
+    exception ← ⍬
+    nexpect_orig ← nexpect ← ⎕NS⍬
+
+    ∇ {Z}←{Conf}run Argument;PRE_test;POST_test;TEST_step;COVER_step;FromSpace
+     
+      load_display_if_not_already_loaded
+      load_salt_scripts_into_current_namespace_if_configured
+     
+      FromSpace←1⊃⎕RSI
+     
+      PRE_test←{}
+      POST_test←{}
+      COVER_step←{}
+      :If 0≠⎕NC'Conf'
+          :If Conf has'cover_target'
+              PRE_test←{{}⎕PROFILE'start'}
+              POST_test←{{}⎕PROFILE'stop'}
+          :EndIf
+      :EndIf
+     
+      :If is_function Argument
+          TEST_step←single_function_test_function
+          COVER_file←Argument,'_coverage.html'
+     
+      :ElseIf is_list_of_functions Argument
+          TEST_step←list_of_functions_test_function
+          COVER_file←'list_coverage.html'
+     
+      :ElseIf is_file Argument
+          TEST_step←file_test_function
+          COVER_file←(get_file_name Argument),'_coverage.html'
+     
+      :ElseIf is_dir Argument
+          test_files←test_files_in_dir Argument
+          TEST_step←test_dir_function
+          Argument←test_files
+      :EndIf
+     
+      :If 0≠⎕NC'Conf'
+          :If Conf has'cover_target'
+              COVER_step←{Conf,←⊂('cover_file'COVER_file)
+                  generate_coverage_page Conf}
+          :EndIf
+      :EndIf
+     
+      PRE_test ⍬
+      Z←FromSpace TEST_step Argument
+      POST_test ⍬
+      COVER_step ⍬
+    ∇
+
+    ∇ load_display_if_not_already_loaded
+      :If 0=⎕NC'#.DISPLAY'
+          'DISPLAY'#.⎕CY'display'
+      :EndIf
+    ∇
+
+    ∇ load_salt_scripts_into_current_namespace_if_configured
+      :If 0≠⎕NC'#.UT.appdir'
+          :If ⍬≢#.UT.appdir
+              ⎕SE.SALT.Load #.UT.appdir,'src/*.dyalog -target=#'
+              ⎕SE.SALT.Load #.UT.appdir,'test/*.dyalog -target=#'
+          :EndIf
+      :EndIf
+    ∇
+
+    ∇ Z←FromSpace single_function_test_function TestName
+      Z←run_ut FromSpace TestName
+    ∇
+
+    ∇ Z←FromSpace list_of_functions_test_function ListOfNames;t
+      t←⎕TS
+      Z←run_ut¨{FromSpace ⍵}¨ListOfNames
+      t←⎕TS-t
+      ('Test execution report')print_passed_crashed_failed Z t
+    ∇
+
+    ∇ Z←FromSpace file_test_function FilePath;FileNS;Functions;TestFunctions;t
+      FileNS←⎕SE.SALT.Load FilePath,' -target=#'
+      Functions←↓FileNS.⎕NL 3
+      TestFunctions←(is_test¨Functions)/Functions
+      :If (0/⍬,⊂0/'')≡TestFunctions
+          ⎕←'No test functions found'
+          Z←⍬
+      :Else
+          t←⎕TS
+          Z←run_ut¨{FileNS ⍵}¨TestFunctions
+          t←⎕TS-t
+          (FilePath,' tests')print_passed_crashed_failed Z t
+      :EndIf
+    ∇
+
+    ∇ Z←FromSpace test_dir_function Test_files
+      :If Test_files≡⍬/⍬,⊂''
+          ⎕←'No test files found'
+          Z←⍬
+      :Else
+          Z←#.UT.run¨Test_files
+      :EndIf
+    ∇
+
+    ∇ Z←get_file_name Argument;separator
+      separator←⊃⌽(Argument∊'/\')/⍳⍴Argument
+      Z←¯7↓separator↓Argument
+    ∇
+
+    ∇ generate_coverage_page Conf;ProfileData;CoverResults;HTML
+      ProfileData←⎕PROFILE'data'
+      ToCover←retrieve_coverables¨(⊃'cover_target'in Conf)
+      :If (⍴ToCover)≡(⍴⊂1)
+          ToCover←⊃ToCover
+      :EndIf
+      Representations←get_representation¨ToCover
+      CoverResults←ProfileData∘generate_cover_result¨↓ToCover,[1.5]Representations
+      HTML←generate_html CoverResults
+      Conf write_html_to_page HTML
+      ⎕PROFILE'clear'
+    ∇
+
+    ∇ Z←retrieve_coverables Something;nc;functions
+      nc←⎕NC Something
+      :If nc=3
+          Z←Something
+      :ElseIf nc=9
+          functions←strip¨↓⍎Something,'.⎕NL 3'
+          Z←{(Something,'.',⍵)}¨functions
+      :EndIf
+    ∇
+
+    ∇ Z←strip input
+      Z←(input≠' ')/input
+    ∇
+
+    ∇ Z←get_representation Function;nc;rep
+      nc←⎕NC⊂Function
+      :If nc=3.1
+          rep←↓⎕CR Function
+          rep[1]←⊂'∇',⊃rep[1]
+          rep,←⊂'∇'
+          rep←↑rep
+      :Else
+          rep←⎕CR Function
+      :EndIf
+      Z←rep
+    ∇
+
+    ∇ Z←ProfileData generate_cover_result(name representation);Indices;lines;functionlines;covered_lines
+      Indices←({name≡⍵}¨ProfileData[;1])/⍳⍴ProfileData[;1]
+      lines←ProfileData[Indices;2]
+      nc←⎕NC⊂name
+      :If 3.1=nc
+          functionlines←¯2+⍴↓representation
+      :Else
+          functionlines←⊃⍴↓representation
+      :EndIf
+      covered_lines←(⍬∘≢¨lines)/lines
+      Z←(nc lines functionlines covered_lines representation)
+    ∇
+
+    ∇ Z←generate_html CoverResults;Covered;Total;Percentage;CoverageText;ColorizedCode;Timestamp;Page
+      Covered←⊃⊃+/{⍴4⊃⍵}¨CoverResults
+      Total←⊃⊃+/{3⊃⍵}¨CoverResults
+      Percentage←100×Covered÷Total
+      CoverageText←'Coverage: ',Percentage,'% (',Covered,'/',Total,')'
+      ColorizedCode←⊃,/{colorize_code_by_coverage ⍵}¨CoverResults
+      Timestamp←generate_timestamp_text
+      Page←⍬
+      Page,←⊂⍬,'<html>'
+      Page,←⊂⍬,'<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>'
+      Page,←⊂⍬,'<style>pre cov {line-height:80%;}'
+      Page,←⊂⍬,'pre cov {color: green;}'
+      Page,←⊂⍬,'pre uncov {line-height:80%;}'
+      Page,←⊂⍬,'pre uncov {color:red;}</style>'
+      Page,←⊂⍬,CoverageText
+      Page,←⊂⍬,'<pre>'
+      Page,←ColorizedCode
+      Page,←⊂⍬,'</pre>'
+      Page,←Timestamp
+      Page,←⊂⍬,'</html>'
+      Z←Page
+    ∇
+
+    ∇ Z←colorize_code_by_coverage CoverResult;Colors;Ends;Code
+      :If 3.1=⊃CoverResult
+          Colors←(2+3⊃CoverResult)⍴⊂'<uncov>'
+          Colors[1]←⊂''
+          Colors[⍴Colors]←⊂''
+          Ends←(2+3⊃CoverResult)⍴⊂'</uncov>'
+          Ends[1]←⊂''
+          Ends[⍴Ends]←⊂''
+      :Else
+          Colors←(3⊃CoverResult)⍴⊂'<uncov>'
+          Ends←(3⊃CoverResult)⍴⊂'</uncov>'
+      :EndIf
+      Colors[1+4⊃CoverResult]←⊂'<cov>'
+      Ends[1+4⊃CoverResult]←⊂'</cov>'
+      Code←↓5⊃CoverResult
+      Z←Colors,[1.5]Code
+      Z←{⍺,(⎕UCS 13),⍵}/Z,Ends
+    ∇
+
+    ∇ Z←generate_timestamp_text;TS;YYMMDD;HHMMSS
+      TS←⎕TS
+      YYMMDD←⊃{⍺,'-',⍵}/3↑TS
+      HHMMSS←⊃{⍺,':',⍵}/3↑3↓TS
+      Z←'Page generated: ',YYMMDD,'|',HHMMSS
+    ∇
+
+    ∇ Conf write_html_to_page Page;tie;filename
+      filename←(⊃'cover_out'in Conf),(⊃'cover_file'in Conf)
+      :Trap 22
+          tie←filename ⎕NTIE 0
+          filename ⎕NERASE tie
+          filename ⎕NCREATE tie
+      :Else
+          tie←filename ⎕NCREATE 0
+      :EndTrap
+      Simple_array←⍕⊃,/Page
+      (⎕UCS'UTF-8'⎕UCS Simple_array)⎕NAPPEND tie
+    ∇
+
+    ∇ Z←is_function Argument
+      Z←'_TEST'≡¯5↑Argument
+    ∇
+
+    ∇ Z←is_list_of_functions Argument
+      Z←2=≡Argument
+    ∇
+
+    ∇ Z←is_file Argument
+      Z←'.dyalog'≡¯7↑Argument
+    ∇
+
+    ∇ Z←is_dir Argument;attr
+      :If 'Linux'≡5↑⊃'.'⎕WG'APLVersion'
+          Z←'yes'≡⊃⎕CMD'test -d ',Argument,' && echo yes || echo no'
+      :Else
+          'gfa'⎕NA'I kernel32|GetFileAttributes* <0t'
+          :If Z←¯1≠attr←gfa⊂Argument ⍝ If file exists
+              Z←⊃2 16⊤attr           ⍝ Return bit 4
+          :EndIf
+      :EndIf
+    ∇
+
+
+    ∇ Z←test_files_in_dir Argument
+      :If 'Linux'≡5↑⊃'.'⎕WG'APLVersion'
+          Z←⎕SH'find ',Argument,' -name \*_tests.dyalog'
+      :Else
+          #.⎕CY'files'
+          Z←#.Files.Dir Argument,'\*_tests.dyalog'
+          Z←(Argument,'\')∘,¨Z
+      :EndIf
+    ∇
+
+    ∇ Z←run_ut ut_data;returned;crashed;pass;crash;fail;message
+      (returned crashed time)←execute_function ut_data
+      (pass crash fail)←determine_pass_crash_or_fail returned crashed
+      message←determine_message pass fail crashed(2⊃ut_data)returned time
+      print_message_to_screen message
+      Z←(pass crash fail)
+    ∇
+
+    ∇ Z←execute_function ut_data;function;t
+      reset_UT_globals
+      function←(⍕(⊃ut_data[1])),'.',⊃ut_data[2]
+      :Trap sac
+          :If 3.2≡⎕NC⊂function
+              t←⎕TS
+              Z←(⍎function,' ⍬')0
+              t←⎕TS-t
+          :Else
+              t←⎕TS
+              Z←(⍎function)0
+              t←⎕TS-t
+          :EndIf
+     
+      :Else
+          Z←(↑⎕DM)1
+          :If exception≢⍬
+              expect←exception
+              Z[2]←0
+              t←⎕TS-t
+          :EndIf
+      :EndTrap
+      Z,←⊂t
+    ∇
+
+    ∇ reset_UT_globals
+      expect_orig ← expect← ⎕NS⍬
+      exception←⍬
+      nexpect_orig ← nexpect← ⎕NS⍬
+    ∇
+
+    ∇ Z←is_test FunctionName;wsIndex
+      wsIndex←FunctionName⍳' '
+      FunctionName←(wsIndex-1)↑FunctionName
+      Z←'_TEST'≡¯5↑FunctionName
+    ∇
+
+    ∇ Heading print_passed_crashed_failed(ArrayRes time)
+      ⎕←'-----------------------------------------'
+      ⎕←Heading
+      ⎕←'    ⍋  Passed: ',+/{1⊃⍵}¨ArrayRes
+      ⎕←'    ⍟ Crashed: ',+/{2⊃⍵}¨ArrayRes
+      ⎕←'    ⍒  Failed: ',+/{3⊃⍵}¨ArrayRes
+      ⎕←'    ○ Runtime: ',time[5],'m',time[6],'s',time[7],'ms'
+    ∇
+    
+    determine_pass_crash_or_fail←{
+      r c←⍵ ⋄ 0≠c:0 1 0 ⋄ z←(0 0 1)(1 0 0)
+      expect_orig≢expect:(⎕IO+expect≡r)⊃z ⋄ (⎕IO+nexpect≢r)⊃z
+    }
+
+    ∇ Z←determine_message(pass fail crashed name returned time)
+      :If crashed
+          Z←'CRASHED: 'failure_message name returned
+      :ElseIf pass
+          Z←'Passed ',time[5],'m',time[6],'s',time[7],'ms'
+      :Else
+          Z←'FAILED: 'failure_message name returned
+      :EndIf
+    ∇
+
+    ∇ print_message_to_screen message
+      ⎕←message
+    ∇
+
+    ∇ Z←term_to_text Term;Text;Rows
+      Text←#.DISPLAY Term
+      Rows←1⊃⍴Text
+      Z←(Rows 4⍴''),Text
+    ∇
+
+    ∇ Z←Cause failure_message(name returned);hdr;exp;expterm;got;gotterm
+      hdr←Cause,name
+      exp←'Expected'
+      expterm←term_to_text #.UT.expect
+      got←'Got'
+      gotterm←term_to_text returned
+      Z←align_and_join_message_parts hdr exp expterm got gotterm
+    ∇
+
+    ∇ Z←align_and_join_message_parts Parts;hdr;exp;expterm;got;gotterm;R1;C1;R2;C2;W
+      (hdr exp expterm got gotterm)←Parts
+      (R1 C1)←⍴expterm
+      (R2 C2)←⍴gotterm
+      W←⊃⊃⌈/C1 C2(⍴hdr)(⍴exp)(⍴got)
+      Z←(W↑hdr),[0.5](W↑exp)
+      Z←Z⍪(R1 W↑expterm)
+      Z←Z⍪(W↑got)
+      Z←Z⍪(R2 W↑gotterm)
+    ∇
+
+    ∇ Z←confparam in config
+      Z←1↓⊃({confparam≡⊃⍵}¨config)/config
+    ∇
+
+    ∇ Z←config has confparam
+      Z←∨/{confparam≡⊃⍵}¨config
+    ∇
+
+:EndNameSpace

samples/Ant Build System/filenames/ant.xml

@@ -0,0 +1,110 @@
+<?xml version="1.0" encoding="iso-8859-1"?>
+<project name="WebBuild">
+
+    <!-- generate timestamps -->
+    <tstamp />
+
+    <!-- Debugging Macro -->
+    <import file="echopath.xml" />
+
+    <!-- JS build files macro -->
+    <import file="rhinoscript.xml" />
+
+    <!-- Component Build Files -->
+    <import file="setup.xml" />
+    <import file="clean.xml" />
+    <import file="copy.xml" />
+    <import file="file.transform.xml" />
+    <import file="external.tools.xml" />
+    <import file="rename.xml" />
+    <import file="js.xml" />
+    <import file="css.xml" />
+    <import file="img.xml" />
+    <import file="png8.xml" />
+    <import file="yui.xml" />
+    <import file="cdn.xml" />
+    <import file="datauri.xml" />
+    <import file="devlive.xml" />
+
+    <!-- This dirname is the only complete path we know for sure, everything builds off of it -->
+    <dirname property="dir.build" file="${ant.file.WebBuild}" />
+
+    <!-- get name for newly built folder -->
+    <basename property="app.name"       file="${basedir}" />
+
+    <!-- read global properties file -->
+    <property file="${dir.build}\build.properties" />
+
+    <!-- Build Directories -->
+    <property name="dir.build.js"   location="${dir.build}/js" />
+
+    <!-- App Directories -->
+    <property name="dir.app"        location="${dir.result}/${app.name}" />
+    <property name="dir.app.temp"   location="${dir.temp}/${app.name}" />
+    <property name="dir.app.files"  location="${dir.app.temp}/${dir.files}" />
+
+    <!-- Files -->
+    <property name="mapping.js"     location="${dir.app.temp}/${mapping.file.js}" />
+    <property name="mapping.css"    location="${dir.app.temp}/${mapping.file.css}" />
+    <property name="mapping.img"    location="${dir.app.temp}/${mapping.file.img}" />
+    <property name="mapping.swf"    location="${dir.app.temp}/${mapping.file.swf}" />
+    <property name="mapping.fonts"  location="${dir.app.temp}/${mapping.file.fonts}" />
+
+    <!-- Tool Directories -->
+    <property name="dir.bin"    location="${dir.build}/Bin" />
+    <property name="dir.jar"    location="${dir.bin}/jar" />
+
+    <!-- Tool Files -->
+	<property name="tools.compressor"     location="${dir.jar}/${tools.file.compressor}" />
+	<property name="tools.cssembed"       location="${dir.jar}/${tools.file.cssembed}" />
+    <property name="tools.filetransform"  location="${dir.jar}/${tools.file.filetransform}" />
+    <property name="tools.optipng"        location="${dir.bin}/${tools.file.optipng}" />
+    <property name="tools.jpegtran"       location="${dir.bin}/${tools.file.jpegtran}" />
+
+
+    <!-- BUILD TARGETS -->
+
+    <!-- low level utility build targets -->
+
+    <!-- Build the tools -->
+    <target name="-setup.build.tools"
+            depends="-define.filetransform, -define.cssembed, -define.yuicompressor, -define.jsclasspath"
+    />
+
+    <!-- set up filesystem properties -->
+    <target
+        name="-setup"
+        depends="-setup.mode, -setup.conditions, -setup.js, -setup.css, -setup.swf, -setup.img, -setup.fonts, -setup.yui"
+    />
+
+    <!-- utility-ish targets -->
+    <target name="copy"         depends="clean, tools, -copy" />
+    <target name="tools"        depends="-setup.build.tools" />
+    <target name="finalize"     depends="copy, -finalize" />
+    <target name="-prepare"     depends="copy, -setup" />
+
+    <!-- individual component build targets (empty descriptions are to make sure they show in "ant -p") -->
+    <target name="devlive"      depends="-prepare, -devlive"            description="" />
+    <target name="js"           depends="-prepare, -js"                 description="" />
+    <target name="css"          depends="-prepare, -css"                description="" />
+    <target name="rename"       depends="-prepare, -rename"             description="" />
+    <target name="yui"          depends="-prepare, rename, -yui"        description="" />
+    <target name="cdn"          depends="-prepare, -cdn"                description="" />
+
+    <!-- high level build targets (Excluding of images is on purpose here, it's slow) -->
+    <target name="core"
+            depends="devlive, js, css, cdn, rename, yui, -js.inline"
+            description="Core build work"
+    />
+
+    <target name="prod"
+            depends="core, finalize"
+            description="Full Production Build"
+    />
+
+    <!-- debug target -->
+    <target name="debug" depends="-setup">
+        <echoproperties/>
+    </target>
+
+</project>

samples/Ant Build System/filenames/build.xml

@@ -0,0 +1 @@
+ant.xml

samples/Assembly/External Interrupt.a51

@@ -0,0 +1,66 @@
+	ORG	0000h
+	SJMP	START
+	ORG	0003h
+	LCALL	INT0_ISR
+	RETI
+	ORG	000Bh
+	LCALL	T0_ISR
+	RETI
+	ORG	0013h
+	LCALL	INT1_ISR
+	RETI
+	ORG	001Bh
+	LCALL	T1_ISR
+	RETI
+	ORG	0023h
+	LCALL	UART_ISR
+	RETI
+	ORG	0030h
+START:
+	MOV	A,#11111110b
+	SETB	IT0	; Set External Interrupt 0 to be falling edge triggered
+	SETB	EX0	; Enable External Interrut 0
+	SETB	EA	; Enable Interrupt
+LEFT:			
+	CJNE	A,#01111111b,LOOP1
+	JMP	RIGHT
+LOOP1:
+	MOV	P1,A
+	RL	A	
+	LCALL	DELAY
+	SJMP	LEFT	
+RIGHT:
+	CJNE	A,#11111110b,LOOP2
+	JMP	LEFT
+LOOP2:
+	MOV	P1,A
+	RR	A	
+	LCALL	DELAY
+	SJMP	RIGHT
+	
+INT0_ISR:
+	MOV	R1,#3
+FLASH:
+	MOV	P1,#00h
+	LCALL	DELAY
+	MOV	P1,#0FFh
+	LCALL	DELAY
+	DJNZ	R1,FLASH
+	RET
+T0_ISR:
+	RET
+INT1_ISR:
+	RET
+T1_ISR:
+	RET
+UART_ISR:
+	RET
+
+DELAY:	MOV	R5,#20	;R5*20 mS
+D1:     MOV	R6,#40
+D2:     MOV	R7,#249
+	DJNZ	R7,$
+ 	DJNZ	R6,D2
+  	DJNZ	R5,D1
+   	RET
+	END

samples/Assembly/FASM.asm

@@ -0,0 +1,350 @@
+
+; flat assembler interface for Win32
+; Copyright (c) 1999-2014, Tomasz Grysztar.
+; All rights reserved.
+
+	format	PE console
+
+section '.text' code readable executable
+
+start:
+
+	mov	[con_handle],STD_OUTPUT_HANDLE
+	mov	esi,_logo
+	call	display_string
+
+	call	get_params
+	jc	information
+
+	call	init_memory
+
+	mov	esi,_memory_prefix
+	call	display_string
+	mov	eax,[memory_end]
+	sub	eax,[memory_start]
+	add	eax,[additional_memory_end]
+	sub	eax,[additional_memory]
+	shr	eax,10
+	call	display_number
+	mov	esi,_memory_suffix
+	call	display_string
+
+	call	[GetTickCount]
+	mov	[start_time],eax
+
+	call	preprocessor
+	call	parser
+	call	assembler
+	call	formatter
+
+	call	display_user_messages
+	movzx	eax,[current_pass]
+	inc	eax
+	call	display_number
+	mov	esi,_passes_suffix
+	call	display_string
+	call	[GetTickCount]
+	sub	eax,[start_time]
+	xor	edx,edx
+	mov	ebx,100
+	div	ebx
+	or	eax,eax
+	jz	display_bytes_count
+	xor	edx,edx
+	mov	ebx,10
+	div	ebx
+	push	edx
+	call	display_number
+	mov	dl,'.'
+	call	display_character
+	pop	eax
+	call	display_number
+	mov	esi,_seconds_suffix
+	call	display_string
+      display_bytes_count:
+	mov	eax,[written_size]
+	call	display_number
+	mov	esi,_bytes_suffix
+	call	display_string
+	xor	al,al
+	jmp	exit_program
+
+information:
+	mov	esi,_usage
+	call	display_string
+	mov	al,1
+	jmp	exit_program
+
+get_params:
+	mov	[input_file],0
+	mov	[output_file],0
+	mov	[symbols_file],0
+	mov	[memory_setting],0
+	mov	[passes_limit],100
+	call	[GetCommandLine]
+	mov	esi,eax
+	mov	edi,params
+    find_command_start:
+	lodsb
+	cmp	al,20h
+	je	find_command_start
+	cmp	al,22h
+	je	skip_quoted_name
+    skip_name:
+	lodsb
+	cmp	al,20h
+	je	find_param
+	or	al,al
+	jz	all_params
+	jmp	skip_name
+    skip_quoted_name:
+	lodsb
+	cmp	al,22h
+	je	find_param
+	or	al,al
+	jz	all_params
+	jmp	skip_quoted_name
+    find_param:
+	lodsb
+	cmp	al,20h
+	je	find_param
+	cmp	al,'-'
+	je	option_param
+	cmp	al,0Dh
+	je	all_params
+	or	al,al
+	jz	all_params
+	cmp	[input_file],0
+	jne	get_output_file
+	mov	[input_file],edi
+	jmp	process_param
+      get_output_file:
+	cmp	[output_file],0
+	jne	bad_params
+	mov	[output_file],edi
+    process_param:
+	cmp	al,22h
+	je	string_param
+    copy_param:
+	stosb
+	lodsb
+	cmp	al,20h
+	je	param_end
+	cmp	al,0Dh
+	je	param_end
+	or	al,al
+	jz	param_end
+	jmp	copy_param
+    string_param:
+	lodsb
+	cmp	al,22h
+	je	string_param_end
+	cmp	al,0Dh
+	je	param_end
+	or	al,al
+	jz	param_end
+	stosb
+	jmp	string_param
+    option_param:
+	lodsb
+	cmp	al,'m'
+	je	memory_option
+	cmp	al,'M'
+	je	memory_option
+	cmp	al,'p'
+	je	passes_option
+	cmp	al,'P'
+	je	passes_option
+	cmp	al,'s'
+	je	symbols_option
+	cmp	al,'S'
+	je	symbols_option
+    bad_params:
+	stc
+	ret
+    get_option_value:
+	xor	eax,eax
+	mov	edx,eax
+    get_option_digit:
+	lodsb
+	cmp	al,20h
+	je	option_value_ok
+	cmp	al,0Dh
+	je	option_value_ok
+	or	al,al
+	jz	option_value_ok
+	sub	al,30h
+	jc	invalid_option_value
+	cmp	al,9
+	ja	invalid_option_value
+	imul	edx,10
+	jo	invalid_option_value
+	add	edx,eax
+	jc	invalid_option_value
+	jmp	get_option_digit
+    option_value_ok:
+	dec	esi
+	clc
+	ret
+    invalid_option_value:
+	stc
+	ret
+    memory_option:
+	lodsb
+	cmp	al,20h
+	je	memory_option
+	cmp	al,0Dh
+	je	bad_params
+	or	al,al
+	jz	bad_params
+	dec	esi
+	call	get_option_value
+	or	edx,edx
+	jz	bad_params
+	cmp	edx,1 shl (32-10)
+	jae	bad_params
+	mov	[memory_setting],edx
+	jmp	find_param
+    passes_option:
+	lodsb
+	cmp	al,20h
+	je	passes_option
+	cmp	al,0Dh
+	je	bad_params
+	or	al,al
+	jz	bad_params
+	dec	esi
+	call	get_option_value
+	or	edx,edx
+	jz	bad_params
+	cmp	edx,10000h
+	ja	bad_params
+	mov	[passes_limit],dx
+	jmp	find_param
+    symbols_option:
+	mov	[symbols_file],edi
+      find_symbols_file_name:
+	lodsb
+	cmp	al,20h
+	jne	process_param
+	jmp	find_symbols_file_name
+    param_end:
+	dec	esi
+    string_param_end:
+	xor	al,al
+	stosb
+	jmp	find_param
+    all_params:
+	cmp	[input_file],0
+	je	bad_params
+	clc
+	ret
+
+include 'system.inc'
+
+include '..\errors.inc'
+include '..\symbdump.inc'
+include '..\preproce.inc'
+include '..\parser.inc'
+include '..\exprpars.inc'
+include '..\assemble.inc'
+include '..\exprcalc.inc'
+include '..\formats.inc'
+include '..\x86_64.inc'
+include '..\avx.inc'
+
+include '..\tables.inc'
+include '..\messages.inc'
+
+section '.data' data readable writeable
+
+include '..\version.inc'
+
+_copyright db 'Copyright (c) 1999-2014, Tomasz Grysztar',0Dh,0Ah,0
+
+_logo db 'flat assembler  version ',VERSION_STRING,0
+_usage db 0Dh,0Ah
+       db 'usage: fasm <source> [output]',0Dh,0Ah
+       db 'optional settings:',0Dh,0Ah
+       db ' -m <limit>    set the limit in kilobytes for the available memory',0Dh,0Ah
+       db ' -p <limit>    set the maximum allowed number of passes',0Dh,0Ah
+       db ' -s <file>     dump symbolic information for debugging',0Dh,0Ah
+       db 0
+_memory_prefix db '  (',0
+_memory_suffix db ' kilobytes memory)',0Dh,0Ah,0
+_passes_suffix db ' passes, ',0
+_seconds_suffix db ' seconds, ',0
+_bytes_suffix db ' bytes.',0Dh,0Ah,0
+
+align 4
+
+include '..\variable.inc'
+
+con_handle dd ?
+memory_setting dd ?
+start_time dd ?
+bytes_count dd ?
+displayed_count dd ?
+character db ?
+last_displayed rb 2
+
+params rb 1000h
+options rb 1000h
+buffer rb 4000h
+
+stack 10000h
+
+section '.idata' import data readable writeable
+
+  dd 0,0,0,rva kernel_name,rva kernel_table
+  dd 0,0,0,0,0
+
+  kernel_table:
+    ExitProcess dd rva _ExitProcess
+    CreateFile dd rva _CreateFileA
+    ReadFile dd rva _ReadFile
+    WriteFile dd rva _WriteFile
+    CloseHandle dd rva _CloseHandle
+    SetFilePointer dd rva _SetFilePointer
+    GetCommandLine dd rva _GetCommandLineA
+    GetEnvironmentVariable dd rva _GetEnvironmentVariable
+    GetStdHandle dd rva _GetStdHandle
+    VirtualAlloc dd rva _VirtualAlloc
+    VirtualFree dd rva _VirtualFree
+    GetTickCount dd rva _GetTickCount
+    GetSystemTime dd rva _GetSystemTime
+    GlobalMemoryStatus dd rva _GlobalMemoryStatus
+    dd 0
+
+  kernel_name db 'KERNEL32.DLL',0
+
+  _ExitProcess dw 0
+    db 'ExitProcess',0
+  _CreateFileA dw 0
+    db 'CreateFileA',0
+  _ReadFile dw 0
+    db 'ReadFile',0
+  _WriteFile dw 0
+    db 'WriteFile',0
+  _CloseHandle dw 0
+    db 'CloseHandle',0
+  _SetFilePointer dw 0
+    db 'SetFilePointer',0
+  _GetCommandLineA dw 0
+    db 'GetCommandLineA',0
+  _GetEnvironmentVariable dw 0
+    db 'GetEnvironmentVariableA',0
+  _GetStdHandle dw 0
+    db 'GetStdHandle',0
+  _VirtualAlloc dw 0
+    db 'VirtualAlloc',0
+  _VirtualFree dw 0
+    db 'VirtualFree',0
+  _GetTickCount dw 0
+    db 'GetTickCount',0
+  _GetSystemTime dw 0
+    db 'GetSystemTime',0
+  _GlobalMemoryStatus dw 0
+    db 'GlobalMemoryStatus',0
+
+section '.reloc' fixups data readable discardable

samples/BlitzMax/sample.bmx

@@ -0,0 +1,25 @@
+SuperStrict
+
+Framework Brl.StandardIO
+
+Type TMyType
+	Field property:int
+
+	Function A:int(param:int)
+		'do nothing
+	End Function
+
+	Method B:int(param:int)
+		'do nothing
+	End Method
+End Type
+
+
+Global my:TMyType = new TMyType
+?Win32
+	my.A()
+	my.B()
+?Linux
+	my.B()
+	my.A()
+?

samples/C++/16F88.h

@@ -0,0 +1,86 @@
+/*
+ * This file is part of PIC
+ * Copyright © 2012 Rachel Mant (dx-mon@users.sourceforge.net)
+ *
+ * PIC is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Lesser General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * PIC is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+enum PIC16F88Instruction
+{
+	ADDWF,
+	ANDWF,
+	CLRF,
+	CLRW,
+	COMF,
+	DECF,
+	DECFSZ,
+	INCF,
+	INCFSZ,
+	IORWF,
+	MOVF,
+	MOVWF,
+	NOP,
+	RLF,
+	RRF,
+	SUBWF,
+	SWAPF,
+	XORWF,
+	BCF,
+	BSF,
+	BTFSC,
+	BTFSS,
+	ADDLW,
+	ANDLW,
+	CALL,
+	CLRWDT,
+	GOTO,
+	IORLW,
+	MOVLW,
+	RETFIE,
+	RETLW,
+	RETURN,
+	SLEEP,
+	SUBLW,
+	XORLW
+};
+
+class PIC16F88
+{
+public:
+	PIC16F88(ROM *ProgramMemory);
+	void Step();
+
+private:
+	uint8_t q;
+	bool nextIsNop, trapped;
+	Memory *memory;
+	ROM *program;
+	Stack<uint16_t, 8> *CallStack;
+	Register<uint16_t> *PC;
+	Register<> *WREG, *PCL, *STATUS, *PCLATCH;
+	PIC16F88Instruction inst;
+	uint16_t instrWord;
+
+private:
+	void DecodeInstruction();
+	void ProcessInstruction();
+
+	uint8_t GetBank();
+	uint8_t GetMemoryContents(uint8_t partialAddress);
+	void SetMemoryContents(uint8_t partialAddress, uint8_t newVal);
+	void CheckZero(uint8_t value);
+	void StoreValue(uint8_t value, bool updateZero);
+	uint8_t SetCarry(bool val);
+	uint16_t GetPCHFinalBits();
+};

samples/C++/Memory16F88.h

@@ -0,0 +1,32 @@
+/*
+ * This file is part of PIC
+ * Copyright © 2012 Rachel Mant (dx-mon@users.sourceforge.net)
+ *
+ * PIC is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Lesser General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * PIC is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "Memory.h"
+
+class Memory16F88 : public Memory
+{
+private:
+	uint8_t memory[512];
+	std::map<uint32_t, MemoryLocation *> memoryMap;
+
+public:
+	Memory16F88();
+	uint8_t Dereference(uint8_t bank, uint8_t partialAddress);
+	uint8_t *Reference(uint8_t bank, uint8_t partialAddress);
+	uint8_t *operator [](uint32_t ref);
+};

samples/C++/ThreadedQueue.h

@@ -0,0 +1,76 @@
+/*
+ * This file is part of IRCBot
+ * Copyright © 2014 Rachel Mant (dx-mon@users.sourceforge.net)
+ *
+ * IRCBot is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * IRCBot is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __THREADED_QUEUE_H__
+#define __THREADED_QUEUE_H__
+
+#include <pthread.h>
+#include <queue>
+
+template<class T>
+class ThreadedQueue : public std::queue<T>
+{
+private:
+	pthread_mutex_t queueMutex;
+	pthread_cond_t queueCond;
+
+public:
+	ThreadedQueue()
+	{
+		pthread_mutexattr_t mutexAttrs;
+		pthread_condattr_t condAttrs;
+
+		pthread_mutexattr_init(&mutexAttrs);
+		pthread_mutexattr_settype(&mutexAttrs, PTHREAD_MUTEX_ERRORCHECK);
+		pthread_mutex_init(&queueMutex, &mutexAttrs);
+		pthread_mutexattr_destroy(&mutexAttrs);
+
+		pthread_condattr_init(&condAttrs);
+		pthread_condattr_setpshared(&condAttrs, PTHREAD_PROCESS_PRIVATE);
+		pthread_cond_init(&queueCond, &condAttrs);
+		pthread_condattr_destroy(&condAttrs);
+	}
+
+	~ThreadedQueue()
+	{
+		pthread_cond_destroy(&queueCond);
+		pthread_mutex_destroy(&queueMutex);
+	}
+
+	void waitItems()
+	{
+		pthread_mutex_lock(&queueMutex);
+		pthread_cond_wait(&queueCond, &queueMutex);
+		pthread_mutex_unlock(&queueMutex);
+	}
+
+	void signalItems()
+	{
+		pthread_mutex_lock(&queueMutex);
+		pthread_cond_broadcast(&queueCond);
+		pthread_mutex_unlock(&queueMutex);
+	}
+
+	void push(T item)
+	{
+		std::queue<T>::push(item);
+		signalItems();
+	}
+};
+
+#endif /*__THREADED_QUEUE_H__*/

samples/C++/bar.hh

@@ -0,0 +1,10 @@
+class Bar
+{
+  protected:
+
+    char *name;
+
+  public:
+
+    void hello();
+}

samples/C/2D.C

@@ -0,0 +1,145 @@
+#include "2D.h"
+#include <math.h>
+
+void set_vgabasemem(void)
+{
+    ULONG vgabase;
+    SELECTOR tmp;
+    asm mov [tmp], ds
+    dpmi_get_sel_base(&vgabase, tmp);
+    vgabasemem = (char *)(-vgabase + 0xa0000);
+}
+
+void drw_chdis(int mode) // change the display!
+{
+    regs.b.ah = 0x00;   // seet theh display moode
+    regs.b.al = mode;  // change it to the mode like innit
+    regs.h.flags = 0x72;// Set the dingoes kidneys out of FLAGS eh?
+    regs.h.ss = 0;     // Like, totally set the stack segment
+    regs.h.sp = 0;     // Set tha stack pointaaaaahhhhh!!!
+    dpmi_simulate_real_interrupt(0x10, &regs);
+}
+
+void drw_pix(int x, int y, enum COLORS col)
+{
+    *VGAPIX(x, y) = col;
+}
+
+void drw_line(int x0, int y0, int x1, int y1, enum COLORS col)
+{
+    // Going for the optimized version of bresenham's line algo.        
+    int stp = (abs(y0 - y1) > abs(x0 - x1));
+    int tmp, dx, dy, err, yi, i, j; // yi = y excrement
+    if (stp) {
+        // swappity swap
+        tmp = y0;
+        y0 = x0;
+        x0 = tmp;
+        
+        tmp = y1;
+        y1 = x1;
+        x1 = tmp;
+    }
+    // AAAAND NOW WE MUST DO ZEES AGAIN :(
+    // I'm sure there was a func somewhere that does this? :P
+    if (x0 > x1) {
+        tmp = x0;
+        x0 = x1;
+        x1 = tmp;
+        
+        tmp = y0;
+        y0 = y1;
+        y1 = tmp;
+    }
+    dx = (x1 - x0);
+    dy = (abs(y1 - y0));
+    err = (dx / 2);
+
+    if (y0 < y1)
+        yi = 1;
+    else
+        yi = -1;
+    j = y0;
+    for (i = x0; i < x1; i++)
+    {
+        if (stp)
+            *VGAPIX(j, i) = col;
+        else
+            *VGAPIX(i, j) = col;
+
+        err -= dy;
+        if (err < 0) {
+            j += yi;
+            err += dx;
+        }
+    }
+}
+
+void drw_rectl(int x, int y, int w, int h, enum COLORS col)
+{
+    drw_line(x, y, x+w, y, col);
+    drw_line(x+w, y, x+w, y+h, col);
+
+    drw_line(x, y, x, y+h, col);
+    drw_line(x, y+h, x+w+1, y+h, col);
+}
+
+void drw_rectf(int x, int y, int w, int h, enum COLORS col)
+{
+    int i, j;
+    for (j = y; j < x+h; j++) {
+        for (i = x; i < y+w; i++) {
+            *VGAPIX(i, j) = col;
+        }
+    }
+}
+
+void drw_circl(int x, int y, int rad, enum COLORS col)
+{
+    int mang, i; // max angle, haha
+    int px, py;
+    mang = 360; // Yeah yeah I'll switch to rad later
+    for (i = 0; i <= mang; i++)
+    {   
+        px = cos(i)*rad + x; // + px; // causes some really cools effects! :D
+        py = sin(i)*rad + y; // + py;
+        *VGAPIX(px, py) = col;
+    }
+}
+
+void drw_tex(int x, int y, int w, int h, enum COLORS tex[])
+{   // i*w+j
+    int i, j;
+    for (i = 0; i < w; i++)
+    {
+        for (j = 0; j < h; j++)
+        {
+            *VGAPIX(x+i, y+j) = tex[j*w+i];
+        }
+    }
+}
+
+void 2D_init(void)
+{
+    set_vgabasemem();
+    drw_chdis(0x13);
+}
+
+void 2D_exit(void)
+{
+    drw_chdis(3);
+}
+/*
+int main()
+{
+    set_vgabasemem();
+    drw_chdis(0x13);
+
+    while(!kbhit()) {
+        if ((getch()) == 0x1b) // escape
+            break;
+    }
+    drw_chdis(3);
+    return 0;
+}
+*/

samples/C/2D.H

@@ -0,0 +1,29 @@
+#ifndef __2DGFX
+#define __2DGFX
+// Includes
+#include <stdio.h>
+#include <math.h>
+#include <conio.h>
+#include <dpmi.h>
+
+// Defines
+#define VGAPIX(x,y) (vgabasemem + (x) + (y) * 320)
+
+// Variables
+char * vgabasemem;
+DPMI_REGS regs;
+
+// Drawing functions:
+//void setvgabasemem(void);
+void drw_chdis(int mode); // draw_func_change_display
+void drw_pix(int x, int y, enum COLORS col);
+void drw_line(int x0, int y0, int x1, int y1, enum COLORS col);
+void drw_rectl(int x, int y, int w, int h, enum COLORS col);
+void drw_rectf(int x, int y, int w, int h, enum COLORS col);
+void drw_cirl(int x, int y, int rad, enum COLORS col);
+void drw_tex(int x, int y, int w, int h, enum COLORS tex[]);
+void 2D_init(void);
+void 2D_exit(void);
+
+
+#endif

samples/C/ArrowLeft.h

@@ -0,0 +1,93 @@
+/*
+ * This file is part of GTK++ (libGTK++)
+ * Copyright © 2012 Rachel Mant (dx-mon@users.sourceforge.net)
+ *
+ * GTK++ is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Lesser General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * GTK++ is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+/* GdkPixbuf RGBA C-Source image dump */
+
+#ifdef __SUNPRO_C
+#pragma align 4 (ArrowLeft)
+#endif
+#ifdef __GNUC__
+static const uint8_t ArrowLeft[] __attribute__ ((__aligned__ (4))) = 
+#else
+static const uint8_t ArrowLeft[] = 
+#endif
+{ ""
+  /* Pixbuf magic (0x47646b50) */
+  "GdkP"
+  /* length: header (24) + pixel_data (1600) */
+  "\0\0\6X"
+  /* pixdata_type (0x1010002) */
+  "\1\1\0\2"
+  /* rowstride (80) */
+  "\0\0\0P"
+  /* width (20) */
+  "\0\0\0\24"
+  /* height (20) */
+  "\0\0\0\24"
+  /* pixel_data: */
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\377\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\377\0\0\0\377\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\377\0\0\0\377\0\0\0\377\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\377\0\0\0\377\0\0\0\377"
+  "\0\0\0\377\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\377\0\0\0\377\0\0\0\377\0\0\0\377\0\0\0\377\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\377\0\0\0\377\0\0\0\377\0\0\0\377\0\0\0\377"
+  "\0\0\0\377\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\377\0\0\0\377\0\0\0\377\0\0\0\377\0\0\0\377\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\377\0\0\0\377\0\0\0\377\0"
+  "\0\0\377\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\377\0\0\0\377\0\0\0\377\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\377\0\0\0\377\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\377\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
+  "\0\0"};
+
+

samples/C/GLKMatrix4.h

@@ -0,0 +1,903 @@
+//
+//  GLKMatrix4.h
+//  GLKit
+//
+//  Copyright (c) 2011, Apple Inc. All rights reserved.
+//
+
+#ifndef __GLK_MATRIX_4_H
+#define __GLK_MATRIX_4_H
+
+#include <stddef.h>
+#include <stdbool.h>
+#include <math.h>
+
+#if defined(__ARM_NEON__)
+#include <arm_neon.h>
+#endif
+
+#include <GLKit/GLKMathTypes.h>
+#include <GLKit/GLKVector3.h>
+#include <GLKit/GLKVector4.h>
+#include <GLKit/GLKQuaternion.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#pragma mark -
+#pragma mark Prototypes
+#pragma mark -
+
+extern const GLKMatrix4 GLKMatrix4Identity;
+
+/*
+ m30, m31, and m32 correspond to the translation values tx, ty, tz, respectively.
+ */
+static __inline__ GLKMatrix4 GLKMatrix4Make(float m00, float m01, float m02, float m03,
+                                            float m10, float m11, float m12, float m13,
+                                            float m20, float m21, float m22, float m23,
+                                            float m30, float m31, float m32, float m33);
+
+/*
+ m03, m13, and m23 correspond to the translation values tx, ty, tz, respectively.
+ */
+static __inline__ GLKMatrix4 GLKMatrix4MakeAndTranspose(float m00, float m01, float m02, float m03,
+                                                        float m10, float m11, float m12, float m13,
+                                                        float m20, float m21, float m22, float m23,
+                                                        float m30, float m31, float m32, float m33);
+
+/*
+ m[12], m[13], and m[14] correspond to the translation values tx, ty, and tz, respectively.
+ */
+static __inline__ GLKMatrix4 GLKMatrix4MakeWithArray(float values[16]);
+
+/*
+ m[3], m[7], and m[11] correspond to the translation values tx, ty, and tz, respectively.
+ */
+static __inline__ GLKMatrix4 GLKMatrix4MakeWithArrayAndTranspose(float values[16]);
+
+/*
+ row0, row1, and row2's last component should correspond to the translation values tx, ty, and tz, respectively.
+ */
+static __inline__ GLKMatrix4 GLKMatrix4MakeWithRows(GLKVector4 row0,
+                                                    GLKVector4 row1,
+                                                    GLKVector4 row2,
+                                                    GLKVector4 row3);
+
+/*
+ column3's first three components should correspond to the translation values tx, ty, and tz.
+ */
+static __inline__ GLKMatrix4 GLKMatrix4MakeWithColumns(GLKVector4 column0,
+                                                       GLKVector4 column1,
+                                                       GLKVector4 column2,
+                                                       GLKVector4 column3);
+
+/*
+ The quaternion will be normalized before conversion.
+ */
+static __inline__ GLKMatrix4 GLKMatrix4MakeWithQuaternion(GLKQuaternion quaternion);
+
+static __inline__ GLKMatrix4 GLKMatrix4MakeTranslation(float tx, float ty, float tz);
+static __inline__ GLKMatrix4 GLKMatrix4MakeScale(float sx, float sy, float sz);
+static __inline__ GLKMatrix4 GLKMatrix4MakeRotation(float radians, float x, float y, float z);
+
+static __inline__ GLKMatrix4 GLKMatrix4MakeXRotation(float radians);
+static __inline__ GLKMatrix4 GLKMatrix4MakeYRotation(float radians);
+static __inline__ GLKMatrix4 GLKMatrix4MakeZRotation(float radians);
+
+/*
+ Equivalent to gluPerspective.
+ */
+static __inline__ GLKMatrix4 GLKMatrix4MakePerspective(float fovyRadians, float aspect, float nearZ, float farZ);
+
+/*
+ Equivalent to glFrustum.
+ */
+static __inline__ GLKMatrix4 GLKMatrix4MakeFrustum(float left, float right,
+                                                   float bottom, float top,
+                                                   float nearZ, float farZ);
+
+/*
+ Equivalent to glOrtho.
+ */
+static __inline__ GLKMatrix4 GLKMatrix4MakeOrtho(float left, float right,
+                                                 float bottom, float top,
+                                                 float nearZ, float farZ);
+
+/*
+ Equivalent to gluLookAt.
+ */
+static __inline__ GLKMatrix4 GLKMatrix4MakeLookAt(float eyeX, float eyeY, float eyeZ,
+                                                  float centerX, float centerY, float centerZ,
+                                                  float upX, float upY, float upZ);
+
+/*
+ Returns the upper left 3x3 portion of the 4x4 matrix.
+ */
+static __inline__ GLKMatrix3 GLKMatrix4GetMatrix3(GLKMatrix4 matrix);
+/*
+ Returns the upper left 2x2 portion of the 4x4 matrix.
+ */
+static __inline__ GLKMatrix2 GLKMatrix4GetMatrix2(GLKMatrix4 matrix);
+
+/*
+ GLKMatrix4GetRow returns vectors for rows 0, 1, and 2 whose last component will be the translation value tx, ty, and tz, respectively.
+ Valid row values range from 0 to 3, inclusive.
+ */
+static __inline__ GLKVector4 GLKMatrix4GetRow(GLKMatrix4 matrix, int row);
+/*
+ GLKMatrix4GetColumn returns a vector for column 3 whose first three components will be the translation values tx, ty, and tz.
+ Valid column values range from 0 to 3, inclusive.
+ */
+static __inline__ GLKVector4 GLKMatrix4GetColumn(GLKMatrix4 matrix, int column);
+
+/*
+ GLKMatrix4SetRow expects that the vector for row 0, 1, and 2 will have a translation value as its last component.
+ Valid row values range from 0 to 3, inclusive.
+ */
+static __inline__ GLKMatrix4 GLKMatrix4SetRow(GLKMatrix4 matrix, int row, GLKVector4 vector);
+/*
+ GLKMatrix4SetColumn expects that the vector for column 3 will contain the translation values tx, ty, and tz as its first three components, respectively.
+ Valid column values range from 0 to 3, inclusive.
+ */
+static __inline__ GLKMatrix4 GLKMatrix4SetColumn(GLKMatrix4 matrix, int column, GLKVector4 vector);
+
+static __inline__ GLKMatrix4 GLKMatrix4Transpose(GLKMatrix4 matrix);
+
+GLKMatrix4 GLKMatrix4Invert(GLKMatrix4 matrix, bool *isInvertible);
+GLKMatrix4 GLKMatrix4InvertAndTranspose(GLKMatrix4 matrix, bool *isInvertible);
+
+static __inline__ GLKMatrix4 GLKMatrix4Multiply(GLKMatrix4 matrixLeft, GLKMatrix4 matrixRight);
+
+static __inline__ GLKMatrix4 GLKMatrix4Add(GLKMatrix4 matrixLeft, GLKMatrix4 matrixRight);
+static __inline__ GLKMatrix4 GLKMatrix4Subtract(GLKMatrix4 matrixLeft, GLKMatrix4 matrixRight);
+
+static __inline__ GLKMatrix4 GLKMatrix4Translate(GLKMatrix4 matrix, float tx, float ty, float tz);
+static __inline__ GLKMatrix4 GLKMatrix4TranslateWithVector3(GLKMatrix4 matrix, GLKVector3 translationVector);
+/*
+ The last component of the GLKVector4, translationVector, is ignored.
+ */
+static __inline__ GLKMatrix4 GLKMatrix4TranslateWithVector4(GLKMatrix4 matrix, GLKVector4 translationVector);
+
+static __inline__ GLKMatrix4 GLKMatrix4Scale(GLKMatrix4 matrix, float sx, float sy, float sz);
+static __inline__ GLKMatrix4 GLKMatrix4ScaleWithVector3(GLKMatrix4 matrix, GLKVector3 scaleVector);
+/*
+ The last component of the GLKVector4, scaleVector, is ignored.
+ */
+static __inline__ GLKMatrix4 GLKMatrix4ScaleWithVector4(GLKMatrix4 matrix, GLKVector4 scaleVector);
+
+static __inline__ GLKMatrix4 GLKMatrix4Rotate(GLKMatrix4 matrix, float radians, float x, float y, float z);
+static __inline__ GLKMatrix4 GLKMatrix4RotateWithVector3(GLKMatrix4 matrix, float radians, GLKVector3 axisVector);
+/*
+ The last component of the GLKVector4, axisVector, is ignored.
+ */
+static __inline__ GLKMatrix4 GLKMatrix4RotateWithVector4(GLKMatrix4 matrix, float radians, GLKVector4 axisVector);
+
+static __inline__ GLKMatrix4 GLKMatrix4RotateX(GLKMatrix4 matrix, float radians);
+static __inline__ GLKMatrix4 GLKMatrix4RotateY(GLKMatrix4 matrix, float radians);
+static __inline__ GLKMatrix4 GLKMatrix4RotateZ(GLKMatrix4 matrix, float radians);
+
+/*
+ Assumes 0 in the w component.
+ */
+static __inline__ GLKVector3 GLKMatrix4MultiplyVector3(GLKMatrix4 matrixLeft, GLKVector3 vectorRight);
+/*
+ Assumes 1 in the w component.
+ */
+static __inline__ GLKVector3 GLKMatrix4MultiplyVector3WithTranslation(GLKMatrix4 matrixLeft, GLKVector3 vectorRight);
+/*
+ Assumes 1 in the w component and divides the resulting vector by w before returning.
+ */
+static __inline__ GLKVector3 GLKMatrix4MultiplyAndProjectVector3(GLKMatrix4 matrixLeft, GLKVector3 vectorRight);
+
+/*
+ Assumes 0 in the w component.
+ */
+static __inline__ void GLKMatrix4MultiplyVector3Array(GLKMatrix4 matrix, GLKVector3 *vectors, size_t vectorCount);
+/*
+ Assumes 1 in the w component.
+ */
+static __inline__ void GLKMatrix4MultiplyVector3ArrayWithTranslation(GLKMatrix4 matrix, GLKVector3 *vectors, size_t vectorCount);
+/*
+ Assumes 1 in the w component and divides the resulting vector by w before returning.
+ */
+static __inline__ void GLKMatrix4MultiplyAndProjectVector3Array(GLKMatrix4 matrix, GLKVector3 *vectors, size_t vectorCount);
+
+static __inline__ GLKVector4 GLKMatrix4MultiplyVector4(GLKMatrix4 matrixLeft, GLKVector4 vectorRight);
+
+static __inline__ void GLKMatrix4MultiplyVector4Array(GLKMatrix4 matrix, GLKVector4 *vectors, size_t vectorCount);
+
+#pragma mark -
+#pragma mark Implementations
+#pragma mark -
+
+static __inline__ GLKMatrix4 GLKMatrix4Make(float m00, float m01, float m02, float m03,
+                                            float m10, float m11, float m12, float m13,
+                                            float m20, float m21, float m22, float m23,
+                                            float m30, float m31, float m32, float m33)
+{
+    GLKMatrix4 m = { m00, m01, m02, m03,
+                     m10, m11, m12, m13,
+                     m20, m21, m22, m23,
+                     m30, m31, m32, m33 };
+    return m;
+}
+
+static __inline__ GLKMatrix4 GLKMatrix4MakeAndTranspose(float m00, float m01, float m02, float m03,
+                                                        float m10, float m11, float m12, float m13,
+                                                        float m20, float m21, float m22, float m23,
+                                                        float m30, float m31, float m32, float m33)
+{
+    GLKMatrix4 m = { m00, m10, m20, m30,
+                     m01, m11, m21, m31,
+                     m02, m12, m22, m32,
+                     m03, m13, m23, m33 };
+    return m;
+}
+
+static __inline__ GLKMatrix4 GLKMatrix4MakeWithArray(float values[16])
+{
+    GLKMatrix4 m = { values[0], values[1], values[2], values[3],
+                     values[4], values[5], values[6], values[7],
+                     values[8], values[9], values[10], values[11],
+                     values[12], values[13], values[14], values[15] };
+    return m;
+}
+
+static __inline__ GLKMatrix4 GLKMatrix4MakeWithArrayAndTranspose(float values[16])
+{
+#if defined(__ARM_NEON__)
+    float32x4x4_t m = vld4q_f32(values);
+    return *(GLKMatrix4 *)&m;
+#else
+    GLKMatrix4 m = { values[0], values[4], values[8], values[12],
+                     values[1], values[5], values[9], values[13],
+                     values[2], values[6], values[10], values[14],
+                     values[3], values[7], values[11], values[15] };
+    return m;
+#endif
+}
+
+static __inline__ GLKMatrix4 GLKMatrix4MakeWithRows(GLKVector4 row0,
+                                                    GLKVector4 row1,
+                                                    GLKVector4 row2,
+                                                    GLKVector4 row3)
+{
+    GLKMatrix4 m = { row0.v[0], row1.v[0], row2.v[0], row3.v[0],
+                     row0.v[1], row1.v[1], row2.v[1], row3.v[1],
+                     row0.v[2], row1.v[2], row2.v[2], row3.v[2],
+                     row0.v[3], row1.v[3], row2.v[3], row3.v[3] };
+    return m;
+}
+
+static __inline__ GLKMatrix4 GLKMatrix4MakeWithColumns(GLKVector4 column0,
+                                                       GLKVector4 column1,
+                                                       GLKVector4 column2,
+                                                       GLKVector4 column3)
+{
+#if defined(__ARM_NEON__)
+    float32x4x4_t m;
+    m.val[0] = vld1q_f32(column0.v);
+    m.val[1] = vld1q_f32(column1.v);
+    m.val[2] = vld1q_f32(column2.v);
+    m.val[3] = vld1q_f32(column3.v);
+    return *(GLKMatrix4 *)&m;
+#else
+    GLKMatrix4 m = { column0.v[0], column0.v[1], column0.v[2], column0.v[3],
+                     column1.v[0], column1.v[1], column1.v[2], column1.v[3],
+                     column2.v[0], column2.v[1], column2.v[2], column2.v[3],
+                     column3.v[0], column3.v[1], column3.v[2], column3.v[3] };
+    return m;
+#endif
+}
+
+static __inline__ GLKMatrix4 GLKMatrix4MakeWithQuaternion(GLKQuaternion quaternion)
+{
+    quaternion = GLKQuaternionNormalize(quaternion);
+
+    float x = quaternion.q[0];
+    float y = quaternion.q[1];
+    float z = quaternion.q[2];
+    float w = quaternion.q[3];
+
+    float _2x = x + x;
+    float _2y = y + y;
+    float _2z = z + z;
+    float _2w = w + w;
+
+    GLKMatrix4 m = { 1.0f - _2y * y - _2z * z,
+                     _2x * y + _2w * z,
+                     _2x * z - _2w * y,
+                     0.0f,
+                     _2x * y - _2w * z,
+                     1.0f - _2x * x - _2z * z,
+                     _2y * z + _2w * x,
+                     0.0f,
+                     _2x * z + _2w * y,
+                     _2y * z - _2w * x,
+                     1.0f - _2x * x - _2y * y,
+                     0.0f,
+                     0.0f,
+                     0.0f,
+                     0.0f,
+                     1.0f };
+
+    return m;
+}
+
+static __inline__ GLKMatrix4 GLKMatrix4MakeTranslation(float tx, float ty, float tz)
+{
+    GLKMatrix4 m = GLKMatrix4Identity;
+    m.m[12] = tx;
+    m.m[13] = ty;
+    m.m[14] = tz;
+    return m;
+}
+
+static __inline__ GLKMatrix4 GLKMatrix4MakeScale(float sx, float sy, float sz)
+{
+    GLKMatrix4 m = GLKMatrix4Identity;
+    m.m[0] = sx;
+    m.m[5] = sy;
+    m.m[10] = sz;
+    return m;
+}
+
+static __inline__ GLKMatrix4 GLKMatrix4MakeRotation(float radians, float x, float y, float z)
+{
+    GLKVector3 v = GLKVector3Normalize(GLKVector3Make(x, y, z));
+    float cos = cosf(radians);
+    float cosp = 1.0f - cos;
+    float sin = sinf(radians);
+
+    GLKMatrix4 m = { cos + cosp * v.v[0] * v.v[0],
+                     cosp * v.v[0] * v.v[1] + v.v[2] * sin,
+                     cosp * v.v[0] * v.v[2] - v.v[1] * sin,
+                     0.0f,
+                     cosp * v.v[0] * v.v[1] - v.v[2] * sin,
+                     cos + cosp * v.v[1] * v.v[1],
+                     cosp * v.v[1] * v.v[2] + v.v[0] * sin,
+                     0.0f,
+                     cosp * v.v[0] * v.v[2] + v.v[1] * sin,
+                     cosp * v.v[1] * v.v[2] - v.v[0] * sin,
+                     cos + cosp * v.v[2] * v.v[2],
+                     0.0f,
+                     0.0f,
+                     0.0f,
+                     0.0f,
+                     1.0f };
+
+    return m;
+}
+
+static __inline__ GLKMatrix4 GLKMatrix4MakeXRotation(float radians)
+{
+    float cos = cosf(radians);
+    float sin = sinf(radians);
+
+    GLKMatrix4 m = { 1.0f, 0.0f, 0.0f, 0.0f,
+                     0.0f, cos, sin, 0.0f,
+                     0.0f, -sin, cos, 0.0f,
+                     0.0f, 0.0f, 0.0f, 1.0f };
+
+    return m;
+}
+
+static __inline__ GLKMatrix4 GLKMatrix4MakeYRotation(float radians)
+{
+    float cos = cosf(radians);
+    float sin = sinf(radians);
+
+    GLKMatrix4 m = { cos, 0.0f, -sin, 0.0f,
+                     0.0f, 1.0f, 0.0f, 0.0f,
+                     sin, 0.0f, cos, 0.0f,
+                     0.0f, 0.0f, 0.0f, 1.0f };
+
+    return m;
+}
+
+static __inline__ GLKMatrix4 GLKMatrix4MakeZRotation(float radians)
+{
+    float cos = cosf(radians);
+    float sin = sinf(radians);
+
+    GLKMatrix4 m = { cos, sin, 0.0f, 0.0f,
+                     -sin, cos, 0.0f, 0.0f,
+                     0.0f, 0.0f, 1.0f, 0.0f,
+                     0.0f, 0.0f, 0.0f, 1.0f };
+
+    return m;
+}
+
+static __inline__ GLKMatrix4 GLKMatrix4MakePerspective(float fovyRadians, float aspect, float nearZ, float farZ)
+{
+    float cotan = 1.0f / tanf(fovyRadians / 2.0f);
+
+    GLKMatrix4 m = { cotan / aspect, 0.0f, 0.0f, 0.0f,
+                     0.0f, cotan, 0.0f, 0.0f,
+                     0.0f, 0.0f, (farZ + nearZ) / (nearZ - farZ), -1.0f,
+                     0.0f, 0.0f, (2.0f * farZ * nearZ) / (nearZ - farZ), 0.0f };
+
+    return m;
+}
+
+static __inline__ GLKMatrix4 GLKMatrix4MakeFrustum(float left, float right,
+                                                   float bottom, float top,
+                                                   float nearZ, float farZ)
+{
+    float ral = right + left;
+    float rsl = right - left;
+    float tsb = top - bottom;
+    float tab = top + bottom;
+    float fan = farZ + nearZ;
+    float fsn = farZ - nearZ;
+
+    GLKMatrix4 m = { 2.0f * nearZ / rsl, 0.0f, 0.0f, 0.0f,
+                     0.0f, 2.0f * nearZ / tsb, 0.0f, 0.0f,
+                     ral / rsl, tab / tsb, -fan / fsn, -1.0f,
+                     0.0f, 0.0f, (-2.0f * farZ * nearZ) / fsn, 0.0f };
+
+    return m;
+}
+
+static __inline__ GLKMatrix4 GLKMatrix4MakeOrtho(float left, float right,
+                                                 float bottom, float top,
+                                                 float nearZ, float farZ)
+{
+    float ral = right + left;
+    float rsl = right - left;
+    float tab = top + bottom;
+    float tsb = top - bottom;
+    float fan = farZ + nearZ;
+    float fsn = farZ - nearZ;
+
+    GLKMatrix4 m = { 2.0f / rsl, 0.0f, 0.0f, 0.0f,
+                     0.0f, 2.0f / tsb, 0.0f, 0.0f,
+                     0.0f, 0.0f, -2.0f / fsn, 0.0f,
+                     -ral / rsl, -tab / tsb, -fan / fsn, 1.0f };
+
+    return m;
+}
+
+static __inline__ GLKMatrix4 GLKMatrix4MakeLookAt(float eyeX, float eyeY, float eyeZ,
+                                                  float centerX, float centerY, float centerZ,
+                                                  float upX, float upY, float upZ)
+{
+    GLKVector3 ev = { eyeX, eyeY, eyeZ };
+    GLKVector3 cv = { centerX, centerY, centerZ };
+    GLKVector3 uv = { upX, upY, upZ };
+    GLKVector3 n = GLKVector3Normalize(GLKVector3Add(ev, GLKVector3Negate(cv)));
+    GLKVector3 u = GLKVector3Normalize(GLKVector3CrossProduct(uv, n));
+    GLKVector3 v = GLKVector3CrossProduct(n, u);
+
+    GLKMatrix4 m = { u.v[0], v.v[0], n.v[0], 0.0f,
+                     u.v[1], v.v[1], n.v[1], 0.0f,
+                     u.v[2], v.v[2], n.v[2], 0.0f,
+                     GLKVector3DotProduct(GLKVector3Negate(u), ev),
+                     GLKVector3DotProduct(GLKVector3Negate(v), ev),
+                     GLKVector3DotProduct(GLKVector3Negate(n), ev),
+                     1.0f };
+
+    return m;
+}
+
+static __inline__ GLKMatrix3 GLKMatrix4GetMatrix3(GLKMatrix4 matrix)
+{
+    GLKMatrix3 m = { matrix.m[0], matrix.m[1], matrix.m[2],
+                     matrix.m[4], matrix.m[5], matrix.m[6],
+                     matrix.m[8], matrix.m[9], matrix.m[10] };
+    return m;
+}
+
+static __inline__ GLKMatrix2 GLKMatrix4GetMatrix2(GLKMatrix4 matrix)
+{
+    GLKMatrix2 m = { matrix.m[0], matrix.m[1],
+                     matrix.m[4], matrix.m[5] };
+    return m;
+}
+
+static __inline__ GLKVector4 GLKMatrix4GetRow(GLKMatrix4 matrix, int row)
+{
+    GLKVector4 v = { matrix.m[row], matrix.m[4 + row], matrix.m[8 + row], matrix.m[12 + row] };
+    return v;
+}
+
+static __inline__ GLKVector4 GLKMatrix4GetColumn(GLKMatrix4 matrix, int column)
+{
+#if defined(__ARM_NEON__)
+    float32x4_t v = vld1q_f32(&(matrix.m[column * 4]));
+    return *(GLKVector4 *)&v;
+#else
+    GLKVector4 v = { matrix.m[column * 4 + 0], matrix.m[column * 4 + 1], matrix.m[column * 4 + 2], matrix.m[column * 4 + 3] };
+    return v;
+#endif
+}
+
+static __inline__ GLKMatrix4 GLKMatrix4SetRow(GLKMatrix4 matrix, int row, GLKVector4 vector)
+{
+    matrix.m[row] = vector.v[0];
+    matrix.m[row + 4] = vector.v[1];
+    matrix.m[row + 8] = vector.v[2];
+    matrix.m[row + 12] = vector.v[3];
+
+    return matrix;
+}
+
+static __inline__ GLKMatrix4 GLKMatrix4SetColumn(GLKMatrix4 matrix, int column, GLKVector4 vector)
+{
+#if defined(__ARM_NEON__)
+    float *dst = &(matrix.m[column * 4]);
+    vst1q_f32(dst, vld1q_f32(vector.v));
+    return matrix;
+#else
+    matrix.m[column * 4 + 0] = vector.v[0];
+    matrix.m[column * 4 + 1] = vector.v[1];
+    matrix.m[column * 4 + 2] = vector.v[2];
+    matrix.m[column * 4 + 3] = vector.v[3];
+
+    return matrix;
+#endif
+}
+
+static __inline__ GLKMatrix4 GLKMatrix4Transpose(GLKMatrix4 matrix)
+{
+#if defined(__ARM_NEON__)
+    float32x4x4_t m = vld4q_f32(matrix.m);
+    return *(GLKMatrix4 *)&m;
+#else
+    GLKMatrix4 m = { matrix.m[0], matrix.m[4], matrix.m[8], matrix.m[12],
+                     matrix.m[1], matrix.m[5], matrix.m[9], matrix.m[13],
+                     matrix.m[2], matrix.m[6], matrix.m[10], matrix.m[14],
+                     matrix.m[3], matrix.m[7], matrix.m[11], matrix.m[15] };
+    return m;
+#endif
+}
+
+static __inline__ GLKMatrix4 GLKMatrix4Multiply(GLKMatrix4 matrixLeft, GLKMatrix4 matrixRight)
+{
+#if defined(__ARM_NEON__)
+    float32x4x4_t iMatrixLeft = *(float32x4x4_t *)&matrixLeft;
+    float32x4x4_t iMatrixRight = *(float32x4x4_t *)&matrixRight;
+    float32x4x4_t m;
+
+    m.val[0] = vmulq_n_f32(iMatrixLeft.val[0], vgetq_lane_f32(iMatrixRight.val[0], 0));
+    m.val[1] = vmulq_n_f32(iMatrixLeft.val[0], vgetq_lane_f32(iMatrixRight.val[1], 0));
+    m.val[2] = vmulq_n_f32(iMatrixLeft.val[0], vgetq_lane_f32(iMatrixRight.val[2], 0));
+    m.val[3] = vmulq_n_f32(iMatrixLeft.val[0], vgetq_lane_f32(iMatrixRight.val[3], 0));
+
+    m.val[0] = vmlaq_n_f32(m.val[0], iMatrixLeft.val[1], vgetq_lane_f32(iMatrixRight.val[0], 1));
+    m.val[1] = vmlaq_n_f32(m.val[1], iMatrixLeft.val[1], vgetq_lane_f32(iMatrixRight.val[1], 1));
+    m.val[2] = vmlaq_n_f32(m.val[2], iMatrixLeft.val[1], vgetq_lane_f32(iMatrixRight.val[2], 1));
+    m.val[3] = vmlaq_n_f32(m.val[3], iMatrixLeft.val[1], vgetq_lane_f32(iMatrixRight.val[3], 1));
+
+    m.val[0] = vmlaq_n_f32(m.val[0], iMatrixLeft.val[2], vgetq_lane_f32(iMatrixRight.val[0], 2));
+    m.val[1] = vmlaq_n_f32(m.val[1], iMatrixLeft.val[2], vgetq_lane_f32(iMatrixRight.val[1], 2));
+    m.val[2] = vmlaq_n_f32(m.val[2], iMatrixLeft.val[2], vgetq_lane_f32(iMatrixRight.val[2], 2));
+    m.val[3] = vmlaq_n_f32(m.val[3], iMatrixLeft.val[2], vgetq_lane_f32(iMatrixRight.val[3], 2));
+
+    m.val[0] = vmlaq_n_f32(m.val[0], iMatrixLeft.val[3], vgetq_lane_f32(iMatrixRight.val[0], 3));
+    m.val[1] = vmlaq_n_f32(m.val[1], iMatrixLeft.val[3], vgetq_lane_f32(iMatrixRight.val[1], 3));
+    m.val[2] = vmlaq_n_f32(m.val[2], iMatrixLeft.val[3], vgetq_lane_f32(iMatrixRight.val[2], 3));
+    m.val[3] = vmlaq_n_f32(m.val[3], iMatrixLeft.val[3], vgetq_lane_f32(iMatrixRight.val[3], 3));
+
+    return *(GLKMatrix4 *)&m;
+#else
+    GLKMatrix4 m;
+
+    m.m[0]  = matrixLeft.m[0] * matrixRight.m[0]  + matrixLeft.m[4] * matrixRight.m[1]  + matrixLeft.m[8] * matrixRight.m[2]   + matrixLeft.m[12] * matrixRight.m[3];
+	m.m[4]  = matrixLeft.m[0] * matrixRight.m[4]  + matrixLeft.m[4] * matrixRight.m[5]  + matrixLeft.m[8] * matrixRight.m[6]   + matrixLeft.m[12] * matrixRight.m[7];
+	m.m[8]  = matrixLeft.m[0] * matrixRight.m[8]  + matrixLeft.m[4] * matrixRight.m[9]  + matrixLeft.m[8] * matrixRight.m[10]  + matrixLeft.m[12] * matrixRight.m[11];
+	m.m[12] = matrixLeft.m[0] * matrixRight.m[12] + matrixLeft.m[4] * matrixRight.m[13] + matrixLeft.m[8] * matrixRight.m[14]  + matrixLeft.m[12] * matrixRight.m[15];
+
+	m.m[1]  = matrixLeft.m[1] * matrixRight.m[0]  + matrixLeft.m[5] * matrixRight.m[1]  + matrixLeft.m[9] * matrixRight.m[2]   + matrixLeft.m[13] * matrixRight.m[3];
+	m.m[5]  = matrixLeft.m[1] * matrixRight.m[4]  + matrixLeft.m[5] * matrixRight.m[5]  + matrixLeft.m[9] * matrixRight.m[6]   + matrixLeft.m[13] * matrixRight.m[7];
+	m.m[9]  = matrixLeft.m[1] * matrixRight.m[8]  + matrixLeft.m[5] * matrixRight.m[9]  + matrixLeft.m[9] * matrixRight.m[10]  + matrixLeft.m[13] * matrixRight.m[11];
+	m.m[13] = matrixLeft.m[1] * matrixRight.m[12] + matrixLeft.m[5] * matrixRight.m[13] + matrixLeft.m[9] * matrixRight.m[14]  + matrixLeft.m[13] * matrixRight.m[15];
+
+	m.m[2]  = matrixLeft.m[2] * matrixRight.m[0]  + matrixLeft.m[6] * matrixRight.m[1]  + matrixLeft.m[10] * matrixRight.m[2]  + matrixLeft.m[14] * matrixRight.m[3];
+	m.m[6]  = matrixLeft.m[2] * matrixRight.m[4]  + matrixLeft.m[6] * matrixRight.m[5]  + matrixLeft.m[10] * matrixRight.m[6]  + matrixLeft.m[14] * matrixRight.m[7];
+	m.m[10] = matrixLeft.m[2] * matrixRight.m[8]  + matrixLeft.m[6] * matrixRight.m[9]  + matrixLeft.m[10] * matrixRight.m[10] + matrixLeft.m[14] * matrixRight.m[11];
+	m.m[14] = matrixLeft.m[2] * matrixRight.m[12] + matrixLeft.m[6] * matrixRight.m[13] + matrixLeft.m[10] * matrixRight.m[14] + matrixLeft.m[14] * matrixRight.m[15];
+
+	m.m[3]  = matrixLeft.m[3] * matrixRight.m[0]  + matrixLeft.m[7] * matrixRight.m[1]  + matrixLeft.m[11] * matrixRight.m[2]  + matrixLeft.m[15] * matrixRight.m[3];
+	m.m[7]  = matrixLeft.m[3] * matrixRight.m[4]  + matrixLeft.m[7] * matrixRight.m[5]  + matrixLeft.m[11] * matrixRight.m[6]  + matrixLeft.m[15] * matrixRight.m[7];
+	m.m[11] = matrixLeft.m[3] * matrixRight.m[8]  + matrixLeft.m[7] * matrixRight.m[9]  + matrixLeft.m[11] * matrixRight.m[10] + matrixLeft.m[15] * matrixRight.m[11];
+	m.m[15] = matrixLeft.m[3] * matrixRight.m[12] + matrixLeft.m[7] * matrixRight.m[13] + matrixLeft.m[11] * matrixRight.m[14] + matrixLeft.m[15] * matrixRight.m[15];
+
+    return m;
+#endif
+}
+
+static __inline__ GLKMatrix4 GLKMatrix4Add(GLKMatrix4 matrixLeft, GLKMatrix4 matrixRight)
+{
+#if defined(__ARM_NEON__)
+    float32x4x4_t iMatrixLeft = *(float32x4x4_t *)&matrixLeft;
+    float32x4x4_t iMatrixRight = *(float32x4x4_t *)&matrixRight;
+    float32x4x4_t m;
+
+    m.val[0] = vaddq_f32(iMatrixLeft.val[0], iMatrixRight.val[0]);
+    m.val[1] = vaddq_f32(iMatrixLeft.val[1], iMatrixRight.val[1]);
+    m.val[2] = vaddq_f32(iMatrixLeft.val[2], iMatrixRight.val[2]);
+    m.val[3] = vaddq_f32(iMatrixLeft.val[3], iMatrixRight.val[3]);
+
+    return *(GLKMatrix4 *)&m;
+#else
+    GLKMatrix4 m;
+
+    m.m[0] = matrixLeft.m[0] + matrixRight.m[0];
+    m.m[1] = matrixLeft.m[1] + matrixRight.m[1];
+    m.m[2] = matrixLeft.m[2] + matrixRight.m[2];
+    m.m[3] = matrixLeft.m[3] + matrixRight.m[3];
+
+    m.m[4] = matrixLeft.m[4] + matrixRight.m[4];
+    m.m[5] = matrixLeft.m[5] + matrixRight.m[5];
+    m.m[6] = matrixLeft.m[6] + matrixRight.m[6];
+    m.m[7] = matrixLeft.m[7] + matrixRight.m[7];
+
+    m.m[8] = matrixLeft.m[8] + matrixRight.m[8];
+    m.m[9] = matrixLeft.m[9] + matrixRight.m[9];
+    m.m[10] = matrixLeft.m[10] + matrixRight.m[10];
+    m.m[11] = matrixLeft.m[11] + matrixRight.m[11];
+
+    m.m[12] = matrixLeft.m[12] + matrixRight.m[12];
+    m.m[13] = matrixLeft.m[13] + matrixRight.m[13];
+    m.m[14] = matrixLeft.m[14] + matrixRight.m[14];
+    m.m[15] = matrixLeft.m[15] + matrixRight.m[15];
+
+    return m;
+#endif
+}
+
+static __inline__ GLKMatrix4 GLKMatrix4Subtract(GLKMatrix4 matrixLeft, GLKMatrix4 matrixRight)
+{
+#if defined(__ARM_NEON__)
+    float32x4x4_t iMatrixLeft = *(float32x4x4_t *)&matrixLeft;
+    float32x4x4_t iMatrixRight = *(float32x4x4_t *)&matrixRight;
+    float32x4x4_t m;
+
+    m.val[0] = vsubq_f32(iMatrixLeft.val[0], iMatrixRight.val[0]);
+    m.val[1] = vsubq_f32(iMatrixLeft.val[1], iMatrixRight.val[1]);
+    m.val[2] = vsubq_f32(iMatrixLeft.val[2], iMatrixRight.val[2]);
+    m.val[3] = vsubq_f32(iMatrixLeft.val[3], iMatrixRight.val[3]);
+
+    return *(GLKMatrix4 *)&m;
+#else
+    GLKMatrix4 m;
+
+    m.m[0] = matrixLeft.m[0] - matrixRight.m[0];
+    m.m[1] = matrixLeft.m[1] - matrixRight.m[1];
+    m.m[2] = matrixLeft.m[2] - matrixRight.m[2];
+    m.m[3] = matrixLeft.m[3] - matrixRight.m[3];
+
+    m.m[4] = matrixLeft.m[4] - matrixRight.m[4];
+    m.m[5] = matrixLeft.m[5] - matrixRight.m[5];
+    m.m[6] = matrixLeft.m[6] - matrixRight.m[6];
+    m.m[7] = matrixLeft.m[7] - matrixRight.m[7];
+
+    m.m[8] = matrixLeft.m[8] - matrixRight.m[8];
+    m.m[9] = matrixLeft.m[9] - matrixRight.m[9];
+    m.m[10] = matrixLeft.m[10] - matrixRight.m[10];
+    m.m[11] = matrixLeft.m[11] - matrixRight.m[11];
+
+    m.m[12] = matrixLeft.m[12] - matrixRight.m[12];
+    m.m[13] = matrixLeft.m[13] - matrixRight.m[13];
+    m.m[14] = matrixLeft.m[14] - matrixRight.m[14];
+    m.m[15] = matrixLeft.m[15] - matrixRight.m[15];
+
+    return m;
+#endif
+}
+
+static __inline__ GLKMatrix4 GLKMatrix4Translate(GLKMatrix4 matrix, float tx, float ty, float tz)
+{
+    GLKMatrix4 m = { matrix.m[0], matrix.m[1], matrix.m[2], matrix.m[3],
+                     matrix.m[4], matrix.m[5], matrix.m[6], matrix.m[7],
+                     matrix.m[8], matrix.m[9], matrix.m[10], matrix.m[11],
+                     matrix.m[0] * tx + matrix.m[4] * ty + matrix.m[8] * tz + matrix.m[12],
+                     matrix.m[1] * tx + matrix.m[5] * ty + matrix.m[9] * tz + matrix.m[13],
+                     matrix.m[2] * tx + matrix.m[6] * ty + matrix.m[10] * tz + matrix.m[14],
+                     matrix.m[15] };
+    return m;
+}
+
+static __inline__ GLKMatrix4 GLKMatrix4TranslateWithVector3(GLKMatrix4 matrix, GLKVector3 translationVector)
+{
+    GLKMatrix4 m = { matrix.m[0], matrix.m[1], matrix.m[2], matrix.m[3],
+                     matrix.m[4], matrix.m[5], matrix.m[6], matrix.m[7],
+                     matrix.m[8], matrix.m[9], matrix.m[10], matrix.m[11],
+                     matrix.m[0] * translationVector.v[0] + matrix.m[4] * translationVector.v[1] + matrix.m[8] * translationVector.v[2] + matrix.m[12],
+                     matrix.m[1] * translationVector.v[0] + matrix.m[5] * translationVector.v[1] + matrix.m[9] * translationVector.v[2] + matrix.m[13],
+                     matrix.m[2] * translationVector.v[0] + matrix.m[6] * translationVector.v[1] + matrix.m[10] * translationVector.v[2] + matrix.m[14],
+                     matrix.m[15] };
+    return m;
+}
+
+static __inline__ GLKMatrix4 GLKMatrix4TranslateWithVector4(GLKMatrix4 matrix, GLKVector4 translationVector)
+{
+    GLKMatrix4 m = { matrix.m[0], matrix.m[1], matrix.m[2], matrix.m[3],
+                     matrix.m[4], matrix.m[5], matrix.m[6], matrix.m[7],
+                     matrix.m[8], matrix.m[9], matrix.m[10], matrix.m[11],
+                     matrix.m[0] * translationVector.v[0] + matrix.m[4] * translationVector.v[1] + matrix.m[8] * translationVector.v[2] + matrix.m[12],
+                     matrix.m[1] * translationVector.v[0] + matrix.m[5] * translationVector.v[1] + matrix.m[9] * translationVector.v[2] + matrix.m[13],
+                     matrix.m[2] * translationVector.v[0] + matrix.m[6] * translationVector.v[1] + matrix.m[10] * translationVector.v[2] + matrix.m[14],
+                     matrix.m[15] };
+    return m;
+}
+
+static __inline__ GLKMatrix4 GLKMatrix4Scale(GLKMatrix4 matrix, float sx, float sy, float sz)
+{
+#if defined(__ARM_NEON__)
+    float32x4x4_t iMatrix = *(float32x4x4_t *)&matrix;
+    float32x4x4_t m;
+
+    m.val[0] = vmulq_n_f32(iMatrix.val[0], (float32_t)sx);
+    m.val[1] = vmulq_n_f32(iMatrix.val[1], (float32_t)sy);
+    m.val[2] = vmulq_n_f32(iMatrix.val[2], (float32_t)sz);
+    m.val[3] = iMatrix.val[3];
+
+    return *(GLKMatrix4 *)&m;
+#else
+    GLKMatrix4 m = { matrix.m[0] * sx, matrix.m[1] * sx, matrix.m[2] * sx, matrix.m[3] * sx,
+                     matrix.m[4] * sy, matrix.m[5] * sy, matrix.m[6] * sy, matrix.m[7] * sy,
+                     matrix.m[8] * sz, matrix.m[9] * sz, matrix.m[10] * sz, matrix.m[11] * sz,
+                     matrix.m[12], matrix.m[13], matrix.m[14], matrix.m[15] };
+    return m;
+#endif
+}
+
+static __inline__ GLKMatrix4 GLKMatrix4ScaleWithVector3(GLKMatrix4 matrix, GLKVector3 scaleVector)
+{
+#if defined(__ARM_NEON__)
+    float32x4x4_t iMatrix = *(float32x4x4_t *)&matrix;
+    float32x4x4_t m;
+
+    m.val[0] = vmulq_n_f32(iMatrix.val[0], (float32_t)scaleVector.v[0]);
+    m.val[1] = vmulq_n_f32(iMatrix.val[1], (float32_t)scaleVector.v[1]);
+    m.val[2] = vmulq_n_f32(iMatrix.val[2], (float32_t)scaleVector.v[2]);
+    m.val[3] = iMatrix.val[3];
+
+    return *(GLKMatrix4 *)&m;
+#else
+    GLKMatrix4 m = { matrix.m[0] * scaleVector.v[0], matrix.m[1] * scaleVector.v[0], matrix.m[2] * scaleVector.v[0], matrix.m[3] * scaleVector.v[0],
+                     matrix.m[4] * scaleVector.v[1], matrix.m[5] * scaleVector.v[1], matrix.m[6] * scaleVector.v[1], matrix.m[7] * scaleVector.v[1],
+                     matrix.m[8] * scaleVector.v[2], matrix.m[9] * scaleVector.v[2], matrix.m[10] * scaleVector.v[2], matrix.m[11] * scaleVector.v[2],
+                     matrix.m[12], matrix.m[13], matrix.m[14], matrix.m[15] };
+    return m;
+#endif
+}
+
+static __inline__ GLKMatrix4 GLKMatrix4ScaleWithVector4(GLKMatrix4 matrix, GLKVector4 scaleVector)
+{
+#if defined(__ARM_NEON__)
+    float32x4x4_t iMatrix = *(float32x4x4_t *)&matrix;
+    float32x4x4_t m;
+
+    m.val[0] = vmulq_n_f32(iMatrix.val[0], (float32_t)scaleVector.v[0]);
+    m.val[1] = vmulq_n_f32(iMatrix.val[1], (float32_t)scaleVector.v[1]);
+    m.val[2] = vmulq_n_f32(iMatrix.val[2], (float32_t)scaleVector.v[2]);
+    m.val[3] = iMatrix.val[3];
+
+    return *(GLKMatrix4 *)&m;
+#else
+    GLKMatrix4 m = { matrix.m[0] * scaleVector.v[0], matrix.m[1] * scaleVector.v[0], matrix.m[2] * scaleVector.v[0], matrix.m[3] * scaleVector.v[0],
+                     matrix.m[4] * scaleVector.v[1], matrix.m[5] * scaleVector.v[1], matrix.m[6] * scaleVector.v[1], matrix.m[7] * scaleVector.v[1],
+                     matrix.m[8] * scaleVector.v[2], matrix.m[9] * scaleVector.v[2], matrix.m[10] * scaleVector.v[2], matrix.m[11] * scaleVector.v[2],
+                     matrix.m[12], matrix.m[13], matrix.m[14], matrix.m[15] };
+    return m;
+#endif
+}
+
+static __inline__ GLKMatrix4 GLKMatrix4Rotate(GLKMatrix4 matrix, float radians, float x, float y, float z)
+{
+    GLKMatrix4 rm = GLKMatrix4MakeRotation(radians, x, y, z);
+    return GLKMatrix4Multiply(matrix, rm);
+}
+
+static __inline__ GLKMatrix4 GLKMatrix4RotateWithVector3(GLKMatrix4 matrix, float radians, GLKVector3 axisVector)
+{
+    GLKMatrix4 rm = GLKMatrix4MakeRotation(radians, axisVector.v[0], axisVector.v[1], axisVector.v[2]);
+    return GLKMatrix4Multiply(matrix, rm);
+}
+
+static __inline__ GLKMatrix4 GLKMatrix4RotateWithVector4(GLKMatrix4 matrix, float radians, GLKVector4 axisVector)
+{
+    GLKMatrix4 rm = GLKMatrix4MakeRotation(radians, axisVector.v[0], axisVector.v[1], axisVector.v[2]);
+    return GLKMatrix4Multiply(matrix, rm);
+}
+
+static __inline__ GLKMatrix4 GLKMatrix4RotateX(GLKMatrix4 matrix, float radians)
+{
+    GLKMatrix4 rm = GLKMatrix4MakeXRotation(radians);
+    return GLKMatrix4Multiply(matrix, rm);
+}
+
+static __inline__ GLKMatrix4 GLKMatrix4RotateY(GLKMatrix4 matrix, float radians)
+{
+    GLKMatrix4 rm = GLKMatrix4MakeYRotation(radians);
+    return GLKMatrix4Multiply(matrix, rm);
+}
+
+static __inline__ GLKMatrix4 GLKMatrix4RotateZ(GLKMatrix4 matrix, float radians)
+{
+    GLKMatrix4 rm = GLKMatrix4MakeZRotation(radians);
+    return GLKMatrix4Multiply(matrix, rm);
+}
+
+static __inline__ GLKVector3 GLKMatrix4MultiplyVector3(GLKMatrix4 matrixLeft, GLKVector3 vectorRight)
+{
+    GLKVector4 v4 = GLKMatrix4MultiplyVector4(matrixLeft, GLKVector4Make(vectorRight.v[0], vectorRight.v[1], vectorRight.v[2], 0.0f));
+    return GLKVector3Make(v4.v[0], v4.v[1], v4.v[2]);
+}
+
+static __inline__ GLKVector3 GLKMatrix4MultiplyVector3WithTranslation(GLKMatrix4 matrixLeft, GLKVector3 vectorRight)
+{
+    GLKVector4 v4 = GLKMatrix4MultiplyVector4(matrixLeft, GLKVector4Make(vectorRight.v[0], vectorRight.v[1], vectorRight.v[2], 1.0f));
+    return GLKVector3Make(v4.v[0], v4.v[1], v4.v[2]);
+}
+
+static __inline__ GLKVector3 GLKMatrix4MultiplyAndProjectVector3(GLKMatrix4 matrixLeft, GLKVector3 vectorRight)
+{
+    GLKVector4 v4 = GLKMatrix4MultiplyVector4(matrixLeft, GLKVector4Make(vectorRight.v[0], vectorRight.v[1], vectorRight.v[2], 1.0f));
+    return GLKVector3MultiplyScalar(GLKVector3Make(v4.v[0], v4.v[1], v4.v[2]), 1.0f / v4.v[3]);
+}
+
+static __inline__ void GLKMatrix4MultiplyVector3Array(GLKMatrix4 matrix, GLKVector3 *vectors, size_t vectorCount)
+{
+    size_t i;
+    for (i=0; i < vectorCount; i++)
+        vectors[i] = GLKMatrix4MultiplyVector3(matrix, vectors[i]);
+}
+
+static __inline__ void GLKMatrix4MultiplyVector3ArrayWithTranslation(GLKMatrix4 matrix, GLKVector3 *vectors, size_t vectorCount)
+{
+    size_t i;
+    for (i=0; i < vectorCount; i++)
+        vectors[i] = GLKMatrix4MultiplyVector3WithTranslation(matrix, vectors[i]);
+}
+
+static __inline__ void GLKMatrix4MultiplyAndProjectVector3Array(GLKMatrix4 matrix, GLKVector3 *vectors, size_t vectorCount)
+{
+    size_t i;
+    for (i=0; i < vectorCount; i++)
+        vectors[i] = GLKMatrix4MultiplyAndProjectVector3(matrix, vectors[i]);
+}
+
+static __inline__ GLKVector4 GLKMatrix4MultiplyVector4(GLKMatrix4 matrixLeft, GLKVector4 vectorRight)
+{
+#if defined(__ARM_NEON__)
+    float32x4x4_t iMatrix = *(float32x4x4_t *)&matrixLeft;
+    float32x4_t v;
+
+    iMatrix.val[0] = vmulq_n_f32(iMatrix.val[0], (float32_t)vectorRight.v[0]);
+    iMatrix.val[1] = vmulq_n_f32(iMatrix.val[1], (float32_t)vectorRight.v[1]);
+    iMatrix.val[2] = vmulq_n_f32(iMatrix.val[2], (float32_t)vectorRight.v[2]);
+    iMatrix.val[3] = vmulq_n_f32(iMatrix.val[3], (float32_t)vectorRight.v[3]);
+
+    iMatrix.val[0] = vaddq_f32(iMatrix.val[0], iMatrix.val[1]);
+    iMatrix.val[2] = vaddq_f32(iMatrix.val[2], iMatrix.val[3]);
+
+    v = vaddq_f32(iMatrix.val[0], iMatrix.val[2]);
+
+    return *(GLKVector4 *)&v;
+#else
+    GLKVector4 v = { matrixLeft.m[0] * vectorRight.v[0] + matrixLeft.m[4] * vectorRight.v[1] + matrixLeft.m[8] * vectorRight.v[2] + matrixLeft.m[12] * vectorRight.v[3],
+                     matrixLeft.m[1] * vectorRight.v[0] + matrixLeft.m[5] * vectorRight.v[1] + matrixLeft.m[9] * vectorRight.v[2] + matrixLeft.m[13] * vectorRight.v[3],
+                     matrixLeft.m[2] * vectorRight.v[0] + matrixLeft.m[6] * vectorRight.v[1] + matrixLeft.m[10] * vectorRight.v[2] + matrixLeft.m[14] * vectorRight.v[3],
+                     matrixLeft.m[3] * vectorRight.v[0] + matrixLeft.m[7] * vectorRight.v[1] + matrixLeft.m[11] * vectorRight.v[2] + matrixLeft.m[15] * vectorRight.v[3] };
+    return v;
+#endif
+}
+
+static __inline__ void GLKMatrix4MultiplyVector4Array(GLKMatrix4 matrix, GLKVector4 *vectors, size_t vectorCount)
+{
+    size_t i;
+    for (i=0; i < vectorCount; i++)
+        vectors[i] = GLKMatrix4MultiplyVector4(matrix, vectors[i]);
+}
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* __GLK_MATRIX_4_H */

samples/C/NWMan.h

@@ -0,0 +1,99 @@
+#ifndef _NME_WMAN_H
+#define _NME_WMAN_H
+
+// Internal window manager API
+
+#include "NCompat.h"
+
+START_HEAD
+
+#include "NPos.h"
+#include "NUtil.h"
+#include "NTypes.h"
+
+NTS(NWMan_event);
+
+NSTRUCT(NWMan, {
+    // Init stuff
+    bool (*init)();
+    bool (*destroy)();
+
+    // Window stuff
+    bool (*create_window)();
+    bool (*destroy_window)();
+
+    void (*swap_buffers)();
+
+    // Event stuff
+    bool (*next_event)(NWMan_event* event);
+
+    // Time stuff
+    uint (*get_millis)();
+    void (*sleep)(uint millis);
+
+    // Info
+    int rshift_key;
+    int lshift_key;
+    int left_key;
+    int right_key;
+});
+
+NENUM(NWMan_event_type, {
+    N_WMAN_MOUSE_MOVE = 0,
+    N_WMAN_MOUSE_BUTTON = 1,
+    N_WMAN_MOUSE_WHEEL = 2,
+
+    N_WMAN_KEYBOARD = 10,
+
+    N_WMAN_QUIT = 20,
+    N_WMAN_RESIZE = 21,
+    N_WMAN_FOCUS = 22
+});
+
+#define N_WMAN_MOUSE_LEFT 0
+#define N_WMAN_MOUSE_RIGHT 1
+#define N_WMAN_MOUSE_MIDDLE 2
+
+NSTRUCT(NWMan_event, {
+    NWMan_event_type type;
+
+    union {
+        // Mouse
+
+        NPos2i mouse_pos;
+
+        struct {
+            short id;
+            bool state;
+        } mouse_button;
+
+        signed char mouse_wheel; // 1 if up, -1 if down
+
+        // Keyboard
+
+        struct {
+            int key;
+            bool state;
+        } keyboard;
+
+        // Window
+
+        bool window_quit; // Will always be true if WM_QUIT
+
+        NPos2i window_size;
+
+        bool window_focus;
+    };
+});
+
+NWMan_event NWMan_event_new(NWMan_event_type type);
+
+
+bool NWMan_init();
+bool NWMan_destroy();
+
+extern NWMan N_WMan;
+
+END_HEAD
+
+#endif

samples/C/Nightmare.h

@@ -0,0 +1,27 @@
+#ifndef _NMEX_NIGHTMARE_H
+#define _NMEX_NIGHTMARE_H
+
+//#define NMEX
+
+#include "../src/NCompat.h"
+
+START_HEAD
+
+#include "../src/NTypes.h"
+#include "../src/NUtil.h"
+#include "../src/NPorting.h"
+#include "../src/NGlobals.h"
+#include "../src/NLog.h"
+#include "../src/NWMan.h"
+#include "../src/NRsc.h"
+#include "../src/NShader.h"
+#include "../src/NSquare.h"
+#include "../src/NImage.h"
+#include "../src/NSprite.h"
+#include "../src/NSpritesheet.h"
+#include "../src/NEntity.h"
+#include "../src/Game.h"
+
+END_HEAD
+
+#endif

samples/C/ntru_encrypt.h

@@ -0,0 +1,89 @@
+/*
+ * Copyright (C) 2014 FH Bielefeld
+ *
+ * This file is part of a FH Bielefeld project.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
+ * MA  02110-1301  USA
+ */
+
+/**
+ * @file ntru_encrypt.h
+ * Header for the internal API of ntru_encrypt.c.
+ * @brief header for encrypt.c
+ */
+
+#ifndef PQC_ENCRYPT_H
+#define PQC_ENCRYPT_H
+
+
+#include "ntru_params.h"
+#include "ntru_poly.h"
+#include "ntru_string.h"
+
+#include <fmpz_poly.h>
+#include <fmpz.h>
+
+
+/**
+ * encrypt the msg, using the math:
+ * e = (h ∗ r) + m (mod q)
+ *
+ * e = the encrypted poly
+ *
+ * h = the public key
+ *
+ * r = the random poly
+ *
+ * m = the message poly
+ *
+ * q = large mod
+ *
+ * @param msg_tern the message to encrypt, in ternary format
+ * @param pub_key the public key
+ * @param rnd the random poly (should have relatively small
+ * coefficients, but not restricted to {-1, 0, 1})
+ * @param out the output poly which is in the range {0, q-1}
+ * (not ternary!) [out]
+ * @param params ntru_params the ntru context
+ */
+void
+ntru_encrypt_poly(
+		const fmpz_poly_t msg_tern,
+		const fmpz_poly_t pub_key,
+		const fmpz_poly_t rnd,
+		fmpz_poly_t out,
+		const ntru_params *params);
+
+/**
+ * Encrypt a message in the form of a null-terminated char array and
+ * return a string.
+ *
+ * @param msg the message
+ * @param pub_key the public key
+ * @param rnd the random poly (should have relatively small
+ * coefficients, but not restricted to {-1, 0, 1})
+ * @param params ntru_params the ntru context
+ * @return the newly allocated encrypted string
+ */
+string *
+ntru_encrypt_string(
+		const string *msg,
+		const fmpz_poly_t pub_key,
+		const fmpz_poly_t rnd,
+		const ntru_params *params);
+
+
+#endif /* PQC_ENCRYPT_H */

samples/Ceylon/Foo.ceylon

@@ -1,13 +1,13 @@
-doc "Test function for Ceylon"
-by  "Enrique"
+"Test function for Ceylon"
+by ("Enrique")
 shared void test() {
-  print("test");
+    print("test");
 }
 
-doc "Test class for Ceylon"
+"Test class for Ceylon"
 shared class Test(name) satisfies Comparable<Test> {
     shared String name;
-    shared actual String string = "Test " name ".";
+    shared actual String string = "Test ``name``.";
 
     shared actual Comparison compare(Test other) {
         return name<=>other.name;

samples/Chapel/distributions.chpl

@@ -0,0 +1,304 @@
+//
+// Distributions Primer
+//
+// This primer demonstrates uses of some of Chapel's standard
+// distributions.  To use these distributions in a Chapel program,
+// the respective module must be used:
+//
+use BlockDist, CyclicDist, BlockCycDist, ReplicatedDist;
+use DimensionalDist2D, ReplicatedDim, BlockCycDim;
+
+//
+// For each distribution, we'll create a distributed domain and array
+// and then initialize it just to give a brief flavor of how the
+// distribution maps across locales.  Running this example on 6
+// locales does a nice job of illustrating the distribution
+// characteristics.
+//
+// All of these distributions support options to map to a different
+// virtual locale grid than the one used by default (a
+// multidimensional factoring of the built-in Locales array), as well
+// as to control the amount of parallelism used in data parallel
+// loops.  See the Standard Distributions chapter of the language spec
+// for more details.
+//
+
+//
+// Make the program size configurable from the command line.
+//
+config const n = 8;
+
+//
+// Declare a 2-dimensional domain Space that we will later use to
+// initialize the distributed domains.
+//
+const Space = {1..n, 1..n};
+
+//
+// The Block distribution distributes a bounding box from
+// n-dimensional space across the target locale array viewed as an
+// n-dimensional virtual locale grid.  The bounding box is blocked
+// into roughly equal portions across the locales.  Note that domains
+// declared over a Block distribution can also store indices outside
+// of the bounding box; the bounding box is merely used to compute
+// the blocking of space.
+//
+// In this example, we declare a 2-dimensional Block-distributed
+// domain BlockSpace and a Block-distributed array BA declared over
+// the domain.
+//
+const BlockSpace = Space dmapped Block(boundingBox=Space);
+var BA: [BlockSpace] int;
+
+//
+// To illustrate how the index set is distributed across locales,
+// we'll use a forall loop to initialize each array element to the
+// locale ID that stores that index/element/iteration.
+//
+forall ba in BA do
+  ba = here.id;
+
+//
+// Output the Block-distributed array to visually see how the elements
+// are partitioned across the locales.
+//
+writeln("Block Array Index Map");
+writeln(BA);
+writeln();
+
+//
+// Most of Chapel's standard distributions support an optional
+// targetLocales argument that permits you to pass in your own
+// array of locales to be targeted.  In general, the targetLocales
+// argument should match the rank of the distribution.  So for
+// example, to map a Block to a [numLocales x 1] view of the
+// locale set, one could do something like this:
+
+//
+// We start by creating our own array of the locale values.  Here
+// we use the standard array reshape function for convenience,
+// but more generally, this array could be accessed/assigned like any
+// other.
+//
+
+var MyLocaleView = {0..#numLocales, 1..1};
+var MyLocales: [MyLocaleView] locale = reshape(Locales, MyLocaleView);
+
+//
+// Then we'll declare a distributed domain/array that targets
+// this view of the locales:
+// 
+
+const BlockSpace2 = Space dmapped Block(boundingBox=Space,
+                                        targetLocales=MyLocales);
+var BA2: [BlockSpace2] int;
+
+//
+// Then we'll do a similar computation as before to verify where
+// everything ended up:
+//
+forall ba in BA2 do
+  ba = here.id;
+
+writeln("Block Array Index Map");
+writeln(BA2);
+writeln();
+
+
+
+//
+// Next, we'll perform a similar computation for the Cyclic distribution.
+// Cyclic distributions start at a designated n-dimensional index and
+// distribute the n-dimensional space across an n-dimensional array
+// of locales in a round-robin fashion (in each dimension).  As with
+// the Block distribution, domains may be declared using the
+// distribution who have lower indices that the starting index; that
+// value should just be considered a parameterization of how the
+// distribution is defined.
+//
+const CyclicSpace = Space dmapped Cyclic(startIdx=Space.low);
+var CA: [CyclicSpace] int;
+
+forall ca in CA do
+  ca = here.id;
+
+writeln("Cyclic Array Index Map");
+writeln(CA);
+writeln();
+
+
+//
+// Next, we'll declare a Block-Cyclic distribution.  These
+// distributions also deal out indices in a round-robin fashion,
+// but rather than dealing out singleton indices, they deal out blocks
+// of indices.  Thus, the BlockCyclic distribution is parameterized
+// by a starting index (as with Cyclic) and a block size (per
+// dimension) specifying how large the chunks to be dealt out are.
+// 
+const BlkCycSpace = Space dmapped BlockCyclic(startIdx=Space.low, 
+                                              blocksize=(2, 3));
+var BCA: [BlkCycSpace] int;
+
+forall bca in BCA do
+  bca = here.id;
+
+writeln("Block-Cyclic Array Index Map");
+writeln(BCA);
+writeln();
+
+
+//
+// The ReplicatedDist distribution is different: each of the
+// original domain's indices - and the corresponding array elements -
+// is replicated onto each locale. (Note: consistency among these
+// array replicands is NOT maintained automatically.)
+//
+// This replication is observable in some cases but not others,
+// as shown below. Note: this behavior may change in the future.
+//
+const ReplicatedSpace = Space dmapped ReplicatedDist();
+var RA: [ReplicatedSpace] int;
+
+// The replication is observable - this visits each replicand.
+forall ra in RA do
+  ra = here.id;
+
+writeln("Replicated Array Index Map, ", RA.numElements, " elements total");
+writeln(RA);
+writeln();
+
+//
+// The replication is observable when the replicated array is
+// on the left-hand side. If the right-hand side is not replicated,
+// it is copied into each replicand.
+// We illustrate this using a non-distributed array.
+//
+var A: [Space] int = [(i,j) in Space] i*100 + j;
+RA = A;
+writeln("Replicated Array after being array-assigned into");
+writeln(RA);
+writeln();
+
+//
+// Analogously, each replicand will be visited and
+// other participated expressions will be computed on each locale
+// (a) when the replicated array is assigned a scalar:
+//       RA = 5;
+// (b) when it appears first in a zippered forall loop:
+//       forall (ra, a) in zip(RA, A) do ...;
+// (c) when it appears in a for loop:
+//       for ra in RA do ...;
+//
+// Zippering (RA,A) or (A,RA) in a 'for' loop will generate
+// an error due to their different number of elements.
+
+// Let RA store the Index Map again, for the examples below.
+forall ra in RA do
+  ra = here.id;
+
+//
+// Only the local replicand is accessed - replication is NOT observable
+// and consistency is NOT maintained - when:
+// (a) the replicated array is indexed - an individual element is read...
+//
+on Locales(0) do
+  writeln("on ", here, ": ", RA(Space.low));
+on Locales(LocaleSpace.high) do
+  writeln("on ", here, ": ", RA(Space.low));
+writeln();
+
+// ...or an individual element is written;
+on Locales(LocaleSpace.high) do
+  RA(Space.low) = 7777;
+
+writeln("Replicated Array after being indexed into");
+writeln(RA);
+writeln();
+
+//
+// (b) the replicated array is on the right-hand side of an assignment...
+//
+on Locales(LocaleSpace.high) do
+  A = RA + 4;
+writeln("Non-Replicated Array after assignment from Replicated Array + 4");
+writeln(A);
+writeln();
+
+//
+// (c) ...or, generally, the replicated array or domain participates
+//     in a zippered forall loop, but not in the first position.
+//     The loop could look like:
+//
+//       forall (a, (i,j), ra) in (A, ReplicatedSpace, RA) do ...;
+//
+
+
+//
+// The DimensionalDist2D distribution lets us build a 2D distribution
+// as a composition of specifiers for individual dimensions.
+// Under such a "dimensional" distribution each dimension is handled
+// independently of the other.
+//
+// The dimension specifiers are similar to the corresponding multi-dimensional
+// distributions in constructor arguments and index-to-locale mapping rules.
+// However, instead of an array of locales, a specifier constructor
+// accepts just the number of locales that the indices in the corresponding
+// dimension will be distributed across.
+//
+// The DimensionalDist2D constructor requires:
+// * an [0..nl1-1, 0..nl2-1] array of locales, where
+//   nl1 and nl2 are the number of locales in each dimension, and
+// * two dimension specifiers, created for nl1 and nl2 locale counts, resp.
+//
+// Presently, the following dimension specifiers are available
+// (shown here with their constructor arguments):
+//
+// * ReplicatedDim(numLocales)
+// * BlockDim(numLocales, boundingBoxLow, boundingBoxHigh)
+// * BlockCyclicDim(lowIdx, blockSize, numLocales)
+//
+
+//
+// The following example creates a dimensional distribution that
+// replicates over 2 locales (when available) in the first dimemsion
+// and distributes using block-cyclic distribution in the second dimension.
+// The example computes nl1 and nl2 and reshapes MyLocales correspondingly.
+//
+
+var (nl1, nl2) = if numLocales == 1 then (1, 1) else (2, numLocales/2);
+MyLocaleView = {0..#nl1, 0..#nl2};
+MyLocales = reshape(Locales[0..#nl1*nl2], MyLocaleView);
+
+const DimReplicatedBlockcyclicSpace = Space
+  dmapped DimensionalDist2D(MyLocales,
+                            new ReplicatedDim(numLocales = nl1),
+                            new BlockCyclicDim(numLocales = nl2,
+                                               lowIdx = 1, blockSize = 2));
+
+var DRBA: [DimReplicatedBlockcyclicSpace] int;
+
+// The ReplicatedDim specifier always accesses the local replicand.
+// (This differs from how the ReplicatedDist distribution works.)
+//
+// This example visits each replicand. The behavior is the same
+// regardless of the second index into MyLocales below.
+
+for locId1 in 0..#nl1 do on MyLocales[locId1, 0] {
+
+  forall drba in DRBA do
+    drba = here.id;
+
+  writeln("Dimensional2D(Replicated,BlockCyclic) Array Index Map",
+          " from ", here);
+
+  // Technicality: 'writeln(DRBA)' would read DRBA always on Locale 0.
+  // Since we want to see what DRBA contains on the current locale,
+  // we use 'Helper' that is mapped using the default distribution.
+  // 'Helper = DRBA' captures the view of DRBA on the current locale,
+  // which we then print out.
+
+  const Helper: [Space] int = DRBA;
+  writeln(Helper);
+  writeln();
+
+}

samples/Chapel/hello.chpl

@@ -0,0 +1 @@
+writeln("Hello, world!");    // print 'Hello, world!' to the console

samples/Chapel/nbody.chpl

@@ -0,0 +1,147 @@
+/* The Computer Language Benchmarks Game
+   http://benchmarksgame.alioth.debian.org/
+
+   contributed by Albert Sidelnik
+   modified by Brad Chamberlain
+*/
+
+
+//
+// The number of timesteps to simulate; may be set via the command-line
+//
+config const n = 10000;
+
+//
+// Constants representing pi, the solar mass, and the number of days per year
+//
+const pi = 3.141592653589793,
+      solarMass = 4 * pi**2,
+      daysPerYear = 365.24;
+
+//
+// a record representing one of the bodies in the solar system
+//
+record body {
+  var pos: 3*real;
+  var v: 3*real;
+  var mass: real;  // does not change after it is set up
+}
+
+//
+// the array of bodies that we'll be simulating
+//
+var bodies = [/* sun */
+              new body(mass = solarMass),
+
+              /* jupiter */
+              new body(pos = ( 4.84143144246472090e+00,
+                              -1.16032004402742839e+00,
+                              -1.03622044471123109e-01),
+                         v = ( 1.66007664274403694e-03 * daysPerYear,
+                               7.69901118419740425e-03 * daysPerYear,
+                              -6.90460016972063023e-05 * daysPerYear),
+                      mass =   9.54791938424326609e-04 * solarMass),
+  
+              /* saturn */
+              new body(pos = ( 8.34336671824457987e+00,
+                               4.12479856412430479e+00,
+                              -4.03523417114321381e-01),
+                         v = (-2.76742510726862411e-03 * daysPerYear,
+                               4.99852801234917238e-03 * daysPerYear,
+                               2.30417297573763929e-05 * daysPerYear),
+                      mass =   2.85885980666130812e-04 * solarMass),
+
+              /* uranus */
+              new body(pos = ( 1.28943695621391310e+01,
+                              -1.51111514016986312e+01,
+                              -2.23307578892655734e-01),
+                         v = ( 2.96460137564761618e-03 * daysPerYear,
+                               2.37847173959480950e-03 * daysPerYear,
+                              -2.96589568540237556e-05 * daysPerYear),
+                      mass =   4.36624404335156298e-05 * solarMass),
+
+              /* neptune */
+              new body(pos = ( 1.53796971148509165e+01,
+                              -2.59193146099879641e+01,
+                               1.79258772950371181e-01),
+                         v = ( 2.68067772490389322e-03 * daysPerYear,
+                               1.62824170038242295e-03 * daysPerYear,
+                              -9.51592254519715870e-05 * daysPerYear),
+                      mass =   5.15138902046611451e-05 * solarMass)
+              ];
+
+//
+// the number of bodies to be simulated
+//
+const numbodies = bodies.numElements;
+
+//
+// The computation involves initializing the sun's velocity,
+// writing the initial energy, advancing the system through 'n'
+// timesteps, and writing the final energy.
+//
+proc main() {
+  initSun();
+
+  writef("%.9r\n", energy());
+  for 1..n do
+    advance(0.01);
+  writef("%.9r\n", energy());
+}
+
+//
+// compute the sun's initial velocity
+//
+proc initSun() {
+  const p = + reduce (for b in bodies do (b.v * b.mass));
+  bodies[1].v = -p / solarMass;
+}
+
+//
+// advance the positions and velocities of all the bodies
+//
+proc advance(dt) {
+  for i in 1..numbodies {
+    for j in i+1..numbodies {
+      updateVelocities(bodies[i], bodies[j]);
+      
+      inline proc updateVelocities(ref b1, ref b2) {
+        const dpos = b1.pos - b2.pos,
+               mag = dt / sqrt(sumOfSquares(dpos))**3;
+        
+        b1.v -= dpos * b2.mass * mag;
+        b2.v += dpos * b1.mass * mag;
+      }
+    }
+  }
+  
+  for b in bodies do
+    b.pos += dt * b.v;
+}
+
+//
+// compute the energy of the bodies
+//
+proc energy() {
+  var e = 0.0;
+  
+  for i in 1..numbodies {
+    const b1 = bodies[i];
+    
+    e += 0.5 * b1.mass * sumOfSquares(b1.v);
+    
+    for j in i+1..numbodies {
+      const b2 = bodies[j];
+      
+      e -= (b1.mass * b2.mass) / sqrt(sumOfSquares(b1.pos - b2.pos));
+    }
+  }
+  
+  return e;
+}
+
+//
+// a helper routine to compute the sum of squares of a 3-tuple's components
+//
+inline proc sumOfSquares(x)
+  return x(1)**2 + x(2)**2 + x(3)**2;

samples/Chapel/quicksort.chpl

@@ -0,0 +1,145 @@
+//
+// An example of a parallel quick sort implementation that uses
+// "cobegin" to make each recursive call in parallel and "serial" to
+// limit the number of threads.
+//
+
+use Random, Time; // for random number generation and the Timer class
+
+var timer: Timer; // to time the sort
+
+config var n: int = 2**15;      // the size of the array to be sorted
+config var thresh: int = 1;     // the recursive depth to serialize
+config var verbose: int = 0;    // print out this many elements in array
+config var timing: bool = true; // set timing to false to disable timer
+
+var A: [1..n] real; // array of real numbers
+
+//
+// initialize array with random numbers
+//
+fillRandom(A);
+
+//
+// print out front of array if verbose flag is set
+//
+if verbose > 0 then
+  writeln("A[1..", verbose, "] = ", A[1..verbose]);
+
+//
+// start timer, call parallel quick sort routine, stop timer
+//
+if timing then timer.start();
+pqsort(A, thresh);
+if timing then timer.stop();
+
+//
+// report sort time
+//
+if timing then writeln("sorted in ", timer.elapsed(), " seconds");
+
+//
+// print out front of array if verbose flag is set
+//   values should now be in sorted order
+//
+if verbose > 0 then
+  writeln("A[1..", verbose, "] = ", A[1..verbose]);
+
+//
+// verify that array is sorted or halt
+//
+for i in 2..n do
+  if A(i) < A(i-1) then
+    halt("A(", i-1, ") == ", A(i-1), " > A(", i, ") == ", A(i));
+
+writeln("verification success");
+
+//
+// pqsort -- parallel quick sort
+//
+//   arr: generic 1D array of values (real, int, ...)
+//   thresh: number of recursive calls to make before serializing
+//   low: lower bound of array to start sort at, defaults to whole array
+//   high: upper bound of array to stop sort at, defaults to whole array
+//
+proc pqsort(arr: [],
+           thresh: int,
+           low: int = arr.domain.low,
+           high: int = arr.domain.high) where arr.rank == 1 {
+
+  //
+  // base case: arr[low..high] is small enough to bubble sort
+  //
+  if high - low < 8 {
+    bubbleSort(arr, low, high);
+    return;
+  }
+
+  //
+  // determine pivot and partition arr[low..high]
+  //
+  const pivotVal = findPivot();
+  const pivotLoc = partition(pivotVal);
+
+  //
+  // make recursive calls to parallel quick sort each unsorted half of
+  // the array; if thresh is 0 or less, start executing conquer tasks
+  // serially
+  //
+  serial thresh <= 0 do cobegin {
+    pqsort(arr, thresh-1, low, pivotLoc-1);
+    pqsort(arr, thresh-1, pivotLoc+1, high);
+  }
+
+  //
+  // findPivot -- helper routine to find pivot value using simple
+  //              median-of-3 method, returns pivot value
+  //
+  proc findPivot() {
+    const mid = low + (high-low+1) / 2;
+
+    if arr(mid) < arr(low) then arr(mid) <=> arr(low);
+    if arr(high) < arr(low) then arr(high) <=> arr(low);
+    if arr(high) < arr(mid) then arr(high) <=> arr(mid);
+
+    const pivotVal = arr(mid);
+    arr(mid) = arr(high-1);
+    arr(high-1) = pivotVal;
+
+    return pivotVal;
+  }
+
+  //
+  // partition -- helper routine to partition array such that all
+  //              values less than pivot are to its left and all
+  //              values greater than pivot are to its right, returns
+  //              pivot location
+  //
+  proc partition(pivotVal) {
+    var ilo = low, ihi = high-1;
+    while (ilo < ihi) {
+      do { ilo += 1; } while arr(ilo) < pivotVal;
+      do { ihi -= 1; } while pivotVal < arr(ihi);
+      if (ilo < ihi) {
+        arr(ilo) <=> arr(ihi);
+      }
+    }
+    arr(high-1) = arr(ilo);
+    arr(ilo) = pivotVal;
+    return ilo;
+  }
+}
+
+//
+// bubbleSort -- bubble sort for base case of quick sort
+//
+//   arr: generic 1D array of values (real, int, ...)
+//   low: lower bound of array to start sort at
+//   high: upper bound of array to stop sort at
+//
+proc bubbleSort(arr: [], low: int, high: int) where arr.rank == 1 {
+  for i in low..high do
+    for j in low..high-1 do
+      if arr(j) > arr(j+1) then
+        arr(j) <=> arr(j+1);
+}

samples/Clean/GenHylo.dcl

@@ -0,0 +1,11 @@
+definition module GenHylo
+
+import StdGeneric, GenMap
+
+:: Fix f = In (f .(Fix f))
+Out :: !u:(Fix v:a) -> v:(a w:(Fix v:a)), [u <= w]
+
+hylo :: ((.f .b) -> .b) (.a -> (.f .a)) -> (.a -> .b) | gMap{|*->*|} f
+cata :: (u:(f .a) -> .a) -> (Fix u:f) -> .a | gMap{|*->*|} f
+ana :: (.a -> u:(f .a)) -> .a -> (Fix u:f) | gMap{|*->*|} f
+

samples/Clean/GenMap.dcl

@@ -0,0 +1,9 @@
+definition module GenMap
+
+import StdGeneric
+
+generic gMap a b :: .a -> .b
+derive gMap c, UNIT, PAIR, EITHER, CONS, FIELD, OBJECT, {}, {!} 
+
+derive gMap [], (,), (,,),  (,,,), (,,,,), (,,,,,), (,,,,,,), (,,,,,,,)
+

samples/Clean/GenMap.icl

@@ -0,0 +1,19 @@
+implementation module GenMap
+
+import StdClass, StdArray, StdInt, StdFunc
+import StdGeneric, _Array
+
+generic gMap a b :: .a -> .b
+gMap{|c|} x 					= x
+gMap{|UNIT|} x 					= x
+gMap{|PAIR|} fx fy (PAIR x y) 	= PAIR (fx x) (fy y) 
+gMap{|EITHER|} fl fr (LEFT x) 	= LEFT (fl x)
+gMap{|EITHER|} fl fr (RIGHT x) 	= RIGHT (fr x)
+gMap{|CONS|} f (CONS x) 		= CONS (f x)
+gMap{|FIELD|} f (FIELD x) 		= FIELD (f x)
+gMap{|OBJECT|} f (OBJECT x) 	= OBJECT (f x)
+gMap{|{}|} f xs 				= mapArray f xs
+gMap{|{!}|} f xs				= mapArray f xs
+
+derive gMap [], (,), (,,),  (,,,), (,,,,), (,,,,,), (,,,,,,), (,,,,,,,)
+

samples/Clean/fsieve.icl

@@ -0,0 +1,54 @@
+module fsieve
+
+/*
+The Fast Sieve of Eratosthenes.
+
+A sequential and optimized version of the sieve of Eratosthenes.
+The program calculates a list of the first NrOfPrime primes.
+The result of the program is the NrOfPrimes'th prime.
+
+Strictness annotations have been added because the strictness analyser
+is not able to deduce all strictness information. Removal of these !'s
+will make the program about 20% slower.
+
+On a machine without a math coprocessor the execution of this
+program might take a (very) long time. Set NrOfPrimes to a smaller value.
+*/
+
+import StdClass; // RWS
+import StdInt, StdReal
+     
+NrOfPrimes :== 3000 
+	
+//	The sieve algorithm: generate an infinite list of all primes.
+
+Primes::[Int]
+Primes = pr where pr = [5 : Sieve 7 4 pr]
+
+Sieve::Int !Int [Int] -> [Int]
+Sieve g i prs
+	| IsPrime prs g (toInt (sqrt (toReal g)))	=  [g : Sieve` g i prs]
+												=  Sieve (g + i) (6 - i) prs
+
+Sieve`::Int Int [Int] -> [Int]
+Sieve` g i prs =  Sieve (g + i) (6 - i) prs
+
+IsPrime::[Int] !Int Int -> Bool
+IsPrime [f:r] pr bd | f>bd 			=  True
+					| pr rem f==0	=  False
+									=  IsPrime r pr bd
+								  
+//	Select is used to get the NrOfPrimes'th prime from the infinite list.
+
+Select::[x] Int -> x
+Select [f:r] 1 =  f
+Select [f:r] n =  Select r (n - 1)
+
+
+/*	The Start rule: Select the NrOfPrimes'th prime from the list of primes
+	generated by Primes.
+*/
+
+Start::Int
+Start = Select [2, 3 : Primes] NrOfPrimes
+

samples/Clean/sem.icl

@@ -0,0 +1,99 @@
+module monadicSemantics
+
+import StdEnv, StdGeneric, GenMap, GenHylo
+
+/* For fun I implemented the recursive datastructre Exp and Stm as fixpoints
+   This helps us define recursive functions on them (only a little bit though)
+   However deriving gMap for Fix did not works out of the box
+   I had to remove some uniqueness typing in GenMap and GenHylo */
+:: Op      = Plus | Minus | Times | Rem | Equal | LessThan
+:: Var     :== String
+
+:: ExpP a  = Int Int | Var Var | Op Op a a
+:: Exp     :== Fix ExpP
+
+:: StmP a  = Assign Var Exp | If Exp a a | While Exp a | Seq a a | Cont
+:: Stm     :== Fix StmP
+
+derive gMap ExpP, StmP, Fix
+
+// Environment. Semantics is basically Env -> Env
+:: Env :== Var -> Int
+:: Sem :== Env -> (Int, Env)
+empty = \v . 0
+
+// return
+rtn :: Int -> Sem
+rtn i = \e. (i, e)
+
+// the usual bind
+(>>=) infixl 1 :: Sem (Int->Sem) -> Sem
+(>>=) x y = \e. (\(i,e2).y i e2) (x e)
+(>>|) infixl 1 :: Sem Sem -> Sem
+(>>|) x y = x >>= \_. y
+
+// read variable from environment
+read :: Var -> Sem
+read v = \e. (e v, e)
+
+// assign value to give variable in environment
+write :: Var Int -> Sem
+write v i = \e. (i, \w. if (w==v) i (e w))
+
+// semantics
+class sem a :: a -> Sem
+
+operator :: Op -> Int -> Int -> Int
+operator Plus     = (+)
+operator Minus    = (-)
+operator Times    = (*)
+operator Rem      = rem
+operator Equal    = \x y . if (x==y) 1 0
+operator LessThan = \x y . if (x< y)  1 0
+
+// semantics of expressions
+instance sem Exp where
+	sem x = cata phi x where
+		phi (Int n)     = rtn n
+		phi (Var v)     = read v
+		phi (Op op x y) = x >>= \v1. y >>= return o (operator op v1)
+
+// semantics of statments
+// NOTE: while will always return 0, as it might not even be executed
+instance sem Stm where
+	sem x = cata phi x where
+		phi (Assign v e)     = sem e >>= write v
+		phi (If e s1 s2)     = sem e >>= \b . if (b<>0) s1 s2
+		phi stm=:(While e s) = sem e >>= \b . if (b<>0) (s >>| phi stm) (phi Cont)
+		phi (Seq s1 s2)      = s1 >>| s2    // Here the cata *finally* pays off :D
+		phi Cont             = rtn 0
+
+// convenience functions
+int    = In o Int
+var    = In o Var
+op o   = In o2 (Op o)
+assign = In o2 Assign
+ifte e = In o2 (If e)
+while  = In o2 While
+seq    = In o2 Seq
+cont   = In Cont
+
+// test case, also testing the new operator <
+pEuclides =
+	while (op LessThan (int 0) (var "b"))(
+		seq (assign "r" (op Rem (var "a") (var "b")))
+		(seq (assign "a" (var "b"))
+		( (assign "b" (var "r")))
+		)
+	)
+
+Start = fst (program start) where
+	program = sem pEuclides >>| read "a"
+	start "a" = 9
+	start "b" = 12
+	start _ = 0
+
+// Helper
+(o2) infixr 9
+(o2) f g x :== f o (g x)
+

samples/Clean/stack.dcl

@@ -0,0 +1,14 @@
+definition module stack
+
+:: Stack a
+
+newStack :: (Stack a)
+push :: a (Stack a) -> Stack a
+pushes :: [a] (Stack a) -> Stack a
+pop :: (Stack a) -> Stack a
+popn :: Int (Stack a) -> Stack a
+top :: (Stack a) -> a
+topn :: Int (Stack a) -> [a]
+elements :: (Stack a) -> [a]
+count :: (Stack a) -> Int
+

samples/Clean/stack.icl

@@ -0,0 +1,33 @@
+implementation module stack
+import StdEnv
+
+:: Stack a :== [a]
+
+newStack :: (Stack a)
+newStack = []
+
+push :: a (Stack a) -> Stack a
+push x s = [x:s]
+
+pushes :: [a] (Stack a) -> Stack a
+pushes x s = x ++ s
+
+pop :: (Stack a) -> Stack a
+pop [] = abort "Cannot use pop on an empty stack"
+pop [e:s] = s
+
+popn :: Int (Stack a) -> Stack a
+popn n s  = drop n s
+
+top :: (Stack a) -> a
+top [] = abort "Cannot use top on an empty stack"
+top [e:s] = e
+
+topn :: Int (Stack a) -> [a]
+topn n s = take n s
+elements :: (Stack a) -> [a]
+elements s = s
+
+count :: (Stack a) -> Int
+count s = length s
+

samples/Clean/streams.dcl

@@ -0,0 +1,16 @@
+definition module streams
+
+import StdEnv
+
+instance zero [Real]
+instance one [Real]
+instance + [Real]        
+instance - [Real]
+instance * [Real]
+instance / [Real]
+
+X :: [Real]
+invert :: [Real] -> [Real]
+pow :: [Real] Int -> [Real]
+(shuffle) infixl 7 :: [Real] [Real] -> [Real]
+

samples/Clean/streams.icl

@@ -0,0 +1,49 @@
+implementation module streams
+
+import StdEnv
+
+instance zero [Real]
+where
+        zero = [] //Infinite row of zeroes represented as empty list to ease computation
+
+instance one [Real]
+where
+        one = [1.0:zero]
+
+instance + [Real]
+where
+        (+) [s:s`] [t:t`] = [s+t:s`+t`]
+        (+) [s:s`] [] = [s:s`]
+        (+) [] [t:t`] = [t:t`]
+        (+) [] [] = []
+        
+instance - [Real]
+where
+        (-) [s:s`] [t:t`] = [s-t:s`-t`]
+        (-) [s:s`] [] = [s:s`]
+        (-) [] [t:t`] = [-1.0] * [t:t`]
+        (-) [] [] = []
+
+instance * [Real]
+where
+        (*) [s:s`] [t:t`] = [s*t:s`*[t:t`]+[s]*t`]
+        (*) _ _ = []
+
+instance / [Real]
+where
+        (/) s t = s * (invert t)
+
+X :: [Real]
+X = [0.0:one]
+
+invert :: [Real] -> [Real]
+invert [s:s`] = [1.0/s:(invert [s:s`]) * s` * [-1.0/s]]
+
+pow :: [Real] Int -> [Real]
+pow s 0 = one
+pow s n = s * pow s (n-1)
+
+(shuffle) infixl 7 :: [Real] [Real] -> [Real]
+(shuffle) [s:s`] [t:t`] = [s*t:s` shuffle [t:t`] + [s:s`] shuffle t`]
+(shuffle) _ _ = []
+

samples/Clojure/index.cljs.hl

@@ -0,0 +1,146 @@
+;; Copyright (c) Alan Dipert and Micha Niskin. All rights reserved.
+;; The use and distribution terms for this software are covered by the
+;; Eclipse Public License 1.0 (http://opensource.org/licenses/eclipse-1.0.php)
+;; which can be found in the file epl-v10.html at the root of this distribution.
+;; By using this software in any fashion, you are agreeing to be bound by
+;; the terms of this license.
+;; You must not remove this notice, or any other, from this software.
+
+(page "index.html"
+  (:refer-clojure :exclude [nth])
+  (:require
+   [tailrecursion.hoplon.reload        :refer [reload-all]]
+   [tailrecursion.hoplon.util          :refer [nth name pluralize]]
+   [tailrecursion.hoplon.storage-atom  :refer [local-storage]]))
+
+;; utility functions ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(declare route state editing)
+
+(reload-all)
+
+(def mapvi  (comp vec map-indexed))
+
+(defn dissocv [v i]
+  (let [z (- (dec (count v)) i)]
+    (cond (neg?  z) v
+          (zero? z) (pop v)
+          (pos?  z) (into (subvec v 0 i) (subvec v (inc i))))))
+
+(defn decorate [todo route editing i]
+  (let [{done? :completed text :text} todo]
+    (-> todo (assoc :editing (= editing i)
+                    :visible (and (not (empty? text))
+                                  (or (= "#/" route)
+                                      (and (= "#/active" route) (not done?))
+                                      (and (= "#/completed" route) done?)))))))
+
+;; persisted state cell (AKA: stem cell) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(def   state        (-> (cell []) (local-storage ::store)))
+
+;; local state cells ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(defc  loaded?      false)
+(defc  editing      nil)
+(def   route        (route-cell "#/"))
+
+;; formula cells (computed state) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(defc= completed    (filter :completed state))
+(defc= active       (remove :completed state))
+(defc= plural-item  (pluralize "item" (count active)))
+(defc= todos        (mapvi #(list %1 (decorate %2 route editing %1)) state))
+
+;; state transition functions ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(defn  todo        [t]   {:completed false :text t})
+(defn  destroy!    [i]   (swap! state dissocv i))
+(defn  done!       [i v] (swap! state assoc-in [i :completed] v))
+(defn  clear-done! [& _] (swap! state #(vec (remove :completed %))))
+(defn  new!        [t]   (when (not (empty? t)) (swap! state conj (todo t))))
+(defn  all-done!   [v]   (swap! state #(mapv (fn [x] (assoc x :completed v)) %)))
+(defn  editing!    [i v] (reset! editing (if v i nil)))
+(defn  text!       [i v] (if (empty? v) (destroy! i) (swap! state assoc-in [i :text] v)))
+
+;; page ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(html :lang "en"
+  (head
+    (meta :charset "utf-8")
+    (meta :http-equiv "X-UA-Compatible" :content "IE=edge,chrome=1")
+    (link :rel "stylesheet" :href "base.css")
+    (title "Hoplon • TodoMVC"))
+  (body
+    (noscript
+      (div :id "noscript"
+        (p "JavaScript is required to view this page.")))
+    (div
+      (section :id "todoapp"
+        (header :id "header"
+          (h1 "todos")
+          (form :on-submit #(do (new! (val-id :new-todo))
+                                (do! (by-id :new-todo) :value ""))
+            (input
+              :id "new-todo"
+              :type "text"
+              :autofocus true
+              :placeholder "What needs to be done?"
+              :on-blur #(do! (by-id :new-todo) :value ""))))
+        (section
+          :id "main"
+          :do-toggle (cell= (not (and (empty? active) (empty? completed))))
+          (input
+            :id "toggle-all"
+            :type "checkbox"
+            :do-attr (cell= {:checked (empty? active)}) 
+            :on-click #(all-done! (val-id :toggle-all)))
+          (label :for "toggle-all"
+            "Mark all as complete")
+          (ul :id "todo-list"
+            (loop-tpl
+              :reverse true
+              :bind-ids [done# edit#]
+              :bindings [[i {edit? :editing done? :completed todo-text :text show? :visible}] todos] 
+              (li
+                :do-class (cell= {:completed done? :editing edit?}) 
+                :do-toggle show?
+                (div :class "view" :on-dblclick #(editing! @i true)
+                  (input
+                    :id done# 
+                    :type "checkbox"
+                    :class "toggle"
+                    :do-attr (cell= {:checked done?}) 
+                    :on-click #(done! @i (val-id done#)))
+                  (label (text "~{todo-text}"))
+                  (button
+                    :type "submit"
+                    :class "destroy"
+                    :on-click  #(destroy! @i)))
+                (form :on-submit #(editing! @i false)
+                  (input
+                    :id edit#
+                    :type "text"
+                    :class "edit"
+                    :do-value todo-text
+                    :do-focus edit?
+                    :on-blur #(when @edit? (editing! @i false))
+                    :on-change #(when @edit? (text! @i (val-id edit#)))))))))
+        (footer 
+          :id "footer"
+          :do-toggle (cell= (not (and (empty? active) (empty? completed))))
+          (span :id "todo-count"
+            (strong (text "~(count active) "))
+            (span (text "~{plural-item} left")))
+          (ul :id "filters"
+            (li (a :href "#/"          :do-class (cell= {:selected (= "#/" route)})          "All"))
+            (li (a :href "#/active"    :do-class (cell= {:selected (= "#/active" route)})    "Active"))
+            (li (a :href "#/completed" :do-class (cell= {:selected (= "#/completed" route)}) "Completed")))
+          (button
+            :type      "submit"
+            :id        "clear-completed"
+            :on-click  #(clear-done!)
+            (text "Clear completed (~(count completed))"))))
+      (footer :id "info" 
+        (p "Double-click to edit a todo")
+        (p "Part of " (a :href "http://github.com/tailrecursion/hoplon-demos/" "hoplon-demos")))))) 

samples/CoffeeScript/example.cjsx

@@ -0,0 +1,40 @@
+###* @cjsx React.DOM ###
+define 'myProject.ReactExampleComponent', [
+  'React'
+  'myProject.ExampleStore'
+  'myProject.ExampleActions'
+  'myProject.ReactExampleTable'
+], (React, ExampleStore, ExampleActions, ReactExampleTable ) ->
+
+  ReactExampleComponent = React.createClass
+    mixins: [ListenMixin]
+
+    getInitialState: ->
+      rows: ExampleStore.getRows()
+      meta: ExampleStore.getMeta()
+
+    componentWillMount: ->
+      @listenTo ExampleStore
+
+    componentDidMount: ->
+      ExampleActions.getExampleData()
+
+    onStoreChange: ->
+      if this.isMounted()
+        @setState
+          rows: ExampleStore.getRows()
+          meta: ExampleStore.getMeta()
+
+    componentWillUnmount: ->
+      @stopListening ExampleStore
+
+    render: ->
+      <div className="page-wrap">
+          <header>
+            <strong> {@state.title} </strong>
+          <header>
+            <ReactExampleTable
+              rows={@state.rows},
+              meta={@state.meta}
+            />
+      </div>

samples/ColdFusion CFC/exampleScript.cfc

@@ -0,0 +1,239 @@
+/**
+********************************************************************************
+ContentBox - A Modular Content Platform
+Copyright 2012 by Luis Majano and Ortus Solutions, Corp
+www.gocontentbox.org | www.luismajano.com | www.ortussolutions.com
+********************************************************************************
+Apache License, Version 2.0
+
+Copyright Since [2012] [Luis Majano and Ortus Solutions,Corp]
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+********************************************************************************
+* A generic content service for content objects
+*/
+component extends="coldbox.system.orm.hibernate.VirtualEntityService" singleton{
+
+	// DI
+	property name="settingService"			inject="id:settingService@cb";
+	property name="cacheBox"				inject="cachebox";
+	property name="log"						inject="logbox:logger:{this}";
+	property name="customFieldService" 	 	inject="customFieldService@cb";
+	property name="categoryService" 	 	inject="categoryService@cb";
+	property name="commentService" 	 		inject="commentService@cb";
+	property name="contentVersionService"	inject="contentVersionService@cb";
+	property name="authorService"			inject="authorService@cb";
+	property name="populator"				inject="wirebox:populator";
+	property name="systemUtil"				inject="SystemUtil@cb";
+	
+	/*
+	* Constructor
+	* @entityName.hint The content entity name to bind this service to.
+	*/
+	ContentService function init(entityName="cbContent"){
+		// init it
+		super.init(entityName=arguments.entityName, useQueryCaching=true);
+
+		// Test scope coloring in pygments
+		this.colorTestVar = "Just for testing pygments!";
+		cookie.colorTestVar = "";
+		client.colorTestVar = ""
+		session.colorTestVar = "";
+		application.colorTestVar = "";
+
+		return this;
+	}
+
+	/**
+	* Clear all content caches
+	* @async.hint Run it asynchronously or not, defaults to false
+	*/
+	function clearAllCaches(boolean async=false){
+		var settings = settingService.getAllSettings(asStruct=true);
+		// Get appropriate cache provider
+		var cache = cacheBox.getCache( settings.cb_content_cacheName );
+		cache.clearByKeySnippet(keySnippet="cb-content",async=arguments.async);
+		return this;
+	}
+
+	/**
+	* Clear all page wrapper caches
+	* @async.hint Run it asynchronously or not, defaults to false
+	*/
+	function clearAllPageWrapperCaches(boolean async=false){
+		var settings = settingService.getAllSettings(asStruct=true);
+		// Get appropriate cache provider
+		var cache = cacheBox.getCache( settings.cb_content_cacheName );
+		cache.clearByKeySnippet(keySnippet="cb-content-pagewrapper",async=arguments.async);
+		return this;
+	}
+
+	/**
+	* Clear all page wrapper caches
+	* @slug.hint The slug partial to clean on
+	* @async.hint Run it asynchronously or not, defaults to false
+	*/
+	function clearPageWrapperCaches(required any slug, boolean async=false){
+		var settings = settingService.getAllSettings(asStruct=true);
+		// Get appropriate cache provider
+		var cache = cacheBox.getCache( settings.cb_content_cacheName );
+		cache.clearByKeySnippet(keySnippet="cb-content-pagewrapper-#arguments.slug#",async=arguments.async);
+		return this;
+	}
+
+	/**
+	* Clear a page wrapper cache
+	* @slug.hint The slug to clean
+	* @async.hint Run it asynchronously or not, defaults to false
+	*/
+	function clearPageWrapper(required any slug, boolean async=false){
+		var settings = settingService.getAllSettings(asStruct=true);
+		// Get appropriate cache provider
+		var cache = cacheBox.getCache( settings.cb_content_cacheName );
+		cache.clear("cb-content-pagewrapper-#arguments.slug#/");
+		return this;
+	}
+
+	/**
+	* Searches published content with cool paramters, remember published content only
+	* @searchTerm.hint The search term to search
+	* @max.hint The maximum number of records to paginate
+	* @offset.hint The offset in the pagination
+	* @asQuery.hint Return as query or array of objects, defaults to array of objects
+	* @sortOrder.hint The sorting of the search results, defaults to publishedDate DESC
+	* @isPublished.hint Search for published, non-published or both content objects [true, false, 'all']
+	* @searchActiveContent.hint Search only content titles or both title and active content. Defaults to both.
+	*/
+	function searchContent(
+		any searchTerm="", 
+		numeric max=0, 
+		numeric offset=0, 
+		boolean asQuery=false, 
+		any sortOrder="publishedDate DESC", 
+		any isPublished=true, 
+		boolean searchActiveContent=true){
+
+		var results = {};
+		var c = newCriteria();
+
+		// only published content
+		if( isBoolean( arguments.isPublished ) ){
+			// Published bit
+			c.isEq( "isPublished", javaCast( "Boolean", arguments.isPublished ) );
+			// Published eq true evaluate other params
+			if( arguments.isPublished ){
+				c.isLt("publishedDate", now() )
+				.$or( c.restrictions.isNull("expireDate"), c.restrictions.isGT("expireDate", now() ) )
+				.isEq("passwordProtection","");
+			}
+		}
+
+		// Search Criteria
+		if( len( arguments.searchTerm ) ){
+			// like disjunctions
+			c.createAlias("activeContent","ac");
+			// Do we search title and active content or just title?
+			if( arguments.searchActiveContent ){
+				c.$or( c.restrictions.like("title","%#arguments.searchTerm#%"),
+				  	  c.restrictions.like("ac.content", "%#arguments.searchTerm#%") );
+			}
+			else{
+				c.like( "title", "%#arguments.searchTerm#%" ); 
+			}
+		}
+
+		// run criteria query and projections count
+		results.count = c.count( "contentID" );
+		results.content = c.resultTransformer( c.DISTINCT_ROOT_ENTITY )
+							.list(offset=arguments.offset, max=arguments.max, sortOrder=arguments.sortOrder, asQuery=arguments.asQuery);
+	
+		return results;
+	}
+
+/********************************************* PRIVATE *********************************************/
+	
+
+	/**
+	* Update the content hits
+	* @contentID.hint The content id to update
+	*/
+	private function syncUpdateHits(required contentID){
+		var q = new Query(sql="UPDATE cb_content SET hits = hits + 1 WHERE contentID = #arguments.contentID#").execute();
+		return this;
+	}
+	
+	
+	private function closureTest(){
+		methodCall(
+			param1,
+			function( arg1, required arg2 ){
+				var settings = settingService.getAllSettings(asStruct=true);
+				// Get appropriate cache provider
+				var cache = cacheBox.getCache( settings.cb_content_cacheName );
+				cache.clear("cb-content-pagewrapper-#arguments.slug#/");
+				return this;
+			},
+			param1
+		);
+	}
+	
+	private function StructliteralTest(){
+		return {
+			foo = bar,
+			brad = 'Wood',
+			func = function( arg1, required arg2 ){
+				var settings = settingService.getAllSettings(asStruct=true);
+				// Get appropriate cache provider
+				var cache = cacheBox.getCache( settings.cb_content_cacheName );
+				cache.clear("cb-content-pagewrapper-#arguments.slug#/");
+				return this;
+			},
+			array = [
+				1,
+				2,
+				3,
+				4,
+				5,
+				'test',
+				'testing',
+				'testerton',
+				{
+					foo = true,
+					brad = false,
+					wood = null
+				}
+			],
+			last = "final"
+		};
+	}
+	
+	private function arrayliteralTest(){
+		return [
+			1,
+			2,
+			3,
+			4,
+			5,
+			'test',
+			'testing',
+			'testerton',
+			{
+				foo = true,
+				brad = false,
+				wood = null
+			},
+			'testy-von-testavich'
+		];
+	}
+	
+}

samples/ColdFusion CFC/exampleTag.cfc

@@ -0,0 +1,18 @@
+<cfcomponent>
+	
+	<cffunction name="init" access="public" returntype="any">
+		<cfargument name="arg1" type="any" required="true">
+		<cfset this.myVariable = arguments.arg1>
+
+		<cfreturn this>
+	</cffunction>
+
+	<cffunction name="testFunc" access="private" returntype="void">
+		<cfargument name="arg1" type="any" required="false">
+		
+		<cfif structKeyExists(arguments, "arg1")>
+			<cfset writeoutput("Argument exists")>
+		</cfif>
+	</cffunction>
+	
+</cfcomponent>

samples/ColdFusion/example.cfm

@@ -0,0 +1,50 @@
+<!--- cfcomment --->
+<!--- nested <!--- cfcomment ---> --->
+<!--- multi-line
+nested
+<!---
+cfcomment
+--->
+--->
+<!-- html comment -->
+<html>
+<head>
+<title>Date Functions</title>
+</head>
+<body>
+<cfset RightNow = Now()>
+<cfoutput>
+ #RightNow#<br />
+ #DateFormat(RightNow)#<br />
+ #DateFormat(RightNow,"mm/dd/yy")#<br />
+ #TimeFormat(RightNow)#<br />
+ #TimeFormat(RightNow,"hh:mm tt")#<br />
+ #IsDate(RightNow)#<br />
+ #IsDate("January 31, 2007")#<br />
+ #IsDate("foo")#<br />
+ #DaysInMonth(RightNow)#
+</cfoutput>
+<cfset x="x">
+<cfset y="y">
+<cfset z="z">
+<cfoutput group="x">
+    #x#
+    <cfoutput>#y#</cfoutput>
+    #z#
+</cfoutput>
+</body>
+</html>
+
+<cfset person = "Paul">
+<cfset greeting = "Hello #person#">
+
+<cfset greeting = "Hello" & " world!">
+<cfset a = 5>
+<cfset b = 10>
+<cfset c = a^b>
+<cfset c = a MOD b>
+<cfset c = a / b>
+<cfset c = a * b>
+<cfset c = a + b>
+<cfset c = a - b>
+<!--- <!-- another <!--- nested --> ---> comment --->

samples/Component Pascal/Example.cp

@@ -0,0 +1,130 @@
+MODULE ObxControls;
+(**
+    project         = "BlackBox"
+    organization    = "www.oberon.ch"
+    contributors    = "Oberon microsystems"
+    version         = "System/Rsrc/About"
+    copyright       = "System/Rsrc/About"
+    license         = "Docu/BB-License"
+    changes         = ""
+    issues          = ""
+
+**)
+
+IMPORT Dialog, Ports, Properties, Views;
+
+CONST beginner = 0; advanced = 1; expert = 2; guru = 3;    (* user classes *)
+
+TYPE
+    View = POINTER TO RECORD (Views.View)
+        size: INTEGER    (* border size in mm *)
+    END;
+
+VAR
+    data*: RECORD
+        class*: INTEGER;    (* current user class *)
+        list*: Dialog.List;    (* list of currently available sizes, derived from class *)
+        width*: INTEGER    (* width of next view to be opened. Derived from
+                                    class, or entered through a text entry field *)
+    END;
+
+    predef: ARRAY 6 OF INTEGER;    (* table of predefined sizes *)
+
+
+PROCEDURE SetList;
+BEGIN
+    IF data.class = beginner THEN
+        data.list.SetLen(1);
+        data.list.SetItem(0, "default")
+    ELSIF data.class = advanced THEN
+        data.list.SetLen(4);
+        data.list.SetItem(0, "default");
+        data.list.SetItem(1, "small");
+        data.list.SetItem(2, "medium");
+        data.list.SetItem(3, "large");
+    ELSE
+        data.list.SetLen(6);
+        data.list.SetItem(0, "default");
+        data.list.SetItem(1, "small");
+        data.list.SetItem(2, "medium");
+        data.list.SetItem(3, "large");
+        data.list.SetItem(4, "tiny");
+        data.list.SetItem(5, "huge");
+    END
+END SetList;
+
+(* View *)
+
+PROCEDURE (v: View) CopyFromSimpleView (source: Views.View);
+BEGIN
+    v.size := source(View).size
+END CopyFromSimpleView;
+
+PROCEDURE (v: View) Restore (f: Views.Frame; l, t, r, b: INTEGER);
+BEGIN    (* fill view with a red square of size v.size *)
+    IF v.size = 0 THEN v.size := predef[0] END;    (* lazy initialization of size *)
+    f.DrawRect(0, 0, v.size, v.size, Ports.fill, Ports.red)
+END Restore;
+
+PROCEDURE (v: View) HandlePropMsg (VAR msg: Views.PropMessage);
+BEGIN
+    WITH msg: Properties.SizePref DO
+        IF v.size = 0 THEN v.size := predef[0] END;    (* lazy initialization of size *)
+        msg.w := v.size; msg.h := v.size    (* tell environment about desired width and height *)
+    ELSE    (* ignore other messages *)
+    END
+END HandlePropMsg;
+
+(* notifiers *)
+
+PROCEDURE ClassNotify* (op, from, to: INTEGER);
+BEGIN    (* react to change in data.class *)
+    IF op = Dialog.changed THEN
+        IF (to = beginner) OR (to = advanced) & (data.list.index > 3) THEN
+            (* if class is reduced, make sure that selection contains legal elements *)
+            data.list.index := 0; data.width := predef[0];    (* modify interactor *)
+            Dialog.Update(data)    (* redraw controls where necessary *)
+        END;
+        SetList;
+        Dialog.UpdateList(data.list)    (* reconstruct list box contents *)
+    END
+END ClassNotify;
+
+PROCEDURE ListNotify* (op, from, to: INTEGER);
+BEGIN    (* reacto to change in data.list (index to was selected) *)
+    IF op = Dialog.changed THEN
+        data.width := predef[to];    (* modify interactor *)
+        Dialog.Update(data)    (* redraw controls where necessary *)
+    END
+END ListNotify;
+
+(* guards *)
+
+PROCEDURE ListGuard* (VAR par: Dialog.Par);
+BEGIN    (* disable list box for a beginner *)
+    par.disabled := data.class = beginner
+END ListGuard;
+
+PROCEDURE WidthGuard* (VAR par: Dialog.Par);
+BEGIN    (* make text entry field read-only if user is not guru *)
+    par.readOnly := data.class # guru
+END WidthGuard;
+
+(* commands *)
+
+PROCEDURE Open*;
+    VAR v: View;
+BEGIN
+    NEW(v);    (* create and initialize a new view *)
+    v.size := data.width * Ports.mm;    (* define view's size in function of class *)
+    Views.OpenAux(v, "Example")    (* open the view in a window *)
+END Open;
+
+BEGIN    (* initialization of global variables *)
+    predef[0] := 40; predef[1] := 30; predef[2] := 50;    (* predefined sizes *)
+    predef[3] := 70; predef[4] := 20; predef[5] := 100;
+    data.class := beginner;    (* default values *)
+    data.list.index := 0;
+    data.width := predef[0];
+    SetList
+END ObxControls.

samples/Component Pascal/Example2.cps

@@ -0,0 +1,71 @@
+MODULE ObxFact;
+(**
+    project         = "BlackBox"
+    organization    = "www.oberon.ch"
+    contributors    = "Oberon microsystems"
+    version         = "System/Rsrc/About"
+    copyright       = "System/Rsrc/About"
+    license         = "Docu/BB-License"
+    changes         = ""
+    issues          = ""
+
+**)
+
+IMPORT
+    Stores, Models, TextModels, TextControllers, Integers;
+
+PROCEDURE Read(r: TextModels.Reader; VAR x: Integers.Integer);
+    VAR i, len, beg: INTEGER; ch: CHAR; buf: POINTER TO ARRAY OF CHAR;
+BEGIN
+    r.ReadChar(ch);
+    WHILE ~r.eot & (ch <= " ") DO r.ReadChar(ch) END;
+    ASSERT(~r.eot & (((ch >= "0") & (ch <= "9")) OR (ch = "-")));
+    beg := r.Pos() - 1; len := 0;
+    REPEAT INC(len); r.ReadChar(ch) UNTIL r.eot OR (ch < "0") OR (ch > "9");
+    NEW(buf, len + 1);
+    i := 0; r.SetPos(beg);
+    REPEAT r.ReadChar(buf[i]); INC(i) UNTIL i = len;
+    buf[i] := 0X;
+    Integers.ConvertFromString(buf^, x)
+END Read;
+
+PROCEDURE Write(w: TextModels.Writer; x: Integers.Integer);
+    VAR i: INTEGER;
+BEGIN
+    IF Integers.Sign(x) < 0 THEN w.WriteChar("-") END;
+    i := Integers.Digits10Of(x);
+    IF i # 0 THEN
+        REPEAT DEC(i); w.WriteChar(Integers.ThisDigit10(x, i)) UNTIL i = 0
+    ELSE w.WriteChar("0")
+    END
+END Write;
+
+PROCEDURE Compute*;
+    VAR beg, end, i, n: INTEGER; ch: CHAR;
+        s: Stores.Operation;
+        r: TextModels.Reader; w: TextModels.Writer; attr: TextModels.Attributes;
+        c: TextControllers.Controller;
+        x: Integers.Integer;
+BEGIN
+    c := TextControllers.Focus();
+    IF (c # NIL) & c.HasSelection() THEN
+        c.GetSelection(beg, end);
+        r := c.text.NewReader(NIL); r.SetPos(beg); r.ReadChar(ch);
+        WHILE ~r.eot & (beg < end) & (ch <= " ") DO r.ReadChar(ch); INC(beg) END;
+        IF ~r.eot & (beg < end) THEN
+            r.ReadPrev; Read(r, x);
+            end := r.Pos(); r.ReadPrev; attr :=r.attr;
+            IF (Integers.Sign(x) > 0) & (Integers.Compare(x, Integers.Long(MAX(LONGINT))) <= 0) THEN
+                n := SHORT(Integers.Short(x)); i := 2; x := Integers.Long(1);
+                WHILE i <= n DO x := Integers.Product(x, Integers.Long(i)); INC(i) END;
+                Models.BeginScript(c.text, "computation", s);
+                c.text.Delete(beg, end);
+                w := c.text.NewWriter(NIL); w.SetPos(beg); w.SetAttr(attr);
+                Write(w, x);
+                Models.EndScript(c.text, s)
+            END
+        END
+    END
+END Compute;
+
+END ObxFact.

samples/Cool/list.cl

@@ -0,0 +1,26 @@
+(* This simple example of a list class is adapted from an example in the
+   Cool distribution. *)
+
+class List {
+   isNil() : Bool { true };
+   head()  : Int { { abort(); 0; } };
+   tail()  : List { { abort(); self; } };
+   cons(i : Int) : List {
+      (new Cons).init(i, self)
+   };
+};
+
+class Cons inherits List {
+   car : Int;	-- The element in this list cell
+   cdr : List;	-- The rest of the list
+   isNil() : Bool { false };
+   head()  : Int { car };
+   tail()  : List { cdr };
+   init(i : Int, rest : List) : List {
+      {
+	 car <- i;
+	 cdr <- rest;
+	 self;
+      }
+   };
+};

samples/Cool/sample.cl

@@ -0,0 +1,71 @@
+(* Refer to Alex Aiken, "The Cool Reference Manual":
+    http://theory.stanford.edu/~aiken/software/cool/cool-manual.pdf
+   for language specification.
+*)
+
+-- Exhibit various language constructs
+class Sample {
+    testCondition(x: Int): Bool {
+        if x = 0
+        then false
+        else
+            if x < (1 + 2) * 3
+            then true
+            else false
+            fi
+        fi
+    };
+
+    testLoop(y: Int): Bool {
+        while y > 0 loop
+        {
+            if not condition(y)
+            then y <- y / 2
+            else y <- y - 1;
+        }
+        pool
+    };
+
+    testAssign(z: Int): Bool {
+        i : Int;
+        i <- ~z;
+    };
+
+    testCase(var: Sample): SELF_TYPE {
+        io : IO <- new IO;
+        case var of
+            a : A => io.out_string("Class type is A\n");
+            b : B => io.out_string("Class type is B\n");
+            s : Sample => io.out_string("Class type is Sample\n");
+            o : Object => io.out_string("Class type is object\n");
+        esac
+    };
+
+    testLet(i: Int): Int {
+        let (a: Int in
+            let(b: Int <- 3, c: Int <- 4 in
+                {
+                    a <- 2;
+                    a * b * 2 / c;
+                }
+            )
+        )
+    };
+};
+
+-- Used to test subclasses
+class A inherits Sample {};
+class B inherits A {};
+
+class C {
+    main() : Int {
+        (new Sample).testLet(1)
+    };
+};
+
+-- "Hello, world" example
+class Main inherits IO {
+    main(): SELF_TYPE {
+        out_string("Hello, World.\n")
+    };
+};

samples/Cycript/utils.cy

@@ -0,0 +1,580 @@
+(function(utils) {
+	// Load C functions declared in utils.loadFuncs
+	var shouldLoadCFuncs = true;
+	// Expose the C functions to cycript's global scope
+	var shouldExposeCFuncs = true;
+	// Expose C constants to cycript's global scope
+	var shouldExposeConsts = true;
+	// Expose functions defined here to cycript's global scope
+	var shouldExposeFuncs = true;
+	// Which functions to expose
+	var funcsToExpose = ["exec", "include", "sizeof", "logify", "apply", "str2voidPtr", "voidPtr2str", "double2voidPtr", "voidPtr2double", "isMemoryReadable", "isObject", "makeStruct"];
+	
+	// C functions that utils.loadFuncs loads
+	var CFuncsDeclarations = [
+		// <stdlib.h>
+		"void *calloc(size_t num, size_t size)",
+		// <string.h>
+		"char *strcpy(char *restrict dst, const char *restrict src)",
+		"char *strdup(const char *s1)",
+		"void* memset(void* dest, int ch, size_t count)",
+		// <stdio.h>
+		"FILE *fopen(const char *, const char *)",
+		"int fclose(FILE *)",
+		"size_t fread(void *restrict, size_t, size_t, FILE *restrict)",
+		"size_t fwrite(const void *restrict, size_t, size_t, FILE *restrict)",
+		// <mach.h>
+		"mach_port_t mach_task_self()",
+		"kern_return_t task_for_pid(mach_port_name_t target_tport, int pid, mach_port_name_t *tn)",
+		"kern_return_t mach_vm_protect(vm_map_t target_task, mach_vm_address_t address, mach_vm_size_t size, boolean_t set_maximum, vm_prot_t new_protection)",
+		"kern_return_t mach_vm_write(vm_map_t target_task, mach_vm_address_t address, vm_offset_t data, mach_msg_type_number_t dataCnt)",
+		"kern_return_t mach_vm_read(vm_map_t target_task, mach_vm_address_t address, mach_vm_size_t size, vm_offset_t *data, mach_msg_type_number_t *dataCnt)",
+	];
+	
+	/*
+		Replacement for eval that can handle @encode etc.
+		
+		Usage:
+			cy# utils.exec("@encode(void *(int, char))")
+			@encode(void*(int,char))
+	*/
+	utils.exec = function(str) {
+		var mkdir = @encode(int (const char *, int))(dlsym(RTLD_DEFAULT, "mkdir"));
+		var tempnam = @encode(char *(const char *, const char *))(dlsym(RTLD_DEFAULT, "tempnam"));
+		var fopen = @encode(void *(const char *, const char *))(dlsym(RTLD_DEFAULT, "fopen"));
+		var fclose = @encode(int (void *))(dlsym(RTLD_DEFAULT, "fclose"));
+		var fwrite = @encode(int (const char *, int, int, void *))(dlsym(RTLD_DEFAULT, "fwrite"));
+		var symlink = @encode(int (const char *, const char *))(dlsym(RTLD_DEFAULT, "symlink"));
+		var unlink = @encode(int (const char *))(dlsym(RTLD_DEFAULT, "unlink"));
+		var getenv = @encode(const char *(const char *))(dlsym(RTLD_DEFAULT, "getenv"));
+		var setenv = @encode(int (const char *, const char *, int))(dlsym(RTLD_DEFAULT, "setenv"));
+		
+		var libdir = "/usr/lib/cycript0.9";
+		var dir = libdir + "/tmp";
+
+		mkdir(dir, 0777);
+		
+		// This is needed because tempnam seems to ignore the first argument on i386
+		var old_tmpdir = getenv("TMPDIR");
+		setenv("TMPDIR", dir, 1);
+
+		// No freeing :(
+		var f = tempnam(dir, "exec-");
+		setenv("TMPDIR", old_tmpdir, 1);
+		if(!f) {
+			return false;
+		}
+
+		symlink(f, f + ".cy");
+		
+		str = "exports.result = " + str;
+
+		var handle = fopen(f, "w");
+		fwrite(str, str.length, 1, handle);
+		fclose(handle);
+		
+		var r;
+		var except = null;
+		try {
+			r = require(f.replace(libdir + "/", ""));
+		} catch(e) {
+			except = e;
+		}
+
+		unlink(f + ".cy");
+		unlink(f);
+		
+		if(except !== null) {
+			throw except;
+		}
+
+		return r.result;
+	};
+	
+	/*
+		Applies known typedefs
+		Used in utils.include and utils.makeStruct
+		
+		Usage:
+			cy# utils.applyTypedefs("mach_vm_address_t")
+			"uint64_t"
+	*/
+	utils.applyTypedefs = function(str) {
+		var typedefs = {
+			"struct": "",
+			"restrict": "",
+			"FILE": "void",
+			"size_t": "uint64_t",
+			"uintptr_t": "unsigned long",
+			"kern_return_t": "int",
+			"mach_port_t": "unsigned int",
+			"mach_port_name_t": "unsigned int",
+			"vm_offset_t": "unsigned long",
+			"vm_size_t": "unsigned long",
+			"mach_vm_address_t": "uint64_t",
+			"mach_vm_offset_t": "uint64_t",
+			"mach_vm_size_t": "uint64_t",
+			"vm_map_offset_t": "uint64_t",
+			"vm_map_address_t": "uint64_t",
+			"vm_map_size_t": "uint64_t",
+			"mach_port_context_t": "uint64_t",
+			"vm_map_t": "unsigned int",
+			"boolean_t": "unsigned int",
+			"vm_prot_t": "int",
+			"mach_msg_type_number_t": "unsigned int",
+			"cpu_type_t": "int",
+			"cpu_subtype_t": "int",
+			"cpu_threadtype_t": "int",
+		};
+		
+		for(var k in typedefs) {
+			str = str.replace(new RegExp("(\\s|\\*|,|\\(|^)" + k + "(\\s|\\*|,|\\)|$)", "g"), "$1" + typedefs[k] + "$2");
+		}
+		
+		return str;
+	};
+	
+	/*
+		Parses a C function declaration and returns the function name and cycript type
+		If load is true, tries to load it into cycript using utils.exec
+		
+		Usage:
+			cy# var str = "void *calloc(size_t num, size_t size)";
+			"void *calloc(size_t num, size_t size)"
+			cy# utils.include(str)
+			["calloc","@encode(void *(uint64_t num,  uint64_t size))(140735674376857)"]
+			cy# var ret = utils.include(str, true)
+			["calloc",0x7fff93e0e299]
+			cy# ret[1].type
+			@encode(void*(unsigned long long int,unsigned long long int))
+			cy# ret[1](100, 1)
+			0x100444100
+	*/
+	utils.include = function(str, load) {
+		var re = /^\s*([^(]*(?:\s+|\*))(\w*)\s*\(([^)]*)\)\s*;?\s*$/;
+		var match = re.exec(str);
+		if(!match) {
+			return -1;
+		}
+		var rType = utils.applyTypedefs(match[1]);
+		var name = match[2];
+		var args = match[3];
+
+		var argsRe = /([^,]+)(?:,|$)/g;
+		var argsTypes = [];
+		while((match = argsRe.exec(args)) !== null) {
+			var type = utils.applyTypedefs(match[1]);
+			argsTypes.push(type);
+		}
+		
+		var encodeString = "@encode(";
+		encodeString += rType + "(";
+		encodeString += argsTypes.join(", ") + "))";
+
+		var fun = dlsym(RTLD_DEFAULT, name);
+		if(fun !== null) {
+			encodeString += "(" + fun + ")";
+			if(load) {
+				return [name, utils.exec(encodeString)];
+			}
+		} else if(load) {
+			throw "Function couldn't be found with dlsym!";
+		}
+
+		return [name, encodeString];
+	};
+	
+	/*
+		Loads the function declaration in the defs array using utils.exec and exposes to cycript's global scope
+		Is automatically called if shouldLoadCFuncs is true
+	*/
+	utils.funcs = {};
+	utils.loadfuncs = function(expose) {
+		for(var i = 0; i < CFuncsDeclarations.length; i++) {
+			try {
+				var o = utils.include(CFuncsDeclarations[i], true);
+				utils.funcs[o[0]] = o[1];
+				if(expose) {
+					Cycript.all[o[0]] = o[1];
+				}
+			} catch(e) {
+				system.print("Failed to load function: " + i);
+				try {
+					system.print(utils.include(CFuncsDeclarations[i]));
+				} catch(e2) {
+					
+				}
+			}
+		}
+	};
+	
+	/*
+		Calculates the size of a type like the C operator sizeof
+		
+		Usage:
+			cy# utils.sizeof(int)
+			4
+			cy# utils.sizeof(@encode(void *))
+			8
+			cy# utils.sizeof("mach_vm_address_t")
+			8
+	*/
+	utils.sizeof = function(type) {
+		if(typeof type === "string") {
+			type = utils.applyTypedefs(type);
+			type = utils.exec("@encode(" + type + ")");
+		}
+		
+		// (const) char * has "infinite" preceision
+		if(type.toString().slice(-1) === "*") {
+			return utils.sizeof(@encode(void *));
+		}
+		
+		// float and double
+		if(type.toString() === @encode(float).toString()) {
+			return 4;
+		} else if (type.toString() === @encode(double).toString()) {
+			return 8;
+		}
+
+		var typeInstance = type(0);
+		
+		if(typeInstance instanceof Object) {
+			// Arrays
+			if("length" in typeInstance) {
+				return typeInstance.length * utils.sizeof(typeInstance.type);
+			}
+			
+			// Structs
+			if(typeInstance.toString() === "[object Struct]") {
+				var typeStr = type.toString();
+				var arrayTypeStr = "[2" + typeStr + "]";
+				var arrayType = new Type(arrayTypeStr);
+				
+				var arrayInstance = new arrayType;
+				
+				return @encode(void *)(&(arrayInstance[1])) - @encode(void *)(&(arrayInstance[0]));
+			}
+		}
+		
+		for(var i = 0; i < 5; i++) {
+			var maxSigned = Math.pow(2, 8 * Math.pow(2, i) - 1) - 1;
+			if(i === 3) {
+				// Floating point fix ;^)
+				maxSigned /= 1000;
+			}
+
+			// can't use !== or sizeof(void *) === 0.5
+			if(type(maxSigned) != maxSigned) {
+				return Math.pow(2, i - 1);
+			}
+		}
+	};
+	
+	/*
+		Logs a specific message sent to an instance of a class like logify.pl in theos
+		Requires Cydia Substrate (com.saurik.substrate.MS) and NSLog (org.cycript.NSLog) modules
+		Returns the old message returned by MS.hookMessage (Note: this is not just the old message!)
+		
+		Usage:
+			cy# var oldm = utils.logify(objc_getMetaClass(NSNumber), @selector(numberWithDouble:))
+			...
+			cy# var n = [NSNumber numberWithDouble:1.5]
+			2014-07-28 02:26:39.805 cycript[71213:507] +[<NSNumber: 0x10032d0c4> numberWithDouble:1.5]
+			2014-07-28 02:26:39.806 cycript[71213:507]  = 1.5
+			@1.5
+	*/
+	utils.logify = function(cls, sel) {
+		@import com.saurik.substrate.MS;
+		@import org.cycript.NSLog;
+		
+		var oldm = {};
+		
+		MS.hookMessage(cls, sel, function() {
+			var args = [].slice.call(arguments);
+			
+			var selFormat = sel.toString().replace(/:/g, ":%@ ").trim();
+			var logFormat = "%@[<%@: 0x%@> " + selFormat + "]";
+			
+			var standardArgs = [logFormat, class_isMetaClass(cls)? "+": "-", cls.toString(), (&this).valueOf().toString(16)];
+			var logArgs = standardArgs.concat(args);
+			
+			NSLog.apply(null, logArgs);
+			
+			var r = oldm->apply(this, arguments);
+			
+			if(r !== undefined) {
+				NSLog(" = %@", r);
+			}
+			
+			return r;
+		}, oldm);
+		
+		return oldm;
+	};
+	
+	/*
+		Calls a C function by providing its name and arguments
+		Doesn't support structs
+		Return value is always a void pointer
+		
+		Usage:
+			cy# utils.apply("printf", ["%s %.3s, %d -> %c, float: %f\n", "foo", "barrrr", 97, 97, 1.5])
+			foo bar, 97 -> a, float: 1.500000
+			0x22
+	*/
+	utils.apply = function(fun, args) {
+		if(!(args instanceof Array)) {
+			throw "Args needs to be an array!";
+		}
+		
+		var argc = args.length;
+		var voidPtr = @encode(void *);
+		var argTypes = [];
+		for(var i = 0; i < argc; i++) {
+			var argType = voidPtr;
+			
+			var arg = args[i];
+			if(typeof arg === "string") {
+				argType = @encode(char *);
+			}
+			if(typeof arg === "number" && arg % 1 !== 0) {
+				argType = @encode(double);
+			}
+			
+			argTypes.push(argType);
+		}
+		
+		var type = voidPtr.functionWith.apply(voidPtr, argTypes);
+		
+		if(typeof fun === "string") {
+			fun = dlsym(RTLD_DEFAULT, fun);
+		}
+		
+		if(!fun) {
+			throw "Function not found!";
+		}
+
+		return type(fun).apply(null, args);
+	};
+	
+	/*
+		Converts a string (char *) to a void pointer (void *)
+		You can't cast to strings to void pointers and vice versa in cycript. Blame saurik.
+		
+		Usage:
+			cy# var voidPtr = utils.str2voidPtr("foobar")
+			0x100331590
+			cy# utils.voidPtr2str(voidPtr)
+			"foobar"
+	*/
+	utils.str2voidPtr = function(str) {
+		var strdup = @encode(void *(char *))(dlsym(RTLD_DEFAULT, "strdup"));
+		return strdup(str);
+	};
+	
+	/*
+		The inverse function of str2voidPtr
+	*/
+	utils.voidPtr2str = function(voidPtr) {
+		var strdup = @encode(char *(void *))(dlsym(RTLD_DEFAULT, "strdup"));
+		return strdup(voidPtr);
+	};
+	
+	/*
+		Converts a double into a void pointer
+		This can be used to view the binary representation of a floating point number
+		
+		Usage:
+			cy# var n = utils.double2voidPtr(-1.5)
+			0xbff8000000000000
+			cy# utils.voidPtr2double(n)
+			-1.5
+	*/
+	utils.double2voidPtr = function(n) {
+		var doublePtr = new double;
+		*doublePtr = n;
+		
+		var voidPtrPtr = @encode(void **)(doublePtr);
+		
+		return *voidPtrPtr;
+	};
+	
+	/*
+		The inverse function of double2voidPtr
+	*/
+	utils.voidPtr2double = function(voidPtr) {
+		var voidPtrPtr = new @encode(void **);
+		*voidPtrPtr = voidPtr;
+		
+		var doublePtr = @encode(double *)(voidPtrPtr);
+		
+		return *doublePtr;
+	};
+	
+	/*
+		Determines in a safe way if a memory location is readable
+		
+		Usage:
+			cy# utils.isMemoryReadable(0)
+			false
+			cy# utils.isMemoryReadable(0x1337)
+			false
+			cy# utils.isMemoryReadable(NSObject)
+			true
+			cy# var a = malloc(100); utils.isMemoryReadable(a)
+			true
+	*/
+	utils.isMemoryReadable = function(ptr) {
+		if(typeof ptr === "string") {
+			return true;
+		}
+		
+		var fds = new @encode(int [2]);
+		utils.apply("pipe", [fds]);
+		var result = utils.apply("write", [fds[1], ptr, 1]) == 1;
+		
+		utils.apply("close", [fds[0]]);
+		utils.apply("close", [fds[1]]);
+		
+		return result;
+	};
+	
+	/*
+		Determines in a safe way if the memory location contains an Objective-C object
+
+		Usage:
+			cy# utils.isObject(0)
+			false
+			cy# utils.isObject(0x1337)
+			false
+			cy# utils.isObject(NSObject)
+			true
+			cy# utils.isObject(objc_getMetaClass(NSObject))
+			true
+			cy# utils.isObject([new NSObject init])
+			true
+			cy# var a = malloc(100); utils.isObject(a)
+			false
+			cy# *@encode(void **)(a) = NSObject; utils.isObject(a)
+			true
+	*/
+	utils.isObject = function(obj) {
+		obj = @encode(void *)(obj);
+		var lastObj = -1;
+		
+		function objc_isa_ptr(obj) {
+			// See http://www.sealiesoftware.com/blog/archive/2013/09/24/objc_explain_Non-pointer_isa.html
+			var objc_debug_isa_class_mask = 0x00000001fffffffa;
+			obj = (obj & 1)? (obj & objc_debug_isa_class_mask): obj;
+			
+			if((obj & (utils.sizeof(@encode(void *)) - 1)) != 0) {
+				return null;
+			} else {
+				return obj;
+			}
+		}
+		
+		function ptrValue(obj) {
+			return obj? obj.valueOf(): null;
+		}
+		
+		var foundMetaClass = false;
+		
+		for(obj = objc_isa_ptr(obj); utils.isMemoryReadable(obj); ) {
+			obj = *@encode(void **)(obj);
+			
+			if(ptrValue(obj) == ptrValue(lastObj)) {
+				foundMetaClass = true;
+				break;
+			}
+			
+			lastObj = obj;
+		}
+		
+		if(!foundMetaClass) {
+			return false;
+		}
+		
+		if(lastObj === -1 || lastObj === null) {
+			return false;
+		}
+		
+		var obj_class = objc_isa_ptr(@encode(void **)(obj)[1]);
+		
+		if(!utils.isMemoryReadable(obj_class)) {
+			return false;
+		}
+		
+		var metaclass = objc_isa_ptr(@encode(void **)(obj_class)[0]);
+		var superclass = objc_isa_ptr(@encode(void **)(obj_class)[1]);
+		
+		return ptrValue(obj) == ptrValue(metaclass) && superclass == null;
+	};
+	
+	/*
+		Creates a cycript struct type from a C struct definition
+		
+		Usage:
+			cy# var foo = makeStruct("int a; short b; char c; uint64_t d; double e;", "foo");
+			@encode(foo)
+			cy# var f = new foo
+			&{a:0,b:0,c:0,d:0,e:0}
+			cy# f->a = 100; f
+			&{a:100,b:0,c:0,d:0,e:0}
+			cy# *@encode(int *)(f)
+			100
+	*/
+	utils.makeStruct = function(str, name) {		
+		var fieldRe = /(?:\s|\n)*([^;]+\s*(?:\s|\*))([^;]+)\s*;/g;
+		
+		if(!name) {
+			name = "struct" + Math.floor(Math.random() * 100000);
+		}
+		var typeStr = "{" + name + "=";
+		
+		while((match = fieldRe.exec(str)) !== null) {
+			var fieldType = utils.applyTypedefs(match[1]);
+			var fieldName = match[2];
+			var encodedType = utils.exec("@encode(" + fieldType + ")").toString();
+			
+			typeStr += '"' + fieldName + '"' + encodedType;
+		}
+		
+		typeStr += "}";
+		
+		return new Type(typeStr);
+	};
+	
+	// Various constants
+	utils.constants = {
+		VM_PROT_NONE:       0x0,
+		VM_PROT_READ:       0x1,
+		VM_PROT_WRITE:      0x2,
+		VM_PROT_EXECUTE:    0x4,
+		VM_PROT_NO_CHANGE:  0x8,
+		VM_PROT_COPY:       0x10,
+		VM_PROT_WANTS_COPY: 0x10,
+		VM_PROT_IS_MASK:    0x40,
+	};
+	var c = utils.constants;
+	c.VM_PROT_DEFAULT = c.VM_PROT_READ | c.VM_PROT_WRITE;
+	c.VM_PROT_ALL =     c.VM_PROT_READ | c.VM_PROT_WRITE | c.VM_PROT_EXECUTE;
+	
+	if(shouldExposeConsts) {
+		for(var k in c) {
+			Cycript.all[k] = c[k];
+		}
+	}
+	
+	if(shouldExposeFuncs) {
+		for(var i = 0; i < funcsToExpose.length; i++) {
+			var name = funcsToExpose[i];
+			Cycript.all[name] = utils[name];
+		}
+	}
+	
+	if(shouldLoadCFuncs) {
+		utils.loadfuncs(shouldExposeCFuncs);
+	}
+})(exports);

samples/EmberScript/momentComponent.em

@@ -0,0 +1,23 @@
+class App.FromNowView extends Ember.View
+    tagName: 'time'
+    template: Ember.Handlebars.compile '{{view.output}}'
+    output: ~>
+        return moment(@value).fromNow()
+
+    didInsertElement: ->
+        @tick()
+
+    tick: ->
+        f = ->
+            @notifyPropertyChange 'output'
+            @tick()
+
+        nextTick = Ember.run.later(this, f, 1000)
+        @set 'nextTick', nextTick
+
+    willDestroyElement: ->
+        nextTick = @nextTick
+        Ember.run.cancel nextTick
+
+Ember.Handlebars.helper 'fromNow', App.FromNowView
+

samples/F#/Combinators.fs

@@ -0,0 +1,49 @@
+namespace Nessos.FsPickler.Combinators
+
+    open Nessos.FsPickler
+    open Nessos.FsPickler.Json
+
+    /// Json pickling methods
+    [<RequireQualifiedAccess>]
+    module Json =
+
+        let private jsonSerializer = lazy(FsPickler.CreateJson(omitHeader = true))
+
+        /// <summary>
+        ///     Pickles a value to Json.
+        /// </summary>
+        /// <param name="pickler">utilized pickler.</param>
+        /// <param name="value">input value.</param>
+        let pickle (pickler : Pickler<'T>) (value : 'T) : string =
+            jsonSerializer.Value.PickleToString (pickler, value)
+
+        /// <summary>
+        ///     Unpickles a value from Json.
+        /// </summary>
+        /// <param name="pickler">utilized pickler.</param>
+        /// <param name="pickle">input pickle.</param>
+        let unpickle (pickler : Pickler<'T>) (pickle : string) : 'T =
+            jsonSerializer.Value.UnPickleOfString (pickler, pickle)
+
+
+    /// Bson pickling methods
+    [<RequireQualifiedAccess>]
+    module Bson =
+
+        let private bsonPickler = lazy(FsPickler.CreateBson())
+
+        /// <summary>
+        ///     Pickles a value to Bson.
+        /// </summary>
+        /// <param name="pickler">utilized pickler.</param>
+        /// <param name="value">input value.</param>
+        let pickle (pickler : Pickler<'T>) (value : 'T) : byte [] =
+            bsonPickler.Value.Pickle (pickler, value)
+
+        /// <summary>
+        ///     Unpickles a value from bson.
+        /// </summary>
+        /// <param name="pickler">utilized pickler.</param>
+        /// <param name="pickle">input pickle.</param>
+        let unpickle (pickler : Pickler<'T>) (pickle : byte []) : 'T =
+            bsonPickler.Value.UnPickle (pickler, pickle)

samples/F#/JsonFormat.fs

@@ -0,0 +1,65 @@
+namespace Nessos.FsPickler.Json
+
+    open System
+    open System.IO
+    open System.Text
+
+    open Newtonsoft.Json
+
+    open Nessos.FsPickler
+
+    /// <summary>
+    ///     Factory methods for the Json serialization format.
+    /// </summary>
+    type JsonPickleFormatProvider internal (indent, omitHeader) as self =
+
+        let isCustomSeq isTopLevelSequence = 
+            isTopLevelSequence && self.OmitHeader && self.UseCustomTopLevelSequenceSeparator
+
+        let mutable sequenceSeparator = " "
+
+        member val Indent = indent with get,set
+        member val OmitHeader = omitHeader with get,set
+        member val UseCustomTopLevelSequenceSeparator = false with get,set
+
+        member __.SequenceSeparator
+            with get () = sequenceSeparator
+            and set sep =
+                if sep <> null && String.IsNullOrWhiteSpace sep then
+                    sequenceSeparator <- sep
+                else
+                    invalidArg "SequenceSeparator" "should be non-null whitespace."
+
+        interface ITextPickleFormatProvider with
+            member __.Name = "Json"
+
+            // see discussion : https://github.com/nessos/FsPickler/issues/17
+            member __.DefaultEncoding = new UTF8Encoding(false) :> Encoding
+
+            member __.CreateWriter (stream, encoding, isTopLevelSequence, leaveOpen) =
+#if NET40
+                if leaveOpen then raise <| new NotSupportedException("'leaveOpen' not supported in .NET 40.")
+                let sw = new StreamWriter(stream, encoding)
+#else
+                let sw = new StreamWriter(stream, encoding, 1024, leaveOpen)
+#endif
+                let jw = new JsonTextWriter(sw)
+                new JsonPickleWriter(jw, __.OmitHeader, __.Indent, isCustomSeq isTopLevelSequence, sequenceSeparator, leaveOpen) :> _
+
+            member __.CreateReader (stream, encoding, isTopLevelSequence, leaveOpen) =
+#if NET40
+                if leaveOpen then raise <| new NotSupportedException("'leaveOpen' not supported in .NET 40.")
+                let sr = new StreamReader(stream, encoding)
+#else
+                let sr = new StreamReader(stream, encoding, true, 1024, leaveOpen)
+#endif
+                let jr = new JsonTextReader(sr)
+                new JsonPickleReader(jr, __.OmitHeader, isCustomSeq isTopLevelSequence, leaveOpen) :> _
+
+            member __.CreateWriter (textWriter, isTopLevelSequence, leaveOpen) =
+                let jw = new JsonTextWriter(textWriter)
+                new JsonPickleWriter(jw, __.OmitHeader, __.Indent, isCustomSeq isTopLevelSequence, sequenceSeparator, leaveOpen) :> _
+
+            member __.CreateReader (textReader, isTopLevelSequence, leaveOpen) =
+                let jr = new JsonTextReader(textReader)
+                new JsonPickleReader(jr, __.OmitHeader, isCustomSeq isTopLevelSequence, leaveOpen) :> _

samples/F#/JsonReader.fs

@@ -0,0 +1,202 @@
+namespace Nessos.FsPickler.Json
+
+    open System
+    open System.Collections.Generic
+    open System.Globalization
+    open System.IO
+    open System.Numerics
+    open System.Text
+
+    open Newtonsoft.Json
+
+    open Nessos.FsPickler
+
+    /// <summary>
+    ///     Json format deserializer
+    /// </summary>
+    type internal JsonPickleReader (jsonReader : JsonReader, omitHeader, isTopLevelSequence, leaveOpen) =
+
+        do
+            jsonReader.CloseInput <- not leaveOpen
+            jsonReader.SupportMultipleContent <- isTopLevelSequence
+
+        let isBsonReader = match jsonReader with :? Bson.BsonReader -> true | _ -> false
+
+        let mutable depth = 0
+        let arrayStack = new Stack<int> ()
+        do arrayStack.Push Int32.MinValue
+
+        // do not write tag if omitting header or array element
+        let omitTag () = (omitHeader && depth = 0) || arrayStack.Peek() = depth - 1
+
+        interface IPickleFormatReader with
+            
+            member __.BeginReadRoot (tag : string) =
+                do jsonReader.Read() |> ignore
+                    
+                if omitHeader then () else
+
+                if jsonReader.TokenType <> JsonToken.StartObject then raise <| new FormatException("invalid json root object.")
+                else
+                    do jsonReader.MoveNext()
+                    let version = jsonReader.ReadPrimitiveAs<string> false "FsPickler"
+                    if version <> jsonFormatVersion then
+                        let v = Version(version)
+                        raise <| new FormatException(sprintf "Invalid FsPickler format version %O." version)
+
+                    let sTag = jsonReader.ReadPrimitiveAs<string> false "type"
+                    if tag <> sTag then
+                        raise <| new InvalidPickleTypeException(tag, sTag)
+
+            member __.EndReadRoot () = 
+                if not omitHeader then jsonReader.Read() |> ignore
+
+            member __.BeginReadObject (tag : string) =
+                
+                if not <| omitTag () then
+                    jsonReader.ReadProperty tag
+                    jsonReader.MoveNext ()
+
+                if isTopLevelSequence && depth = 0 then
+                    arrayStack.Push depth
+                    depth <- depth + 1
+                    ObjectFlags.IsSequenceHeader
+
+                else
+                    match jsonReader.TokenType with
+                    | JsonToken.Null -> ObjectFlags.IsNull
+                    | JsonToken.StartArray ->
+                        jsonReader.MoveNext()
+                        arrayStack.Push depth
+                        depth <- depth + 1
+                        ObjectFlags.IsSequenceHeader
+
+                    | JsonToken.StartObject ->
+                        do jsonReader.MoveNext()
+                        depth <- depth + 1
+
+                        if jsonReader.ValueAs<string> () = "_flags" then
+                            jsonReader.MoveNext()
+                            let csvFlags = jsonReader.ValueAs<string>()
+                            jsonReader.MoveNext()
+                            parseFlagCsv csvFlags
+                        else
+                            ObjectFlags.None
+
+                    | token -> raise <| new FormatException(sprintf "expected start of Json object but was '%O'." token)
+
+
+            member __.EndReadObject () =
+                if isTopLevelSequence && depth = 1 then
+                    arrayStack.Pop () |> ignore
+                    depth <- depth - 1
+                    jsonReader.Read() |> ignore
+                else
+                    match jsonReader.TokenType with
+                    | JsonToken.Null -> ()
+                    | JsonToken.EndObject -> depth <- depth - 1
+                    | JsonToken.EndArray ->
+                        arrayStack.Pop() |> ignore
+                        depth <- depth - 1
+
+                    | token -> raise <| new FormatException(sprintf "expected end of Json object but was '%O'." token)
+
+                    if omitHeader && depth = 0 then ()
+                    else jsonReader.Read() |> ignore
+
+            member __.SerializeUnionCaseNames = true
+
+            member __.PreferLengthPrefixInSequences = false
+            member __.ReadNextSequenceElement () = 
+                if isTopLevelSequence && depth = 1 then
+                    jsonReader.TokenType <> JsonToken.None
+                else
+                    jsonReader.TokenType <> JsonToken.EndArray
+
+            member __.ReadCachedObjectId () = jsonReader.ReadPrimitiveAs<int64> false "id"
+
+            member __.ReadBoolean tag = jsonReader.ReadPrimitiveAs<bool> (omitTag ()) tag
+            member __.ReadByte tag = jsonReader.ReadPrimitiveAs<int64> (omitTag ()) tag |> byte
+            member __.ReadSByte tag = jsonReader.ReadPrimitiveAs<int64> (omitTag ()) tag |> sbyte
+
+            member __.ReadInt16 tag = jsonReader.ReadPrimitiveAs<int64> (omitTag ()) tag |> int16
+            member __.ReadInt32 tag = jsonReader.ReadPrimitiveAs<int64> (omitTag ()) tag |> int
+            member __.ReadInt64 tag = jsonReader.ReadPrimitiveAs<int64> (omitTag ()) tag
+
+            member __.ReadUInt16 tag = jsonReader.ReadPrimitiveAs<int64> (omitTag ()) tag |> uint16
+            member __.ReadUInt32 tag = jsonReader.ReadPrimitiveAs<int64> (omitTag ()) tag |> uint32
+            member __.ReadUInt64 tag = jsonReader.ReadPrimitiveAs<int64> (omitTag ()) tag |> uint64
+
+            member __.ReadSingle tag =
+                if not <| omitTag () then
+                    jsonReader.ReadProperty tag
+                    jsonReader.MoveNext()
+
+                let value =
+                    match jsonReader.TokenType with
+                    | JsonToken.Float -> jsonReader.ValueAs<double> () |> single
+                    | JsonToken.String -> Single.Parse(jsonReader.ValueAs<string>(), CultureInfo.InvariantCulture)
+                    | _ -> raise <| new FormatException("not a float.")
+
+                jsonReader.Read() |> ignore
+                value
+                
+            member __.ReadDouble tag =
+                if not <| omitTag () then
+                    jsonReader.ReadProperty tag
+                    jsonReader.MoveNext()
+
+                let value =
+                    match jsonReader.TokenType with
+                    | JsonToken.Float -> jsonReader.ValueAs<double> ()
+                    | JsonToken.String -> Double.Parse(jsonReader.ValueAs<string>(), CultureInfo.InvariantCulture)
+                    | _ -> raise <| new FormatException("not a float.")
+
+                jsonReader.Read() |> ignore
+                value
+
+            member __.ReadChar tag = let value = jsonReader.ReadPrimitiveAs<string> (omitTag ()) tag in value.[0]
+            member __.ReadString tag = jsonReader.ReadPrimitiveAs<string> (omitTag ()) tag
+            member __.ReadBigInteger tag = jsonReader.ReadPrimitiveAs<string> (omitTag ()) tag |> BigInteger.Parse
+
+            member __.ReadGuid tag = 
+                if isBsonReader then 
+                    jsonReader.ReadPrimitiveAs<Guid> (omitTag ()) tag
+                else
+                    jsonReader.ReadPrimitiveAs<string> (omitTag ()) tag |> Guid.Parse
+
+            member __.ReadTimeSpan tag = jsonReader.ReadPrimitiveAs<string> (omitTag ()) tag |> TimeSpan.Parse
+            member __.ReadDecimal tag = jsonReader.ReadPrimitiveAs<string> (omitTag ()) tag |> decimal
+
+            // BSON spec mandates the use of Unix time; 
+            // this has millisecond precision which results in loss of accuracy w.r.t. ticks
+            // since the goal of FsPickler is to offer faithful representations of .NET objects
+            // we choose to override the spec and serialize ticks outright.
+            // see also https://json.codeplex.com/discussions/212067 
+            member __.ReadDate tag = 
+                if isBsonReader then
+                    let ticks = jsonReader.ReadPrimitiveAs<int64> (omitTag ()) tag
+                    DateTime(ticks)
+                else
+                    jsonReader.ReadPrimitiveAs<DateTime> (omitTag ()) tag
+
+            member __.ReadBytes tag =
+                if not <| omitTag () then
+                    jsonReader.ReadProperty tag
+                    jsonReader.Read() |> ignore
+
+                let bytes =
+                    if jsonReader.TokenType = JsonToken.Null then null
+                    elif isBsonReader then jsonReader.ValueAs<byte []> ()
+                    else
+                        let base64 = jsonReader.ValueAs<string> ()
+                        Convert.FromBase64String base64
+
+                jsonReader.Read() |> ignore
+
+                bytes
+
+            member __.IsPrimitiveArraySerializationSupported = false
+            member __.ReadPrimitiveArray _ _ = raise <| new NotImplementedException()
+
+            member __.Dispose () = (jsonReader :> IDisposable).Dispose()

samples/F#/JsonSerializer.fs

@@ -0,0 +1,85 @@
+namespace Nessos.FsPickler.Json
+
+    open System
+
+    open Nessos.FsPickler
+
+    type internal OAttribute = System.Runtime.InteropServices.OptionalAttribute
+    type internal DAttribute = System.Runtime.InteropServices.DefaultParameterValueAttribute
+
+    /// <summary>
+    ///     Json pickler instance.
+    /// </summary>
+    type JsonSerializer =
+        inherit FsPicklerTextSerializer
+        
+        val private format : JsonPickleFormatProvider
+
+        /// <summary>
+        ///     Initializes a new Json pickler instance.
+        /// </summary>
+        /// <param name="indent">indent out Json pickles.</param>
+        /// <param name="omitHeader">omit FsPickler header in Json pickles.</param>
+        /// <param name="typeConverter">specify a custom type name converter.</param>
+        new ([<O;D(null)>] ?indent, [<O;D(null)>] ?omitHeader, [<O;D(null)>] ?typeConverter) =
+            let indent = defaultArg indent false
+            let omitHeader = defaultArg omitHeader false
+            let json = new JsonPickleFormatProvider(indent, omitHeader)
+            { 
+                inherit FsPicklerTextSerializer(json, ?typeConverter = typeConverter)
+                format = json    
+            }
+
+        /// <summary>
+        ///     Gets or sets whether Json output should be indented.
+        /// </summary>
+        member x.Indent
+            with get () = x.format.Indent
+            and set b = x.format.Indent <- b
+
+        /// <summary>
+        ///     Gets or sets whether FsPickler headers should be ignored in pickle format.
+        /// </summary>
+        member x.OmitHeader
+            with get () = x.format.OmitHeader
+            and set b = x.format.OmitHeader <- b
+
+        /// <summary>
+        ///     Gets or sets a non-null whitespace string that serves as a custom, top-level sequence separator.
+        /// </summary>
+        member x.SequenceSeparator
+            with get () = x.format.SequenceSeparator
+            and set sep = x.format.SequenceSeparator <- sep
+
+        /// <summary>
+        ///     Gets or sets whether top-level sequences should be serialized using the custom separator.
+        /// </summary>
+        member x.UseCustomTopLevelSequenceSeparator
+            with get () = x.format.UseCustomTopLevelSequenceSeparator
+            and set e = x.format.UseCustomTopLevelSequenceSeparator <- e
+
+    /// <summary>
+    ///     BSON pickler instance.
+    /// </summary>
+    type BsonSerializer([<O;D(null)>] ?typeConverter) =
+        inherit FsPicklerSerializer(new BsonPickleFormatProvider(), ?typeConverter = typeConverter)
+
+
+    /// FsPickler static methods.
+    type FsPickler =
+
+        /// <summary>
+        ///     Initializes a new Json pickler instance.
+        /// </summary>
+        /// <param name="indent">indent out Json pickles.</param>
+        /// <param name="omitHeader">omit FsPickler header in Json pickles.</param>
+        /// <param name="typeConverter">specify a custom type name converter.</param>
+        static member CreateJson([<O;D(null)>] ?indent, [<O;D(null)>] ?omitHeader, [<O;D(null)>] ?typeConverter) = 
+            new JsonSerializer(?indent = indent, ?omitHeader = omitHeader, ?typeConverter = typeConverter)
+
+        /// <summary>
+        ///     Initializes a new Bson pickler instance.
+        /// </summary>
+        /// <param name="typeConverter">specify a custom type name converter.</param>
+        static member CreateBson([<O;D(null)>] ?typeConverter) = 
+            new BsonSerializer(?typeConverter = typeConverter)

samples/F#/JsonWriter.fs

@@ -0,0 +1,142 @@
+namespace Nessos.FsPickler.Json
+
+    open System
+    open System.IO
+    open System.Collections.Generic
+
+    open Newtonsoft.Json
+
+    open Nessos.FsPickler
+
+    /// <summary>
+    ///     Json format serializer.
+    /// </summary>
+    type internal JsonPickleWriter (jsonWriter : JsonWriter, omitHeader, indented, isTopLevelSequence, separator, leaveOpen) =
+
+        do 
+            jsonWriter.Formatting <- if indented then Formatting.Indented else Formatting.None
+            jsonWriter.CloseOutput <- not leaveOpen
+
+        let isBsonWriter = match jsonWriter with :? Bson.BsonWriter -> true | _ -> false
+
+        let mutable depth = 0
+        let mutable isTopLevelSequenceHead = false
+        let mutable currentValueIsNull = false
+
+        let arrayStack = new Stack<int> ()
+        do arrayStack.Push Int32.MinValue
+
+        // do not write tag if omitting header or array element
+        let omitTag () = (omitHeader && depth = 0) || arrayStack.Peek() = depth - 1
+
+        interface IPickleFormatWriter with
+            
+            member __.BeginWriteRoot (tag : string) =
+                if omitHeader then () else
+
+                jsonWriter.WriteStartObject()
+                writePrimitive jsonWriter false "FsPickler" jsonFormatVersion
+                writePrimitive jsonWriter false "type" tag
+
+            member __.EndWriteRoot () = 
+                if not omitHeader then jsonWriter.WriteEnd()
+
+            member __.BeginWriteObject (tag : string) (flags : ObjectFlags) =
+
+                if not <| omitTag () then
+                    jsonWriter.WritePropertyName tag
+
+                if flags.HasFlag ObjectFlags.IsNull then
+                    currentValueIsNull <- true
+                    jsonWriter.WriteNull()
+
+                elif flags.HasFlag ObjectFlags.IsSequenceHeader then
+                    if isTopLevelSequence && depth = 0 then
+                        isTopLevelSequenceHead <- true
+                    else
+                        jsonWriter.WriteStartArray()
+
+                    arrayStack.Push depth
+                    depth <- depth + 1
+                else
+                    jsonWriter.WriteStartObject()
+                    depth <- depth + 1
+
+                    if flags = ObjectFlags.None then ()
+                    else
+                        let flagCsv = mkFlagCsv flags
+                        writePrimitive jsonWriter false "_flags" flagCsv
+
+            member __.EndWriteObject () = 
+                if currentValueIsNull then 
+                    currentValueIsNull <- false
+                else
+                    depth <- depth - 1
+                    if arrayStack.Peek () = depth then
+                        if isTopLevelSequence && depth = 0 then ()
+                        else
+                            jsonWriter.WriteEndArray()
+
+                        arrayStack.Pop () |> ignore
+                    else
+                        jsonWriter.WriteEndObject()
+
+            member __.SerializeUnionCaseNames = true
+
+            member __.PreferLengthPrefixInSequences = false
+            member __.WriteNextSequenceElement hasNext =
+                if isTopLevelSequence && depth = 1 then
+                    if isTopLevelSequenceHead then
+                        isTopLevelSequenceHead <- false
+                    else
+                        jsonWriter.WriteWhitespace separator
+
+            member __.WriteCachedObjectId id = writePrimitive jsonWriter false "id" id
+
+            member __.WriteBoolean (tag : string) value = writePrimitive jsonWriter (omitTag ()) tag value
+            member __.WriteByte (tag : string) value = writePrimitive jsonWriter (omitTag ()) tag value
+            member __.WriteSByte (tag : string) value = writePrimitive jsonWriter (omitTag ()) tag value
+
+            member __.WriteInt16 (tag : string) value = writePrimitive jsonWriter (omitTag ()) tag value
+            member __.WriteInt32 (tag : string) value = writePrimitive jsonWriter (omitTag ()) tag value
+            member __.WriteInt64 (tag : string) value = writePrimitive jsonWriter (omitTag ()) tag value
+
+            member __.WriteUInt16 (tag : string) value = writePrimitive jsonWriter (omitTag ()) tag value
+            member __.WriteUInt32 (tag : string) value = writePrimitive jsonWriter (omitTag ()) tag value
+            member __.WriteUInt64 (tag : string) value = writePrimitive jsonWriter (omitTag ()) tag value
+
+            member __.WriteSingle (tag : string) value = writePrimitive jsonWriter (omitTag ()) tag value
+            member __.WriteDouble (tag : string) value = writePrimitive jsonWriter (omitTag ()) tag value
+            member __.WriteDecimal (tag : string) value = writePrimitive jsonWriter (omitTag ()) tag (string value)
+
+            member __.WriteChar (tag : string) value = writePrimitive jsonWriter (omitTag ()) tag value
+            member __.WriteString (tag : string) value = writePrimitive jsonWriter (omitTag ()) tag value
+            member __.WriteBigInteger (tag : string) value = writePrimitive jsonWriter (omitTag ()) tag (string value)
+
+            member __.WriteGuid (tag : string) value = writePrimitive jsonWriter (omitTag ()) tag value
+            member __.WriteTimeSpan (tag : string) value = writePrimitive jsonWriter (omitTag ()) tag (string value)
+
+            // BSON spec mandates the use of Unix time; 
+            // this has millisecond precision which results in loss of accuracy w.r.t. ticks
+            // since the goal of FsPickler is to offer faithful representations of .NET objects
+            // we choose to override the spec and serialize ticks outright.
+            // see also https://json.codeplex.com/discussions/212067 
+            member __.WriteDate (tag : string) value = 
+                if isBsonWriter then
+                    writePrimitive jsonWriter (omitTag ()) tag value.Ticks
+                else
+                    writePrimitive jsonWriter (omitTag ()) tag value
+
+            member __.WriteBytes (tag : string) (value : byte []) =
+                if not <| omitTag () then 
+                    jsonWriter.WritePropertyName tag
+
+                if obj.ReferenceEquals(value, null) then
+                    jsonWriter.WriteNull()
+                else
+                    jsonWriter.WriteValue value
+
+            member __.IsPrimitiveArraySerializationSupported = false
+            member __.WritePrimitiveArray _ _ = raise <| NotSupportedException()
+
+            member __.Dispose () = jsonWriter.Flush()

samples/F#/PerformanceTesters.fs

@@ -0,0 +1,68 @@
+namespace Nessos.FsPickler.Tests
+
+    open PerfUtil
+    open PerfUtil.NUnit
+
+    open NUnit.Framework
+
+    open Nessos.FsPickler
+    open Nessos.FsPickler.Json
+
+    [<AbstractClass>]
+    type PerfTester () =
+        inherit NUnitPerf<Serializer> ()
+
+        let tests = PerfTest.OfModuleMarker<PerformanceTests.Marker> ()
+
+        override __.PerfTests = tests
+
+
+    type ``Serializer Comparison`` () =
+        inherit PerfTester()
+
+        let fsp = FsPickler.initBinary()
+        let bfs = new BinaryFormatterSerializer() :> Serializer
+        let ndc = new NetDataContractSerializer() :> Serializer
+        let jdn = new JsonDotNetSerializer() :> Serializer
+        let bdn = new JsonDotNetBsonSerializer () :> Serializer
+        let pbn = new ProtoBufSerializer() :> Serializer
+        let ssj = new ServiceStackJsonSerializer() :> Serializer
+        let sst = new ServiceStackTypeSerializer() :> Serializer
+
+        let comparer = new WeightedComparer(spaceFactor = 0.2, leastAcceptableImprovementFactor = 1.)
+        let tester = new ImplementationComparer<_>(fsp, [bfs;ndc;jdn;bdn;pbn;ssj;sst], throwOnError = true, warmup = true, comparer = comparer)
+
+        override __.PerfTester = tester :> _
+        
+
+    type ``FsPickler Formats Comparison`` () =
+        inherit PerfTester ()
+
+        let binary = FsPickler.initBinary()
+        let json = FsPickler.initJson()
+        let bson = FsPickler.initBson()
+        let xml = FsPickler.initXml()
+
+        let tester = new ImplementationComparer<_>(binary, [json ; bson; xml], warmup = true, throwOnError = false)
+
+        override __.PerfTester = tester :> _
+
+
+    type ``Past FsPickler Versions Comparison`` () =
+        inherit PerfTester ()
+
+        let persistResults = true
+        let persistenceFile = "fspPerf.xml"
+
+        let fsp = FsPickler.initBinary()
+        let version = typeof<FsPickler>.Assembly.GetName().Version
+        let comparer = new WeightedComparer(spaceFactor = 0.2, leastAcceptableImprovementFactor = 0.8)
+        let tester = 
+            new PastImplementationComparer<Serializer>(
+                fsp, version, historyFile = persistenceFile, throwOnError = true, warmup = true, comparer = comparer)
+
+        override __.PerfTester = tester :> _
+
+        [<TestFixtureTearDown>]
+        member __.Persist() =
+            if persistResults then tester.PersistCurrentResults ()

samples/F#/PerformanceTests.fs

@@ -0,0 +1,207 @@
+namespace Nessos.FsPickler.Tests
+
+    open System
+    open System.Collections.Generic
+
+    open PerfUtil
+
+    open Nessos.FsPickler
+    open Nessos.FsPickler.Tests.Serializer
+    open Nessos.FsPickler.Tests.TestTypes
+
+    module PerformanceTests =
+
+        type Marker = class end
+
+        let guid = Guid.NewGuid()
+
+        [<PerfTest(1000)>]
+        let ``Value: Guid`` s = roundtrip guid s
+
+        let date = DateTime.Now
+
+        [<PerfTest(1000)>]
+        let ``Value: DateTime`` s = roundtrip date s
+
+        [<PerfTest(10000)>]
+        let ``Value: String`` s = roundtrip stringValue s
+
+
+        let boxed = box ([| 1 .. 1000 |], "lorem ipsum")
+
+        [<PerfTest(1000)>]
+        let ``Boxed Object`` s = roundtrip boxed s
+
+        let fsClass = new Class(42, stringValue)
+
+        [<PerfTest(10000)>]
+        let ``Class: Simple F# Class`` s = roundtrip fsClass s
+
+        let serializableClass = new SerializableClass<_>(42, stringValue, [|1..1000|])
+
+        [<PerfTest(10000)>]
+        let ``Class: ISerializable`` s = roundtrip serializableClass s
+
+        let boxedClass = box(Some 42)
+
+        [<PerfTest(10000)>]
+        let ``Subtype Resolution`` s = roundtrip boxedClass s
+
+        let floatArray = Array.init 100000 (fun i -> float i)
+
+        [<PerfTest(10)>]
+        let ``Array: Float`` s = roundtrip floatArray s
+
+        let intArray = Array.init 100000 id
+
+        [<PerfTest(10)>]
+        let ``Array: Int`` s = roundtrip intArray s
+
+        let stringArray = Array.init 10000 (fun i -> stringValue + string i)
+        
+        [<PerfTest(100)>]
+        let ``Array: String`` s = roundtrip stringArray s
+
+        let kvarr = [|1..10000|] |> Array.map (fun i -> i, string i)
+
+        [<PerfTest(100)>]
+        let ``Array: Key-Value Pairs`` s = roundtrip kvarr s
+
+        let duArray = [| for i in 1 .. 10000 -> (Something ("asdasdasdas", i)) |]
+
+        [<PerfTest(100)>]
+        let ``Array: Discriminated Unions`` s = roundtrip duArray s
+
+        let objArray = 
+            [| 
+                box 2; box 3; box "hello" ; box <| Some 3; box(2,3) ; 
+                box <| new Class(2, stringValue) ; box <| new SerializableClass<int option>(2, stringValue, Some 12); 
+                box stringValue 
+            |]
+
+        [<PerfTest(1000)>]
+        let ``Array: Objects`` s = roundtrip objArray s
+
+
+        let array3D = Array3D.init 100 100 100 (fun i j k -> float (i * j + k))
+
+        [<PerfTest(10)>]
+        let ``Array: Rank-3 Float`` s = roundtrip array3D s
+
+        let bclDict = dict [ for i in 1 .. 1000 -> (string i, i)]
+
+        [<PerfTest(100)>]
+        let ``.NET Dictionary`` s = roundtrip bclDict s
+
+        let bclStack = new Stack<string>([for i in 1 .. 1000 -> string i])
+
+        [<PerfTest(100)>]
+        let ``.NET Stack`` s = roundtrip bclStack s
+
+        let bclList = new List<string * int>([for i in 1 .. 1000 -> string i, i])
+
+        [<PerfTest(100)>]
+        let ``.NET List`` s = roundtrip bclList s
+
+        let bclSet = new SortedSet<_>([for i in 1 .. 1000 -> string i])
+
+        [<PerfTest(100)>]
+        let ``.NET Set`` s = roundtrip bclSet s
+
+        let smallTuple = (1, DateTime.Now,"hello")
+
+        [<PerfTest(10000)>]
+        let ``FSharp: Tuple Small`` s = roundtrip smallTuple s
+
+        let largeTuple = (stringValue, 1, 2, 3, true, "", Some(3.14, [2]), 3, 2, 1, stringValue)
+
+        [<PerfTest(10000)>]
+        let ``FSharp: Tuple Large`` s =
+            roundtrip largeTuple s
+
+        let intList = [1..1000]
+
+        [<PerfTest(1000)>]
+        let ``FSharp: List Int`` s = roundtrip intList s
+
+        let stringList = [ for i in 1 .. 1000 -> stringValue + string i ]
+
+        [<PerfTest(1000)>]
+        let ``FSharp: List String`` s = roundtrip stringList s
+
+        let pairList = [ for i in 1 .. 1000 -> (string i, i) ]
+
+        [<PerfTest(1000)>]
+        let ``FSharp: List Key-Value`` s = roundtrip pairList s
+
+        let nestedLst = let n = [1..1000] in [for _ in 1 .. 100 -> n]
+
+        [<PerfTest(1000)>]
+        let ``FSharp: List Nested`` s = roundtrip nestedLst s
+
+        let union = SomethingElse(stringValue, 42, box (Some 42))
+
+        [<PerfTest(10000)>]
+        let ``FSharp: Union`` s = roundtrip union s
+
+        let record = { Int = 42 ; String = stringValue ; Tuple = (13, "") }
+
+        [<PerfTest(10000)>]
+        let ``FSharp: Record`` s = roundtrip record s
+
+        let peano = int2Peano 100
+
+        [<PerfTest(100)>]
+        let ``FSharp: Peano Rectype`` s = roundtrip peano s
+
+        let closure = (@) [ Some([1..100], Set.ofList [1..100]) ]
+
+        [<PerfTest(1000)>]
+        let ``FSharp: Curried Function`` s = roundtrip closure s
+
+        let binTree = mkTree 10
+
+        [<PerfTest(100)>]
+        let ``FSharp: Binary Tree`` s = roundtrip binTree s
+
+        let intSet = [1..1000] |> List.map string |> set
+
+        [<PerfTest(1000)>]
+        let ``FSharp: Set`` s = roundtrip intSet s
+
+        let fsMap = [1..1000] |> Seq.map (fun i -> (string i,i)) |> Map.ofSeq
+
+        [<PerfTest(1000)>]
+        let ``FSharp: Map`` s = roundtrip fsMap s
+
+        let testType = typeof<int * string option * Map<int * string [], string ref option>>
+
+        [<PerfTest(1000)>]
+        let ``Reflection: Type`` s = roundtrip testType s
+
+        let quotationSmall = <@ fun x -> pown 2 x @>
+
+        let quotationLarge =
+            <@
+                async {
+                    let rec fibAsync n =
+                        async {
+                            match n with
+                            | _ when n < 0 -> return invalidArg "negative" "n"
+                            | _ when n < 2 -> return n
+                            | n ->
+                                let! fn = fibAsync (n-1)
+                                let! fnn = fibAsync (n-2)
+                                return fn + fnn
+                        }
+
+                    let! values = [1..100] |> Seq.map fibAsync |> Async.Parallel
+                    return Seq.sum values
+                }
+            @>
+
+        [<PerfTest(10000)>]
+        let ``FSharp: Quotation Small`` s = roundtrip quotationSmall s
+
+        [<PerfTest(1000)>]
+        let ``FSharp: Quotation Large`` s = roundtrip quotationLarge s

samples/F#/sample.fs

@@ -0,0 +1,15 @@
+module Sample
+
+open System
+
+type Foo =
+    {
+        Bar : string
+    }
+
+type Baz = interface end
+
+let Sample1(xs : int list) : string =
+    xs
+    |> List.map (fun x -> string x)
+    |> String.concat ","

samples/FORTRAN/sample1.f

@@ -0,0 +1,25 @@
+c comment
+* comment
+
+      program main
+
+      end
+
+      subroutine foo( i, x, b )
+      INTEGER            i
+      REAL               x
+      LOGICAL            b
+
+      if( i.ne.0 ) then
+         call bar( -i )
+      end if
+
+      return
+      end
+
+      double complex function baz()
+
+      baz = (0.0d0,0.0d0)
+
+      return 
+      end

samples/FORTRAN/sample1.for

@@ -0,0 +1,25 @@
+c comment
+* comment
+
+      program main
+
+      end
+
+      subroutine foo( i, x, b )
+      INTEGER            i
+      REAL               x
+      LOGICAL            b
+
+      if( i.ne.0 ) then
+         call bar( -i )
+      end if
+
+      return
+      end
+
+      double complex function baz()
+
+      baz = (0.0d0,0.0d0)
+
+      return 
+      end

samples/FORTRAN/sample2.f

@@ -0,0 +1,25 @@
+      PROGRAM MAIN
+
+      END
+
+C comment
+* comment
+
+      SUBROUTINE foo( i, x, b )
+      INTEGER            i
+      REAL               x
+      LOGICAL            b
+
+      IF( i.NE.0 ) THEN
+         CALL bar( -i )
+      END IF
+
+      RETURN
+      END
+
+      DOUBLE COMPLEX FUNCTION baz()
+
+      baz = (0.0d0,0.0d0)
+
+      RETURN 
+      END

samples/FORTRAN/sample3.F

@@ -0,0 +1,25 @@
+c comment
+* comment
+
+      program main
+
+      end
+
+      subroutine foo( i, x, b )
+      INTEGER            i
+      REAL               x
+      LOGICAL            b
+
+      if( i.ne.0 ) then
+         call bar( -i )
+      end if
+
+      return
+      end
+
+      double complex function baz()
+
+      baz = (0.0d0,0.0d0)
+
+      return 
+      end

samples/Forth/bitmap.frt

@@ -0,0 +1,8 @@
+\ Bit arrays.
+: bits ( u1 -- u2 ) 7 + 3 rshift ;
+: bitmap ( u "name" -- ) create bits here over erase allot
+   does> ( u -- a x ) over 3 rshift +  1 rot 7 and lshift ;
+: bit@ ( a x -- f ) swap c@ and ;
+: 1bit ( a x -- ) over c@ or swap c! ;
+: 0bit ( a x -- ) invert over c@ and swap c! ;
+: bit! ( f a x -- ) rot if 1bit else 0bit then ;

samples/Forth/core.f

@@ -0,0 +1,252 @@
+: immediate   lastxt @ dup c@ negate swap c! ;
+
+: \   source nip >in ! ; immediate \ Copyright 2004, 2012 Lars Brinkhoff
+
+: char \ ( "word" -- char )
+    bl-word here 1+ c@ ;
+
+: ahead  here 0 , ;
+
+: resolve   here swap ! ;
+
+: '   bl-word here find 0branch [ ahead ] exit [ resolve ] 0 ;
+
+: postpone-nonimmediate   [ ' literal , ' compile, ] literal , ;
+
+: create   dovariable_code header, reveal ;
+
+create postponers
+    ' postpone-nonimmediate ,
+    ' abort ,
+    ' , ,
+
+: word \ ( char "<chars>string<char>" -- caddr )
+    drop bl-word here ;
+
+: postpone \ ( C: "word" -- )
+    bl word find 1+ cells  postponers + @ execute ; immediate
+
+: unresolved \ ( C: "word" -- orig )
+    postpone postpone  postpone ahead ; immediate
+
+: chars \ ( n1 -- n2 )
+    ;
+
+: else \ ( -- ) ( C: orig1 -- orig2 )
+    unresolved branch swap resolve ; immediate
+
+: if \ ( flag -- ) ( C: -- orig )
+    unresolved 0branch ; immediate
+
+: then \ ( -- ) ( C: orig -- )
+    resolve ; immediate
+
+: [char] \ ( "word" -- )
+    char  postpone literal ; immediate
+
+: (does>)   lastxt @ dodoes_code over >code ! r> swap >does ! ;
+
+: does>   postpone (does>) ; immediate
+
+: begin \ ( -- ) ( C: -- dest )
+    here ; immediate
+
+: while \ ( x -- ) ( C: dest -- orig dest )
+    unresolved 0branch swap ; immediate
+
+: repeat \ ( -- ) ( C: orig dest -- )
+    postpone branch ,  resolve ; immediate
+
+: until \ ( x -- ) ( C: dest -- )
+    postpone 0branch , ; immediate
+
+: recurse   lastxt @ compile, ; immediate
+
+: pad \ ( -- addr )
+    here 1024 + ;
+
+: parse \ ( char "string<char>" -- addr n )
+    pad >r  begin
+	source? if <source 2dup <> else 0 0 then
+    while
+	r@ c!  r> 1+ >r
+    repeat  2drop  pad r> over - ;
+
+: ( \ ( "string<paren>" -- )
+    [ char ) ] literal parse 2drop ; immediate
+    \ TODO: If necessary, refill and keep parsing.
+
+: string, ( addr n -- )
+    here over allot align  swap cmove ;
+
+: (s") ( -- addr n ) ( R: ret1 -- ret2 )
+    r> dup @ swap cell+ 2dup + aligned >r swap ;
+
+create squote   128 allot
+
+: s" ( "string<quote>" -- addr n )
+    state @ if
+	postpone (s")  [char] " parse  dup ,  string,
+    else
+	[char] " parse  >r squote r@ cmove  squote r>
+    then ; immediate
+
+: (abort") ( ... addr n -- ) ( R: ... -- )
+    cr type cr abort ;
+
+: abort" ( ... x "string<quote>" -- ) ( R: ... -- )
+    postpone if  postpone s"  postpone (abort")  postpone then ; immediate
+
+\ ----------------------------------------------------------------------
+
+( Core words. )
+
+\ TODO: #
+\ TODO: #>
+\ TODO: #s
+
+: and  ( x y -- x&y )   nand invert ;
+
+: *   1 2>r 0 swap begin r@ while
+         r> r> swap 2dup dup + 2>r and if swap over + swap then dup +
+      repeat r> r> 2drop drop ;
+
+\ TODO: */mod
+
+: +loop ( -- ) ( C: nest-sys -- )
+    postpone (+loop)  postpone 0branch  ,  postpone unloop ; immediate
+
+: space   bl emit ;
+
+: ?.-  dup 0 < if [char] - emit negate then ;
+
+: digit   [char] 0 + emit ;
+
+: (.)   base @ /mod  ?dup if recurse then  digit ;
+
+: ." ( "string<quote>" -- )   postpone s"  postpone type ; immediate
+
+: . ( x -- )   ?.- (.) space ;
+
+: postpone-number ( caddr -- )
+    0 0 rot count >number dup 0= if
+	2drop nip
+	postpone (literal)  postpone (literal)  postpone ,
+	postpone literal  postpone ,
+    else
+	." Undefined: " type cr abort
+    then ;
+
+' postpone-number  postponers cell+  !
+
+: / ( x y -- x/y )   /mod nip ;
+
+: 0< ( n -- flag )   0 < ;
+
+: 1- ( n -- n-1 )   -1 + ;
+
+: 2! ( x1 x2 addr -- )   swap over ! cell+ ! ;
+
+: 2* ( n -- 2n )   dup + ;
+
+\ Kernel: 2/
+
+: 2@ ( addr -- x1 x2 )   dup cell+ @ swap @ ;
+
+\ Kernel: 2drop
+\ Kernel: 2dup
+
+\ TODO: 2over ( x1 x2 x3 x4 -- x1 x2 x3 x4 x1 x2 )
+\           3 pick 3 pick ;
+
+\ TODO: 2swap
+
+\ TODO: <#
+
+: abs ( n -- |n| )
+    dup 0< if negate then ;
+
+\ TODO: accept
+
+: c, ( n -- )
+    here c!  1 chars allot ;
+
+: char+ ( n1 -- n2 )
+    1+ ;
+
+: constant   create , does> @ ;
+
+: decimal ( -- )
+    10 base ! ;
+
+: depth ( -- n )
+    data_stack 100 cells +  'SP @  - /cell /  2 - ;
+
+: do ( n1 n2 -- ) ( R: -- loop-sys ) ( C: -- do-sys )
+    postpone 2>r  here ; immediate
+
+\ TODO: environment?
+\ TODO: evaluate
+\ TODO: fill
+\ TODO: fm/mod )
+\ TODO: hold
+
+: j ( -- x1 ) ( R: x1 x2 x3 -- x1 x2 x3 )
+    'RP @ 3 cells + @ ;
+
+\ TODO: leave
+
+: loop ( -- ) ( C: nest-sys -- )
+    postpone 1  postpone (+loop)
+    postpone 0branch  ,
+    postpone unloop ; immediate
+
+: lshift   begin ?dup while 1- swap dup + swap repeat ;
+
+: rshift   1 begin over while dup + swap 1- swap repeat nip
+           2>r 0 1 begin r@ while
+              r> r> 2dup swap dup + 2>r and if swap over + swap then dup +
+           repeat r> r> 2drop drop ;
+
+: max ( x y -- max[x,y] )
+    2dup > if drop else nip then ;
+
+\ Kernel: min
+\ TODO:   mod
+\ TODO:   move
+
+: (quit) ( R: ... -- )
+    return_stack 100 cells + 'RP !
+    0 'source-id !  tib ''source !  #tib ''#source !
+    postpone [
+    begin
+	refill
+    while
+	interpret  state @ 0= if ." ok" cr then
+    repeat
+    bye ;
+
+' (quit)  ' quit >body cell+  !
+
+\ TODO: s>d
+\ TODO: sign
+\ TODO: sm/rem
+
+: spaces ( n -- )
+    0 do space loop ;
+
+\ TODO: u.
+
+: signbit ( -- n )   -1 1 rshift invert ;
+
+: xor ( x y -- x^y )    2dup nand >r r@ nand swap r> nand nand ;
+
+: u<  ( x y -- flag )  signbit xor swap signbit xor > ;
+
+\ TODO: um/mod
+
+: variable ( "word" -- )
+    create /cell allot ;
+
+: ['] \ ( C: "word" -- )
+    ' postpone literal ; immediate

samples/Forth/core.for

@@ -0,0 +1,252 @@
+: immediate   lastxt @ dup c@ negate swap c! ;
+
+: \   source nip >in ! ; immediate \ Copyright 2004, 2012 Lars Brinkhoff
+
+: char \ ( "word" -- char )
+    bl-word here 1+ c@ ;
+
+: ahead  here 0 , ;
+
+: resolve   here swap ! ;
+
+: '   bl-word here find 0branch [ ahead ] exit [ resolve ] 0 ;
+
+: postpone-nonimmediate   [ ' literal , ' compile, ] literal , ;
+
+: create   dovariable_code header, reveal ;
+
+create postponers
+    ' postpone-nonimmediate ,
+    ' abort ,
+    ' , ,
+
+: word \ ( char "<chars>string<char>" -- caddr )
+    drop bl-word here ;
+
+: postpone \ ( C: "word" -- )
+    bl word find 1+ cells  postponers + @ execute ; immediate
+
+: unresolved \ ( C: "word" -- orig )
+    postpone postpone  postpone ahead ; immediate
+
+: chars \ ( n1 -- n2 )
+    ;
+
+: else \ ( -- ) ( C: orig1 -- orig2 )
+    unresolved branch swap resolve ; immediate
+
+: if \ ( flag -- ) ( C: -- orig )
+    unresolved 0branch ; immediate
+
+: then \ ( -- ) ( C: orig -- )
+    resolve ; immediate
+
+: [char] \ ( "word" -- )
+    char  postpone literal ; immediate
+
+: (does>)   lastxt @ dodoes_code over >code ! r> swap >does ! ;
+
+: does>   postpone (does>) ; immediate
+
+: begin \ ( -- ) ( C: -- dest )
+    here ; immediate
+
+: while \ ( x -- ) ( C: dest -- orig dest )
+    unresolved 0branch swap ; immediate
+
+: repeat \ ( -- ) ( C: orig dest -- )
+    postpone branch ,  resolve ; immediate
+
+: until \ ( x -- ) ( C: dest -- )
+    postpone 0branch , ; immediate
+
+: recurse   lastxt @ compile, ; immediate
+
+: pad \ ( -- addr )
+    here 1024 + ;
+
+: parse \ ( char "string<char>" -- addr n )
+    pad >r  begin
+	source? if <source 2dup <> else 0 0 then
+    while
+	r@ c!  r> 1+ >r
+    repeat  2drop  pad r> over - ;
+
+: ( \ ( "string<paren>" -- )
+    [ char ) ] literal parse 2drop ; immediate
+    \ TODO: If necessary, refill and keep parsing.
+
+: string, ( addr n -- )
+    here over allot align  swap cmove ;
+
+: (s") ( -- addr n ) ( R: ret1 -- ret2 )
+    r> dup @ swap cell+ 2dup + aligned >r swap ;
+
+create squote   128 allot
+
+: s" ( "string<quote>" -- addr n )
+    state @ if
+	postpone (s")  [char] " parse  dup ,  string,
+    else
+	[char] " parse  >r squote r@ cmove  squote r>
+    then ; immediate
+
+: (abort") ( ... addr n -- ) ( R: ... -- )
+    cr type cr abort ;
+
+: abort" ( ... x "string<quote>" -- ) ( R: ... -- )
+    postpone if  postpone s"  postpone (abort")  postpone then ; immediate
+
+\ ----------------------------------------------------------------------
+
+( Core words. )
+
+\ TODO: #
+\ TODO: #>
+\ TODO: #s
+
+: and  ( x y -- x&y )   nand invert ;
+
+: *   1 2>r 0 swap begin r@ while
+         r> r> swap 2dup dup + 2>r and if swap over + swap then dup +
+      repeat r> r> 2drop drop ;
+
+\ TODO: */mod
+
+: +loop ( -- ) ( C: nest-sys -- )
+    postpone (+loop)  postpone 0branch  ,  postpone unloop ; immediate
+
+: space   bl emit ;
+
+: ?.-  dup 0 < if [char] - emit negate then ;
+
+: digit   [char] 0 + emit ;
+
+: (.)   base @ /mod  ?dup if recurse then  digit ;
+
+: ." ( "string<quote>" -- )   postpone s"  postpone type ; immediate
+
+: . ( x -- )   ?.- (.) space ;
+
+: postpone-number ( caddr -- )
+    0 0 rot count >number dup 0= if
+	2drop nip
+	postpone (literal)  postpone (literal)  postpone ,
+	postpone literal  postpone ,
+    else
+	." Undefined: " type cr abort
+    then ;
+
+' postpone-number  postponers cell+  !
+
+: / ( x y -- x/y )   /mod nip ;
+
+: 0< ( n -- flag )   0 < ;
+
+: 1- ( n -- n-1 )   -1 + ;
+
+: 2! ( x1 x2 addr -- )   swap over ! cell+ ! ;
+
+: 2* ( n -- 2n )   dup + ;
+
+\ Kernel: 2/
+
+: 2@ ( addr -- x1 x2 )   dup cell+ @ swap @ ;
+
+\ Kernel: 2drop
+\ Kernel: 2dup
+
+\ TODO: 2over ( x1 x2 x3 x4 -- x1 x2 x3 x4 x1 x2 )
+\           3 pick 3 pick ;
+
+\ TODO: 2swap
+
+\ TODO: <#
+
+: abs ( n -- |n| )
+    dup 0< if negate then ;
+
+\ TODO: accept
+
+: c, ( n -- )
+    here c!  1 chars allot ;
+
+: char+ ( n1 -- n2 )
+    1+ ;
+
+: constant   create , does> @ ;
+
+: decimal ( -- )
+    10 base ! ;
+
+: depth ( -- n )
+    data_stack 100 cells +  'SP @  - /cell /  2 - ;
+
+: do ( n1 n2 -- ) ( R: -- loop-sys ) ( C: -- do-sys )
+    postpone 2>r  here ; immediate
+
+\ TODO: environment?
+\ TODO: evaluate
+\ TODO: fill
+\ TODO: fm/mod )
+\ TODO: hold
+
+: j ( -- x1 ) ( R: x1 x2 x3 -- x1 x2 x3 )
+    'RP @ 3 cells + @ ;
+
+\ TODO: leave
+
+: loop ( -- ) ( C: nest-sys -- )
+    postpone 1  postpone (+loop)
+    postpone 0branch  ,
+    postpone unloop ; immediate
+
+: lshift   begin ?dup while 1- swap dup + swap repeat ;
+
+: rshift   1 begin over while dup + swap 1- swap repeat nip
+           2>r 0 1 begin r@ while
+              r> r> 2dup swap dup + 2>r and if swap over + swap then dup +
+           repeat r> r> 2drop drop ;
+
+: max ( x y -- max[x,y] )
+    2dup > if drop else nip then ;
+
+\ Kernel: min
+\ TODO:   mod
+\ TODO:   move
+
+: (quit) ( R: ... -- )
+    return_stack 100 cells + 'RP !
+    0 'source-id !  tib ''source !  #tib ''#source !
+    postpone [
+    begin
+	refill
+    while
+	interpret  state @ 0= if ." ok" cr then
+    repeat
+    bye ;
+
+' (quit)  ' quit >body cell+  !
+
+\ TODO: s>d
+\ TODO: sign
+\ TODO: sm/rem
+
+: spaces ( n -- )
+    0 do space loop ;
+
+\ TODO: u.
+
+: signbit ( -- n )   -1 1 rshift invert ;
+
+: xor ( x y -- x^y )    2dup nand >r r@ nand swap r> nand nand ;
+
+: u<  ( x y -- flag )  signbit xor swap signbit xor > ;
+
+\ TODO: um/mod
+
+: variable ( "word" -- )
+    create /cell allot ;
+
+: ['] \ ( C: "word" -- )
+    ' postpone literal ; immediate

samples/Forth/core.fs

@@ -0,0 +1,252 @@
+: immediate   lastxt @ dup c@ negate swap c! ;
+
+: \   source nip >in ! ; immediate \ Copyright 2004, 2012 Lars Brinkhoff
+
+: char \ ( "word" -- char )
+    bl-word here 1+ c@ ;
+
+: ahead  here 0 , ;
+
+: resolve   here swap ! ;
+
+: '   bl-word here find 0branch [ ahead ] exit [ resolve ] 0 ;
+
+: postpone-nonimmediate   [ ' literal , ' compile, ] literal , ;
+
+: create   dovariable_code header, reveal ;
+
+create postponers
+    ' postpone-nonimmediate ,
+    ' abort ,
+    ' , ,
+
+: word \ ( char "<chars>string<char>" -- caddr )
+    drop bl-word here ;
+
+: postpone \ ( C: "word" -- )
+    bl word find 1+ cells  postponers + @ execute ; immediate
+
+: unresolved \ ( C: "word" -- orig )
+    postpone postpone  postpone ahead ; immediate
+
+: chars \ ( n1 -- n2 )
+    ;
+
+: else \ ( -- ) ( C: orig1 -- orig2 )
+    unresolved branch swap resolve ; immediate
+
+: if \ ( flag -- ) ( C: -- orig )
+    unresolved 0branch ; immediate
+
+: then \ ( -- ) ( C: orig -- )
+    resolve ; immediate
+
+: [char] \ ( "word" -- )
+    char  postpone literal ; immediate
+
+: (does>)   lastxt @ dodoes_code over >code ! r> swap >does ! ;
+
+: does>   postpone (does>) ; immediate
+
+: begin \ ( -- ) ( C: -- dest )
+    here ; immediate
+
+: while \ ( x -- ) ( C: dest -- orig dest )
+    unresolved 0branch swap ; immediate
+
+: repeat \ ( -- ) ( C: orig dest -- )
+    postpone branch ,  resolve ; immediate
+
+: until \ ( x -- ) ( C: dest -- )
+    postpone 0branch , ; immediate
+
+: recurse   lastxt @ compile, ; immediate
+
+: pad \ ( -- addr )
+    here 1024 + ;
+
+: parse \ ( char "string<char>" -- addr n )
+    pad >r  begin
+	source? if <source 2dup <> else 0 0 then
+    while
+	r@ c!  r> 1+ >r
+    repeat  2drop  pad r> over - ;
+
+: ( \ ( "string<paren>" -- )
+    [ char ) ] literal parse 2drop ; immediate
+    \ TODO: If necessary, refill and keep parsing.
+
+: string, ( addr n -- )
+    here over allot align  swap cmove ;
+
+: (s") ( -- addr n ) ( R: ret1 -- ret2 )
+    r> dup @ swap cell+ 2dup + aligned >r swap ;
+
+create squote   128 allot
+
+: s" ( "string<quote>" -- addr n )
+    state @ if
+	postpone (s")  [char] " parse  dup ,  string,
+    else
+	[char] " parse  >r squote r@ cmove  squote r>
+    then ; immediate
+
+: (abort") ( ... addr n -- ) ( R: ... -- )
+    cr type cr abort ;
+
+: abort" ( ... x "string<quote>" -- ) ( R: ... -- )
+    postpone if  postpone s"  postpone (abort")  postpone then ; immediate
+
+\ ----------------------------------------------------------------------
+
+( Core words. )
+
+\ TODO: #
+\ TODO: #>
+\ TODO: #s
+
+: and  ( x y -- x&y )   nand invert ;
+
+: *   1 2>r 0 swap begin r@ while
+         r> r> swap 2dup dup + 2>r and if swap over + swap then dup +
+      repeat r> r> 2drop drop ;
+
+\ TODO: */mod
+
+: +loop ( -- ) ( C: nest-sys -- )
+    postpone (+loop)  postpone 0branch  ,  postpone unloop ; immediate
+
+: space   bl emit ;
+
+: ?.-  dup 0 < if [char] - emit negate then ;
+
+: digit   [char] 0 + emit ;
+
+: (.)   base @ /mod  ?dup if recurse then  digit ;
+
+: ." ( "string<quote>" -- )   postpone s"  postpone type ; immediate
+
+: . ( x -- )   ?.- (.) space ;
+
+: postpone-number ( caddr -- )
+    0 0 rot count >number dup 0= if
+	2drop nip
+	postpone (literal)  postpone (literal)  postpone ,
+	postpone literal  postpone ,
+    else
+	." Undefined: " type cr abort
+    then ;
+
+' postpone-number  postponers cell+  !
+
+: / ( x y -- x/y )   /mod nip ;
+
+: 0< ( n -- flag )   0 < ;
+
+: 1- ( n -- n-1 )   -1 + ;
+
+: 2! ( x1 x2 addr -- )   swap over ! cell+ ! ;
+
+: 2* ( n -- 2n )   dup + ;
+
+\ Kernel: 2/
+
+: 2@ ( addr -- x1 x2 )   dup cell+ @ swap @ ;
+
+\ Kernel: 2drop
+\ Kernel: 2dup
+
+\ TODO: 2over ( x1 x2 x3 x4 -- x1 x2 x3 x4 x1 x2 )
+\           3 pick 3 pick ;
+
+\ TODO: 2swap
+
+\ TODO: <#
+
+: abs ( n -- |n| )
+    dup 0< if negate then ;
+
+\ TODO: accept
+
+: c, ( n -- )
+    here c!  1 chars allot ;
+
+: char+ ( n1 -- n2 )
+    1+ ;
+
+: constant   create , does> @ ;
+
+: decimal ( -- )
+    10 base ! ;
+
+: depth ( -- n )
+    data_stack 100 cells +  'SP @  - /cell /  2 - ;
+
+: do ( n1 n2 -- ) ( R: -- loop-sys ) ( C: -- do-sys )
+    postpone 2>r  here ; immediate
+
+\ TODO: environment?
+\ TODO: evaluate
+\ TODO: fill
+\ TODO: fm/mod )
+\ TODO: hold
+
+: j ( -- x1 ) ( R: x1 x2 x3 -- x1 x2 x3 )
+    'RP @ 3 cells + @ ;
+
+\ TODO: leave
+
+: loop ( -- ) ( C: nest-sys -- )
+    postpone 1  postpone (+loop)
+    postpone 0branch  ,
+    postpone unloop ; immediate
+
+: lshift   begin ?dup while 1- swap dup + swap repeat ;
+
+: rshift   1 begin over while dup + swap 1- swap repeat nip
+           2>r 0 1 begin r@ while
+              r> r> 2dup swap dup + 2>r and if swap over + swap then dup +
+           repeat r> r> 2drop drop ;
+
+: max ( x y -- max[x,y] )
+    2dup > if drop else nip then ;
+
+\ Kernel: min
+\ TODO:   mod
+\ TODO:   move
+
+: (quit) ( R: ... -- )
+    return_stack 100 cells + 'RP !
+    0 'source-id !  tib ''source !  #tib ''#source !
+    postpone [
+    begin
+	refill
+    while
+	interpret  state @ 0= if ." ok" cr then
+    repeat
+    bye ;
+
+' (quit)  ' quit >body cell+  !
+
+\ TODO: s>d
+\ TODO: sign
+\ TODO: sm/rem
+
+: spaces ( n -- )
+    0 do space loop ;
+
+\ TODO: u.
+
+: signbit ( -- n )   -1 1 rshift invert ;
+
+: xor ( x y -- x^y )    2dup nand >r r@ nand swap r> nand nand ;
+
+: u<  ( x y -- flag )  signbit xor swap signbit xor > ;
+
+\ TODO: um/mod
+
+: variable ( "word" -- )
+    create /cell allot ;
+
+: ['] \ ( C: "word" -- )
+    ' postpone literal ; immediate

samples/Forth/core1.F

@@ -0,0 +1,252 @@
+: immediate   lastxt @ dup c@ negate swap c! ;
+
+: \   source nip >in ! ; immediate \ Copyright 2004, 2012 Lars Brinkhoff
+
+: char \ ( "word" -- char )
+    bl-word here 1+ c@ ;
+
+: ahead  here 0 , ;
+
+: resolve   here swap ! ;
+
+: '   bl-word here find 0branch [ ahead ] exit [ resolve ] 0 ;
+
+: postpone-nonimmediate   [ ' literal , ' compile, ] literal , ;
+
+: create   dovariable_code header, reveal ;
+
+create postponers
+    ' postpone-nonimmediate ,
+    ' abort ,
+    ' , ,
+
+: word \ ( char "<chars>string<char>" -- caddr )
+    drop bl-word here ;
+
+: postpone \ ( C: "word" -- )
+    bl word find 1+ cells  postponers + @ execute ; immediate
+
+: unresolved \ ( C: "word" -- orig )
+    postpone postpone  postpone ahead ; immediate
+
+: chars \ ( n1 -- n2 )
+    ;
+
+: else \ ( -- ) ( C: orig1 -- orig2 )
+    unresolved branch swap resolve ; immediate
+
+: if \ ( flag -- ) ( C: -- orig )
+    unresolved 0branch ; immediate
+
+: then \ ( -- ) ( C: orig -- )
+    resolve ; immediate
+
+: [char] \ ( "word" -- )
+    char  postpone literal ; immediate
+
+: (does>)   lastxt @ dodoes_code over >code ! r> swap >does ! ;
+
+: does>   postpone (does>) ; immediate
+
+: begin \ ( -- ) ( C: -- dest )
+    here ; immediate
+
+: while \ ( x -- ) ( C: dest -- orig dest )
+    unresolved 0branch swap ; immediate
+
+: repeat \ ( -- ) ( C: orig dest -- )
+    postpone branch ,  resolve ; immediate
+
+: until \ ( x -- ) ( C: dest -- )
+    postpone 0branch , ; immediate
+
+: recurse   lastxt @ compile, ; immediate
+
+: pad \ ( -- addr )
+    here 1024 + ;
+
+: parse \ ( char "string<char>" -- addr n )
+    pad >r  begin
+	source? if <source 2dup <> else 0 0 then
+    while
+	r@ c!  r> 1+ >r
+    repeat  2drop  pad r> over - ;
+
+: ( \ ( "string<paren>" -- )
+    [ char ) ] literal parse 2drop ; immediate
+    \ TODO: If necessary, refill and keep parsing.
+
+: string, ( addr n -- )
+    here over allot align  swap cmove ;
+
+: (s") ( -- addr n ) ( R: ret1 -- ret2 )
+    r> dup @ swap cell+ 2dup + aligned >r swap ;
+
+create squote   128 allot
+
+: s" ( "string<quote>" -- addr n )
+    state @ if
+	postpone (s")  [char] " parse  dup ,  string,
+    else
+	[char] " parse  >r squote r@ cmove  squote r>
+    then ; immediate
+
+: (abort") ( ... addr n -- ) ( R: ... -- )
+    cr type cr abort ;
+
+: abort" ( ... x "string<quote>" -- ) ( R: ... -- )
+    postpone if  postpone s"  postpone (abort")  postpone then ; immediate
+
+\ ----------------------------------------------------------------------
+
+( Core words. )
+
+\ TODO: #
+\ TODO: #>
+\ TODO: #s
+
+: and  ( x y -- x&y )   nand invert ;
+
+: *   1 2>r 0 swap begin r@ while
+         r> r> swap 2dup dup + 2>r and if swap over + swap then dup +
+      repeat r> r> 2drop drop ;
+
+\ TODO: */mod
+
+: +loop ( -- ) ( C: nest-sys -- )
+    postpone (+loop)  postpone 0branch  ,  postpone unloop ; immediate
+
+: space   bl emit ;
+
+: ?.-  dup 0 < if [char] - emit negate then ;
+
+: digit   [char] 0 + emit ;
+
+: (.)   base @ /mod  ?dup if recurse then  digit ;
+
+: ." ( "string<quote>" -- )   postpone s"  postpone type ; immediate
+
+: . ( x -- )   ?.- (.) space ;
+
+: postpone-number ( caddr -- )
+    0 0 rot count >number dup 0= if
+	2drop nip
+	postpone (literal)  postpone (literal)  postpone ,
+	postpone literal  postpone ,
+    else
+	." Undefined: " type cr abort
+    then ;
+
+' postpone-number  postponers cell+  !
+
+: / ( x y -- x/y )   /mod nip ;
+
+: 0< ( n -- flag )   0 < ;
+
+: 1- ( n -- n-1 )   -1 + ;
+
+: 2! ( x1 x2 addr -- )   swap over ! cell+ ! ;
+
+: 2* ( n -- 2n )   dup + ;
+
+\ Kernel: 2/
+
+: 2@ ( addr -- x1 x2 )   dup cell+ @ swap @ ;
+
+\ Kernel: 2drop
+\ Kernel: 2dup
+
+\ TODO: 2over ( x1 x2 x3 x4 -- x1 x2 x3 x4 x1 x2 )
+\           3 pick 3 pick ;
+
+\ TODO: 2swap
+
+\ TODO: <#
+
+: abs ( n -- |n| )
+    dup 0< if negate then ;
+
+\ TODO: accept
+
+: c, ( n -- )
+    here c!  1 chars allot ;
+
+: char+ ( n1 -- n2 )
+    1+ ;
+
+: constant   create , does> @ ;
+
+: decimal ( -- )
+    10 base ! ;
+
+: depth ( -- n )
+    data_stack 100 cells +  'SP @  - /cell /  2 - ;
+
+: do ( n1 n2 -- ) ( R: -- loop-sys ) ( C: -- do-sys )
+    postpone 2>r  here ; immediate
+
+\ TODO: environment?
+\ TODO: evaluate
+\ TODO: fill
+\ TODO: fm/mod )
+\ TODO: hold
+
+: j ( -- x1 ) ( R: x1 x2 x3 -- x1 x2 x3 )
+    'RP @ 3 cells + @ ;
+
+\ TODO: leave
+
+: loop ( -- ) ( C: nest-sys -- )
+    postpone 1  postpone (+loop)
+    postpone 0branch  ,
+    postpone unloop ; immediate
+
+: lshift   begin ?dup while 1- swap dup + swap repeat ;
+
+: rshift   1 begin over while dup + swap 1- swap repeat nip
+           2>r 0 1 begin r@ while
+              r> r> 2dup swap dup + 2>r and if swap over + swap then dup +
+           repeat r> r> 2drop drop ;
+
+: max ( x y -- max[x,y] )
+    2dup > if drop else nip then ;
+
+\ Kernel: min
+\ TODO:   mod
+\ TODO:   move
+
+: (quit) ( R: ... -- )
+    return_stack 100 cells + 'RP !
+    0 'source-id !  tib ''source !  #tib ''#source !
+    postpone [
+    begin
+	refill
+    while
+	interpret  state @ 0= if ." ok" cr then
+    repeat
+    bye ;
+
+' (quit)  ' quit >body cell+  !
+
+\ TODO: s>d
+\ TODO: sign
+\ TODO: sm/rem
+
+: spaces ( n -- )
+    0 do space loop ;
+
+\ TODO: u.
+
+: signbit ( -- n )   -1 1 rshift invert ;
+
+: xor ( x y -- x^y )    2dup nand >r r@ nand swap r> nand nand ;
+
+: u<  ( x y -- flag )  signbit xor swap signbit xor > ;
+
+\ TODO: um/mod
+
+: variable ( "word" -- )
+    create /cell allot ;
+
+: ['] \ ( C: "word" -- )
+    ' postpone literal ; immediate

samples/Forth/enum.frt

@@ -0,0 +1,7 @@
+\ Implements ENUM.
+
+\ Double DOES>!
+: enum   create 0 ,  does> create dup @ 1 rot +! ,  does> @ ;
+
+\ But this is simpler.
+: enum   create 0 ,  does> dup @ constant 1 swap +! ;

samples/Forth/macros.frt

@@ -0,0 +1,8 @@
+\ Simplifies compiling words.
+
+: [[     ; immediate
+: '<>    >in @ ' swap >in ! <> ;
+: (]])   begin dup '<> while postpone postpone repeat drop ;
+: ]]     ['] [[ (]]) ; immediate
+
+( Usage:   : foo ]] dup * [[ ; immediate   : bar 42 foo . ; )

samples/G-code/duettest.g

@@ -0,0 +1,57 @@
+; RepRapPro Ormerod
+; Board test GCodes
+M111 S1; Debug on
+G21 ; mm
+G90 ; Absolute positioning
+M83 ; Extrusion relative
+M906 X800 Y800 Z800 E800 ; Motor currents (mA)
+T0 ; Extruder 0
+G1 X50 F500
+G1 X0
+G4 P500
+G1 Y50 F500
+G1 Y0
+G4 P500
+G1 Z20 F200
+G1 Z0
+G4 P500
+G1 E20 F200
+G1 E-20
+G4 P500
+M106 S255
+G4 P500
+M106 S0
+G4 P500
+M105
+G10 P0 S100
+T0
+M140 S100
+G4 P5000
+M105
+G4 P5000
+M105
+G4 P5000
+M105
+G4 P5000
+M105
+G4 P5000
+M105
+G4 P5000
+M105
+G4 P5000
+M105
+G4 P5000
+M105
+G4 P5000
+M105
+G4 P5000
+M105
+G4 P5000
+M105
+G4 P5000
+M105
+M0
+
+
+
+