Merge protobuf_benchmarks and query_benchmark

[osa1]

This merges two benchmark packages into one to make it easier to run benchmarks
on Golem and locally.

api_benchmark is not merged: it runs a web server and benchmarks performance
in an actual browser. We should revisit those benchmarks to see if they make
sense and see if it's a good idea to merge them into the new benchmark package
in this PR.

Changes:

• Remove protobuf_benchmarks, query_benchmark. Move benchmarks in both to
  benchmarks.

• Use benchmark_harness in all benchmarks. Previously only
  protobuf_benchmarks used benchmark_harness.

• Because benchmark_harness reports 10x the actual number
  (Why is the benchmark reporting 10-times higher values? dart-lang/tools#791), to be compatible
  with the old query_benchmark results (which are stored in Golem) and
  generally report accurate results, BenchmarkBase class overrides the
  relevant method(s) from benchmark_harness.BenchmarkBase to divide the
  result by 10 before reporting.

  Outputs before:

  RunTimeRaw(protobuf_decode): 973 us
  RunTimeRaw(protobuf_decode_json): 3914 us
  RunTimeRaw(protobuf_encode): 3135 us
  RunTimeRaw(protobuf_encode_json): 4606 us
  RunTimeRaw(protobuf_decode): 30 us
  

  Outputs after:

  protobuf_query_decode(RunTimeRaw): 973.4907766990291 us.
  protobuf_query_decode_json(RunTimeRaw): 3925.9745098039216 us.
  protobuf_query_encode(RunTimeRaw): 3115.623076923077 us.
  protobuf_query_encode_json(RunTimeRaw): 4568.952272727272 us.
  protobuf_query_set_nested_value(RunTimeRaw): 18.9181422625804 us.
  

  All benchmarks (query_benchmark and others) use this new class.

• Directory structure:

  • datasets/ has benchmark inputs (i.e. binary encoded protos)

  • protos/ has benchmark protos

  • lib/ has code common to all benchmarks (file operations, benchmark class)

  • tool/compile_protos.sh compiles all protos

  • tool/run_protoc_plugin.sh runs protoc plugin from ../protoc_plugin.
    Used by compile_protos.sh.

  • tool/compile_benchmarks.dart compiles benchmark programs to native and
    JS, in parallel

  • bin/ contains benchmarks

  • protoc_version is used by Golem to download the right version of protoc

• Each benchmark now has its own executable. This is to allow benchmarking
  changes in isolation. For example, to avoid changing encoding benchmark
  results when decoding code was modified, because of global analyses or JIT.

• protobuf_benchmarks/ inputs no longer use benchmarks.proto. Instead
  the input files are now the payload fields of BenchmarkDatasets, and
  input file name specifies message type.

Fixes #613

Golem cl 4855277

---

cc @mkustermann

[sigurdm]

LGTM

[mkustermann]

TODO: Do we need to tweak number of iterations in query_benchmark
benchmarks?

👍 Would be good numbers stay compatible so we keep old history.

[mkustermann]

Another thing you may want to consider: Check the pubspec.lock file (generated by pub get from pubspec.yaml) into the repository (only for the benchmark folder). This will ensure that if a new minor version of a dependency (e.g. package:fixnum) is published, we won't get performance result changes at random commits - instead upgrades of 3rd party packages will be visible by the commit that updates the lock file.

[osa1]

I'm updating Golem to use benchmarks added in this PR. Once that works I'll merge this PR.

[sortie]

A couple thoughts :)

[sortie]

Consider another name for this class to avoid confusion between the upstream BenchmarkBase and this modified one.

I am planning on fixing the 10x bug in BenchmarkBase once I have time to think up a migration scheme