README.org

* COMMENT -*- mode: org -*-
#+Date: 2016-12-16
Time-stamp: <2017-01-25>

* database binding benchmark
I want to know the efficiency of the db binding for different language,
libraries and database servers.

- table:
  CREATE TABLE foo
  (id integer primary key,
   bar integer);
  INSERT INTO foo (id, bar) VALUES (1, 1);

- run many times:
  SELECT * FROM foo
  UPDATE foo SET bar=bar+1 WHERE id=1

* benchmark results
each entry has format: db_server, compiler, libraries
all build is release mode (with -O or equivalent).

| db    | language/runtime  | lib               | select (s) | update (s) |
|-------+-------------------+-------------------+------------+------------|
| pg9.4 | python/cpython2.7 | psycopg2          |       0.37 |       26.3 |
| pg9.4 | C/gcc             | libpq5            |       0.32 |       26.5 |
| pg9.4 | haskell/ghc8.0.1  | postgresql-simple |       0.63 |       27.2 |
| pg9.4 | haskell/ghc7.10.3 | postgresql-simple |       0.61 |       26.6 |
| pg9.4 | rust/rust1.13.0   | postgres          |       0.50 |       26.3 |
| pg9.4 | go/go1.7.3        | pq                |       0.37 |       27.2 |
| pg9.4 | racket/racket6.5  | built-in db       |       1.99 |       26.7 |

- Remarks:
  - python and go's db binding is as fast as C.
  - rust and haskell is slower. C is 1.65x faster.
  - racket is much slower. C is 5x faster.

** postgres 9.4, cpython 2.7, psycopg2==2.6.1
http://initd.org/psycopg/docs/usage.html
0.35
28.77
--
0.38
26.15
--
0.37
26.35
** postgres 9.4, C, libpq5
https://www.postgresql.org/docs/9.4/static/libpq.html
0.33
26.6
--
0.32
26.5
--
0.30
26.2

//side note: why adding -shared in LDFLAGS will just segfault the program?
** postgres 9.4, lts-7.13, ghc 8.0.1, postgresql-simple
0.71
27.5
--
0.63
27.2
--
0.61
26.2
** postgres 9.4, lts-5.18, ghc 7.10.3, postgresql-simple
0.61
26.6
--
0.59
26.8
--
0.63
26.6
** postgres 9.4, rust 1.13.0, postgres 0.13
0.48
26.3
--
0.53
26.2
--
0.50
28.0
** postgres 9.4, go 1.7.3, pq
0.385
27.0
--
0.367
27.6
--
0.423
30.7
--
0.362
27.2
--
0.372
26.4
** postgres 9.4, racket 6.5
2.21
26.72
--
1.85
26.63
--
2.19
26.78

- I think I maybe testing disk IO performance. should use the pg bench cluster
  for benchmarking. bench cluster listen on port 5433.

  when running against bench cluster.
  result is
  2.15
  2.29
  --
  2.10
  2.21

  So write performance is definitely testing disk IO performance. Read
  performance is related to the language and runtime.
- try compile it with raco exe first.
  2.00
  27.84
  --
  2.02
  27.62

  it's about the same.

- try use query-rows instead of query.
  1.78
  26.94
  --
  2.02
  29.08
  --
  1.99
  26.79

  it's still the slowest.