3TS-Coo

3TS-Coo is a consistency check tool for transactional databases. Coo checks the consistency of databases in accurate (all types of anomalies), user-friendly (SQL-based test), and cost-effective (one-time checking in a few minutes) ways. You can check out the code and detailed result on GitHub branch. Contributions in any kind are welcome.

The original and executed schedules are available for analysis and debugging. You can click each hyperlink for more detailed test cases and executed results. The formal expressions of test cases are also available.

The result behaviors are classified into two types, i.e., anomaly (A) and consistency. For anomaly occurrence, data anomalies are not recognized by databases, resulting in data inconsistencies, meaning the executed schedule with no equivalent serializable execution (or a Partial Order Pair (POP) cycle). While for the consistent performance, databases either pass (P) the anomaly test cases with a serializable result (no POP cycle) or rollback transactions due to rules (R), deadlock detection (D), or timeout (T) reached. The isolation levels are Serializable (SER), Repeatable Read (RR), Read Committed (RC), Read Uncommitted (RU), and Snapshot Isolation (SI).

Interesting results

Please check out the below results of Oracle 21.3.0, TDSQL 2.0.1, and PostgreSQL 12.4.

                                                                                                                                                   
No. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35
Databases
Versions
Report date		Test caseDirty ReadNon-repeatable ReadIntermediate ReadIntermediate Read CommittedLost Self UpdateWrite-read SkewWrite-read Skew CommittedDouble-write Skew 1Double-writeSkew 1 CommittedDouble-write Skew 2Read SkewRead Skew 2Read Skew 2 CommittedStep RATDirty WriteFull-writeFull-write CommittedLost UpdateLost Self Update CommittedDouble-write Skew 2 CommittedFull-write SkewFull-write Skew CommittedRead-write Skew 1Read-write Skew 2Read-write Skew 2 CommittedStep WATNon-repeatable Read CommittedLost Update CommittedRead Skew CommittedRead-writeSkew 1 CommittedWrite SkewWrite Skew CommittedStep IATNon-repeatable Read PredicateWrite Skew Predicate
Oracle
v21.3.0
2022.05.26		 SER P P P P R A A R R R P P P A R R R R R R D D R R R D P R P R A R R P R
RC P P P P P A P P P P P P P A P P P A P P D D A A A D A A A A A A A P A
TDSQL
v2.0.1
2022.05.26		 SER P P P P P R R R R R R R R R P P P R P R D D R R R D P R R R R R R P R
RR P P P P P A P P P P P P P A P P P A P P D D A A A D P A P A A A A P A
RC P P P P P A P P P P P P P A P P P A P P D D A A A D A A A A A A A P A
PostgreSQL
v12.4
2022.05.26		 SER P P P P R R R R R R P P P R R R R R R R D D R R R D P R P R R R R P R
RR P P P P R A A R R R P P P A R R R R R R D D R R R D P R P R A A A P A
RC P P P P P A P P P P P P P A P P P A P P D D A A A D A A A A A A A P A

Tested results

The below is the result of checking MySQL 8.0.20, MyRocks 8.0.26, SQL Server 15.0, TiDB 4.0.5/5.4.0, Oracle 12.1.0, OceanBase XE 2.2.50, OceanBase CE 3.1.2, Greenplum 6.20, CockroachDB 19.2.2, and MongoDB 4.4.4.

                                                                                                                                           
No. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
Databases
Versions
Report date		Test caseDirty ReadNon-repeatable ReadIntermediate ReadIntermediate Read CommittedLost Self UpdateWrite-read SkewWrite-read Skew CommittedDouble-write Skew 1Double-writeSkew 1 CommittedDouble-write Skew 2Read SkewRead Skew 2Read Skew 2 CommittedStep RATDirty WriteFull-writeFull-write CommittedLost UpdateLost Self Update CommittedDouble-write Skew 2 CommittedFull-write SkewFull-write Skew CommittedRead-write Skew 1Read-write Skew 2Read-write Skew 2 CommittedStep WATNon-repeatable Read CommittedLost Update CommittedRead Skew CommittedRead-writeSkew 1 CommittedWrite SkewWrite Skew CommittedStep IAT
MySQL
v8.0.20
2022.04.30		 SER P P P P P D D D D D D D D D P P P D P D D D D D D D P D D D D D D
RR P P P P P A P P P P P P P A P P P A P P D D A A A D P A P A A A A
RC P P P P P A P P P P P P P A P P P A P P D D A A A D A A A A A A A
RU A A A A P A A A A A A A A A P P P A P A D D A A A D A A A A A A A
MyRocks
v8.0.26
2022.04.30		 SER P P P P P D D D D D D D D D P P P D P D D D D D D D P D D D D D D
RR P P P P P A P P P P P P P A P P P A P P D D A A A D P A P A A A A
RC P P P P P A P P P P P P P A P P P A P P D D A A A D A A A A A A A
RU A A A A P A A A A A A A A A P P P A P A D D A A A D A A A A A A A
SQL Server
v15.0
2022.04.30		 SER P P P P P D D D D D D D D D P P P P P D D D D D D D P P D D D D D
SI P P P P R A A R R R P P P A R R R R R R D D R R R D P R P R A A A
RR P P P P P D D D D D D D D D P P P P P D D D D D D D P P D D D D D
RC P A P P P D D D D D A A A D P P P A P D D D A A A D A A A A A A A
RU A A A A P A A A A A A A A A P P P A P A D D A A A D A A A A A A A
TiDB
v4.0.5/v5.4.0
2022.04.30		 OPT P P P P R A A R R R P P P A R R R R R R R R R R R R P R P R A A A
RR P P P P P A A P P P P P P A P P P A P P D D A A A D P A P A A A A
RC P P P P P A P P P P P P P A P P P A P P D D A A A D A A A A A A A
Oracle
v12.1.0
2022.04.30		 SER P P P P R A A R R R P P P A R R R R R R D D R R R D P R P R A R R
RC P P P P P A P P P P P P P A P P P A P P D D A A A D A A A A A A A
OceanBase
XE v2.2.50
2022.04.30		 SER P P P P R A A R R R P P P A R R R R R R T T R R R T P R P R A A A
RR P P P P R A A R R R P P P A R R R R R R T T R R R T P R P R A A A
RC P P P P P A P P P P P P P A P P P A P P T T A A A T A A A A A A A
OceanBase
CE v3.1.0
2022.04.30		 RR P P P P P A P P P P P P P A P P P A P P T T A A A T A A A A A A A
RC P P P P P A P P P P P P P A P P P A P P T T A A A T A A A A A A A
Greenplum
v6.20.0
2022.04.30		 SER P P P P R A A R R R P P P A R R R R R R D D R R R D P R P R A A A
RC P P P P P A P P P P P P P A P P P A P P D D A A A P A A A A A A A
CockroachDB
v19.2.2
2022.04.30		 SER P P P P P P P R R P P P P P P P P R P P D D R R R D P R P R R R R
MongoDB
v4.4.4
2022.04.30		 SI P P P P P A A R R R P P P A R R R R R R R R R R R R P R P R A A A

Follow up

We are testing the scenarios with predicates. Please stay tuned!

Contact

axingguchen(AT)tencent(dot)com; blueseali(AT)tencent(dot)com