|
1 | 1 | # MySQL Binary Protocol |
2 | 2 |
|
3 | | -Vitess 2.1 is adding alpha support for the MySQL binary protocol (refered to as the |
4 | | -`protocol` in the rest of this document). This allows existing applications to |
5 | | -connect to Vitess directly without any change, or new driver / connector. |
| 3 | +Vitess supports MySQL binary protocol. This allows existing applications to connect to Vitess directly without any change, or without using a new driver or connector. This is now the recommended and the most popular protocol for connecting to Vitess. |
6 | 4 |
|
7 | | -However, this also has limitations. The RPC protocol traditionally exposed by |
8 | | -Vitess is richer in features, and allows finer grain control of the query |
9 | | -behaviours. |
10 | | - |
11 | | -This document explores the limitations of using this protocol. |
12 | | - |
13 | | -## Protocol Limitations |
14 | | - |
15 | | -The following features are not represented in the protocol. |
| 5 | +# Features of RPC protocol not supported by SQL protocol |
16 | 6 |
|
17 | 7 | ### Bind Variables |
| 8 | +The RPC protocol supports bind variables which allows Vitess to cache query plans providing much better execution times. |
18 | 9 |
|
19 | | -In the traditional connector, queries are sent as plain text to the server. This |
20 | | -protocol does not support bind variable. |
21 | | - |
22 | | -The Prepared Statement part of the API can support bind variables, but it |
23 | | -requires per-connection state, and using binary bind variables, which are much |
24 | | -harder to implement. It is also not recommended. |
25 | | - |
26 | | -Most database drivers (JDBC, go, ...) support bind variables. These end up being |
27 | | -implemented as client-side bind variables, where the values are printed in the |
28 | | -SQL statement by the client, and re-interpreted by the server. |
29 | | - |
30 | | -A Vitess Connector, on the other end, can send the Bind Variable map to the |
31 | | -server along with the query. The query plan is then cached by both vtgate and |
32 | | -vttablet, providing much better execution times. |
33 | | - |
34 | | -Note we added the `normalize_queries` command line parameter to vtgate to |
35 | | -mitigate this problem. With this flag, vtgate will try to extract bind variables |
36 | | -from full queries. This makes the vttablet side optimized, but still costs extra |
37 | | -CPU on the vtgate side. |
38 | | - |
39 | | -### Tablet Type |
40 | | - |
41 | | -The regular Vitess API provides the ability to specify the tablet type to |
42 | | -target: `master`, `replica`, `rdonly`. The current MySQL connector we created |
43 | | -only uses the `master` type. |
44 | | - |
45 | | -Note we could implement a different policy for this. |
46 | | - |
47 | | -### Transaction Type |
48 | | - |
49 | | -The regular Vitess API provides the ability to specify the transaction type: one |
50 | | -shard only, or 2PC. The current MYSQL connector uses the transaction type |
51 | | -provided to vtgate by the `transaction_mode`, which usually is the highest |
52 | | -transaction level allowed by vtgate, not the default one. |
53 | | - |
54 | | -### Streaming Queries |
55 | | - |
56 | | -The Vitess RPC protocol supports both non-streaming queries (for web-like |
57 | | -traffic), and streaming queries (for data analysis traffic). The current MySQL |
58 | | -connector only uses non-streaming queries. |
59 | | - |
60 | | -This could be changed with a flag. Or we could try to be smart: anything within |
61 | | -a transaction could be non-streaming query, anything outside a transaction could |
62 | | -be streaming. |
63 | | - |
64 | | -### Extra Query Features |
65 | | - |
66 | | -There are even more Vitess specific features in the API, that are not |
67 | | -represented in the MySQL server API. Event Tokens for cache invalidation, Update |
68 | | -Stream, Messages, ... These seem somewhat impossible to implement using the |
69 | | -MySQL binary protocol. |
| 10 | +### Event Tokens |
| 11 | +The RPC protocols allows you to use event tokens to get the latest binlog position. These can be used for cache invalidation. |
70 | 12 |
|
71 | | -### Security |
72 | | - |
73 | | -The current RPC protocol uses the security provided by the RPC protocol, TLS for |
74 | | -gRPC. We then use the certificate name as the user name for our authorization |
75 | | -(table ACLs). |
76 | | - |
77 | | -The MySQL server connector requires user names and passwords, that need to |
78 | | -maintained, rotated, ... We do not include TLS support yet, although it can be |
79 | | -easily added. We also need to add the username as authenticated user to use for |
80 | | -our authorization. |
81 | | - |
82 | | -We have plans to provide an LDAP plug-in to authenticate the users (but no firm |
83 | | -development plan for it yet). |
| 13 | +### Update Stream |
| 14 | +Update stream allows you to subscribe to changing rows. |
84 | 15 |
|
85 | 16 | ### Query Multiplexing |
86 | | - |
87 | | -gRPC can multiplex multiple request / responses on the same TCP connection. The |
88 | | -MySQL server protocol is very synchronous, and can only have one request in |
89 | | -flight at any given time. |
90 | | - |
91 | | -## Recommended Use Cases |
92 | | - |
93 | | -With all these limitations, why did we even bother implementing this? Well, |
94 | | -there is something to be said for drop-in replacement and ease of use: |
95 | | - |
96 | | -* When migrating an existing application to Vitess, it is useful to run Vitess |
97 | | - on top of an existing database, and only change the server address in the |
98 | | - client code to point at vtgate, and not change anything else. This allows for |
99 | | - an easier transition, and then the client code can be update later to use a |
100 | | - better connector. |
101 | | - |
102 | | -* Some tools only support MySQL server protocol, and not connectors / not Vitess |
103 | | - (yet!). |
104 | | - |
105 | | -* Most of the mentioned limitations are going to affect production systems |
106 | | - running at a somewhat high load. For smaller workloads, none of this really |
107 | | - matters. |
108 | | - |
109 | | -* A lot more programming languages have MySQL connectors than what we support |
110 | | - already (we have connectors for Java, Python, PHP, Go). They can use the MySQL |
111 | | - server protocol until a native connector for gRPC can be immplemented. |
112 | | - |
113 | | -## Future Features |
114 | | - |
115 | | -We are thinking about the following features to make this connector more useful: |
116 | | - |
117 | | -* Add SSL support. |
118 | | - |
119 | | -* Provided more authentication schemes than `mysql_native_password`. In |
120 | | - particular, `sha256` seems easy. |
121 | | - |
122 | | -* Use an LDAP client to validate users and passwords on the server side. |
123 | | - |
124 | | -* Implement a lot more DBA features in the protocol. Statements like `SHOW |
125 | | - TABLES` are not supported yet. They will make the MySQL binary protocol much |
126 | | - more useful. |
127 | | - |
128 | | -* Possibly add hints to the queries so they can use more advanced features. Like |
129 | | - provide a `commit 2pc` syntax to enable 2PC for some commits. |
130 | | - |
131 | | -* Provide direct SQL access for some features like Vitess Messages. |
132 | | - |
| 17 | +Ability to multiplex multiple request/responses on the same TCP connection. |
0 commit comments