Skip to content

Commit 83c22bc

Browse files
authored
Merge pull request vitessio#3853 from jvaidya/fix_binary_proto_doc
MySQL binary protocol is done - fix doc to reflect this
2 parents 210495b + e50cded commit 83c22bc

File tree

1 file changed

+8
-123
lines changed

1 file changed

+8
-123
lines changed

doc/MySQLServerProtocol.md

Lines changed: 8 additions & 123 deletions
Original file line numberDiff line numberDiff line change
@@ -1,132 +1,17 @@
11
# MySQL Binary Protocol
22

3-
Vitess 2.1 is adding alpha support for the MySQL binary protocol (refered to as the
4-
`protocol` in the rest of this document). This allows existing applications to
5-
connect to Vitess directly without any change, or new driver / connector.
3+
Vitess supports MySQL binary protocol. This allows existing applications to connect to Vitess directly without any change, or without using a new driver or connector. This is now the recommended and the most popular protocol for connecting to Vitess.
64

7-
However, this also has limitations. The RPC protocol traditionally exposed by
8-
Vitess is richer in features, and allows finer grain control of the query
9-
behaviours.
10-
11-
This document explores the limitations of using this protocol.
12-
13-
## Protocol Limitations
14-
15-
The following features are not represented in the protocol.
5+
# Features of RPC protocol not supported by SQL protocol
166

177
### Bind Variables
8+
The RPC protocol supports bind variables which allows Vitess to cache query plans providing much better execution times.
189

19-
In the traditional connector, queries are sent as plain text to the server. This
20-
protocol does not support bind variable.
21-
22-
The Prepared Statement part of the API can support bind variables, but it
23-
requires per-connection state, and using binary bind variables, which are much
24-
harder to implement. It is also not recommended.
25-
26-
Most database drivers (JDBC, go, ...) support bind variables. These end up being
27-
implemented as client-side bind variables, where the values are printed in the
28-
SQL statement by the client, and re-interpreted by the server.
29-
30-
A Vitess Connector, on the other end, can send the Bind Variable map to the
31-
server along with the query. The query plan is then cached by both vtgate and
32-
vttablet, providing much better execution times.
33-
34-
Note we added the `normalize_queries` command line parameter to vtgate to
35-
mitigate this problem. With this flag, vtgate will try to extract bind variables
36-
from full queries. This makes the vttablet side optimized, but still costs extra
37-
CPU on the vtgate side.
38-
39-
### Tablet Type
40-
41-
The regular Vitess API provides the ability to specify the tablet type to
42-
target: `master`, `replica`, `rdonly`. The current MySQL connector we created
43-
only uses the `master` type.
44-
45-
Note we could implement a different policy for this.
46-
47-
### Transaction Type
48-
49-
The regular Vitess API provides the ability to specify the transaction type: one
50-
shard only, or 2PC. The current MYSQL connector uses the transaction type
51-
provided to vtgate by the `transaction_mode`, which usually is the highest
52-
transaction level allowed by vtgate, not the default one.
53-
54-
### Streaming Queries
55-
56-
The Vitess RPC protocol supports both non-streaming queries (for web-like
57-
traffic), and streaming queries (for data analysis traffic). The current MySQL
58-
connector only uses non-streaming queries.
59-
60-
This could be changed with a flag. Or we could try to be smart: anything within
61-
a transaction could be non-streaming query, anything outside a transaction could
62-
be streaming.
63-
64-
### Extra Query Features
65-
66-
There are even more Vitess specific features in the API, that are not
67-
represented in the MySQL server API. Event Tokens for cache invalidation, Update
68-
Stream, Messages, ... These seem somewhat impossible to implement using the
69-
MySQL binary protocol.
10+
### Event Tokens
11+
The RPC protocols allows you to use event tokens to get the latest binlog position. These can be used for cache invalidation.
7012

71-
### Security
72-
73-
The current RPC protocol uses the security provided by the RPC protocol, TLS for
74-
gRPC. We then use the certificate name as the user name for our authorization
75-
(table ACLs).
76-
77-
The MySQL server connector requires user names and passwords, that need to
78-
maintained, rotated, ... We do not include TLS support yet, although it can be
79-
easily added. We also need to add the username as authenticated user to use for
80-
our authorization.
81-
82-
We have plans to provide an LDAP plug-in to authenticate the users (but no firm
83-
development plan for it yet).
13+
### Update Stream
14+
Update stream allows you to subscribe to changing rows.
8415

8516
### Query Multiplexing
86-
87-
gRPC can multiplex multiple request / responses on the same TCP connection. The
88-
MySQL server protocol is very synchronous, and can only have one request in
89-
flight at any given time.
90-
91-
## Recommended Use Cases
92-
93-
With all these limitations, why did we even bother implementing this? Well,
94-
there is something to be said for drop-in replacement and ease of use:
95-
96-
* When migrating an existing application to Vitess, it is useful to run Vitess
97-
on top of an existing database, and only change the server address in the
98-
client code to point at vtgate, and not change anything else. This allows for
99-
an easier transition, and then the client code can be update later to use a
100-
better connector.
101-
102-
* Some tools only support MySQL server protocol, and not connectors / not Vitess
103-
(yet!).
104-
105-
* Most of the mentioned limitations are going to affect production systems
106-
running at a somewhat high load. For smaller workloads, none of this really
107-
matters.
108-
109-
* A lot more programming languages have MySQL connectors than what we support
110-
already (we have connectors for Java, Python, PHP, Go). They can use the MySQL
111-
server protocol until a native connector for gRPC can be immplemented.
112-
113-
## Future Features
114-
115-
We are thinking about the following features to make this connector more useful:
116-
117-
* Add SSL support.
118-
119-
* Provided more authentication schemes than `mysql_native_password`. In
120-
particular, `sha256` seems easy.
121-
122-
* Use an LDAP client to validate users and passwords on the server side.
123-
124-
* Implement a lot more DBA features in the protocol. Statements like `SHOW
125-
TABLES` are not supported yet. They will make the MySQL binary protocol much
126-
more useful.
127-
128-
* Possibly add hints to the queries so they can use more advanced features. Like
129-
provide a `commit 2pc` syntax to enable 2PC for some commits.
130-
131-
* Provide direct SQL access for some features like Vitess Messages.
132-
17+
Ability to multiplex multiple request/responses on the same TCP connection.

0 commit comments

Comments
 (0)