Professional Documents
Culture Documents
(Draft v14)
Page 1 of 28
Table of Contents
Overview..................................................................................................................................... 4
Introduction................................................................................................................................. 5
ZKProto Servers........................................................................................................................... 7
ZKProto Sync Basic Operation............................................................................................. 7
ZKProto Sync Client Basic Operation Flow..........................................................................7
ZKProto Sync Server Basic Operation Flow.........................................................................8
ZKProtoSyncOpenInfo................................................................................................. 8
ZKProtoSyncAckInfo.................................................................................................... 9
ZKProtoSyncOperations.............................................................................................. 9
ZKProtoSyncOperation................................................................................................ 9
ZKProto Control Basic Operation....................................................................................... 11
ZKProto Control Client Basic Operation Flow.....................................................................11
ZKProto Control Server Basic Operation Flow...................................................................11
ZKProtoControlAction................................................................................................ 11
Compression............................................................................................................................. 13
Encryption................................................................................................................................. 14
ZKTeco's ZKProto Server........................................................................................................... 15
Setting up......................................................................................................................... 15
Server rules....................................................................................................................... 15
Open......................................................................................................................... 15
Pull............................................................................................................................ 15
Push.......................................................................................................................... 16
Control server supported features.................................................................................... 16
Thrid-Party Development using ZKProto................................................................................... 17
Using ZKTeco's ZKProto Server......................................................................................... 17
Simple sync client tutorial......................................................................................... 17
Appendix A: Apache Thrift......................................................................................................... 24
Appendix B: FAQ (Frequently Asked Questions)........................................................................25
What is ZKProto?............................................................................................................... 25
What is Apache Thrift?...................................................................................................... 25
What transport protocol does ZKProto use?......................................................................25
What DB engine should I use to make a ZKProto server/client implementation?..............25
What language can I use to make a ZKProto server/client implementation?....................25
What tools do I need to make a ZKProto server/client implementation?...........................25
Do I need any external library to make a ZKProto server/client?......................................25
Do I need to write my own server?................................................................................... 25
How do I add ZKProto support to my existing software?...................................................25
How to implement a client that synchronizes into several zones?...................................26
Does ZKProto support compression?................................................................................. 26
Does ZKProto support encryption?.................................................................................... 26
What happens to local data if I change a client's zone?....................................................26
Why there's a sync server and a control server?..............................................................26
Why are most table IDs very big and sometimes also negative?......................................26
What does server message Push called from non-replicated client, ignoring... means? 26
Should my client open and close the socket after each pull call or keep it open?............26
Why control server password is hashed using BCrypt and not a more common hash
algorithm like MD5/SHA1?................................................................................................. 27
Page 2 of 28
Appendix C: Authors.................................................................................................................. 28
Page 3 of 28
Overview
ZKProto aims to provide a modern, fast and reliable server-centralized protocol for DB synchronization
between clients (with/from ZKTeco devices that support it, or for other purposes too). A quick view of ZKProto's
characteristics.
Pros
Cons
It allows normal work even when remote connection is not Requires data duplication in all clients and server.
available.
Read-only queries are performed locally, avoiding DB engine Cannot read any data without getting replicated first.
workload and network-bandwidth consumption.
Database abstraction: each client can freely decide which DB
backend to use.
Automatic almost real-time database synchronization.
Support for several network-level protocols: raw TCP, HTTP,
JSON...
Support for several programming languages in both client and
server implementations.
Secure data transmission: RSA+AES encryption
Versioning: clients/server running different protocol versions
can work together.
No external libraries dependency (this is generally true, but for
some target languages some libraries might be required, e.g.
Java and SLF4J).
Multi-platform support (Windows, Linux, Mac, UNIX...).
Page 4 of 28
Introduction
ZKProto is a reliable, fast, modern, secure and flexible communication protocol based on the concepts of
middleware and RPC (Remote Procedure Call). The middleware (in case of ZKProto we use Apache Thrift) takes care
of all the raw communication aspects, like sockets and everything related to TCP/IP stack, so we can focus on writing
the actual database synchronization and needed features. The RPC part refers to the ability to call remote
procedures/methods/functions like they were local.
ZKProto is a centralized protocol as shown in the above diagram, which means the clients never actually
connect between themselves but all the synchronization is done through the server. The sole and unique purpose of
the server is to keep all clients synchronized and handle any data conflicts that might arise. As an abstraction, you can
think of the synchronization prodecure like all clients are actually using the same database.
NOTE: the server DB is private should never be accessed directly from any source that is not the ZKProto server itself.
ZKProto is not tied to any database structure and thus can be used over any database design, structure and
engine. ZKProto does not make any assumptions about the structure/data of the database being synchronized. It thus
can be used to synchronize clients with totally different DB engines. In this schema, clients are responsible to update
their DB as they see fit.
ZKProto server supports zones. This means the server can be configured to group clients so these clients only
synchronize data between the members of their group and not with other clients. Each group is called a zone in
ZKProto terminology. For example, if clients A and B belong to zone 1, and clients C and D belong to zone 2, changes
in client A will only be reflected in client B, not C and D, even if all clients A, B, C, and D are connected to the same
server and on the same network.
Page 5 of 28
ZKProto supports writing clients and servers in several programming languages like Java, C++, Python, PHP,
Erlang and many others. Clients and servers can be written in totally different languages. Even each client can be in a
different language. For example, client A could be coded in C++ while client B is in Java; and server in PHP. Being
language-agnostic also means platform-agnostic, as ZKProto client/server will work as long as platform X supports the
language in which it was built. While ZKProto base code is platform-agnostic, the developer must take care of writing
portable code if this is needed.
ZKProto supports any transport protocol that can run over TCP, which includes raw binary, HTTP, HTTPS,
JSON and several others, even custom ones.
Both ZKProto client and server do not depend on any external library to work. A special tool (the Apache Thrift
compiler, detailed in Appendix A) will generate the appropriate source code in the targeted programming language to
include in your project and have access to ZKProto functionality. Also ZKProto does not depend on any platformspecific external library (DLL, SO, etc...).
ZKProto supports different version in client and server. This means ZKProto will work even if client and server
have different versions of the protocol. Even if so, it is highly recommended to have at least latest server version (that
is, all your clients should have equal or previous ZKProto version than your server).
ZKProto supports AES encryption for secure data communication. AES is a standard encryption algorithm
(ISO/IEC 18033-3) that was designed to be very secure and fast. Currently ZKProto only supports 256-bit key size.
Support for other key sizes is planned for future releases. Please make sure your country laws allow the usage of AES
256 before activating it.
Page 6 of 28
ZKProto Servers
ZKProto defines 2 types of server: DB-synchronization server (from now on referred to as sync server) and
server-control server (from now on referred to as control server).
Control server: provides information about and allows configuring sync server (i.e. currently connected clients).
Clients are always the ones to initiate connection with the server.
open First call for a client when it opens a new connection. Sends information about client (i.e. client ID). You
need to call this once per connection and before any call to pull/push.
push Called from client when it has new DB modifications to be synchronized to the server, or to inform the
server it correctly received and processed sent DB modifications (after a pull).
pull Called from client when it wants to retreive any DB modifications that happened. Server can also request
client to perform a push through this call.
All calls are blocking (they do not return until processing is finished). If any error happens, the server will throw
a remote exception that extends TException (this is a Thrift internal exception). If call returns without exception, it was
successful. If an exception is thrown/caught, then the call has failed for the reasons explained in the exception specific
subclass and in the exception message.
NOTE: there's no multithreading support for the same client. That is, you cannot call the RPCs concurrently, only
sequentially.
open to start communication with server. Client sends some information about itself to help prepare the server
to communicate with it. Server can reject the open call with a ZKProtoException for various reasons that are
server-side implementation-dependent.
2.
pull operations, store them if necessary and execute them if any. Keep in mind that a pull can be multi-part. In
this case the client needs to keep pulling until there no more operations before doing any other call (namely
push).
3.
push if server orders device to push (through PUSH operation), if client decides it has new data to push, or to
acknowledge result of fetched operations to the server, or all of the previous. Note that with ACKNOWLEDGE
action you tell the server if pulled operations executed successfully or not (see ZKProtoOperation).
4.
If you want to close the socket now, you can do it. Otherwise go to step 2 again (for suggestions about pros
and cons for keeping socket open or closing it, see the FAQ).
In case opening communication with the server fails, the client should retry connecting after X milliseconds. In
case the communication suddenly drops, client will restart communication from step 1.
Page 7 of 28
How to implement this depends on target programming language and target data storage.
ZKProto server starts listening on configured port (defaults to 4372). The server must accept multiple incoming
connections in this same port.
2.
Once a device connects, it will call open. What the server does with this information is implementationdependent, as well as what to require from a client (for example, a server can require clients to have the clock
correctly set up).
3.
The client calls pull. We check if we have any pending operations for this device and send them. How to check
pending operations is implementation-dependent.
4.
The client calls push to send new data to the server or to confirm received data. Whether server accepts or not
a push call from a client and how it informs the client about this last situation is implementation-dependent.
ZKProtoSyncOpenInfo
ZKProtoOpenInfo is client information sent to server just after connecting (before any pull/push call). This
information must be sent each time the client re-connects to server as well.
// Synchronization information
struct ZKProtoSyncOpenInfo {
1: required i32 protocolVersion,
// ZKProto version
2: required i64 clientId,
// Client UUID
3: required string customId,
// Customer ID
4: optional i32 maxOperations,
// Max size of ZKProtoOperations (in bytes)
5: optional string publicKey,
// Public key for encryption start
6: required i64 timestamp,
// Current client UTC time
7: optional ZKProtoSyncTableFilter tableFilter, // Table filters if any
}
protocolVersion client ZKProto version implementation. As of the writing this document, this value has to be
1.
clientId client UUID. This is the unique identifier for this client. To avoid possible UUID collisions we suggest
you generate this number randomly or use a value known to be almost unique (e.g. CPU ID).
customId client customized ID. This string can be anything and it is not processed by ZKProto. It's for
Page 8 of 28
maxOperations maximum size in bytes for operations as answer to pull call. This defines maximum data a
pull will return. If not set, there will be virtually no limit. Recommended to set it in low-memory clients (e.g.
embedded devices).
publicKey base16-encoded RSA public key to encrypt AES key with (see Encryption section). If not set, there
will be no encryption.
timestamp client UTC time as milliseconds since January 1st 1970 00:00. How the server handles this field is
implementation-dependent.
tableFilter a data structure that defines about which tables the server should send this client updates. It can
be a white or a black list. If not set, server will synchronize all tables in this client's zone.
ZKProtoSyncAckInfo
ZKProtoSyncAckInfo is the data structure returned from open call.
// Structure returned from open() call
struct ZKProtoSyncAckInfo {
1: optional string encryptedKey,
}
encryptedKey base 16-encoded encrypted AES key to be used to encrypt data. This key is encrypted with
RSA public key sent by client in open call. This key will only encrypt data field of the ZKProtoOperation
structure. Note that if data is encrypted and compressed, it must be first compressed then encrypted (and the
opposite when receiving, that is first decrypted then decompressed).
ZKProtoSyncOperations
ZKProtoSyncOperations is a ZKProtoOperation list wrapper.
// Wrapper for a list of operations
struct ZKProtoSyncOperations {
1: required list<ZKProtoSyncOperation> operations,
2: required bool last,
}
// List of operations
// Is this last part of a multipart?
last if true, server still has operations pending to be pulled by this client, which requires the client to pull
again; if false, no more operations are pending.
ZKProtoSyncOperation
ZKProtoOperation represents one operation to be performed in the client (it can be a DB modification, but not
Page 9 of 28
//
//
//
//
//
id This operation universal 64-bit identifier. This ID should never repeat. Can be ommitted or set to -1 if the
operation does not need to be identified or it's not a real operation.
action ZKProtoAction that defines which action this operation is describing. Current possible actions are:
NOP No Operation, does nothing.
INSERT Informs of new data in DB.
UPDATE Informs of data update in DB.
DELETE Informs of data removal in DB.
PUSH Informs client it has to push any pending operations it has.
ACKNOWLEDGE Informs server about status of client after executing operations from a pull.
timestamp When this operation was perfomed (UTC as milliseconds since January 1st 1970 00:00)
data String that represents the data for the operation. The format of this data changes depending on the
operation action.
UPDATE
table=tableName;pk=field1=value1,[...]fieldN=valueN,;
field1=newField1Value;
[...]fieldN=newFieldNValue; where tableName is the name of the table to update; fieldN=valueN are
update conditions; fieldN is the field name; newFieldNValue is the new value to set to fieldN. Example:
table=entity;pk=_id=1245124;enabled=1 is equivalent to SQL sentence UPDATE entity SET enabled
= 1 WHERE _id = 1245124.
compressed Indicates if data field is compressed (see Compression section). Note that if data is also
encrypted, compression is done before encryption (otherwise the compression would be useless, see this
discussion about why).
Page 10 of 28
For a complete ZKProto client example in Java, you can check com.zktechnology.simplezkprotoclient.zkproto.
ZKProtoClient source code.
open - First call for a client when it opens a new connection. Sends information about client.
execute Executes one given control operation and returns the result if any.
2.
execute to perform a control operation (i.e. change client zone) or to update current server status (get sync
client statuses, get existing zones, etc...).
ZKProto server starts listening on configured port (defaults to 4373). Each control client will just connect once
to this port. Of course the server accepts multiple clients in this port, as with the sync server.
2.
Once a device connects, it will call open. Here we initialize information about the client.
3.
The client eventually calls execute. Server checks the requested action, performs it and returns data only if
action requires data back. If action does not return data, then empty result means OK. If anything has gone
wrong, a ZKProtoException will be thrown.
ZKTeco already provides a ZKProto server, so you usually do not have to implement this part unless you want
a specific server.
ZKProtoControlAction
ZKProtoControlAction is a Thrift enumeration that defines control functionality offered by the control server and
possible actions for control clients.
// Operation to be performed
struct ZKProtoControlOperation {
1: required ZKProtoControlTarget target,
2: required ZKProtoControlAction action,
3: optional binary data,
4: optional bool compressed,
}
//
//
//
//
Control target
Action performed in operation
Data for the action (if any)
Flag if data is compressed
target target this action will affect. Each target only accepts a specific subset of actions. Current possible
targets are:
Page 11 of 28
action which action to execute for specified target. Current possible actions are:
NOP no action, used to include plain data, for example an answer to a data petition.
GET_ALL_CLIENTS valid target: CLIENT. No parameters. Retreives all known clients in server
Page 12 of 28
Compression
Compression is made using ZLIB compression algorithm and it is only applied to ZKProtoOperation.data string
field/member. Note that compressed data has no ZLIB header. Most modern programming languages already include
classes to handle ZLIB, and older ones have libraries for it.
Page 13 of 28
Encryption
Encryption schema uses 2 different standard encryption algorithms: RSA and AES. These two algorithms are
today's standard algorithms for assymmetric and symmetric encryption respectively. Exact (hash and padding) RSA
and AES algorithms being used are RSA/ECB/PKCS1Padding and AES/ECB/PCKCS5Padding respectively. RSA is
used to share the AES key used to actually encrypt data. The process flow is as follows:
1.
Client generates an RSA key pair. We will call the public key CA and the private key CB. On opening
communication with server, it sends CA to the server (see ZKProtoOpenInfo data structure).
2.
Once a client connects, server generates an AES key to use for this client. We will call this key SK. The client
CA key is used to encrypt SK and send back to client (see ZKProtoAckInfo data structure).
3.
Client uses CB to decrypt SK from server. Now both sides have the SK key and can use this key for secure
communciation.
Note that if compression and encryption are both used, you must first compress then encrypt, otherwise the
compression will have small to no effect (due to the high entropy from AES encryption).
Please make sure your country laws allow the usage of AES encryption and under which restrictions.
Page 14 of 28
Setting up
You need an up and working PostgreSQL 9.1+ server, configured in the default port (5432) and accessable
with the user postgres and password postgres.
Server rules
In this section we will explain ZKTeco's ZKProto Server internal logic. Please note that this is merely
informative. Any ZKProto client implementation should just execute the commands received by the server.
Open
1.
If this client does not exist, create an entry for it to log that this client tried to open.
2.
If this client already called open on this same socket connection, throw an exception.
3.
Check if client ZKProto version is equal or lower than server's. If not, throw an exception.
4.
5.
6.
Assign this client to its zone. If no zone assigned, assign to default zone (limbo).
Pull
1.
If this client didn't call open on this same socket connection, throw an exception.
2.
If this client has been replicated, go to next step. Otherwise prepare to send this client initialization operations
(see Policy below) and clear any pending operations this client might have. Go to 4.
3.
If this client has any pending operations, go to next step. Otherwise check if any new pending changes to send
and put them as pending to send.
4.
There are 2 different policies for initializing a client: RESET and PUSH. RESET policy will replicate server DB
to client (thus client will lose any data it has previous to this). PUSH policy will first instruct the client client to push its
data first before getting replicated from server DB (thus no data is lost from this client). PUSH policy is much slower
Page 15 of 28
and consumes much more bandwidth and traffic than RESET policy.
Push
1.
If this client didn't call open on this same socket connection, throw an exception.
2.
If zone has been changed, discard any operations from this client.
3.
If operation is DB operation (INSERT, UPDATE, DELETE) and client is replicated, execute it. If client is not
replicated, throw an exception. If there's an error executing the DB operation, server can determine if this
error is harmful to data integrity or not: it can either throw an exception and stop processing, or insert it in a
conflict table.
2.
If operation is an ACKNOWLEDGE, see if it has failed or succeeded. If failed, set client as not-replicated
(see consequences of this on Pull). If success, mark as replicated and clear pending operations.
For a correct working of the protocol, please only send ACKNOWLEDGE with no failed operations from your
client when previous pull operations were successfully executed. If any error happened, immediately abort operation
execution and inform the server through ACKNOWLEDGE.
Get/set the following control server settings: TCP port, debug logs, authentication, and password. Password is
hashed using secure BCrypt hashing over optional -but recommended- AES encryption communication.
Get/set the following sync server settings: TCP port, and debug logging.
Get all existing sync zones, and get all clients associated to a given zone.
Create a new zone, edit and remove an existing one. Zone names are restricted to lowercase english letters,
numbers and underscore, and cannot be start with number. There's no limit to number of zones. You can only
remove a zone with no clients associated.
Create a new client, edit and remove an existing one. Client names have no restrictions. When creating a
client, you can specify its UUID, its code and its description. When editing a client, you can edit its description,
its authorization status, and its policy (see next point).
Page 16 of 28
Ubuntu 12.04, although we won't be using any Linux-specific tools/methods, so the instructions can be applied
to any other OS (Windows, Mac...).
Java 7 or above installed (we will use OpenJDK 1.7, you're free to use your favorite Java implementation).
Build the Apache Thrift compiler (see Appendix A on how to build it). This step will automatically generate a
Java library: libthrift-0.9.1.jar.
ZKProto Apache Thrift Interface file for sync server: zkprotosync.thrift (should be included in the ZKProto
SDK).
Now that we have the basic tools, let's start with the implementation itself:
Page 17 of 28
1.
Make sure your Java JDK is properly installed. Open a console and run java -version.
2.
Create a directory called zkproto tutorial wherever you feel like. We will be using it as our operations HQ.
3.
Extract Eclipse 4.3. Run eclipse executable. It will ask you for a workspace location, we create a new directory
under zkproto tutorial called zkproto workspace to use as Eclipse workspace.
4.
Close the Welcome window and you'll be presented with Eclipse's default workspace layout.
Page 18 of 28
5.
Create a new Java project: File > New > Java Project. You will be presented with the following window, please
fill as indicated and click Finish.
6.
The project should appear in the Package Explorer window. Create a new folder libs inside the project. Rightclick on the project: New > Folder and name it libs. Copy libthrift-0.9.1.jar to this folder. Include the library into
the project build path: right-click on the project > Build Path > Configure Build Path... > Libraries > Add JARs...
and select libthrift-0.9.1.jar. Close OK all windows. You also need to include the SL4FJ logger library (which is
almost-standard Java library for logging) with the same procedure as you did with libthrift-0.9.1.jar. You can
download it from here (the exact JAR to include in the build path is slf4j-api-X.X.X.jar (X.X.X being the current
version number). This library requirement is exclusively for Java.
7.
Copy zkprotosync.thrift to zkproto tutorial directory. In a console, move (cd) to zkproto tutorial directory and
generate the Java classes and library by executing thrift -gen java zkprotosync.thrift. This will create a
directory called gen-java where you can find the Java classes.
8.
Expand the project and right-click on src > New > Package. Name it com.zktechnology.zkproto and click
Finish. Copy the Thrfit-generated Java classes to this package. You can copy them in the your OS file explorer
and paste in Eclipse (right-click on the created package and paste).
Page 19 of 28
9.
Now we have everything ready to start implementing our ZKProto client. To start, we create a new class called
ZKProtoClient: right-click on the package > New > Class and name it ZKProtoClient. Click Finish.
10. We need an entry point for our example program, so we create a main static method in ZKProtoClient.
package com.zktechnology.zkproto;
public class ZKProtoClient {
public static void main (final String[] args) {
}
}
11. First thing to do is to open a TCP connection to the ZKProto server (assuming server is running on the same
machine on default port 4372), which is as simple as follows:
final TTransport transport = new TSocket("localhost", 4372);
transport.open();
12. Now we need to identify our client to the server. For this, we must first create a ZKProto sync client based on
our previous TCP socket, then identify to the sync server using open. This step can be implemented as
follows:
final Client client = new ZKProtoSyncService.Client(
new TBinaryProtocol(transport));
final int protocolVersion = 1;
final long clientUUID = 1;
final ZKProtoSyncOpenInfo openInfo = new ZKProtoSyncOpenInfo(
protocolVersion, clientUUID, "ZKProto test client",
System.currentTimeMillis());
client.open(openInfo);
13. Next, we want to pull, which is first thing a client should do after open. Since we already have a ZKProto sync
client instance, we will use it to pull. Since this is just an example, we will just output the result of this pull.
final ZKProtoSyncOperations operations = client.pull();
Page 20 of 28
System.out.println(operations);
14. After pull, we have to inform the server we received (and eventually executed) all operations successfully. For
this, we will push an ACKNOWLEDGE operation.
final ZKProtoSyncOperation ackOperation = new ZKProtoSyncOperation(-1,
ZKProtoSyncAction.ACKNOWLEDGE);
final List<ZKProtoSyncOperation> opsToPush = new ArrayList<>();
opsToPush.add(ackOperation);
client.push(opsToPush);
System.out.println("Ack success");
java.util.List;
java.util.ArrayList;
org.apache.thrift.TException;
org.apache.thrift.protocol.TBinaryProtocol;
org.apache.thrift.transport.TSocket;
org.apache.thrift.transport.TTransport;
com.zktechnology.zkproto.ZKProtoSyncService.Client;
Page 21 of 28
Note that if you execute this client against a ZKProto server for the first time, it will raise a NOT_AUTHORIZED
exception:
Exception in
at
at
at
at
at
at
at
This is absolutely normal because this client is not authorized and when calling open, the server will refuse to
complete this call. After authorizing the client with RESET policy, the output of this client example should be as
following:
Connection with server established
Opening
Open success
Pulled operations successfully
ZKProtoSyncOperations(operations:[ZKProtoSyncOperation(id:-1, action:DELETE, timestamp:0, data:74 61 62 6C 65
3D 65 6D 70 6C 6F 79 65 65 5F 6C 6F 67 69 6E 5F 63 6F 6D 62 69 6E 61 74 69 6F 6E), ZKProtoSyncOperation(id:1, action:INSERT, data:74 61 62 6C 65 3D 65 6D 70 6C 6F 79 65 65 5F 6C 6F 67 69 6E 5F 63 6F 6D 62 69 6E 61 74
69 6F 6E 3B 6C 6F 67 69 6E 5F 63 6F 6D 62 69 6E 61 74 69 6F 6E 3D 33 3B 69 64 5F 65 6D 70 6C 6F 79 65 65 3D
31 3B), ZKProtoSyncOperation(id:-1, action:DELETE, timestamp:0, data:74 61 62 6C 65 3D 72 6F 6C 65 32 65 6E
74 69 74 79), ZKProtoSyncOperation(id:-1, action:INSERT, data:74 61 62 6C 65 3D 72 6F 6C 65 32 65 6E 74 69 74
79 3B 69 64 5F 72 6F 6C 65 3D 31 3B 69 64 5F 65 6E 74 69 74 79 3D 31 3B), ZKProtoSyncOperation(id:-1,
action:INSERT, data:74 61 62 6C 65 3D 72 6F 6C 65 32 65 6E 74 69 74 79 3B 69 64 5F 72 6F 6C 65 3D 32 3B 69 64
5F 65 6E 74 69 74 79 3D 31 3B), ZKProtoSyncOperation(id:-1, action:INSERT, data:74 61 62 6C 65 3D 72 6F 6C 65
32 65 6E 74 69 74 79 3B 69 64 5F 72 6F 6C 65 3D 33 3B 69 64 5F 65 6E 74 69 74 79 3D 31 3B),
ZKProtoSyncOperation(id:-1, action:INSERT, data:74 61 62 6C 65 3D 72 6F 6C 65 32 65 6E 74 69 74 79 3B 69 64
5F 72 6F 6C 65 3D 34 3B 69 64 5F 65 6E 74 69 74 79 3D 31 3B), ZKProtoSyncOperation(id:-1, action:DELETE,
timestamp:0, data:74 61 62 6C 65 3D 72 6F 6C 65), ZKProtoSyncOperation(id:-1, action:INSERT, data:74 61 62 6C
65 3D 72 6F 6C 65 3B 6E 61 6D 65 3D 53 75 70 65 72 20 41 64 6D 69 6E 3B 5F 69 64 3D 31 3B),
ZKProtoSyncOperation(id:-1, action:INSERT, data:74 61 62 6C 65 3D 72 6F 6C 65 3B 6E 61 6D 65 3D 45 6E 72 6F
6C 6C 65 72 3B 5F 69 64 3D 32 3B), [cut for brievity...]], last:true)
Ack success
Finished successfully
Note that obviously this is not a real working client because operations are not actually executed. How to
execute the operations is highly dependent on underlying DB, any ORM layers, language, and other client
characteristics. We think that with explanations given in ZKProtoSyncOperation section it should be relatively
simple to execute any operation received from the server.
The server output after running with this sample client should be as following:
Wed May 28 10:47:53 CEST
Wed May 28 10:47:53 CEST
Wed May 28 10:47:53 CEST
Wed May 28 10:47:53 CEST
Wed May 28 10:47:56 CEST
Wed May 28 10:47:56 CEST
127.0.0.1:43963, ZKProto
Wed May 28 10:47:56 CEST
Wed May 28 10:47:56 CEST
Wed May 28 10:47:56 CEST
Wed May 28 10:47:58 CEST
Wed May 28 10:47:58 CEST
Wed May 28 10:47:58 CEST
Wed May 28 10:47:58 CEST
Wed May 28 10:47:58 CEST
Wed May 28 10:47:58 CEST
Wed May 28 10:47:58 CEST
Wed May 28 10:47:58 CEST
2014 - >==================================================<
2014 - Starting ZKProto Server 0.16.1
2014 - Starting server ZKProtoSync on port 4372
2014 - Starting server ZKProtoControl on port 4373
2014 - SYNC >> ZKProto test client: open()
2014 - SYNC >> ZKProto test client: ZKProto test client (UUID: 1, IP:
version: 1)
2014 - SYNC >> ZKProto test client: Buffer capacity: 2147483647 bytes
2014 - SYNC >> ZKProto test client: Authorized with policy RESET
2014 - SYNC >> ZKProto test client: Policy: RESET
2014 - SYNC >> ZKProto test client: Zone: limbo
2014 - SYNC >> ZKProto test client: NOT replicated
2014 - SYNC >> ZKProto test client: Doesn't need encryption
2014 - SYNC >> ZKProto test client: pull()
2014 - SYNC >> ZKProto test client: Preparing RESET policy
2014 - SYNC >> ZKProto test client: Initialization op count: 161
2014 - SYNC >> ZKProto test client: Pull returning count: 161
2014 - SYNC >> ZKProto test client: Pull isLast: true
Page 22 of 28
CEST
CEST
CEST
CEST
2014
2014
2014
2014
SYNC
SYNC
SYNC
SYNC
>>
>>
>>
>>
ZKProto
ZKProto
ZKProto
ZKProto
test
test
test
test
client:
client:
client:
client:
push()
Pushed : 1
Processing index 0
Executing pushed operation: id: -1 | action:
CEST
CEST
CEST
CEST
2014
2014
2014
2014
Page 23 of 28
It has one of the best performances over Java in currently available middlewares.
Supports several programming languages (C++, Java, Python, PHP, Ruby, Erlang, Perl, Haskell, C#, Cocoa
(Objective-C), JavaScript, Node.js, Smalltalk, OCaml, Delphi, and many others).
Platform independent.
It was developped by ex-Google engineers for Facebook, and currently Facebook uses it as middleware.
Please note this is how to install Apache Thrift at the time of writing this document. For latest Thrift and latest
instructions, please visit Apache Thrift site. Also note that you need latest ZKProto Thrift interface definition (contact
current ZKProto maintainer for access to this file). Take into account that there are 2 of these files: one for sync
(zkserver.thrift), the other for control (zkservercontrol.thrift).
1.
Download latest Thrift (0.9.1 as of this writing) source code from here (I suggest first check latest version
here).
2.
Compile with necessary language support following these instructions. Depending on the target language
(Java, PHP, Ruby...), Thrift might generate an additional library to link in your project.
3.
Generate the classes/data structures for your target language by feeding the Thrift interface definition to the
thrift executable. For example:
thrift -gen target zkserver.thrift
where target is the target language. Supported targets (at the writing of this document) are: cpp (C++), csharp
(C#), d (D), delphi (Delphi), erl (Erlang), go (Go), hs (Haskell), java (Java), js (JavaScript), ocaml (OCaml), perl
(Perl), php (PHP), py.tornado (Python), py.twisted (Python), py (Python), rb (Ruby).
Example: thrift -gen java zkserver.thrift will generate required language classes for server/client.
4.
Use the generated classes (and library if any) to implement ZKProto server/client.
For more information about internal Apache Thrift workings, you can refer to the whitepaper (PDF).
Page 24 of 28
Page 25 of 28
Why are most table IDs very big and sometimes also negative?
Because operation IDs are 64-bit numbers generated randomly to avoid ID collisions in case clients are offline when
this row is created. The probability of a collision using this system is one in 1.84467440710 or
0.000000000000000001% for each table.
What does server message Push called from non-replicated client, ignoring... means?
This message means a client that is not replicated tried to push operations to the server. A non-replicated client is a
client has not been successfully initialized.
Should my client open and close the socket after each pull call or keep it open?
Each method has its pros and cons, you should decide which one is better suited to your context:
Open and close socket: costs more time and bandwidth due to to connection establishement: TCP handshake,
OS has to allocate resources for the socket, launch a new process/thread, call ZKProto open, etc....
Keeping socket open: costs memory and connections due to maintaining the socket resources. If socket is idle
it has no bandwidth or CPU cost, but the memory is still reserved.
Some scenarios and our suggestions:
Limited bandwidth: keep sockets open to avoid connection establishement overhead. An already-open TCP
connection has no bandwidth cost at all.
A lot clients: open and close sockets to free server memory and workload.
Client pulling interval is short: keep sockets open to avoid opening/closing too much in too short time.
Page 26 of 28
Why control server password is hashed using BCrypt and not a more common hash
algorithm like MD5/SHA1?
First, why not MD5/SHA256/SHA1, etc...? These are all general purpose hash functions, designed to calculate
a digest of huge amounts of data in as short a time as possible. This means they are fast, and that is precisely why
they're bad: someone who stole your password DB using a modern server can try every single possible MD5 password
hash (assuming passwords are lowercase, alphanumeric, and 6 characters long) in around 40 seconds. Now why
Bcrypt? Simply because its slow. It introduces a work factor, which allows you to determine how expensive the hash
function will be. Because of this, as computers get faster you can increase the work factor and the hash will get slower.
Page 27 of 28
Appendix C: Authors
Page 28 of 28