FlatBuffers 编码原理

FlatBuffers的Java库只提供了如下的4个类：

./com/google/flatbuffers/Constants.java
./com/google/flatbuffers/FlatBufferBuilder.java
./com/google/flatbuffers/Struct.java
./com/google/flatbuffers/Table.java

Constants 类定义FlatBuffers中可用的基本原始数据类型的长度：

public class Constants {
    // Java doesn't seem to have these.
    /** The number of bytes in an `byte`. */
    static final int SIZEOF_BYTE = 1;
    /** The number of bytes in a `short`. */
    static final int SIZEOF_SHORT = 2;
    /** The number of bytes in an `int`. */
    static final int SIZEOF_INT = 4;
    /** The number of bytes in an `float`. */
    static final int SIZEOF_FLOAT = 4;
    /** The number of bytes in an `long`. */
    static final int SIZEOF_LONG = 8;
    /** The number of bytes in an `double`. */
    static final int SIZEOF_DOUBLE = 8;
    /** The number of bytes in a file identifier. */
    static final int FILE_IDENTIFIER_LENGTH = 4;
}

FlatBufferBuilder 用于FlatBuffers编码，它会将我们的结构化数据序列化为字节数组。我们借助于 FlatBufferBuilder 在 ByteBuffer 中放置基本数据类型的数据、数组、字符串及对象。ByteBuffer 用于处理字节序，在序列化时，它将数据按适当的字节序进行序列化，在发序列化时，它将多个字节转换为适当的数据类型。在 .fbs 文件中定义的 table 和 struct，为它们生成的Java 类会继承 Table 和 Struct。

在反序列化时，输入的ByteBuffer数据被当作字节数组，Table提供了针对字节数组的操作，生成的Java类负责对这些数据进行解释。对于FlatBuffers编码的数据，无需进行解码，只需进行解释。在编译 .fbs 文件时，每个字段在这段数据中的位置将被确定。每个字段的类型及长度将被硬编码进生成的Java类。

Struct 类的代码也比较简洁：

package com.google.flatbuffers;

import java.nio.ByteBuffer;

/// @cond FLATBUFFERS_INTERNAL

/**
 * All structs in the generated code derive from this class, and add their own accessors.
 */
public class Struct {
  /** Used to hold the position of the `bb` buffer. */
  protected int bb_pos;
  /** The underlying ByteBuffer to hold the data of the Struct. */
  protected ByteBuffer bb;
}

整体的结构如下图：

在序列化结构化数据时，我们首先需要创建一个 FlatBufferBuilder ，在这个对象的创建过程中会分配或从调用者那里获取 ByteBuffer，序列化的数据将保存在这个 ByteBuffer中：

   /**
    * Start with a buffer of size `initial_size`, then grow as required.
    *
    * @param initial_size The initial size of the internal buffer to use.
    */
    public FlatBufferBuilder(int initial_size) {
        if (initial_size <= 0) initial_size = 1;
        space = initial_size;
        bb = newByteBuffer(initial_size);
    }

   /**
    * Start with a buffer of 1KiB, then grow as required.
    */
    public FlatBufferBuilder() {
        this(1024);
    }

    /**
     * Alternative constructor allowing reuse of {@link ByteBuffer}s.  The builder
     * can still grow the buffer as necessary.  User classes should make sure
     * to call {@link #dataBuffer()} to obtain the resulting encoded message.
     *
     * @param existing_bb The byte buffer to reuse.
     */
    public FlatBufferBuilder(ByteBuffer existing_bb) {
        init(existing_bb);
    }

    /**
     * Alternative initializer that allows reusing this object on an existing
     * `ByteBuffer`. This method resets the builder's internal state, but keeps
     * objects that have been allocated for temporary storage.
     *
     * @param existing_bb The byte buffer to reuse.
     * @return Returns `this`.
     */
    public FlatBufferBuilder init(ByteBuffer existing_bb){
        bb = existing_bb;
        bb.clear();
        bb.order(ByteOrder.LITTLE_ENDIAN);
        minalign = 1;
        space = bb.capacity();
        vtable_in_use = 0;
        nested = false;
        finished = false;
        object_start = 0;
        num_vtables = 0;
        vector_num_elems = 0;
        return this;
    }

    static ByteBuffer newByteBuffer(int capacity) {
        ByteBuffer newbb = ByteBuffer.allocate(capacity);
        newbb.order(ByteOrder.LITTLE_ENDIAN);
        return newbb;
    }

下面我们更详细地分析基本数据类型数据、数组及对象的序列化过程。ByteBuffer 为小尾端的。

FlatBuffers编码基本数据类型

FlatBuffer 的基本数据类型主要包括如下这些：

Boolean
Byte
Short
Int
Long
Float
Double

FlatBufferBuilder 提供了三组方法用于操作这些数据：

    public void putBoolean(boolean x);
    public void putByte   (byte    x);
    public void putShort  (short   x);
    public void putInt    (int     x);
    public void putLong   (long    x);
    public void putFloat  (float   x);
    public void putDouble (double  x);

    public void addBoolean(boolean x);
    public void addByte   (byte    x);
    public void addShort  (short   x);
    public void addInt    (int     x);
    public void addLong   (long    x);
    public void addFloat  (float   x);
    public void addDouble (double  x);

    public void addBoolean(int o, boolean x, boolean d);
    public void addByte(int o, byte x, int d);
    public void addShort(int o, short x, int d);
    public void addInt    (int o, int     x, int     d);
    public void addLong   (int o, long    x, long    d);
    public void addFloat  (int o, float   x, double  d);
    public void addDouble (int o, double  x, double  d);

putXXX 那一组，直接地将一个数据放入 ByteBuffer 中，它们的实现基本如下面这样：

    public void putBoolean(boolean x) {
        bb.put(space -= Constants.SIZEOF_BYTE, (byte) (x ? 1 : 0));
    }

    public void putByte(byte x) {
        bb.put(space -= Constants.SIZEOF_BYTE, x);
    }

    public void putShort(short x) {
        bb.putShort(space -= Constants.SIZEOF_SHORT, x);
    }

Boolean值会被先转为byte类型再放入 ByteBuffer。另外一点值得注意的是，数据是从 ByteBuffer 的结尾处开始放置的，space用于记录最近放入的数据的位置及剩余的空间。

addXXX(XXX x) 那一组在放入数据之前会先做对齐处理，并在需要时扩展 ByteBuffer 的容量：

    static ByteBuffer growByteBuffer(ByteBuffer bb) {
        int old_buf_size = bb.capacity();
        if ((old_buf_size & 0xC0000000) != 0)  // Ensure we don't grow beyond what fits in an int.
            throw new AssertionError("FlatBuffers: cannot grow buffer beyond 2 gigabytes.");
        int new_buf_size = old_buf_size << 1;
        bb.position(0);
        ByteBuffer nbb = newByteBuffer(new_buf_size);
        nbb.position(new_buf_size - old_buf_size);
        nbb.put(bb);
        return nbb;
    }

   public void pad(int byte_size) {
       for (int i = 0; i < byte_size; i++) bb.put(--space, (byte) 0);
   }

    public void prep(int size, int additional_bytes) {
        // Track the biggest thing we've ever aligned to.
        if (size > minalign) minalign = size;
        // Find the amount of alignment needed such that `size` is properly
        // aligned after `additional_bytes`
        int align_size = ((~(bb.capacity() - space + additional_bytes)) + 1) & (size - 1);
        // Reallocate the buffer if needed.
        while (space < align_size + size + additional_bytes) {
            int old_buf_size = bb.capacity();
            bb = growByteBuffer(bb);
            space += bb.capacity() - old_buf_size;
        }
        pad(align_size);
    }

    public void addBoolean(boolean x) {
        prep(Constants.SIZEOF_BYTE, 0);
        putBoolean(x);
    }

    public void addInt(int x) {
        prep(Constants.SIZEOF_INT, 0);
        putInt(x);
    }

对齐是数据存放的起始位置相对于ByteBuffer的结束位置的对齐，additional bytes被认为是不需要对齐的，且在必要的时候会在ByteBuffer可用空间的结尾处填充值为0的字节。在扩展 ByteBuffer 的空间时，老的ByteBuffer被放在新ByteBuffer的结尾处。

addXXX(int o, XXX x, YYY y) 这一组方法在放入数据之后，会将 vtable 中对应位置的值更新为最近放入的数据的offset。

    public void addShort(int o, short x, int d) {
        if (force_defaults || x != d) {
            addShort(x);
            slot(o);
        }
    }

    public void slot(int voffset) {
        vtable[voffset] = offset();
    }

后面我们在分析编码对象时再来详细地了解vtable。

基本上，在我们的应用程序代码中不要直接调用这些方法，它们主要在构造对象时用于存储对象的基本数据类型字段。

网易云新用户大礼包：https://www.163yun.com/gift

本文来自网易实践者社区，经作者韩鹏飞授权发布。