最新博客
最新资源下载
HTTP/2首部压缩的OkHttp3实现（下篇）
达芬奇密码2018-08-17 12:59请求间首部字段内容的复用
HPACK中，最重要的优化就是消除请求间冗余的首部字段。在实现上，主要有两个方面，一是前面看到的首部字段的索引表示，另一方面则是动态表的维护。
HTTP/2中数据发送方向和数据接收方向各有一个动态表。通信的双方，一端发送方向的动态表需要与另一端接收方向的动态表保持一致，反之亦然。
HTTP/2的连接复用及请求并发执行指的是逻辑上的并发。由于底层传输还是用的TCP协议，因而，发送方发送数据的顺序，与接收方接收数据的顺序是一致的。
数据发送方在发送一个请求的首部数据时会顺便维护自己的动态表，接收方在收到首部数据时，也需要立马维护自己接收方向的动态表，然后将解码之后的首部字段列表dispatch出去。
如果通信双方同时在进行2个HTTP请求，分别称为Req1和Req2，假设在发送方Req1的头部字段列表先发送，Req2的头部字段后发送。接收方必然先收到Req1的头部字段列表，然后是Req2的。如果接收方在收到Req1的头部字段列表后，没有立即解码，而是等Req2的首部字段列表接收并处理完成之后，再来处理Req1的，则两端的动态表必然是不一致的。
这里来看一下OkHttp3中的动态表维护。
发送方向的动态表，在 okhttp3.internal.framed.Hpack.Writer 中维护。在HTTP/2中，动态表的最大大小在连接建立的初期会进行协商，后面在数据收发过程中也会进行更新。
在编码头部字段列表的 writeHeaders(List
headerBlock) 中，会在需要的时候，将头部字段插入动态表，具体来说，就是在头部字段的名字不在静态表中，同时 名-值对不在动态表中的情况。将头部字段插入动态表的过程如下：
private void clearDynamicTable() {
Arrays.fill(dynamicTable, null);
nextHeaderIndex = dynamicTable.length - 1;
headerCount = 0;
dynamicTableByteCount = 0;
}
/** Returns the count of entries evicted. */
private int evictToRecoverBytes(int bytesToRecover) {
int entriesToEvict = 0;
if (bytesToRecover > 0) {
// determine how many headers need to be evicted.
for (int j = dynamicTable.length - 1; j >= nextHeaderIndex && bytesToRecover > 0; j--) {
bytesToRecover -= dynamicTable[j].hpackSize;
dynamicTableByteCount -= dynamicTable[j].hpackSize;
headerCount--;
entriesToEvict++;
}
System.arraycopy(dynamicTable, nextHeaderIndex + 1, dynamicTable,
nextHeaderIndex + 1 + entriesToEvict, headerCount);
Arrays.fill(dynamicTable, nextHeaderIndex + 1, nextHeaderIndex + 1 + entriesToEvict, null);
nextHeaderIndex += entriesToEvict;
}
return entriesToEvict;
}
private void insertIntoDynamicTable(Header entry) {
int delta = entry.hpackSize;
// if the new or replacement header is too big, drop all entries.
if (delta > maxDynamicTableByteCount) {
clearDynamicTable();
return;
}
// Evict headers to the required length.
int bytesToRecover = (dynamicTableByteCount + delta) - maxDynamicTableByteCount;
evictToRecoverBytes(bytesToRecover);
if (headerCount + 1 > dynamicTable.length) { // Need to grow the dynamic table.
Header[] doubled = new Header[dynamicTable.length * 2];
System.arraycopy(dynamicTable, 0, doubled, dynamicTable.length, dynamicTable.length);
nextHeaderIndex = dynamicTable.length - 1;
dynamicTable = doubled;
}
int index = nextHeaderIndex--;
dynamicTable[index] = entry;
headerCount++;
dynamicTableByteCount += delta;
}
动态表占用的空间超出限制时，老的头部字段将被移除。在OkHttp3中，动态表是一个自后向前生长的表。
在数据的接收防线，okhttp3.internal.http2.Http2Reader 的 nextFrame(Handler handler) 会不停从网络读取一帧帧的数据：
public boolean nextFrame(Handler handler) throws IOException {
try {
source.require(9); // Frame header size
} catch (IOException e) {
return false; // This might be a normal socket close.
}
/* 0 1 2 3
* 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
* +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
* | Length (24) |
* +---------------+---------------+---------------+
* | Type (8) | Flags (8) |
* +-+-+-----------+---------------+-------------------------------+
* |R| Stream Identifier (31) |
* +=+=============================================================+
* | Frame Payload (0...) ...
* +---------------------------------------------------------------+
*/
int length = readMedium(source);
if (length < 0 || length > INITIAL_MAX_FRAME_SIZE) {
throw ioException("FRAME_SIZE_ERROR: %s", length);
}
byte type = (byte) (source.readByte() & 0xff);
byte flags = (byte) (source.readByte() & 0xff);
int streamId = (source.readInt() & 0x7fffffff); // Ignore reserved bit.
if (logger.isLoggable(FINE)) logger.fine(frameLog(true, streamId, length, type, flags));
switch (type) {
case TYPE_DATA:
readData(handler, length, flags, streamId);
break;
case TYPE_HEADERS:
readHeaders(handler, length, flags, streamId);
break;
读到头部块时，会立即维护本地接收方向的动态表：
private void readHeaders(Handler handler, int length, byte flags, int streamId)
throws IOException {
if (streamId == 0) throw ioException("PROTOCOL_ERROR: TYPE_HEADERS streamId == 0");
boolean endStream = (flags & FLAG_END_STREAM) != 0;
short padding = (flags & FLAG_PADDED) != 0 ? (short) (source.readByte() & 0xff) : 0;
if ((flags & FLAG_PRIORITY) != 0) {
readPriority(handler, streamId);
length -= 5; // account for above read.
}
length = lengthWithoutPadding(length, flags, padding);
List<Header> headerBlock = readHeaderBlock(length, padding, flags, streamId);
handler.headers(endStream, streamId, -1, headerBlock);
}
private List<Header> readHeaderBlock(int length, short padding, byte flags, int streamId)
throws IOException {
continuation.length = continuation.left = length;
continuation.padding = padding;
continuation.flags = flags;
continuation.streamId = streamId;
// TODO: Concat multi-value headers with 0x0, except COOKIE, which uses 0x3B, 0x20.
// http://tools.ietf.org/html/draft-ietf-httpbis-http2-17#section-8.1.2.5
hpackReader.readHeaders();
return hpackReader.getAndResetHeaderList();
}
okhttp3.internal.http2.Hpack.Reader的readHeaders()如下：
static final class Reader {
private final List<Header> headerList = new ArrayList<>();
private final BufferedSource source;
private final int headerTableSizeSetting;
private int maxDynamicTableByteCount;
// Visible for testing.
Header[] dynamicTable = new Header[8];
// Array is populated back to front, so new entries always have lowest index.
int nextHeaderIndex = dynamicTable.length - 1;
int headerCount = 0;
int dynamicTableByteCount = 0;
Reader(int headerTableSizeSetting, Source source) {
this(headerTableSizeSetting, headerTableSizeSetting, source);
}
Reader(int headerTableSizeSetting, int maxDynamicTableByteCount, Source source) {
this.headerTableSizeSetting = headerTableSizeSetting;
this.maxDynamicTableByteCount = maxDynamicTableByteCount;
this.source = Okio.buffer(source);
}
int maxDynamicTableByteCount() {
return maxDynamicTableByteCount;
}
private void adjustDynamicTableByteCount() {
if (maxDynamicTableByteCount < dynamicTableByteCount) {
if (maxDynamicTableByteCount == 0) {
clearDynamicTable();
} else {
evictToRecoverBytes(dynamicTableByteCount - maxDynamicTableByteCount);
}
}
}
private void clearDynamicTable() {
headerList.clear();
Arrays.fill(dynamicTable, null);
nextHeaderIndex = dynamicTable.length - 1;
headerCount = 0;
dynamicTableByteCount = 0;
}
/** Returns the count of entries evicted. */
private int evictToRecoverBytes(int bytesToRecover) {
int entriesToEvict = 0;
if (bytesToRecover > 0) {
// determine how many headers need to be evicted.
for (int j = dynamicTable.length - 1; j >= nextHeaderIndex && bytesToRecover > 0; j--) {
bytesToRecover -= dynamicTable[j].hpackSize;
dynamicTableByteCount -= dynamicTable[j].hpackSize;
headerCount--;
entriesToEvict++;
}
System.arraycopy(dynamicTable, nextHeaderIndex + 1, dynamicTable,
nextHeaderIndex + 1 + entriesToEvict, headerCount);
nextHeaderIndex += entriesToEvict;
}
return entriesToEvict;
}
/**
* Read {@code byteCount} bytes of headers from the source stream. This implementation does not
* propagate the never indexed flag of a header.
*/
void readHeaders() throws IOException {
while (!source.exhausted()) {
int b = source.readByte() & 0xff;
if (b == 0x80) { // 10000000
throw new IOException("index == 0");
} else if ((b & 0x80) == 0x80) { // 1NNNNNNN
int index = readInt(b, PREFIX_7_BITS);
readIndexedHeader(index - 1);
} else if (b == 0x40) { // 01000000
readLiteralHeaderWithIncrementalIndexingNewName();
} else if ((b & 0x40) == 0x40) { // 01NNNNNN
int index = readInt(b, PREFIX_6_BITS);
readLiteralHeaderWithIncrementalIndexingIndexedName(index - 1);
} else if ((b & 0x20) == 0x20) { // 001NNNNN
maxDynamicTableByteCount = readInt(b, PREFIX_5_BITS);
if (maxDynamicTableByteCount < 0
|| maxDynamicTableByteCount > headerTableSizeSetting) {
throw new IOException("Invalid dynamic table size update " + maxDynamicTableByteCount);
}
adjustDynamicTableByteCount();
} else if (b == 0x10 || b == 0) { // 000?0000 - Ignore never indexed bit.
readLiteralHeaderWithoutIndexingNewName();
} else { // 000?NNNN - Ignore never indexed bit.
int index = readInt(b, PREFIX_4_BITS);
readLiteralHeaderWithoutIndexingIndexedName(index - 1);
}
}
}
public List<Header> getAndResetHeaderList() {
List<Header> result = new ArrayList<>(headerList);
headerList.clear();
return result;
}
private void readIndexedHeader(int index) throws IOException {
if (isStaticHeader(index)) {
Header staticEntry = STATIC_HEADER_TABLE[index];
headerList.add(staticEntry);
} else {
int dynamicTableIndex = dynamicTableIndex(index - STATIC_HEADER_TABLE.length);
if (dynamicTableIndex < 0 || dynamicTableIndex > dynamicTable.length - 1) {
throw new IOException("Header index too large " + (index + 1));
}
headerList.add(dynamicTable[dynamicTableIndex]);
}
}
// referencedHeaders is relative to nextHeaderIndex + 1.
private int dynamicTableIndex(int index) {
return nextHeaderIndex + 1 + index;
}
private void readLiteralHeaderWithoutIndexingIndexedName(int index) throws IOException {
ByteString name = getName(index);
ByteString value = readByteString();
headerList.add(new Header(name, value));
}
private void readLiteralHeaderWithoutIndexingNewName() throws IOException {
ByteString name = checkLowercase(readByteString());
ByteString value = readByteString();
headerList.add(new Header(name, value));
}
private void readLiteralHeaderWithIncrementalIndexingIndexedName(int nameIndex)
throws IOException {
ByteString name = getName(nameIndex);
ByteString value = readByteString();
insertIntoDynamicTable(-1, new Header(name, value));
}
private void readLiteralHeaderWithIncrementalIndexingNewName() throws IOException {
ByteString name = checkLowercase(readByteString());
ByteString value = readByteString();
insertIntoDynamicTable(-1, new Header(name, value));
}
private ByteString getName(int index) {
if (isStaticHeader(index)) {
return STATIC_HEADER_TABLE[index].name;
} else {
return dynamicTable[dynamicTableIndex(index - STATIC_HEADER_TABLE.length)].name;
}
}
private boolean isStaticHeader(int index) {
return index >= 0 && index <= STATIC_HEADER_TABLE.length - 1;
}
/** index == -1 when new. */
private void insertIntoDynamicTable(int index, Header entry) {
headerList.add(entry);
int delta = entry.hpackSize;
if (index != -1) { // Index -1 == new header.
delta -= dynamicTable[dynamicTableIndex(index)].hpackSize;
}
// if the new or replacement header is too big, drop all entries.
if (delta > maxDynamicTableByteCount) {
clearDynamicTable();
return;
}
// Evict headers to the required length.
int bytesToRecover = (dynamicTableByteCount + delta) - maxDynamicTableByteCount;
int entriesEvicted = evictToRecoverBytes(bytesToRecover);
if (index == -1) { // Adding a value to the dynamic table.
if (headerCount + 1 > dynamicTable.length) { // Need to grow the dynamic table.
Header[] doubled = new Header[dynamicTable.length * 2];
System.arraycopy(dynamicTable, 0, doubled, dynamicTable.length, dynamicTable.length);
nextHeaderIndex = dynamicTable.length - 1;
dynamicTable = doubled;
}
index = nextHeaderIndex--;
dynamicTable[index] = entry;
headerCount++;
} else { // Replace value at same position.
index += dynamicTableIndex(index) + entriesEvicted;
dynamicTable[index] = entry;
}
dynamicTableByteCount += delta;
}
HTTP/2中数据收发两端的动态表一致性主要是依赖TCP来实现的。
Done。
网易云新用户大礼包：https://www.163yun.com/gift
本文来自网易实践者社区，经作者韩鹏飞授权发布。
