Preliminary HTTP/1.1 Performance Evaluation
January 15, 1997
This talk has been obsoleted by Talk of February 10
Henrik Frystyk Nielsen, W3C
Jim Gettys, W3C / Digital
Anselm Baird-Smith, W3C
Eric Prudhommeaux, W3C
Purpose of this Evaluation
-
Evaluate the gains of using HTTP/1.1 for basic web transport
-
Does HTTP/1.1 fulfill its design goals?
-
For the network? (network traffic/behavior)
-
For the end user? (perceived speed)
-
Test Persistent Connections, Pipelining and Range Requests
-
Tested two common situations
-
First Time Retrieval (Filling the cache)
-
Subsequent Retrievals (Cache validation)
-
Have not investigated benefits of HTTP/1.1 caching facilities or
Transfer-Encoding (e.g. gzip compression)
-
This work is still very much "Work In Progress"
Test Setup
-
Applications used for Testing
-
libwww 5.1 robot as client (C based)
-
Jigsaw 1.03 as server 1 (Java based)
-
Apache 1.2 beta 1 as server 2 (C based)
-
Limited use of Netscape Communicator
-
Platforms used
-
Servers running on Linux (2.0.18)
-
Clients running on Alphas (4.0, 4.0a)
-
Windows NT used on low bandwidth (dialup)
Test Setup cont.
-
HTTP Client Implementations
-
HTTP/1.0 with 6 simultaneous connections
-
HTTP/1.1 with 1 persistent connection
-
HTTP/1.1 with 1 persistent connection using pipelined requests
-
Server HTTP Implementations
Pipelining and Output Buffering
-
Pipelining allows multiple outstanding requests, reducing round trips
-
Responses are still serialized - difference is timing
-
Buffering packs TCPs segments better
-
Reduces number of packets (and server context switches) required for same
"work"
-
When to Flush?
-
If the data in the output buffer exceeds 1K
-
If the data is buffered longer than N ms
-
If the application explicitly requests it
-
Experimenting with Nagles algorithm
-
Turned on/off in both client and server
-
No differences seen (due to good buffering???)
Description of Tests
-
Mix of Microsoft and Netscape pages ("Microscape home page")
-
1 HTML document (~ 40K)
-
41 Inlined images ( ~ 130K)
-
Cache Load Test
-
1 GET request on HTML document
-
GET requests on 41 inlined images
-
Think visit a site for the first time
-
Cache Validation Test
-
1 GET request on HTML document
-
Simulated cache validation with 41 HEAD requests
-
Think revisit a site youve been to before
Network Environments Tested
-
High Bandwidth, Low Latency
-
LAN 10 Mbit Ethernet (RTT ~ 1ms)
-
High Bandwidth, High Latency
-
WAN between MIT and LBL (RTT ~ 75ms)
-
Low Bandwidth, High Latency
-
PPP over 28.8 modem (Local ISP to MIT, RTT ~ 150ms)
-
Note: The following results were accidentally taken against a Jigsaw
server without our buffering improvements; final results for HTTP/1.1 with
pipelining should be better than those quoted here. The following results
are probably roughly similar to the Apache 1.1b1 server (without the connection
close "feature").
Results - LAN Load/Validation
10Mb Ethernet
Jigsaw /
libwww |
HTTP/1.0 |
HTTP/1.1
Persistent |
HTTP/1.1
Pipelined |
Total
number of
packets |
596/461 |
311/175 |
232/126 |
Total
elapsed time
(seconds) |
1.61/1.48 |
1.72/1.46 |
1.35/1.21 |
Results - WAN Load/Validation
MIT - LBL, RTT ~75ms
Jigsaw /
libwww |
HTTP/1.0 |
HTTP/1.1
Persistent |
HTTP/1.1
Pipelined |
Total
number of
packets |
640/478 |
339/185 |
232/113 |
Total
elapsed time
(seconds) |
3.23/2.95 |
6.67/5.45 |
2.49/1.96 |
Results - PPP Load
28.8Kbaud modem, via Local ISP to MIT, RTT ~150ms
Jigsaw /
libwww |
HTTP/1.0 *) |
HTTP/1.1
Persistent |
HTTP/1.1
Pipelined |
Total
number of
packets |
489 |
|
288 |
Total
elapsed time
(seconds) |
65.05 |
|
54.38 |
*) HTTP/1.0 data measured using Netscape Communicator
Summary (in our tests)
-
Roughly half of the packets in HTTP/1.0 are TCP open/close
-
Such packets are not "congestion controlled"
-
Significant further gain by using buffering with pipelining
-
Validation may use less than 1/5 packets of HTTP/1.0
-
Cache load may use less than 1/2 packets of HTTP/1.0
-
HTTP/1.1 up to twice as fast as HTTP/1.0 (elapsed time)
-
Server also gains when using pipelining
-
Client implementation required some care and effort; server implementation
was very easy, though slight care in output buffering makes a significant
difference in performance
-
Nagle is not a bottleneck
-
Recommend that it is turned off
Summary (Continued)
-
Server performance also increases when using pipelining
-
Apache 1.2b1 was faster than Jigsaw, but used more packets than Jigsaw with
output buffering mods. (we are working with the Apache group)
-
More Work is Needed
-
Complete PPP data, retake data and re-reduce
-
Range requests for "poor man's multiplexing"
-
test current hypothesis: Apache runs much faster than Jigsaw, and therefore
elapsed time measurements will be more favorable to HTTP/1.1 than those reported
here.
Conclusions
-
Pipelined implementation required to outperform HTTP/1.0's elapsed time
-
Buffered Pipelined implementation also reduced network traffic
-
In our tests, HTTP/1.1 w. Pipelining using a single connection ALWAYS
outperformed HTTP/1.0 using multiple connections
-
HTTP/1.1 will help the Internets problems significantly, once deployed.
The improvement in packets will be at least a factor of two, with much lower
congestion
-
Users will see observably higher performance
-
HTTP/1.1 should be deployed as soon as possible
More Information
-
Performance Overview
-
HTTP Protocol
-
W3C