How to Serve Web Clients in China
Until recently, all users of Flexport.com were located within the US and Europe with <100ms ping times to Amazon’s US-East data centers. With the opening of a new office and warehouse in China, however, Flexport.com needed a network upgrade to support the distant and highly regulated Chinese internet.
These tips may be helpful for any company that’s expanding into China (and expects to have web users there).
The challenges of operating a website in China
The Chinese government requires all websites to obtain ICP licenses. Any website without an ICP license may be blocked by ISPs without warning and without recourse. Check out https://en.greatfire.org/analyzer for more information on websites blocked in China.
To get an ICP license:
- The government requires a Chinese enterprise business license and a copy of the photo ID of a Chinese employee, and communicates exclusively in Chinese. Plan on entering into business within China before applying for an ICP license.
- The applicant must initiate the ICP licensing process with their Chinese ISP company. The ISP collects some preliminary information and initiates contact with the local government.
- The local government responds with further questions based on the industry in which the applicant is involved. Media companies tend to receive greater scrutiny than other industries.
- The ISP then forwards the local government’s response to the applicant company. Again, this response will be in Chinese and tailored to the company, not just a form letter, so it is important to have a native Chinese speaker at the company.
- After some amount of back and forth, the government responds with an ICP license number that must be displayed on the front page of the domain. The license is specific to the ISP, the domain, and the domain’s owner.
Operating a fast website in China when servers are in the US
We require encrypted communication (HTTPS) on all of our web traffic. In order to establish this encrypted channel:
- The client web browser sends a packet to www.flexport.com indicating that it wants to start an encrypted channel.
- The server at www.flexport.com accepts the request to establish a connection and lists the encryption methods available.
- The client web browser chooses an encryption method that it supports and sends a “challenge” packet with data that the server must encrypt with its private key.
- The server responds with the encrypted response.
- The client verifies that the server response matches the server’s public certificate, and if it matches, it begins requesting webpages from the server through the encrypted connection.
This back-and-forth doesn’t take much time at either end, but the latency between one end sending a packet and the other end receiving the packet can take some time.
In New York this latency might be 20ms, while the latency in San Francisco is 60ms. In China the latency can be as high as 250ms.
That corresponds to connection initiation times of 80ms, 240ms, and 1,000ms — just to begin requesting webpages (ignoring HTTP2 and other advances in web technology).
Partnering with Cloudflare to make it happen
Cloudflare is a Content Distribution Network (CDN), and they partner with ISPs in China to extend their CDN services within mainland China. To get access to this, contact Cloudflare; it’s part of their Enterprise plan.
Cloudflare and their ISP partner in China were our point of contact during the ICP licensing (as described above). The back-and-forth of ICP questions lasted about 6 weeks. Once this was finished, Cloudflare flipped a switch and enabled their Chinese network for Flexport.com.
Cloudflare decreases this connection initiation time by providing endpoints all over the globe. By connecting from San Francisco to Cloudflare’s data center in San Jose, my latency is under 10ms and my connection initiation time is 40ms. Our office in Amsterdam achieves a 60ms connection initiation time. Our clients connecting from within China reach Cloudflare’s data centers in China and achieve a similar 60ms connection initiation time.
1a) Create DNS entries at CDN so that they know what traffic we plan on routing through them.
1b) Create an SSL key/cert for use with the CDN. The key and cert should only be used by them so that in the case the key were exposed we would only have to deactivate that certificate and could serve directly from our servers in the meantime.
1c) Upload the SSL key and certificate to Cloudflare. Their web interface checks the validity of the key/cert pair and automatically enables them for the specified domains. You might also enable copying the key/cert pair to servers within China to improve SSL connection times.
2) Test the CDN. Dig `www.flexport.cdn.cloudflare.com` to see what ip address www.flexport.com will have on Cloudflare.
2b) Then add `<ip> www.flexport.com` to your /etc/hosts file so that your computer will use the CDN
2c) Request assets that should be available through the CDN. The HTML of the homepage should be available `wget https://www.flexport.com/` and a recent asset `wget https://www.flexport.com/assets/pages/team/RyanPetersen-67c04f4aaf819715fecc600a46cd6018acd36423a34c34d26df973df24f271db.jpg`
2d) Check the response headers for information from the CDN. Each CDN provider has their “Cache status” header which should read “Hit” or “Miss” depending on whether the CDN found or didn’t find the file in their cache.
2e) Compare performance. SSL Connect time should be low. Upon cache Hit the time waiting for a response should be near zero and the download speed should be fast. Upon a cache miss:
This request initialized a connection to the CDN, which forwarded the request to our server where it rendered then responded with the rendered HTML. Upon a cache hit:
This request re-used an existing connection to the CDN from the first request and requested a cached asset which was returned essentially immediately.
3) Test within China.
Perhaps the most difficult part of serving China is determining what else doesn’t work from within China. We analyzed our site and 3rd-party integrations using https://en.greatfire.org/analyzer to understand what we needed to address.
One of our front-end engineers then travelled to China to tailor our site to their needs. He configured our server to check if the client is in China, then removed calls to services blocked within China, like Google Analytics and Google Maps.
The process of launching a website in China is going to be difficult no matter how you do it, but Cloudflare helped us get things done more smoothly. We sought out Cloudflare to help us get set up in China, and wound up utilizing them to speed up our connection times for all of our users across the world.
Interested in solving these types of problems? Follow us here or on Twitter to learn more about interesting problems in the world of freight, or if you’re ready to take the next step, we’re hiring! Check out our current openings!