CDN Hosting at Google Cloud Storage

Note: these are preliminary notes about using Google Cloud Storage as a CDN service

Preliminaries

• The documentation home is https://developers.google.com/storage/index.
• The main tool is gsutil.
• Note: due to rapid development a lot of information on the web uses outdated syntax (but is still useful, e.g., https://gist.github.com/mhulse/4362104).

Copying

Notes:

• gsutil option: -m multithreaded (higher load+cost, faster)

• cp options: recursively (-R), compressed (-z), verbose (-v) https://developers.google.com/storage/docs/gsutil/commands/cp

• Note: compressed is necessary to get gzipped delivery. Google Cloud storage does not compress on the fly but should decompress on the fly for old browsers that can't accept decompressed files.

    gsutil -m cp -Rzv mathjax gs://mathjax-cdn/
  

This also works within GCS, i.e., bucket to bucket. Note that metadata isn't preserved by default (ACLs etc); use -p option to preserve them (warning: this incurs higher costs).

gsutil -m cp -Rp gs://bucket1/file1 gs://bucket2/

(You'll still need to set the CORS for each bucket though; see below.)

make bucket public

Notes:

• should only be needed once

• https://developers.google.com/storage/docs/gsutil/commands/acl#set

    gsutil -m setacl -R -a public-read gs://mathjax-cdn
  

set CORS

Notes:

• should only be needed once
• https://developers.google.com/storage/docs/cross-origin

create XML file with

<?xml version="1.0" encoding="UTF-8"?>
<CorsConfig>
  <Cors>
        <Origins>
            <Origin>*</Origin>
        </Origins>
        <Methods>
            <Method>GET</Method>
            <Method>POST</Method>
            <Method>HEAD</Method>
        </Methods>
        <ResponseHeaders>
            <ResponseHeader>*</ResponseHeader>
        </ResponseHeaders>
        <MaxAgeSec>86400</MaxAgeSec>
    </Cors>
</CorsConfig>

Save as cors.xml and run

gsutil cors set cors.xml gs://fonts.acmecompany.com/

mime-type headers

Notes:

• https://developers.google.com/storage/docs/gsutil/commands/setmeta

• the ** wildcard includes subdirectories, https://developers.google.com/storage/docs/gsutil/addlhelp/WildcardNames

• EOT and SVG content-type headers are detected correctly during upload.

• WOFF is set to the old mime type x-font/woff; that works but we decided to go with the future-proof font-woff. Either change it as below or see advice here.

• For otf:

    gsutil setmeta -r -h "Content-Type:font/opentype" -h "Cache-Control:public, max-age=3600" -h "Content-Disposition" gs://mathjax-cdn/mathjax/**.otf
  

• For otf:

    gsutil setmeta -r -h "Content-Type:application/font-woff" -h "Cache-Control:public, max-age=3600" -h "Content-Disposition" gs://mathjax-cdn/mathjax/**.woff
  

caching

This is a one time task. Proposed caching is 3 months for all but MathJax.js which gets 1 week.

    $ gsutil -m setmeta -r -h "Cache-Control:public, max-age=7776000" gs://mathjax-cdn/mathjax/
    $ gsutil -m setmeta -r -h "Cache-Control:public, max-age=3600" gs://mathjax-cdn/mathjax/**/MathJax.js

This is theoretically risky. Since caching can only be specified after making the file public, it could theoretically happen that MathJax.js is called and cached before the second command runs -- fixing it to a 3 month caching that cannot be undone.

Well, it didn't happen. So we're good.

Logging

Another one time task. Instructions are at https://developers.google.com/storage/docs/gsutil/commands/logging

CNAME

To use a CNAME for a bucket, the bucket needs to be named like the CNAME, see https://developers.google.com/storage/docs/reference-uris#cname.

You also need to verify domain ownership; see https://support.google.com/webmasters/answer/35179?hl=en.

TODO

• tests.
  • CORS [vital]
    • A simple test: switch mathjax.org over to the origin. user browsershots/browserstack etc to check rendering across IE, FF, Chrome, Safari.
  • mimetype [non-vital]
    • write script to curl -I all woff and otf files, compare header.
• document log analysis