Skip to content

CDN Hosting at Google Cloud Storage

Jump to bottom
Peter Krautzberger edited this page Jun 2, 2014 · 34 revisions

Note: these are notes about using Google Cloud Storage to host a copy of MathJax. We use GCS in combination with CloudFlare which means these instructions are not always optimal if you are working in a pure GCS setup.

Preliminaries

Setting a default ACL (access control list)

Notes:

This is a one-time task

The GCS default for access control is private. GCS does not allow for ACLs to persist when files are overwritten.

Since we update files in \latest with every releases and files in beta during beta runs, we need to prevent GCS from marking those new files as private (breaking public access to the CDN).

The solution is to set the default ACL for the bucket once following the instructions in the gsutil documentation.

To set this either use

$ gsutil defacl set public-read gs://our-bucket

Or create a suitable XML file containing

    <Entry>
        <Scope type="AllUsers"/>
        <Permission>READ</Permission>
    </Entry>

Copying

Notes:

  • gsutil option: -m multithreaded (higher load+cost but faster)

  • cp options: recursively (-R), compressed (-z), verbose (-v) https://developers.google.com/storage/docs/gsutil/commands/cp

  • Note: compressed is necessary to get gzipped delivery. Google Cloud storage does not compress on the fly but should decompress on the fly for old browsers that can't accept decompressed files.

      gsutil -m cp -Rzv mathjax gs://our-bucket/

This also works within GCS, i.e., bucket to bucket.

Note that metadata isn't preserved by default (public access, caching, custom headers etc), especially when updating a bucket from local files (TODO there must be some way to specicy metadata?).

If you copy within GCS from bucket to bucket, you can use the -p option to preserve metadata (warning: this incurs higher costs).

gsutil -m cp -Rp gs://bucket1/file1 gs://bucket2/

(You'll still need to set the CORS for each bucket though; see below.)

make files public

Notes:

This has to be done whenever files are uploaded (TODO: automate!)

    gsutil -m acl set -R -a public-read gs://our-bucket/path/to/file_or_wildcard

See also Caching below

set CORS

Notes:

Create an XML file with

<?xml version="1.0" encoding="UTF-8"?>
<CorsConfig>
  <Cors>
        <Origins>
            <Origin>*</Origin>
        </Origins>
        <Methods>
            <Method>GET</Method>
            <Method>POST</Method>
            <Method>HEAD</Method>
        </Methods>
        <ResponseHeaders>
            <ResponseHeader>*</ResponseHeader>
        </ResponseHeaders>
        <MaxAgeSec>86400</MaxAgeSec>
    </Cors>
</CorsConfig>

Save as cors.xml and run

gsutil cors set cors.xml gs://our-bucket

mime-type headers

Notes:

Caching

Note Since we are combining GCS with CloudFlare's caching, we will have short caching times on GCS.

This has to be repeated whenever files get uploaded from local.

    $ gsutil -m setmeta -r -h "Cache-Control:public, max-age=3600" gs://our-bucket/mathjax/

CNAME

To use a CNAME for a bucket, the bucket needs to be named like the CNAME, see https://developers.google.com/storage/docs/reference-uris#cname.

You also need to verify domain ownership; see https://support.google.com/webmasters/answer/35179?hl=en.

Testing

  • CORS allow-origin [vital]
    • A simple test: switch mathjax.org over to the GCS origin (bucket.storage.googlapis.com) and use saucelabs to check rendering across IE, FF, Chrome, Safari.
    • Since CORS is set on bucket level, we should not have to worry about it.
  • font mimetypes [non-vital]
    • TODO write script to curl -I all woff and otf files, compare header.

Logging

We don't log on GCS anymore since we combine it with CloudFlare (and get their stats).

For the record, instructions are at https://developers.google.com/storage/docs/gsutil/commands/logging

MathJax Wiki

Clone this wiki locally