Skip to content

Custom Builders

Jump to bottom Edit New page
ig0774 edited this page Jan 19, 2016 · 16 revisions

Custom Builders

This is a work-in-progress to add some documentation on how to write custom builders for LaTeXTools.

Since the release on March 13, 2014 (v3.1.0), LaTeXTools has had support for custom build systems, in addition to the default build system, called the traditional builder. Support for custom builders is provided by means of small Python scripts, called builders, that interact with the LaTeXTools build system. In order to write a basic builder it is a good idea to have some basic familiarity with the Python language.

LaTeXTools comes packaged with a small sample builder to demonstrate the basics of the builder system, called SimpleBuilder which can be used as a reference for what builders can do.

Note that if you are interested in creating your own builder, please pay attention to the Caveats section below.

The Basics

A really simple Builder

Every builder consists of a single Python class called "WhateverBuilder" (so, for example, "TraditionalBuilder", "SimpleBuilder", etc.) which is a sub-class of a class called "PdfBuilder". Note that the class name should be unique (i.e., it can't share a name with any of the built-in builders) and it must end with "Builder". Each builder class implements a single generator function called commands() which is responsible for generating a list of commands to be run.

Below is a really simple builder that does nothing to demonstrate the basic structure of a builder:

# the pdfBuilder module will always be available on the PYTHONPATH
from pdfBuilder import PdfBuilder

class ReallySimpleBuilder(PdfBuilder):
    # for now, we are ignoring all of the arguments passed to the builder
    def __init__(self, *args):
        # call the __init__ method of PdfBuilder
        super(ReallySimpleBuilder, self).__init__(*args)
        
        # now we do the initialization for this builder
        # the only thing that must be set here is the builder name
        self.name = "Really Simple Builder"
    
    # commands is a generator function that yields the commands to be run
    def commands(self):
        # display a message in the build output console
        self.display("\n\nReallySimpleBuilder")
        
        # yield is how we pass the command to be run back to LaTeXTools
        #
        # each yield should yield a tuple consisting of the command to be run
        # and a message to be displayed, if any
        #
        # for the ReallySimpleBuilder, we yield ("", None) which tells
        # LaTeXTools there is no command to be run and no message to be
        # displayed
        yield ("", None)

To use this, save it to a file called "reallySimpleBuilder.py" in your Sublime Text User package (you can find this folder by selecting Preferences|Browse Packages... or running the Preferences: Browse Packages command). Then, in your LaTeXTools preferences, change the "builder" setting to "reallySimple" and change the "builder_path" setting to "User". Try compiling a document. You should see the following:

[Compiling ...]

ReallySimpleBuilder

Notice how the message we set using self.display() gets displayed.

Also notice how the name of the Python file matches the name of the builder (except with the first letter lower-cased) and the name given to the setting. These must match in order for LaTeXTools to be able to find and execute your builder.

Generating Basic Commands

The PdfBuilder base class provides access to some very basic information about the tex document being compiled, which can be gathered from the following variables:

Variable Description
self.tex_root the full path of the main tex document, e.g. C:\path\to\tex_root.tex
self.tex_dir the full path to the directory containing the main tex document, e.g. C:\path\to
self.tex_name the name of the main tex document, e.g. tex_root.tex
self.base_name the name of the main tex document without the extension, e.g. tex_root
self.tex_ext the extension of the main tex document, e.g. tex

(Note that all of these refer to the main tex document, specified using the %!TEX root directive or the "TEXroot" setting)

With this is mind, we can now write a builder that actually does something useful. Below is a sample builder that simply runs the standard pdflatex, bibtex, pdflatex, pdflatex pattern.

from pdfBuilder import PdfBuilder

# here we define the commands to be used
# commands are passed to subprocess.Popen which prefers a list of
# arguments to a string
PDFLATEX = ["pdflatex", "-interaction=nonstopmode", "-synctex=1"]
BIBTEX = ["bibtex"]

class BasicBuilder(PdfBuilder):
    def __init__(self, *args):
        super(BasicBuilder, self).__init__(*args)
        
        # now we do the initialization for this builder
        self.name = "Basic Builder"

    def commands(self):
        self.display("\n\nBasicBuilder: ")

        # first run of pdflatex
        # this tells LaTeXTools to run:
        #  pdflatex -interaction=nonstopmode -synctex=1 tex_root
        # note that we append the base_name of the file to the command here
        yield(PDFLATEX + [self.base_name], "Running pdflatex...")

        # LaTeXTools has run pdflatex and returned control to the builder
        # here we just add text saying the step is done, to give some
        # feedback
        self.display("done.\n")

        # now run bibtex
        yield(BIBTEX + [self.base_name], "Running bibtex...")

        self.display("done.\n")

        # second run of pdflatex
        yield(PDFLATEX + [self.base_name], "Running pdflatex again...")

        self.display("done.\n")

        # third run of pdflatex
        yield(PDFLATEX + [self.base_name], "Running pdflatex for the last time...")

        self.display("done.\n")

To use this, save it to a file called "basicBuilder.py" then change your "builder" setting to "basic". When you compile a document, you should see the following output:

[Compiling ...]

BasicBuilder: Running pdflatex...done.
Running bibtex...done.
Running pdflatex again...done.
Running pdflatex for the last time...done.

...

Since this builder actually does a build, there may be additional messages displayed after the last message from our builder. These usually come from LaTeXTools log-parsing code which is run after the build completes.

Interacting with Output

Of course, sometimes it is necessary not just to run a series of commands, but also to react to the output of those commands to determine the next step in the process. This is what the SimpleBuilder does to determine whether or not to run BibTeX, searching the output for a particular pattern that pdflatex generates to determine whether or not to run BibTeX.

The output of the previously run command is available after LaTeXTools returns control to the builder in the variable self.out. This consists of anything written to STDOUT and STDERR, i.e., all the messages you would see if running the command from the terminal / command line.

Building on our previous example, here's a builder that checks to see if BibTeX (only) needs to be run. This example makes use of Python's re library, which provides operations for dealing with regular expressions, a way of matching patterns in strings.

from pdfBuilder import PdfBuilder

import re

PDFLATEX = ["pdflatex", "-interaction=nonstopmode", "-synctex=1"]
BIBTEX = ["bibtex"]

# here we define a regular expression to match the output expected
# if we need to run bibtex
# this matches any lines like:
#  Warning: Citation: `aristotle:ethics' on page 2 undefined
CITATIONS_REGEX = re.compile(r"Warning: Citation `.+' on page \d+ undefined")

class BibTeXBuilder(PdfBuilder):
    def __init__(self, *args):
        super(BibTeXBuilder, self).__init__(*args)
        
        # now we do the initialization for this builder
        self.name = "BibTeX Builder"

    def commands(self):
        self.display("\n\nBibTeXBuilder: ")

        # we always run pdflatex
        yield(PDFLATEX + [self.base_name], "Running pdflatex...")

        # here control has returned to the builder from LaTeXTools
        # we display the same message as last time...
        self.display("done.\n")

        # and now we check the output to see if bibtex needs to be run
        # search will scan the entire output for any match of the
        # pattern we defined above
        if CITATIONS_REGEX.search(self.out):
            # if a matching bit of text is found, we need to run bibtex
            # now run bibtex
            yield(BIBTEX + [self.base_name], "Running bibtex...")

            self.display("done.\n")

            # we only need to run the second and third runs of pdflatex if we
            # actually ran bibtex, so these remain inside the same `if` block
            # code to run bibtex

            # second run of pdflatex
            yield(PDFLATEX + [self.base_name], "Running pdflatex again...")

            self.display("done.\n")

            # third run of pdflatex
            yield(PDFLATEX + [self.base_name], "Running pdflatex for the last time...")

            self.display("done.\n")

To use this builder, save it to a file called "bibTeXBuilder.py" and change the "builder" setting to "bibTeX". When using this builder, you should see that it only runs bibtex and the final two pdflatex commands if you have any citations.

More Advanced Topics

The following sections deal with more advanced concepts in creating builders or with features that need careful handling for one reason or another.

Allowing the user to set Options

Sometimes having a static series of commands to build a document is not enough and you'd want to give the user an opportunity to, for example, tell the builder what command to run to generate a bibliography. LaTeXTools provides your builder with access to the settings in the build_settings block of your LaTeXTools preferences through the variable self.builder_settings. Note that when allowing the use of settings it is important to verify that values you get make sense and that if no value is supplied, you provide a sane default.

Our builder from the previous example could be modified to support either bibtex or biber as the bibliography program depending on a user setting like so:

from pdfBuilder import PdfBuilder

import re
import sublime

PDFLATEX = ["pdflatex", "-interaction=nonstopmode", "-synctex=1"]
# notice that we do not define the bibliography command here, since it will
# depend on settings that can only be known when our builder is initialized

CITATIONS_REGEX = re.compile(r"Warning: Citation `.+' on page \d+ undefined")

class BibBuilder(PdfBuilder):
    def __init__(self, *args):
        super(BibBuilder, self).__init__(*args)
        
        # now we do the initialization for this builder
        self.name = "Bibliography Builder"

        # here we get which bibliography command to use from the
        # builder_settings
        # notice that we draw this setting from the platform-specific portion
        # of the builder_settings block. this allows the setting to be changed
        # for each platform
        self.bibtex = self.builder_settings.get(
            sublime.platform(), {}).get('bibtex') or 'bibtex'
        # notice that or clause here ensures that self.bibtex will be set to
        # 'bibtex' if the 'bibtex' setting is unset or blank.

    def commands(self):
        self.display("\n\nBibBuilder: ")

        # we always run pdflatex
        yield(PDFLATEX + [self.base_name], "Running pdflatex...")

        # here control has returned to the builder from LaTeXTools
        self.display("done.\n")

        # and now we check the output to see if bibtex needs to be run
        if CITATIONS_REGEX.search(self.out):
            # if a matching bit of text is found, we need to run the configured
            # bibtex command
            # note that we wrap this value in a list to ensure that a list is
            # yielded to LaTeXTools
            yield([self.bibtex] + [self.base_name], "Running " + self.bibtex + "...")

            self.display("done.\n")

            # second run of pdflatex
            yield(PDFLATEX + [self.base_name], "Running pdflatex again...")

            self.display("done.\n")

            # third run of pdflatex
            yield(PDFLATEX + [self.base_name], "Running pdflatex for the last time...")

            self.display("done.\n")

To use this builder, save it to a file called "bibBuilder.py" and change the "builder" setting to "bib". You should be able to change what command gets run when a bibliography is needed by changing the "bibtex" setting in "builder_settings".

Assuming you are on OS X, you might use something like this to run biber instead of bibtex with this builder:

"builder_settings": {
    "osx": {
        "bibtex": "biber"
    }
}

Important Notice that all interaction with the settings occurred in builder's __init__() function. This is to ensure that the builder works on ST2 as well as ST3. For more on this, see the caveats below.

Caveats

What LaTeXTools expects

LaTeXTools Expects

  • That everything yielded from a builder will be a tuple consisting of a command to run and a message to display. The command can be either an argument list or a string or a subprocess.Popen object. If the command is an empty string ("") or None, LaTeXTools will assume that your builder is finished. The message should either be a string or None.
  • That the final output of the builder will be a PDF document in the same directory as the main file and named $main_file.pdf. So, if the main file is my_amazing_document.tex, the output should be my_amazing_document.pdf.
  • That the LaTeX log will be available in the same directory as the main file and named $main_file.log. So, if the main file is my_amazing_document.tex, the log file should be my_amazing_document.log.

If you don't adhere to these expectations, it is likely to cause errors or breakage of certain features.

Running PDFLaTeX and friends

For the most part, subject to the above caveats, you are free to run pdflatex and other commands as you'd like. However, there are a couple of other considerations to be aware of:

  • These commands will be run in a non-interactive environment. Therefore, it is a good idea to ensure that LaTeX is run with either -interaction=nonstopmode or -interaction=batchmode set, as otherwise the process will hang on an error.
  • Forward search requires a .synctex.gz, which contains mappings between line and column numbers and positions in the PDF file. If you want forward-search to function, you should ensure that LaTeX is run with -synctex=1 (or some other non-zero number).
  • The error reports that LaTeXTools generates are only as good as the data it can extract from the log. The data is more useful with -file-line-error is enabled. This is the default on most TeX installs, but if you have changed it, you may want to pass -file-line-error to your invocation of LaTeX.

Threading, the Sublime API, and Settings

LaTeX builds tend to be quite long-running processes and running them causes the thread they are running in to be unavailable to execute any other code for long periods of time. For this reason, the commands() generator every builder defines is run on its own thread. For the most part, this need not concern you. However, if you need to interact with the sublime API for any reason, it is advisable that you do as much as possible in the __init__() method. Unlike commands(), __init__() is called on the main Sublime thread. Builders that access the sublime API inside the commands() function are likely to break on ST2, whose API is not thread-safe. This is also why any processing of settings should happen in the __init__() function and not in the commands() generator.

Add a custom footer
Clone this wiki locally