persistence.tex

% TODO: class diagram of Resort and ReservationSystem.
% TODO: would this work better showing persistence first, then hydration?

\chapter{Persistence and hydration}

Often, we want some of our objects to maintain their existence between
executions of the program. They're intended to be durable and lasting, and so
we need a way to record their details in some kind of permanent record so they
can be resurrected later.

\index{profile@\texttt{Profile}}
\index{user@\texttt{User}}
Examples abound. Consider a social media app, where new users can register,
login, post messages, friend each other, \textit{etc.} We'd likely store
information about all this in various \texttt{User}, \texttt{Post},
\texttt{Profile}, and \texttt{FriendRequest} objects. But it sure would be a
bummer if all that data disappeared any time the server needed to be
restarted!

\index{drawing@\texttt{Drawing}}
\index{line@\texttt{Line}}
\index{rectangle@\texttt{Rectangle}}
Or consider a drawing application, which we can use to create figures like
lines, rectangles, circles, and so forth, to create the kinds of diagrams
contained in this book. Our \texttt{Drawing}, \texttt{Line}, and
\texttt{Rectangle} objects, whose instances held information about position,
size, and color, would be next to useless if they weren't able to store
themselves permanently somehow, to be reloaded in a later execution of the
app. It would be like a word processor without a ``save'' function.

\index{persistence}
\index{hydration}
What we want is a way to \textbf{persist} (or save) an object, and
\textbf{hydrate} (or restore) it later. Persistence is taking an ephemeral,
in-memory object and writing it out to some form of permanent storage -- a
file in the filesystem, a database, a network drive, or anything else that
will stay put even when the program ends. Hydration is the reverse process:
resurrecting that stored version of the entity's state into a living,
breathing object once again.

\section{Java object serialization}

\index{serialization}
\index{objectoutputstream@\texttt{ObjectOutputStream}}
\index{serializable@\texttt{Serializable}}
There are several ways to implement such operations. One, which I don't
recommend, is built into Java and is called ``serialization.'' The
\texttt{java.io} package has two classes -- \texttt{ObjectOutputStream} and
\texttt{ObjectInputStream} -- for this purpose. The idea is that just as you can
write primitive types like integers and strings to a stream with familiar file
I/O operations, you can also read and write bona fide objects
themselves.\footnote{All objects that are saved and restored in this way must
be from classes that are declared to ``implement'' the
\texttt{java.io.Serializable} ``interface.'' We'll talk about implementing
interfaces in Section~\ref{sec:interfaces}.} Just open an
\texttt{ObjectOutputStream} to a file and call \texttt{.writeObject()}, and
the object you pass, in its entirety, will be saved in the file.
\texttt{ObjectInputStream.readObject()} performs the reverse process.

\index{binary format}
\index{text format}
\index{vim@\texttt{vim}}
This all sounds like a great idea, but there's some serious practical
difficulties to consider. For one, \texttt{ObjectOutputStream} stores objects
in a \textbf{binary format} rather than a \textbf{text format}. This means
that the files it produces can't be easily analyzed -- they're compact, but
opaque, sequences of 1's and 0's. If you try to open such a file in a text
editor like \texttt{vim} to inspect what objects and values it contains,
you'll get gobbledy-gook. The upshot is that it's nearly impossible to debug
your program or even figure out what was saved.

\index{forwards-compatible}
\index{java file@\texttt{.java} file}
Just as problematic is the fact that serialization is not
\textbf{forwards-compatible}. When you write an object to a stream in this
way, the data that is persisted is hard-wired to the particular version of the
class the object is a member of. This isn't a problem if your code is
completely, permanently stable. But during a development cycle, you're
constantly changing the nature of many classes, including their instance
variables, names, and types. As soon as you change a class in any significant
way, \textit{bam!!} all previously stored copies of objects are now
unreadable. When you try to hydrate them with \texttt{ObjectInputStream},
it'll break saying, ``whoa, the class of this stored object was an old version
of the class -- it doesn't match the current \texttt{.java} file.'' And that
data is, for all intents and purposes, lost.

I've learned the hard way that when you persist objects, you want to store
them in a format which is transparent and forwards-compatible. You need to be
able to inspect exactly what was stored, and have that data be readable even
if you change the class's definition later on.

\section{Using a database}

\index{database}
\index{connection}
An excellent solution for persistence and hydration is to use a Database
Management System (DBMS) like MySQL, PostgreSQL, or MongoDB. These products
specialize in this very task. Some, like those with ``SQL'' in the name,
create \textbf{relational databases} which hold rectangular tables of data
records, sort of like gigantic spreadsheets. Others are called
\textbf{non-relational} or \textbf{NoSQL databases} and can store data in a
more flexible, non-rectangular format. Programs in Java (or any other
language) can create \textbf{connections} to either kind of database, and
read/write to them with standard commands.

The reason I won't go further into this option is because it's really a
separate topic from OOA\&D, and merits a whole course in its own right (at
UMW, it's CPSC 350). Once you learn that material, it's really the best way to
do the whole persistence/hydration thing.

\section{Using plain text files}

\index{text file}
\index{file!I/O}
In this book, I'll just explain how to do caveman-style persistence and
hydration: using plain text files. This will allow you to leverage your basic
file I/O knowledge from previous programming courses to achieve our purpose of
making objects persistent.

The first thing we need to decide is how to organize our storage. Should each
object have its own file? Should all objects of the same class be stored in
one file? Or should \textit{everything} we want to persist be stored in one
file?

There are arguments for all of these, and it really comes down to the needs of
the application. If we have a file-based save/restore paradigm, like our
drawing editor example from earlier, it probably makes sense to store
everything in a single file. When the user chooses ``save,'' we write their
entire drawing to the filename of their choice, and when they choose ``open,''
we read it back.

In a social network scenario, on the other hand, we probably want more
granularity. As soon as a user posts a new message or changes their status, we
want to create or re-write \textit{one} file (not our entire memory footprint)
with the updated information. This way we can write in little snippets as we
go, creating a bunch of files in a directory that collectively represent the
entire state of the application. When the system is taken down and rebooted
later on, it reads all those files to reconstruct its previous state.

\subsection{Example: a resort reservation system}
\label{sec:resortExample}

\index{resort@\texttt{Resort}}
\index{reservationSystem@\texttt{ReservationSystem}}
Here, let's assume that we want the first scenario: a single file that
contains all relevant information to the application. We'll create part of a
simple hotel reservation system, which stores and retrieves information about
various resort destinations. It consists of just two classes: \texttt{Resort},
objects of which represent individual hotels; and \texttt{ReservationSystem},
the main program.

For simplicity, I'm only going to show the parts of the program that are
related to persistence and hydration. (Clearly there are lots of other things
the code would need to do, like display lists of matching hotels in response
to searches, and actually make reservations.)

The key idea is this. Each class whose objects we want to save needs two
things:

\begin{compactitem}

\index{printWriter@\texttt{PrintWriter}}
\index{fileWriter@\texttt{FileWriter}}
\item For persistence, a \texttt{.persist()} method that writes the object out
to the \texttt{PrintWriter}\footnote{\texttt{PrintWriter} is a class in
\texttt{java.io} that has \texttt{.println()} methods for various data types.
I like it for that reason. Otherwise, it's essentially the same as any other
kind of \texttt{Writer}, like a \texttt{FileWriter}.} passed.

\index{scanner@\texttt{Scanner}}
\item For hydration, a constructor that takes a \texttt{Scanner}\footnote{A
\texttt{Scanner} is a class in \texttt{java.util} that can parse primitive
types from a stream of text.} and creates a new object based on the
information queued up on that \texttt{Scanner}.

\end{compactitem}

Let's do the \texttt{Resort} class first. The key question is: what do we want
the persisted form of a \texttt{Resort} object to look like? Any way of
writing all the necessary information in a way that we can unambiguously get
it back will do. I vote for this:

\begin{Verbatim}[fontsize=\small,samepage=true,frame=lines]
Westword Spa and Surf
312-555-1234
5 stars
$$$
This luxurious beach property
is a pleasure for all who visit.
Dogs and cats welcome too!
.
\end{Verbatim}

The textual representation of a \texttt{Resort} consists of the following
parts:

\begin{compactenum}
\item A single line of text giving the resort's name.
\item A single line of text giving the resort's telephone number.
\item A single line of text with a digit from 1 to 5, followed by a space and
the String ``\texttt{stars}'' (or ``\texttt{star}'').
\item A single line of text with dollar signs, the number of which indicates
the expense of the resort (on a scale of \$ to \$\$\$\$).
\item One or more lines of text giving a description of the resort.
\item A line containing only a period. \label{item:delimiter}
\end{compactenum}

\index{delimiter}
As you'll remember from your previous programming courses, that last item
(\#\ref{item:delimiter}) is called a \textbf{delimiter} since it ``delimits''
(marks the end of) the resort entry.

Now, for the Java class. Figure~\ref{fig:resort1} \textit{almost} has what we
want. Look at it carefully.

\begin{figure}[ht]
\centering
\begin{Verbatim}[fontsize=\footnotesize,samepage=true,frame=single]
class Resort {
  private String name, desc, phone;
  private int rating, cost;

  Resort(Scanner s) {
      this.name = s.nextLine();
      this.phone = s.nextLine();
      this.rating = s.nextInt();
      s.nextLine();    // read and discard the " stars" part.
      this.cost = s.nextLine().length();
      this.desc = "";
      String next = s.nextLine();
      while (!next.equals(".")) {
          this.desc += next + "\n";
          next = s.nextLine();
      }
  }
  
  void persist(PrintWriter pw) {
      pw.println(this.name);
      pw.println(this.phone);
      pw.println(this.rating + " " +
            (this.rating == 1 ? "star" : "stars"));
      for (int i=0; i<this.cost; i++) {
          pw.print("$");
      }
      pw.println();  // newline for the $$$
      pw.println(this.desc);
      pw.println(".");
  }
}
\end{Verbatim}
\caption{Our first cut at the \texttt{Resort} class.}
\label{fig:resort1}
\end{figure}

\subsubsection{\texttt{Resort}: hydration}

\index{constructor}
\index{cursor}
\index{nextInt@\texttt{.nextInt()}}
\index{nextLine@\texttt{.nextLine()}}
In Figure~\ref{fig:resort1}, the constructor is used for hydration. Perhaps the
trickiest thing about it is that we \textit{pass it a \texttt{Scanner} object}.
Even if you've worked with \texttt{Scanner} before, this may be a new idea to
you: passing one around between objects so that different objects can read
different parts of a file. But that's precisely what we're doing here. Remember
that with file I/O, you have a \textbf{cursor} open to a file, which is always
``at'' a particular location within the file. Reading from the \texttt{Scanner}
(perhaps using one of the \texttt{.next*()} methods like \texttt{.nextInt()} or
\texttt{.nextLine()}) \textbf{advances} the cursor through the file to the next
bit.

So when a client calls ``\texttt{new Resort(someScanner)}'', they're saying
``this \texttt{Scanner}'s cursor is currently positioned right at the
beginning of some resort information. So please hydrate one \texttt{Resort}
object for me by reading just that much of the file.''

The constructor proceeds to do just that. It reads the name, phone number,
rating, cost, and description, in that order, ending when it reaches (and
reads past) the period (``\texttt{.}'') delimiter. I'll let you glance through
the code and see if you agree with my implementation here. There are a few
interesting nuances, like:

\begin{compactitem}

\item When reading the rating (``\texttt{4 stars}'') we call
\texttt{.nextInt()} to read the number part, and then \texttt{.nextLine()} to
read past and discard the rest of the line.

\item When reading the cost (``\texttt{\$\$\$\$}'') we take the length of the
string (the number of dollar signs) and store that number.

\item When reading the possibly-multi-line description, we continually keep a
lookout for the delimiter. As long as the lines we read are \textit{not} the
period on a line by itself, we append them to the end of our \texttt{desc}
inst var, along with the newline character (``\texttt{\textbackslash n}'') that
\texttt{.nextLine()} discards.

\end{compactitem}

All just fiddly stuff.

\subsubsection{\texttt{Resort}: persistence}

The inverse process of our constructor is our \texttt{.persist()} method,
which works similarly: given a \texttt{PrintWriter} -- to which other
information has quite possibly already been written -- write out the text
representation of this \texttt{Resort} object.

Just a few things of note here:

\begin{compactitem}

\index{conditional expression}
\index{? : (conditional expression)}
\item If you haven't seen the wacky syntax on the
``\texttt{this.rating}'' lines, it's worth knowing. It's called a ``conditional
expression'' or a ``question-mark-colon operator'' and is a compact way of
sticking a short \texttt{if}/\texttt{else} onto one line. It means ``if the
boolean expression before the question mark is true, replace this whole
expression with the thing before the colon; otherwise, replace it with the
thing after the colon.'' Less than legible to beginners, but a nice shortcut
for experts.

\item A simple \texttt{for} loop converts the integer \texttt{cost} into a
string of dollar signs. We also have an explicit \texttt{.println()} after
that loop so that we include the line feed after those dollar signs.

\index{delimiter}
\item We explicitly write the delimiter (``\texttt{.}'') at the end of the
description, so that when read back by the hydration process, the constructor
knows where the description stops.

\end{compactitem}

Clearly the persistence and hydration code for a class need to be kept in
lock-step with each other. Adding a new field to the former, for instance,
necessitates us reading that field in the latter.

\subsubsection{Detecting the end of a sequence}

I said above that Figure~\ref{fig:resort1} is \textit{almost} what we want.
We're going to make one change to it. When a client calls ``\texttt{new
Resort(aScanner)}'', it's expecting the constructor to hydrate a
\texttt{Resort} object and hand it over. But consider: what if there are no
more \texttt{Resort} objects in the file?

Let's zoom out a second and appreciate the context in which the
\texttt{Resort} constructor will be called. We've made the design decision to
store all our application's information in a single file. This means that our
save file is going to have lots of resorts in it, back-to-back-to-back, each
separated from the following one by the period delimiter.

This in turn means that our code in \texttt{ReservationSystem} is going to be
calling ``\texttt{new Resort(aScanner)}'' \textit{in a loop} to hydrate all
the objects. This will work great until the point at which we hit the end of
the list. What then?

\index{exception}
In that case, the \texttt{Resort} constructor can't be allowed to finish,
because \textit{it doesn't make sense for an object to be constructed at all}.
So what should the constructor do in that case? Throw an \texttt{Exception},
of course, as we learned in Chapter~\ref{ch:exceptions}. This indicates to the
client that the instantiation \textit{failed} -- there is no \texttt{Resort}
object to be constructed, and the client should do what's appropriate in that
event (in this case, simply stop reading from the file).

\index{inheritance}
\index{noMoreResortsException@\texttt{NoMoreResortsException}}
The exception we throw can either be a ``plain old \texttt{Exception},'' or a
special type of exception that we create for this purpose. The latter approach
requires the technique of \textit{inheritance}, which we won't cover until the
next chapter. But I'll give you a preview. By simply creating a one-line
\texttt{NoMoreResortsException.java} file:

\begin{Verbatim}[fontsize=\small,samepage=true,frame=single]
class NoMoreResortsException extends Exception {}
\end{Verbatim}

voil\`{a}, we can now throw \texttt{NoMoreResortsException}s in addition to
plain old \texttt{Exception}s. This increases code readability (it's obvious
from the class name what this kind of exception indicates) and also allows us
to distinguish this type of exception from other kinds of things that might go
wrong (for instance, a bad filename or other problem accessing the
filesystem).

The only other question is: how does the constructor know when it's reached
the end of the sequence of resorts? Depending on how we choose to structure
the file, it might be when the end of the file is reached, or when some other
delimiter is encountered. Let's do the second case, and say that at the end of
the resorts list, our file will contain a line with these five characters:
``\texttt{-END-}''. When we hit this, we'll know there are no more resorts to be
hydrated.

Figure~\ref{fig:modifiedResort} has the modified \texttt{Resort} constructor.
It's exactly the same except that after reading what it supposes is the
resort's name, the constructor sanity-checks that against the string
``\texttt{--END-}''. If they're equal, it realizes that it didn't read a
resort's name after all, but instead \textit{it hit the end of the list.} In
response, it abandons the rest of the constructor, \textit{refuses to
instantiate an object}, and throws the exception back to the caller.


\begin{figure}[ht]
\centering
\begin{Verbatim}[fontsize=\footnotesize,samepage=true,frame=single]
    Resort(Scanner s) throws NoMoreResortsException {
        this.name = s.nextLine();
        if (this.name.equals("-END-")) {
            throw new NoMoreResortsException();
        }
        this.phone = s.nextLine();
        this.rating = s.nextInt();
        s.nextLine();    // read and discard the " stars" part.
        this.cost = s.nextLine().length();
        this.desc = "";
        String next = s.nextLine();
        while (!next.equals(".")) {
            this.desc += next + "\n";
            next = s.nextLine();
        }
    }
\end{Verbatim}
\caption{The modified \texttt{Resort} constructor.}
\label{fig:modifiedResort}
\end{figure}

\subsubsection{\texttt{ReservationSystem}: hydration}

And now, what does our overarching ``read and write the whole file'' code look
like? Again, it depends on the structure of the file itself. Let's say it's
formatted as in Figure~\ref{fig:wholeFile}. This file is comprised of the
following parts:

\begin{compactenum}
\item A line with the resort chain name.
\item The year this data is applicable to, followed by some other text
(``\texttt{ season}'').
\item Another line of preamble (the hyphens), which we just have to read
past.
\item A sequence of resort entries, each of which is delimited with a period.
\item The line ``\texttt{-END-}'' to indicate the end of the sequence.
\item A copyright notice, which we just have to read past.
\end{compactenum}


\begin{figure}[ht]
\centering
\begin{Verbatim}[fontsize=\scriptsize,samepage=true,frame=lines]
Holiday Inn resorts
2019 season
-------------------------
Westword Spa and Surf
312-555-1234
5 stars
$$$
This luxurious beach property is a pleasure
for all who visit. Dogs and cats welcome too!
.
Roadkill Motel
306-555-4444
1 star
$$
The 1970's-style armchairs aren't the problem:
the holes in the walls and the odorific properties
are. Traveler beware!
.
ABC Inn
123-456-7890
3 stars
$$  
Nothin' fancy, nothin' broken. Your basic motel 
for the budget (but hygiene-conscious) traveler.
.   
-END- 
Copyright (C) 2019
\end{Verbatim}
\caption{The file format for our resort reservation system.}
\label{fig:wholeFile}
\end{figure}

\index{constructor}
\index{try/catch@\texttt{try}/\texttt{catch}}

Given this structure, our \texttt{ReservationSystem}'s constructor should work
as in Figure~\ref{fig:reservationSysClass}
(p.~\pageref{fig:reservationSysClass}). Check out that
\texttt{try}/\texttt{catch}/\texttt{while} loop construct very carefully. It's
short, but it's also a doozy. Inside the body of the \texttt{try} block, we
have what appears to be an \textit{infinite} loop: after all,
\texttt{while(true)} means ``forever.'' So we instantiate a \texttt{Resort}
object and add it to our \texttt{ArrayList}, do it again, do it again, do it
again...

\begin{figure}[ht]
\centering
\begin{Verbatim}[fontsize=\footnotesize,samepage=true,frame=single]
public class ReservationSystem {
    private int year;
    private String chainName;
    private ArrayList<Resort> resorts;

    ReservationSystem(String filename) throws Exception {
        this.resorts = new ArrayList<Resort>();
        Scanner s = new Scanner(new FileReader(filename));
        this.chainName = s.nextLine();
        this.year = s.nextInt();
        s.nextLine();    // Read past " season"
        s.nextLine();    // Read past line of hyphens

        // Read all the resorts.
        try {
            while (true) {
                Resort r = new Resort(s);
                this.resorts.add(r);
            }
        } catch (NoResortException e) {
        }

        s.nextLine();    // Read past copyright
    }
}
\end{Verbatim}
\caption{The \texttt{ReservationSystem} class's constructor.}
\label{fig:reservationSysClass}
\end{figure}


But it's not really infinite and here's why: eventually, the \texttt{Resort}
constructor is going to discover that there aren't any more resort entries in
the file. When we ask it to hydrate the fourth \texttt{Resort} from
Figure~\ref{fig:wholeFile}, the constructor won't return: instead, it'll throw
a \texttt{NoMoreResortsException}. When that happens, we pop out of the
\texttt{try} block and into the \texttt{catch} block, \textit{which is outside
the body of the loop}. So we're officially done with the loop at that point,
and carry on to read the copyright information at the end and we're done.

\index{catch@\texttt{catch}}
And what does the \texttt{catch} block do? \textit{Nothing.} That may seem
pretty weird, but a moment's thought will convince you otherwise. What should
we do when we reach the end of the resorts? \textit{Simply carry on.} Even
though it's an \texttt{Exception}, it's not really an ``error.'' It just
means, ``oh, you wanted another \texttt{Resort} object, but actually there
aren't any. Carry on with your other business, Mr.~Client.''

To be crystal clear, here's what happens in the \texttt{try}/\texttt{while}
construct when we read Figure~\ref{fig:wholeFile}:

\begin{compactenum}
\item It hydrates a \texttt{Resort} object, and adds it to the
\texttt{resorts} \texttt{ArrayList}.
\item It hydrates a \texttt{Resort} object, and adds it to the
\texttt{resorts} \texttt{ArrayList}.
\item It hydrates a \texttt{Resort} object, and adds it to the
\texttt{resorts} \texttt{ArrayList}.
\item It tries to hydrate a \texttt{Resort} object...but catches the exception
instead, pops out of the \texttt{try} block (and therefore also the
\texttt{while} loop) entirely, and goes to the \texttt{catch} block.
\item The catch block does nothing.
\item It carries on reading the rest of the file.
\end{compactenum}

Note carefully that \textit{no exception is thrown from the constructor of the
\texttt{ReservationSystem} class in this case!} Rather, the exceptions
thrown by the \textit{\texttt{Resort}} constructor are caught and dealt with
by the \texttt{ReservationSystem} constructor. The only time the
\texttt{ReservationSystem} constructor would throw an \texttt{Exception} is if
something went wrong when trying to read the file (in which case an
\texttt{Exception}, rather than a \texttt{NoMoreResortsException}, would be
generated).

\subsubsection{\texttt{ReservationSystem}: persistence}

Finally, the \texttt{.persist()} method of the \texttt{ReservationSystem}
class:

\begin{Verbatim}[fontsize=\small,samepage=true,frame=single]
    void persist(PrintWriter pw) {
        pw.println(this.chainName);
        pw.println("" + this.year + " season");
        pw.println("-----------------------------------");
        for (Resort r : this.resorts) {
            r.persist(pw);
        }
        pw.println("Copyright (C) " + year);
        pw.close();
    }
\end{Verbatim}

It's quite simple. The main part is simply calling the \texttt{.persist()}
method on each \texttt{Resort} object, so that they can persist themselves.
The stuff before and after is just for the boilerplate.