\documentclass[12pt]{book} \usepackage[T1]{fontenc} \usepackage[utf8]{inputenc} \usepackage{lmodern} \usepackage{hyperref} \usepackage{graphicx} \usepackage[english]{babel} \usepackage{listings} \usepackage{amsmath} \usepackage{amsthm} \usepackage{amssymb} \usepackage{natbib} \usepackage{stmaryrd} \usepackage{xypic} \lstset{% basicstyle=\ttfamily% } %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % 'dedication' environment: To add a dedication paragraph at the start of book % % Source: http://www.tug.org/pipermail/texhax/2010-June/015184.html % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \newenvironment{dedication} { \cleardoublepage \thispagestyle{empty} \vspace*{\stretch{1}} \hfill\begin{minipage}[t]{0.66\textwidth} \raggedright } { \end{minipage} \vspace*{\stretch{3}} \clearpage } %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Chapter quote at the start of chapter % % Source: http://tex.stackexchange.com/a/53380 % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \makeatletter \renewcommand{\@chapapp}{}% Not necessary... \newenvironment{chapquote}[2][2em] {\setlength{\@tempdima}{#1}% \def\chapquote@author{#2}% \parshape 1 \@tempdima \dimexpr\textwidth-2\@tempdima\relax% \itshape} {\par\normalfont\hfill--\ \chapquote@author\hspace*{\@tempdima}\par\bigskip} \makeatother %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \newcommand{\itm}[1]{\ensuremath{\mathit{#1}}} \newcommand{\Stmt}{\itm{stmt}} \newcommand{\Exp}{\itm{exp}} \newcommand{\Instr}{\itm{instr}} \newcommand{\Prog}{\itm{prog}} \newcommand{\Arg}{\itm{arg}} \newcommand{\Int}{\itm{int}} \newcommand{\Var}{\itm{var}} \newcommand{\Op}{\itm{op}} \newcommand{\key}[1]{\texttt{#1}} \newcommand{\READ}{(\key{read})} \newcommand{\UNIOP}[2]{(\key{#1}\,#2)} \newcommand{\BINOP}[3]{(\key{#1}\,#2\,#3)} \newcommand{\LET}[3]{(\key{let}\,([#1\;#2])\,#3)} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \title{\Huge \textbf{Essentials of Compilation} \\ \huge An Incremental Approach} \author{\textsc{Jeremy G. Siek} \thanks{\url{http://homes.soic.indiana.edu/jsiek/}} } \begin{document} \frontmatter \maketitle \begin{dedication} This book is dedicated to the programming languages group at Indiana University. \end{dedication} \tableofcontents %\listoffigures %\listoftables \mainmatter %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \chapter*{Preface} \cite{Sarkar:2004fk} \cite{Keep:2012aa} \cite{Ghuloum:2006bh} %\section*{Structure of book} % You might want to add short description about each chapter in this book. %\section*{About the companion website} %The website\footnote{\url{https://github.com/amberj/latex-book-template}} for %this file contains: %\begin{itemize} % \item A link to (freely downlodable) latest version of this document. % \item Link to download LaTeX source for this document. % \item Miscellaneous material (e.g. suggested readings etc). %\end{itemize} \section*{Acknowledgments} Need to give thanks to \begin{itemize} \item Kent Dybvig \item Daniel P. Friedman \item Oscar Waddell \item Abdulaziz Ghuloum \item Dipanwita Sarkar \end{itemize} %\mbox{}\\ %\noindent Amber Jain \\ %\noindent \url{http://amberj.devio.us/} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \chapter{Integers and Variables} \label{ch:int-exp} %\begin{chapquote}{Author's name, \textit{Source of this quote}} %``This is a quote and I don't know who said this.'' %\end{chapquote} \section{The $S_0$ Language} The $S_0$ language includes integers, operations on integers, (arithmetic and input), and variable definitions. The syntax of the $S_0$ language is defined by the grammar in Figure~\ref{fig:s0-syntax}. This language is rich enough to exhibit several compilation techniques but simple enough so that we can implement a compiler for it in two weeks of hard work. To give the reader a feeling for the scale of this first compiler, the instructor solution for the $S_0$ compiler consists of 6 recursive functions and a few small helper functions that together span 256 lines of code. \begin{figure}[htbp] \centering \fbox{ \begin{minipage}{0.85\textwidth} \[ \begin{array}{lcl} \Op &::=& \key{+} \mid \key{-} \mid \key{*} \mid \key{read} \\ \Exp &::=& \Int \mid (\Op \; \Exp^{*}) \mid \Var \mid \LET{\Var}{\Exp}{\Exp} \end{array} \] \end{minipage} } \caption{The syntax of the $S_0$ language. The abbreviation \Op{} is short for operator, \Exp{} is short for expression, \Int{} for integer, and \Var{} for variable.} \label{fig:s0-syntax} \end{figure} The result of evaluating an expression is a value. For $S_0$, values are integers. To make it straightforward to map these integers onto x86-64 assembly~\citep{Matz:2013aa}, we restrict the integers to just those representable with 64-bits, the range $-2^{63}$ to $2^{63}$. We will walk through some examples of $S_0$ programs, commenting on aspects of the language that will be relevant to compiling it. We start with one of the simplest $S_0$ programs; it adds two integers. \[ \BINOP{+}{10}{32} \] The result is $42$, as you might expected. % The next example demonstrates that expressions may be nested within each other, in this case nesting several additions and negations. \[ \BINOP{+}{10}{ \UNIOP{-}{ \BINOP{+}{12}{20} } } \] What is the result of the above program? The \key{let} construct stores a value in a variable which can then be used within the body of the \key{let}. So the following program stores $32$ in $x$ and then computes $\BINOP{+}{10}{x}$, producing $42$. \[ \LET{x}{ \BINOP{+}{12}{20} }{ \BINOP{+}{10}{x} } \] When there are multiple \key{let}'s for the same variable, the closest enclosing \key{let} is used. Consider the following program with two \key{let}'s that define variables named $x$. \[ \LET{x}{32}{ \BINOP{+}{ \LET{x}{10}{x} }{ x } } \] For the purposes of showing which variable uses correspond to which definitions, the following shows the $x$'s annotated with subscripts to distinguish them. \[ \LET{x_1}{32}{ \BINOP{+}{ \LET{x_2}{10}{x_2} }{ x_1 } } \] The \key{read} operation prompts the user of the program for an integer. Given an input of $10$, the following program produces $42$. \[ \BINOP{+}{(\key{read})}{32} \] We include the \key{read} operation in $S_0$ to demonstrate that order of evaluation can make a different. Given the input $52$ then $10$, the following produces $42$ (and not $-42$). \[ \LET{x}{\READ}{ \LET{y}{\READ}{ \BINOP{-}{x}{y} } } \] The initializing expression is always evaluated before the body of the \key{let}, so in the above, the \key{read} for $x$ is performed before the \key{read} for $y$. % The behavior of the following program is somewhat subtle because Scheme does not specify an evaluation order for arguments of an operator such as $-$. \[ \BINOP{-}{\READ}{\READ} \] Given the input $42$ then $10$, the above program can result in either $42$ or $-42$, depending on the whims of the Scheme implementation. The goal for this chapter is to implement a compiler that translates any program $p \in S_0$ into a x86-64 assembly program $p'$ such that the assembly program exhibits the same behavior on an x86 computer as the $S_0$ program running in a Scheme implementation. \[ \xymatrix{ p \in S_0 \ar[rr]^{\text{compile}} \ar[drr]_{\text{run in Scheme}\quad} && p' \in \text{x86-64} \ar[d]^{\quad\text{run on an x86 machine}}\\ & & n \in \mathbb{Z} } \] In the next section we introduce enough of the x86-64 assembly language to compile $S_0$. \section{x86-64 Assembly} An x86-64 program is a sequence of instructions. The instructions manipulate 16 variables called \emph{registers} and can also load and store values into \emph{memory}. Memory is a mapping of 64-bit addresses to 64-bit values. The syntax $n(r)$ is used to read the address $a$ stored in register $r$ and then offset it by $n$ bytes (8 bits), producing the address $a + n$. The arithmetic instructions, such as $\key{addq}\,s\,d$, read from the source $s$ and destination argument $d$, apply the arithmetic operation, then stores the result in the destination $d$. In this case, computing $d \gets d + s$. The move instruction, $\key{movq}\,s\,d$ reads from $s$ and stores the result in $d$. The $\key{callq}\,\mathit{label}$ instruction executes the procedure specified by the label, which we shall use to implement \key{read}. Figure~\ref{fig:x86-a} defines the syntax for this subset of the x86-64 assembly language. \begin{figure}[tbp] \fbox{ \begin{minipage}{0.96\textwidth} \[ \begin{array}{lcl} \itm{register} &::=& \key{rsp} \mid \key{rbp} \mid \key{rax} \mid \key{rbx} \mid \key{rcx} \mid \key{rdx} \mid \key{rsi} \mid \key{rdi} \mid \\ && \key{r8} \mid \key{r9} \mid \key{r10} \mid \key{r11} \mid \key{r12} \mid \key{r13} \mid \key{r14} \mid \key{r15} \\ \Arg &::=& \key{\$}\Int \mid \key{\%}\itm{register} \mid \Int(\key{\%}\itm{register}) \\ \Instr &::=& \key{addq} \; \Arg \; \Arg \mid \key{subq} \; \Arg \; \Arg \mid \key{imulq} \; \Arg \; \Arg \mid \key{negq} \; \Arg \mid \\ && \key{movq} \; \Arg \; \Arg \mid \key{callq} \; \mathit{label} \mid \key{pushq}\;\Arg \mid \key{popq};\Arg \mid \key{retq} \\ \Prog &::= & \key{.globl \_main}\\ & & \key{\_main:} \; \Instr^{*} \end{array} \] \end{minipage} } \caption{A subset of the x86-64 assembly language.} \label{fig:x86-a} \end{figure} Figure~\ref{fig:p0-x86} depicts an x86-64 program that is equivalent to $\BINOP{+}{10}{32}$. The \key{globl} directive says that the \key{\_main} procedure is externally visible, which is necessary so that the operating system can call it. The label \key{\_main:} indicates the beginning of the \key{\_main} procedure. The instruction $\key{movq}\,\$10, \%\key{rax}$ puts $10$ into the register \key{rax}. The following instruction $\key{addq}\,\key{\$}32, \key{\%rax}$ adds $32$ to the $10$ in \key{rax} and puts the result, $42$, back into \key{rax}. The instruction \key{retq} finishes the \key{\_main} function by returning the integer in the \key{rax} register to the operating system. \begin{figure}[htbp] \centering \begin{minipage}{0.6\textwidth} \begin{lstlisting} .globl _main _main: movq $10, %rax addq $32, %rax retq \end{lstlisting} \end{minipage} \caption{A simple x86-64 program equivalent to $\BINOP{+}{10}{32}$.} \label{fig:p0-x86} \end{figure} The next example exhibits the use of memory. Figure~\ref{fig:p1-x86} lists an x86-64 program that is equivalent to $\BINOP{+}{52}{ \UNIOP{-}{10} }$. To understand how this x86-64 program uses memory, we need to explain a region of memory called called the \emph{procedure call stack} (\emph{stack} for short). The stack consists of a separate \emph{frame} for each procedure call. The memory layout for an individual frame is shown in Figure~\ref{fig:frame}. The register \key{rsp} is called the \emph{stack pointer} and points to the item at the top of the stack. The stack grows downward in memory, so we increase the size of the stack by subtracting from the stack pointer. The frame size is required to be a multiple of 16 bytes. The register \key{rbp} is the \emph{base pointer} which serves two purposes: 1) it saves the location of the stack pointer for the procedure that called the current one and 2) it is used to access variables associated with the current procedure. We number the variables from $1$ to $n$. Variable $1$ is stored at address $-8\key{(\%rbp)}$, variable $2$ at $-16\key{(\%rbp)}$, etc. \begin{figure} \centering \begin{minipage}{0.6\textwidth} \begin{lstlisting} .globl _main _main: pushq %rbp movq %rsp, %rbp subq $16, %rsp movq $10, -8(%rbp) negq -8(%rbp) movq $52, %rax addq -8(%rbp), %rax addq $16, %rsp popq %rbp retq \end{lstlisting} \end{minipage} \caption{An x86-64 program equivalent to $\BINOP{+}{52}{\UNIOP{-}{10} }$.} \label{fig:p1-x86} \end{figure} \begin{figure} \centering \begin{tabular}{|r|l|} \hline Position & Contents \\ \hline 8(\key{\%rbp}) & return address \\ 0(\key{\%rbp}) & old \key{rbp} \\ -8(\key{\%rbp}) & variable $1$ \\ -16(\key{\%rbp}) & variable $2$ \\ \ldots & \ldots \\ 0(\key{\%rsp}) & variable $n$\\ \hline \end{tabular} \caption{Memory layout of a frame.} \label{fig:frame} \end{figure} Getting back to the program in Figure~\ref{fig:p1-x86}, the first three instructions are the typical prelude for a procedure. The instruction \key{pushq \%rbp} saves the base pointer for the procedure that called the current one onto the stack and subtracts $8$ from the stack pointer. The second instruction \key{movq \%rsp, \%rbp} changes the base pointer to the top of the stack. The instruction \key{subq \$16, \%rsp} moves the stack pointer down to make enough room for storing variables. This program just needs one variable ($8$ bytes) but because the frame size is required to be a multiple of 16 bytes, it rounds to 16 bytes. The next four instructions carry out the work of computing $\BINOP{+}{52}{\UNIOP{-}{10} }$. The first instruction \key{movq \$10, -8(\%rbp)} stores $10$ in variable $1$. The instruction \key{negq -8(\%rbp)} changes variable $1$ to $-10$. The \key{movq \$52, \%rax} places $52$ in the register \key{rax} and \key{addq -8(\%rbp), \%rax} adds the contents of variable $1$ to \key{rax}, at which point \key{rax} contains $42$. The last three instructions are the typical conclusion of a procedure. The \key{addq \$16, \%rsp} instruction moves the stack pointer back to point at the old base pointer. The amount added here needs to match the amount that was subtracted in the prelude of the procedure. Then \key{popq \%rbp} returns the old base pointer to \key{rbp} and adds $8$ to the stack pointer. The \key{retq} instruction jumps back to the procedure that called this one and subtracts 8 from the stack pointer. \section{Planning the route from $S_0$ to x86-64} To compile one language to another it helps to focus on the differences between the two languages. It is these differences that the compiler will need to bridge. What are the differences between $S_0$ and x86-64 assembly? Here we list some of the most important the differences. \begin{enumerate} \item Variables in $S_0$ can overshadow other variables with the same name. The registers and memory locations of x86-64 all have unique names. \item An argument to an $S_0$ operator can be any expression, whereas x86-64 instructions restrict their arguments to integers, registers, and memory locations. \item x86-64 arithmetic instructions typically take two arguments and update the second argument in place. In contrast, $S_0$ arithmetic operations only read their arguments and produce a new value. \item An $S_0$ program can have any number of variables whereas x86-64 has only 16 registers. \end{enumerate} We ease the challenge of compiling from $S_0$ to x86 by breaking down the problem into several steps, dealing with the above differences one at a time. Further, we identify an intermediate language named $C_0$, roughly half-way between $S_0$ and x86, to provide a rest stop along the way. The name $C_0$ comes from this language being vaguely similar to the $C$ language. first two differences discussed above, regarding variables and nested expressions, are handled by the passes \textsf{uniquify} and \textsf{flatten} that bring us to $C_0$. \[\large \xymatrix@=55pt{ S_0 \ar[r]^-{\textsf{uniquify}} & S_0 \ar[r]^-{\textsf{flatten}} & C_0 } \] The syntax for $C_0$ is defined in Figure~\ref{fig:c0-syntax}. The $C_0$ language supports the same operators as $S_0$ but the arguments of operators are now restricted to just variables and integers. The \key{let} construct of $S_0$ is replaced by an assignment statement and there is a \key{return} construct to specify the return value of the program. A program consists of a sequence of statements that include at least one \key{return} statement. \begin{figure}[htbp] \[ \begin{array}{lcl} \Arg &::=& \Int \mid \Var \\ \Exp &::=& \Arg \mid (\Op \; \Arg^{*})\\ \Stmt &::=& (\key{assign} \; \Var \; \Exp) \mid (\key{return}\; \Exp) \\ \Prog & ::= & \Stmt^{+} \end{array} \] \caption{The $C_0$ intermediate language.} \label{fig:c0-syntax} \end{figure} To get from $C_0$ to x86-64 assembly requires three more steps, which we discuss below. \[\large \xymatrix@=55pt{ C_0 \ar[r]^-{\textsf{select\_instr.}} & \text{x86}^{*} \ar[r]^-{\textsf{assign\_homes}} & \text{x86}^{*} \ar[r]^-{\textsf{spill\_code}} & \text{x86} } \] We handle the third difference listed above, concerning the format of arithmetic instructions, in the \textsf{select\_instructions} pass. The result of this pass produces programs consisting of x86-64 instructions that use variables. % As there are only 16 registers, we cannot always map variables to registers. Fortunately, the stack can grow arbitrarily, so we can always map variables to locations on the stack. This is handled in the \textsf{assign\_homes} pass. The topic of Chapter~\ref{ch:register-allocation} is implementing a smarter approach in which we make a best-effort to map variables to registers, resorting to the stack only when necessary. % The final pass in our journey to x86 handles an indiosycracy of x86 assembly. Many x86 instructions have two arguments but only one of the arguments may be a memory reference. Because we are mapping variables to stack locations, many of our generated instructions will violate this restriction. The purpose of the \textsf{spill\_code} pass is to patch up this problem by replacing each bad instructions with a short sequence of instructions that use the \key{rax} register. \section{Uniquify} The purpose of this pass is to make sure that each \key{let} uses a unique variable name. For example, the \textsf{uniquify} pass could translate \[ \LET{x}{32}{ \BINOP{+}{ \LET{x}{10}{x} }{ x } } \] to \[ \LET{x.1}{32}{ \BINOP{+}{ \LET{x.2}{10}{x.2} }{ x.1 } } \] We recommend implementing \textsf{uniquify} as a recursive function that mostly just copies the input program. However, when encountering a \key{let}, it should generate a unique name for the variable (the Racket function \key{gensym} is handy for this) and associate the old name with the new unique name in an association list. The \textsf{uniquify} function will need to access this association list when it gets to a variable reference, so we add another paramter to \textsf{uniquify} for the association list. \section{Flatten} \section{Select Instructions} \section{Assign Homes} \section{Spill Code} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \chapter{Register Allocation} \label{ch:register-allocation} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \chapter{Booleans, Conditions, and Type Checking} \label{ch:bool-types} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \chapter{Tuples and Heap Allocation} \label{ch:tuples} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \chapter{Functions} \label{ch:functions} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \chapter{Lexically Scoped Functions} \label{ch:lambdas} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \chapter{The Dynamic Type} \label{ch:type-dynamic} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \chapter{Mutable Lists} \label{ch:mutable-lists} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \chapter{Parametric Polymorphism} \label{ch:parametric-polymorphism} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \chapter{High-level Optimization} \label{ch:high-level-optimization} \bibliographystyle{plainnat} \bibliography{all} \end{document} %% LocalWords: Dybvig Waddell Abdulaziz Ghuloum Dipanwita %% LocalWords: Sarkar lcl Matz aa representable