From b36adc8686938af533f3dfce6368f74c37016186 Mon Sep 17 00:00:00 2001 From: Ryan Bernstein Date: Wed, 19 Aug 2020 09:04:29 -0400 Subject: [PATCH 01/11] Added docs for compiler optimizations --- src/reference-manual/_bookdown.yml | 1 + .../compiler-optimizations.Rmd | 504 ++++++++++++++++++ 2 files changed, 505 insertions(+) create mode 100644 src/reference-manual/compiler-optimizations.Rmd diff --git a/src/reference-manual/_bookdown.yml b/src/reference-manual/_bookdown.yml index 1355e15a3..28ce0834d 100644 --- a/src/reference-manual/_bookdown.yml +++ b/src/reference-manual/_bookdown.yml @@ -30,6 +30,7 @@ rmd_files: [ "usage.Rmd", "reproducibility.Rmd", + "compiler-optimizations.Rmd", "licenses.Rmd", "references.Rmd" diff --git a/src/reference-manual/compiler-optimizations.Rmd b/src/reference-manual/compiler-optimizations.Rmd new file mode 100644 index 000000000..be16c447c --- /dev/null +++ b/src/reference-manual/compiler-optimizations.Rmd @@ -0,0 +1,504 @@ +The Stanc3 compiler can attempt to optimize a Stan program as it is compiled. The optimized program has the same behavior as the unoptimized program, but it may be faster, more memory efficient, or more numerically stable. + +This section introduces the available optimization options and describes their effect. + +You can see a printout of a representation of the Stan program after the optimizations have been applied with the Stanc3 command-line option `--debug-optimized-mir-pretty`. You can see an analogous representation of the program before optimizations have been applied with `--debug-transformed-mir-pretty`. + + +# Optimization levels + +To turn on optimizations, the user specifies the desired optimization *level*. The level specifies the set of optimizations which are turned on. Optimizations which are turned on are then applied in a specific order, and some are applied repeatedly. + +Optimization levels are specified by the numbers 0-4: + +- **O0** No optimizations are applied. +- **O1** Only optimizations which are simple, do not dramatically change the program, and are unlikely noticeably slow down compile times are applied. +- **O2** All optimizations are applied which are unlikely to significantly increase the size of the output program. +- **O3** All optimizations are applied. + +The levels include these optimizations: + +- **O0** includes no optimizations. +- **O1** includes: + - Dead code elimination + - Auto-differentiation level optimization +- **O2** includes optimizations specified by **O1** and also: + - One step loop unrolling + - Constant propagation + - Expression propagation + - Copy propagation + - Partial evaluation + - Lazy code motion +- **O3** includes optimizations specified by **O2** and also: + - Function inlining + - Static loop unrolling + +In addition, **O3** will apply more repetitions of the optimizations, which may increase compile times. + + +# The `--optimize-numerically-close` option + +Using the `--optimize-numerically-close` option will disallow all optimizations which are likely to result in a program with nontrivial numerical differences from the unoptimized program. Some code transformations, such as replacing `log(1-x)` with the builtin `log1m(x)`, may result in slight numerical differences that are detectable when sampling from the program with a fixed random seed. While these transformations do not result in incorrect code, and infact are often more stable than the original code, they are sometimes undesirable for testing purposes. + + +# Optimization descriptions + + +## **O1** Optimizations + + +### Dead code elimination + +Dead code is code that does not have any effect on the behavior of the program. Code is not dead if it affects `target`, the value of any outside-observable variable like transformed parameters or generated quantities, or side effects such as print statements. Removing dead code can speed up a program by avoiding unnecessary computations. + +Example Stan program: + +``` +model { + int i; + i = 5; + for (j in 1:10); + if (0) { + print("Dead code"); + } else { + print("Hi!"); + } +} +``` + +Compiler representation of program **before dead code elimination** (simplified from the output of `--debug-transformed-mir-pretty`): + +``` +log_prob { + int i = 5; + for(j in 1:10) { + ; + } + if(0) { + FnPrint__("Dead code"); + } else { + FnPrint__("Hi!"); + } +} +``` + +Compiler representation of program **after dead code elimination** (simplified from the output of `--debug-optimized-mir-pretty`): + +``` +log_prob { + int i; + FnPrint__("Hi!"); +} +``` + + +### Auto-differentiation level optimization + +Stan variables can have two auto-differentiation (AD) *levels*: AD or non-AD. AD variables carry gradient information with them, which allows Stan to calculate the log-density gradient, but they also have more overhead than non-AD variables. It is therefore inefficient for a variable to be AD unnecessarily. AD-level optimization sets every variable to be non-AD unless its gradient is necessary. + +Example Stan program: + +``` +data { + real y; +} +model { + real x = y + 1; +} +``` + +Compiler representation of program **before AD-level optimization** (simplified from the output of `--debug-transformed-mir-pretty`): + +``` +input_vars { + real y; +} + +log_prob { + real x = (y + 1); +} +``` + +Compiler representation of program **after AD-level optimization** (simplified from the output of `--debug-optimized-mir-pretty`): + +``` +input_vars { + real y; +} + +log_prob { + data real x = (y + 1); +} +``` + + +## **O2** Optimizations + + +### One step loop unrolling + +One step loop unrolling is similar to [static loop unrolling](#orge4599d2), but it only 'unrolls' the first iteration of a loop, and can therefore work even when the total number of iterations is not predictable. This can speed up a program by providing more opportunities for further optimizations such as partial evaluation and lazy code motion. + +Example Stan program: + +``` +data { + int n; +} +transformed data { + int x = 0; + for (i in 1:n) { + x += i; + } +} +``` + +Compiler representation of program **before one step static loop unrolling** (simplified from the output of `--debug-transformed-mir-pretty`): + +``` +prepare_data { + data int n = FnReadData__("n")[1]; + data int x = 0; + for(i in 1:n) { + x = (x + i); + } +} +``` + +Compiler representation of program **after one step static loop unrolling** (simplified from the output of `--debug-optimized-mir-pretty`): + +``` +prepare_data { + data int n = FnReadData__("n")[1]; + int x = 0; + if((n >= 1)) { + x = (x + 1); + for(i in (1 + 1):n) { + x = (x + i); + } + } +} +``` + + +### Constant propagation + +Constant propagation replaces uses of a variable which is known to have a constant value `C` with that constant `C`. This removes the overhead of looking up the variable, and also makes many other optimizations possible (such as static loop unrolling and partial evaluation). + +Example Stan program: + +``` +transformed data { + int n = 100; + int a[n]; + for (i in 1:n) { + a[i] = i; + } +} +``` + +Compiler representation of program **before constant propagation** (simplified from the output of `--debug-transformed-mir-pretty`): + +``` +prepare_data { + data int n = 100; + data array[int, n] a; + for(i in 1:n) { + a[i] = i; + } +} +``` + +Compiler representation of program **after constant propagation** (simplified from the output of `--debug-optimized-mir-pretty`): + +``` +prepare_data { + data int n = 100; + data array[int, 100] a; + for(i in 1:100) { + a[i] = i; + } +} +``` + + +### Expression propagation + + Constant propagation replaces uses of a variable which is known to have a constant value `E` with that constant `E`. This often results in recalculation of the expression, but provides more opportunities for further optimizations such as partial evaluation. Expression propagation is always followed by [lazy code motion](#org0c605fa) to avoid unnecessarily recomputing expressions. + +Example Stan program: + +``` +data { + int m; +} +transformed data { + int n = m+1; + int a[n]; + for (i in 1:n-1) { + a[i] = i; + } +} +``` + +Compiler representation of program **before expression propagation** (simplified from the output of `--debug-transformed-mir-pretty`): + +``` +prepare_data { + data int m = FnReadData__("m")[1]; + data int n = (m + 1); + data array[int, n] a; + for(i in 1:(n - 1)) { + a[i] = i; + } +} +``` + +Compiler representation of program **after expression propagation** (simplified from the output of `--debug-optimized-mir-pretty`): + +``` +prepare_data { + data int m = FnReadData__("m")[1]; + data int n = (m + 1); + data array[int, (m + 1)] a; + for(i in 1:((m + 1) - 1)) { + a[i] = i; + } +} +``` + + +### Copy propagation + +Copy propagation is similar to [expression propagation](#org4e2cadd), but only propagates variables rather than arbitrary expressions. This can reduce the complexity of the code for other optimizations such as expression propagation. + +Example Stan program: + +``` +model { + int i = 1; + int j = i; + int k = i + j; +} +``` + +Compiler representation of program **before copy propagation** (simplified from the output of `--debug-transformed-mir-pretty`): + +``` +log_prob { + int i = 1; + int j = i; + int k = (i + j); +} +``` + +Compiler representation of program **after copy propagation** (simplified from the output of `--debug-optimized-mir-pretty`): + +``` +log_prob { + int i = 1; + int j = i; + int k = (i + i); +} +``` + + +### Partial evaluation + +Partial evaluation searches for expressions that can be replaced with a faster, simpler, more memory efficient, or more numerically stable expression that has the same meaning. + +Example Stan program: + +``` +model { + real a = 1 + 1; + real b = log(1 - a); + real c = a + b * 5; +} +``` + +Compiler representation of program **before partial evaluation** (simplified from the output of `--debug-transformed-mir-pretty`): + +``` +log_prob { + real a = (1 + 1); + real b = log((1 - a)); + real c = (a + (b * 5)); +} +``` + +Compiler representation of program **after partial evaluation** (simplified from the output of `--debug-optimized-mir-pretty`): + +``` +log_prob { + real a = 2; + real b = log1m(a); + real c = fma(b, 5, a); +} +``` + + +### Lazy code motion + + Lazy code motion rearranges the statements and expressions in a program with the goals of: + +- Avoiding computing expressions more than once, and +- Computing expressions as late as possible (to minimize the strain on the working memory set). + +To accomplish these goals, lazy code motion will perform optimizations such as: + +- Moving a repeatedly calculated expression its own variable (also referred to as *common-subexpression elimination*) +- Moving an expression outside of a loop, if it doesn't need to be in the loop (also referred to as *loop-invariant code motion*) + +Lazy code motion can make some programs significantly more efficient by avoiding redundant or early computations. + +Example Stan program: + +``` +model { + real x; + real y; + real z; + + for (i in 1:10) { + x = sqrt(10); + y = sqrt(i); + } + z = sqrt(10); +} +``` + +Compiler representation of program \*before lazy code motion (simplified from the output of `--debug-transformed-mir-pretty`): + +``` +log_prob { + real x; + real y; + real z; + for(i in 1:10) { + x = sqrt(10); + y = sqrt(i); + } + z = sqrt(10); +} +``` + +Compiler representation of program \*after lazy code motion (simplified from the output of `--debug-optimized-mir-pretty`): + +``` +log_prob { + data real lcm_sym4__; + data real lcm_sym3__; + real x; + real y; + lcm_sym4__ = sqrt(10); + real z; + for(i in 1:10) { + x = lcm_sym4__; + y = sqrt(i); + } + z = lcm_sym4__; +} +``` + + +## **O3** Optimizations + + +### Function inlining + +Function inlining replaces each function call to each user-defined function `f` with the body of `f`. It does this by copying the function body to the call site and doing the appropriate renaming of the argument variables. This can speed up a program by avoiding the overhead of a function call and providing more opportunities for further optimizations (such as partial evaluation). + +Example Stan program: + +``` +functions { + int incr(int x) { + int y = 1; + return x + y; + } +} +transformed data { + int a = 2; + int b = incr(a); +} +``` + +Compiler representation of program **before function inlining** (simplified from the output of `--debug-transformed-mir-pretty`): + +``` +functions { + int incr(int x) { + int y = 1; + return (x + y); + } +} + +prepare_data { + data int a = 2; + data int b = incr(a); +} +``` + +Compiler representation of program **after function inlining** (simplified from the output of `--debug-optimized-mir-pretty`): + +``` +prepare_data { + data int a; + a = 2; + data int b; + data int inline_sym1__; + data int inline_sym3__; + inline_sym3__ = 0; + for(inline_sym4__ in 1:1) { + int inline_sym2__; + inline_sym2__ = 1; + inline_sym3__ = 1; + inline_sym1__ = (a + inline_sym2__); + break; + } + b = inline_sym1__; +} +``` + +In this code, the `for` loop and `break` is used to simulate the behavior of a `return` statement. The value to be returned is held in `inline_sym1__`. The flag variable `inline_sym3__` indicates whether a return has occurred and is necessary to handle `return` statements nested inside loops within the function body. + + +### Static loop unrolling + + Static loop unrolling takes a loop that has a predictable number of iterations `X` and replaces it by writing out the loop body `X` times. The loop index in each repeat is replaced with the appropriate constant. This can speed up a program by avoiding the overhead of a loop and providing more opportunities for further optimizations (such as partial evaluation). + +Example Stan program: + +``` +transformed data { + int x = 0; + for (i in 1:4) { + x += i; + } +} +``` + +Compiler representation of program **before static loop unrolling** (simplified from the output of `--debug-transformed-mir-pretty`): + +``` +prepare_data { + data int x = 0; + for(i in 1:4) { + x = (x + i); + } +} +``` + +Compiler representation of program **after static loop unrolling** (simplified from the output of `--debug-optimized-mir-pretty`): + +``` +prepare_data { + data int x; + x = 0; + x = (x + 1); + x = (x + 2); + x = (x + 3); + x = (x + 4); +} +``` From 01be048e0e7bea2976d3ba14eca46f9ec620397b Mon Sep 17 00:00:00 2001 From: Steve Bronder Date: Thu, 23 Dec 2021 14:12:45 -0500 Subject: [PATCH 02/11] fix wording --- src/reference-manual/compiler-optimizations.Rmd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/reference-manual/compiler-optimizations.Rmd b/src/reference-manual/compiler-optimizations.Rmd index d53a2f522..d12acc94b 100644 --- a/src/reference-manual/compiler-optimizations.Rmd +++ b/src/reference-manual/compiler-optimizations.Rmd @@ -9,7 +9,7 @@ You can see a printout of a representation of the Stan program after the optimiz To turn on optimizations, the user specifies the desired optimization *level*. The level specifies the set of optimizations which are turned on. Optimizations which are turned on are then applied in a specific order, and some are applied repeatedly. -Optimization levels are specified by the numbers 0-4: +Optimization levels are specified by the numbers 0-1 and the 'experimental' tag: - **O0** No optimizations are applied. - **O1** Only optimizations which are simple, do not dramatically change the program, and are unlikely noticeably slow down compile times are applied. From 11a737620dfdef7584f2531ee775b2b8fcf1589f Mon Sep 17 00:00:00 2001 From: Steve Bronder Date: Thu, 23 Dec 2021 14:18:13 -0500 Subject: [PATCH 03/11] fix wording --- .../compiler-optimizations.Rmd | 221 +++++++++--------- 1 file changed, 108 insertions(+), 113 deletions(-) diff --git a/src/reference-manual/compiler-optimizations.Rmd b/src/reference-manual/compiler-optimizations.Rmd index d12acc94b..592c01cbb 100644 --- a/src/reference-manual/compiler-optimizations.Rmd +++ b/src/reference-manual/compiler-optimizations.Rmd @@ -83,6 +83,79 @@ log_prob { } ``` +#### Constant propagation + +Constant propagation replaces uses of a variable which is known to have a constant value `C` with that constant `C`. This removes the overhead of looking up the variable, and also makes many other optimizations possible (such as static loop unrolling and partial evaluation). + +Example Stan program: + +``` +transformed data { + int n = 100; + int a[n]; + for (i in 1:n) { + a[i] = i; + } +} +``` + +Compiler representation of program **before constant propagation** (simplified from the output of `--debug-transformed-mir-pretty`): + +``` +prepare_data { + data int n = 100; + data array[int, n] a; + for(i in 1:n) { + a[i] = i; + } +} +``` + +Compiler representation of program **after constant propagation** (simplified from the output of `--debug-optimized-mir-pretty`): + +``` +prepare_data { + data int n = 100; + data array[int, 100] a; + for(i in 1:100) { + a[i] = i; + } +} +``` + +#### Copy propagation + +Copy propagation is similar to [expression propagation](#org4e2cadd), but only propagates variables rather than arbitrary expressions. This can reduce the complexity of the code for other optimizations such as expression propagation. + +Example Stan program: + +``` +model { + int i = 1; + int j = i; + int k = i + j; +} +``` + +Compiler representation of program **before copy propagation** (simplified from the output of `--debug-transformed-mir-pretty`): + +``` +log_prob { + int i = 1; + int j = i; + int k = (i + j); +} +``` + +Compiler representation of program **after copy propagation** (simplified from the output of `--debug-optimized-mir-pretty`): + +``` +log_prob { + int i = 1; + int j = i; + int k = (i + i); +} +``` #### Auto-differentiation level optimization @@ -123,6 +196,41 @@ log_prob { } ``` +#### Partial evaluation + +Partial evaluation searches for expressions that can be replaced with a faster, simpler, more memory efficient, or more numerically stable expression that has the same meaning. + +Example Stan program: + +``` +model { + real a = 1 + 1; + real b = log(1 - a); + real c = a + b * 5; +} +``` + +Compiler representation of program **before partial evaluation** (simplified from the output of `--debug-transformed-mir-pretty`): + +``` +log_prob { + real a = (1 + 1); + real b = log((1 - a)); + real c = (a + (b * 5)); +} +``` + +Compiler representation of program **after partial evaluation** (simplified from the output of `--debug-optimized-mir-pretty`): + +``` +log_prob { + real a = 2; + real b = log1m(a); + real c = fma(b, 5, a); +} +``` + + ### **Oexperimental** Optimizations @@ -172,48 +280,6 @@ prepare_data { } ``` - -#### Constant propagation - -Constant propagation replaces uses of a variable which is known to have a constant value `C` with that constant `C`. This removes the overhead of looking up the variable, and also makes many other optimizations possible (such as static loop unrolling and partial evaluation). - -Example Stan program: - -``` -transformed data { - int n = 100; - int a[n]; - for (i in 1:n) { - a[i] = i; - } -} -``` - -Compiler representation of program **before constant propagation** (simplified from the output of `--debug-transformed-mir-pretty`): - -``` -prepare_data { - data int n = 100; - data array[int, n] a; - for(i in 1:n) { - a[i] = i; - } -} -``` - -Compiler representation of program **after constant propagation** (simplified from the output of `--debug-optimized-mir-pretty`): - -``` -prepare_data { - data int n = 100; - data array[int, 100] a; - for(i in 1:100) { - a[i] = i; - } -} -``` - - #### Expression propagation Constant propagation replaces uses of a variable which is known to have a constant value `E` with that constant `E`. This often results in recalculation of the expression, but provides more opportunities for further optimizations such as partial evaluation. Expression propagation is always followed by [lazy code motion](#org0c605fa) to avoid unnecessarily recomputing expressions. @@ -259,77 +325,6 @@ prepare_data { } ``` - -#### Copy propagation - -Copy propagation is similar to [expression propagation](#org4e2cadd), but only propagates variables rather than arbitrary expressions. This can reduce the complexity of the code for other optimizations such as expression propagation. - -Example Stan program: - -``` -model { - int i = 1; - int j = i; - int k = i + j; -} -``` - -Compiler representation of program **before copy propagation** (simplified from the output of `--debug-transformed-mir-pretty`): - -``` -log_prob { - int i = 1; - int j = i; - int k = (i + j); -} -``` - -Compiler representation of program **after copy propagation** (simplified from the output of `--debug-optimized-mir-pretty`): - -``` -log_prob { - int i = 1; - int j = i; - int k = (i + i); -} -``` - - -#### Partial evaluation - -Partial evaluation searches for expressions that can be replaced with a faster, simpler, more memory efficient, or more numerically stable expression that has the same meaning. - -Example Stan program: - -``` -model { - real a = 1 + 1; - real b = log(1 - a); - real c = a + b * 5; -} -``` - -Compiler representation of program **before partial evaluation** (simplified from the output of `--debug-transformed-mir-pretty`): - -``` -log_prob { - real a = (1 + 1); - real b = log((1 - a)); - real c = (a + (b * 5)); -} -``` - -Compiler representation of program **after partial evaluation** (simplified from the output of `--debug-optimized-mir-pretty`): - -``` -log_prob { - real a = 2; - real b = log1m(a); - real c = fma(b, 5, a); -} -``` - - #### Lazy code motion Lazy code motion rearranges the statements and expressions in a program with the goals of: From 1519b984da88cc50ce451bdcadc0d5e479e6ad12 Mon Sep 17 00:00:00 2001 From: Ryan Bernstein Date: Tue, 18 Jan 2022 14:05:11 -0500 Subject: [PATCH 04/11] Specify optimization level default, fix typos --- src/reference-manual/compiler-optimizations.Rmd | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/src/reference-manual/compiler-optimizations.Rmd b/src/reference-manual/compiler-optimizations.Rmd index 592c01cbb..57a2b9c6f 100644 --- a/src/reference-manual/compiler-optimizations.Rmd +++ b/src/reference-manual/compiler-optimizations.Rmd @@ -9,12 +9,14 @@ You can see a printout of a representation of the Stan program after the optimiz To turn on optimizations, the user specifies the desired optimization *level*. The level specifies the set of optimizations which are turned on. Optimizations which are turned on are then applied in a specific order, and some are applied repeatedly. -Optimization levels are specified by the numbers 0-1 and the 'experimental' tag: +Optimization levels are specified by the numbers 0 and 1 and the 'experimental' tag: - **O0** No optimizations are applied. - **O1** Only optimizations which are simple, do not dramatically change the program, and are unlikely noticeably slow down compile times are applied. - **Oexperimental** All optimizations are applied. Some of these are not thorougly tested and may not always improve a programs performance. +O0 is the default setting. + The levels include these optimizations: - **O0** includes no optimizations. @@ -334,7 +336,7 @@ prepare_data { To accomplish these goals, lazy code motion will perform optimizations such as: -- Moving a repeatedly calculated expression its own variable (also referred to as *common-subexpression elimination*) +- Moving a repeatedly calculated expression to its own variable (also referred to as *common-subexpression elimination*) - Moving an expression outside of a loop if it does not need to be in the loop (also referred to as *loop-invariant code motion*) Lazy code motion can make some programs significantly more efficient by avoiding redundant or early computations. From a15e8fca3a19d6ae173e9cc3bd9542f5fa900f13 Mon Sep 17 00:00:00 2001 From: rybern Date: Tue, 18 Jan 2022 14:07:35 -0500 Subject: [PATCH 05/11] Adjust wording of optimization doc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Rok Češnovar --- src/reference-manual/compiler-optimizations.Rmd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/reference-manual/compiler-optimizations.Rmd b/src/reference-manual/compiler-optimizations.Rmd index 57a2b9c6f..56ae4bca2 100644 --- a/src/reference-manual/compiler-optimizations.Rmd +++ b/src/reference-manual/compiler-optimizations.Rmd @@ -1,4 +1,4 @@ -The Stanc3 compiler can attempt to optimize a Stan program as it is compiled. The optimized program has the same behavior as the unoptimized program, but it may be faster, more memory efficient, or more numerically stable. +The Stanc3 compiler can optimize the code of Stan model during compilation. The optimized model code behaves the same as unoptimized code, but it may be faster, more memory efficient, or more numerically stable. This section introduces the available optimization options and describes their effect. From 8017df068928271ee2730bb347235b6867717151 Mon Sep 17 00:00:00 2001 From: Ryan Bernstein Date: Tue, 18 Jan 2022 14:23:48 -0500 Subject: [PATCH 06/11] Adjust wording of optimization doc --- src/reference-manual/compiler-optimizations.Rmd | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/reference-manual/compiler-optimizations.Rmd b/src/reference-manual/compiler-optimizations.Rmd index 57a2b9c6f..843e36f26 100644 --- a/src/reference-manual/compiler-optimizations.Rmd +++ b/src/reference-manual/compiler-optimizations.Rmd @@ -2,8 +2,7 @@ The Stanc3 compiler can attempt to optimize a Stan program as it is compiled. Th This section introduces the available optimization options and describes their effect. -You can see a printout of a representation of the Stan program after the optimizations have been applied with the Stanc3 command-line option `--debug-optimized-mir-pretty`. You can see an analogous representation of the program before optimizations have been applied with `--debug-transformed-mir-pretty`. - +To print out a representation of the optimized Stan program, use the Stanc3 command-line flag `--debug-optimized-mir-pretty`. To see an analagous representation of the Stan program prior to optimization, use the flag `--debug-transformed-mir-pretty`. # Optimization levels From e15fd238c06b9701c5edfc1eab5bb903e83ab492 Mon Sep 17 00:00:00 2001 From: Ryan Bernstein Date: Tue, 18 Jan 2022 14:24:29 -0500 Subject: [PATCH 07/11] Adjust wording of optimization doc --- src/reference-manual/compiler-optimizations.Rmd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/reference-manual/compiler-optimizations.Rmd b/src/reference-manual/compiler-optimizations.Rmd index 08699a9be..894d060d7 100644 --- a/src/reference-manual/compiler-optimizations.Rmd +++ b/src/reference-manual/compiler-optimizations.Rmd @@ -2,7 +2,7 @@ The Stanc3 compiler can optimize the code of Stan model during compilation. The This section introduces the available optimization options and describes their effect. -To print out a representation of the optimized Stan program, use the Stanc3 command-line flag `--debug-optimized-mir-pretty`. To see an analagous representation of the Stan program prior to optimization, use the flag `--debug-transformed-mir-pretty`. +To print out a representation of the optimized Stan program, use the Stanc3 command-line flag `--debug-optimized-mir-pretty`. To print an analagous representation of the Stan program prior to optimization, use the flag `--debug-transformed-mir-pretty`. # Optimization levels From 00f2854f9ce3cb5d039b461fbf6c7448396adec7 Mon Sep 17 00:00:00 2001 From: rybern Date: Tue, 18 Jan 2022 14:25:20 -0500 Subject: [PATCH 08/11] Adjust wording in optimization doc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Rok Češnovar --- src/reference-manual/compiler-optimizations.Rmd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/reference-manual/compiler-optimizations.Rmd b/src/reference-manual/compiler-optimizations.Rmd index 894d060d7..7c6677f49 100644 --- a/src/reference-manual/compiler-optimizations.Rmd +++ b/src/reference-manual/compiler-optimizations.Rmd @@ -6,7 +6,7 @@ To print out a representation of the optimized Stan program, use the Stanc3 comm # Optimization levels -To turn on optimizations, the user specifies the desired optimization *level*. The level specifies the set of optimizations which are turned on. Optimizations which are turned on are then applied in a specific order, and some are applied repeatedly. +To turn optimizations on, the user specifies the desired optimization *level*. The level specifies the set of optimizations to use. The chosen optimizations are used in a specific order, with some of them applied repeatedly. Optimization levels are specified by the numbers 0 and 1 and the 'experimental' tag: From 71d442e60a4c26ddeee292de22f3a54dccc583af Mon Sep 17 00:00:00 2001 From: rybern Date: Tue, 18 Jan 2022 14:25:38 -0500 Subject: [PATCH 09/11] Adjust wording in optimization doc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Rok Češnovar --- src/reference-manual/compiler-optimizations.Rmd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/reference-manual/compiler-optimizations.Rmd b/src/reference-manual/compiler-optimizations.Rmd index 7c6677f49..2a03e8423 100644 --- a/src/reference-manual/compiler-optimizations.Rmd +++ b/src/reference-manual/compiler-optimizations.Rmd @@ -11,7 +11,7 @@ To turn optimizations on, the user specifies the desired optimization *level*. T Optimization levels are specified by the numbers 0 and 1 and the 'experimental' tag: - **O0** No optimizations are applied. -- **O1** Only optimizations which are simple, do not dramatically change the program, and are unlikely noticeably slow down compile times are applied. +- **O1** Optimizations that are simple, do not dramatically change the program, and are unlikely to noticeably slow down compile times are applied. - **Oexperimental** All optimizations are applied. Some of these are not thorougly tested and may not always improve a programs performance. O0 is the default setting. From 12060bb1fe7b6d85f0a49165bb0a7d764edb1891 Mon Sep 17 00:00:00 2001 From: rybern Date: Tue, 18 Jan 2022 14:25:55 -0500 Subject: [PATCH 10/11] Adjust wording in optimization doc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Rok Češnovar --- src/reference-manual/compiler-optimizations.Rmd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/reference-manual/compiler-optimizations.Rmd b/src/reference-manual/compiler-optimizations.Rmd index 2a03e8423..806a6403e 100644 --- a/src/reference-manual/compiler-optimizations.Rmd +++ b/src/reference-manual/compiler-optimizations.Rmd @@ -42,7 +42,7 @@ In addition, **Oexperimental** will apply more repetitions of the optimizations, #### Dead code elimination -Dead code is code that does not have any effect on the behavior of the program. Code is not dead if it affects `target`, the value of any outside-observable variable like transformed parameters or generated quantities, or side effects such as print statements. Removing dead code can speed up a program by avoiding unnecessary computations. +Dead code is code that does not affect the behavior of the program. Code is not dead if it affects `target`, the value of any outside-observable variable like transformed parameters or generated quantities, or side effects such as print statements. Removing dead code can speed up a program by avoiding unnecessary computations. Example Stan program: From 37efc7554d2c2dfecf7507bd11e56eaa2b11b890 Mon Sep 17 00:00:00 2001 From: rybern Date: Tue, 18 Jan 2022 14:26:32 -0500 Subject: [PATCH 11/11] Adjust wording in optimization doc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Rok Češnovar --- src/reference-manual/compiler-optimizations.Rmd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/reference-manual/compiler-optimizations.Rmd b/src/reference-manual/compiler-optimizations.Rmd index 806a6403e..107ef2a62 100644 --- a/src/reference-manual/compiler-optimizations.Rmd +++ b/src/reference-manual/compiler-optimizations.Rmd @@ -199,7 +199,7 @@ log_prob { #### Partial evaluation -Partial evaluation searches for expressions that can be replaced with a faster, simpler, more memory efficient, or more numerically stable expression that has the same meaning. +Partial evaluation searches for expressions that we can replace with a faster, simpler, more memory efficient, or more numerically stable expression with the same meaning. Example Stan program: