From 327edc2467ce70718346268090369814c41fe630 Mon Sep 17 00:00:00 2001 From: Jammy2211 Date: Thu, 30 Apr 2026 21:44:07 +0100 Subject: [PATCH] docs: convert 30 prose .rst files to MyST .md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Keeps docs/api/*.rst and docs/_templates/*.rst as RST — autosummary-driven pages don't gain readability from a MyST conversion. Adds `dollarmath` to myst_enable_extensions for the existing math blocks in overview/the_basics.md and science_examples/astronomy.md. Part of #1245. Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/conf.py | 1 + docs/cookbooks/analysis.md | 686 +++++++++ docs/cookbooks/analysis.rst | 695 --------- docs/cookbooks/configs.md | 279 ++++ docs/cookbooks/configs.rst | 285 ---- docs/cookbooks/model.md | 1231 ++++++++++++++++ docs/cookbooks/model.rst | 1253 ----------------- docs/cookbooks/multi_level_model.md | 497 +++++++ docs/cookbooks/multi_level_model.rst | 513 ------- docs/cookbooks/multiple_datasets.md | 621 ++++++++ docs/cookbooks/multiple_datasets.rst | 632 --------- docs/cookbooks/result.md | 754 ++++++++++ docs/cookbooks/result.rst | 782 ---------- docs/cookbooks/samples.md | 718 ++++++++++ docs/cookbooks/samples.rst | 742 ---------- docs/cookbooks/search.md | 385 +++++ docs/cookbooks/search.rst | 404 ------ docs/features/graphical.md | 194 +++ docs/features/graphical.rst | 195 --- docs/features/interpolate.md | 212 +++ docs/features/interpolate.rst | 217 --- docs/features/search_chaining.md | 217 +++ docs/features/search_chaining.rst | 221 --- ..._grid_search.rst => search_grid_search.md} | 261 ++-- docs/features/sensitivity_mapping.md | 233 +++ docs/features/sensitivity_mapping.rst | 241 ---- docs/general/citations.md | 16 + docs/general/citations.rst | 18 - docs/general/configs.md | 24 + docs/general/configs.rst | 27 - docs/general/credits.md | 11 + docs/general/credits.rst | 12 - docs/general/{roadmap.rst => roadmap.md} | 51 +- docs/general/{software.rst => software.md} | 29 +- docs/general/workspace.md | 48 + docs/general/workspace.rst | 54 - docs/index.md | 218 +++ docs/index.rst | 219 --- docs/installation/conda.md | 43 + docs/installation/conda.rst | 44 - docs/installation/overview.md | 41 + docs/installation/overview.rst | 45 - docs/installation/pip.md | 54 + docs/installation/pip.rst | 55 - docs/installation/source.md | 45 + docs/installation/source.rst | 46 - docs/installation/troubleshooting.md | 29 + docs/installation/troubleshooting.rst | 32 - docs/overview/backup.md | 377 +++++ docs/overview/backup.rst | 380 ----- docs/overview/scientific_workflow.md | 608 ++++++++ docs/overview/scientific_workflow.rst | 621 -------- ...cal_methods.rst => statistical_methods.md} | 220 ++- docs/overview/the_basics.md | 655 +++++++++ docs/overview/the_basics.rst | 679 --------- docs/science_examples/astronomy.md | 325 +++++ docs/science_examples/astronomy.rst | 328 ----- 57 files changed, 8794 insertions(+), 9029 deletions(-) create mode 100644 docs/cookbooks/analysis.md delete mode 100644 docs/cookbooks/analysis.rst create mode 100644 docs/cookbooks/configs.md delete mode 100644 docs/cookbooks/configs.rst create mode 100644 docs/cookbooks/model.md delete mode 100644 docs/cookbooks/model.rst create mode 100644 docs/cookbooks/multi_level_model.md delete mode 100644 docs/cookbooks/multi_level_model.rst create mode 100644 docs/cookbooks/multiple_datasets.md delete mode 100644 docs/cookbooks/multiple_datasets.rst create mode 100644 docs/cookbooks/result.md delete mode 100644 docs/cookbooks/result.rst create mode 100644 docs/cookbooks/samples.md delete mode 100644 docs/cookbooks/samples.rst create mode 100644 docs/cookbooks/search.md delete mode 100644 docs/cookbooks/search.rst create mode 100644 docs/features/graphical.md delete mode 100644 docs/features/graphical.rst create mode 100644 docs/features/interpolate.md delete mode 100644 docs/features/interpolate.rst create mode 100644 docs/features/search_chaining.md delete mode 100644 docs/features/search_chaining.rst rename docs/features/{search_grid_search.rst => search_grid_search.md} (51%) create mode 100644 docs/features/sensitivity_mapping.md delete mode 100644 docs/features/sensitivity_mapping.rst create mode 100644 docs/general/citations.md delete mode 100644 docs/general/citations.rst create mode 100644 docs/general/configs.md delete mode 100644 docs/general/configs.rst create mode 100644 docs/general/credits.md delete mode 100644 docs/general/credits.rst rename docs/general/{roadmap.rst => roadmap.md} (66%) rename docs/general/{software.rst => software.md} (50%) create mode 100644 docs/general/workspace.md delete mode 100644 docs/general/workspace.rst create mode 100644 docs/index.md delete mode 100644 docs/index.rst create mode 100644 docs/installation/conda.md delete mode 100644 docs/installation/conda.rst create mode 100644 docs/installation/overview.md delete mode 100644 docs/installation/overview.rst create mode 100644 docs/installation/pip.md delete mode 100644 docs/installation/pip.rst create mode 100644 docs/installation/source.md delete mode 100644 docs/installation/source.rst create mode 100644 docs/installation/troubleshooting.md delete mode 100644 docs/installation/troubleshooting.rst create mode 100644 docs/overview/backup.md delete mode 100644 docs/overview/backup.rst create mode 100644 docs/overview/scientific_workflow.md delete mode 100644 docs/overview/scientific_workflow.rst rename docs/overview/{statistical_methods.rst => statistical_methods.md} (79%) create mode 100644 docs/overview/the_basics.md delete mode 100644 docs/overview/the_basics.rst create mode 100644 docs/science_examples/astronomy.md delete mode 100644 docs/science_examples/astronomy.rst diff --git a/docs/conf.py b/docs/conf.py index cdd05935d..5fd60ed53 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -73,6 +73,7 @@ myst_enable_extensions = [ "colon_fence", "deflist", + "dollarmath", ] myst_heading_anchors = 3 diff --git a/docs/cookbooks/analysis.md b/docs/cookbooks/analysis.md new file mode 100644 index 000000000..b49f15553 --- /dev/null +++ b/docs/cookbooks/analysis.md @@ -0,0 +1,686 @@ +(analysis)= + +# Analysis + +The `Analysis` class is the interface between the data and model, whereby a `log_likelihood_function` is defined +and called by the non-linear search to fit the model. + +This cookbook provides an overview of how to use and extend `Analysis` objects in **PyAutoFit**. + +**Contents:** + +- **Example**: A simple example of an analysis class which can be adapted for you use-case. +- **Customization**: Customizing an analysis class with different data inputs and editing the `log_likelihood_function`. +- **Visualization**: Using a `visualize` method so that model-specific visuals are output to hard-disk. +- **Custom Result**: Return a custom Result object with methods specific to your model fitting problem. +- **Latent Variables**: Adding a `compute_latent_variables` method to the analysis to output latent variables to hard-disk. +- **Custom Output**: Add methods which output model-specific results to hard-disk in the `files` folder (e.g. as .json files) to aid in the interpretation of results. + +## Example + +An example simple `Analysis` class, to remind ourselves of the basic structure and inputs. + +This can be adapted for your use case. + +```python +class Analysis(af.Analysis): + def __init__(self, data: np.ndarray, noise_map: np.ndarray): + """ + The `Analysis` class acts as an interface between the data and model in **PyAutoFit**. + + Its `log_likelihood_function` defines how the model is fitted to the data and it is + called many times by the non-linear search fitting algorithm. + + In this example the `Analysis` `__init__` constructor only contains the `data` + and `noise-map`, but it can be easily extended to include other quantities. + + Parameters + ---------- + data + A 1D numpy array containing the data (e.g. a noisy 1D signal) fitted in the + workspace examples. + noise_map + A 1D numpy array containing the noise values of the data, used for computing + the goodness of fit metric, the log likelihood. + """ + super().__init__() + + self.data = data + self.noise_map = noise_map + + def log_likelihood_function(self, instance) -> float: + """ + Returns the log likelihood of a fit of a 1D Gaussian to the dataset. + + The data is fitted using an `instance` of the `Gaussian` class where + its `model_data_from` is called in order to create a + model data representation of the Gaussian that is fitted to the data. + """ + + xvalues = np.arange(self.data.shape[0]) + + model_data = instance.model_data_from(xvalues=xvalues) + + residual_map = self.data - model_data + chi_squared_map = (residual_map / self.noise_map) ** 2.0 + chi_squared = sum(chi_squared_map) + noise_normalization = np.sum(np.log(2 * np.pi * self.noise_map ** 2.0)) + log_likelihood = -0.5 * (chi_squared + noise_normalization) + + return log_likelihood +``` + +An instance of the analysis class is created as follows. + +```python +dataset_path = path.join("dataset", "example_1d", "gaussian_x1") +data = af.util.numpy_array_from_json(file_path=path.join(dataset_path, "data.json")) +noise_map = af.util.numpy_array_from_json( + file_path=path.join(dataset_path, "noise_map.json") +) + +analysis = Analysis(data=data, noise_map=noise_map) +``` + +## Customization + +The `Analysis` class can be fully customized to be suitable for your model-fit. + +For example, additional inputs can be included in the `__init__` constructor and used in the `log_likelihood_function`. +if they are required for your `log_likelihood_function` to work. + +The example below includes three additional inputs: + +- Instead of inputting a `noise_map`, a `noise_covariance_matrix` is input, which means that corrrlated noise is + : accounted for in the `log_likelihood_function`. +- A `mask` is input which masks the data such that certain data points are omitted from the log likelihood +- A `kernel` is input which can account for certain blurring operations during data acquisition. + +```python +class Analysis(af.Analysis): + def __init__( + self, + data: np.ndarray, + noise_covariance_matrix: np.ndarray, + mask: np.ndarray, + kernel: np.ndarray + ): + """ + The `Analysis` class which has had its inputs edited for a different model-fit. + + Parameters + ---------- + data + A 1D numpy array containing the data (e.g. a noisy 1D signal) fitted + in the workspace examples. + noise_covariance_matrix + A 2D numpy array containing the noise values and their covariances + for the data, used for computing the + goodness of fit whilst accounting for correlated noise. + mask + A 1D numpy array containing a mask, where `True` values mean a data + point is masked and is omitted from + the log likelihood. + kernel + A 1D numpy array containing the blurring kernel of the data, used + for creating the model data. + """ + super().__init__() + + self.data = data + self.noise_covariance_matrix = noise_covariance_matrix + self.mask = mask + self.kernel = kernel + + def log_likelihood_function(self, instance) -> float: + """ + The `log_likelihood_function` now has access to + the `noise_covariance_matrix`, `mask` and `kernel`, input above. + """ + print(self.noise_covariance_matrix) + print(self.mask) + print(self.kernel) + + """ + We do not provide a specific example of how to use these inputs + in the `log_likelihood_function` as they are specific to your + model fitting problem. + + The key point is that any inputs required to compute the log + likelihood can be passed into the `__init__` constructor of the + `Analysis` class and used in the `log_likelihood_function`. + """ + + log_likelihood = None + + return log_likelihood +``` + +An instance of the analysis class is created as follows. + +```python +dataset_path = path.join("dataset", "example_1d", "gaussian_x1") +data = af.util.numpy_array_from_json(file_path=path.join(dataset_path, "data.json")) + +noise_covariance_matrix = np.ones(shape=(data.shape[0], data.shape[0])) +mask = np.full(fill_value=False, shape=data.shape) +kernel = np.full(fill_value=1.0, shape=data.shape) + +analysis = Analysis( + data=data, noise_covariance_matrix=noise_covariance_matrix, mask=mask, kernel=kernel +) +``` + +## Visualization + +If a `name` is input into a non-linear search, all results are output to hard-disk in a folder. + +By overwriting the `Visualizer` object of an `Analysis` class with a custom `Visualizer` class, custom results of the +model-fit can be visualized during the model-fit. + +The `Visualizer` below has the methods `visualize_before_fit` and `visualize`, which perform model specific +visualization will also be output into an `image` folder, for example as `.png` files. + +This uses the maximum log likelihood model of the model-fit inferred so far. + +Visualization of the results of the search, such as the corner plot of what is called the "Probability Density +Function", are also automatically output during the model-fit on the fly. + +```python +class Visualizer(af.Visualizer): + + @staticmethod + def visualize_before_fit( + analysis, + paths: af.DirectoryPaths, + model: af.AbstractPriorModel + ): + """ + Before a model-fit, the `visualize_before_fit` method is called to perform visualization. + + The function receives as input an instance of the `Analysis` class which is being used to perform the fit, + which is used to perform the visualization (e.g. it contains the data and noise map which are plotted). + + This can output visualization of quantities which do not change during the model-fit, for example the + data and noise-map. + + The `paths` object contains the path to the folder where the visualization should be output, which is determined + by the non-linear search `name` and other inputs. + """ + + import matplotlib.pyplot as plt + + xvalues = np.arange(analysis.data.shape[0]) + + plt.errorbar( + x=xvalues, + y=analysis.data, + yerr=analysis.noise_map, + color="k", + ecolor="k", + elinewidth=1, + capsize=2, + ) + plt.title("Maximum Likelihood Fit") + plt.xlabel("x value of profile") + plt.ylabel("Profile Normalization") + plt.savefig(path.join(paths.image_path, f"data.png")) + plt.clf() + + @staticmethod + def visualize( + analysis, + paths: af.DirectoryPaths, + instance, + during_analysis + ): + """ + During a model-fit, the `visualize` method is called throughout the non-linear search. + + The function receives as input an instance of the `Analysis` class which is being used to perform the fit, + which is used to perform the visualization (e.g. it generates the model data which is plotted). + + The `instance` passed into the visualize method is maximum log likelihood solution obtained by the model-fit + so far and it can be used to provide on-the-fly images showing how the model-fit is going. + + The `paths` object contains the path to the folder where the visualization should be output, which is determined + by the non-linear search `name` and other inputs. + """ + xvalues = np.arange(analysis.data.shape[0]) + + model_data = instance.model_data_from(xvalues=xvalues) + residual_map = analysis.data - model_data + + """ + The visualizer now outputs images of the best-fit results to hard-disk (checkout `visualizer.py`). + """ + import matplotlib.pyplot as plt + + plt.errorbar( + x=xvalues, + y=analysis.data, + yerr=analysis.noise_map, + color="k", + ecolor="k", + elinewidth=1, + capsize=2, + ) + plt.plot(xvalues, model_data, color="r") + plt.title("Maximum Likelihood Fit") + plt.xlabel("x value of profile") + plt.ylabel("Profile Normalization") + plt.savefig(path.join(paths.image_path, f"model_fit.png")) + plt.clf() + + plt.errorbar( + x=xvalues, + y=residual_map, + yerr=analysis.noise_map, + color="k", + ecolor="k", + elinewidth=1, + capsize=2, + ) + plt.title("Residuals of Maximum Likelihood Fit") + plt.xlabel("x value of profile") + plt.ylabel("Residual") + plt.savefig(path.join(paths.image_path, f"model_fit.png")) + plt.clf() +``` + +The `Analysis` class is defined following the same API as before, but now with its `Visualizer` class attribute +overwritten with the `Visualizer` class above. + +```python +class Analysis(af.Analysis): + + """ + This over-write means the `Visualizer` class is used for visualization throughout the model-fit. + + This `VisualizerExample` object is in the `autofit.example.visualize` module and is used to customize the + plots output during the model-fit. + + It has been extended with visualize methods that output visuals specific to the fitting of `1D` data. + """ + Visualizer = Visualizer + + def __init__(self, data, noise_map): + """ + An Analysis class which illustrates visualization. + """ + super().__init__() + + self.data = data + self.noise_map = noise_map + + def log_likelihood_function(self, instance): + """ + The `log_likelihood_function` is identical to the example above + """ + xvalues = np.arange(self.data.shape[0]) + + model_data = instance.model_data_from(xvalues=xvalues) + residual_map = self.data - model_data + chi_squared_map = (residual_map / self.noise_map) ** 2.0 + chi_squared = sum(chi_squared_map) + noise_normalization = np.sum(np.log(2 * np.pi * noise_map**2.0)) + log_likelihood = -0.5 * (chi_squared + noise_normalization) + + return log_likelihood +``` + +## Custom Result + +The `Result` object is returned by a non-linear search after running the following code: + +```python +result = search.fit(model=model, analysis=analysis) +``` + +The result can be can be customized to include additional information about the model-fit that is specific to your +model-fitting problem. + +For example, for fitting 1D profiles, the `Result` could include the maximum log likelihood model 1D data: + +```python +print(result.max_log_likelihood_model_data_1d) +``` + +In other examples, this quantity has been manually computed after the model-fit has completed. + +The custom result API allows us to do this. First, we define a custom `Result` class, which includes the property +`max_log_likelihood_model_data_1d`. + +```python +class ResultExample(af.Result): + + @property + def max_log_likelihood_model_data_1d(self) -> np.ndarray: + """ + Returns the maximum log likelihood model's 1D model data. + + This is an example of how we can pass the `Analysis` class a custom `Result` object and extend this result + object with new properties that are specific to the model-fit we are performing. + """ + xvalues = np.arange(self.analysis.data.shape[0]) + + return self.instance.model_data_from(xvalues=xvalues) +``` + +The custom result has access to the analysis class, meaning that we can use any of its methods or properties to +compute custom result properties. + +To make it so that the `ResultExample` object above is returned by the search we overwrite the `Result` class attribute +of the `Analysis` and define a `make_result` object describing what we want it to contain: + +```python +class Analysis(af.Analysis): + + """ + This overwrite means the `ResultExample` class is returned after the model-fit. + """ + Result = ResultExample + + def __init__(self, data, noise_map): + """ + An Analysis class which illustrates custom results. + """ + super().__init__() + + self.data = data + self.noise_map = noise_map + + def log_likelihood_function(self, instance): + """ + The `log_likelihood_function` is identical to the example above + """ + xvalues = np.arange(self.data.shape[0]) + + model_data = instance.model_data_from(xvalues=xvalues) + residual_map = self.data - model_data + chi_squared_map = (residual_map / self.noise_map) ** 2.0 + chi_squared = sum(chi_squared_map) + noise_normalization = np.sum(np.log(2 * np.pi * noise_map**2.0)) + log_likelihood = -0.5 * (chi_squared + noise_normalization) + + return log_likelihood + + def make_result( + self, + samples_summary: af.SamplesSummary, + paths: af.AbstractPaths, + samples: Optional[af.SamplesPDF] = None, + search_internal: Optional[object] = None, + analysis: Optional[object] = None, + ) -> Result: + """ + Returns the `Result` of the non-linear search after it is completed. + + The result type is defined as a class variable in the `Analysis` class (see top of code under the python code + `class Analysis(af.Analysis)`. + + The result can be manually overwritten by a user to return a user-defined result object, which can be extended + with additional methods and attribute specific to the model-fit. + + This example class does example this, whereby the analysis result has been overwritten with the `ResultExample` + class, which contains a property `max_log_likelihood_model_data_1d` that returns the model data of the + best-fit model. This API means you can customize your result object to include whatever attributes you want + and therefore make a result object specific to your model-fit and model-fitting problem. + + The `Result` object you return can be customized to include: + + - The samples summary, which contains the maximum log likelihood instance and median PDF model. + + - The paths of the search, which are used for loading the samples and search internal below when a search + is resumed. + + - The samples of the non-linear search (e.g. MCMC chains) also stored in `samples.csv`. + + - The non-linear search used for the fit in its internal representation, which is used for resuming a search + and making bespoke visualization using the search's internal results. + + - The analysis used to fit the model (default disabled to save memory, but option may be useful for certain + projects). + + Parameters + ---------- + samples_summary + The summary of the samples of the non-linear search, which include the maximum log likelihood instance and + median PDF model. + paths + An object describing the paths for saving data (e.g. hard-disk directories or entries in sqlite database). + samples + The samples of the non-linear search, for example the chains of an MCMC run. + search_internal + The internal representation of the non-linear search used to perform the model-fit. + analysis + The analysis used to fit the model. + + Returns + ------- + Result + The result of the non-linear search, which is defined as a class variable in the `Analysis` class. + """ + return self.Result( + samples_summary=samples_summary, + paths=paths, + samples=samples, + search_internal=search_internal, + analysis=self + ) +``` + +For the sake of brevity, we do not run the code below, but the following code would work: + +```python +result = search.fit(model=model, analysis=analysis) +print(result.max_log_likelihood_model_data_1d) +``` + +## Latent Variables + +A latent variable is not a model parameter but can be derived from the model. Its value and errors may be of interest +and aid in the interpretation of a model-fit. + +For example, for the simple 1D Gaussian example, it could be the full-width half maximum (FWHM) of the Gaussian. +This is not included in the model but can be easily derived from the Gaussian's sigma value. + +By overwriting the Analysis class's `compute_latent_variables` method we can manually specify latent variables that +are calculated. If the search has a `name`, these are output to a `latent.csv` file, which mirrors +the `samples.csv` file. + +There may also be a `latent.results` and `latent_summary.json` files output. The `output.yaml` config file +contains settings customizing what files are output and how often. + +```python +class Analysis(af.Analysis): + def __init__(self, data, noise_map): + """ + An Analysis class which illustrates latent variables. + """ + super().__init__() + + self.data = data + self.noise_map = noise_map + + def log_likelihood_function(self, instance): + """ + The `log_likelihood_function` is identical to the example above + """ + xvalues = np.arange(self.data.shape[0]) + + model_data = instance.model_data_from(xvalues=xvalues) + residual_map = self.data - model_data + chi_squared_map = (residual_map / self.noise_map) ** 2.0 + chi_squared = sum(chi_squared_map) + noise_normalization = np.sum(np.log(2 * np.pi * noise_map**2.0)) + log_likelihood = -0.5 * (chi_squared + noise_normalization) + + return log_likelihood + + def compute_latent_variables(self, instance) -> Dict[str, float]: + """ + A latent variable is not a model parameter but can be derived from the model. Its value and errors may be + of interest and aid in the interpretation of a model-fit. + + For example, for the simple 1D Gaussian example, it could be the full-width half maximum (FWHM) of the + Gaussian. This is not included in the model but can be easily derived from the Gaussian's sigma value. + + By overwriting this method we can manually specify latent variables that are calculated and output to + a `latent.csv` file, which mirrors the `samples.csv` file. + + In the example below, the `latent.csv` file will contain one column with the FWHM of every Gausian model + sampled by the non-linear search. + + This function is called for every non-linear search sample, where the `instance` passed in corresponds to + each sample. + + Parameters + ---------- + instance + The instances of the model which the latent variable is derived from. + + Returns + ------- + A dictionary mapping every latent variable name to its value. + + """ + return { + "fwhm": instance.fwhm + } +``` + +Outputting latent variables manually after a fit is complete is simple, just call +the `analysis.compute_latent_variables()` function. + +For many use cases, the best set disables autofit latent variable output during a fit via +the `output.yaml` file and perform it manually after completing a successful model-fit. This will save computational +run time by not computing latent variables during a any model-fit which is unsuccessful. + +```python +analysis = Analysis(data=data, noise_map=noise_map) + +# You need to have run a fit to retrieve a result to do this. + +analysis.compute_latent_variables(samples=result.samples) +``` + +Analysing and interpreting latent variables is described fully in the result cookbook. + +However, in brief, the `latent_samples` object is a `Samples` object and uses the same API as samples objects. + +```python +print(latent_samples.median_pdf().fwhm) +``` + +## Custom Output + +When performing fits which output results to hard-disc, a `files` folder is created containing .json / .csv files of +the model, samples, search, etc. + +These files are human readable and help one quickly inspect and interpret results. + +By extending an `Analysis` class with the methods `save_attributes` and `save_results`, +custom files can be written to the `files` folder to further aid this inspection. + +These files can then also be loaded via the database, as described in the database cookbook. + +```python +class Analysis(af.Analysis): + def __init__(self, data: np.ndarray, noise_map: np.ndarray): + """ + Standard Analysis class example used throughout PyAutoFit examples. + """ + super().__init__() + + self.data = data + self.noise_map = noise_map + + def log_likelihood_function(self, instance) -> float: + """ + Standard log likelihood function used throughout PyAutoFit examples. + """ + + xvalues = np.arange(self.data.shape[0]) + + model_data = instance.model_data_from(xvalues=xvalues) + + residual_map = self.data - model_data + chi_squared_map = (residual_map / self.noise_map) ** 2.0 + chi_squared = sum(chi_squared_map) + noise_normalization = np.sum(np.log(2 * np.pi * self.noise_map**2.0)) + log_likelihood = -0.5 * (chi_squared + noise_normalization) + + return log_likelihood + + def save_attributes(self, paths: af.DirectoryPaths): + """ + Before the non-linear search begins, this routine saves attributes + of the `Analysis` object to the `files` folder such that they can + be loaded after the analysis using PyAutoFit's database and aggregator tools. + + For this analysis, it uses the `AnalysisDataset` object's method to + output the following: + + - The dataset's data as a .json file. + - The dataset's noise-map as a .json file. + + These are accessed using the aggregator via `agg.values("data")` + and `agg.values("noise_map")`. + + Parameters + ---------- + paths + The paths object which manages all paths, e.g. where + the non-linear search outputs are stored, visualization, and the + pickled objects used by the aggregator output by this function. + """ + # The path where data.json is saved, e.g. output/dataset_name/unique_id/files/data.json + + file_path = paths._files_path / "data.json" + + with open(file_path, "w+") as f: + json.dump(self.data.tolist(), f, indent=4) + + # The path where noise_map.json is saved, e.g. output/noise_mapset_name/unique_id/files/noise_map.json + + file_path = paths._files_path / "noise_map.json" + + with open(file_path, "w+") as f: + json.dump(self.noise_map.tolist(), f, indent=4) + + def save_results(self, paths: af.DirectoryPaths, result: af.Result): + """ + At the end of a model-fit, this routine saves attributes of the `Analysis` + object to the `files` folder such that they can be loaded after the analysis + using PyAutoFit's database and aggregator tools. + + For this analysis it outputs the following: + + - The maximum log likelihood model data as a .json file. + + This is accessed using the aggregator via `agg.values("model_data")`. + + Parameters + ---------- + paths + The paths object which manages all paths, e.g. where the + non-linear search outputs are stored, visualization and the pickled + objects used by the aggregator output by this function. + result + The result of a model fit, including the non-linear search, samples + and maximum likelihood model. + """ + xvalues = np.arange(self.data.shape[0]) + + instance = result.max_log_likelihood_instance + + model_data = instance.model_data_from(xvalues=xvalues) + + # The path where model_data.json is saved, e.g. output/dataset_name/unique_id/files/model_data.json + + file_path = (path.join(paths._files_path, "model_data.json"),) + + with open(file_path, "w+") as f: + json.dump(model_data, f, indent=4) +``` diff --git a/docs/cookbooks/analysis.rst b/docs/cookbooks/analysis.rst deleted file mode 100644 index 15b4023bd..000000000 --- a/docs/cookbooks/analysis.rst +++ /dev/null @@ -1,695 +0,0 @@ -.. _analysis: - -Analysis -======== - -The ``Analysis`` class is the interface between the data and model, whereby a ``log_likelihood_function`` is defined -and called by the non-linear search to fit the model. - -This cookbook provides an overview of how to use and extend ``Analysis`` objects in **PyAutoFit**. - -**Contents:** - -- **Example**: A simple example of an analysis class which can be adapted for you use-case. -- **Customization**: Customizing an analysis class with different data inputs and editing the ``log_likelihood_function``. -- **Visualization**: Using a `visualize` method so that model-specific visuals are output to hard-disk. -- **Custom Result**: Return a custom Result object with methods specific to your model fitting problem. -- **Latent Variables**: Adding a `compute_latent_variables` method to the analysis to output latent variables to hard-disk. -- **Custom Output**: Add methods which output model-specific results to hard-disk in the ``files`` folder (e.g. as .json files) to aid in the interpretation of results. - -Example -------- - -An example simple ``Analysis`` class, to remind ourselves of the basic structure and inputs. - -This can be adapted for your use case. - -.. code-block:: python - - class Analysis(af.Analysis): - def __init__(self, data: np.ndarray, noise_map: np.ndarray): - """ - The `Analysis` class acts as an interface between the data and model in **PyAutoFit**. - - Its `log_likelihood_function` defines how the model is fitted to the data and it is - called many times by the non-linear search fitting algorithm. - - In this example the `Analysis` `__init__` constructor only contains the `data` - and `noise-map`, but it can be easily extended to include other quantities. - - Parameters - ---------- - data - A 1D numpy array containing the data (e.g. a noisy 1D signal) fitted in the - workspace examples. - noise_map - A 1D numpy array containing the noise values of the data, used for computing - the goodness of fit metric, the log likelihood. - """ - super().__init__() - - self.data = data - self.noise_map = noise_map - - def log_likelihood_function(self, instance) -> float: - """ - Returns the log likelihood of a fit of a 1D Gaussian to the dataset. - - The data is fitted using an `instance` of the `Gaussian` class where - its `model_data_from` is called in order to create a - model data representation of the Gaussian that is fitted to the data. - """ - - xvalues = np.arange(self.data.shape[0]) - - model_data = instance.model_data_from(xvalues=xvalues) - - residual_map = self.data - model_data - chi_squared_map = (residual_map / self.noise_map) ** 2.0 - chi_squared = sum(chi_squared_map) - noise_normalization = np.sum(np.log(2 * np.pi * self.noise_map ** 2.0)) - log_likelihood = -0.5 * (chi_squared + noise_normalization) - - return log_likelihood - -An instance of the analysis class is created as follows. - -.. code-block:: python - - dataset_path = path.join("dataset", "example_1d", "gaussian_x1") - data = af.util.numpy_array_from_json(file_path=path.join(dataset_path, "data.json")) - noise_map = af.util.numpy_array_from_json( - file_path=path.join(dataset_path, "noise_map.json") - ) - - analysis = Analysis(data=data, noise_map=noise_map) - -Customization -------------- - -The ``Analysis`` class can be fully customized to be suitable for your model-fit. - -For example, additional inputs can be included in the ``__init__`` constructor and used in the ``log_likelihood_function``. -if they are required for your ``log_likelihood_function`` to work. - -The example below includes three additional inputs: - -- Instead of inputting a ``noise_map``, a ``noise_covariance_matrix`` is input, which means that corrrlated noise is - accounted for in the ``log_likelihood_function``. - -- A ``mask`` is input which masks the data such that certain data points are omitted from the log likelihood - -- A ``kernel`` is input which can account for certain blurring operations during data acquisition. - -.. code-block:: python - - class Analysis(af.Analysis): - def __init__( - self, - data: np.ndarray, - noise_covariance_matrix: np.ndarray, - mask: np.ndarray, - kernel: np.ndarray - ): - """ - The `Analysis` class which has had its inputs edited for a different model-fit. - - Parameters - ---------- - data - A 1D numpy array containing the data (e.g. a noisy 1D signal) fitted - in the workspace examples. - noise_covariance_matrix - A 2D numpy array containing the noise values and their covariances - for the data, used for computing the - goodness of fit whilst accounting for correlated noise. - mask - A 1D numpy array containing a mask, where `True` values mean a data - point is masked and is omitted from - the log likelihood. - kernel - A 1D numpy array containing the blurring kernel of the data, used - for creating the model data. - """ - super().__init__() - - self.data = data - self.noise_covariance_matrix = noise_covariance_matrix - self.mask = mask - self.kernel = kernel - - def log_likelihood_function(self, instance) -> float: - """ - The `log_likelihood_function` now has access to - the `noise_covariance_matrix`, `mask` and `kernel`, input above. - """ - print(self.noise_covariance_matrix) - print(self.mask) - print(self.kernel) - - """ - We do not provide a specific example of how to use these inputs - in the `log_likelihood_function` as they are specific to your - model fitting problem. - - The key point is that any inputs required to compute the log - likelihood can be passed into the `__init__` constructor of the - `Analysis` class and used in the `log_likelihood_function`. - """ - - log_likelihood = None - - return log_likelihood - -An instance of the analysis class is created as follows. - -.. code-block:: python - - dataset_path = path.join("dataset", "example_1d", "gaussian_x1") - data = af.util.numpy_array_from_json(file_path=path.join(dataset_path, "data.json")) - - noise_covariance_matrix = np.ones(shape=(data.shape[0], data.shape[0])) - mask = np.full(fill_value=False, shape=data.shape) - kernel = np.full(fill_value=1.0, shape=data.shape) - - analysis = Analysis( - data=data, noise_covariance_matrix=noise_covariance_matrix, mask=mask, kernel=kernel - ) - -Visualization -------------- - -If a ``name`` is input into a non-linear search, all results are output to hard-disk in a folder. - -By overwriting the ``Visualizer`` object of an ``Analysis`` class with a custom ``Visualizer`` class, custom results of the -model-fit can be visualized during the model-fit. - -The ``Visualizer`` below has the methods ``visualize_before_fit`` and ``visualize``, which perform model specific -visualization will also be output into an ``image`` folder, for example as ``.png`` files. - -This uses the maximum log likelihood model of the model-fit inferred so far. - -Visualization of the results of the search, such as the corner plot of what is called the "Probability Density -Function", are also automatically output during the model-fit on the fly. - -.. code-block:: python - - class Visualizer(af.Visualizer): - - @staticmethod - def visualize_before_fit( - analysis, - paths: af.DirectoryPaths, - model: af.AbstractPriorModel - ): - """ - Before a model-fit, the `visualize_before_fit` method is called to perform visualization. - - The function receives as input an instance of the `Analysis` class which is being used to perform the fit, - which is used to perform the visualization (e.g. it contains the data and noise map which are plotted). - - This can output visualization of quantities which do not change during the model-fit, for example the - data and noise-map. - - The `paths` object contains the path to the folder where the visualization should be output, which is determined - by the non-linear search `name` and other inputs. - """ - - import matplotlib.pyplot as plt - - xvalues = np.arange(analysis.data.shape[0]) - - plt.errorbar( - x=xvalues, - y=analysis.data, - yerr=analysis.noise_map, - color="k", - ecolor="k", - elinewidth=1, - capsize=2, - ) - plt.title("Maximum Likelihood Fit") - plt.xlabel("x value of profile") - plt.ylabel("Profile Normalization") - plt.savefig(path.join(paths.image_path, f"data.png")) - plt.clf() - - @staticmethod - def visualize( - analysis, - paths: af.DirectoryPaths, - instance, - during_analysis - ): - """ - During a model-fit, the `visualize` method is called throughout the non-linear search. - - The function receives as input an instance of the `Analysis` class which is being used to perform the fit, - which is used to perform the visualization (e.g. it generates the model data which is plotted). - - The `instance` passed into the visualize method is maximum log likelihood solution obtained by the model-fit - so far and it can be used to provide on-the-fly images showing how the model-fit is going. - - The `paths` object contains the path to the folder where the visualization should be output, which is determined - by the non-linear search `name` and other inputs. - """ - xvalues = np.arange(analysis.data.shape[0]) - - model_data = instance.model_data_from(xvalues=xvalues) - residual_map = analysis.data - model_data - - """ - The visualizer now outputs images of the best-fit results to hard-disk (checkout `visualizer.py`). - """ - import matplotlib.pyplot as plt - - plt.errorbar( - x=xvalues, - y=analysis.data, - yerr=analysis.noise_map, - color="k", - ecolor="k", - elinewidth=1, - capsize=2, - ) - plt.plot(xvalues, model_data, color="r") - plt.title("Maximum Likelihood Fit") - plt.xlabel("x value of profile") - plt.ylabel("Profile Normalization") - plt.savefig(path.join(paths.image_path, f"model_fit.png")) - plt.clf() - - plt.errorbar( - x=xvalues, - y=residual_map, - yerr=analysis.noise_map, - color="k", - ecolor="k", - elinewidth=1, - capsize=2, - ) - plt.title("Residuals of Maximum Likelihood Fit") - plt.xlabel("x value of profile") - plt.ylabel("Residual") - plt.savefig(path.join(paths.image_path, f"model_fit.png")) - plt.clf() - -The `Analysis` class is defined following the same API as before, but now with its `Visualizer` class attribute -overwritten with the `Visualizer` class above. - -.. code-block:: python - - class Analysis(af.Analysis): - - """ - This over-write means the `Visualizer` class is used for visualization throughout the model-fit. - - This `VisualizerExample` object is in the `autofit.example.visualize` module and is used to customize the - plots output during the model-fit. - - It has been extended with visualize methods that output visuals specific to the fitting of `1D` data. - """ - Visualizer = Visualizer - - def __init__(self, data, noise_map): - """ - An Analysis class which illustrates visualization. - """ - super().__init__() - - self.data = data - self.noise_map = noise_map - - def log_likelihood_function(self, instance): - """ - The `log_likelihood_function` is identical to the example above - """ - xvalues = np.arange(self.data.shape[0]) - - model_data = instance.model_data_from(xvalues=xvalues) - residual_map = self.data - model_data - chi_squared_map = (residual_map / self.noise_map) ** 2.0 - chi_squared = sum(chi_squared_map) - noise_normalization = np.sum(np.log(2 * np.pi * noise_map**2.0)) - log_likelihood = -0.5 * (chi_squared + noise_normalization) - - return log_likelihood - -Custom Result -------------- - -The ``Result`` object is returned by a non-linear search after running the following code: - -.. code-block:: python - - result = search.fit(model=model, analysis=analysis) - -The result can be can be customized to include additional information about the model-fit that is specific to your -model-fitting problem. - -For example, for fitting 1D profiles, the ``Result`` could include the maximum log likelihood model 1D data: - -.. code-block:: python - - print(result.max_log_likelihood_model_data_1d) - -In other examples, this quantity has been manually computed after the model-fit has completed. - -The custom result API allows us to do this. First, we define a custom ``Result`` class, which includes the property -``max_log_likelihood_model_data_1d``. - -.. code-block:: python - - class ResultExample(af.Result): - - @property - def max_log_likelihood_model_data_1d(self) -> np.ndarray: - """ - Returns the maximum log likelihood model's 1D model data. - - This is an example of how we can pass the `Analysis` class a custom `Result` object and extend this result - object with new properties that are specific to the model-fit we are performing. - """ - xvalues = np.arange(self.analysis.data.shape[0]) - - return self.instance.model_data_from(xvalues=xvalues) - -The custom result has access to the analysis class, meaning that we can use any of its methods or properties to -compute custom result properties. - -To make it so that the ``ResultExample`` object above is returned by the search we overwrite the ``Result`` class attribute -of the ``Analysis`` and define a ``make_result`` object describing what we want it to contain: - -.. code-block:: python - - class Analysis(af.Analysis): - - """ - This overwrite means the `ResultExample` class is returned after the model-fit. - """ - Result = ResultExample - - def __init__(self, data, noise_map): - """ - An Analysis class which illustrates custom results. - """ - super().__init__() - - self.data = data - self.noise_map = noise_map - - def log_likelihood_function(self, instance): - """ - The `log_likelihood_function` is identical to the example above - """ - xvalues = np.arange(self.data.shape[0]) - - model_data = instance.model_data_from(xvalues=xvalues) - residual_map = self.data - model_data - chi_squared_map = (residual_map / self.noise_map) ** 2.0 - chi_squared = sum(chi_squared_map) - noise_normalization = np.sum(np.log(2 * np.pi * noise_map**2.0)) - log_likelihood = -0.5 * (chi_squared + noise_normalization) - - return log_likelihood - - def make_result( - self, - samples_summary: af.SamplesSummary, - paths: af.AbstractPaths, - samples: Optional[af.SamplesPDF] = None, - search_internal: Optional[object] = None, - analysis: Optional[object] = None, - ) -> Result: - """ - Returns the `Result` of the non-linear search after it is completed. - - The result type is defined as a class variable in the `Analysis` class (see top of code under the python code - `class Analysis(af.Analysis)`. - - The result can be manually overwritten by a user to return a user-defined result object, which can be extended - with additional methods and attribute specific to the model-fit. - - This example class does example this, whereby the analysis result has been overwritten with the `ResultExample` - class, which contains a property `max_log_likelihood_model_data_1d` that returns the model data of the - best-fit model. This API means you can customize your result object to include whatever attributes you want - and therefore make a result object specific to your model-fit and model-fitting problem. - - The `Result` object you return can be customized to include: - - - The samples summary, which contains the maximum log likelihood instance and median PDF model. - - - The paths of the search, which are used for loading the samples and search internal below when a search - is resumed. - - - The samples of the non-linear search (e.g. MCMC chains) also stored in `samples.csv`. - - - The non-linear search used for the fit in its internal representation, which is used for resuming a search - and making bespoke visualization using the search's internal results. - - - The analysis used to fit the model (default disabled to save memory, but option may be useful for certain - projects). - - Parameters - ---------- - samples_summary - The summary of the samples of the non-linear search, which include the maximum log likelihood instance and - median PDF model. - paths - An object describing the paths for saving data (e.g. hard-disk directories or entries in sqlite database). - samples - The samples of the non-linear search, for example the chains of an MCMC run. - search_internal - The internal representation of the non-linear search used to perform the model-fit. - analysis - The analysis used to fit the model. - - Returns - ------- - Result - The result of the non-linear search, which is defined as a class variable in the `Analysis` class. - """ - return self.Result( - samples_summary=samples_summary, - paths=paths, - samples=samples, - search_internal=search_internal, - analysis=self - ) - -For the sake of brevity, we do not run the code below, but the following code would work: - -.. code-block:: python - - result = search.fit(model=model, analysis=analysis) - print(result.max_log_likelihood_model_data_1d) - -Latent Variables ----------------- - -A latent variable is not a model parameter but can be derived from the model. Its value and errors may be of interest -and aid in the interpretation of a model-fit. - -For example, for the simple 1D Gaussian example, it could be the full-width half maximum (FWHM) of the Gaussian. -This is not included in the model but can be easily derived from the Gaussian's sigma value. - -By overwriting the Analysis class's ``compute_latent_variables`` method we can manually specify latent variables that -are calculated. If the search has a ``name``, these are output to a ``latent.csv`` file, which mirrors -the ``samples.csv`` file. - -There may also be a ``latent.results`` and ``latent_summary.json`` files output. The ``output.yaml`` config file -contains settings customizing what files are output and how often. - -.. code-block:: python - - class Analysis(af.Analysis): - def __init__(self, data, noise_map): - """ - An Analysis class which illustrates latent variables. - """ - super().__init__() - - self.data = data - self.noise_map = noise_map - - def log_likelihood_function(self, instance): - """ - The `log_likelihood_function` is identical to the example above - """ - xvalues = np.arange(self.data.shape[0]) - - model_data = instance.model_data_from(xvalues=xvalues) - residual_map = self.data - model_data - chi_squared_map = (residual_map / self.noise_map) ** 2.0 - chi_squared = sum(chi_squared_map) - noise_normalization = np.sum(np.log(2 * np.pi * noise_map**2.0)) - log_likelihood = -0.5 * (chi_squared + noise_normalization) - - return log_likelihood - - def compute_latent_variables(self, instance) -> Dict[str, float]: - """ - A latent variable is not a model parameter but can be derived from the model. Its value and errors may be - of interest and aid in the interpretation of a model-fit. - - For example, for the simple 1D Gaussian example, it could be the full-width half maximum (FWHM) of the - Gaussian. This is not included in the model but can be easily derived from the Gaussian's sigma value. - - By overwriting this method we can manually specify latent variables that are calculated and output to - a `latent.csv` file, which mirrors the `samples.csv` file. - - In the example below, the `latent.csv` file will contain one column with the FWHM of every Gausian model - sampled by the non-linear search. - - This function is called for every non-linear search sample, where the `instance` passed in corresponds to - each sample. - - Parameters - ---------- - instance - The instances of the model which the latent variable is derived from. - - Returns - ------- - A dictionary mapping every latent variable name to its value. - - """ - return { - "fwhm": instance.fwhm - } - -Outputting latent variables manually after a fit is complete is simple, just call -the ``analysis.compute_latent_variables()`` function. - -For many use cases, the best set disables autofit latent variable output during a fit via -the ``output.yaml`` file and perform it manually after completing a successful model-fit. This will save computational -run time by not computing latent variables during a any model-fit which is unsuccessful. - -.. code-block:: python - - analysis = Analysis(data=data, noise_map=noise_map) - - # You need to have run a fit to retrieve a result to do this. - - analysis.compute_latent_variables(samples=result.samples) - -Analysing and interpreting latent variables is described fully in the result cookbook. - -However, in brief, the `latent_samples` object is a `Samples` object and uses the same API as samples objects. - -.. code-block:: python - - print(latent_samples.median_pdf().fwhm) - -Custom Output -------------- - -When performing fits which output results to hard-disc, a ``files`` folder is created containing .json / .csv files of -the model, samples, search, etc. - -These files are human readable and help one quickly inspect and interpret results. - -By extending an ``Analysis`` class with the methods ``save_attributes`` and ``save_results``, -custom files can be written to the ``files`` folder to further aid this inspection. - -These files can then also be loaded via the database, as described in the database cookbook. - -.. code-block:: python - - class Analysis(af.Analysis): - def __init__(self, data: np.ndarray, noise_map: np.ndarray): - """ - Standard Analysis class example used throughout PyAutoFit examples. - """ - super().__init__() - - self.data = data - self.noise_map = noise_map - - def log_likelihood_function(self, instance) -> float: - """ - Standard log likelihood function used throughout PyAutoFit examples. - """ - - xvalues = np.arange(self.data.shape[0]) - - model_data = instance.model_data_from(xvalues=xvalues) - - residual_map = self.data - model_data - chi_squared_map = (residual_map / self.noise_map) ** 2.0 - chi_squared = sum(chi_squared_map) - noise_normalization = np.sum(np.log(2 * np.pi * self.noise_map**2.0)) - log_likelihood = -0.5 * (chi_squared + noise_normalization) - - return log_likelihood - - def save_attributes(self, paths: af.DirectoryPaths): - """ - Before the non-linear search begins, this routine saves attributes - of the `Analysis` object to the `files` folder such that they can - be loaded after the analysis using PyAutoFit's database and aggregator tools. - - For this analysis, it uses the `AnalysisDataset` object's method to - output the following: - - - The dataset's data as a .json file. - - The dataset's noise-map as a .json file. - - These are accessed using the aggregator via `agg.values("data")` - and `agg.values("noise_map")`. - - Parameters - ---------- - paths - The paths object which manages all paths, e.g. where - the non-linear search outputs are stored, visualization, and the - pickled objects used by the aggregator output by this function. - """ - # The path where data.json is saved, e.g. output/dataset_name/unique_id/files/data.json - - file_path = paths._files_path / "data.json" - - with open(file_path, "w+") as f: - json.dump(self.data.tolist(), f, indent=4) - - # The path where noise_map.json is saved, e.g. output/noise_mapset_name/unique_id/files/noise_map.json - - file_path = paths._files_path / "noise_map.json" - - with open(file_path, "w+") as f: - json.dump(self.noise_map.tolist(), f, indent=4) - - def save_results(self, paths: af.DirectoryPaths, result: af.Result): - """ - At the end of a model-fit, this routine saves attributes of the `Analysis` - object to the `files` folder such that they can be loaded after the analysis - using PyAutoFit's database and aggregator tools. - - For this analysis it outputs the following: - - - The maximum log likelihood model data as a .json file. - - This is accessed using the aggregator via `agg.values("model_data")`. - - Parameters - ---------- - paths - The paths object which manages all paths, e.g. where the - non-linear search outputs are stored, visualization and the pickled - objects used by the aggregator output by this function. - result - The result of a model fit, including the non-linear search, samples - and maximum likelihood model. - """ - xvalues = np.arange(self.data.shape[0]) - - instance = result.max_log_likelihood_instance - - model_data = instance.model_data_from(xvalues=xvalues) - - # The path where model_data.json is saved, e.g. output/dataset_name/unique_id/files/model_data.json - - file_path = (path.join(paths._files_path, "model_data.json"),) - - with open(file_path, "w+") as f: - json.dump(model_data, f, indent=4) \ No newline at end of file diff --git a/docs/cookbooks/configs.md b/docs/cookbooks/configs.md new file mode 100644 index 000000000..9fa13afaf --- /dev/null +++ b/docs/cookbooks/configs.md @@ -0,0 +1,279 @@ +(configs)= + +# Configs + +Configuration files are used to control the behaviour model components in **PyAutoFit**, which perform the +following tasks: + +- Specify the default priors of model components, so that a user does not have to manually specify priors every time they create a model. +- Specify labels of every parameter, which are used for plotting and visualizing results. + +This cookbook illustrates how to create configuration files for your own model components, so that they can be used +with **PyAutoFit**. + +**Contents:** + +- **No Config Behaviour**: An example of what happens when a model component does not have a config file. +- **Template**: A template config file for specifying default model component priors. +- **Modules**: Writing prior config files based on the Python module the model component Python class is contained in. +- **Labels**: Config files which specify the labels of model component parameters for visualization. + +## No Config Behaviour + +The examples seen so far have used `Gaussian` and `Exponential` model components, which have configuration files in +the `autofit_workspace/config/priors` folder which define their priors and labels. + +If a model component does not have a configuration file and we try to use it in a fit, **PyAutoFit** will raise an +error. + +Lets illustrate this by setting up the usual Gaussian object, but naming it `GaussianNoConfig` so that it does +not have a config file. + +```python +class GaussianNoConfig: + def __init__( + self, + centre=0.0, # <- PyAutoFit recognises these constructor arguments + normalization=0.1, # <- are the Gaussian`s model parameters. + sigma=0.01, + ): + """ + Represents a 1D `Gaussian` profile, which does not have a config file set up. + """ + self.centre = centre + self.normalization = normalization + self.sigma = sigma + + def model_data_from(self, xvalues: np.ndarray) -> np.ndarray: + """ + The usual method that returns the 1D data of the `Gaussian` profile. + """ + transformed_xvalues = xvalues - self.centre + + return np.multiply( + np.divide(self.normalization, self.sigma * np.sqrt(2.0 * np.pi)), + np.exp(-0.5 * np.square(np.divide(transformed_xvalues, self.sigma))), + ) +``` + +When we try make this a `Model` and fit it, **PyAutoFit** raises an error, as it does not know where the priors +of the `GaussianNoConfig` are located. + +```python +model = af.Model(GaussianNoConfig) + +dataset_path = path.join("dataset", "example_1d", "gaussian_x1") +data = af.util.numpy_array_from_json(file_path=path.join(dataset_path, "data.json")) +noise_map = af.util.numpy_array_from_json( + file_path=path.join(dataset_path, "noise_map.json") +) + +analysis = af.ex.Analysis(data=data, noise_map=noise_map) + +search = af.DynestyStatic() + +result = search.fit(model=model, analysis=analysis) +``` + +In all other examples, the fits runs because the priors have been defined in one of two ways: + +- They were manually input in the example script. +- They were loaded via config files "behind the scenes". + +Checkout the folder `autofit_workspace/config/priors`, where .yaml files defining the priors of the `Gaussian` and +`Exponential` model components are located. These are the config files that **PyAutoFit** loads in the background +in order to setup the default priors of these model components. + +If we do not manually override priors, these are the priors that will be used by default when a model-fit is performed. + +## Templates + +For your model-fitting task, you therefore should set up a config file for every model component you defining its +default priors. + +Next, inspect the `TemplateObject.yaml` priors configuration file in `autofit_workspace/config/priors`. + +You should see the following text: + +```bash +parameter0: + type: Uniform + lower_limit: 0.0 + upper_limit: 1.0 +parameter1: + type: TruncatedGaussian + mean: 0.0 + sigma: 0.1 + lower_limit: 0.0 + upper_limit: inf +parameter2: + type: Uniform + lower_limit: 0.0 + upper_limit: 10.0 +``` + +This specifies the default priors on two parameters, named `parameter0` and `parameter1`. + +The `type` is the type of prior assumed by **PyAutoFit** by default for its corresponding parameter, where in this +example: + +- `parameter0` is given a `UniformPrior` with limits between 0.0 and 1.0. +- `parameter1` a `GaussianPrior` with mean 0.0 and sigma 1.0. +- `parameter2` is given a `UniformPrior` with limits between 0.0 and 10.0. + +The `lower_limit` and `upper_limit` of a `GaussianPrior` define the boundaries of what parameter values are +physically allowed. If a model-component is given a value outside these limits during model-fitting the model is +instantly resampled and discarded. + +We can easily adapt this template for any model component, for example the `GaussianNoConfig`. + +First, copy and paste the `TemplateObject.yaml` file to create a new file called `GaussianNoConfig.yaml`. + +The name of the class is matched to the name of the configuration file, therefore it is a requirement that the +configuration file is named `GaussianNoConfig.yaml` so that **PyAutoFit** can associate it with the `GaussianNoConfig` +Python class. + +Now perform the follow changes to the `.yaml` file: + +- Rename `parameter0` to `centre` and updates its uniform prior to be from a `lower_limit` of 0.0 and an `upper_limit` of 100.0. +- Rename `parameter1` to `normalization`. +- Rename `parameter2` to `sigma`. + +The `.yaml` file should read as follows: + +```bash +centre: + type: Uniform + lower_limit: 0.0 + upper_limit: 100.0 +normalization: + type: TruncatedGaussian + mean: 0.0 + sigma: 0.1 + lower_limit: 0.0 + upper_limit: inf +sigma: + type: Uniform + lower_limit: 0.0 + upper_limit: 10.0 +``` + +We should now be able to make a `Model` of the `GaussianNoConfig` class and fit it, without manually specifying +the priors. + +You may need to reset your Jupyter notebook's kernel for the changes to the `.yaml` file to take effect. + +```python +model = af.Model(GaussianNoConfig) + +dataset_path = path.join("dataset", "example_1d", "gaussian_x1") +data = af.util.numpy_array_from_json(file_path=path.join(dataset_path, "data.json")) +noise_map = af.util.numpy_array_from_json( + file_path=path.join(dataset_path, "noise_map.json") +) + +analysis = af.ex.Analysis(data=data, noise_map=noise_map) + +search = af.DynestyStatic() + +result = search.fit(model=model, analysis=analysis) +``` + +## Modules + +For larger projects, it may not be ideal to have to write a .yaml file for every Python class which acts as a model +component. + +We instead would prefer them to be in their own dedicated Python module. + +Suppose the `Gaussian` and `Exponential` model components were contained in a module named `profiles.py` in your +project's source code. + +You could then write a priors .yaml config file following the format given in the example config file +`autofit_workspace/config/priors/profiles.yaml`, noting that there is a paring between the module name +(`profiles.py`) and the name of the `.yaml` file (`profiles.yaml`). + +The file `autofit_workspace/config/priors/template_module.yaml` provides the tempolate for module based prior +configs and reads as follows: + +```bash +ModelComponent0: + parameter0: + type: Uniform + lower_limit: 0.0 + upper_limit: 1.0 + parameter1: + type: LogUniform + lower_limit: 1.0e-06 + upper_limit: 1000000.0 + parameter2: + type: Uniform + lower_limit: 0.0 + upper_limit: 25.0 +ModelComponent1: + parameter0: + type: Uniform + lower_limit: 0.0 + upper_limit: 1.0 + parameter1: + type: LogUniform + lower_limit: 1.0e-06 + upper_limit: 1000000.0 + parameter2: + type: Uniform + lower_limit: 0.0 + upper_limit: 1.0 +``` + +This looks very similar to `TemplateObject`, the only differences are: + +- It now contains the model-component class name in the configuration file, e.g. `ModelComponent0`, `ModelComponent1`. +- It includes multiple model-components, whereas `TemplateObject.yaml` corresponded to only one model component. + +## Labels + +There is an optional configs which associate model parameters with labels: + +`autofit_workspace/config/notation.yaml` + +It includes a `label` section which pairs every parameter with a label, which is used when visualizing results +(e.g. these labels are used when creating a corner plot). + +```bash +label: + label: + sigma: \sigma + centre: x + normalization: norm + parameter0: a + parameter1: b + parameter2: c + rate: \lambda +``` + +It also contains a `superscript` section which pairs every model-component label with a superscript, so that +models with the same parameter names (e.g. `centre` can be distinguished). + +```bash +label: + superscript: + Exponential: e + Gaussian: g + ModelComponent0: M0 + ModelComponent1: M1 +``` + +The `label_format` section sets Python formatting options for every parameter, controlling how they display in +the `model.results` file. + +```bash +label_format: + format: + sigma: '{:.2f}' + centre: '{:.2f}' + normalization: '{:.2f}' + parameter0: '{:.2f}' + parameter1: '{:.2f}' + parameter2: '{:.2f}' + rate: '{:.2f}' +``` diff --git a/docs/cookbooks/configs.rst b/docs/cookbooks/configs.rst deleted file mode 100644 index 469caf94e..000000000 --- a/docs/cookbooks/configs.rst +++ /dev/null @@ -1,285 +0,0 @@ -.. _configs: - -Configs -======= - -Configuration files are used to control the behaviour model components in **PyAutoFit**, which perform the -following tasks: - -- Specify the default priors of model components, so that a user does not have to manually specify priors every time they create a model. - -- Specify labels of every parameter, which are used for plotting and visualizing results. - -This cookbook illustrates how to create configuration files for your own model components, so that they can be used -with **PyAutoFit**. - -**Contents:** - -- **No Config Behaviour**: An example of what happens when a model component does not have a config file. -- **Template**: A template config file for specifying default model component priors. -- **Modules**: Writing prior config files based on the Python module the model component Python class is contained in. -- **Labels**: Config files which specify the labels of model component parameters for visualization. - -No Config Behaviour -------------------- - -The examples seen so far have used ``Gaussian`` and ``Exponential`` model components, which have configuration files in -the ``autofit_workspace/config/priors`` folder which define their priors and labels. - -If a model component does not have a configuration file and we try to use it in a fit, **PyAutoFit** will raise an -error. - -Lets illustrate this by setting up the usual Gaussian object, but naming it ``GaussianNoConfig`` so that it does -not have a config file. - -.. code-block:: python - - class GaussianNoConfig: - def __init__( - self, - centre=0.0, # <- PyAutoFit recognises these constructor arguments - normalization=0.1, # <- are the Gaussian`s model parameters. - sigma=0.01, - ): - """ - Represents a 1D `Gaussian` profile, which does not have a config file set up. - """ - self.centre = centre - self.normalization = normalization - self.sigma = sigma - - def model_data_from(self, xvalues: np.ndarray) -> np.ndarray: - """ - The usual method that returns the 1D data of the `Gaussian` profile. - """ - transformed_xvalues = xvalues - self.centre - - return np.multiply( - np.divide(self.normalization, self.sigma * np.sqrt(2.0 * np.pi)), - np.exp(-0.5 * np.square(np.divide(transformed_xvalues, self.sigma))), - ) - -When we try make this a ``Model`` and fit it, **PyAutoFit** raises an error, as it does not know where the priors -of the ``GaussianNoConfig`` are located. - -.. code-block:: python - - model = af.Model(GaussianNoConfig) - - dataset_path = path.join("dataset", "example_1d", "gaussian_x1") - data = af.util.numpy_array_from_json(file_path=path.join(dataset_path, "data.json")) - noise_map = af.util.numpy_array_from_json( - file_path=path.join(dataset_path, "noise_map.json") - ) - - analysis = af.ex.Analysis(data=data, noise_map=noise_map) - - search = af.DynestyStatic() - - result = search.fit(model=model, analysis=analysis) - -In all other examples, the fits runs because the priors have been defined in one of two ways: - -- They were manually input in the example script. -- They were loaded via config files "behind the scenes". - -Checkout the folder ``autofit_workspace/config/priors``, where .yaml files defining the priors of the ``Gaussian`` and -``Exponential`` model components are located. These are the config files that **PyAutoFit** loads in the background -in order to setup the default priors of these model components. - -If we do not manually override priors, these are the priors that will be used by default when a model-fit is performed. - -Templates ---------- - -For your model-fitting task, you therefore should set up a config file for every model component you defining its -default priors. - -Next, inspect the ``TemplateObject.yaml`` priors configuration file in ``autofit_workspace/config/priors``. - -You should see the following text: - -.. code-block:: bash - - parameter0: - type: Uniform - lower_limit: 0.0 - upper_limit: 1.0 - parameter1: - type: TruncatedGaussian - mean: 0.0 - sigma: 0.1 - lower_limit: 0.0 - upper_limit: inf - parameter2: - type: Uniform - lower_limit: 0.0 - upper_limit: 10.0 - -This specifies the default priors on two parameters, named ``parameter0`` and ``parameter1``. - -The ``type`` is the type of prior assumed by **PyAutoFit** by default for its corresponding parameter, where in this -example: - -- ``parameter0`` is given a ``UniformPrior`` with limits between 0.0 and 1.0. -- ``parameter1`` a ``GaussianPrior`` with mean 0.0 and sigma 1.0. -- ``parameter2`` is given a ``UniformPrior`` with limits between 0.0 and 10.0. - -The ``lower_limit`` and ``upper_limit`` of a ``GaussianPrior`` define the boundaries of what parameter values are -physically allowed. If a model-component is given a value outside these limits during model-fitting the model is -instantly resampled and discarded. - -We can easily adapt this template for any model component, for example the ``GaussianNoConfig``. - -First, copy and paste the ``TemplateObject.yaml`` file to create a new file called ``GaussianNoConfig.yaml``. - -The name of the class is matched to the name of the configuration file, therefore it is a requirement that the -configuration file is named ``GaussianNoConfig.yaml`` so that **PyAutoFit** can associate it with the ``GaussianNoConfig`` -Python class. - -Now perform the follow changes to the ``.yaml`` file: - -- Rename ``parameter0`` to ``centre`` and updates its uniform prior to be from a ``lower_limit`` of 0.0 and an ``upper_limit`` of 100.0. -- Rename ``parameter1`` to ``normalization``. -- Rename ``parameter2`` to ``sigma``. - -The ``.yaml`` file should read as follows: - -.. code-block:: bash - - centre: - type: Uniform - lower_limit: 0.0 - upper_limit: 100.0 - normalization: - type: TruncatedGaussian - mean: 0.0 - sigma: 0.1 - lower_limit: 0.0 - upper_limit: inf - sigma: - type: Uniform - lower_limit: 0.0 - upper_limit: 10.0 - -We should now be able to make a ``Model`` of the ``GaussianNoConfig`` class and fit it, without manually specifying -the priors. - -You may need to reset your Jupyter notebook's kernel for the changes to the ``.yaml`` file to take effect. - -.. code-block:: python - - model = af.Model(GaussianNoConfig) - - dataset_path = path.join("dataset", "example_1d", "gaussian_x1") - data = af.util.numpy_array_from_json(file_path=path.join(dataset_path, "data.json")) - noise_map = af.util.numpy_array_from_json( - file_path=path.join(dataset_path, "noise_map.json") - ) - - analysis = af.ex.Analysis(data=data, noise_map=noise_map) - - search = af.DynestyStatic() - - result = search.fit(model=model, analysis=analysis) - -Modules -------- - -For larger projects, it may not be ideal to have to write a .yaml file for every Python class which acts as a model -component. - -We instead would prefer them to be in their own dedicated Python module. - -Suppose the ``Gaussian`` and ``Exponential`` model components were contained in a module named ``profiles.py`` in your -project's source code. - -You could then write a priors .yaml config file following the format given in the example config file -``autofit_workspace/config/priors/profiles.yaml``, noting that there is a paring between the module name -(``profiles.py``) and the name of the ``.yaml`` file (``profiles.yaml``). - -The file ``autofit_workspace/config/priors/template_module.yaml`` provides the tempolate for module based prior -configs and reads as follows: - -.. code-block:: bash - - ModelComponent0: - parameter0: - type: Uniform - lower_limit: 0.0 - upper_limit: 1.0 - parameter1: - type: LogUniform - lower_limit: 1.0e-06 - upper_limit: 1000000.0 - parameter2: - type: Uniform - lower_limit: 0.0 - upper_limit: 25.0 - ModelComponent1: - parameter0: - type: Uniform - lower_limit: 0.0 - upper_limit: 1.0 - parameter1: - type: LogUniform - lower_limit: 1.0e-06 - upper_limit: 1000000.0 - parameter2: - type: Uniform - lower_limit: 0.0 - upper_limit: 1.0 - -This looks very similar to ``TemplateObject``, the only differences are: - -- It now contains the model-component class name in the configuration file, e.g. ``ModelComponent0``, ``ModelComponent1``. -- It includes multiple model-components, whereas ``TemplateObject.yaml`` corresponded to only one model component. - -Labels ------- - -There is an optional configs which associate model parameters with labels: - -``autofit_workspace/config/notation.yaml`` - -It includes a ``label`` section which pairs every parameter with a label, which is used when visualizing results -(e.g. these labels are used when creating a corner plot). - -.. code-block:: bash - - label: - label: - sigma: \sigma - centre: x - normalization: norm - parameter0: a - parameter1: b - parameter2: c - rate: \lambda - -It also contains a ``superscript`` section which pairs every model-component label with a superscript, so that -models with the same parameter names (e.g. ``centre`` can be distinguished). - -.. code-block:: bash - - label: - superscript: - Exponential: e - Gaussian: g - ModelComponent0: M0 - ModelComponent1: M1 - -The ``label_format`` section sets Python formatting options for every parameter, controlling how they display in -the ``model.results`` file. - -.. code-block:: bash - - label_format: - format: - sigma: '{:.2f}' - centre: '{:.2f}' - normalization: '{:.2f}' - parameter0: '{:.2f}' - parameter1: '{:.2f}' - parameter2: '{:.2f}' - rate: '{:.2f}' \ No newline at end of file diff --git a/docs/cookbooks/model.md b/docs/cookbooks/model.md new file mode 100644 index 000000000..02853d8b4 --- /dev/null +++ b/docs/cookbooks/model.md @@ -0,0 +1,1231 @@ +(model)= + +# Model + +Model composition is the process of defining a probabilistic model as a collection of model components, which are +ultimate fitted to a dataset via a non-linear search. + +This cookbook provides an overview of basic model composition tools. + +**Contents:** + +**Models:** + +If first describes how to use the `af.Model` object to define models with a single model component from single +Python classes, with the following sections: + +- **Python Class Template**: The template of a model component written as a Python class. +- **Model Composition (Model)**: Creating a model via `af.Model()`. +- **Priors (Model)**: How the default priors of a model are set and how to customize them. +- **Instances (Model)**: Creating an instance of a model via input parameters. +- **Model Customization (Model)**: Customizing a model (e.g. fixing parameters or linking them to one another). +- **Tuple Parameters (Model)**: Defining model components with parameters that are tuples. +- **Json Output (Model)**: Output a model in human readable text via a .json file and loading it back again. + +**Collections:** + +It then describes how to use the `af.Collection` object to define models with many model components from multiple +Python classes, with the following sections: + +- **Model Composition (Collection)**: Creating a model via `af.Collection()`. +- **Priors (Collection)**: How the default priors of a collection are set and how to customize them. +- **Instances (Collection)**: Create an instance of a collection via input parameters. +- **Model Customization (Collection)**: Customize a collection (e.g. fixing parameters or linking them to one another). +- **Json Output (Collection)**: Output a collection in human readable text via a .json file and loading it back again. +- **Extensible Models (Collection)**: Using collections to extend models with new model components, including the use of Python dictionaries and lists. + +**Arrays:** + +The cookbook next describes using NumPy arrays via tbe `af.Array` object to compose models, where each entry of the +array is a free parameters, therefore offering maximum flexibility with the number of free parameter. This has +the following sections: + +> - **Model Composition (af.Array)**: Composing models using NumPy arrays and af.Array\`(). +> +> - **Prior Customization (af.Array)**: How to customize the priors of a numpy array model. +> +> - **Instances (af.Array)**: Create an instance of a numpy array model via input parameters. +> +> - **Model Customization (af.Array):** Customize a numpy array model (e.g. fixing parameters or linking them to one another). +> +> - **Json Output (af.Array)**: Output a numpy array model in human readable text via a .json file and loading it back again. +> +> - **Extensible Models (af.Array)**: Using numpy arrays to compose models with a flexible number of parameters. + +## Python Class Template + +A model component is written as a Python class using the following format: + +- The name of the class is the name of the model component, in this case, “Gaussian”. +- The input arguments of the constructor are the parameters of the mode (here `centre`, `normalization` and `sigma`). +- The default values of the input arguments tell PyAutoFit whether a parameter is a single-valued float or a multi-valued tuple. + +We define a 1D Gaussian model component to illustrate model composition in PyAutoFit. + +```python +class Gaussian: + def __init__( + self, + centre : float = 30.0, # <- **PyAutoFit** recognises these constructor arguments + normalization : float = 1.0, # <- are the Gaussian``s model parameters. + sigma : float = 5.0, + ): + self.centre = centre + self.normalization = normalization + self.sigma = sigma +``` + +## Model Composition (Model) + +We can instantiate a Python class as a model component using `af.Model()`. + +```python +model = af.Model(Gaussian) +``` + +The model has 3 free parameters, corresponding to the 3 parameters defined above (`centre`, `normalization` +and `sigma`). + +Each parameter has a prior associated with it, meaning they are fitted for if the model is passed to a non-linear +search. + +```python +print(f"Model Total Free Parameters = {model.total_free_parameters}") +``` + +If we print the `info` attribute of the model we get information on all of the parameters and their priors. + +```python +print(model.info) +``` + +This gives the following output: + +```bash +Total Free Parameters = 3 + +model Gaussian (N=3) + +centre UniformPrior [1], lower_limit = 0.0, upper_limit = 100.0 +normalization LogUniformPrior [2], lower_limit = 1e-06, upper_limit = 1000000.0 +sigma UniformPrior [3], lower_limit = 0.0, upper_limit = 25.0 +``` + +## Priors (Model) + +The model has a set of default priors, which have been loaded from a config file in the PyAutoFit workspace. + +The config cookbook describes how to setup config files in order to produce custom priors, which means that you do not +need to manually specify priors in your Python code every time you compose a model. + +If you do not setup config files, all priors must be manually specified before you fit the model, as shown below. + +```python +model = af.Model(Gaussian) +model.centre = af.UniformPrior(lower_limit=0.0, upper_limit=100.0) +model.normalization = af.LogUniformPrior(lower_limit=1e-4, upper_limit=1e4) +model.sigma = af.GaussianPrior(mean=0.0, sigma=1.0, lower_limit=0.0, upper_limit=1e5) +``` + +## Instances (Model) + +Instances of the model components above (created via `af.Model`) can be created, where an input `vector` of +parameters is mapped to create an instance of the Python class of the model. + +We first need to know the order of parameters in the model, so we know how to define the input `vector`. This +information is contained in the models `paths` attribute: + +```python +print(model.paths) +``` + +The paths appear as follows: + +```bash +[('centre',), ('normalization',), ('sigma',)] +``` + +We create an `instance` of the `Gaussian` class via the model where `centre=30.0`, `normalization=2.0` and `sigma=3.0`. + +```python +instance = model.instance_from_vector(vector=[30.0, 2.0, 3.0]) + +print("Model Instance: \n") +print(instance) + +print("Instance Parameters \n") +print("centre = ", instance.centre) +print("normalization = ", instance.normalization) +print("sigma = ", instance.sigma) +``` + +This gives the following output: + +```bash +Model Instance: +<__main__.Gaussian object at 0x7f6f11d437c0> + +Instance Parameters + +centre = 30.0 +normalization = 2.0 +sigma = 3.0 +``` + +We can create an `instance` by inputting unit values (e.g. between 0.0 and 1.0) which are mapped to the input values +via the priors. + +The inputs of 0.5 below are mapped as follows: + +- `centre`: goes to 0.5 because this is the midpoint of a `UniformPrior` with `lower_limit=0.0` and `upper_limit=1.0`. +- `normalization` goes to 1.0 because this is the midpoint of the `LogUniformPrior`' with `lower_limit=1e-4` and `upper_limit=1e4` corresponding to log10 space. +- `sigma`: goes to 0.0 because this is the `mean` of the `GaussianPrior`. + +```python +instance = model.instance_from_unit_vector(unit_vector=[0.5, 0.5, 0.5]) + +print("Model Instance:\n") +print(instance) + +print("\nInstance Parameters \n") +print("centre = ", instance.centre) +print("normalization = ", instance.normalization) +print("sigma = ", instance.sigma) +``` + +This gives the following output: + +```bash +Model Instance: +<__main__.Gaussian object at 0x7f6f11d43f70> + +Instance Parameters + +centre = 50.0 +normalization = 1.0 +sigma = 0.0 +``` + +We can create instances of the `Gaussian` using the median value of the prior of every parameter. + +```python +instance = model.instance_from_prior_medians() + +print("Instance Parameters \n") +print("centre = ", instance.centre) +print("normalization = ", instance.normalization) +print("sigma = ", instance.sigma) +``` + +This gives the following output: + +```bash +Instance Parameters + +centre = 50.0 +normalization = 1.0 +sigma = 0.0 +``` + +We can create a random instance, where the random values are unit values drawn between 0.0 and 1.0. + +This means the parameter values of this instance are randomly drawn from the priors. + +```python +model = af.Model(Gaussian) +instance = model.random_instance() +``` + +## Model Customization (Model) + +We can fix a free parameter to a specific value (reducing the dimensionality of parameter space by 1): + +```python +model = af.Model(Gaussian) +model.centre = 0.0 +``` + +We can link two parameters together such they always assume the same value (reducing the dimensionality of +parameter space by 1): + +```python +model.centre = model.normalization +``` + +Offsets between linked parameters or with certain values are possible: + +```python +model.centre = model.normalization + model.sigma +``` + +Assertions remove regions of parameter space (but do not reduce the dimensionality of parameter space): + +```python +model.add_assertion(model.sigma > 5.0) +model.add_assertion(model.centre > model.normalization) +``` + +The customized model can be inspected by printing its `info` attribute. + +```python +print(model.info) +``` + +This gives the following output: + +```bash +Total Free Parameters = 2 + +model Gaussian (N=2) + centre SumPrior (N=2) + +centre + self LogUniformPrior [14], lower_limit = 1e-06, upper_limit = 1000000.0 + other UniformPrior [15], lower_limit = 0.0, upper_limit = 25.0 +normalization LogUniformPrior [14], lower_limit = 1e-06, upper_limit = 1000000.0 +sigma UniformPrior [15], lower_limit = 0.0, upper_limit = 25.0 +``` + +The overwriting of priors shown above can be achieved via the following alternative API: + +```python +model = af.Model( + Gaussian, + centre=af.UniformPrior(lower_limit=0.0, upper_limit=1.0), + normalization=af.LogUniformPrior(lower_limit=1e-4, upper_limit=1e4), + sigma=af.GaussianPrior(mean=0.0, sigma=1.0), +) +``` + +This API can also be used for fixing a parameter to a certain value: + +```python +model = af.Model(Gaussian, centre=0.0) +``` + +## Tuple Parameters (Model) + +The `Gaussian` model component above only has parameters that are single-valued floats. + +Parameters can also be tuples, which is useful for defining model components where certain parameters are naturally +grouped together. + +For example, we can define a 2D Gaussian with a center that has two coordinates and therefore free parameters, (x, y), +using a tuple. + +```python +class Gaussian2D: + def __init__( + self, + centre: Tuple[float, float] = (0.0, 0.0), # <- **PyAutoFit** recognises these constructor arguments + normalization: float = 0.1, # <- are the Gaussian``s model parameters. + sigma: float = 1.0, + ): + self.centre = centre + self.normalization = normalization + self.sigma = sigma +``` + +The model's `total_free_parameters` attribute now includes 4 free parameters, as the tuple `centre` parameter accounts +for 2 free parameters. + +```python +model = af.Model(Gaussian2D) + +print(f"Model Total Free Parameters = {model.total_free_parameters}") +``` + +This information is again displayed in the `info` attribute: + +```python +print(model.info) +``` + +This gives the following output: + +```bash +Total Free Parameters = 4 + +model Gaussian2D (N=4) + +centre + centre_0 UniformPrior [3], lower_limit = 0.0, upper_limit = 100.0 + centre_1 UniformPrior [4], lower_limit = 0.0, upper_limit = 100.0 +normalization LogUniformPrior [5], lower_limit = 1e-06, upper_limit = 1000000.0 +sigma UniformPrior [6], lower_limit = 0.0, upper_limit = 25.0 +``` + +Here are examples of how model customization can be applied to a model with tuple parameters: + +```python +model = af.Model(Gaussian2D) +model.centre = (0.0, 0.0) + +model.centre_0 = model.normalization + +model.centre_1 = model.normalization + model.sigma + +model.add_assertion(model.centre_0 > model.normalization) +``` + +## Json Outputs (Model) + +A model has a `dict` attribute, which expresses all information about the model as a Python dictionary. + +By printing this dictionary we can therefore get a concise summary of the model. + +```python +model = af.Model(Gaussian) + +print(model.dict()) +``` + +This gives the following output: + +```bash +{ + 'class_path': '__main__.Gaussian', 'type': 'model', + 'centre': {'lower_limit': 0.0, 'upper_limit': 100.0, 'type': 'Uniform'}, + 'normalization': {'lower_limit': 1e-06, 'upper_limit': 1000000.0, 'type': 'LogUniform'}, + 'sigma': {'lower_limit': 0.0, 'upper_limit': 25.0, 'type': 'Uniform'} +} +``` + +The dictionary representation printed above can be saved to hard disk as a `.json` file. + +This means we can save any **PyAutoFit** model to hard-disk in a human readable format. + +Checkout the file `autofit_workspace/*/cookbooks/jsons/model.json` to see the model written as a .json. + +```python +model_path = path.join("scripts", "cookbooks", "jsons") + +os.makedirs(model_path, exist_ok=True) + +model_file = path.join(model_path, "model.json") + +with open(model_file, "w+") as f: + json.dump(model.dict(), f, indent=4) +``` + +We can load the model from its `.json` file, meaning that one can easily save a model to hard disk and load it +elsewhere. + +```python +model = af.Model.from_json(file=model_file) +``` + +## Model Composition (Collection) + +To illustrate `Collection` objects we define a second model component, representing a `Exponential` profile. + +```python +class Exponential: + def __init__( + self, + centre=0.0, # <- PyAutoFit recognises these constructor arguments are the model + normalization=0.1, # <- parameters of the Exponential. + rate=0.01, + ): + self.centre = centre + self.normalization = normalization + self.rate = rate +``` + +To instantiate multiple Python classes into a combined model component we combine the `af.Collection()` and `af.Model()` +objects. + +By passing the key word arguments `gaussian` and `exponential` below, these are used as the names of the attributes of +instances created using this model (which is illustrated clearly below). + +```python +model = af.Collection(gaussian=af.Model(Gaussian), exponential=af.Model(Exponential)) +``` + +We can check the model has a `total_free_parameters` of 6, meaning the 3 parameters defined +above (`centre`, `normalization`, `sigma` and `rate`) for both the `Gaussian` and `Exponential` classes all have +priors associated with them . + +This also means each parameter is fitted for if we fitted the model to data via a non-linear search. + +```python +print(f"Model Total Free Parameters = {model.total_free_parameters}") +``` + +Printing the `info` attribute of the model gives us information on all of the parameters. + +```python +print(model.info) +``` + +This gives the following output: + +```bash +Total Free Parameters = 6 + +model Collection (N=6) + gaussian Gaussian (N=3) + exponential Exponential (N=3) + +gaussian + centre UniformPrior [39], lower_limit = 0.0, upper_limit = 100.0 + normalization LogUniformPrior [40], lower_limit = 1e-06, upper_limit = 1000000.0 + sigma UniformPrior [41], lower_limit = 0.0, upper_limit = 25.0 +exponential + centre UniformPrior [42], lower_limit = 0.0, upper_limit = 100.0 + normalization LogUniformPrior [43], lower_limit = 1e-06, upper_limit = 1000000.0 + rate UniformPrior [44], lower_limit = 0.0, upper_limit = 1.0 +``` + +## Priors (Collection) + +The model has a set of default priors, which have been loaded from a config file in the PyAutoFit workspace. + +The configs cookbook describes how to setup config files in order to produce custom priors, which means that you do not +need to manually specify priors in your Python code every time you compose a model. + +If you do not setup config files, all priors must be manually specified before you fit the model, as shown below. + +```python +model.gaussian.centre = af.UniformPrior(lower_limit=0.0, upper_limit=100.0) +model.gaussian.normalization = af.UniformPrior(lower_limit=0.0, upper_limit=1e2) +model.gaussian.sigma = af.UniformPrior(lower_limit=0.0, upper_limit=30.0) +model.exponential.centre = af.UniformPrior(lower_limit=0.0, upper_limit=100.0) +model.exponential.normalization = af.UniformPrior(lower_limit=0.0, upper_limit=1e2) +model.exponential.rate = af.UniformPrior(lower_limit=0.0, upper_limit=10.0) +``` + +When creating a model via a `Collection`, there is no need to actually pass the python classes as an `af.Model()` +because **PyAutoFit** implicitly assumes they are to be created as a `Model()`. + +This enables more concise code, whereby the following code: + +```python +model = af.Collection(gaussian=af.Model(Gaussian), exponential=af.Model(Exponential)) +``` + +Can instead be written as: + +```python +model = af.Collection(gaussian=Gaussian, exponential=Exponential) +``` + +## Instances (Collection) + +We can create an instance of collection containing both the `Gaussian` and `Exponential` classes using this model. + +We create an `instance` where: + +- The `Gaussian` class has `centre=30.0`, `normalization=2.0` and `sigma=3.0`. +- The `Exponential` class has `centre=60.0`, `normalization=4.0` and ``` rate=1.0`` ```. + +```python +instance = model.instance_from_vector(vector=[30.0, 2.0, 3.0, 60.0, 4.0, 1.0]) +``` + +Because we passed the key word arguments `gaussian` and `exponential` above, these are the names of the attributes of +instances created using this model (e.g. this is why we write `instance.gaussian`): + +```python +print("Model Instance: \n") +print(instance) + +print("Instance Parameters \n") +print("centre (Gaussian) = ", instance.gaussian.centre) +print("normalization (Gaussian) = ", instance.gaussian.normalization) +print("sigma (Gaussian) = ", instance.gaussian.sigma) +print("centre (Exponential) = ", instance.exponential.centre) +print("normalization (Exponential) = ", instance.exponential.normalization) +print("rate (Exponential) = ", instance.exponential.rate) +``` + +This gives the following output: + +```bash +Model Instance: + + +Instance Parameters + +centre (Gaussian) = 30.0 +normalization (Gaussian) = 2.0 +sigma (Gaussian) = 3.0 +centre (Exponential) = 60.0 +normalization (Exponential) = 4.0 +rate (Exponential) = 1.0 +``` + +Alternatively, the instance's variables can also be accessed as a list, whereby instead of using attribute names +(e.g. `gaussian_0`) we input the list index. + +Note that the order of the instance model components is determined from the order the components are input into the +`Collection`. + +For example, for the line `af.Collection(gaussian=gaussian, exponential=exponential)`, the first entry in the list +is the gaussian because it is the first input to the `Collection`. + +```python +print("centre (Gaussian) = ", instance[0].centre) +print("normalization (Gaussian) = ", instance[0].normalization) +print("sigma (Gaussian) = ", instance[0].sigma) +print("centre (Gaussian) = ", instance[1].centre) +print("normalization (Gaussian) = ", instance[1].normalization) +print("rate (Exponential) = ", instance[1].rate) +``` + +This gives the following output: + +```bash +centre (Gaussian) = 30.0 +normalization (Gaussian) = 2.0 +sigma (Gaussian) = 3.0 +centre (Exponential) = 60.0 +normalization (Exponential) = 4.0 +rate (Exponential) = 1.0 +``` + +## Model Customization (Collection) + +By setting up each Model first the model can be customized using either of the API’s shown above: + +```python +gaussian = af.Model(Gaussian) +gaussian.normalization = 1.0 +gaussian.sigma = af.GaussianPrior(mean=0.0, sigma=1.0) + +exponential = af.Model(Exponential) +exponential.centre = 50.0 +exponential.add_assertion(exponential.rate > 5.0) + +model = af.Collection(gaussian=gaussian, exponential=exponential) + +print(model.info) +``` + +This gives the following output: + +```bash + +``` + +Total Free Parameters = 4 + +> model Collection (N=4) +> +> : gaussian Gaussian (N=2) +> exponential Exponential (N=2) +> +> gaussian +> +> : centre UniformPrior [71], lower_limit = 0.0, upper_limit = 100.0 +> normalization 1.0 +> sigma GaussianPrior [70], mean = 0.0, sigma = 1.0 +> +> exponential +> +> : centre 50.0 +> normalization LogUniformPrior [72], lower_limit = 1e-06, upper_limit = 1000000.0 +> rate UniformPrior [73], lower_limit = 0.0, upper_limit = 1.0 + +Below is an alternative API that can be used to create the same model as above. + +Which API is used is up to the user and which they find most intuitive. + +```python +gaussian = af.Model( + Gaussian, normalization=1.0, sigma=af.GaussianPrior(mean=0.0, sigma=1.0) +) +exponential = af.Model(Exponential, centre=50.0) +exponential.add_assertion(exponential.rate > 5.0) + +model = af.Collection(gaussian=gaussian, exponential=exponential) + +print(model.info) +``` + +This gives the following output: + +```bash +Total Free Parameters = 4 + +model Collection (N=4) + gaussian Gaussian (N=2) + exponential Exponential (N=2) + +gaussian + centre UniformPrior [63], lower_limit = 0.0, upper_limit = 100.0 + normalization 1.0 + sigma GaussianPrior [66], mean = 0.0, sigma = 1.0 +exponential + centre 50.0 + normalization LogUniformPrior [68], lower_limit = 1e-06, upper_limit = 1000000.0 + rate UniformPrior [69], lower_limit = 0.0, upper_limit = 1.0 +``` + +After creating the model as a `Collection` we can customize it afterwards: + +```python +model = af.Collection(gaussian=Gaussian, exponential=Exponential) + +model.gaussian.normalization = 1.0 +model.gaussian.sigma = af.GaussianPrior(mean=0.0, sigma=1.0) + +model.exponential.centre = 50.0 +model.exponential.add_assertion(exponential.rate > 5.0) + +print(model.info) +``` + +This gives the following output: + +```bash +Total Free Parameters = 4 + +model Collection (N=4) + gaussian Gaussian (N=2) + exponential Exponential (N=2) + +gaussian + centre UniformPrior [71], lower_limit = 0.0, upper_limit = 100.0 + normalization 1.0 + sigma GaussianPrior [70], mean = 0.0, sigma = 1.0 +exponential + centre 50.0 + normalization LogUniformPrior [72], lower_limit = 1e-06, upper_limit = 1000000.0 + rate UniformPrior [73], lower_limit = 0.0, upper_limit = 1.0 +``` + +## JSon Outputs (Collection) + +A `Collection` has a `dict` attribute, which express all information about the model as a Python dictionary. + +By printing this dictionary we can therefore get a concise summary of the model. + +```python +model = af.Model(Gaussian) + +print(model.dict()) +``` + +This gives the following output: + +```bash +{ + 'type': 'collection', + 'gaussian': { + 'class_path': '__main__.Gaussian', 'type': 'model', + 'centre': {'lower_limit': 0.0, 'upper_limit': 100.0, 'type': 'Uniform'}, + 'normalization': 1.0, 'sigma': {'lower_limit': -inf, 'upper_limit': inf, 'type': 'Gaussian', 'mean': 0.0, 'sigma': 1.0}}, + 'exponential': { + 'class_path': '__main__.Exponential', 'type': 'model', + 'centre': 50.0, + 'normalization': {'lower_limit': 1e-06, 'upper_limit': 1000000.0, 'type': 'LogUniform'}, + 'rate': {'lower_limit': 0.0, 'upper_limit': 1.0, 'type': 'Uniform'}} +} +``` + +Python dictionaries can easily be saved to hard disk as a `.json` file. + +This means we can save any **PyAutoFit** model to hard-disk. + +Checkout the file `autofit_workspace/*/model/jsons/collection.json` to see the model written as a .json. + +```python +model_path = path.join("scripts", "model", "jsons") + +os.makedirs(model_path, exist_ok=True) + +model_file = path.join(model_path, "collection.json") + +with open(model_file, "w+") as f: + json.dump(model.dict(), f, indent=4) +``` + +We can load the model from its `.json` file, meaning that one can easily save a model to hard disk and load it +elsewhere. + +```python +model = af.Model.from_json(file=model_file) + +print(f"\n Model via Json Prior Count = {model.prior_count}") +``` + +## Extensible Models (Collection) + +There is no limit to the number of components we can use to set up a model via a `Collection`. + +```python +model = af.Collection( + gaussian_0=Gaussian, + gaussian_1=Gaussian, + exponential_0=Exponential, + exponential_1=Exponential, + exponential_2=Exponential, +) + +print(model.info) +``` + +This gives the following output: + +```bash +Total Free Parameters = 15 + +model Collection (N=15) + gaussian_0 Gaussian (N=3) + gaussian_1 Gaussian (N=3) + exponential_0 Exponential (N=3) + exponential_1 Exponential (N=3) + exponential_2 Exponential (N=3) + +gaussian_0 + centre UniformPrior [91], lower_limit = 0.0, upper_limit = 100.0 + normalization LogUniformPrior [92], lower_limit = 1e-06, upper_limit = 1000000.0 + sigma UniformPrior [93], lower_limit = 0.0, upper_limit = 25.0 +gaussian_1 + centre UniformPrior [94], lower_limit = 0.0, upper_limit = 100.0 + normalization LogUniformPrior [95], lower_limit = 1e-06, upper_limit = 1000000.0 + sigma UniformPrior [96], lower_limit = 0.0, upper_limit = 25.0 +exponential_0 + centre UniformPrior [97], lower_limit = 0.0, upper_limit = 100.0 + normalization LogUniformPrior [98], lower_limit = 1e-06, upper_limit = 1000000.0 + rate UniformPrior [99], lower_limit = 0.0, upper_limit = 1.0 +exponential_1 + centre UniformPrior [100], lower_limit = 0.0, upper_limit = 100.0 + normalization LogUniformPrior [101], lower_limit = 1e-06, upper_limit = 1000000.0 + rate UniformPrior [102], lower_limit = 0.0, upper_limit = 1.0 +exponential_2 + centre UniformPrior [103], lower_limit = 0.0, upper_limit = 100.0 + normalization LogUniformPrior [104], lower_limit = 1e-06, upper_limit = 1000000.0 + rate UniformPrior [105], lower_limit = 0.0, upper_limit = 1.0 +Total Free Parameters = 6 + +model Collection (N=6) + gaussian_0 Gaussian (N=3) + gaussian_1 Gaussian (N=3) + +gaussian_0 + centre UniformPrior [106], lower_limit = 0.0, upper_limit = 100.0 + normalization LogUniformPrior [107], lower_limit = 1e-06, upper_limit = 1000000.0 + sigma UniformPrior [108], lower_limit = 0.0, upper_limit = 25.0 +gaussian_1 + centre UniformPrior [109], lower_limit = 0.0, upper_limit = 100.0 + normalization LogUniformPrior [110], lower_limit = 1e-06, upper_limit = 1000000.0 + sigma UniformPrior [111], lower_limit = 0.0, upper_limit = 25.0 +Total Free Parameters = 6 + +model Collection (N=6) + gaussian_0 Gaussian (N=3) + gaussian_1 Gaussian (N=3) + +gaussian_0 + centre UniformPrior [112], lower_limit = 0.0, upper_limit = 100.0 + normalization LogUniformPrior [113], lower_limit = 1e-06, upper_limit = 1000000.0 + sigma UniformPrior [114], lower_limit = 0.0, upper_limit = 25.0 +gaussian_1 + centre UniformPrior [115], lower_limit = 0.0, upper_limit = 100.0 + normalization LogUniformPrior [116], lower_limit = 1e-06, upper_limit = 1000000.0 + sigma UniformPrior [117], lower_limit = 0.0, upper_limit = 25.0 +``` + +A model can be created via `af.Collection()` where a dictionary of `af.Model()` objects are passed to it. + +The two models created below are identical- one uses the API detailed above whereas the second uses a dictionary. + +```python +model = af.Collection(gaussian_0=Gaussian, gaussian_1=Gaussian) + +model_dict = {"gaussian_0": Gaussian, "gaussian_1": Gaussian} +model = af.Collection(**model_dict) +``` + +The keys of the dictionary passed to the model (e.g. `gaussian_0` and `gaussian_1` above) are used to create the +names of the attributes of instances of the model. + +```python +instance = model.instance_from_vector(vector=[1.0, 2.0, 3.0, 4.0, 5.0, 6.0]) + +print("Model Instance: \n") +print(instance) + +print("Instance Parameters \n") +print("centre (Gaussian) = ", instance.gaussian_0.centre) +print("normalization (Gaussian) = ", instance.gaussian_0.normalization) +print("sigma (Gaussian) = ", instance.gaussian_0.sigma) +print("centre (Gaussian) = ", instance.gaussian_1.centre) +print("normalization (Gaussian) = ", instance.gaussian_1.normalization) +print("sigma (Gaussian) = ", instance.gaussian_1.sigma) +``` + +This gives the following output: + +```bash +Model Instance: + + +Instance Parameters: + +centre (Gaussian) = 1.0 +normalization (Gaussian) = 2.0 +sigma (Gaussian) = 3.0 +centre (Gaussian) = 4.0 +normalization (Gaussian) = 5.0 +sigma (Gaussian) = 6.0 +``` + +A list of model components can also be passed to an `af.Collection` to create a model: + +```python +model = af.Collection([Gaussian, Gaussian]) + +print(model.info) +``` + +When a list is used, there is no string with which to name the model components (e.g. we do not input `gaussian_0` +and `gaussian_1` anywhere. + +The `instance` therefore can only be accessed via list indexing. + +```python +instance = model.instance_from_vector(vector=[1.0, 2.0, 3.0, 4.0, 5.0, 6.0]) + +print("Model Instance: \n") +print(instance) + +print("Instance Parameters \n") +print("centre (Gaussian) = ", instance[0].centre) +print("normalization (Gaussian) = ", instance[0].normalization) +print("sigma (Gaussian) = ", instance[0].sigma) +print("centre (Gaussian) = ", instance[1].centre) +print("normalization (Gaussian) = ", instance[1].normalization) +print("sigma (Gaussian) = ", instance[1].sigma) +``` + +This gives the following output: + +```bash +Model Instance: + + +Instance Parameters: + +centre (Gaussian) = 1.0 +normalization (Gaussian) = 2.0 +sigma (Gaussian) = 3.0 +centre (Gaussian) = 4.0 +normalization (Gaussian) = 5.0 +sigma (Gaussian) = 6.0 +``` + +## Model Composition (af.Array) + +Models can be composed using NumPy arrays, where each element of the array is a free parameter. + +This offers a lot more flexibility than using `Model` and `Collection` objects, as the number of parameters in the +model is chosen on initialization via the input of the `shape` attribute. + +For many use cases, this flexibility is key to ensuring model composition is as easy as possible, for example when +a part of the model being fitted is a matrix of parameters which may change shape depending on the dataset being +fitted. + +To compose models using NumPy arrays, we use the `af.Array` object. + +```python +model = af.Array( + shape=(2, 2), + prior=af.GaussianPrior(mean=0.0, sigma=1.0), +) +``` + +Each element of the array is a free parameter, which for `shape=(2,2)` means the model has 4 free parameters. + +```python +print(f"Model Total Free Parameters = {model.total_free_parameters}") +``` + +The `info` attribute of the model gives information on all of the parameters and their priors. + +```python +print(model.info) +``` + +This gives the following output: + +```bash +Total Free Parameters = 4 + +model Array (N=4) + indices list (N=0) + +shape (2, 2) +indices + 0 (0, 0) + 1 (0, 1) + 2 (1, 0) + 3 (1, 1) +prior_0_0 GaussianPrior [124], mean = 0.0, sigma = 1.0 +prior_0_1 GaussianPrior [125], mean = 0.0, sigma = 1.0 +prior_1_0 GaussianPrior [126], mean = 0.0, sigma = 1.0 +prior_1_1 GaussianPrior [127], mean = 0.0, sigma = 1.0 +``` + +## Prior Customization (af.Array) + +The prior of every parameter in the array is set via the `prior` input above. + +NumPy array models do not currently support default priors via config files, so all priors must be manually specified. + +The prior of every parameter in the array can be customized by normal NumPy array indexing: + +```python +model = af.Array(shape=(2, 2), prior=af.GaussianPrior(mean=0.0, sigma=1.0)) + +model.array[0, 0] = af.UniformPrior(lower_limit=0.0, upper_limit=1.0) +model.array[0, 1] = af.LogUniformPrior(lower_limit=1e-4, upper_limit=1e4) +model.array[1, 0] = af.GaussianPrior(mean=0.0, sigma=2.0) +``` + +The `info` attribute shows the customized priors. + +```python +print(model.info) +``` + +The output is as follows: + +```bash +Total Free Parameters = 4 + +model Array (N=4) + indices list (N=0) + +shape (2, 2) +indices + 0 (0, 0) + 1 (0, 1) + 2 (1, 0) + 3 (1, 1) +prior_0_0 UniformPrior [133], lower_limit = 0.0, upper_limit = 1.0 +prior_0_1 LogUniformPrior [134], lower_limit = 0.0001, upper_limit = 10000.0 +prior_1_0 GaussianPrior [135], mean = 0.0, sigma = 2.0 +prior_1_1 GaussianPrior [132], mean = 0.0, sigma = 1.0 +``` + +## Instances (af.Array) + +Instances of numpy array model components can be created, where an input `vector` of parameters is mapped to create +an instance of the Python class of the model. + +If the priors of the numpy array are not customized, ordering of parameters goes from element [0,0] to [0,1] to [1,0], +as shown by the `paths` attribute. + +```python +model = af.Array( + shape=(2, 2), + prior=af.GaussianPrior(mean=0.0, sigma=1.0), +) + +print(model.paths) +``` + +The output is as follows: + +```bash +['prior_0_0', 'prior_0_1', 'prior_1_0', 'prior_1_1'] +``` + +An instance can then be created by passing a vector of parameters to the model via the `instance_from_vector` method. + +The `instance` created is a NumPy array, where each element is the value passed in the vector. + +```python +instance = model.instance_from_vector(vector=[0.0, 1.0, 2.0, 3.0]) + +print("\nModel Instance:") +print(instance) +``` + +The output is as follows: + +```bash +Model Instance: +[[0. 1.] +[2. 3.]] +``` + +Prior customization changes the order of the parameters, therefore if you customize the priors of the numpy +array you must check the ordering of the parameters in the `paths` attribute before passing a vector to +the `instance_from_vector` + +```python +model[0, 0] = af.UniformPrior(lower_limit=0.0, upper_limit=1.0) +model[0, 1] = af.LogUniformPrior(lower_limit=1e-4, upper_limit=1e4) +model[1, 0] = af.GaussianPrior(mean=0.0, sigma=2.0) + +print(model.paths) +``` + +The output is as follows: + +```bash +[('prior_1_1',), ('prior_0_0',), ('prior_0_1',), ('prior_1_0',)] +``` + +If we create a vector and print its values from this customized model: + +```python +instance = model.instance_from_vector(vector=[0.0, 1.0, 2.0, 3.0]) + +print("\nModel Instance:") +print(instance) +``` + +The output is as follows: + +```bash +Model Instance: +[[1. 2.] + [3. 0.]] +``` + +## Model Customization (af.Array) + +The model customization API for numpy array models is the same as for `af.Model` and `af.Collection` objects. + +```python +model = af.Array( + shape=(2, 2), + prior=af.GaussianPrior(mean=0.0, sigma=1.0), +) + +model[0,0] = 50.0 +model[0,1] = model[1,0] +model.add_assertion(model[1,1] > 0.0) + +print(model.info) +``` + +The output is as follows: + +```bash Total Free Parameters = 2 +model Array (N=2) + indices list (N=0) + +shape (2, 2) +indices + 0 (0, 0) + 1 (0, 1) + 2 (1, 0) + 3 (1, 1) +prior_0_0 50.0 +prior_0_1 - prior_1_0 GaussianPrior [147], mean = 0.0, sigma = 1.0 +prior_1_1 GaussianPrior [148], mean = 0.0, sigma = 1.0 +``` + +## JSon Outputs (af.Array) + +An `Array` has a `dict` attribute, which express all information about the model as a Python dictionary. + +By printing this dictionary we can therefore get a concise summary of the model. + +```python +model = af.Array( + shape=(2, 2), + prior=af.GaussianPrior(mean=0.0, sigma=1.0), +) + +print(model.dict()) +``` + +Python dictionaries can easily be saved to hard disk as a `.json` file. + +This means we can save any **PyAutoFit** model to hard-disk. + +Checkout the file `autofit_workspace/*/model/jsons/array.json` to see the model written as a .json. + +```python +model_path = path.join("scripts", "model", "jsons") + +os.makedirs(model_path, exist_ok=True) + +model_file = path.join(model_path, "array.json") + +with open(model_file, "w+") as f: + json.dump(model.dict(), f, indent=4) +``` + +We can load the model from its `.json` file, meaning that one can easily save a model to hard disk and load it +elsewhere. + +```python +model = af.Array.from_json(file=model_file) + +print(f"\n Model via Json Prior Count = {model.prior_count}") +``` + +## Extensible Models (af.Array) + +For `Model` objects, the number of parameters is fixed to those listed in the input Python class when the model is +created. + +For `Collection` objects, the use of dictionaries and lists allows for the number of parameters to be extended, but it +was still tied to the input Python classes when the model was created. + +For `Array` objects, the number of parameters is fully customizable, you choose the shape of the array and therefore +the number of parameters in the model when you create it. + +This makes `Array` objects the most extensible and flexible way to compose models. + +You can also combine `Array` objects with `Collection` objects to create models with a mix of fixed and extensible +parameters. + +```python +model = af.Collection( + gaussian=Gaussian, + array=af.Array(shape=(3, 2), prior=af.GaussianPrior(mean=0.0, sigma=1.0)) +) + +model.gaussian.sigma = 2.0 +model.array[0, 0] = 1.0 + +print(model.info) +``` + +The output is as follows: + +```python +Total Free Parameters = 7 + +model Collection (N=7) + gaussian Gaussian (N=2) + array Array (N=5) + indices list (N=0) + +gaussian + centre UniformPrior [165], lower_limit = 0.0, upper_limit = 100.0 + normalization LogUniformPrior [166], lower_limit = 1e-06, upper_limit = 1000000.0 + sigma 2.0 +array + shape (3, 2) + indices + 0 (0, 0) + 1 (0, 1) + 2 (1, 0) + 3 (1, 1) + 4 (2, 0) + 5 (2, 1) + prior_0_0 1.0 + prior_0_1 GaussianPrior [160], mean = 0.0, sigma = 1.0 + prior_1_0 GaussianPrior [161], mean = 0.0, sigma = 1.0 + prior_1_1 GaussianPrior [162], mean = 0.0, sigma = 1.0 + prior_2_0 GaussianPrior [163], mean = 0.0, sigma = 1.0 + prior_2_1 GaussianPrior [164], mean = 0.0, sigma = 1.0 +``` + +## Wrap Up + +This cookbook shows how to compose models consisting of multiple components using the `af.Model()` +and `af.Collection()` object. + +Advanced model composition uses multi-level models, which compose models from hierarchies of Python classes. This is +described in the multi-level model cookbook. diff --git a/docs/cookbooks/model.rst b/docs/cookbooks/model.rst deleted file mode 100644 index 38e0667c7..000000000 --- a/docs/cookbooks/model.rst +++ /dev/null @@ -1,1253 +0,0 @@ -.. _model: - -Model -===== - -Model composition is the process of defining a probabilistic model as a collection of model components, which are -ultimate fitted to a dataset via a non-linear search. - -This cookbook provides an overview of basic model composition tools. - -**Contents:** - -**Models:** - -If first describes how to use the ``af.Model`` object to define models with a single model component from single -Python classes, with the following sections: - -- **Python Class Template**: The template of a model component written as a Python class. -- **Model Composition (Model)**: Creating a model via ``af.Model()``. -- **Priors (Model)**: How the default priors of a model are set and how to customize them. -- **Instances (Model)**: Creating an instance of a model via input parameters. -- **Model Customization (Model)**: Customizing a model (e.g. fixing parameters or linking them to one another). -- **Tuple Parameters (Model)**: Defining model components with parameters that are tuples. -- **Json Output (Model)**: Output a model in human readable text via a .json file and loading it back again. - -**Collections:** - -It then describes how to use the ``af.Collection`` object to define models with many model components from multiple -Python classes, with the following sections: - -- **Model Composition (Collection)**: Creating a model via ``af.Collection()``. -- **Priors (Collection)**: How the default priors of a collection are set and how to customize them. -- **Instances (Collection)**: Create an instance of a collection via input parameters. -- **Model Customization (Collection)**: Customize a collection (e.g. fixing parameters or linking them to one another). -- **Json Output (Collection)**: Output a collection in human readable text via a .json file and loading it back again. -- **Extensible Models (Collection)**: Using collections to extend models with new model components, including the use of Python dictionaries and lists. - -**Arrays:** - -The cookbook next describes using NumPy arrays via tbe `af.Array` object to compose models, where each entry of the -array is a free parameters, therefore offering maximum flexibility with the number of free parameter. This has -the following sections: - - - **Model Composition (af.Array)**: Composing models using NumPy arrays and `af.Array`(). - - **Prior Customization (af.Array)**: How to customize the priors of a numpy array model. - - **Instances (af.Array)**: Create an instance of a numpy array model via input parameters. - - **Model Customization (af.Array):** Customize a numpy array model (e.g. fixing parameters or linking them to one another). - - **Json Output (af.Array)**: Output a numpy array model in human readable text via a .json file and loading it back again. - - **Extensible Models (af.Array)**: Using numpy arrays to compose models with a flexible number of parameters. - -Python Class Template ---------------------- - -A model component is written as a Python class using the following format: - -- The name of the class is the name of the model component, in this case, “Gaussian”. - -- The input arguments of the constructor are the parameters of the mode (here ``centre``, ``normalization`` and ``sigma``). - -- The default values of the input arguments tell PyAutoFit whether a parameter is a single-valued float or a multi-valued tuple. - -We define a 1D Gaussian model component to illustrate model composition in PyAutoFit. - -.. code-block:: python - - class Gaussian: - def __init__( - self, - centre : float = 30.0, # <- **PyAutoFit** recognises these constructor arguments - normalization : float = 1.0, # <- are the Gaussian``s model parameters. - sigma : float = 5.0, - ): - self.centre = centre - self.normalization = normalization - self.sigma = sigma - -Model Composition (Model) -------------------------- - -We can instantiate a Python class as a model component using ``af.Model()``. - -.. code-block:: python - - model = af.Model(Gaussian) - -The model has 3 free parameters, corresponding to the 3 parameters defined above (``centre``, ``normalization`` -and ``sigma``). - -Each parameter has a prior associated with it, meaning they are fitted for if the model is passed to a non-linear -search. - -.. code-block:: python - - print(f"Model Total Free Parameters = {model.total_free_parameters}") - -If we print the ``info`` attribute of the model we get information on all of the parameters and their priors. - -.. code-block:: python - - print(model.info) - -This gives the following output: - -.. code-block:: bash - - Total Free Parameters = 3 - - model Gaussian (N=3) - - centre UniformPrior [1], lower_limit = 0.0, upper_limit = 100.0 - normalization LogUniformPrior [2], lower_limit = 1e-06, upper_limit = 1000000.0 - sigma UniformPrior [3], lower_limit = 0.0, upper_limit = 25.0 - -Priors (Model) --------------- - -The model has a set of default priors, which have been loaded from a config file in the PyAutoFit workspace. - -The config cookbook describes how to setup config files in order to produce custom priors, which means that you do not -need to manually specify priors in your Python code every time you compose a model. - -If you do not setup config files, all priors must be manually specified before you fit the model, as shown below. - -.. code-block:: python - - model = af.Model(Gaussian) - model.centre = af.UniformPrior(lower_limit=0.0, upper_limit=100.0) - model.normalization = af.LogUniformPrior(lower_limit=1e-4, upper_limit=1e4) - model.sigma = af.GaussianPrior(mean=0.0, sigma=1.0, lower_limit=0.0, upper_limit=1e5) - -Instances (Model) ------------------ - -Instances of the model components above (created via ``af.Model``) can be created, where an input ``vector`` of -parameters is mapped to create an instance of the Python class of the model. - -We first need to know the order of parameters in the model, so we know how to define the input ``vector``. This -information is contained in the models ``paths`` attribute: - -.. code-block:: python - - print(model.paths) - -The paths appear as follows: - -.. code-block:: bash - - [('centre',), ('normalization',), ('sigma',)] - -We create an ``instance`` of the ``Gaussian`` class via the model where ``centre=30.0``, ``normalization=2.0`` and ``sigma=3.0``. - -.. code-block:: python - - instance = model.instance_from_vector(vector=[30.0, 2.0, 3.0]) - - print("Model Instance: \n") - print(instance) - - print("Instance Parameters \n") - print("centre = ", instance.centre) - print("normalization = ", instance.normalization) - print("sigma = ", instance.sigma) - -This gives the following output: - -.. code-block:: bash - - Model Instance: - <__main__.Gaussian object at 0x7f6f11d437c0> - - Instance Parameters - - centre = 30.0 - normalization = 2.0 - sigma = 3.0 - -We can create an ``instance`` by inputting unit values (e.g. between 0.0 and 1.0) which are mapped to the input values -via the priors. - -The inputs of 0.5 below are mapped as follows: - -- ``centre``: goes to 0.5 because this is the midpoint of a ``UniformPrior`` with ``lower_limit=0.0`` and ``upper_limit=1.0``. - -- ``normalization`` goes to 1.0 because this is the midpoint of the ``LogUniformPrior``' with ``lower_limit=1e-4`` and ``upper_limit=1e4`` corresponding to log10 space. - -- ``sigma``: goes to 0.0 because this is the ``mean`` of the ``GaussianPrior``. - -.. code-block:: python - - instance = model.instance_from_unit_vector(unit_vector=[0.5, 0.5, 0.5]) - - print("Model Instance:\n") - print(instance) - - print("\nInstance Parameters \n") - print("centre = ", instance.centre) - print("normalization = ", instance.normalization) - print("sigma = ", instance.sigma) - -This gives the following output: - -.. code-block:: bash - - Model Instance: - <__main__.Gaussian object at 0x7f6f11d43f70> - - Instance Parameters - - centre = 50.0 - normalization = 1.0 - sigma = 0.0 - -We can create instances of the ``Gaussian`` using the median value of the prior of every parameter. - -.. code-block:: python - - instance = model.instance_from_prior_medians() - - print("Instance Parameters \n") - print("centre = ", instance.centre) - print("normalization = ", instance.normalization) - print("sigma = ", instance.sigma) - -This gives the following output: - -.. code-block:: bash - - Instance Parameters - - centre = 50.0 - normalization = 1.0 - sigma = 0.0 - -We can create a random instance, where the random values are unit values drawn between 0.0 and 1.0. - -This means the parameter values of this instance are randomly drawn from the priors. - -.. code-block:: python - - model = af.Model(Gaussian) - instance = model.random_instance() - -Model Customization (Model) ---------------------------- - -We can fix a free parameter to a specific value (reducing the dimensionality of parameter space by 1): - -.. code-block:: python - - model = af.Model(Gaussian) - model.centre = 0.0 - -We can link two parameters together such they always assume the same value (reducing the dimensionality of -parameter space by 1): - -.. code-block:: python - - model.centre = model.normalization - -Offsets between linked parameters or with certain values are possible: - -.. code-block:: python - - model.centre = model.normalization + model.sigma - -Assertions remove regions of parameter space (but do not reduce the dimensionality of parameter space): - -.. code-block:: python - - model.add_assertion(model.sigma > 5.0) - model.add_assertion(model.centre > model.normalization) - -The customized model can be inspected by printing its `info` attribute. - -.. code-block:: python - - print(model.info) - -This gives the following output: - -.. code-block:: bash - - Total Free Parameters = 2 - - model Gaussian (N=2) - centre SumPrior (N=2) - - centre - self LogUniformPrior [14], lower_limit = 1e-06, upper_limit = 1000000.0 - other UniformPrior [15], lower_limit = 0.0, upper_limit = 25.0 - normalization LogUniformPrior [14], lower_limit = 1e-06, upper_limit = 1000000.0 - sigma UniformPrior [15], lower_limit = 0.0, upper_limit = 25.0 - -The overwriting of priors shown above can be achieved via the following alternative API: - -.. code-block:: python - - model = af.Model( - Gaussian, - centre=af.UniformPrior(lower_limit=0.0, upper_limit=1.0), - normalization=af.LogUniformPrior(lower_limit=1e-4, upper_limit=1e4), - sigma=af.GaussianPrior(mean=0.0, sigma=1.0), - ) - -This API can also be used for fixing a parameter to a certain value: - -.. code-block:: python - - model = af.Model(Gaussian, centre=0.0) - - -Tuple Parameters (Model) ------------------------- - -The `Gaussian` model component above only has parameters that are single-valued floats. - -Parameters can also be tuples, which is useful for defining model components where certain parameters are naturally -grouped together. - -For example, we can define a 2D Gaussian with a center that has two coordinates and therefore free parameters, (x, y), -using a tuple. - -.. code-block:: python - - class Gaussian2D: - def __init__( - self, - centre: Tuple[float, float] = (0.0, 0.0), # <- **PyAutoFit** recognises these constructor arguments - normalization: float = 0.1, # <- are the Gaussian``s model parameters. - sigma: float = 1.0, - ): - self.centre = centre - self.normalization = normalization - self.sigma = sigma - -The model's `total_free_parameters` attribute now includes 4 free parameters, as the tuple `centre` parameter accounts -for 2 free parameters. - -.. code-block:: python - - model = af.Model(Gaussian2D) - - print(f"Model Total Free Parameters = {model.total_free_parameters}") - -This information is again displayed in the `info` attribute: - -.. code-block:: python - - print(model.info) - -This gives the following output: - -.. code-block:: bash - - Total Free Parameters = 4 - - model Gaussian2D (N=4) - - centre - centre_0 UniformPrior [3], lower_limit = 0.0, upper_limit = 100.0 - centre_1 UniformPrior [4], lower_limit = 0.0, upper_limit = 100.0 - normalization LogUniformPrior [5], lower_limit = 1e-06, upper_limit = 1000000.0 - sigma UniformPrior [6], lower_limit = 0.0, upper_limit = 25.0 - -Here are examples of how model customization can be applied to a model with tuple parameters: - -.. code-block:: python - - model = af.Model(Gaussian2D) - model.centre = (0.0, 0.0) - - model.centre_0 = model.normalization - - model.centre_1 = model.normalization + model.sigma - - model.add_assertion(model.centre_0 > model.normalization) - -Json Outputs (Model) --------------------- - -A model has a ``dict`` attribute, which expresses all information about the model as a Python dictionary. - -By printing this dictionary we can therefore get a concise summary of the model. - -.. code-block:: python - - model = af.Model(Gaussian) - - print(model.dict()) - -This gives the following output: - -.. code-block:: bash - - { - 'class_path': '__main__.Gaussian', 'type': 'model', - 'centre': {'lower_limit': 0.0, 'upper_limit': 100.0, 'type': 'Uniform'}, - 'normalization': {'lower_limit': 1e-06, 'upper_limit': 1000000.0, 'type': 'LogUniform'}, - 'sigma': {'lower_limit': 0.0, 'upper_limit': 25.0, 'type': 'Uniform'} - } - -The dictionary representation printed above can be saved to hard disk as a ``.json`` file. - -This means we can save any **PyAutoFit** model to hard-disk in a human readable format. - -Checkout the file ``autofit_workspace/*/cookbooks/jsons/model.json`` to see the model written as a .json. - -.. code-block:: python - - model_path = path.join("scripts", "cookbooks", "jsons") - - os.makedirs(model_path, exist_ok=True) - - model_file = path.join(model_path, "model.json") - - with open(model_file, "w+") as f: - json.dump(model.dict(), f, indent=4) - -We can load the model from its ``.json`` file, meaning that one can easily save a model to hard disk and load it -elsewhere. - -.. code-block:: python - - model = af.Model.from_json(file=model_file) - -Model Composition (Collection) ------------------------------- - -To illustrate ``Collection`` objects we define a second model component, representing a ``Exponential`` profile. - -.. code-block:: python - - class Exponential: - def __init__( - self, - centre=0.0, # <- PyAutoFit recognises these constructor arguments are the model - normalization=0.1, # <- parameters of the Exponential. - rate=0.01, - ): - self.centre = centre - self.normalization = normalization - self.rate = rate - -To instantiate multiple Python classes into a combined model component we combine the ``af.Collection()`` and ``af.Model()`` -objects. - -By passing the key word arguments ``gaussian`` and ``exponential`` below, these are used as the names of the attributes of -instances created using this model (which is illustrated clearly below). - -.. code-block:: python - - model = af.Collection(gaussian=af.Model(Gaussian), exponential=af.Model(Exponential)) - -We can check the model has a ``total_free_parameters`` of 6, meaning the 3 parameters defined -above (``centre``, ``normalization``, ``sigma`` and ``rate``) for both the ``Gaussian`` and ``Exponential`` classes all have -priors associated with them . - -This also means each parameter is fitted for if we fitted the model to data via a non-linear search. - -.. code-block:: python - - print(f"Model Total Free Parameters = {model.total_free_parameters}") - -Printing the ``info`` attribute of the model gives us information on all of the parameters. - -.. code-block:: python - - print(model.info) - -This gives the following output: - -.. code-block:: bash - - Total Free Parameters = 6 - - model Collection (N=6) - gaussian Gaussian (N=3) - exponential Exponential (N=3) - - gaussian - centre UniformPrior [39], lower_limit = 0.0, upper_limit = 100.0 - normalization LogUniformPrior [40], lower_limit = 1e-06, upper_limit = 1000000.0 - sigma UniformPrior [41], lower_limit = 0.0, upper_limit = 25.0 - exponential - centre UniformPrior [42], lower_limit = 0.0, upper_limit = 100.0 - normalization LogUniformPrior [43], lower_limit = 1e-06, upper_limit = 1000000.0 - rate UniformPrior [44], lower_limit = 0.0, upper_limit = 1.0 - -Priors (Collection) -------------------- - -The model has a set of default priors, which have been loaded from a config file in the PyAutoFit workspace. - -The configs cookbook describes how to setup config files in order to produce custom priors, which means that you do not -need to manually specify priors in your Python code every time you compose a model. - -If you do not setup config files, all priors must be manually specified before you fit the model, as shown below. - -.. code-block:: python - - model.gaussian.centre = af.UniformPrior(lower_limit=0.0, upper_limit=100.0) - model.gaussian.normalization = af.UniformPrior(lower_limit=0.0, upper_limit=1e2) - model.gaussian.sigma = af.UniformPrior(lower_limit=0.0, upper_limit=30.0) - model.exponential.centre = af.UniformPrior(lower_limit=0.0, upper_limit=100.0) - model.exponential.normalization = af.UniformPrior(lower_limit=0.0, upper_limit=1e2) - model.exponential.rate = af.UniformPrior(lower_limit=0.0, upper_limit=10.0) - -When creating a model via a ``Collection``, there is no need to actually pass the python classes as an ``af.Model()`` -because **PyAutoFit** implicitly assumes they are to be created as a ``Model()``. - -This enables more concise code, whereby the following code: - -.. code-block:: python - - model = af.Collection(gaussian=af.Model(Gaussian), exponential=af.Model(Exponential)) - -Can instead be written as: - -.. code-block:: python - - model = af.Collection(gaussian=Gaussian, exponential=Exponential) - -Instances (Collection) ----------------------- - -We can create an instance of collection containing both the ``Gaussian`` and ``Exponential`` classes using this model. - -We create an ``instance`` where: - -- The ``Gaussian`` class has ``centre=30.0``, ``normalization=2.0`` and ``sigma=3.0``. -- The ``Exponential`` class has ``centre=60.0``, ``normalization=4.0`` and ``rate=1.0````. - -.. code-block:: python - - instance = model.instance_from_vector(vector=[30.0, 2.0, 3.0, 60.0, 4.0, 1.0]) - -Because we passed the key word arguments ``gaussian`` and ``exponential`` above, these are the names of the attributes of -instances created using this model (e.g. this is why we write ``instance.gaussian``): - -.. code-block:: python - - print("Model Instance: \n") - print(instance) - - print("Instance Parameters \n") - print("centre (Gaussian) = ", instance.gaussian.centre) - print("normalization (Gaussian) = ", instance.gaussian.normalization) - print("sigma (Gaussian) = ", instance.gaussian.sigma) - print("centre (Exponential) = ", instance.exponential.centre) - print("normalization (Exponential) = ", instance.exponential.normalization) - print("rate (Exponential) = ", instance.exponential.rate) - -This gives the following output: - -.. code-block:: bash - - Model Instance: - - - Instance Parameters - - centre (Gaussian) = 30.0 - normalization (Gaussian) = 2.0 - sigma (Gaussian) = 3.0 - centre (Exponential) = 60.0 - normalization (Exponential) = 4.0 - rate (Exponential) = 1.0 - -Alternatively, the instance's variables can also be accessed as a list, whereby instead of using attribute names -(e.g. ``gaussian_0``) we input the list index. - -Note that the order of the instance model components is determined from the order the components are input into the -``Collection``. - -For example, for the line ``af.Collection(gaussian=gaussian, exponential=exponential)``, the first entry in the list -is the gaussian because it is the first input to the ``Collection``. - -.. code-block:: python - - print("centre (Gaussian) = ", instance[0].centre) - print("normalization (Gaussian) = ", instance[0].normalization) - print("sigma (Gaussian) = ", instance[0].sigma) - print("centre (Gaussian) = ", instance[1].centre) - print("normalization (Gaussian) = ", instance[1].normalization) - print("rate (Exponential) = ", instance[1].rate) - -This gives the following output: - -.. code-block:: bash - - centre (Gaussian) = 30.0 - normalization (Gaussian) = 2.0 - sigma (Gaussian) = 3.0 - centre (Exponential) = 60.0 - normalization (Exponential) = 4.0 - rate (Exponential) = 1.0 - -Model Customization (Collection) --------------------------------- - -By setting up each Model first the model can be customized using either of the API’s shown above: - -.. code-block:: python - - gaussian = af.Model(Gaussian) - gaussian.normalization = 1.0 - gaussian.sigma = af.GaussianPrior(mean=0.0, sigma=1.0) - - exponential = af.Model(Exponential) - exponential.centre = 50.0 - exponential.add_assertion(exponential.rate > 5.0) - - model = af.Collection(gaussian=gaussian, exponential=exponential) - - print(model.info) - -This gives the following output: - -.. code-block:: bash - -Total Free Parameters = 4 - - model Collection (N=4) - gaussian Gaussian (N=2) - exponential Exponential (N=2) - - gaussian - centre UniformPrior [71], lower_limit = 0.0, upper_limit = 100.0 - normalization 1.0 - sigma GaussianPrior [70], mean = 0.0, sigma = 1.0 - exponential - centre 50.0 - normalization LogUniformPrior [72], lower_limit = 1e-06, upper_limit = 1000000.0 - rate UniformPrior [73], lower_limit = 0.0, upper_limit = 1.0 - -Below is an alternative API that can be used to create the same model as above. - -Which API is used is up to the user and which they find most intuitive. - -.. code-block:: python - - gaussian = af.Model( - Gaussian, normalization=1.0, sigma=af.GaussianPrior(mean=0.0, sigma=1.0) - ) - exponential = af.Model(Exponential, centre=50.0) - exponential.add_assertion(exponential.rate > 5.0) - - model = af.Collection(gaussian=gaussian, exponential=exponential) - - print(model.info) - -This gives the following output: - -.. code-block:: bash - - Total Free Parameters = 4 - - model Collection (N=4) - gaussian Gaussian (N=2) - exponential Exponential (N=2) - - gaussian - centre UniformPrior [63], lower_limit = 0.0, upper_limit = 100.0 - normalization 1.0 - sigma GaussianPrior [66], mean = 0.0, sigma = 1.0 - exponential - centre 50.0 - normalization LogUniformPrior [68], lower_limit = 1e-06, upper_limit = 1000000.0 - rate UniformPrior [69], lower_limit = 0.0, upper_limit = 1.0 - -After creating the model as a ``Collection`` we can customize it afterwards: - -.. code-block:: python - - model = af.Collection(gaussian=Gaussian, exponential=Exponential) - - model.gaussian.normalization = 1.0 - model.gaussian.sigma = af.GaussianPrior(mean=0.0, sigma=1.0) - - model.exponential.centre = 50.0 - model.exponential.add_assertion(exponential.rate > 5.0) - - print(model.info) - -This gives the following output: - -.. code-block:: bash - - Total Free Parameters = 4 - - model Collection (N=4) - gaussian Gaussian (N=2) - exponential Exponential (N=2) - - gaussian - centre UniformPrior [71], lower_limit = 0.0, upper_limit = 100.0 - normalization 1.0 - sigma GaussianPrior [70], mean = 0.0, sigma = 1.0 - exponential - centre 50.0 - normalization LogUniformPrior [72], lower_limit = 1e-06, upper_limit = 1000000.0 - rate UniformPrior [73], lower_limit = 0.0, upper_limit = 1.0 - -JSon Outputs (Collection) -------------------------- - -A ``Collection`` has a ``dict`` attribute, which express all information about the model as a Python dictionary. - -By printing this dictionary we can therefore get a concise summary of the model. - -.. code-block:: python - - model = af.Model(Gaussian) - - print(model.dict()) - -This gives the following output: - -.. code-block:: bash - - { - 'type': 'collection', - 'gaussian': { - 'class_path': '__main__.Gaussian', 'type': 'model', - 'centre': {'lower_limit': 0.0, 'upper_limit': 100.0, 'type': 'Uniform'}, - 'normalization': 1.0, 'sigma': {'lower_limit': -inf, 'upper_limit': inf, 'type': 'Gaussian', 'mean': 0.0, 'sigma': 1.0}}, - 'exponential': { - 'class_path': '__main__.Exponential', 'type': 'model', - 'centre': 50.0, - 'normalization': {'lower_limit': 1e-06, 'upper_limit': 1000000.0, 'type': 'LogUniform'}, - 'rate': {'lower_limit': 0.0, 'upper_limit': 1.0, 'type': 'Uniform'}} - } - -Python dictionaries can easily be saved to hard disk as a ``.json`` file. - -This means we can save any **PyAutoFit** model to hard-disk. - -Checkout the file ``autofit_workspace/*/model/jsons/collection.json`` to see the model written as a .json. - -.. code-block:: python - - model_path = path.join("scripts", "model", "jsons") - - os.makedirs(model_path, exist_ok=True) - - model_file = path.join(model_path, "collection.json") - - with open(model_file, "w+") as f: - json.dump(model.dict(), f, indent=4) - -We can load the model from its ``.json`` file, meaning that one can easily save a model to hard disk and load it -elsewhere. - -.. code-block:: python - - model = af.Model.from_json(file=model_file) - - print(f"\n Model via Json Prior Count = {model.prior_count}") - -Extensible Models (Collection) ------------------------------- - -There is no limit to the number of components we can use to set up a model via a ``Collection``. - -.. code-block:: python - - model = af.Collection( - gaussian_0=Gaussian, - gaussian_1=Gaussian, - exponential_0=Exponential, - exponential_1=Exponential, - exponential_2=Exponential, - ) - - print(model.info) - -This gives the following output: - -.. code-block:: bash - - Total Free Parameters = 15 - - model Collection (N=15) - gaussian_0 Gaussian (N=3) - gaussian_1 Gaussian (N=3) - exponential_0 Exponential (N=3) - exponential_1 Exponential (N=3) - exponential_2 Exponential (N=3) - - gaussian_0 - centre UniformPrior [91], lower_limit = 0.0, upper_limit = 100.0 - normalization LogUniformPrior [92], lower_limit = 1e-06, upper_limit = 1000000.0 - sigma UniformPrior [93], lower_limit = 0.0, upper_limit = 25.0 - gaussian_1 - centre UniformPrior [94], lower_limit = 0.0, upper_limit = 100.0 - normalization LogUniformPrior [95], lower_limit = 1e-06, upper_limit = 1000000.0 - sigma UniformPrior [96], lower_limit = 0.0, upper_limit = 25.0 - exponential_0 - centre UniformPrior [97], lower_limit = 0.0, upper_limit = 100.0 - normalization LogUniformPrior [98], lower_limit = 1e-06, upper_limit = 1000000.0 - rate UniformPrior [99], lower_limit = 0.0, upper_limit = 1.0 - exponential_1 - centre UniformPrior [100], lower_limit = 0.0, upper_limit = 100.0 - normalization LogUniformPrior [101], lower_limit = 1e-06, upper_limit = 1000000.0 - rate UniformPrior [102], lower_limit = 0.0, upper_limit = 1.0 - exponential_2 - centre UniformPrior [103], lower_limit = 0.0, upper_limit = 100.0 - normalization LogUniformPrior [104], lower_limit = 1e-06, upper_limit = 1000000.0 - rate UniformPrior [105], lower_limit = 0.0, upper_limit = 1.0 - Total Free Parameters = 6 - - model Collection (N=6) - gaussian_0 Gaussian (N=3) - gaussian_1 Gaussian (N=3) - - gaussian_0 - centre UniformPrior [106], lower_limit = 0.0, upper_limit = 100.0 - normalization LogUniformPrior [107], lower_limit = 1e-06, upper_limit = 1000000.0 - sigma UniformPrior [108], lower_limit = 0.0, upper_limit = 25.0 - gaussian_1 - centre UniformPrior [109], lower_limit = 0.0, upper_limit = 100.0 - normalization LogUniformPrior [110], lower_limit = 1e-06, upper_limit = 1000000.0 - sigma UniformPrior [111], lower_limit = 0.0, upper_limit = 25.0 - Total Free Parameters = 6 - - model Collection (N=6) - gaussian_0 Gaussian (N=3) - gaussian_1 Gaussian (N=3) - - gaussian_0 - centre UniformPrior [112], lower_limit = 0.0, upper_limit = 100.0 - normalization LogUniformPrior [113], lower_limit = 1e-06, upper_limit = 1000000.0 - sigma UniformPrior [114], lower_limit = 0.0, upper_limit = 25.0 - gaussian_1 - centre UniformPrior [115], lower_limit = 0.0, upper_limit = 100.0 - normalization LogUniformPrior [116], lower_limit = 1e-06, upper_limit = 1000000.0 - sigma UniformPrior [117], lower_limit = 0.0, upper_limit = 25.0 - -A model can be created via ``af.Collection()`` where a dictionary of ``af.Model()`` objects are passed to it. - -The two models created below are identical- one uses the API detailed above whereas the second uses a dictionary. - -.. code-block:: python - - model = af.Collection(gaussian_0=Gaussian, gaussian_1=Gaussian) - - model_dict = {"gaussian_0": Gaussian, "gaussian_1": Gaussian} - model = af.Collection(**model_dict) - - -The keys of the dictionary passed to the model (e.g. ``gaussian_0`` and ``gaussian_1`` above) are used to create the -names of the attributes of instances of the model. - -.. code-block:: python - - instance = model.instance_from_vector(vector=[1.0, 2.0, 3.0, 4.0, 5.0, 6.0]) - - print("Model Instance: \n") - print(instance) - - print("Instance Parameters \n") - print("centre (Gaussian) = ", instance.gaussian_0.centre) - print("normalization (Gaussian) = ", instance.gaussian_0.normalization) - print("sigma (Gaussian) = ", instance.gaussian_0.sigma) - print("centre (Gaussian) = ", instance.gaussian_1.centre) - print("normalization (Gaussian) = ", instance.gaussian_1.normalization) - print("sigma (Gaussian) = ", instance.gaussian_1.sigma) - - -This gives the following output: - -.. code-block:: bash - - Model Instance: - - - Instance Parameters: - - centre (Gaussian) = 1.0 - normalization (Gaussian) = 2.0 - sigma (Gaussian) = 3.0 - centre (Gaussian) = 4.0 - normalization (Gaussian) = 5.0 - sigma (Gaussian) = 6.0 - -A list of model components can also be passed to an ``af.Collection`` to create a model: - -.. code-block:: python - - model = af.Collection([Gaussian, Gaussian]) - - print(model.info) - -When a list is used, there is no string with which to name the model components (e.g. we do not input ``gaussian_0`` -and ``gaussian_1`` anywhere. - -The ``instance`` therefore can only be accessed via list indexing. - -.. code-block:: python - - instance = model.instance_from_vector(vector=[1.0, 2.0, 3.0, 4.0, 5.0, 6.0]) - - print("Model Instance: \n") - print(instance) - - print("Instance Parameters \n") - print("centre (Gaussian) = ", instance[0].centre) - print("normalization (Gaussian) = ", instance[0].normalization) - print("sigma (Gaussian) = ", instance[0].sigma) - print("centre (Gaussian) = ", instance[1].centre) - print("normalization (Gaussian) = ", instance[1].normalization) - print("sigma (Gaussian) = ", instance[1].sigma) - -This gives the following output: - -.. code-block:: bash - - Model Instance: - - - Instance Parameters: - - centre (Gaussian) = 1.0 - normalization (Gaussian) = 2.0 - sigma (Gaussian) = 3.0 - centre (Gaussian) = 4.0 - normalization (Gaussian) = 5.0 - sigma (Gaussian) = 6.0 - -Model Composition (af.Array) ----------------------------- - -Models can be composed using NumPy arrays, where each element of the array is a free parameter. - -This offers a lot more flexibility than using ``Model`` and ``Collection`` objects, as the number of parameters in the -model is chosen on initialization via the input of the ``shape`` attribute. - -For many use cases, this flexibility is key to ensuring model composition is as easy as possible, for example when -a part of the model being fitted is a matrix of parameters which may change shape depending on the dataset being -fitted. - -To compose models using NumPy arrays, we use the ``af.Array`` object. - -.. code-block:: python - - model = af.Array( - shape=(2, 2), - prior=af.GaussianPrior(mean=0.0, sigma=1.0), - ) - -Each element of the array is a free parameter, which for ``shape=(2,2)`` means the model has 4 free parameters. - -.. code-block:: python - - print(f"Model Total Free Parameters = {model.total_free_parameters}") - -The ``info`` attribute of the model gives information on all of the parameters and their priors. - -.. code-block:: python - - print(model.info) - -This gives the following output: - -.. code-block:: bash - - Total Free Parameters = 4 - - model Array (N=4) - indices list (N=0) - - shape (2, 2) - indices - 0 (0, 0) - 1 (0, 1) - 2 (1, 0) - 3 (1, 1) - prior_0_0 GaussianPrior [124], mean = 0.0, sigma = 1.0 - prior_0_1 GaussianPrior [125], mean = 0.0, sigma = 1.0 - prior_1_0 GaussianPrior [126], mean = 0.0, sigma = 1.0 - prior_1_1 GaussianPrior [127], mean = 0.0, sigma = 1.0 - -Prior Customization (af.Array) ------------------------------- - -The prior of every parameter in the array is set via the ``prior`` input above. - -NumPy array models do not currently support default priors via config files, so all priors must be manually specified. - -The prior of every parameter in the array can be customized by normal NumPy array indexing: - -.. code-block:: python - - model = af.Array(shape=(2, 2), prior=af.GaussianPrior(mean=0.0, sigma=1.0)) - - model.array[0, 0] = af.UniformPrior(lower_limit=0.0, upper_limit=1.0) - model.array[0, 1] = af.LogUniformPrior(lower_limit=1e-4, upper_limit=1e4) - model.array[1, 0] = af.GaussianPrior(mean=0.0, sigma=2.0) - -The ``info`` attribute shows the customized priors. - -.. code-block:: python - - print(model.info) - -The output is as follows: - -.. code-block:: bash - - Total Free Parameters = 4 - - model Array (N=4) - indices list (N=0) - - shape (2, 2) - indices - 0 (0, 0) - 1 (0, 1) - 2 (1, 0) - 3 (1, 1) - prior_0_0 UniformPrior [133], lower_limit = 0.0, upper_limit = 1.0 - prior_0_1 LogUniformPrior [134], lower_limit = 0.0001, upper_limit = 10000.0 - prior_1_0 GaussianPrior [135], mean = 0.0, sigma = 2.0 - prior_1_1 GaussianPrior [132], mean = 0.0, sigma = 1.0 - -Instances (af.Array) --------------------- - -Instances of numpy array model components can be created, where an input ``vector`` of parameters is mapped to create -an instance of the Python class of the model. - -If the priors of the numpy array are not customized, ordering of parameters goes from element [0,0] to [0,1] to [1,0], -as shown by the ``paths`` attribute. - -.. code-block:: python - - model = af.Array( - shape=(2, 2), - prior=af.GaussianPrior(mean=0.0, sigma=1.0), - ) - - print(model.paths) - -The output is as follows: - -.. code-block:: bash - - ['prior_0_0', 'prior_0_1', 'prior_1_0', 'prior_1_1'] - -An instance can then be created by passing a vector of parameters to the model via the ``instance_from_vector`` method. - -The ``instance`` created is a NumPy array, where each element is the value passed in the vector. - -.. code-block:: python - - instance = model.instance_from_vector(vector=[0.0, 1.0, 2.0, 3.0]) - - print("\nModel Instance:") - print(instance) - -The output is as follows: - -.. code-block:: bash - - Model Instance: - [[0. 1.] - [2. 3.]] - -Prior customization changes the order of the parameters, therefore if you customize the priors of the numpy -array you must check the ordering of the parameters in the ``paths`` attribute before passing a vector to -the ``instance_from_vector`` - - -.. code-block:: python - - model[0, 0] = af.UniformPrior(lower_limit=0.0, upper_limit=1.0) - model[0, 1] = af.LogUniformPrior(lower_limit=1e-4, upper_limit=1e4) - model[1, 0] = af.GaussianPrior(mean=0.0, sigma=2.0) - - print(model.paths) - -The output is as follows: - -.. code-block:: bash - - [('prior_1_1',), ('prior_0_0',), ('prior_0_1',), ('prior_1_0',)] - -If we create a vector and print its values from this customized model: - -.. code-block:: python - - instance = model.instance_from_vector(vector=[0.0, 1.0, 2.0, 3.0]) - - print("\nModel Instance:") - print(instance) - -The output is as follows: - -.. code-block:: bash - - Model Instance: - [[1. 2.] - [3. 0.]] - -Model Customization (af.Array) ------------------------------- - -The model customization API for numpy array models is the same as for ``af.Model`` and ``af.Collection`` objects. - -.. code-block:: python - - model = af.Array( - shape=(2, 2), - prior=af.GaussianPrior(mean=0.0, sigma=1.0), - ) - - model[0,0] = 50.0 - model[0,1] = model[1,0] - model.add_assertion(model[1,1] > 0.0) - - print(model.info) - -The output is as follows: - -.. code-block:: bash - Total Free Parameters = 2 - - model Array (N=2) - indices list (N=0) - - shape (2, 2) - indices - 0 (0, 0) - 1 (0, 1) - 2 (1, 0) - 3 (1, 1) - prior_0_0 50.0 - prior_0_1 - prior_1_0 GaussianPrior [147], mean = 0.0, sigma = 1.0 - prior_1_1 GaussianPrior [148], mean = 0.0, sigma = 1.0 - - -JSon Outputs (af.Array) ------------------------- - -An ``Array`` has a ``dict`` attribute, which express all information about the model as a Python dictionary. - -By printing this dictionary we can therefore get a concise summary of the model. - -.. code-block:: python - - model = af.Array( - shape=(2, 2), - prior=af.GaussianPrior(mean=0.0, sigma=1.0), - ) - - print(model.dict()) - -Python dictionaries can easily be saved to hard disk as a ``.json`` file. - -This means we can save any **PyAutoFit** model to hard-disk. - -Checkout the file ``autofit_workspace/*/model/jsons/array.json`` to see the model written as a .json. - -.. code-block:: python - - model_path = path.join("scripts", "model", "jsons") - - os.makedirs(model_path, exist_ok=True) - - model_file = path.join(model_path, "array.json") - - with open(model_file, "w+") as f: - json.dump(model.dict(), f, indent=4) - -We can load the model from its ``.json`` file, meaning that one can easily save a model to hard disk and load it -elsewhere. - -.. code-block:: python - - model = af.Array.from_json(file=model_file) - - print(f"\n Model via Json Prior Count = {model.prior_count}") - -Extensible Models (af.Array) ----------------------------- - -For ``Model`` objects, the number of parameters is fixed to those listed in the input Python class when the model is -created. - -For ``Collection`` objects, the use of dictionaries and lists allows for the number of parameters to be extended, but it -was still tied to the input Python classes when the model was created. - -For ``Array`` objects, the number of parameters is fully customizable, you choose the shape of the array and therefore -the number of parameters in the model when you create it. - -This makes ``Array`` objects the most extensible and flexible way to compose models. - -You can also combine ``Array`` objects with ``Collection`` objects to create models with a mix of fixed and extensible -parameters. - -.. code-block:: python - - model = af.Collection( - gaussian=Gaussian, - array=af.Array(shape=(3, 2), prior=af.GaussianPrior(mean=0.0, sigma=1.0)) - ) - - model.gaussian.sigma = 2.0 - model.array[0, 0] = 1.0 - - print(model.info) - -The output is as follows: - -.. code-block:: python - - Total Free Parameters = 7 - - model Collection (N=7) - gaussian Gaussian (N=2) - array Array (N=5) - indices list (N=0) - - gaussian - centre UniformPrior [165], lower_limit = 0.0, upper_limit = 100.0 - normalization LogUniformPrior [166], lower_limit = 1e-06, upper_limit = 1000000.0 - sigma 2.0 - array - shape (3, 2) - indices - 0 (0, 0) - 1 (0, 1) - 2 (1, 0) - 3 (1, 1) - 4 (2, 0) - 5 (2, 1) - prior_0_0 1.0 - prior_0_1 GaussianPrior [160], mean = 0.0, sigma = 1.0 - prior_1_0 GaussianPrior [161], mean = 0.0, sigma = 1.0 - prior_1_1 GaussianPrior [162], mean = 0.0, sigma = 1.0 - prior_2_0 GaussianPrior [163], mean = 0.0, sigma = 1.0 - prior_2_1 GaussianPrior [164], mean = 0.0, sigma = 1.0 - - -Wrap Up -------- - -This cookbook shows how to compose models consisting of multiple components using the ``af.Model()`` -and ``af.Collection()`` object. - -Advanced model composition uses multi-level models, which compose models from hierarchies of Python classes. This is -described in the multi-level model cookbook. - diff --git a/docs/cookbooks/multi_level_model.md b/docs/cookbooks/multi_level_model.md new file mode 100644 index 000000000..81e1c5472 --- /dev/null +++ b/docs/cookbooks/multi_level_model.md @@ -0,0 +1,497 @@ +(multi-level-model)= + +# Multi Level Model + +A multi level model is one where one or more of the input parameters in the model components `__init__` +constructor are Python classes, as opposed to a float or tuple. + +The `af.Model()` object treats these Python classes as model components, enabling the composition of models where +model components are grouped within other Python classes, in an object oriented fashion. + +This enables complex models which are intiutive and extensible to be composed. + +This cookbook provides an overview of multi-level model composition. + +**Contents:** + +- **Python Class Template**: The template of multi level model components written as a Python class. +- **Model Composition**: How to compose a multi-level model using the `af.Model()` object. +- **Instances**: Creating an instance of a multi-level model via input parameters. +- **Why Use Multi-Level Models?**: A description of the benefits of using multi-level models compared to a `Collection`. +- **Model Customization**: Customizing a multi-level model (e.g. fixing parameters or linking them to one another). +- **Alternative API**: Alternative API for multi-level models which may be more concise and readable for certain models. +- **Json Output (Model)**: Output a multi-level model in human readable text via a .json file and loading it back again. + +## Python Class Template + +A multi-level model uses standard model components, which are written as a Python class with the usual format +where the inputs of the `__init__` constructor are the model parameters. + +```python +class Gaussian: + def __init__( + self, + normalization=1.0, # <- **PyAutoFit** recognises these constructor arguments + sigma=5.0, # <- are the Gaussian``s model parameters. + ): + self.normalization = normalization + self.sigma = sigma +``` + +The unique aspect of a multi-level model is that a Python class can then be defined where the inputs +of its `__init__` constructor are instances of these model components. + +In the example below, the Python class which will be used to demonstrate a multi-level has an input `gaussian_list`, +which takes as input a list of instances of the `Gaussian` class above. + +This class will represent many individual `Gaussian`'s, which share the same `centre` but have their own unique +`normalization` and `sigma` values. + +```python +class MultiLevelGaussians: + def __init__( + self, + higher_level_centre: float = 50.0, # The centre of all Gaussians in the multi level component. + gaussian_list: List[Gaussian] = None, # Contains a list of Gaussians + ): + self.higher_level_centre = higher_level_centre + + self.gaussian_list = gaussian_list +``` + +## Model Composition + +A multi-level model is instantiated via the af.Model() command, which is passed: + +- `MultiLevelGaussians`: To tell it that the model component will be a `MultiLevelGaussians` object. +- `gaussian_list`: One or more `Gaussian`'s, each of which are created as an `af.Model()` object with free parameters. + +```python +model = af.Model( + MultiLevelGaussians, gaussian_list=[af.Model(Gaussian), af.Model(Gaussian)] +) +``` + +The multi-level model consists of two `Gaussian`'s, where their centres are shared as a parameter in the higher level +model component. + +Total number of parameters is N=5 (x2 `normalizations`, ``` x2 ``sigma ```'s and x1 `higher_level_centre`). + +```python +print(f"Model Total Free Parameters = {model.total_free_parameters}") +``` + +The structure of the multi-level model, including the hierarchy of Python classes, is shown in the `model.info`. + +```python +print(model.info) +``` + +This gives the following output: + +```bash +Total Free Parameters = 5 + +model MultiLevelGaussians (N=5) + gaussian_list Collection (N=4) + 0 Gaussian (N=2) + 1 Gaussian (N=2) + +higher_level_centre UniformPrior [5], lower_limit = 0.0, upper_limit = 100.0 +gaussian_list + 0 + normalization LogUniformPrior [1], lower_limit = 1e-06, upper_limit = 1000000.0 + sigma UniformPrior [2], lower_limit = 0.0, upper_limit = 25.0 + 1 + normalization LogUniformPrior [3], lower_limit = 1e-06, upper_limit = 1000000.0 + sigma UniformPrior [4], lower_limit = 0.0, upper_limit = 25.0 +``` + +## Instances + +Instances of a multi-level model can be created, where an input `vector` of parameters is mapped to create an instance +of the Python class of the model. + +We first need to know the order of parameters in the model, so we know how to define the input `vector`. This +information is contained in the models `paths` attribute. + +```python +print(model.paths) +``` + +This gives the following output: + +```bash +[ + ('gaussian_list', '0', 'normalization'), + ('gaussian_list', '0', 'sigma'), + ('gaussian_list', '1', 'normalization'), + ('gaussian_list', '1', 'sigma'), + ('higher_level_centre',) +] +``` + +We now create an instance via a multi-level model. + +Its attributes are structured differently to models composed via the `Collection` object.. + +```python +instance = model.instance_from_vector(vector=[1.0, 2.0, 3.0, 4.0, 5.0]) + +print("Model Instance: \n") +print(instance) + +print("Instance Parameters \n") +print("Normalization (Gaussian 0) = ", instance.gaussian_list[0].normalization) +print("Sigma (Gaussian 0) = ", instance.gaussian_list[0].sigma) +print("Normalization (Gaussian 0) = ", instance.gaussian_list[1].normalization) +print("Sigma (Gaussian 0) = ", instance.gaussian_list[1].sigma) +print("Higher Level Centre= ", instance.higher_level_centre) +``` + +This gives the following output: + +```bash +Model Instance: + +<__main__.MultiLevelGaussians object at 0x7f5273ccd0f0> +Instance Parameters + +Normalization (Gaussian 0) = 1.0 +Sigma (Gaussian 0) = 2.0 +Normalization (Gaussian 0) = 3.0 +Sigma (Gaussian 0) = 4.0 +Higher Level Centre= 5.0 +``` + +## Why Use Multi Level Models? + +An identical model in terms of functionality could of been created via the `Collection` object as follows: + +```python +class GaussianCentre: + def __init__( + self, + centre=30.0, # <- **PyAutoFit** recognises these constructor arguments + normalization=1.0, # <- are the Gaussian``s model parameters. + sigma=5.0, + ): + self.centre = centre + self.normalization = normalization + self.sigma = sigma + + +model = af.Collection(gaussian_0=GaussianCentre, gaussian_1=GaussianCentre) + +model.gaussian_0.centre = model.gaussian_1.centre +``` + +This raises the question of when to use a `Collection` and when to use multi-level models? + +The answer depends on the structure of the models you are composing and fitting. + +Many problems have models which have a natural multi-level structure. + +For example, imagine a dataset had 3 separate groups of 1D `Gaussian`'s, where each group had multiple Gaussians with +a shared centre. + +This model is concise and easy to define using the multi-level API: + +```python +group_0 = af.Model(MultiLevelGaussians, gaussian_list=3 * [Gaussian]) + +group_1 = af.Model(MultiLevelGaussians, gaussian_list=3 * [Gaussian]) + +group_2 = af.Model(MultiLevelGaussians, gaussian_list=3 * [Gaussian]) + +model = af.Collection(group_0=group_0, group_1=group_1, group_2=group_2) +``` + +Composing the same model without the multi-level model is less concise, less readable and prone to error: + +```python +group_0 = af.Collection( + gaussian_0=GaussianCentre, gaussian_1=GaussianCentre, gaussian_2=GaussianCentre +) + +group_0.gaussian_0.centre = group_0.gaussian_1.centre +group_0.gaussian_0.centre = group_0.gaussian_2.centre +group_0.gaussian_1.centre = group_0.gaussian_2.centre + +group_1 = af.Collection( + gaussian_0=GaussianCentre, gaussian_1=GaussianCentre, gaussian_2=GaussianCentre +) + +group_1.gaussian_0.centre = group_1.gaussian_1.centre +group_1.gaussian_0.centre = group_1.gaussian_2.centre +group_1.gaussian_1.centre = group_1.gaussian_2.centre + +group_2 = af.Collection( + gaussian_0=GaussianCentre, gaussian_1=GaussianCentre, gaussian_2=GaussianCentre +) + +group_2.gaussian_0.centre = group_2.gaussian_1.centre +group_2.gaussian_0.centre = group_2.gaussian_2.centre +group_2.gaussian_1.centre = group_2.gaussian_2.centre + +model = af.Collection(group_0=group_0, group_1=group_1, group_2=group_2) +``` + +Here is what the `model.info` looks like: + +```bash +Total Free Parameters = 21 + +model Collection (N=21) + group_0 MultiLevelGaussians (N=7) + gaussian_list Collection (N=6) + 0 Gaussian (N=2) + 1 Gaussian (N=2) + 2 Gaussian (N=2) + group_1 MultiLevelGaussians (N=7) + gaussian_list Collection (N=6) + 0 Gaussian (N=2) + 1 Gaussian (N=2) + 2 Gaussian (N=2) + group_2 MultiLevelGaussians (N=7) + gaussian_list Collection (N=6) + 0 Gaussian (N=2) + 1 Gaussian (N=2) + 2 Gaussian (N=2) + +group_0 + higher_level_centre UniformPrior [6], lower_limit = 0.0, upper_limit = 100.0 + gaussian_list + 0 + normalization LogUniformPrior [7], lower_limit = 1e-06, upper_limit = 1000000.0 + sigma UniformPrior [8], lower_limit = 0.0, upper_limit = 25.0 + 1 + normalization LogUniformPrior [9], lower_limit = 1e-06, upper_limit = 1000000.0 + sigma UniformPrior [10], lower_limit = 0.0, upper_limit = 25.0 + 2 + normalization LogUniformPrior [11], lower_limit = 1e-06, upper_limit = 1000000.0 + sigma UniformPrior [12], lower_limit = 0.0, upper_limit = 25.0 +group_1 + higher_level_centre UniformPrior [13], lower_limit = 0.0, upper_limit = 100.0 + gaussian_list + 0 + normalization LogUniformPrior [14], lower_limit = 1e-06, upper_limit = 1000000.0 + sigma UniformPrior [15], lower_limit = 0.0, upper_limit = 25.0 + 1 + normalization LogUniformPrior [16], lower_limit = 1e-06, upper_limit = 1000000.0 + sigma UniformPrior [17], lower_limit = 0.0, upper_limit = 25.0 + 2 + normalization LogUniformPrior [18], lower_limit = 1e-06, upper_limit = 1000000.0 + sigma UniformPrior [19], lower_limit = 0.0, upper_limit = 25.0 +group_2 + higher_level_centre UniformPrior [20], lower_limit = 0.0, upper_limit = 100.0 + gaussian_list + 0 + normalization LogUniformPrior [21], lower_limit = 1e-06, upper_limit = 1000000.0 + sigma UniformPrior [22], lower_limit = 0.0, upper_limit = 25.0 + 1 + normalization LogUniformPrior [23], lower_limit = 1e-06, upper_limit = 1000000.0 + sigma UniformPrior [24], lower_limit = 0.0, upper_limit = 25.0 + 2 + normalization LogUniformPrior [25], lower_limit = 1e-06, upper_limit = 1000000.0 + sigma UniformPrior [26], lower_limit = 0.0, upper_limit = 25.0 +``` + +In many situations, multi-levels models are more extensible than the `Collection` API. + +For example, imagine we wanted to add even more 1D profiles to a group with a shared `centre`. This can easily be +achieved using the multi-level API: + +```python +multi = af.Model( + MultiLevelGaussians, + gaussian_list=[Gaussian, Gaussian, Exponential, YourProfileHere] +) +``` + +Composing the same model using just a `Model` and `Collection` is again possible, but would be even more cumbersome, +less readable and is not extensible. + +## Model Customization + +To customize the higher level parameters of a multi-level the usual model API is used: + +```python +multi = af.Model(MultiLevelGaussians, gaussian_list=[Gaussian, Gaussian]) + +multi.higher_level_centre = af.UniformPrior(lower_limit=0.0, upper_limit=100.0) +``` + +To customize a multi-level model instantiated via lists, each model component is accessed via its index: + +```python +multi = af.Model(MultiLevelGaussians, gaussian_list=[Gaussian, Gaussian]) + +group_level = af.Model(MultiLevelGaussians, gaussian_list=[Gaussian, Gaussian]) + +group_level.gaussian_list[0].normalization = group_level.gaussian_list[1].normalization +``` + +Any combination of the API’s shown above can be used for customizing this model: + +```python +gaussian_0 = af.Model(Gaussian) +gaussian_1 = af.Model(Gaussian) + +gaussian_0.normalization = gaussian_1.normalization + +group_level = af.Model( + MultiLevelGaussians, gaussian_list=[gaussian_0, gaussian_1, af.Model(Gaussian)] +) + +group_level.higher_level_centre = 1.0 +group_level.gaussian_list[2].normalization = group_level.gaussian_list[1].normalization +``` + +Here is what the `model.info` looks like: + +```bash +Total Free Parameters = 4 + +model MultiLevelGaussians (N=4) + gaussian_list Collection (N=4) + 0 Gaussian (N=2) + 1 Gaussian (N=2) + 2 Gaussian (N=2) + +higher_level_centre 1.0 +gaussian_list + 0 + normalization LogUniformPrior [45], lower_limit = 1e-06, upper_limit = 1000000.0 + sigma UniformPrior [44], lower_limit = 0.0, upper_limit = 25.0 + 1 + normalization LogUniformPrior [45], lower_limit = 1e-06, upper_limit = 1000000.0 + sigma UniformPrior [46], lower_limit = 0.0, upper_limit = 25.0 + 2 + normalization LogUniformPrior [45], lower_limit = 1e-06, upper_limit = 1000000.0 + sigma UniformPrior [48], lower_limit = 0.0, upper_limit = 25.0 +``` + +## Alternative API + +A multi-level model can be instantiated where each model sub-component is setup using a name (as opposed to a list). + +This means no list input parameter is required in the Python class of the model component, but we do need to include +the `**kwargs` input. + +```python +class MultiLevelGaussians: + def __init__(self, higher_level_centre=1.0, **kwargs): + self.higher_level_centre = higher_level_centre + + +model = af.Model( + MultiLevelGaussians, gaussian_0=af.Model(Gaussian), gaussian_1=af.Model(Gaussian) +) + +instance = model.instance_from_vector(vector=[1.0, 2.0, 3.0, 4.0, 5.0]) + +print("Instance Parameters \n") +print("Normalization (Gaussian 0) = ", instance.gaussian_0.normalization) +print("Sigma (Gaussian 0) = ", instance.gaussian_0.sigma) +print("Normalization (Gaussian 0) = ", instance.gaussian_1.normalization) +print("Sigma (Gaussian 0) = ", instance.gaussian_1.sigma) +print("Higher Level Centre= ", instance.higher_level_centre) +``` + +This gives the following output: + +```bash +Instance Parameters + +Normalization (Gaussian 0) = 1.0 +Sigma (Gaussian 0) = 2.0 +Normalization (Gaussian 0) = 3.0 +Sigma (Gaussian 0) = 4.0 +Higher Level Centre= 5.0 +``` + +The use of Python dictionaries illustrated in previous cookbooks can also be used with multi-level models. + +```python +model_dict = {"gaussian_0": Gaussian, "gaussian_1": Gaussian} + +model = af.Model(MultiLevelGaussians, **model_dict) + +print(f"Multi-level Model Prior Count = {model.prior_count}") + +instance = model.instance_from_vector(vector=[1.0, 2.0, 3.0, 4.0, 5.0]) + +print("Instance Parameters \n") +print("Normalization (Gaussian 0) = ", instance.gaussian_0.normalization) +print("Sigma (Gaussian 0) = ", instance.gaussian_0.sigma) +print("Normalization (Gaussian 0) = ", instance.gaussian_1.normalization) +print("Sigma (Gaussian 0) = ", instance.gaussian_1.sigma) +print("Higher Level Centre= ", instance.higher_level_centre) +``` + +This gives the following output: + +```bash +Instance Parameters + +Normalization (Gaussian 0) = 1.0 +Sigma (Gaussian 0) = 2.0 +Normalization (Gaussian 0) = 3.0 +Sigma (Gaussian 0) = 4.0 +Higher Level Centre= 5.0 +``` + +## JSon Outputs + +A model has a `dict` attribute, which expresses all information about the model as a Python dictionary. + +By printing this dictionary we can therefore get a concise summary of the model. + +```python +model = af.Model(Gaussian) + +print(model.dict()) +``` + +This gives the following output: + +```bash +{ +'class_path': '__main__.Gaussian', 'type': 'model', +'normalization': {'lower_limit': 1e-06, 'upper_limit': 1000000.0, 'type': 'LogUniform'}, +'sigma': {'lower_limit': 0.0, 'upper_limit': 25.0, 'type': 'Uniform'} +} +``` + +The dictionary representation printed above can be saved to hard disk as a `.json` file. + +This means we can save any multi-level model to hard-disk in a human readable format. + +Checkout the file `autofit_workspace/*/cookbooks/jsons/group_level_model.json` to see the model written as a .json. + +```python +model_path = path.join("scripts", "cookbooks", "jsons") + +os.makedirs(model_path, exist_ok=True) + +model_file = path.join(model_path, "multi_level_model.json") + +with open(model_file, "w+") as f: + json.dump(model.dict(), f, indent=4) +``` + +We can load the model from its `.json` file, meaning that one can easily save a model to hard disk and load it +elsewhere. + +```python +model = af.Model.from_json(file=model_file) +``` + +## Wrap Up + +This cookbook shows how to multi-level models consisting of multiple components using the `af.Model()` +and `af.Collection()` objects. + +You should think carefully about whether your model fitting problem can use multi-level models, as they can make +your model definition more concise and extensible. diff --git a/docs/cookbooks/multi_level_model.rst b/docs/cookbooks/multi_level_model.rst deleted file mode 100644 index 0c7b5e2b5..000000000 --- a/docs/cookbooks/multi_level_model.rst +++ /dev/null @@ -1,513 +0,0 @@ -.. _multi_level_model: - -Multi Level Model -================= - -A multi level model is one where one or more of the input parameters in the model components ``__init__`` -constructor are Python classes, as opposed to a float or tuple. - -The ``af.Model()`` object treats these Python classes as model components, enabling the composition of models where -model components are grouped within other Python classes, in an object oriented fashion. - -This enables complex models which are intiutive and extensible to be composed. - -This cookbook provides an overview of multi-level model composition. - -**Contents:** - -- **Python Class Template**: The template of multi level model components written as a Python class. -- **Model Composition**: How to compose a multi-level model using the ``af.Model()`` object. -- **Instances**: Creating an instance of a multi-level model via input parameters. -- **Why Use Multi-Level Models?**: A description of the benefits of using multi-level models compared to a ``Collection``. -- **Model Customization**: Customizing a multi-level model (e.g. fixing parameters or linking them to one another). -- **Alternative API**: Alternative API for multi-level models which may be more concise and readable for certain models. -- **Json Output (Model)**: Output a multi-level model in human readable text via a .json file and loading it back again. - -Python Class Template ---------------------- - -A multi-level model uses standard model components, which are written as a Python class with the usual format -where the inputs of the ``__init__`` constructor are the model parameters. - -.. code-block:: python - - class Gaussian: - def __init__( - self, - normalization=1.0, # <- **PyAutoFit** recognises these constructor arguments - sigma=5.0, # <- are the Gaussian``s model parameters. - ): - self.normalization = normalization - self.sigma = sigma - - -The unique aspect of a multi-level model is that a Python class can then be defined where the inputs -of its ``__init__`` constructor are instances of these model components. - -In the example below, the Python class which will be used to demonstrate a multi-level has an input ``gaussian_list``, -which takes as input a list of instances of the ``Gaussian`` class above. - -This class will represent many individual ``Gaussian``'s, which share the same ``centre`` but have their own unique -``normalization`` and ``sigma`` values. - -.. code-block:: python - - class MultiLevelGaussians: - def __init__( - self, - higher_level_centre: float = 50.0, # The centre of all Gaussians in the multi level component. - gaussian_list: List[Gaussian] = None, # Contains a list of Gaussians - ): - self.higher_level_centre = higher_level_centre - - self.gaussian_list = gaussian_list - -Model Composition ------------------ - -A multi-level model is instantiated via the af.Model() command, which is passed: - -- ``MultiLevelGaussians``: To tell it that the model component will be a ``MultiLevelGaussians`` object. -- ``gaussian_list``: One or more ``Gaussian``'s, each of which are created as an ``af.Model()`` object with free parameters. - -.. code-block:: python - - model = af.Model( - MultiLevelGaussians, gaussian_list=[af.Model(Gaussian), af.Model(Gaussian)] - ) - - -The multi-level model consists of two ``Gaussian``'s, where their centres are shared as a parameter in the higher level -model component. - -Total number of parameters is N=5 (x2 ``normalizations``, ``x2 ``sigma``'s and x1 ``higher_level_centre``). - -.. code-block:: python - - print(f"Model Total Free Parameters = {model.total_free_parameters}") - -The structure of the multi-level model, including the hierarchy of Python classes, is shown in the ``model.info``. - -.. code-block:: python - - print(model.info) - -This gives the following output: - -.. code-block:: bash - - Total Free Parameters = 5 - - model MultiLevelGaussians (N=5) - gaussian_list Collection (N=4) - 0 Gaussian (N=2) - 1 Gaussian (N=2) - - higher_level_centre UniformPrior [5], lower_limit = 0.0, upper_limit = 100.0 - gaussian_list - 0 - normalization LogUniformPrior [1], lower_limit = 1e-06, upper_limit = 1000000.0 - sigma UniformPrior [2], lower_limit = 0.0, upper_limit = 25.0 - 1 - normalization LogUniformPrior [3], lower_limit = 1e-06, upper_limit = 1000000.0 - sigma UniformPrior [4], lower_limit = 0.0, upper_limit = 25.0 - -Instances ---------- - -Instances of a multi-level model can be created, where an input ``vector`` of parameters is mapped to create an instance -of the Python class of the model. - -We first need to know the order of parameters in the model, so we know how to define the input ``vector``. This -information is contained in the models ``paths`` attribute. - -.. code-block:: python - - print(model.paths) - -This gives the following output: - -.. code-block:: bash - - [ - ('gaussian_list', '0', 'normalization'), - ('gaussian_list', '0', 'sigma'), - ('gaussian_list', '1', 'normalization'), - ('gaussian_list', '1', 'sigma'), - ('higher_level_centre',) - ] - -We now create an instance via a multi-level model. - -Its attributes are structured differently to models composed via the ``Collection`` object.. - -.. code-block:: python - - instance = model.instance_from_vector(vector=[1.0, 2.0, 3.0, 4.0, 5.0]) - - print("Model Instance: \n") - print(instance) - - print("Instance Parameters \n") - print("Normalization (Gaussian 0) = ", instance.gaussian_list[0].normalization) - print("Sigma (Gaussian 0) = ", instance.gaussian_list[0].sigma) - print("Normalization (Gaussian 0) = ", instance.gaussian_list[1].normalization) - print("Sigma (Gaussian 0) = ", instance.gaussian_list[1].sigma) - print("Higher Level Centre= ", instance.higher_level_centre) - -This gives the following output: - -.. code-block:: bash - - Model Instance: - - <__main__.MultiLevelGaussians object at 0x7f5273ccd0f0> - Instance Parameters - - Normalization (Gaussian 0) = 1.0 - Sigma (Gaussian 0) = 2.0 - Normalization (Gaussian 0) = 3.0 - Sigma (Gaussian 0) = 4.0 - Higher Level Centre= 5.0 - -Why Use Multi Level Models? ---------------------------- - -An identical model in terms of functionality could of been created via the ``Collection`` object as follows: - -.. code-block:: python - - class GaussianCentre: - def __init__( - self, - centre=30.0, # <- **PyAutoFit** recognises these constructor arguments - normalization=1.0, # <- are the Gaussian``s model parameters. - sigma=5.0, - ): - self.centre = centre - self.normalization = normalization - self.sigma = sigma - - - model = af.Collection(gaussian_0=GaussianCentre, gaussian_1=GaussianCentre) - - model.gaussian_0.centre = model.gaussian_1.centre - - -This raises the question of when to use a ``Collection`` and when to use multi-level models? - -The answer depends on the structure of the models you are composing and fitting. - -Many problems have models which have a natural multi-level structure. - -For example, imagine a dataset had 3 separate groups of 1D ``Gaussian``'s, where each group had multiple Gaussians with -a shared centre. - -This model is concise and easy to define using the multi-level API: - -.. code-block:: python - - group_0 = af.Model(MultiLevelGaussians, gaussian_list=3 * [Gaussian]) - - group_1 = af.Model(MultiLevelGaussians, gaussian_list=3 * [Gaussian]) - - group_2 = af.Model(MultiLevelGaussians, gaussian_list=3 * [Gaussian]) - - model = af.Collection(group_0=group_0, group_1=group_1, group_2=group_2) - - -Composing the same model without the multi-level model is less concise, less readable and prone to error: - -.. code-block:: python - - group_0 = af.Collection( - gaussian_0=GaussianCentre, gaussian_1=GaussianCentre, gaussian_2=GaussianCentre - ) - - group_0.gaussian_0.centre = group_0.gaussian_1.centre - group_0.gaussian_0.centre = group_0.gaussian_2.centre - group_0.gaussian_1.centre = group_0.gaussian_2.centre - - group_1 = af.Collection( - gaussian_0=GaussianCentre, gaussian_1=GaussianCentre, gaussian_2=GaussianCentre - ) - - group_1.gaussian_0.centre = group_1.gaussian_1.centre - group_1.gaussian_0.centre = group_1.gaussian_2.centre - group_1.gaussian_1.centre = group_1.gaussian_2.centre - - group_2 = af.Collection( - gaussian_0=GaussianCentre, gaussian_1=GaussianCentre, gaussian_2=GaussianCentre - ) - - group_2.gaussian_0.centre = group_2.gaussian_1.centre - group_2.gaussian_0.centre = group_2.gaussian_2.centre - group_2.gaussian_1.centre = group_2.gaussian_2.centre - - model = af.Collection(group_0=group_0, group_1=group_1, group_2=group_2) - -Here is what the `model.info` looks like: - -.. code-block:: bash - - Total Free Parameters = 21 - - model Collection (N=21) - group_0 MultiLevelGaussians (N=7) - gaussian_list Collection (N=6) - 0 Gaussian (N=2) - 1 Gaussian (N=2) - 2 Gaussian (N=2) - group_1 MultiLevelGaussians (N=7) - gaussian_list Collection (N=6) - 0 Gaussian (N=2) - 1 Gaussian (N=2) - 2 Gaussian (N=2) - group_2 MultiLevelGaussians (N=7) - gaussian_list Collection (N=6) - 0 Gaussian (N=2) - 1 Gaussian (N=2) - 2 Gaussian (N=2) - - group_0 - higher_level_centre UniformPrior [6], lower_limit = 0.0, upper_limit = 100.0 - gaussian_list - 0 - normalization LogUniformPrior [7], lower_limit = 1e-06, upper_limit = 1000000.0 - sigma UniformPrior [8], lower_limit = 0.0, upper_limit = 25.0 - 1 - normalization LogUniformPrior [9], lower_limit = 1e-06, upper_limit = 1000000.0 - sigma UniformPrior [10], lower_limit = 0.0, upper_limit = 25.0 - 2 - normalization LogUniformPrior [11], lower_limit = 1e-06, upper_limit = 1000000.0 - sigma UniformPrior [12], lower_limit = 0.0, upper_limit = 25.0 - group_1 - higher_level_centre UniformPrior [13], lower_limit = 0.0, upper_limit = 100.0 - gaussian_list - 0 - normalization LogUniformPrior [14], lower_limit = 1e-06, upper_limit = 1000000.0 - sigma UniformPrior [15], lower_limit = 0.0, upper_limit = 25.0 - 1 - normalization LogUniformPrior [16], lower_limit = 1e-06, upper_limit = 1000000.0 - sigma UniformPrior [17], lower_limit = 0.0, upper_limit = 25.0 - 2 - normalization LogUniformPrior [18], lower_limit = 1e-06, upper_limit = 1000000.0 - sigma UniformPrior [19], lower_limit = 0.0, upper_limit = 25.0 - group_2 - higher_level_centre UniformPrior [20], lower_limit = 0.0, upper_limit = 100.0 - gaussian_list - 0 - normalization LogUniformPrior [21], lower_limit = 1e-06, upper_limit = 1000000.0 - sigma UniformPrior [22], lower_limit = 0.0, upper_limit = 25.0 - 1 - normalization LogUniformPrior [23], lower_limit = 1e-06, upper_limit = 1000000.0 - sigma UniformPrior [24], lower_limit = 0.0, upper_limit = 25.0 - 2 - normalization LogUniformPrior [25], lower_limit = 1e-06, upper_limit = 1000000.0 - sigma UniformPrior [26], lower_limit = 0.0, upper_limit = 25.0 - -In many situations, multi-levels models are more extensible than the ``Collection`` API. - -For example, imagine we wanted to add even more 1D profiles to a group with a shared ``centre``. This can easily be -achieved using the multi-level API: - -.. code-block:: python - - multi = af.Model( - MultiLevelGaussians, - gaussian_list=[Gaussian, Gaussian, Exponential, YourProfileHere] - ) - -Composing the same model using just a ``Model`` and ``Collection`` is again possible, but would be even more cumbersome, -less readable and is not extensible. - -Model Customization -------------------- - -To customize the higher level parameters of a multi-level the usual model API is used: - -.. code-block:: python - - multi = af.Model(MultiLevelGaussians, gaussian_list=[Gaussian, Gaussian]) - - multi.higher_level_centre = af.UniformPrior(lower_limit=0.0, upper_limit=100.0) - -To customize a multi-level model instantiated via lists, each model component is accessed via its index: - -.. code-block:: python - - multi = af.Model(MultiLevelGaussians, gaussian_list=[Gaussian, Gaussian]) - - group_level = af.Model(MultiLevelGaussians, gaussian_list=[Gaussian, Gaussian]) - - group_level.gaussian_list[0].normalization = group_level.gaussian_list[1].normalization - -Any combination of the API’s shown above can be used for customizing this model: - -.. code-block:: python - - gaussian_0 = af.Model(Gaussian) - gaussian_1 = af.Model(Gaussian) - - gaussian_0.normalization = gaussian_1.normalization - - group_level = af.Model( - MultiLevelGaussians, gaussian_list=[gaussian_0, gaussian_1, af.Model(Gaussian)] - ) - - group_level.higher_level_centre = 1.0 - group_level.gaussian_list[2].normalization = group_level.gaussian_list[1].normalization - -Here is what the ``model.info`` looks like: - -.. code-block:: bash - - Total Free Parameters = 4 - - model MultiLevelGaussians (N=4) - gaussian_list Collection (N=4) - 0 Gaussian (N=2) - 1 Gaussian (N=2) - 2 Gaussian (N=2) - - higher_level_centre 1.0 - gaussian_list - 0 - normalization LogUniformPrior [45], lower_limit = 1e-06, upper_limit = 1000000.0 - sigma UniformPrior [44], lower_limit = 0.0, upper_limit = 25.0 - 1 - normalization LogUniformPrior [45], lower_limit = 1e-06, upper_limit = 1000000.0 - sigma UniformPrior [46], lower_limit = 0.0, upper_limit = 25.0 - 2 - normalization LogUniformPrior [45], lower_limit = 1e-06, upper_limit = 1000000.0 - sigma UniformPrior [48], lower_limit = 0.0, upper_limit = 25.0 - -Alternative API ---------------- - -A multi-level model can be instantiated where each model sub-component is setup using a name (as opposed to a list). - -This means no list input parameter is required in the Python class of the model component, but we do need to include -the ``**kwargs`` input. - -.. code-block:: python - - class MultiLevelGaussians: - def __init__(self, higher_level_centre=1.0, **kwargs): - self.higher_level_centre = higher_level_centre - - - model = af.Model( - MultiLevelGaussians, gaussian_0=af.Model(Gaussian), gaussian_1=af.Model(Gaussian) - ) - - instance = model.instance_from_vector(vector=[1.0, 2.0, 3.0, 4.0, 5.0]) - - print("Instance Parameters \n") - print("Normalization (Gaussian 0) = ", instance.gaussian_0.normalization) - print("Sigma (Gaussian 0) = ", instance.gaussian_0.sigma) - print("Normalization (Gaussian 0) = ", instance.gaussian_1.normalization) - print("Sigma (Gaussian 0) = ", instance.gaussian_1.sigma) - print("Higher Level Centre= ", instance.higher_level_centre) - -This gives the following output: - -.. code-block:: bash - - Instance Parameters - - Normalization (Gaussian 0) = 1.0 - Sigma (Gaussian 0) = 2.0 - Normalization (Gaussian 0) = 3.0 - Sigma (Gaussian 0) = 4.0 - Higher Level Centre= 5.0 - -The use of Python dictionaries illustrated in previous cookbooks can also be used with multi-level models. - -.. code-block:: python - - model_dict = {"gaussian_0": Gaussian, "gaussian_1": Gaussian} - - model = af.Model(MultiLevelGaussians, **model_dict) - - print(f"Multi-level Model Prior Count = {model.prior_count}") - - instance = model.instance_from_vector(vector=[1.0, 2.0, 3.0, 4.0, 5.0]) - - print("Instance Parameters \n") - print("Normalization (Gaussian 0) = ", instance.gaussian_0.normalization) - print("Sigma (Gaussian 0) = ", instance.gaussian_0.sigma) - print("Normalization (Gaussian 0) = ", instance.gaussian_1.normalization) - print("Sigma (Gaussian 0) = ", instance.gaussian_1.sigma) - print("Higher Level Centre= ", instance.higher_level_centre) - -This gives the following output: - -.. code-block:: bash - - Instance Parameters - - Normalization (Gaussian 0) = 1.0 - Sigma (Gaussian 0) = 2.0 - Normalization (Gaussian 0) = 3.0 - Sigma (Gaussian 0) = 4.0 - Higher Level Centre= 5.0 - -JSon Outputs ------------- - -A model has a ``dict`` attribute, which expresses all information about the model as a Python dictionary. - -By printing this dictionary we can therefore get a concise summary of the model. - -.. code-block:: python - - model = af.Model(Gaussian) - - print(model.dict()) - -This gives the following output: - -.. code-block:: bash - - { - 'class_path': '__main__.Gaussian', 'type': 'model', - 'normalization': {'lower_limit': 1e-06, 'upper_limit': 1000000.0, 'type': 'LogUniform'}, - 'sigma': {'lower_limit': 0.0, 'upper_limit': 25.0, 'type': 'Uniform'} - } - - -The dictionary representation printed above can be saved to hard disk as a ``.json`` file. - -This means we can save any multi-level model to hard-disk in a human readable format. - -Checkout the file ``autofit_workspace/*/cookbooks/jsons/group_level_model.json`` to see the model written as a .json. - -.. code-block:: python - - model_path = path.join("scripts", "cookbooks", "jsons") - - os.makedirs(model_path, exist_ok=True) - - model_file = path.join(model_path, "multi_level_model.json") - - with open(model_file, "w+") as f: - json.dump(model.dict(), f, indent=4) - - -We can load the model from its ``.json`` file, meaning that one can easily save a model to hard disk and load it -elsewhere. - -.. code-block:: python - - model = af.Model.from_json(file=model_file) - -Wrap Up -------- - -This cookbook shows how to multi-level models consisting of multiple components using the ``af.Model()`` -and ``af.Collection()`` objects. - -You should think carefully about whether your model fitting problem can use multi-level models, as they can make -your model definition more concise and extensible. - diff --git a/docs/cookbooks/multiple_datasets.md b/docs/cookbooks/multiple_datasets.md new file mode 100644 index 000000000..3cb4abde9 --- /dev/null +++ b/docs/cookbooks/multiple_datasets.md @@ -0,0 +1,621 @@ +(multiple-datasets)= + +# Multiple Datasets + +This cookbook illustrates how to fit multiple datasets simultaneously, where each dataset is fitted by a different +`Analysis` class. + +The `Analysis` classes are combined to give an overall log likelihood function that is the sum of the +individual log likelihood functions, which a single model is fitted to via non-linear search. + +If one has multiple observations of the same signal, it is often desirable to fit them simultaneously. This ensures +that better constraints are placed on the model, as the full amount of information in the datasets is used. + +In some scenarios, the signal may vary across the datasets in a way that requires that the model is updated +accordingly. **PyAutoFit** provides tools to customize the model composition such that specific parameters of the model +vary across the datasets. + +This cookbook illustrates using observations of 3 1D Gaussians, which have the same `centre` (which is the same +for the model fitted to each dataset) but different `normalization` and `sigma` values (which vary for the model +fitted to each dataset). + +It is common for each individual dataset to only constrain specific aspects of a model. The high level of model +customization provided by **PyAutoFit** ensures that composing a model that is appropriate for fitting large and diverse +datasets is straight forward. This is because different `Analysis` classes can be written for each dataset and combined. + +**Contents:** + +- **Model-Fit**: Setup a model-fit to 3 datasets to illustrate multi-dataset fitting. +- **Analysis List**: Create a list of `Analysis` objects, one for each dataset, which are fitted simultaneously. +- **Analysis Factor**: Wrap each `Analysis` object in an `AnalysisFactor`, which pairs it with the model and prepares it for model fitting. +- **Factor Graph**: Combine all `AnalysisFactor` objects into a `FactorGraphModel`, which represents a global model fit to multiple datasets. +- **Result List**: Use the output of fits to multiple datasets which are a list of `Result` objects. +- **Variable Model Across Datasets**: Fit a model where certain parameters vary across the datasets whereas others stay fixed. +- **Relational Model**: Fit models where certain parameters vary across the dataset as a user defined relation (e.g. `y = mx + c`). +- **Different Analysis Classes**: Fit multiple datasets where each dataset is fitted by a different `Analysis` class, meaning that datasets with different formats can be fitted simultaneously. +- **Interpolation**: Fit multiple datasets with a model one-by-one and interpolation over a smoothly varying parameter (e.g. time) to infer the model between datasets. +- **Individual Sequential Searches**: Fit multiple datasets where each dataset is fitted one-by-one sequentially. +- **Hierarchical / Graphical Models**: Use hierarchical / graphical models to fit multiple datasets simultaneously, which fit for global trends in the model across the datasets. + +## Model Fit + +Load 3 1D Gaussian datasets from .json files in the directory `autofit_workspace/dataset/`. + +All three datasets contain an identical signal, therefore fitting the same model to all three datasets simultaneously +is appropriate. + +Each dataset has a different noise realization, therefore fitting them simultaneously will offer improved constraints +over individual fits. + +```python +dataset_size = 3 + +data_list = [] +noise_map_list = [] + +for dataset_index in range(dataset_size): + dataset_path = path.join( + "dataset", "example_1d", f"gaussian_x1_identical_{dataset_index}" + ) + + data = af.util.numpy_array_from_json(file_path=path.join(dataset_path, "data.json")) + data_list.append(data) + + noise_map = af.util.numpy_array_from_json( + file_path=path.join(dataset_path, "noise_map.json") + ) + noise_map_list.append(noise_map) +``` + +Plot all 3 datasets, including their error bars. + +```python +for data, noise_map in zip(data_list, noise_map_list): + xvalues = range(data.shape[0]) + + plt.errorbar( + x=xvalues, + y=data, + yerr=noise_map, + color="k", + ecolor="k", + linestyle=" ", + elinewidth=1, + capsize=2, + ) + plt.show() + plt.close() +``` + +Here is what the plots look like: + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/feature/docs_update/docs/images/multi_data_0.png +:alt: Alternative text +:width: 300 +``` + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/feature/docs_update/docs/images/multi_data_1.png +:alt: Alternative text +:width: 300 +``` + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/feature/docs_update/docs/images/multi_data_2.png +:alt: Alternative text +:width: 300 +``` + +Create our model corresponding to a single 1D Gaussian that is fitted to all 3 datasets simultaneously. + +```python +model = af.Model(af.ex.Gaussian) + +model.centre = af.UniformPrior(lower_limit=0.0, upper_limit=100.0) +model.normalization = af.LogUniformPrior(lower_limit=1e-2, upper_limit=1e2) +model.sigma = af.GaussianPrior( + mean=10.0, sigma=5.0, lower_limit=0.0, upper_limit=np.inf +) +``` + +## Analysis List + +Set up three instances of the `Analysis` class which fit 1D Gaussian. + +```python +analysis_list = [] + +for data, noise_map in zip(data_list, noise_map_list): + + analysis = af.ex.Analysis(data=data, noise_map=noise_map) + analysis_list.append(analysis) +``` + +## Analysis Factor + +Each analysis object is wrapped in an `AnalysisFactor`, which pairs it with the model and prepares it for use in a +factor graph. This step allows us to flexibly define how each dataset relates to the model. + +The term "Factor" comes from factor graphs, a type of probabilistic graphical model. In this context, each factor +represents the connection between one dataset and the shared model. + +```python +analysis_factor_list = [] + +for analysis in analysis_list: + + analysis_factor = af.AnalysisFactor(prior_model=model, analysis=analysis) + + analysis_factor_list.append(analysis_factor) +``` + +## Factor Graph + +All `AnalysisFactor` objects are combined into a `FactorGraphModel`, which represents a global model fit to +multiple datasets using a graphical model structure. + +The key outcomes of this setup are: + +> - The individual log likelihoods from each `Analysis` object are summed to form the total log likelihood +> evaluated during the model-fitting process. +> - Results from all datasets are output to a unified directory, with subdirectories for visualizations +> from each analysis object, as defined by their `visualize` methods. + +This is a basic use of **PyAutoFit**'s graphical modeling capabilities, which support advanced hierarchical +and probabilistic modeling for large, multi-dataset analyses. + +```python +factor_graph = af.FactorGraphModel(*analysis_factor_list) +``` + +To inspect the model, we print `factor_graph.global_prior_model.info`. + +```python +print(factor_graph.global_prior_model.info) +``` + +To fit multiple datasets, we pass the `FactorGraphModel` to a non-linear search. + +Unlike single-dataset fitting, we now pass the `factor_graph.global_prior_model` as the model and +the `factor_graph` itself as the analysis object. + +This structure enables simultaneous fitting of multiple datasets in a consistent and scalable way. + +```python +search = af.DynestyStatic( + path_prefix="features", sample="rwalk", name="multiple_datasets_simple" +) + +result_list = search.fit(model=factor_graph.global_prior_model, analysis=factor_graph) +``` + +## Result List + +The result object returned by the fit is a list of the `Result` objects, which is described in the result cookbook. + +Each `Result` in the list corresponds to each `Analysis` object in the `analysis_list` we passed to the fit. + +The same model was fitted across all analyses, thus every `Result` in the `result_list` contains the same information +on the samples and the same `max_log_likelihood_instance`. + +```python +print(result_list[0].max_log_likelihood_instance.centre) +print(result_list[0].max_log_likelihood_instance.normalization) +print(result_list[0].max_log_likelihood_instance.sigma) + +print(result_list[1].max_log_likelihood_instance.centre) +print(result_list[1].max_log_likelihood_instance.normalization) +print(result_list[1].max_log_likelihood_instance.sigma) +``` + +This gives the following output: + +```bash +49.99110500540554 +24.793778321608457 +10.067848301502565 +49.99110500540554 +24.793778321608457 +10.067848301502565 +``` + +We can plot the model-fit to each dataset by iterating over the results: + +```python +for data, result in zip(data_list, result_list): + instance = result.max_log_likelihood_instance + + model_data = instance.model_data_from( + xvalues=np.arange(data.shape[0]) + ) + + plt.errorbar( + x=xvalues, + y=data, + yerr=noise_map, + color="k", + ecolor="k", + elinewidth=1, + capsize=2, + ) + plt.plot(xvalues, model_data, color="r") + plt.title("Dynesty model fit to 1D Gaussian dataset.") + plt.xlabel("x values of profile") + plt.ylabel("Profile normalization") + plt.show() + plt.close() +``` + +The image appears as follows: + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/feature/docs_update/docs/images/multi_model_data_0.png +:alt: Alternative text +:width: 300 +``` + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/feature/docs_update/docs/images/multi_model_data_1.png +:alt: Alternative text +:width: 300 +``` + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/feature/docs_update/docs/images/multi_model_data_2.png +:alt: Alternative text +:width: 300 +``` + +## Variable Model Across Datasets + +The same model was fitted to every dataset simultaneously because all 3 datasets contained an identical signal with +only the noise varying across the datasets. + +If the signal varied across the datasets, we would instead want to fit a different model to each dataset. The model +composition can be updated by changing the model passed to each `AnalysisFactor`. + +We will use an example of 3 1D Gaussians which have the same `centre` but the `normalization` and `sigma` vary across +datasets: + +```python +dataset_path = path.join("dataset", "example_1d", "gaussian_x1_variable") + +dataset_name_list = ["sigma_0", "sigma_1", "sigma_2"] + +data_list = [] +noise_map_list = [] + +for dataset_name in dataset_name_list: + dataset_time_path = path.join(dataset_path, dataset_name) + + data = af.util.numpy_array_from_json( + file_path=path.join(dataset_time_path, "data.json") + ) + noise_map = af.util.numpy_array_from_json( + file_path=path.join(dataset_time_path, "noise_map.json") + ) + + data_list.append(data) + noise_map_list.append(noise_map) +``` + +Plotting these datasets shows that the `normalization` and\`\` `sigma` of each Gaussian vary. + +```python +for data, noise_map in zip(data_list, noise_map_list): + xvalues = range(data.shape[0]) + + af.ex.plot_profile_1d(xvalues=xvalues, profile_1d=data) +``` + +The images appear as follows: + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/feature/docs_update/docs/images/multi_model_data_0.png +:alt: Alternative text +:width: 300 +``` + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/feature/docs_update/docs/images/multi_model_data_1.png +:alt: Alternative text +:width: 300 +``` + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/feature/docs_update/docs/images/multi_model_data_2.png +:alt: Alternative text +:width: 300 +``` + +The `centre` of all three 1D Gaussians are the same in each dataset, but their `normalization` and `sigma` values +are decreasing. + +We will therefore fit a model to all three datasets simultaneously, whose `centre` is the same for all 3 datasets but +the `normalization` and `sigma` vary. + +To do that, we use a summed list of `Analysis` objects, where each `Analysis` object contains a different dataset. + +```python +analysis_list = [] + +for data, noise_map in zip(data_list, noise_map_list): + analysis = af.ex.Analysis(data=data, noise_map=noise_map) + analysis_list.append(analysis) +``` + +We now update the model passed to each AnalysisFactor object to compose a model where: + +> - The `centre` values of the Gaussian fitted to every dataset in every `Analysis` object are identical. +> - The\`\`normalization\`\` and `sigma` value of the every Gaussian fitted to every dataset in every `Analysis` object +> are different. + +The model has 7 free parameters in total, x1 shared `centre`, x3 unique `normalization`'s and x3 unique `sigma`'s. + +We do this by overwriting the `normalization` and `sigma` variables of the model passed to each `AnalysisFactor` object +with new priors, that make them free parameters of the model. + +```python +analysis_factor_list = [] + +for analysis in analysis_list: + + model_analysis = model.copy() + + model_analysis.normalization = af.LogUniformPrior( + lower_limit=1e-2, upper_limit=1e2 + ) + model_analysis.sigma = af.GaussianPrior( + mean=10.0, sigma=5.0, lower_limit=0.0, upper_limit=np.inf + ) + + analysis_factor = af.AnalysisFactor(prior_model=model_analysis, analysis=analysis) + + analysis_factor_list.append(analysis_factor) +``` + +To inspect this model, with extra parameters for each dataset created, we print `factor_graph.global_prior_model.info`. + +```python +factor_graph = af.FactorGraphModel(*analysis_factor_list) + +print(factor_graph.global_prior_model.info) +``` + +This gives the following output: + +```bash +Total Free Parameters = 7 + +model GlobalPriorModel (N=7) + 0 - 2 Gaussian (N=3) + +0 - 2 + centre UniformPrior [3], lower_limit = 0.0, upper_limit = 100.0 +0 + normalization LogUniformPrior [6], lower_limit = 0.01, upper_limit = 100.0 + sigma GaussianPrior [7], mean = 10.0, sigma = 5.0 +1 + normalization LogUniformPrior [8], lower_limit = 0.01, upper_limit = 100.0 + sigma GaussianPrior [9], mean = 10.0, sigma = 5.0 +2 + normalization LogUniformPrior [10], lower_limit = 0.01, upper_limit = 100.0 + sigma GaussianPrior [11], mean = 10.0, sigma = 5.0 +``` + +Fit this model to the data using dynesty. + +```python +search = af.DynestyStatic( + path_prefix="features", sample="rwalk", name="multiple_datasets_free_sigma" +) + +result_list = search.fit(model=factor_graph.global_prior_model, analysis=factor_graph) +``` + +The `normalization` and `sigma` values of the maximum log likelihood models fitted to each dataset are different, +which is shown by printing the `sigma` values of the maximum log likelihood instances of each result. + +The `centre` values of the maximum log likelihood models fitted to each dataset are the same. + +```python +for result in result_list: + instance = result.max_log_likelihood_instance + + print("Max Log Likelihood Model:") + print("Centre = ", instance.centre) + print("Normalization = ", instance.normalization) + print("Sigma = ", instance.sigma) + print() +``` + +This gives the following output: + +```bash +Max Log Likelihood Model: +Centre = 50.06514422642149 +Normalization = 50.25307503344711 +Sigma = 10.021209148841097 + +Max Log Likelihood Model: +Centre = 50.06514422642149 +Normalization = 50.21937758886209 +Sigma = 20.143565300562734 + +Max Log Likelihood Model: +Centre = 50.06514422642149 +Normalization = 50.35148002406068 +Sigma = 30.49164712448904 +``` + +## Relational Model + +In the model above, two extra free parameters (``` normalization and ``sigma ```) were added for every dataset. + +For just 3 datasets the model stays low dimensional and this is not a problem. However, for 30+ datasets the model +will become complex and difficult to fit. + +In these circumstances, one can instead compose a model where the parameters vary smoothly across the datasets +via a user defined relation. + +Below, we compose a model where the `sigma` value fitted to each dataset is computed according to: + +```bash +``y = m * x + c`` : ``sigma`` = sigma_m * x + sigma_c`` +``` + +Where x is an integer number specifying the index of the dataset (e.g. 1, 2 and 3). + +By defining a relation of this form, `sigma_m` and `sigma_c` are the only free parameters of the model which vary +across the datasets. + +Of more datasets are added the number of model parameters therefore does not increase. + +```python +model = af.Collection(gaussian=af.Model(af.ex.Gaussian)) + +sigma_m = af.UniformPrior(lower_limit=-10.0, upper_limit=10.0) +sigma_c = af.UniformPrior(lower_limit=-10.0, upper_limit=10.0) + +x_list = [1.0, 2.0, 3.0] + +analysis_factor_list = [] + +for x, analysis in zip(x_list, analysis_list): + sigma_relation = (sigma_m * x) + sigma_c + + model_analysis = model.copy() + model_analysis.gaussian.sigma = sigma_relation + + analysis_factor = af.AnalysisFactor(prior_model=model_analysis, analysis=analysis) + + analysis_factor_list.append(analysis_factor) +``` + +The factor graph is created and its info can be printed after the relational model has been defined. + +```python +factor_graph = af.FactorGraphModel(*analysis_factor_list) + +print(factor_graph.global_prior_model.info) +``` + +This gives the following output: + +```bash +Total Free Parameters = 4 + +model GlobalPriorModel (N=4) + 0 - 2 Collection (N=4) + gaussian Gaussian (N=4) + sigma SumPrior (N=2) + self MultiplePrior (N=1) + +factor + include_prior_factors True +0 - 2 + gaussian + centre UniformPrior [12], lower_limit = 0.0, upper_limit = 100.0 + normalization LogUniformPrior [13], lower_limit = 1e-06, upper_limit = 1000000.0 + sigma + self + sigma_m UniformPrior [15], lower_limit = -10.0, upper_limit = 10.0 + sigma_c UniformPrior [16], lower_limit = -10.0, upper_limit = 10.0 +0 + gaussian + sigma + self + x 1.0 +1 + gaussian + sigma + self + x 2.0 +2 + gaussian + sigma + self + x 3.0 +``` + +We can fit the model as per usual. + +```python +search = af.DynestyStatic( + path_prefix="features", sample="rwalk", name="multiple_datasets_relation" +) + +result_list = search.fit(model=factor_graph.global_prior_model, analysis=factor_graph) +``` + +The `centre` and `sigma` values of the maximum log likelihood models fitted to each dataset are different, +which is shown by printing the `sigma` values of the maximum log likelihood instances of each result. + +They now follow the relation we defined above. + +The `centre` normalization of the maximum log likelihood models fitted to each dataset are the same. + +```python +result_list = search.fit(model=model, analysis=analysis) + +for result in result_list: + instance = result.max_log_likelihood_instance + + print("Max Log Likelihood Model:") + print("Centre = ", instance.centre) + print("Normalization = ", instance.normalization) + print("Sigma = ", instance.sigma) + print() +``` + +This gives the following output: + +```bash +Max Log Likelihood Model: +Centre = 50.04124738060383 +Normalization = 50.330187946622246 +Sigma = 10.04918613466697 + +Max Log Likelihood Model: +Centre = 50.04124738060383 +Normalization = 50.330187946622246 +Sigma = 20.04864425755685 + +Max Log Likelihood Model: +Centre = 50.04124738060383 +Normalization = 50.330187946622246 +Sigma = 30.048102380446732 +``` + +## Different Analysis Objects + +For simplicity, this example used a single `Analysis` class which fitted 1D Gaussian's to 1D data. + +For many problems one may have multiple datasets which are quite different in their format and structure. In this +situation, one can simply define unique `Analysis` objects for each type of dataset, which will contain a +unique `log_likelihood_function` and methods for visualization. + +## Hierarchical / Graphical Models + +The analysis factor API illustrated here can then be used to fit this large variety of datasets, noting that the +the model can also be customized as necessary for fitting models to multiple datasets that are different in their +format and structure. + +This allows us to fit large heterogeneous datasets simultaneously, but also forms the basis of the graphical +modeling API which can be used to fit complex models, such as hierarchical models, to extract more information +from large datasets. + +**PyAutoFit** has a dedicated feature set for fitting hierarchical and graphical models and interested readers should +checkout the hierarchical and graphical modeling +chapter of **HowToFit** () + +## Interpolation + +One may have many datasets which vary according to a smooth function, for example a dataset taken over time where +the signal varies smoothly as a function of time. + +This could be fitted using the tools above, all at once. However, in many use cases this is not possible due to the +model complexity, number of datasets or computational time. + +An alternative approach is to fit each dataset individually, and then interpolate the results over the smoothly +varying parameter (e.g. time) to estimate the model parameters at any point. + +**PyAutoFit** has interpolation tools to do exactly this, which are described in the `features/interpolation.ipynb` +example. + +## Wrap Up + +We have shown how **PyAutoFit** can fit large datasets simultaneously, using custom models that vary specific +parameters across the dataset. diff --git a/docs/cookbooks/multiple_datasets.rst b/docs/cookbooks/multiple_datasets.rst deleted file mode 100644 index 9fbebe9d4..000000000 --- a/docs/cookbooks/multiple_datasets.rst +++ /dev/null @@ -1,632 +0,0 @@ -.. _multiple_datasets: - -Multiple Datasets -================= - -This cookbook illustrates how to fit multiple datasets simultaneously, where each dataset is fitted by a different -``Analysis`` class. - -The ``Analysis`` classes are combined to give an overall log likelihood function that is the sum of the -individual log likelihood functions, which a single model is fitted to via non-linear search. - -If one has multiple observations of the same signal, it is often desirable to fit them simultaneously. This ensures -that better constraints are placed on the model, as the full amount of information in the datasets is used. - -In some scenarios, the signal may vary across the datasets in a way that requires that the model is updated -accordingly. **PyAutoFit** provides tools to customize the model composition such that specific parameters of the model -vary across the datasets. - -This cookbook illustrates using observations of 3 1D Gaussians, which have the same ``centre`` (which is the same -for the model fitted to each dataset) but different ``normalization`` and ``sigma`` values (which vary for the model -fitted to each dataset). - -It is common for each individual dataset to only constrain specific aspects of a model. The high level of model -customization provided by **PyAutoFit** ensures that composing a model that is appropriate for fitting large and diverse -datasets is straight forward. This is because different ``Analysis`` classes can be written for each dataset and combined. - -**Contents:** - -- **Model-Fit**: Setup a model-fit to 3 datasets to illustrate multi-dataset fitting. -- **Analysis List**: Create a list of ``Analysis`` objects, one for each dataset, which are fitted simultaneously. -- **Analysis Factor**: Wrap each ``Analysis`` object in an ``AnalysisFactor``, which pairs it with the model and prepares it for model fitting. -- **Factor Graph**: Combine all ``AnalysisFactor`` objects into a ``FactorGraphModel``, which represents a global model fit to multiple datasets. -- **Result List**: Use the output of fits to multiple datasets which are a list of ``Result`` objects. -- **Variable Model Across Datasets**: Fit a model where certain parameters vary across the datasets whereas others stay fixed. -- **Relational Model**: Fit models where certain parameters vary across the dataset as a user defined relation (e.g. ``y = mx + c``). -- **Different Analysis Classes**: Fit multiple datasets where each dataset is fitted by a different ``Analysis`` class, meaning that datasets with different formats can be fitted simultaneously. -- **Interpolation**: Fit multiple datasets with a model one-by-one and interpolation over a smoothly varying parameter (e.g. time) to infer the model between datasets. -- **Individual Sequential Searches**: Fit multiple datasets where each dataset is fitted one-by-one sequentially. -- **Hierarchical / Graphical Models**: Use hierarchical / graphical models to fit multiple datasets simultaneously, which fit for global trends in the model across the datasets. - -Model Fit ---------- - -Load 3 1D Gaussian datasets from .json files in the directory ``autofit_workspace/dataset/``. - -All three datasets contain an identical signal, therefore fitting the same model to all three datasets simultaneously -is appropriate. - -Each dataset has a different noise realization, therefore fitting them simultaneously will offer improved constraints -over individual fits. - -.. code-block:: python - - dataset_size = 3 - - data_list = [] - noise_map_list = [] - - for dataset_index in range(dataset_size): - dataset_path = path.join( - "dataset", "example_1d", f"gaussian_x1_identical_{dataset_index}" - ) - - data = af.util.numpy_array_from_json(file_path=path.join(dataset_path, "data.json")) - data_list.append(data) - - noise_map = af.util.numpy_array_from_json( - file_path=path.join(dataset_path, "noise_map.json") - ) - noise_map_list.append(noise_map) - -Plot all 3 datasets, including their error bars. - -.. code-block:: python - - for data, noise_map in zip(data_list, noise_map_list): - xvalues = range(data.shape[0]) - - plt.errorbar( - x=xvalues, - y=data, - yerr=noise_map, - color="k", - ecolor="k", - linestyle=" ", - elinewidth=1, - capsize=2, - ) - plt.show() - plt.close() - -Here is what the plots look like: - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/feature/docs_update/docs/images/multi_data_0.png - :width: 300 - :alt: Alternative text - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/feature/docs_update/docs/images/multi_data_1.png - :width: 300 - :alt: Alternative text - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/feature/docs_update/docs/images/multi_data_2.png - :width: 300 - :alt: Alternative text - -Create our model corresponding to a single 1D Gaussian that is fitted to all 3 datasets simultaneously. - -.. code-block:: python - - model = af.Model(af.ex.Gaussian) - - model.centre = af.UniformPrior(lower_limit=0.0, upper_limit=100.0) - model.normalization = af.LogUniformPrior(lower_limit=1e-2, upper_limit=1e2) - model.sigma = af.GaussianPrior( - mean=10.0, sigma=5.0, lower_limit=0.0, upper_limit=np.inf - ) - -Analysis List -------------- - -Set up three instances of the ``Analysis`` class which fit 1D Gaussian. - -.. code-block:: python - - analysis_list = [] - - for data, noise_map in zip(data_list, noise_map_list): - - analysis = af.ex.Analysis(data=data, noise_map=noise_map) - analysis_list.append(analysis) - -Analysis Factor ---------------- - -Each analysis object is wrapped in an ``AnalysisFactor``, which pairs it with the model and prepares it for use in a -factor graph. This step allows us to flexibly define how each dataset relates to the model. - -The term "Factor" comes from factor graphs, a type of probabilistic graphical model. In this context, each factor -represents the connection between one dataset and the shared model. - -.. code-block:: python - - analysis_factor_list = [] - - for analysis in analysis_list: - - analysis_factor = af.AnalysisFactor(prior_model=model, analysis=analysis) - - analysis_factor_list.append(analysis_factor) - -Factor Graph ------------- - -All ``AnalysisFactor`` objects are combined into a ``FactorGraphModel``, which represents a global model fit to -multiple datasets using a graphical model structure. - -The key outcomes of this setup are: - - - The individual log likelihoods from each ``Analysis`` object are summed to form the total log likelihood - evaluated during the model-fitting process. - - - Results from all datasets are output to a unified directory, with subdirectories for visualizations - from each analysis object, as defined by their ``visualize`` methods. - -This is a basic use of **PyAutoFit**'s graphical modeling capabilities, which support advanced hierarchical -and probabilistic modeling for large, multi-dataset analyses. - -.. code-block:: python - - factor_graph = af.FactorGraphModel(*analysis_factor_list) - -To inspect the model, we print `factor_graph.global_prior_model.info`. - -.. code-block:: python - - print(factor_graph.global_prior_model.info) - -To fit multiple datasets, we pass the `FactorGraphModel` to a non-linear search. - -Unlike single-dataset fitting, we now pass the `factor_graph.global_prior_model` as the model and -the `factor_graph` itself as the analysis object. - -This structure enables simultaneous fitting of multiple datasets in a consistent and scalable way. - -.. code-block:: python - - search = af.DynestyStatic( - path_prefix="features", sample="rwalk", name="multiple_datasets_simple" - ) - - result_list = search.fit(model=factor_graph.global_prior_model, analysis=factor_graph) - -Result List ------------ - -The result object returned by the fit is a list of the ``Result`` objects, which is described in the result cookbook. - -Each ``Result`` in the list corresponds to each ``Analysis`` object in the ``analysis_list`` we passed to the fit. - -The same model was fitted across all analyses, thus every ``Result`` in the ``result_list`` contains the same information -on the samples and the same ``max_log_likelihood_instance``. - -.. code-block:: python - - print(result_list[0].max_log_likelihood_instance.centre) - print(result_list[0].max_log_likelihood_instance.normalization) - print(result_list[0].max_log_likelihood_instance.sigma) - - print(result_list[1].max_log_likelihood_instance.centre) - print(result_list[1].max_log_likelihood_instance.normalization) - print(result_list[1].max_log_likelihood_instance.sigma) - -This gives the following output: - -.. code-block:: bash - - 49.99110500540554 - 24.793778321608457 - 10.067848301502565 - 49.99110500540554 - 24.793778321608457 - 10.067848301502565 - -We can plot the model-fit to each dataset by iterating over the results: - -.. code-block:: python - - for data, result in zip(data_list, result_list): - instance = result.max_log_likelihood_instance - - model_data = instance.model_data_from( - xvalues=np.arange(data.shape[0]) - ) - - plt.errorbar( - x=xvalues, - y=data, - yerr=noise_map, - color="k", - ecolor="k", - elinewidth=1, - capsize=2, - ) - plt.plot(xvalues, model_data, color="r") - plt.title("Dynesty model fit to 1D Gaussian dataset.") - plt.xlabel("x values of profile") - plt.ylabel("Profile normalization") - plt.show() - plt.close() - -The image appears as follows: - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/feature/docs_update/docs/images/multi_model_data_0.png - :width: 300 - :alt: Alternative text - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/feature/docs_update/docs/images/multi_model_data_1.png - :width: 300 - :alt: Alternative text - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/feature/docs_update/docs/images/multi_model_data_2.png - :width: 300 - :alt: Alternative text - -Variable Model Across Datasets ------------------------------- - -The same model was fitted to every dataset simultaneously because all 3 datasets contained an identical signal with -only the noise varying across the datasets. - -If the signal varied across the datasets, we would instead want to fit a different model to each dataset. The model -composition can be updated by changing the model passed to each ``AnalysisFactor``. - -We will use an example of 3 1D Gaussians which have the same ``centre`` but the ``normalization`` and ``sigma`` vary across -datasets: - -.. code-block:: python - - dataset_path = path.join("dataset", "example_1d", "gaussian_x1_variable") - - dataset_name_list = ["sigma_0", "sigma_1", "sigma_2"] - - data_list = [] - noise_map_list = [] - - for dataset_name in dataset_name_list: - dataset_time_path = path.join(dataset_path, dataset_name) - - data = af.util.numpy_array_from_json( - file_path=path.join(dataset_time_path, "data.json") - ) - noise_map = af.util.numpy_array_from_json( - file_path=path.join(dataset_time_path, "noise_map.json") - ) - - data_list.append(data) - noise_map_list.append(noise_map) - -Plotting these datasets shows that the ``normalization`` and`` ``sigma`` of each Gaussian vary. - -.. code-block:: python - - for data, noise_map in zip(data_list, noise_map_list): - xvalues = range(data.shape[0]) - - af.ex.plot_profile_1d(xvalues=xvalues, profile_1d=data) - -The images appear as follows: - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/feature/docs_update/docs/images/multi_model_data_0.png - :width: 300 - :alt: Alternative text - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/feature/docs_update/docs/images/multi_model_data_1.png - :width: 300 - :alt: Alternative text - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/feature/docs_update/docs/images/multi_model_data_2.png - :width: 300 - :alt: Alternative text - - -The ``centre`` of all three 1D Gaussians are the same in each dataset, but their ``normalization`` and ``sigma`` values -are decreasing. - -We will therefore fit a model to all three datasets simultaneously, whose ``centre`` is the same for all 3 datasets but -the ``normalization`` and ``sigma`` vary. - -To do that, we use a summed list of ``Analysis`` objects, where each ``Analysis`` object contains a different dataset. - -.. code-block:: python - - analysis_list = [] - - for data, noise_map in zip(data_list, noise_map_list): - analysis = af.ex.Analysis(data=data, noise_map=noise_map) - analysis_list.append(analysis) - -We now update the model passed to each ``AnalysisFactor ``object to compose a model where: - - - The ``centre`` values of the Gaussian fitted to every dataset in every ``Analysis`` object are identical. - - - The``normalization`` and ``sigma`` value of the every Gaussian fitted to every dataset in every ``Analysis`` object - are different. - -The model has 7 free parameters in total, x1 shared ``centre``, x3 unique ``normalization``'s and x3 unique ``sigma``'s. - -We do this by overwriting the ``normalization`` and ``sigma`` variables of the model passed to each ``AnalysisFactor`` object -with new priors, that make them free parameters of the model. - -.. code-block:: python - - analysis_factor_list = [] - - for analysis in analysis_list: - - model_analysis = model.copy() - - model_analysis.normalization = af.LogUniformPrior( - lower_limit=1e-2, upper_limit=1e2 - ) - model_analysis.sigma = af.GaussianPrior( - mean=10.0, sigma=5.0, lower_limit=0.0, upper_limit=np.inf - ) - - analysis_factor = af.AnalysisFactor(prior_model=model_analysis, analysis=analysis) - - analysis_factor_list.append(analysis_factor) - -To inspect this model, with extra parameters for each dataset created, we print `factor_graph.global_prior_model.info`. - -.. code-block:: python - - factor_graph = af.FactorGraphModel(*analysis_factor_list) - - print(factor_graph.global_prior_model.info) - -This gives the following output: - -.. code-block:: bash - - Total Free Parameters = 7 - - model GlobalPriorModel (N=7) - 0 - 2 Gaussian (N=3) - - 0 - 2 - centre UniformPrior [3], lower_limit = 0.0, upper_limit = 100.0 - 0 - normalization LogUniformPrior [6], lower_limit = 0.01, upper_limit = 100.0 - sigma GaussianPrior [7], mean = 10.0, sigma = 5.0 - 1 - normalization LogUniformPrior [8], lower_limit = 0.01, upper_limit = 100.0 - sigma GaussianPrior [9], mean = 10.0, sigma = 5.0 - 2 - normalization LogUniformPrior [10], lower_limit = 0.01, upper_limit = 100.0 - sigma GaussianPrior [11], mean = 10.0, sigma = 5.0 - -Fit this model to the data using dynesty. - -.. code-block:: python - - search = af.DynestyStatic( - path_prefix="features", sample="rwalk", name="multiple_datasets_free_sigma" - ) - - result_list = search.fit(model=factor_graph.global_prior_model, analysis=factor_graph) - - -The ``normalization`` and ``sigma`` values of the maximum log likelihood models fitted to each dataset are different, -which is shown by printing the ``sigma`` values of the maximum log likelihood instances of each result. - -The ``centre`` values of the maximum log likelihood models fitted to each dataset are the same. - -.. code-block:: python - - for result in result_list: - instance = result.max_log_likelihood_instance - - print("Max Log Likelihood Model:") - print("Centre = ", instance.centre) - print("Normalization = ", instance.normalization) - print("Sigma = ", instance.sigma) - print() - -This gives the following output: - -.. code-block:: bash - - Max Log Likelihood Model: - Centre = 50.06514422642149 - Normalization = 50.25307503344711 - Sigma = 10.021209148841097 - - Max Log Likelihood Model: - Centre = 50.06514422642149 - Normalization = 50.21937758886209 - Sigma = 20.143565300562734 - - Max Log Likelihood Model: - Centre = 50.06514422642149 - Normalization = 50.35148002406068 - Sigma = 30.49164712448904 - -Relational Model ----------------- - -In the model above, two extra free parameters (``normalization and ``sigma``) were added for every dataset. - -For just 3 datasets the model stays low dimensional and this is not a problem. However, for 30+ datasets the model -will become complex and difficult to fit. - -In these circumstances, one can instead compose a model where the parameters vary smoothly across the datasets -via a user defined relation. - -Below, we compose a model where the ``sigma`` value fitted to each dataset is computed according to: - - -.. code-block:: bash - - ``y = m * x + c`` : ``sigma`` = sigma_m * x + sigma_c`` - -Where x is an integer number specifying the index of the dataset (e.g. 1, 2 and 3). - -By defining a relation of this form, ``sigma_m`` and ``sigma_c`` are the only free parameters of the model which vary -across the datasets. - -Of more datasets are added the number of model parameters therefore does not increase. - -.. code-block:: python - - model = af.Collection(gaussian=af.Model(af.ex.Gaussian)) - - sigma_m = af.UniformPrior(lower_limit=-10.0, upper_limit=10.0) - sigma_c = af.UniformPrior(lower_limit=-10.0, upper_limit=10.0) - - x_list = [1.0, 2.0, 3.0] - - analysis_factor_list = [] - - for x, analysis in zip(x_list, analysis_list): - sigma_relation = (sigma_m * x) + sigma_c - - model_analysis = model.copy() - model_analysis.gaussian.sigma = sigma_relation - - analysis_factor = af.AnalysisFactor(prior_model=model_analysis, analysis=analysis) - - analysis_factor_list.append(analysis_factor) - -The factor graph is created and its info can be printed after the relational model has been defined. - -.. code-block:: python - - factor_graph = af.FactorGraphModel(*analysis_factor_list) - - print(factor_graph.global_prior_model.info) - -This gives the following output: - -.. code-block:: bash - - Total Free Parameters = 4 - - model GlobalPriorModel (N=4) - 0 - 2 Collection (N=4) - gaussian Gaussian (N=4) - sigma SumPrior (N=2) - self MultiplePrior (N=1) - - factor - include_prior_factors True - 0 - 2 - gaussian - centre UniformPrior [12], lower_limit = 0.0, upper_limit = 100.0 - normalization LogUniformPrior [13], lower_limit = 1e-06, upper_limit = 1000000.0 - sigma - self - sigma_m UniformPrior [15], lower_limit = -10.0, upper_limit = 10.0 - sigma_c UniformPrior [16], lower_limit = -10.0, upper_limit = 10.0 - 0 - gaussian - sigma - self - x 1.0 - 1 - gaussian - sigma - self - x 2.0 - 2 - gaussian - sigma - self - x 3.0 - -We can fit the model as per usual. - -.. code-block:: python - - search = af.DynestyStatic( - path_prefix="features", sample="rwalk", name="multiple_datasets_relation" - ) - - result_list = search.fit(model=factor_graph.global_prior_model, analysis=factor_graph) - -The ``centre`` and ``sigma`` values of the maximum log likelihood models fitted to each dataset are different, -which is shown by printing the ``sigma`` values of the maximum log likelihood instances of each result. - -They now follow the relation we defined above. - -The ``centre`` normalization of the maximum log likelihood models fitted to each dataset are the same. - -.. code-block:: python - - result_list = search.fit(model=model, analysis=analysis) - - for result in result_list: - instance = result.max_log_likelihood_instance - - print("Max Log Likelihood Model:") - print("Centre = ", instance.centre) - print("Normalization = ", instance.normalization) - print("Sigma = ", instance.sigma) - print() - -This gives the following output: - -.. code-block:: bash - - Max Log Likelihood Model: - Centre = 50.04124738060383 - Normalization = 50.330187946622246 - Sigma = 10.04918613466697 - - Max Log Likelihood Model: - Centre = 50.04124738060383 - Normalization = 50.330187946622246 - Sigma = 20.04864425755685 - - Max Log Likelihood Model: - Centre = 50.04124738060383 - Normalization = 50.330187946622246 - Sigma = 30.048102380446732 - -Different Analysis Objects --------------------------- - -For simplicity, this example used a single `Analysis` class which fitted 1D Gaussian's to 1D data. - -For many problems one may have multiple datasets which are quite different in their format and structure. In this -situation, one can simply define unique `Analysis` objects for each type of dataset, which will contain a -unique `log_likelihood_function` and methods for visualization. - -Hierarchical / Graphical Models -------------------------------- - -The analysis factor API illustrated here can then be used to fit this large variety of datasets, noting that the -the model can also be customized as necessary for fitting models to multiple datasets that are different in their -format and structure. - -This allows us to fit large heterogeneous datasets simultaneously, but also forms the basis of the graphical -modeling API which can be used to fit complex models, such as hierarchical models, to extract more information -from large datasets. - -**PyAutoFit** has a dedicated feature set for fitting hierarchical and graphical models and interested readers should -checkout the hierarchical and graphical modeling -chapter of **HowToFit** (https://github.com/PyAutoLabs/HowToFit/blob/main/notebooks/chapter_3_graphical_models) - -Interpolation -------------- - -One may have many datasets which vary according to a smooth function, for example a dataset taken over time where -the signal varies smoothly as a function of time. - -This could be fitted using the tools above, all at once. However, in many use cases this is not possible due to the -model complexity, number of datasets or computational time. - -An alternative approach is to fit each dataset individually, and then interpolate the results over the smoothly -varying parameter (e.g. time) to estimate the model parameters at any point. - -**PyAutoFit** has interpolation tools to do exactly this, which are described in the `features/interpolation.ipynb` -example. - - -Wrap Up --------- - -We have shown how **PyAutoFit** can fit large datasets simultaneously, using custom models that vary specific -parameters across the dataset. - - diff --git a/docs/cookbooks/result.md b/docs/cookbooks/result.md new file mode 100644 index 000000000..66c1c3ee7 --- /dev/null +++ b/docs/cookbooks/result.md @@ -0,0 +1,754 @@ +(results)= + +# Results + +A non-linear search fits a model to a dataset, returning a `Result` object that contains a lot of information on the +model-fit. + +This cookbook provides a concise reference to the result API. + +The cookbook then describes how the results of a search can be output to hard-disk and loaded back into Python, +either using the `Aggregator` object or by building an sqlite database of results. Result loading supports +queries, so that only the results of interest are returned. + +The samples of the non-linear search, which are used to estimate quantities the maximum likelihood model and +parameter errors, are described separately in the `samples.py` cookbook. + +**Contents:** + +An overview of the `Result` object's functionality is given in the following sections: + +> - **Info**: Print the `info` attribute of the `Result` object to display a summary of the model-fit. +> - **Max Log Likelihood Instance**: Getting the maximum likelihood model instance. +> - **Samples**: Getting the samples of the non-linear search from a result. +> - **Custom Result**: Extending the `Result` object with custom attributes specific to the model-fit. + +The cookbook next describes how results can be output to hard-disk and loaded back into Python via the `Aggregator`: + +> - **Output To Hard-Disk**: Output results to hard-disk so they can be inspected and used to restart a crashed search. +> - **Files**: The files that are stored in the `files` folder that is created when results are output to hard-disk. +> - **Loading From Hard-disk**: Loading results from hard-disk to Python variables via the aggregator. +> - **Generators**: Why loading results uses Python generators to ensure memory efficiency. + +The cookbook next gives examples of how to load all the following results from the database: + +> - **Samples**: The samples of the non-linear search (e.g. all parameter values, log likelihoods, etc.). +> - **Model**: The model fitted by the non-linear search. +> - **Search**: The search used to perform the model-fit. +> - **Samples Info**: Additional information on the samples. +> - **Samples Summary**: A summary of the samples of the non-linear search (e.g. the maximum log likelihood model). +> - **Info**: The `info` dictionary passed to the search. + +The output of results to hard-disk is customizeable and described in the following section: + +> - **Custom Output**: Extend `Analysis` classes to output additional information which can be loaded via the aggregator. + +Using queries to load specific results is described in the following sections\*\*: + +> - **Querying Datasets**: Query based on the name of the dataset. +> - **Querying Searches**: Query based on the name of the search. +> - **Querying Models**: Query based on the model that is fitted. +> - **Querying Results**: Query based on the results of the model-fit. +> - **Querying Logic**: Use logic to combine queries to load specific results (e.g. AND, OR, etc.). + +The final section describes how to use results built in an sqlite database file: + +> - **Database**: Building a database file from the output folder. +> - **Unique Identifiers**: The unique identifier of each model-fit. +> - **Writing Directly To Database**: Writing results directly to the database. + +## Model Fit + +To illustrate results, we need to perform a model-fit in order to create a `Result` object. + +We do this below using the standard API and noisy 1D signal example, which you should be familiar with from other +example scripts. + +Note that the `Gaussian` and `Analysis` classes come via the `af.ex` module, which contains example model components +that are identical to those found throughout the examples. + +```python +dataset_path = path.join("dataset", "example_1d", "gaussian_x1") +data = af.util.numpy_array_from_json(file_path=path.join(dataset_path, "data.json")) +noise_map = af.util.numpy_array_from_json( + file_path=path.join(dataset_path, "noise_map.json") +) + +model = af.Model(af.ex.Gaussian) + +analysis = af.ex.Analysis(data=data, noise_map=noise_map) + +search = af.Emcee( + nwalkers=30, + nsteps=1000, + number_of_cores=1, +) + +result = search.fit(model=model, analysis=analysis) +``` + +## Info + +Printing the `info` attribute shows the overall result of the model-fit in a human readable format. + +```python +print(result.info) +``` + +The output appears as follows: + +```bash +Maximum Log Likelihood -46.68992727 +Maximum Log Posterior -46.64963514 + +model Gaussian (N=3) + +Maximum Log Likelihood Model: + +centre 49.892 +normalization 24.819 +sigma 9.844 + + +Summary (3.0 sigma limits): + +centre 49.89 (49.52, 50.23) +normalization 24.79 (23.96, 25.61) +sigma 9.85 (9.53, 10.21) + + +Summary (1.0 sigma limits): + +centre 49.89 (49.83, 49.96) +normalization 24.79 (24.65, 24.94) +sigma 9.85 (9.78, 9.90) +``` + +The `max_log_likelihood_instance` is the model instance of the maximum log likelihood model, which is the model +that maximizes the likelihood of the data given the model. + +```python +instance = result.max_log_likelihood_instance + +print("Max Log Likelihood `Gaussian` Instance:") +print("Centre = ", instance.centre) +print("Normalization = ", instance.normalization) +print("Sigma = ", instance.sigma) +``` + +The `Samples` class contains all information on the non-linear search samples, for example the value of every parameter +sampled using the fit or an instance of the maximum likelihood model. + +```python +samples = result.samples +``` + +The samples are described in detail separately in the `samples.py` cookbook. + +## Custom Result + +The result can be can be customized to include additional information about the model-fit that is specific to your +model-fitting problem. + +For example, for fitting 1D profiles, the `Result` could include the maximum log likelihood model 1D data: + +`print(result.max_log_likelihood_model_data_1d)` + +In other examples, this quantity has been manually computed after the model-fit has completed. + +The custom result API allows us to do this. First, we define a custom `Result` class, which includes the property +`max_log_likelihood_model_data_1d`. + +```python +class ResultExample(af.Result): + @property + def max_log_likelihood_model_data_1d(self) -> np.ndarray: + """ + Returns the maximum log likelihood model's 1D model data. + + This is an example of how we can pass the `Analysis` class a custom `Result` object and extend this result + object with new properties that are specific to the model-fit we are performing. + """ + xvalues = np.arange(self.analysis.data.shape[0]) + + return self.instance.model_data_from(xvalues=xvalues) +``` + +The custom result has access to the analysis class, meaning that we can use any of its methods or properties to +compute custom result properties. + +To make it so that the `ResultExample` object above is returned by the search we overwrite the `Result` class attribute +of the `Analysis` and define a `make_result` object describing what we want it to contain: + +```python +class Analysis(af.ex.Analysis): + + """ + This overwrite means the `ResultExample` class is returned after the model-fit. + """ + + Result = ResultExample + + def make_result( + self, + samples_summary: af.SamplesSummary, + paths: af.AbstractPaths, + samples: Optional[af.SamplesPDF] = None, + search_internal: Optional[object] = None, + analysis: Optional[object] = None, + ) -> Result: + """ + Returns the `Result` of the non-linear search after it is completed. + + The result type is defined as a class variable in the `Analysis` class (see top of code under the python code + `class Analysis(af.Analysis)`. + + The result can be manually overwritten by a user to return a user-defined result object, which can be extended + with additional methods and attribute specific to the model-fit. + + This example class does example this, whereby the analysis result has been overwritten with the `ResultExample` + class, which contains a property `max_log_likelihood_model_data_1d` that returns the model data of the + best-fit model. This API means you can customize your result object to include whatever attributes you want + and therefore make a result object specific to your model-fit and model-fitting problem. + + The `Result` object you return can be customized to include: + + - The samples summary, which contains the maximum log likelihood instance and median PDF model. + + - The paths of the search, which are used for loading the samples and search internal below when a search + is resumed. + + - The samples of the non-linear search (e.g. MCMC chains) also stored in `samples.csv`. + + - The non-linear search used for the fit in its internal representation, which is used for resuming a search + and making bespoke visualization using the search's internal results. + + - The analysis used to fit the model (default disabled to save memory, but option may be useful for certain + projects). + + Parameters + ---------- + samples_summary + The summary of the samples of the non-linear search, which include the maximum log likelihood instance and + median PDF model. + paths + An object describing the paths for saving data (e.g. hard-disk directories or entries in sqlite database). + samples + The samples of the non-linear search, for example the chains of an MCMC run. + search_internal + The internal representation of the non-linear search used to perform the model-fit. + analysis + The analysis used to fit the model. + + Returns + ------- + Result + The result of the non-linear search, which is defined as a class variable in the `Analysis` class. + """ + return self.Result( + samples_summary=samples_summary, + paths=paths, + samples=samples, + search_internal=search_internal, + analysis=self, + ) +``` + +Using the `Analysis` class above, the `Result` object returned by the search is now a `ResultExample` object. + +```python +analysis = af.ex.Analysis(data=data, noise_map=noise_map) + +search = af.Emcee( + nwalkers=30, + nsteps=1000, +) + +result = search.fit(model=model, analysis=analysis) + +print(result.max_log_likelihood_model_data_1d) +``` + +## Output To Hard-Disk + +By default, a non-linear search does not output its results to hard-disk and its results can only be inspected +in Python via the `result` object. + +However, the results of any non-linear search can be output to hard-disk by passing the `name` and / or `path_prefix` +attributes, which are used to name files and output the results to a folder on your hard-disk. + +This cookbook now runs the three searches with output to hard-disk enabled, so you can see how the results are output +to hard-disk and to then illustrate how they can be loaded back into Python. + +Note that an `info` dictionary is also passed to the search, which includes the date of the model-fit and the exposure +time of the dataset. This information is stored output to hard-disk and can be loaded to help interpret the results. + +```python +info = {"date_of_observation": "01-02-18", "exposure_time": 1000.0} + +dataset_name_list = ["gaussian_x1_0", "gaussian_x1_1", "gaussian_x1_2"] + +model = af.Collection(gaussian=af.ex.Gaussian) + +model.gaussian.centre = af.UniformPrior(lower_limit=0.0, upper_limit=100.0) +model.gaussian.normalization = af.LogUniformPrior(lower_limit=1e-2, upper_limit=1e2) +model.gaussian.sigma = af.GaussianPrior( + mean=10.0, sigma=5.0, lower_limit=0.0, upper_limit=np.inf +) + +for dataset_name in dataset_name_list: + dataset_path = path.join("dataset", "example_1d", dataset_name) + + data = af.util.numpy_array_from_json(file_path=path.join(dataset_path, "data.json")) + noise_map = af.util.numpy_array_from_json( + file_path=path.join(dataset_path, "noise_map.json") + ) + + analysis = af.ex.Analysis(data=data, noise_map=noise_map) + + search = af.DynestyStatic( + name="multi_result_example", + path_prefix=path.join("cookbooks", "result"), + unique_tag=dataset_name, # This makes the unique identifier use the dataset name + nlive=50, + ) + + print( + """ + The non-linear search has begun running. + This Jupyter notebook cell with progress once search has completed, this could take a few minutes! + """ + ) + + result = search.fit(model=model, analysis=analysis, info=info) + +print("Search has finished run - you may now continue the notebook.") +``` + +## Files + +By outputting results to hard-disk, a `files` folder is created containing .json / .csv files of the model, +samples, search, etc, for each fit. + +You should check it out now for the completed fits on your hard-disk. + +A description of all files is as follows: + +> - `model`: The `model` defined above and used in the model-fit (`model.json`). +> - `search`: The non-linear search settings (`search.json`). +> - `samples`: The non-linear search samples (`samples.csv`). +> - `samples_info`: Additional information about the samples (`samples_info.json`). +> - `samples_summary`: A summary of key results of the samples (`samples_summary.json`). +> - `info`: The info dictionary passed to the search (`info.json`). +> - `covariance`: The inferred covariance matrix (`covariance.csv`). +> - `data`: The 1D noisy data used that is fitted (`data.json`). +> - `noise_map`: The 1D noise-map fitted (`noise_map.json`). + +The `samples` and `samples_summary` results contain a lot of repeated information. The `samples` result contains +the full non-linear search samples, for example every parameter sample and its log likelihood. The `samples_summary` +contains a summary of the results, for example the maximum log likelihood model and error estimates on parameters +at 1 and 3 sigma confidence. + +Accessing results via the `samples_summary` is much faster, because as it does not reperform calculations using the full +list of samples. Therefore, if the result you want is accessible via the `samples_summary` you should use it +but if not you can revert to the samples. + +## Loading From Hard-Disk + +The multi-fits above wrote the results to hard-disk in three distinct folders, one for each dataset. + +Their results are loaded using the `Aggregator` object, which finds the results in the output directory and can +load them into Python objects. + +```python +from autofit.aggregator.aggregator import Aggregator + +agg = Aggregator.from_directory( + directory=path.join("multi_result_example"), +) +``` + +## Generators + +Before using the aggregator to inspect results, lets discuss Python generators. + +A generator is an object that iterates over a function when it is called. The aggregator creates all of the objects +that it loads from the database as generators (as opposed to a list, or dictionary, or another Python type). + +This is because generators are memory efficient, as they do not store the entries of the database in memory +simultaneously. This contrasts objects like lists and dictionaries, which store all entries in memory all at once. +If you fit a large number of datasets, lists and dictionaries will use a lot of memory and could crash your computer! + +Once we use a generator in the Python code, it cannot be used again. To perform the same task twice, the +generator must be remade it. This cookbook therefore rarely stores generators as variables and instead uses the +aggregator to create each generator at the point of use. + +To create a generator of a specific set of results, we use the `values` method. This takes the `name` of the +object we want to create a generator of, for example inputting `name=samples` will return the results `Samples` +object. + +## Loading Samples + +```python +samples_gen = agg.values("samples") +``` + +By converting this generator to a list and printing it, it is a list of 3 `SamplesNest` objects, corresponding to +the 3 model-fits performed above. + +```python +print("Samples:\n") +print(samples_gen) +print("Total Samples Objects = ", len(agg), "\n") +``` + +## Loading Model + +The model used to perform the model fit for each of the 3 datasets can be loaded via the aggregator and printed. + +```python +model_gen = agg.values("model") + +for model in model_gen: + print(model.info) +``` + +## Loading Search + +The non-linear search used to perform the model fit can be loaded via the aggregator and printed. + +```python +search_gen = agg.values("search") + +for search in search_gen: + print(search.info) +``` + +## Loading Samples + +The `Samples` class contains all information on the non-linear search samples, for example the value of every parameter +sampled using the fit or an instance of the maximum likelihood model. + +The `Samples` class is described fully in the results cookbook. + +```python +for samples in agg.values("samples"): + + print("The tenth sample`s third parameter") + print(samples.parameter_lists[9][2], "\n") + + instance = samples.max_log_likelihood() + + print("Max Log Likelihood `Gaussian` Instance:") + print("Centre = ", instance.centre) + print("Normalization = ", instance.normalization) + print("Sigma = ", instance.sigma, "\n") +``` + +## Loading Samples Summary + +The samples summary contains a subset of results access via the `Samples`, for example the maximum likelihood model +and parameter error estimates. + +Using the samples method above can be slow, as the quantities have to be computed from all non-linear search samples +(e.g. computing errors requires that all samples are marginalized over). This information is stored directly in the +samples summary and can therefore be accessed instantly. + +```python +for samples_summary in agg.values("samples_summary"): + + instance = samples_summary.max_log_likelihood() + + print("Max Log Likelihood `Gaussian` Instance:") + print("Centre = ", instance.centre) + print("Normalization = ", instance.normalization) + print("Sigma = ", instance.sigma, "\n") +``` + +## Loading Info + +The info dictionary passed to the search, discussed earlier in this cookbook, is accessible. + +```python +for info in agg.values("info"): + print(info["date_of_observation"]) + print(info["exposure_time"]) +``` + +The API for querying is fairly self explanatory. Through the combination of info based queries, model based +queries and result based queries a user has all the tools they need to fit extremely large datasets with many different +models and load only the results they are interested in for inspection and analysis. + +## Custom Output + +The results accessible via the database (e.g. `model`, `samples`) are those contained in the `files` folder. + +By extending an `Analysis` class with the methods `save_attributes` and `save_results`, +custom files can be written to the `files` folder and become accessible via the database. + +To save the objects in a human readable and loaded .json format, the `data` and `noise_map`, which are natively stored +as 1D numpy arrays, are converted to a suitable dictionary output format. This uses the **PyAutoConf** method +`to_dict`. + +```python +class Analysis(af.Analysis): + def __init__(self, data: np.ndarray, noise_map: np.ndarray): + """ + Standard Analysis class example used throughout PyAutoFit examples. + """ + super().__init__() + + self.data = data + self.noise_map = noise_map + + def log_likelihood_function(self, instance) -> float: + """ + Standard log likelihood function used throughout PyAutoFit examples. + """ + + xvalues = np.arange(self.data.shape[0]) + + model_data = instance.model_data_from(xvalues=xvalues) + + residual_map = self.data - model_data + chi_squared_map = (residual_map / self.noise_map) ** 2.0 + chi_squared = sum(chi_squared_map) + noise_normalization = np.sum(np.log(2 * np.pi * self.noise_map**2.0)) + log_likelihood = -0.5 * (chi_squared + noise_normalization) + + return log_likelihood + + def save_attributes(self, paths: af.DirectoryPaths): + """ + Before the non-linear search begins, this routine saves attributes of the `Analysis` object to the `files` + folder such that they can be loaded after the analysis using PyAutoFit's database and aggregator tools. + + For this analysis, it uses the `AnalysisDataset` object's method to output the following: + + - The dataset's data as a .json file. + - The dataset's noise-map as a .json file. + + These are accessed using the aggregator via `agg.values("data")` and `agg.values("noise_map")`. + + Parameters + ---------- + paths + The paths object which manages all paths, e.g. where the non-linear search outputs are stored, + visualization, and the pickled objects used by the aggregator output by this function. + """ + from autoconf.dictable import to_dict + + paths.save_json(name="data", object_dict=to_dict(self.data)) + paths.save_json(name="noise_map", object_dict=to_dict(self.noise_map)) + + def save_results(self, paths: af.DirectoryPaths, result: af.Result): + """ + At the end of a model-fit, this routine saves attributes of the `Analysis` object to the `files` + folder such that they can be loaded after the analysis using PyAutoFit's database and aggregator tools. + + For this analysis it outputs the following: + + - The maximum log likelihood model data as a .json file. + + This is accessed using the aggregator via `agg.values("model_data")`. + + Parameters + ---------- + paths + The paths object which manages all paths, e.g. where the non-linear search outputs are stored, + visualization and the pickled objects used by the aggregator output by this function. + result + The result of a model fit, including the non-linear search, samples and maximum likelihood model. + """ + xvalues = np.arange(self.data.shape[0]) + + instance = result.max_log_likelihood_instance + + model_data = instance.model_data_from(xvalues=xvalues) + + # The path where model_data.json is saved, e.g. output/dataset_name/unique_id/files/model_data.json + + paths.save_json(name="model_data", object_dict=model_data) +``` + +## Querying Datasets + +The aggregator can query the database, returning only specific fits of interested. + +We can query using the `dataset_name` string we input into the model-fit above, in order to get the results +of a fit to a specific dataset. + +For example, querying using the string `gaussian_x1_1` returns results for only the fit using the +second `Gaussian` dataset. + +```python +unique_tag = agg.search.unique_tag +agg_query = agg.query(unique_tag == "gaussian_x1_1") +``` + +As expected, this list has only 1 `SamplesNest` corresponding to the second dataset. + +```python +print(agg_query.values("samples")) +print("Total Samples Objects via dataset_name Query = ", len(agg_query), "\n") +``` + +If we query using an incorrect dataset name we get no results. + +```python +unique_tag = agg.search.unique_tag +agg_query = agg.query(unique_tag == "incorrect_name") +samples_gen = agg_query.values("samples") +``` + +## Querying Searches + +We can query using the `name` of the non-linear search used to fit the model. + +In this cookbook, all three fits used the same search, named `database_example`. Query based on search name in this +example is therefore somewhat pointless. + +However, querying based on the search name is useful for model-fits which use a range of searches, for example +if different non-linear searches are used multiple times. + +As expected, the query using search name below contains all 3 results. + +```python +name = agg.search.name +agg_query = agg.query(name == "database_example") + +print(agg_query.values("samples")) +print("Total Samples Objects via name Query = ", len(agg_query), "\n") +``` + +## Querying Models + +We can query based on the model fitted. + +For example, we can load all results which fitted a `Gaussian` model-component, which in this simple example is all +3 model-fits. + +Querying via the model is useful for loading results after performing many model-fits with many different model +parameterizations to large (e.g. Bayesian model comparison). + +\[Note: the code `agg.model.gaussian` corresponds to the fact that in the `Collection` above, we named the model +component `gaussian`. If this `Collection` had used a different name the code below would change +correspondingly. Models with multiple model components (e.g., `gaussian` and `exponential`) are therefore also easily +accessed via the database.\] + +```python +gaussian = agg.model.gaussian +agg_query = agg.query(gaussian == af.ex.Gaussian) +print("Total Samples Objects via `Gaussian` model query = ", len(agg_query), "\n") +``` + +## Querying Results + +We can query based on the results of the model-fit. + +Below, we query the database to find all fits where the inferred value of `sigma` for the `Gaussian` is less +than 3.0 (which returns only the first of the three model-fits). + +```python +gaussian = agg.model.gaussian +agg_query = agg.query(gaussian.sigma < 3.0) +print("Total Samples Objects In Query `gaussian.sigma < 3.0` = ", len(agg_query), "\n") +``` + +## Querying with Logic + +Advanced queries can be constructed using logic. + +Below, we combine the two queries above to find all results which fitted a `Gaussian` AND (using the & symbol) +inferred a value of sigma less than 3.0. + +The OR logical clause is also supported via the symbol . + +```python +gaussian = agg.model.gaussian +agg_query = agg.query((gaussian == af.ex.Gaussian) & (gaussian.sigma < 3.0)) +print( + "Total Samples Objects In Query `Gaussian & sigma < 3.0` = ", len(agg_query), "\n" +) +``` + +## Database + +The default behaviour of model-fitting results output is to be written to hard-disc in folders. These are simple to +navigate and manually check. + +For small model-fitting tasks this is sufficient, however it does not scale well when performing many model fits to +large datasets, because manual inspection of results becomes time consuming. + +All results can therefore be output to an sqlite3 () relational database, +meaning that results can be loaded into a Jupyter notebook or Python script for inspection, analysis and interpretation. +This database supports advanced querying, so that specific model-fits (e.g., which fit a certain model or dataset) can +be loaded. + +## Unique Identifiers + +We have discussed how every model-fit is given a unique identifier, which is used to ensure that the results of the +model-fit are output to a separate folder on hard-disk. + +Each unique identifier is also used to define every entry of the database as it is built. Unique identifiers +therefore play the same vital role for the database of ensuring that every set of results written to it are unique. + +## Building From Output Folder + +The fits above wrote the results to hard-disk in folders, not as an .sqlite database file. + +We build the database below, where the `database_name` corresponds to the name of your output folder and is also the +name of the `.sqlite` database file that is created. + +If you are fitting a relatively small number of datasets (e.g. 10-100) having all results written to hard-disk (e.g. +for quick visual inspection) and using the database for sample wide analysis is beneficial. + +We can optionally only include completed model-fits but setting `completed_only=True`. + +If you inspect the `output` folder, you will see a `database.sqlite` file which contains the results. + +```python +database_name = "database" + +agg = af.Aggregator.from_database( + filename=f"{database_name}.sqlite", completed_only=False +) + +agg.add_directory(directory=path.join("output", "cookbooks", database_name)) +``` + +## Writing Directly To Database + +Results can be written directly to the .sqlite database file, skipping output to hard-disk entirely, by creating +a session and passing this to the non-linear search. + +The code below shows how to do this, but it is commented out to avoid rerunning the non-linear searches. + +This is ideal for tasks where model-fits to hundreds or thousands of datasets are performed, as it becomes unfeasible +to inspect the results of all fits on the hard-disk. + +Our recommended workflow is to set up database analysis scripts using ~10 model-fits, and then scaling these up +to large samples by writing directly to the database. + +```python +session = af.db.open_database("database.sqlite") + +search = af.DynestyStatic( + name="multi_result_example", + path_prefix=path.join("cookbooks", "result"), + unique_tag=dataset_name, # This makes the unique identifier use the dataset name + session=session, # This can instruct the search to write to the .sqlite database. + nlive=50, +) +``` + +If you run the above code and inspect the `output` folder, you will see a `database.sqlite` file which contains +the results. + +The API for loading a database and creating an aggregator to query is as follows: + +```python +agg = af.Aggregator.from_database("database.sqlite") +``` + +Once we have the Aggregator, we can use it to query the database and load results as we did before. diff --git a/docs/cookbooks/result.rst b/docs/cookbooks/result.rst deleted file mode 100644 index 7102e759f..000000000 --- a/docs/cookbooks/result.rst +++ /dev/null @@ -1,782 +0,0 @@ -.. _results: - -Results -======= - -A non-linear search fits a model to a dataset, returning a `Result` object that contains a lot of information on the -model-fit. - -This cookbook provides a concise reference to the result API. - -The cookbook then describes how the results of a search can be output to hard-disk and loaded back into Python, -either using the `Aggregator` object or by building an sqlite database of results. Result loading supports -queries, so that only the results of interest are returned. - -The samples of the non-linear search, which are used to estimate quantities the maximum likelihood model and -parameter errors, are described separately in the `samples.py` cookbook. - -**Contents:** - -An overview of the `Result` object's functionality is given in the following sections: - - - **Info**: Print the `info` attribute of the `Result` object to display a summary of the model-fit. - - **Max Log Likelihood Instance**: Getting the maximum likelihood model instance. - - **Samples**: Getting the samples of the non-linear search from a result. - - **Custom Result**: Extending the `Result` object with custom attributes specific to the model-fit. - -The cookbook next describes how results can be output to hard-disk and loaded back into Python via the `Aggregator`: - - - **Output To Hard-Disk**: Output results to hard-disk so they can be inspected and used to restart a crashed search. - - **Files**: The files that are stored in the `files` folder that is created when results are output to hard-disk. - - **Loading From Hard-disk**: Loading results from hard-disk to Python variables via the aggregator. - - **Generators**: Why loading results uses Python generators to ensure memory efficiency. - -The cookbook next gives examples of how to load all the following results from the database: - - - **Samples**: The samples of the non-linear search (e.g. all parameter values, log likelihoods, etc.). - - **Model**: The model fitted by the non-linear search. - - **Search**: The search used to perform the model-fit. - - **Samples Info**: Additional information on the samples. - - **Samples Summary**: A summary of the samples of the non-linear search (e.g. the maximum log likelihood model). - - **Info**: The `info` dictionary passed to the search. - -The output of results to hard-disk is customizeable and described in the following section: - - - **Custom Output**: Extend `Analysis` classes to output additional information which can be loaded via the aggregator. - -Using queries to load specific results is described in the following sections**: - - - **Querying Datasets**: Query based on the name of the dataset. - - **Querying Searches**: Query based on the name of the search. - - **Querying Models**: Query based on the model that is fitted. - - **Querying Results**: Query based on the results of the model-fit. - - **Querying Logic**: Use logic to combine queries to load specific results (e.g. AND, OR, etc.). - -The final section describes how to use results built in an sqlite database file: - - - **Database**: Building a database file from the output folder. - - **Unique Identifiers**: The unique identifier of each model-fit. - - **Writing Directly To Database**: Writing results directly to the database. - - - -Model Fit ---------- - -To illustrate results, we need to perform a model-fit in order to create a ``Result`` object. - -We do this below using the standard API and noisy 1D signal example, which you should be familiar with from other -example scripts. - -Note that the ``Gaussian`` and ``Analysis`` classes come via the ``af.ex`` module, which contains example model components -that are identical to those found throughout the examples. - -.. code-block:: python - - dataset_path = path.join("dataset", "example_1d", "gaussian_x1") - data = af.util.numpy_array_from_json(file_path=path.join(dataset_path, "data.json")) - noise_map = af.util.numpy_array_from_json( - file_path=path.join(dataset_path, "noise_map.json") - ) - - model = af.Model(af.ex.Gaussian) - - analysis = af.ex.Analysis(data=data, noise_map=noise_map) - - search = af.Emcee( - nwalkers=30, - nsteps=1000, - number_of_cores=1, - ) - - result = search.fit(model=model, analysis=analysis) - -Info ----- - -Printing the ``info`` attribute shows the overall result of the model-fit in a human readable format. - -.. code-block:: python - - print(result.info) - -The output appears as follows: - -.. code-block:: bash - - Maximum Log Likelihood -46.68992727 - Maximum Log Posterior -46.64963514 - - model Gaussian (N=3) - - Maximum Log Likelihood Model: - - centre 49.892 - normalization 24.819 - sigma 9.844 - - - Summary (3.0 sigma limits): - - centre 49.89 (49.52, 50.23) - normalization 24.79 (23.96, 25.61) - sigma 9.85 (9.53, 10.21) - - - Summary (1.0 sigma limits): - - centre 49.89 (49.83, 49.96) - normalization 24.79 (24.65, 24.94) - sigma 9.85 (9.78, 9.90) - -The `max_log_likelihood_instance` is the model instance of the maximum log likelihood model, which is the model -that maximizes the likelihood of the data given the model. - -.. code-block:: python - - instance = result.max_log_likelihood_instance - - print("Max Log Likelihood `Gaussian` Instance:") - print("Centre = ", instance.centre) - print("Normalization = ", instance.normalization) - print("Sigma = ", instance.sigma) - -The `Samples` class contains all information on the non-linear search samples, for example the value of every parameter -sampled using the fit or an instance of the maximum likelihood model. - -.. code-block:: python - - samples = result.samples - -The samples are described in detail separately in the `samples.py` cookbook. - -Custom Result -------------- - -The result can be can be customized to include additional information about the model-fit that is specific to your -model-fitting problem. - -For example, for fitting 1D profiles, the `Result` could include the maximum log likelihood model 1D data: - -`print(result.max_log_likelihood_model_data_1d)` - -In other examples, this quantity has been manually computed after the model-fit has completed. - -The custom result API allows us to do this. First, we define a custom `Result` class, which includes the property -`max_log_likelihood_model_data_1d`. - -.. code-block:: python - - class ResultExample(af.Result): - @property - def max_log_likelihood_model_data_1d(self) -> np.ndarray: - """ - Returns the maximum log likelihood model's 1D model data. - - This is an example of how we can pass the `Analysis` class a custom `Result` object and extend this result - object with new properties that are specific to the model-fit we are performing. - """ - xvalues = np.arange(self.analysis.data.shape[0]) - - return self.instance.model_data_from(xvalues=xvalues) - -The custom result has access to the analysis class, meaning that we can use any of its methods or properties to -compute custom result properties. - -To make it so that the `ResultExample` object above is returned by the search we overwrite the `Result` class attribute -of the `Analysis` and define a `make_result` object describing what we want it to contain: - -.. code-block:: python - - class Analysis(af.ex.Analysis): - - """ - This overwrite means the `ResultExample` class is returned after the model-fit. - """ - - Result = ResultExample - - def make_result( - self, - samples_summary: af.SamplesSummary, - paths: af.AbstractPaths, - samples: Optional[af.SamplesPDF] = None, - search_internal: Optional[object] = None, - analysis: Optional[object] = None, - ) -> Result: - """ - Returns the `Result` of the non-linear search after it is completed. - - The result type is defined as a class variable in the `Analysis` class (see top of code under the python code - `class Analysis(af.Analysis)`. - - The result can be manually overwritten by a user to return a user-defined result object, which can be extended - with additional methods and attribute specific to the model-fit. - - This example class does example this, whereby the analysis result has been overwritten with the `ResultExample` - class, which contains a property `max_log_likelihood_model_data_1d` that returns the model data of the - best-fit model. This API means you can customize your result object to include whatever attributes you want - and therefore make a result object specific to your model-fit and model-fitting problem. - - The `Result` object you return can be customized to include: - - - The samples summary, which contains the maximum log likelihood instance and median PDF model. - - - The paths of the search, which are used for loading the samples and search internal below when a search - is resumed. - - - The samples of the non-linear search (e.g. MCMC chains) also stored in `samples.csv`. - - - The non-linear search used for the fit in its internal representation, which is used for resuming a search - and making bespoke visualization using the search's internal results. - - - The analysis used to fit the model (default disabled to save memory, but option may be useful for certain - projects). - - Parameters - ---------- - samples_summary - The summary of the samples of the non-linear search, which include the maximum log likelihood instance and - median PDF model. - paths - An object describing the paths for saving data (e.g. hard-disk directories or entries in sqlite database). - samples - The samples of the non-linear search, for example the chains of an MCMC run. - search_internal - The internal representation of the non-linear search used to perform the model-fit. - analysis - The analysis used to fit the model. - - Returns - ------- - Result - The result of the non-linear search, which is defined as a class variable in the `Analysis` class. - """ - return self.Result( - samples_summary=samples_summary, - paths=paths, - samples=samples, - search_internal=search_internal, - analysis=self, - ) - -Using the `Analysis` class above, the `Result` object returned by the search is now a `ResultExample` object. - -.. code-block:: python - - analysis = af.ex.Analysis(data=data, noise_map=noise_map) - - search = af.Emcee( - nwalkers=30, - nsteps=1000, - ) - - result = search.fit(model=model, analysis=analysis) - - print(result.max_log_likelihood_model_data_1d) - -Output To Hard-Disk -------------------- - -By default, a non-linear search does not output its results to hard-disk and its results can only be inspected -in Python via the `result` object. - -However, the results of any non-linear search can be output to hard-disk by passing the `name` and / or `path_prefix` -attributes, which are used to name files and output the results to a folder on your hard-disk. - -This cookbook now runs the three searches with output to hard-disk enabled, so you can see how the results are output -to hard-disk and to then illustrate how they can be loaded back into Python. - -Note that an `info` dictionary is also passed to the search, which includes the date of the model-fit and the exposure -time of the dataset. This information is stored output to hard-disk and can be loaded to help interpret the results. - -.. code-block:: python - - info = {"date_of_observation": "01-02-18", "exposure_time": 1000.0} - - dataset_name_list = ["gaussian_x1_0", "gaussian_x1_1", "gaussian_x1_2"] - - model = af.Collection(gaussian=af.ex.Gaussian) - - model.gaussian.centre = af.UniformPrior(lower_limit=0.0, upper_limit=100.0) - model.gaussian.normalization = af.LogUniformPrior(lower_limit=1e-2, upper_limit=1e2) - model.gaussian.sigma = af.GaussianPrior( - mean=10.0, sigma=5.0, lower_limit=0.0, upper_limit=np.inf - ) - - for dataset_name in dataset_name_list: - dataset_path = path.join("dataset", "example_1d", dataset_name) - - data = af.util.numpy_array_from_json(file_path=path.join(dataset_path, "data.json")) - noise_map = af.util.numpy_array_from_json( - file_path=path.join(dataset_path, "noise_map.json") - ) - - analysis = af.ex.Analysis(data=data, noise_map=noise_map) - - search = af.DynestyStatic( - name="multi_result_example", - path_prefix=path.join("cookbooks", "result"), - unique_tag=dataset_name, # This makes the unique identifier use the dataset name - nlive=50, - ) - - print( - """ - The non-linear search has begun running. - This Jupyter notebook cell with progress once search has completed, this could take a few minutes! - """ - ) - - result = search.fit(model=model, analysis=analysis, info=info) - - print("Search has finished run - you may now continue the notebook.") - -Files ------ - -By outputting results to hard-disk, a `files` folder is created containing .json / .csv files of the model, -samples, search, etc, for each fit. - -You should check it out now for the completed fits on your hard-disk. - -A description of all files is as follows: - - - `model`: The `model` defined above and used in the model-fit (`model.json`). - - `search`: The non-linear search settings (`search.json`). - - `samples`: The non-linear search samples (`samples.csv`). - - `samples_info`: Additional information about the samples (`samples_info.json`). - - `samples_summary`: A summary of key results of the samples (`samples_summary.json`). - - `info`: The info dictionary passed to the search (`info.json`). - - `covariance`: The inferred covariance matrix (`covariance.csv`). - - `data`: The 1D noisy data used that is fitted (`data.json`). - - `noise_map`: The 1D noise-map fitted (`noise_map.json`). - -The `samples` and `samples_summary` results contain a lot of repeated information. The `samples` result contains -the full non-linear search samples, for example every parameter sample and its log likelihood. The `samples_summary` -contains a summary of the results, for example the maximum log likelihood model and error estimates on parameters -at 1 and 3 sigma confidence. - -Accessing results via the `samples_summary` is much faster, because as it does not reperform calculations using the full -list of samples. Therefore, if the result you want is accessible via the `samples_summary` you should use it -but if not you can revert to the `samples. - -Loading From Hard-Disk ----------------------- - -The multi-fits above wrote the results to hard-disk in three distinct folders, one for each dataset. - -Their results are loaded using the `Aggregator` object, which finds the results in the output directory and can -load them into Python objects. - -.. code-block:: python - - from autofit.aggregator.aggregator import Aggregator - - agg = Aggregator.from_directory( - directory=path.join("multi_result_example"), - ) - - -Generators ----------- - -Before using the aggregator to inspect results, lets discuss Python generators. - -A generator is an object that iterates over a function when it is called. The aggregator creates all of the objects -that it loads from the database as generators (as opposed to a list, or dictionary, or another Python type). - -This is because generators are memory efficient, as they do not store the entries of the database in memory -simultaneously. This contrasts objects like lists and dictionaries, which store all entries in memory all at once. -If you fit a large number of datasets, lists and dictionaries will use a lot of memory and could crash your computer! - -Once we use a generator in the Python code, it cannot be used again. To perform the same task twice, the -generator must be remade it. This cookbook therefore rarely stores generators as variables and instead uses the -aggregator to create each generator at the point of use. - -To create a generator of a specific set of results, we use the ``values`` method. This takes the ``name`` of the -object we want to create a generator of, for example inputting ``name=samples`` will return the results ``Samples`` -object. - -Loading Samples ---------------- - -.. code-block:: python - - samples_gen = agg.values("samples") - -By converting this generator to a list and printing it, it is a list of 3 ``SamplesNest`` objects, corresponding to -the 3 model-fits performed above. - -.. code-block:: python - - print("Samples:\n") - print(samples_gen) - print("Total Samples Objects = ", len(agg), "\n") - -Loading Model -------------- - -The model used to perform the model fit for each of the 3 datasets can be loaded via the aggregator and printed. - -.. code-block:: python - - model_gen = agg.values("model") - - for model in model_gen: - print(model.info) - -Loading Search --------------- - -The non-linear search used to perform the model fit can be loaded via the aggregator and printed. - -.. code-block:: python - - search_gen = agg.values("search") - - for search in search_gen: - print(search.info) - -Loading Samples ---------------- - -The `Samples` class contains all information on the non-linear search samples, for example the value of every parameter -sampled using the fit or an instance of the maximum likelihood model. - -The `Samples` class is described fully in the results cookbook. - -.. code-block:: python - - for samples in agg.values("samples"): - - print("The tenth sample`s third parameter") - print(samples.parameter_lists[9][2], "\n") - - instance = samples.max_log_likelihood() - - print("Max Log Likelihood `Gaussian` Instance:") - print("Centre = ", instance.centre) - print("Normalization = ", instance.normalization) - print("Sigma = ", instance.sigma, "\n") - -Loading Samples Summary ------------------------ - -The samples summary contains a subset of results access via the ``Samples``, for example the maximum likelihood model -and parameter error estimates. - -Using the samples method above can be slow, as the quantities have to be computed from all non-linear search samples -(e.g. computing errors requires that all samples are marginalized over). This information is stored directly in the -samples summary and can therefore be accessed instantly. - -.. code-block:: python - - for samples_summary in agg.values("samples_summary"): - - instance = samples_summary.max_log_likelihood() - - print("Max Log Likelihood `Gaussian` Instance:") - print("Centre = ", instance.centre) - print("Normalization = ", instance.normalization) - print("Sigma = ", instance.sigma, "\n") - -Loading Info ------------- - -The info dictionary passed to the search, discussed earlier in this cookbook, is accessible. - -.. code-block:: python - - for info in agg.values("info"): - print(info["date_of_observation"]) - print(info["exposure_time"]) - -The API for querying is fairly self explanatory. Through the combination of info based queries, model based -queries and result based queries a user has all the tools they need to fit extremely large datasets with many different -models and load only the results they are interested in for inspection and analysis. - -Custom Output -------------- - -The results accessible via the database (e.g. ``model``, ``samples``) are those contained in the ``files`` folder. - -By extending an ``Analysis`` class with the methods ``save_attributes`` and ``save_results``, -custom files can be written to the ``files`` folder and become accessible via the database. - -To save the objects in a human readable and loaded .json format, the `data` and `noise_map`, which are natively stored -as 1D numpy arrays, are converted to a suitable dictionary output format. This uses the **PyAutoConf** method -`to_dict`. - -.. code-block:: python - - - class Analysis(af.Analysis): - def __init__(self, data: np.ndarray, noise_map: np.ndarray): - """ - Standard Analysis class example used throughout PyAutoFit examples. - """ - super().__init__() - - self.data = data - self.noise_map = noise_map - - def log_likelihood_function(self, instance) -> float: - """ - Standard log likelihood function used throughout PyAutoFit examples. - """ - - xvalues = np.arange(self.data.shape[0]) - - model_data = instance.model_data_from(xvalues=xvalues) - - residual_map = self.data - model_data - chi_squared_map = (residual_map / self.noise_map) ** 2.0 - chi_squared = sum(chi_squared_map) - noise_normalization = np.sum(np.log(2 * np.pi * self.noise_map**2.0)) - log_likelihood = -0.5 * (chi_squared + noise_normalization) - - return log_likelihood - - def save_attributes(self, paths: af.DirectoryPaths): - """ - Before the non-linear search begins, this routine saves attributes of the `Analysis` object to the `files` - folder such that they can be loaded after the analysis using PyAutoFit's database and aggregator tools. - - For this analysis, it uses the `AnalysisDataset` object's method to output the following: - - - The dataset's data as a .json file. - - The dataset's noise-map as a .json file. - - These are accessed using the aggregator via `agg.values("data")` and `agg.values("noise_map")`. - - Parameters - ---------- - paths - The paths object which manages all paths, e.g. where the non-linear search outputs are stored, - visualization, and the pickled objects used by the aggregator output by this function. - """ - from autoconf.dictable import to_dict - - paths.save_json(name="data", object_dict=to_dict(self.data)) - paths.save_json(name="noise_map", object_dict=to_dict(self.noise_map)) - - def save_results(self, paths: af.DirectoryPaths, result: af.Result): - """ - At the end of a model-fit, this routine saves attributes of the `Analysis` object to the `files` - folder such that they can be loaded after the analysis using PyAutoFit's database and aggregator tools. - - For this analysis it outputs the following: - - - The maximum log likelihood model data as a .json file. - - This is accessed using the aggregator via `agg.values("model_data")`. - - Parameters - ---------- - paths - The paths object which manages all paths, e.g. where the non-linear search outputs are stored, - visualization and the pickled objects used by the aggregator output by this function. - result - The result of a model fit, including the non-linear search, samples and maximum likelihood model. - """ - xvalues = np.arange(self.data.shape[0]) - - instance = result.max_log_likelihood_instance - - model_data = instance.model_data_from(xvalues=xvalues) - - # The path where model_data.json is saved, e.g. output/dataset_name/unique_id/files/model_data.json - - paths.save_json(name="model_data", object_dict=model_data) - -Querying Datasets ------------------ - -The aggregator can query the database, returning only specific fits of interested. - -We can query using the ``dataset_name`` string we input into the model-fit above, in order to get the results -of a fit to a specific dataset. - -For example, querying using the string ``gaussian_x1_1`` returns results for only the fit using the -second ``Gaussian`` dataset. - -.. code-block:: python - - unique_tag = agg.search.unique_tag - agg_query = agg.query(unique_tag == "gaussian_x1_1") - -As expected, this list has only 1 ``SamplesNest`` corresponding to the second dataset. - -.. code-block:: python - - print(agg_query.values("samples")) - print("Total Samples Objects via dataset_name Query = ", len(agg_query), "\n") - -If we query using an incorrect dataset name we get no results. - -.. code-block:: python - - unique_tag = agg.search.unique_tag - agg_query = agg.query(unique_tag == "incorrect_name") - samples_gen = agg_query.values("samples") - -Querying Searches ------------------ - -We can query using the ``name`` of the non-linear search used to fit the model. - -In this cookbook, all three fits used the same search, named ``database_example``. Query based on search name in this -example is therefore somewhat pointless. - -However, querying based on the search name is useful for model-fits which use a range of searches, for example -if different non-linear searches are used multiple times. - -As expected, the query using search name below contains all 3 results. - -.. code-block:: python - - name = agg.search.name - agg_query = agg.query(name == "database_example") - - print(agg_query.values("samples")) - print("Total Samples Objects via name Query = ", len(agg_query), "\n") - -Querying Models ---------------- - -We can query based on the model fitted. - -For example, we can load all results which fitted a ``Gaussian`` model-component, which in this simple example is all -3 model-fits. - -Querying via the model is useful for loading results after performing many model-fits with many different model -parameterizations to large (e.g. Bayesian model comparison). - -[Note: the code ``agg.model.gaussian`` corresponds to the fact that in the ``Collection`` above, we named the model -component ``gaussian``. If this ``Collection`` had used a different name the code below would change -correspondingly. Models with multiple model components (e.g., ``gaussian`` and ``exponential``) are therefore also easily -accessed via the database.] - -.. code-block:: python - - gaussian = agg.model.gaussian - agg_query = agg.query(gaussian == af.ex.Gaussian) - print("Total Samples Objects via `Gaussian` model query = ", len(agg_query), "\n") - -Querying Results ----------------- - -We can query based on the results of the model-fit. - -Below, we query the database to find all fits where the inferred value of ``sigma`` for the ``Gaussian`` is less -than 3.0 (which returns only the first of the three model-fits). - -.. code-block:: python - - gaussian = agg.model.gaussian - agg_query = agg.query(gaussian.sigma < 3.0) - print("Total Samples Objects In Query `gaussian.sigma < 3.0` = ", len(agg_query), "\n") - -Querying with Logic -------------------- - -Advanced queries can be constructed using logic. - -Below, we combine the two queries above to find all results which fitted a ``Gaussian`` AND (using the & symbol) -inferred a value of sigma less than 3.0. - -The OR logical clause is also supported via the symbol |. - -.. code-block:: python - - gaussian = agg.model.gaussian - agg_query = agg.query((gaussian == af.ex.Gaussian) & (gaussian.sigma < 3.0)) - print( - "Total Samples Objects In Query `Gaussian & sigma < 3.0` = ", len(agg_query), "\n" - ) - -Database --------- - -The default behaviour of model-fitting results output is to be written to hard-disc in folders. These are simple to -navigate and manually check. - -For small model-fitting tasks this is sufficient, however it does not scale well when performing many model fits to -large datasets, because manual inspection of results becomes time consuming. - -All results can therefore be output to an sqlite3 (https://docs.python.org/3/library/sqlite3.html) relational database, -meaning that results can be loaded into a Jupyter notebook or Python script for inspection, analysis and interpretation. -This database supports advanced querying, so that specific model-fits (e.g., which fit a certain model or dataset) can -be loaded. - -Unique Identifiers ------------------- - -We have discussed how every model-fit is given a unique identifier, which is used to ensure that the results of the -model-fit are output to a separate folder on hard-disk. - -Each unique identifier is also used to define every entry of the database as it is built. Unique identifiers -therefore play the same vital role for the database of ensuring that every set of results written to it are unique. - -Building From Output Folder ---------------------------- - -The fits above wrote the results to hard-disk in folders, not as an .sqlite database file. - -We build the database below, where the `database_name` corresponds to the name of your output folder and is also the -name of the `.sqlite` database file that is created. - -If you are fitting a relatively small number of datasets (e.g. 10-100) having all results written to hard-disk (e.g. -for quick visual inspection) and using the database for sample wide analysis is beneficial. - -We can optionally only include completed model-fits but setting `completed_only=True`. - -If you inspect the `output` folder, you will see a `database.sqlite` file which contains the results. - -.. code-block:: python - - database_name = "database" - - agg = af.Aggregator.from_database( - filename=f"{database_name}.sqlite", completed_only=False - ) - - agg.add_directory(directory=path.join("output", "cookbooks", database_name)) - -Writing Directly To Database ------------------------------ - -Results can be written directly to the .sqlite database file, skipping output to hard-disk entirely, by creating -a session and passing this to the non-linear search. - -The code below shows how to do this, but it is commented out to avoid rerunning the non-linear searches. - -This is ideal for tasks where model-fits to hundreds or thousands of datasets are performed, as it becomes unfeasible -to inspect the results of all fits on the hard-disk. - -Our recommended workflow is to set up database analysis scripts using ~10 model-fits, and then scaling these up -to large samples by writing directly to the database. - -.. code-block:: python - - session = af.db.open_database("database.sqlite") - - search = af.DynestyStatic( - name="multi_result_example", - path_prefix=path.join("cookbooks", "result"), - unique_tag=dataset_name, # This makes the unique identifier use the dataset name - session=session, # This can instruct the search to write to the .sqlite database. - nlive=50, - ) - -If you run the above code and inspect the `output` folder, you will see a `database.sqlite` file which contains -the results. - -The API for loading a database and creating an aggregator to query is as follows: - -.. code-block:: python - - agg = af.Aggregator.from_database("database.sqlite") - -Once we have the Aggregator, we can use it to query the database and load results as we did before. \ No newline at end of file diff --git a/docs/cookbooks/samples.md b/docs/cookbooks/samples.md new file mode 100644 index 000000000..4bb1593b0 --- /dev/null +++ b/docs/cookbooks/samples.md @@ -0,0 +1,718 @@ +(samples)= + +# Samples + +After a non-linear search has completed, it returns a `Result` object that contains information on fit, such as +the maximum likelihood model instance, the errors on each parameter and the Bayesian evidence. + +This cookbook provides an overview of using the results. + +**Contents:** + +- **Model Fit**: Perform a simple model-fit to create a `Samples` object. +- **Samples**: The `Samples` object contained in the `Result`, containing all non-linear samples (e.g. parameters, log likelihoods, etc.). +- **Parameters**: Accessing the parameters of the model from the samples. +- **Figures Of Merit**: The log likelihood, log prior, log posterior and weight of every accepted sample. +- **Instances**: Returning instances of the model corresponding to a particular sample (e.g. the maximum log likelihood). +- **Posterior / PDF**: The median PDF model instance and PDF vectors of all model parameters via 1D marginalization. +- **Errors**: The errors on every parameter estimated from the PDF, computed via marginalized 1D PDFs at an input sigma. +- **Samples Summary**: A summary of the samples of the non-linear search (e.g. the maximum log likelihood model) which can + be faster to load than the full set of samples. +- **Sample Instance**: The model instance of any accepted sample. +- **Search Plots**: Plots of the non-linear search, for example a corner plot or 1D PDF of every parameter. +- **Maximum Likelihood**: The maximum log likelihood model value. +- **Bayesian Evidence**: The log evidence estimated via a nested sampling algorithm. +- **Collection**: Results created from models defined via a `Collection` object. +- **Lists**: Extracting results as Python lists instead of instances. +- **Latex**: Producing latex tables of results (e.g. for a paper). + +The following sections outline how to use advanced features of the results, which you may skip on a first read: + +- **Derived Quantities**: Computing quantities and errors for quantities and parameters not included directly in the model. +- **Result Extension**: Extend the `Result` object with new attributes and methods (e.g. `max_log_likelihood_model_data`). +- **Samples Filtering**: Filter the `Samples` object to only contain samples fulfilling certain criteria. + +## Model Fit + +To get a `Samples` object, we need to perform a model-fit, which you should be familiar after using a non-linear search. + +``` +result = search.fit(model=model, analysis=analysis) +``` + +## Samples + +The result contains a `Samples` object, which contains all samples of the non-linear search. + +Each sample corresponds to a set of model parameters that were evaluated and accepted by the non linear search, +in this example emcee. + +This includes their log likelihoods, which are used for computing additional information about the model-fit, +for example the error on every parameter. + +Our model-fit used the MCMC algorithm Emcee, so the `Samples` object returned is a `SamplesMCMC` object. + +```python +samples = result.samples + +print("MCMC Samples: \n") +print(samples) +``` + +## Parameters + +The parameters are stored as a list of lists, where: + +> - The outer list is the size of the total number of samples. +> - The inner list is the size of the number of free parameters in the fit. + +```python +samples = result.samples + +print("Sample 5's second parameter value (Gaussian -> normalization):") +print(samples.parameter_lists[4][1]) +print("Sample 10's third parameter value (Gaussian -> sigma)") +print(samples.parameter_lists[9][2], "\n") +``` + +The output appears as follows: + +```bash +Sample 5's second parameter value (Gaussian -> normalization): +1.561170345314133 +Sample 10`s third parameter value (Gaussian -> sigma) +12.617071617003607 +``` + +## Figures Of Merit + +The Samples class contains the log likelihood, log prior, log posterior and weight_list of every accepted sample, where: + +- The `log_likelihood` is the value evaluated in the `log_likelihood_function`. +- The `log_prior` encodes information on how parameter priors map log likelihood values to log posterior values. +- The `log_posterior` is `log_likelihood + log_prior`. +- The `weight` gives information on how samples are combined to estimate the posterior, which depends on type of search used (for `Emcee` they are all 1's meaning they are weighted equally). + +Lets inspect the last 10 values of each for the analysis. + +```python +print("log(likelihood), log(prior), log(posterior) and weight of the tenth sample.") +print(samples.log_likelihood_list[9]) +print(samples.log_prior_list[9]) +print(samples.log_posterior_list[9]) +print(samples.weight_list[9]) +``` + +The output appears as follows: + +```bash +log(likelihood), log(prior), log(posterior) and weight of the tenth sample. +-5056.579275235516 +0.743571372185727 +-5055.83570386333 +1.0 +``` + +## Instances + +Using the `Samples` object many results can be returned as an instance of the model, using the Python class structure +of the model composition. + +For example, we can return the model parameters corresponding to the maximum log likelihood sample. + +```python +instance = samples.max_log_likelihood() + +print("Max Log Likelihood Gaussian Instance:") +print("Centre = ", instance.centre) +print("Normalization = ", instance.normalization) +print("Sigma = ", instance.sigma, "\n") +``` + +The output appears as follows: + +```bash +Max Log Likelihood `Gaussian` Instance: +Centre = 49.891590184286855 +Normalization = 24.8187423966329 +Sigma = 9.844319034011903 +``` + +This makes it straight forward to plot the median PDF model: + +```python +model_data = instance.model_data_from(xvalues=np.arange(data.shape[0])) + +plt.plot(range(data.shape[0]), data) +plt.plot(range(data.shape[0]), model_data) +plt.title("Illustrative model fit to 1D Gaussian profile data.") +plt.xlabel("x values of profile") +plt.ylabel("Profile normalization") +plt.show() +plt.close() +``` + +This plot appears as follows: + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/images/toy_model_fit.png +:alt: Alternative text +:width: 600 +``` + +## Posterior / PDF + +The result contains the full posterior information of our non-linear search, which can be used for parameter +estimation. + +The median pdf vector is available, which estimates every parameter via 1D marginalization of their PDFs. + +```python +instance = samples.median_pdf() + +print("Median PDF Gaussian Instance:") +print("Centre = ", instance.centre) +print("Normalization = ", instance.normalization) +print("Sigma = ", instance.sigma, "\n") +``` + +The output appears as follows: + +```bash +Median PDF `Gaussian` Instance: +Centre = 49.88646575581081 +Normalization = 24.786319329440854 +Sigma = 9.845578558662783 +``` + +## Errors + +Methods for computing error estimates on all parameters are provided. + +This again uses 1D marginalization, now at an input sigma confidence limit. + +```python +instance_upper_sigma = samples.errors_at_upper_sigma(sigma=3.0) +instance_lower_sigma = samples.errors_at_lower_sigma(sigma=3.0) + +print("Upper Error values (at 3.0 sigma confidence):") +print("Centre = ", instance_upper_sigma.centre) +print("Normalization = ", instance_upper_sigma.normalization) +print("Sigma = ", instance_upper_sigma.sigma, "\n") + +print("lower Error values (at 3.0 sigma confidence):") +print("Centre = ", instance_lower_sigma.centre) +print("Normalization = ", instance_lower_sigma.normalization) +print("Sigma = ", instance_lower_sigma.sigma, "\n") +``` + +The output appears as follows: + +```bash +Upper Error values (at 3.0 sigma confidence): +Centre = 0.34351559431248546 +Normalization = 0.8210523662181224 +Sigma = 0.36460084790041236 + +lower Error values (at 3.0 sigma confidence): +Centre = 0.36573975189415364 +Normalization = 0.8277555014351385 +Sigma = 0.318978781734252 +``` + +They can also be returned at the values of the parameters at their error values. + +```python +instance_upper_values = samples.values_at_upper_sigma(sigma=3.0) +instance_lower_values = samples.values_at_lower_sigma(sigma=3.0) + +print("Upper Parameter values w/ error (at 3.0 sigma confidence):") +print("Centre = ", instance_upper_values.centre) +print("Normalization = ", instance_upper_values.normalization) +print("Sigma = ", instance_upper_values.sigma, "\n") + +print("lower Parameter values w/ errors (at 3.0 sigma confidence):") +print("Centre = ", instance_lower_values.centre) +print("Normalization = ", instance_lower_values.normalization) +print("Sigma = ", instance_lower_values.sigma, "\n") +``` + +The output appears as follows: + +```bash +Upper Parameter values w/ error (at 3.0 sigma confidence): +Centre = 50.229981350123296 +Normalization = 25.607371695658976 +Sigma = 10.210179406563196 + +lower Parameter values w/ errors (at 3.0 sigma confidence): +Centre = 49.52072600391666 +Normalization = 23.958563828005715 +Sigma = 9.526599776928531 +``` + +## Samples Summary + +The samples summary contains a subset of results access via the `Samples`, for example the maximum likelihood model +and parameter error estimates. + +Using the samples method above can be slow, as the quantities have to be computed from all non-linear search samples +(e.g. computing errors requires that all samples are marginalized over). This information is stored directly in the +samples summary and can therefore be accessed instantly. + +```python +print(samples.summary().max_log_likelihood_sample) +``` + +## Sample Instance + +A non-linear search retains every model that is accepted during the model-fit. + +We can create an instance of any model -- below we create an instance of the last accepted model. + +```python +instance = samples.from_sample_index(sample_index=-1) + +print("Gaussian Instance of last sample") +print("Centre = ", instance.centre) +print("Normalization = ", instance.normalization) +print("Sigma = ", instance.sigma, "\n") +``` + +The output appears as follows: + +```bash +Gaussian Instance of last sample +Centre = 49.81486592598193 +Normalization = 25.342058160043972 +Sigma = 10.001029545296722 +``` + +## Search Plots + +The Probability Density Functions (PDF's) of the results can be plotted using the Emcee's visualization +tool `corner.py`, which is wrapped via the `EmceePlotter` object. + +```python +plotter = aplt.MCMCPlotter(samples=result.samples) +plotter.corner() +``` + +This plot appears as follows: + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/images/corner.png +:alt: Alternative text +:width: 600 +``` + +## Maximum Likelihood + +The maximum log likelihood value of the model-fit can be estimated by simple taking the maximum of all log +likelihoods of the samples. + +If different models are fitted to the same dataset, this value can be compared to determine which model provides +the best fit (e.g. which model has the highest maximum likelihood)? + +```python +print("Maximum Log Likelihood: \n") +print(max(samples.log_likelihood_list)) +``` + +## Bayesian Evidence + +If a nested sampling non-linear search is used, the evidence of the model is also available which enables Bayesian +model comparison to be performed (given we are using Emcee, which is not a nested sampling algorithm, the log evidence +is None).: + +```python +log_evidence = samples.log_evidence +print(f"Log Evidence: {log_evidence}") +``` + +The output appears as follows: + +```bash +Log Evidence: None +``` + +## Collection + +The examples correspond to a model where `af.Model(Gaussian)` was used to compose the model. + +Below, we illustrate how the results API slightly changes if we compose our model using a `Collection`: + +```python +model = af.Collection(gaussian=af.ex.Gaussian, exponential=af.ex.Exponential) + +analysis = af.ex.Analysis(data=data, noise_map=noise_map) + +search = af.Emcee( + nwalkers=50, + nsteps=1000, + number_of_cores=1, +) + +result = search.fit(model=model, analysis=analysis) +``` + +The `result.info` shows the result for the model with both a `Gaussian` and `Exponential` profile. + +```python +print(result.info) +``` + +The output appears as follows: + +```bash +Maximum Log Likelihood -46.19567314 +Maximum Log Posterior 999953.27251548 + +model Collection (N=6) + gaussian Gaussian (N=3) + exponential Exponential (N=3) + +Maximum Log Likelihood Model: + +gaussian + centre 49.914 + normalization 24.635 + sigma 9.851 +exponential + centre 35.911 + normalization 0.010 + rate 5.219 + + +Summary (3.0 sigma limits): + +gaussian + centre 49.84 (44.87, 53.10) + normalization 24.67 (17.87, 38.81) + sigma 9.82 (6.93, 12.98) +exponential + centre 45.03 (1.03, 98.31) + normalization 0.00 (0.00, 0.67) + rate 4.88 (0.07, 9.91) + + +Summary (1.0 sigma limits): + +gaussian + centre 49.84 (49.76, 49.93) + normalization 24.67 (24.46, 24.86) + sigma 9.82 (9.74, 9.90) +exponential + centre 45.03 (36.88, 54.81) + normalization 0.00 (0.00, 0.00) + rate 4.88 (3.73, 5.68) +``` + +Result instances again use the Python classes used to compose the model. + +However, because our fit uses a `Collection` the `instance` has attribues named according to the names given to the +`Collection`, which above were `gaussian` and `exponential`. + +For complex models, with a large number of model components and parameters, this offers a readable API to interpret +the results. + +```python +instance = samples.max_log_likelihood() + +print("Max Log Likelihood Gaussian Instance:") +print("Centre = ", instance.gaussian.centre) +print("Normalization = ", instance.gaussian.normalization) +print("Sigma = ", instance.gaussian.sigma, "\n") + +print("Max Log Likelihood Exponential Instance:") +print("Centre = ", instance.exponential.centre) +print("Normalization = ", instance.exponential.normalization) +print("Sigma = ", instance.exponential.rate, "\n") +``` + +The output appears as follows: + +```bash +Max Log Likelihood `Gaussian` Instance: +Centre = 49.91396277773068 +Normalization = 24.63471453899279 +Sigma = 9.850878941872832 + +Max Log Likelihood Exponential Instance: +Centre = 35.911326828717904 +Normalization = 0.010107001861903789 +Sigma = 5.2192591581876036 +``` + +## Lists + +All results can alternatively be returned as a 1D list of values, by passing `as_instance=False`: + +```python +max_lh_list = samples.max_log_likelihood(as_instance=False) +print("Max Log Likelihood Model Parameters: \n") +print(max_lh_list, "\n\n") +``` + +The output appears as follows: + +```bash +Max Log Likelihood Model Parameters: + +[49.91396277773068, 24.63471453899279, 9.850878941872832, 35.911326828717904, 0.010107001861903789, 5.2192591581876036] +``` + +The list above does not tell us which values correspond to which parameters. + +The following quantities are available in the `Model`, where the order of their entries correspond to the parameters +in the `ml_vector` above: + +- `paths`: a list of tuples which give the path of every parameter in the `Model`. +- `parameter_names`: a list of shorthand parameter names derived from the `paths`. +- `parameter_labels`: a list of parameter labels used when visualizing non-linear search results (see below). + +For simple models like the one fitted in this tutorial, the quantities below are somewhat redundant. For the +more complex models they are important for tracking the parameters of the model. + +```python +model = samples.model + +print(model.paths) +print(model.parameter_names) +print(model.parameter_labels) +print(model.model_component_and_parameter_names) +print("\n") +``` + +The output appears as follows: + +```bash +[('gaussian', 'centre'), ('gaussian', 'normalization'), ('gaussian', 'sigma'), ('exponential', 'centre'), ('exponential', 'normalization'), ('exponential', 'rate')] +['centre', 'normalization', 'sigma', 'centre', 'normalization', 'rate'] +['x', 'norm', '\\sigma', 'x', 'norm', '\\lambda'] +['gaussian_centre', 'gaussian_normalization', 'gaussian_sigma', 'exponential_centre', 'exponential_normalization', 'exponential_rate'] +``` + +All the methods above are available as lists. + +```python +instance = samples.median_pdf(as_instance=False) +values_at_upper_sigma = samples.values_at_upper_sigma(sigma=3.0, as_instance=False) +values_at_lower_sigma = samples.values_at_lower_sigma(sigma=3.0, as_instance=False) +errors_at_upper_sigma = samples.errors_at_upper_sigma(sigma=3.0, as_instance=False) +errors_at_lower_sigma = samples.errors_at_lower_sigma(sigma=3.0, as_instance=False) +``` + +## Latex + +If you are writing modeling results up in a paper, you can use inbuilt latex tools to create latex table +code which you can copy to your .tex document. + +By combining this with the filtering tools below, specific parameters can be included or removed from the latex. + +Remember that the superscripts of a parameter are loaded from the config file `notation/label.yaml`, providing high +levels of customization for how the parameter names appear in the latex table. This is especially useful if your model +uses the same model components with the same parameter, which therefore need to be distinguished via superscripts. + +```python +latex = af.text.Samples.latex( + samples=result.samples, + median_pdf_model=True, + sigma=3.0, + name_to_label=True, + include_name=True, + include_quickmath=True, + prefix="Example Prefix ", + suffix=" \\[-2pt]", +) + +print(latex) +``` + +The output appears as follows: + +```bash +Example Prefix $x^{\rm{g}} = 49.88^{+0.37}_{-0.35}$ & $norm^{\rm{g}} = 24.83^{+0.82}_{-0.76}$ & $\sigma^{\rm{g}} = 9.84^{+0.35}_{-0.40}$ \[-2pt] +``` + +## Derived Quantities (Advanced) + +The parameters `centre`, `normalization` and `sigma` are the model parameters of the `Gaussian`. They are sampled +directly by the non-linear search and we can therefore use the `Samples` object to easily determine their values and +errors. + +Derived quantities (also called latent variables) are those which are not sampled directly by the non-linear search, +but one may still wish to know their values and errors after the fit is complete. For example, what if we want the +error on the full width half maximum (FWHM) of the Gaussian? + +This is achieved by adding them to the `compute_latent_variables` method of the `Analysis` class, which is called +after the non-linear search has completed. The analysis cookbook illustrates how to do this. + +The example analysis used above includes a `compute_latent_variables` method that computes the FWHM of the Gaussian +profile. + +This leads to a number of noteworthy outputs: + +> - A `latent.results` file is output to the results folder, which includes the value and error of all derived quantities +> based on the non-linear search samples (in this example only the `fwhm`). +> - A `latent/samples.csv` is output which lists every accepted sample's value of every derived quantity, which is again +> analogous to the `samples.csv` file (in this example only the `fwhm`). +> - A `latent/samples_summary.json` is output which acts analogously to `samples_summary.json` but for the derived +> quantities of the model (in this example only the `fwhm`). + +Derived quantities are also accessible via the `Samples` object, following a similar API to the model parameters: + +```python +latent = analysis.compute_latent_samples(result.samples) + +instance = latent.max_log_likelihood() + +print(f"Max Likelihood FWHM: {instance.gaussian.fwhm}") + +instance = latent.median_pdf() + +print(f"Median PDF FWHM {instance.gaussian.fwhm}") +``` + +## Derived Errors (Advanced) + +Computing the errors of a quantity like the `sigma` of the Gaussian is simple, because it is sampled by the non-linear +search. Thus, to get their errors above we used the `Samples` object to simply marginalize over all over parameters +via the 1D Probability Density Function (PDF). + +Computing errors on derived quantities is more tricky, because they are not sampled directly by the non-linear search. +For example, what if we want the error on the full width half maximum (FWHM) of the Gaussian? In order to do this +we need to create the PDF of that derived quantity, which we can then marginalize over using the same function we +use to marginalize model parameters. + +Below, we compute the FWHM of every accepted model sampled by the non-linear search and use this determine the PDF +of the FWHM. When combining the FWHM's we weight each value by its `weight`. For Emcee, an MCMC algorithm, the +weight of every sample is 1, but weights may take different values for other non-linear searches. + +In order to pass these samples to the function `marginalize`, which marginalizes over the PDF of the FWHM to compute +its error, we also pass the weight list of the samples. + +(Computing the error on the FWHM could be done in much simpler ways than creating its PDF from the list of every +sample. We chose this example for simplicity, in order to show this functionality, which can easily be extended to more +complicated derived quantities.) + +```python +fwhm_list = [] + +for sample in samples.sample_list: + instance = sample.instance_for_model(model=samples.model) + + sigma = instance.sigma + + fwhm = 2 * np.sqrt(2 * np.log(2)) * sigma + + fwhm_list.append(fwhm) + +median_fwhm, lower_fwhm, upper_fwhm = af.marginalize( + parameter_list=fwhm_list, sigma=3.0, weight_list=samples.weight_list +) + +print(f"FWHM = {median_fwhm} ({upper_fwhm} {lower_fwhm}") +``` + +The output appears as follows: + +```bash +FWHM = 23.065988076921947 (10.249510919377173 54.67455139997644 +``` + +## Samples Filtering (Advanced) + +Our samples object has the results for all three parameters in our model. However, we might only be interested in the +results of a specific parameter. + +The basic form of filtering specifies parameters via their path, which was printed above via the model and is printed +again below. + +```python +samples = result.samples + +print("Parameter paths in the model which are used for filtering:") +print(samples.model.paths) + +print("All parameters of the very first sample") +print(samples.parameter_lists[0]) + +samples = samples.with_paths([("gaussian", "centre")]) + +print("All parameters of the very first sample (containing only the Gaussian centre.") +print(samples.parameter_lists[0]) + +print("Maximum Log Likelihood Model Instances (containing only the Gaussian centre):\n") +print(samples.max_log_likelihood(as_instance=False)) +``` + +The output appears as follows: + +```bash +Parameter paths in the model which are used for filtering: +[('gaussian', 'centre'), ('gaussian', 'normalization'), ('gaussian', 'sigma'), ('exponential', 'centre'), ('exponential', 'normalization'), ('exponential', 'rate')] + +All parameters of the very first sample +[49.63779704398534, 1.1898799260824928, 12.68275074146554, 50.67597072491201, 0.7836791226321858, 5.07432721731388] + +All parameters of the very first sample (containing only the Gaussian centre. +[49.63779704398534] + +Maximum Log Likelihood Model Instances (containing only the Gaussian centre): +[49.880800628266506] +``` + +Above, we specified each path as a list of tuples of strings. + +This is how the source code internally stores the path to different components of the model, but it is not +in-profile_1d with the PyAutoFIT API used to compose a model. + +We can alternatively use the following API: + +```python +samples = result.samples + +samples = samples.with_paths(["gaussian.centre"]) + +print("All parameters of the very first sample (containing only the Gaussian centre).") +print(samples.parameter_lists[0]) +``` + +The output appears as follows: + +```bash +All parameters of the very first sample (containing only the Gaussian centre). +[49.63779704398534] +``` + +Above, we filtered the `Samples` but asking for all parameters which included the path ("gaussian", "centre"). + +We can alternatively filter the `Samples` object by removing all parameters with a certain path. Below, we remove +the Gaussian's `centre` to be left with 2 parameters; the `normalization` and `sigma`. + +```python +samples = result.samples + +print("Parameter paths in the model which are used for filtering:") +print(samples.model.paths) + +print("All parameters of the very first sample") +print(samples.parameter_lists[0]) + +samples = samples.without_paths(["gaussian.centre"]) + +print( + "All parameters of the very first sample (containing only the Gaussian normalization and sigma)." +) +print(samples.parameter_lists[0]) +``` + +The output appears as follows: + +```bash +Parameter paths in the model which are used for filtering: +[('gaussian', 'centre'), ('gaussian', 'normalization'), ('gaussian', 'sigma'), ('exponential', 'centre'), ('exponential', 'normalization'), ('exponential', 'rate')] +All parameters of the very first sample +[49.63779704398534, 1.1898799260824928, 12.68275074146554, 50.67597072491201, 0.7836791226321858, 5.07432721731388] +All parameters of the very first sample (containing only the Gaussian normalization and sigma). +[1.1898799260824928, 12.68275074146554, 50.67597072491201, 0.7836791226321858, 5.07432721731388] +``` diff --git a/docs/cookbooks/samples.rst b/docs/cookbooks/samples.rst deleted file mode 100644 index f29667a52..000000000 --- a/docs/cookbooks/samples.rst +++ /dev/null @@ -1,742 +0,0 @@ -.. _samples: - -Samples -======= - -After a non-linear search has completed, it returns a ``Result`` object that contains information on fit, such as -the maximum likelihood model instance, the errors on each parameter and the Bayesian evidence. - -This cookbook provides an overview of using the results. - -**Contents:** - -- **Model Fit**: Perform a simple model-fit to create a ``Samples`` object. -- **Samples**: The ``Samples`` object contained in the ``Result``, containing all non-linear samples (e.g. parameters, log likelihoods, etc.). -- **Parameters**: Accessing the parameters of the model from the samples. -- **Figures Of Merit**: The log likelihood, log prior, log posterior and weight of every accepted sample. -- **Instances**: Returning instances of the model corresponding to a particular sample (e.g. the maximum log likelihood). -- **Posterior / PDF**: The median PDF model instance and PDF vectors of all model parameters via 1D marginalization. -- **Errors**: The errors on every parameter estimated from the PDF, computed via marginalized 1D PDFs at an input sigma. -- **Samples Summary**: A summary of the samples of the non-linear search (e.g. the maximum log likelihood model) which can - be faster to load than the full set of samples. -- **Sample Instance**: The model instance of any accepted sample. -- **Search Plots**: Plots of the non-linear search, for example a corner plot or 1D PDF of every parameter. -- **Maximum Likelihood**: The maximum log likelihood model value. -- **Bayesian Evidence**: The log evidence estimated via a nested sampling algorithm. -- **Collection**: Results created from models defined via a ``Collection`` object. -- **Lists**: Extracting results as Python lists instead of instances. -- **Latex**: Producing latex tables of results (e.g. for a paper). - -The following sections outline how to use advanced features of the results, which you may skip on a first read: - -- **Derived Quantities**: Computing quantities and errors for quantities and parameters not included directly in the model. -- **Result Extension**: Extend the ``Result`` object with new attributes and methods (e.g. ``max_log_likelihood_model_data``). -- **Samples Filtering**: Filter the ``Samples`` object to only contain samples fulfilling certain criteria. - -Model Fit ---------- - -To get a `Samples` object, we need to perform a model-fit, which you should be familiar after using a non-linear search. - -.. code-block:: - - result = search.fit(model=model, analysis=analysis) - -Samples -------- - -The result contains a ``Samples`` object, which contains all samples of the non-linear search. - -Each sample corresponds to a set of model parameters that were evaluated and accepted by the non linear search, -in this example emcee. - -This includes their log likelihoods, which are used for computing additional information about the model-fit, -for example the error on every parameter. - -Our model-fit used the MCMC algorithm Emcee, so the ``Samples`` object returned is a ``SamplesMCMC`` object. - -.. code-block:: python - - samples = result.samples - - print("MCMC Samples: \n") - print(samples) - -Parameters ----------- - -The parameters are stored as a list of lists, where: - - - The outer list is the size of the total number of samples. - - The inner list is the size of the number of free parameters in the fit. - -.. code-block:: python - - samples = result.samples - - print("Sample 5's second parameter value (Gaussian -> normalization):") - print(samples.parameter_lists[4][1]) - print("Sample 10's third parameter value (Gaussian -> sigma)") - print(samples.parameter_lists[9][2], "\n") - -The output appears as follows: - -.. code-block:: bash - - Sample 5's second parameter value (Gaussian -> normalization): - 1.561170345314133 - Sample 10`s third parameter value (Gaussian -> sigma) - 12.617071617003607 - -Figures Of Merit ----------------- - -The Samples class contains the log likelihood, log prior, log posterior and weight_list of every accepted sample, where: - -- The ``log_likelihood`` is the value evaluated in the ``log_likelihood_function``. - -- The ``log_prior`` encodes information on how parameter priors map log likelihood values to log posterior values. - -- The ``log_posterior`` is ``log_likelihood + log_prior``. - -- The ``weight`` gives information on how samples are combined to estimate the posterior, which depends on type of search used (for ``Emcee`` they are all 1's meaning they are weighted equally). - -Lets inspect the last 10 values of each for the analysis. - -.. code-block:: python - - print("log(likelihood), log(prior), log(posterior) and weight of the tenth sample.") - print(samples.log_likelihood_list[9]) - print(samples.log_prior_list[9]) - print(samples.log_posterior_list[9]) - print(samples.weight_list[9]) - -The output appears as follows: - -.. code-block:: bash - - log(likelihood), log(prior), log(posterior) and weight of the tenth sample. - -5056.579275235516 - 0.743571372185727 - -5055.83570386333 - 1.0 - -Instances ---------- - -Using the ``Samples`` object many results can be returned as an instance of the model, using the Python class structure -of the model composition. - -For example, we can return the model parameters corresponding to the maximum log likelihood sample. - -.. code-block:: python - - instance = samples.max_log_likelihood() - - print("Max Log Likelihood Gaussian Instance:") - print("Centre = ", instance.centre) - print("Normalization = ", instance.normalization) - print("Sigma = ", instance.sigma, "\n") - -The output appears as follows: - -.. code-block:: bash - - Max Log Likelihood `Gaussian` Instance: - Centre = 49.891590184286855 - Normalization = 24.8187423966329 - Sigma = 9.844319034011903 - -This makes it straight forward to plot the median PDF model: - -.. code-block:: python - - model_data = instance.model_data_from(xvalues=np.arange(data.shape[0])) - - plt.plot(range(data.shape[0]), data) - plt.plot(range(data.shape[0]), model_data) - plt.title("Illustrative model fit to 1D Gaussian profile data.") - plt.xlabel("x values of profile") - plt.ylabel("Profile normalization") - plt.show() - plt.close() - -This plot appears as follows: - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/images/toy_model_fit.png - :width: 600 - :alt: Alternative text - - -Posterior / PDF ---------------- - -The result contains the full posterior information of our non-linear search, which can be used for parameter -estimation. - -The median pdf vector is available, which estimates every parameter via 1D marginalization of their PDFs. - -.. code-block:: python - - instance = samples.median_pdf() - - print("Median PDF Gaussian Instance:") - print("Centre = ", instance.centre) - print("Normalization = ", instance.normalization) - print("Sigma = ", instance.sigma, "\n") - -The output appears as follows: - -.. code-block:: bash - - Median PDF `Gaussian` Instance: - Centre = 49.88646575581081 - Normalization = 24.786319329440854 - Sigma = 9.845578558662783 - -Errors ------- - -Methods for computing error estimates on all parameters are provided. - -This again uses 1D marginalization, now at an input sigma confidence limit. - -.. code-block:: python - - instance_upper_sigma = samples.errors_at_upper_sigma(sigma=3.0) - instance_lower_sigma = samples.errors_at_lower_sigma(sigma=3.0) - - print("Upper Error values (at 3.0 sigma confidence):") - print("Centre = ", instance_upper_sigma.centre) - print("Normalization = ", instance_upper_sigma.normalization) - print("Sigma = ", instance_upper_sigma.sigma, "\n") - - print("lower Error values (at 3.0 sigma confidence):") - print("Centre = ", instance_lower_sigma.centre) - print("Normalization = ", instance_lower_sigma.normalization) - print("Sigma = ", instance_lower_sigma.sigma, "\n") - -The output appears as follows: - -.. code-block:: bash - - Upper Error values (at 3.0 sigma confidence): - Centre = 0.34351559431248546 - Normalization = 0.8210523662181224 - Sigma = 0.36460084790041236 - - lower Error values (at 3.0 sigma confidence): - Centre = 0.36573975189415364 - Normalization = 0.8277555014351385 - Sigma = 0.318978781734252 - -They can also be returned at the values of the parameters at their error values. - -.. code-block:: python - - instance_upper_values = samples.values_at_upper_sigma(sigma=3.0) - instance_lower_values = samples.values_at_lower_sigma(sigma=3.0) - - print("Upper Parameter values w/ error (at 3.0 sigma confidence):") - print("Centre = ", instance_upper_values.centre) - print("Normalization = ", instance_upper_values.normalization) - print("Sigma = ", instance_upper_values.sigma, "\n") - - print("lower Parameter values w/ errors (at 3.0 sigma confidence):") - print("Centre = ", instance_lower_values.centre) - print("Normalization = ", instance_lower_values.normalization) - print("Sigma = ", instance_lower_values.sigma, "\n") - -The output appears as follows: - -.. code-block:: bash - - Upper Parameter values w/ error (at 3.0 sigma confidence): - Centre = 50.229981350123296 - Normalization = 25.607371695658976 - Sigma = 10.210179406563196 - - lower Parameter values w/ errors (at 3.0 sigma confidence): - Centre = 49.52072600391666 - Normalization = 23.958563828005715 - Sigma = 9.526599776928531 - -Samples Summary ---------------- - -The samples summary contains a subset of results access via the `Samples`, for example the maximum likelihood model -and parameter error estimates. - -Using the samples method above can be slow, as the quantities have to be computed from all non-linear search samples -(e.g. computing errors requires that all samples are marginalized over). This information is stored directly in the -samples summary and can therefore be accessed instantly. - -.. code-block:: python - - print(samples.summary().max_log_likelihood_sample) - -Sample Instance ---------------- - -A non-linear search retains every model that is accepted during the model-fit. - -We can create an instance of any model -- below we create an instance of the last accepted model. - -.. code-block:: python - - instance = samples.from_sample_index(sample_index=-1) - - print("Gaussian Instance of last sample") - print("Centre = ", instance.centre) - print("Normalization = ", instance.normalization) - print("Sigma = ", instance.sigma, "\n") - -The output appears as follows: - -.. code-block:: bash - - Gaussian Instance of last sample - Centre = 49.81486592598193 - Normalization = 25.342058160043972 - Sigma = 10.001029545296722 - -Search Plots ------------- - -The Probability Density Functions (PDF's) of the results can be plotted using the Emcee's visualization -tool ``corner.py``, which is wrapped via the ``EmceePlotter`` object. - -.. code-block:: python - - plotter = aplt.MCMCPlotter(samples=result.samples) - plotter.corner() - -This plot appears as follows: - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/images/corner.png - :width: 600 - :alt: Alternative text - -Maximum Likelihood ------------------- - -The maximum log likelihood value of the model-fit can be estimated by simple taking the maximum of all log -likelihoods of the samples. - -If different models are fitted to the same dataset, this value can be compared to determine which model provides -the best fit (e.g. which model has the highest maximum likelihood)? - -.. code-block:: python - - print("Maximum Log Likelihood: \n") - print(max(samples.log_likelihood_list)) - -Bayesian Evidence ------------------ - -If a nested sampling non-linear search is used, the evidence of the model is also available which enables Bayesian -model comparison to be performed (given we are using Emcee, which is not a nested sampling algorithm, the log evidence -is None).: - -.. code-block:: python - - log_evidence = samples.log_evidence - print(f"Log Evidence: {log_evidence}") - -The output appears as follows: - -.. code-block:: bash - - Log Evidence: None - -Collection ----------- - -The examples correspond to a model where ``af.Model(Gaussian)`` was used to compose the model. - -Below, we illustrate how the results API slightly changes if we compose our model using a ``Collection``: - -.. code-block:: python - - model = af.Collection(gaussian=af.ex.Gaussian, exponential=af.ex.Exponential) - - analysis = af.ex.Analysis(data=data, noise_map=noise_map) - - search = af.Emcee( - nwalkers=50, - nsteps=1000, - number_of_cores=1, - ) - - result = search.fit(model=model, analysis=analysis) - -The ``result.info`` shows the result for the model with both a ``Gaussian`` and ``Exponential`` profile. - -.. code-block:: python - - print(result.info) - -The output appears as follows: - -.. code-block:: bash - - Maximum Log Likelihood -46.19567314 - Maximum Log Posterior 999953.27251548 - - model Collection (N=6) - gaussian Gaussian (N=3) - exponential Exponential (N=3) - - Maximum Log Likelihood Model: - - gaussian - centre 49.914 - normalization 24.635 - sigma 9.851 - exponential - centre 35.911 - normalization 0.010 - rate 5.219 - - - Summary (3.0 sigma limits): - - gaussian - centre 49.84 (44.87, 53.10) - normalization 24.67 (17.87, 38.81) - sigma 9.82 (6.93, 12.98) - exponential - centre 45.03 (1.03, 98.31) - normalization 0.00 (0.00, 0.67) - rate 4.88 (0.07, 9.91) - - - Summary (1.0 sigma limits): - - gaussian - centre 49.84 (49.76, 49.93) - normalization 24.67 (24.46, 24.86) - sigma 9.82 (9.74, 9.90) - exponential - centre 45.03 (36.88, 54.81) - normalization 0.00 (0.00, 0.00) - rate 4.88 (3.73, 5.68) - -Result instances again use the Python classes used to compose the model. - -However, because our fit uses a ``Collection`` the ``instance`` has attribues named according to the names given to the -``Collection``, which above were ``gaussian`` and ``exponential``. - -For complex models, with a large number of model components and parameters, this offers a readable API to interpret -the results. - -.. code-block:: python - - instance = samples.max_log_likelihood() - - print("Max Log Likelihood Gaussian Instance:") - print("Centre = ", instance.gaussian.centre) - print("Normalization = ", instance.gaussian.normalization) - print("Sigma = ", instance.gaussian.sigma, "\n") - - print("Max Log Likelihood Exponential Instance:") - print("Centre = ", instance.exponential.centre) - print("Normalization = ", instance.exponential.normalization) - print("Sigma = ", instance.exponential.rate, "\n") - -The output appears as follows: - -.. code-block:: bash - - Max Log Likelihood `Gaussian` Instance: - Centre = 49.91396277773068 - Normalization = 24.63471453899279 - Sigma = 9.850878941872832 - - Max Log Likelihood Exponential Instance: - Centre = 35.911326828717904 - Normalization = 0.010107001861903789 - Sigma = 5.2192591581876036 - -Lists ------ - -All results can alternatively be returned as a 1D list of values, by passing ``as_instance=False``: - -.. code-block:: python - - max_lh_list = samples.max_log_likelihood(as_instance=False) - print("Max Log Likelihood Model Parameters: \n") - print(max_lh_list, "\n\n") - -The output appears as follows: - -.. code-block:: bash - - Max Log Likelihood Model Parameters: - - [49.91396277773068, 24.63471453899279, 9.850878941872832, 35.911326828717904, 0.010107001861903789, 5.2192591581876036] - -The list above does not tell us which values correspond to which parameters. - -The following quantities are available in the ``Model``, where the order of their entries correspond to the parameters -in the ``ml_vector`` above: - -- ``paths``: a list of tuples which give the path of every parameter in the ``Model``. -- ``parameter_names``: a list of shorthand parameter names derived from the ``paths``. -- ``parameter_labels``: a list of parameter labels used when visualizing non-linear search results (see below). - -For simple models like the one fitted in this tutorial, the quantities below are somewhat redundant. For the -more complex models they are important for tracking the parameters of the model. - -.. code-block:: python - - model = samples.model - - print(model.paths) - print(model.parameter_names) - print(model.parameter_labels) - print(model.model_component_and_parameter_names) - print("\n") - -The output appears as follows: - -.. code-block:: bash - - [('gaussian', 'centre'), ('gaussian', 'normalization'), ('gaussian', 'sigma'), ('exponential', 'centre'), ('exponential', 'normalization'), ('exponential', 'rate')] - ['centre', 'normalization', 'sigma', 'centre', 'normalization', 'rate'] - ['x', 'norm', '\\sigma', 'x', 'norm', '\\lambda'] - ['gaussian_centre', 'gaussian_normalization', 'gaussian_sigma', 'exponential_centre', 'exponential_normalization', 'exponential_rate'] - -All the methods above are available as lists. - -.. code-block:: python - - instance = samples.median_pdf(as_instance=False) - values_at_upper_sigma = samples.values_at_upper_sigma(sigma=3.0, as_instance=False) - values_at_lower_sigma = samples.values_at_lower_sigma(sigma=3.0, as_instance=False) - errors_at_upper_sigma = samples.errors_at_upper_sigma(sigma=3.0, as_instance=False) - errors_at_lower_sigma = samples.errors_at_lower_sigma(sigma=3.0, as_instance=False) - -Latex ------ - -If you are writing modeling results up in a paper, you can use inbuilt latex tools to create latex table -code which you can copy to your .tex document. - -By combining this with the filtering tools below, specific parameters can be included or removed from the latex. - -Remember that the superscripts of a parameter are loaded from the config file ``notation/label.yaml``, providing high -levels of customization for how the parameter names appear in the latex table. This is especially useful if your model -uses the same model components with the same parameter, which therefore need to be distinguished via superscripts. - -.. code-block:: python - - latex = af.text.Samples.latex( - samples=result.samples, - median_pdf_model=True, - sigma=3.0, - name_to_label=True, - include_name=True, - include_quickmath=True, - prefix="Example Prefix ", - suffix=" \\[-2pt]", - ) - - print(latex) - -The output appears as follows: - -.. code-block:: bash - - Example Prefix $x^{\rm{g}} = 49.88^{+0.37}_{-0.35}$ & $norm^{\rm{g}} = 24.83^{+0.82}_{-0.76}$ & $\sigma^{\rm{g}} = 9.84^{+0.35}_{-0.40}$ \[-2pt] - -Derived Quantities (Advanced) ------------------------------ - -The parameters ``centre``, ``normalization`` and ``sigma`` are the model parameters of the ``Gaussian``. They are sampled -directly by the non-linear search and we can therefore use the ``Samples`` object to easily determine their values and -errors. - -Derived quantities (also called latent variables) are those which are not sampled directly by the non-linear search, -but one may still wish to know their values and errors after the fit is complete. For example, what if we want the -error on the full width half maximum (FWHM) of the Gaussian? - -This is achieved by adding them to the ``compute_latent_variables`` method of the ``Analysis`` class, which is called -after the non-linear search has completed. The analysis cookbook illustrates how to do this. - -The example analysis used above includes a ``compute_latent_variables`` method that computes the FWHM of the Gaussian -profile. - -This leads to a number of noteworthy outputs: - - - A ``latent.results`` file is output to the results folder, which includes the value and error of all derived quantities - based on the non-linear search samples (in this example only the ``fwhm``). - - - A ``latent/samples.csv`` is output which lists every accepted sample's value of every derived quantity, which is again - analogous to the ``samples.csv`` file (in this example only the ``fwhm``). - - - A ``latent/samples_summary.json`` is output which acts analogously to ``samples_summary.json`` but for the derived - quantities of the model (in this example only the ``fwhm``). - -Derived quantities are also accessible via the ``Samples`` object, following a similar API to the model parameters: - -.. code-block:: python - - latent = analysis.compute_latent_samples(result.samples) - - instance = latent.max_log_likelihood() - - print(f"Max Likelihood FWHM: {instance.gaussian.fwhm}") - - instance = latent.median_pdf() - - print(f"Median PDF FWHM {instance.gaussian.fwhm}") - -Derived Errors (Advanced) -------------------------- - -Computing the errors of a quantity like the ``sigma`` of the Gaussian is simple, because it is sampled by the non-linear -search. Thus, to get their errors above we used the ``Samples`` object to simply marginalize over all over parameters -via the 1D Probability Density Function (PDF). - -Computing errors on derived quantities is more tricky, because they are not sampled directly by the non-linear search. -For example, what if we want the error on the full width half maximum (FWHM) of the Gaussian? In order to do this -we need to create the PDF of that derived quantity, which we can then marginalize over using the same function we -use to marginalize model parameters. - -Below, we compute the FWHM of every accepted model sampled by the non-linear search and use this determine the PDF -of the FWHM. When combining the FWHM's we weight each value by its ``weight``. For Emcee, an MCMC algorithm, the -weight of every sample is 1, but weights may take different values for other non-linear searches. - -In order to pass these samples to the function ``marginalize``, which marginalizes over the PDF of the FWHM to compute -its error, we also pass the weight list of the samples. - -(Computing the error on the FWHM could be done in much simpler ways than creating its PDF from the list of every -sample. We chose this example for simplicity, in order to show this functionality, which can easily be extended to more -complicated derived quantities.) - -.. code-block:: python - - fwhm_list = [] - - for sample in samples.sample_list: - instance = sample.instance_for_model(model=samples.model) - - sigma = instance.sigma - - fwhm = 2 * np.sqrt(2 * np.log(2)) * sigma - - fwhm_list.append(fwhm) - - median_fwhm, lower_fwhm, upper_fwhm = af.marginalize( - parameter_list=fwhm_list, sigma=3.0, weight_list=samples.weight_list - ) - - print(f"FWHM = {median_fwhm} ({upper_fwhm} {lower_fwhm}") - -The output appears as follows: - -.. code-block:: bash - - FWHM = 23.065988076921947 (10.249510919377173 54.67455139997644 - -Samples Filtering (Advanced) ----------------------------- - -Our samples object has the results for all three parameters in our model. However, we might only be interested in the -results of a specific parameter. - -The basic form of filtering specifies parameters via their path, which was printed above via the model and is printed -again below. - -.. code-block:: python - - samples = result.samples - - print("Parameter paths in the model which are used for filtering:") - print(samples.model.paths) - - print("All parameters of the very first sample") - print(samples.parameter_lists[0]) - - samples = samples.with_paths([("gaussian", "centre")]) - - print("All parameters of the very first sample (containing only the Gaussian centre.") - print(samples.parameter_lists[0]) - - print("Maximum Log Likelihood Model Instances (containing only the Gaussian centre):\n") - print(samples.max_log_likelihood(as_instance=False)) - -The output appears as follows: - -.. code-block:: bash - - Parameter paths in the model which are used for filtering: - [('gaussian', 'centre'), ('gaussian', 'normalization'), ('gaussian', 'sigma'), ('exponential', 'centre'), ('exponential', 'normalization'), ('exponential', 'rate')] - - All parameters of the very first sample - [49.63779704398534, 1.1898799260824928, 12.68275074146554, 50.67597072491201, 0.7836791226321858, 5.07432721731388] - - All parameters of the very first sample (containing only the Gaussian centre. - [49.63779704398534] - - Maximum Log Likelihood Model Instances (containing only the Gaussian centre): - [49.880800628266506] - -Above, we specified each path as a list of tuples of strings. - -This is how the source code internally stores the path to different components of the model, but it is not -in-profile_1d with the PyAutoFIT API used to compose a model. - -We can alternatively use the following API: - -.. code-block:: python - - samples = result.samples - - samples = samples.with_paths(["gaussian.centre"]) - - print("All parameters of the very first sample (containing only the Gaussian centre).") - print(samples.parameter_lists[0]) - -The output appears as follows: - -.. code-block:: bash - - All parameters of the very first sample (containing only the Gaussian centre). - [49.63779704398534] - -Above, we filtered the ``Samples`` but asking for all parameters which included the path ("gaussian", "centre"). - -We can alternatively filter the ``Samples`` object by removing all parameters with a certain path. Below, we remove -the Gaussian's ``centre`` to be left with 2 parameters; the ``normalization`` and ``sigma``. - -.. code-block:: python - - samples = result.samples - - print("Parameter paths in the model which are used for filtering:") - print(samples.model.paths) - - print("All parameters of the very first sample") - print(samples.parameter_lists[0]) - - samples = samples.without_paths(["gaussian.centre"]) - - print( - "All parameters of the very first sample (containing only the Gaussian normalization and sigma)." - ) - print(samples.parameter_lists[0]) - -The output appears as follows: - -.. code-block:: bash - - Parameter paths in the model which are used for filtering: - [('gaussian', 'centre'), ('gaussian', 'normalization'), ('gaussian', 'sigma'), ('exponential', 'centre'), ('exponential', 'normalization'), ('exponential', 'rate')] - All parameters of the very first sample - [49.63779704398534, 1.1898799260824928, 12.68275074146554, 50.67597072491201, 0.7836791226321858, 5.07432721731388] - All parameters of the very first sample (containing only the Gaussian normalization and sigma). - [1.1898799260824928, 12.68275074146554, 50.67597072491201, 0.7836791226321858, 5.07432721731388] - diff --git a/docs/cookbooks/search.md b/docs/cookbooks/search.md new file mode 100644 index 000000000..493bfdedc --- /dev/null +++ b/docs/cookbooks/search.md @@ -0,0 +1,385 @@ +(search)= + +# Search + +This cookbook provides an overview of the non-linear searches available in **PyAutoFit**, and how to use them. + +**Contents:** + +It first covers standard options available for all non-linear searches: + +- **Example Fit**: A simple example of a non-linear search to remind us how it works. +- **Output To Hard-Disk**: Output results to hard-disk so they can be inspected and used to restart a crashed search. +- **Output Customization**: Customize the output of a non-linear search to hard-disk. +- **Unique Identifier**: Ensure results are output in unique folders, so tthey do not overwrite each other. +- **Iterations Per Update**: Control how often non-linear searches output results to hard-disk. +- **Parallelization**: Use parallel processing to speed up the sampling of parameter space. +- **Plots**: Perform non-linear search specific visualization using their in-built visualization tools. +- **Start Point**: Manually specify the start point of a non-linear search, or sample a specific region of parameter space. + +It then provides example code for using every search: + +- **Emcee (MCMC)**: The Emcee ensemble sampler MCMC. +- **Zeus (MCMC)**: The Zeus ensemble sampler MCMC. +- **DynestyDynamic (Nested Sampling)**: The Dynesty dynamic nested sampler. +- **DynestyStatic (Nested Sampling)**: The Dynesty static nested sampler. +- **LBFGS**: The L-BFGS scipy optimization. + +## Example Fit + +An example of how to use a `search` to fit a model to data is given in other example scripts, but is shown below +for completeness. + +```python +dataset_path = path.join("dataset", "example_1d", "gaussian_x1") +data = af.util.numpy_array_from_json(file_path=path.join(dataset_path, "data.json")) +noise_map = af.util.numpy_array_from_json( + file_path=path.join(dataset_path, "noise_map.json") +) + +model = af.Model(af.ex.Gaussian) + +analysis = af.ex.Analysis(data=data, noise_map=noise_map) +``` + +It is this line, where the command `af.Emcee()` can be swapped out for the examples provided throughout this +cookbook to use different non-linear searches. + +```python +search = af.Emcee() + +result = search.fit(model=model, analysis=analysis) +``` + +## Output To Hard-Disk + +By default, a non-linear search does not output its results to hard-disk and its results can only be inspected +in a Jupyter Notebook or Python script via the `result` object. + +However, the results of any non-linear search can be output to hard-disk by passing the `name` and / or `path_prefix` +attributes, which are used to name files and output the results to a folder on your hard-disk. + +The benefits of doing this include: + +- Inspecting results via folders on your computer is more efficient than using a Jupyter Notebook for multiple datasets. +- Results are output on-the-fly, making it possible to check that a fit is progressing as expected mid way through. +- Additional information about a fit (e.g. visualization) can be output. +- Unfinished runs can be resumed from where they left off if they are terminated. +- On high performance super computers results often must be output in this way. + +The code below shows how to enable outputting of results to hard-disk: + +```python +search = af.Emcee( + path_prefix=path.join("folder_0", "folder_1"), + name="example_mcmc" +) +``` + +These outputs are fully described in the scientific workflow example. + +## Output Customization + +For large model fitting problems outputs may use up a lot of hard-disk space, therefore full customization of the +outputs is supported. + +This is controlled by the `output.yaml` config file found in the `config` folder of the workspace. This file contains +a full description of all customization options. + +A few examples of the options available include: + +- Control over every file which is output to the `files` folder (e.g. `model.json`, `samples.csv`, etc.). +- For the `samples.csv` file, all samples with a weight below a certain value can be automatically removed. +- Customization of the `samples_summary.json` file, which summarises the results of the model-fit (e.g. the maximum + log likelihood model, the median PDF model and 3 sigma error). These results are computed using the full set of + samples, ensuring samples removal via a weight cut does not impact the results. + +In many use cases, the `samples.csv` takes up the significant majority of the hard-disk space, which for large-scale +model-fitting problems can exceed gigabytes and be prohibitive to the analysis. + +Careful customization of the `output.yaml` file enables a workflow where the `samples.csv` file is never output, +but all important information is output in the `samples_summary.json` file using the full samples to compute all +results to high numerical accuracy. + +## Unique Identifier + +Results are output to a folder which is a collection of random characters, which is the 'unique_identifier' of +the model-fit. This identifier is generated based on the model fitted and search used, such that an identical +combination of model and search generates the same identifier. + +This ensures that rerunning an identical fit will use the existing results to resume the model-fit. In contrast, if +you change the model or search, a new unique identifier will be generated, ensuring that the model-fit results are +output into a separate folder. + +A `unique_tag` can be input into a search, which customizes the unique identifier based on the string you provide. +For example, if you are performing many fits to different datasets, using an identical model and search, you may +wish to provide a unique tag for each dataset such that the model-fit results are output into a different folder. + +```python +search = af.Emcee(unique_tag="example_tag") +``` + +## Iterations Per Update + +If results are output to hard-disk, this occurs every `iterations_per_full_update` number of iterations. + +For certain problems, you may want this value to be low, to inspect the results of the model-fit on a regular basis. +This is especially true if the time it takes for your non-linear search to perform an iteration by evaluating the +log likelihood is long (e.g. > 1s) and your model-fit often goes to incorrect solutions that you want to monitor. + +For other problems, you may want to increase this value, to avoid spending lots of time outputting the results to +hard-disk. This is especially true if the time it takes for your non-linear search to perform an iteration by +evaluating the log likelihood is fast (e.g. < 0.1s) and you are confident your model-fit will find the global +maximum solution given enough iterations. + +```python +search = af.Emcee(iterations_per_full_update=1000) +``` + +## Parallelization + +Many searches support parallelization using the Python ``` ``multiprocessing`` ``` module. + +This distributes the non-linear search analysis over multiple CPU's, speeding up the run-time roughly by the number +of CPUs used. + +To enable parallelization, input a `number_of_cores` greater than 1. You should aim not to exceed the number of +physical cores in your computer, as using more cores than exist may actually slow down the non-linear search. + +```python +search = af.Emcee(number_of_cores=4) +``` + +## Plots + +Every non-linear search supported by **PyAutoFit** has a dedicated `plotter` class that allows the results of the +model-fit to be plotted and inspected. + +This uses that search's in-built visualization libraries, which are fully described in the `plot` package of the +workspace. + +For example, `Emcee` has a corresponding `EmceePlotter`, which is used as follows. + +Checkout the `plot` package for a complete description of the plots that can be made for a given search. + +```python +samples = result.samples + +plotter = aplt.MCMCPlotter(samples=samples) + +plotter.corner( + bins=20, + range=None, + color="k", + hist_bin_factor=1, + smooth=None, + smooth1d=None, + label_kwargs=None, + titles=None, + show_titles=False, + title_fmt=".2f", + title_kwargs=None, + truths=None, + truth_color="#4682b4", + scale_hist=False, + quantiles=None, + verbose=False, + fig=None, + max_n_ticks=5, + top_ticks=False, + use_math_text=False, + reverse=False, + labelpad=0.0, + hist_kwargs=None, + group="posterior", + var_names=None, + filter_vars=None, + coords=None, + divergences=False, + divergences_kwargs=None, + labeller=None, +) +``` + +The Python library GetDist \<>\`\`\_ can also be used to create plots of the +results. + +This is described in the `plot` package of the workspace. + +## Start Point + +For maximum likelihood estimator (MLE) and Markov Chain Monte Carlo (MCMC) non-linear searches, parameter space +sampling is built around having a "location" in parameter space. + +This could simply be the parameters of the current maximum likelihood model in an MLE fit, or the locations of many +walkers in parameter space (e.g. MCMC). + +For many model-fitting problems, we may have an expectation of where correct solutions lie in parameter space and +therefore want our non-linear search to start near that location of parameter space. Alternatively, we may want to +sample a specific region of parameter space, to determine what solutions look like there. + +The start-point API allows us to do this, by manually specifying the start-point of an MLE fit or the start-point of +the walkers in an MCMC fit. Because nested sampling draws from priors, it cannot use the start-point API. + +We now define the start point of certain parameters in the model as follows. + +```python +initializer = af.InitializerParamBounds( + { + model.centre: (49.0, 51.0), + model.normalization: (4.0, 6.0), + model.sigma: (1.0, 2.0), + } +) +``` + +Similar behaviour can be achieved by customizing the priors of a model-fit. We could place `GaussianPrior`'s +centred on the regions of parameter space we want to sample, or we could place tight `UniformPrior`'s on regions +of parameter space we believe the correct answer lies. + +The downside of using priors is that our priors have a direct influence on the parameters we infer and the size +of the inferred parameter errors. By using priors to control the location of our model-fit, we therefore risk +inferring a non-representative model. + +For users more familiar with statistical inference, adjusting ones priors in the way described above leads to +changes in the posterior, which therefore impacts the model inferred. + +## Emcee (MCMC) + +The Emcee sampler is a Markov Chain Monte Carlo (MCMC) Ensemble sampler. It is a Python implementation of the +Goodman & Weare \<>\`\`\_ affine-invariant ensemble MCMC sampler. + +Information about Emcee can be found at the following links: + +- +- + +The following workspace example shows examples of fitting data with Emcee and plotting the results. + +- `autofit_workspace/notebooks/searches/mcmc/Emcee.ipynb` +- `autofit_workspace/notebooks/plot/EmceePlotter.ipynb` + +The following code shows how to use Emcee with all available options. + +```python +search = af.Emcee( + nwalkers=30, + nsteps=1000, + initializer=af.InitializerBall(lower_limit=0.49, upper_limit=0.51), + auto_correlation_settings=af.AutoCorrelationsSettings( + check_for_convergence=True, + check_size=100, + required_length=50, + change_threshold=0.01, + ), +) +``` + +## Zeus (MCMC) + +The Zeus sampler is a Markov Chain Monte Carlo (MCMC) Ensemble sampler. + +Information about Zeus can be found at the following links: + +- +- + +```python +search = af.Zeus( + nwalkers=30, + nsteps=1001, + initializer=af.InitializerBall(lower_limit=0.49, upper_limit=0.51), + auto_correlation_settings=af.AutoCorrelationsSettings( + check_for_convergence=True, + check_size=100, + required_length=50, + change_threshold=0.01, + ), + tune=False, + tolerance=0.05, + patience=5, + maxsteps=10000, + mu=1.0, + maxiter=10000, + vectorize=False, + check_walkers=True, + shuffle_ensemble=True, + light_mode=False, +) +``` + +## DynestyDynamic (Nested Sampling) + +The DynestyDynamic sampler is a Dynamic Nested Sampling algorithm. It is a Python implementation of the +Speagle \<>\`\`\_ algorithm. + +Information about Dynesty can be found at the following links: + +- +- + +```python +search = af.DynestyDynamic( + nlive=50, + bound="multi", + sample="auto", + bootstrap=None, + enlarge=None, + update_interval=None, + walks=25, + facc=0.5, + slices=5, + fmove=0.9, + max_move=100, +) +``` + +## DynestyStatic (Nested Sampling) + +The DynestyStatic sampler is a Static Nested Sampling algorithm. It is a Python implementation of the +Speagle \<>\`\`\_ algorithm. + +Information about Dynesty can be found at the following links: + +- +- + +```python +search = af.DynestyStatic( + nlive=50, + bound="multi", + sample="auto", + bootstrap=None, + enlarge=None, + update_interval=None, + walks=25, + facc=0.5, + slices=5, + fmove=0.9, + max_move=100, +) +``` + +## LBFGS + +The LBFGS sampler is a Local Optimization algorithm. It is a Python implementation of the scipy.optimize.lbfgs +algorithm. + +Information about the L-BFGS method can be found at the following links: + +- + +```python +search = af.LBFGS( + tol=None, + disp=None, + maxcor=10, + ftol=2.220446049250313e-09, + gtol=1e-05, + eps=1e-08, + maxfun=15000, + maxiter=15000, + iprint=-1, + maxls=20, +) +``` diff --git a/docs/cookbooks/search.rst b/docs/cookbooks/search.rst deleted file mode 100644 index e557b1f20..000000000 --- a/docs/cookbooks/search.rst +++ /dev/null @@ -1,404 +0,0 @@ -.. _search: - -Search -====== - -This cookbook provides an overview of the non-linear searches available in **PyAutoFit**, and how to use them. - -**Contents:** - -It first covers standard options available for all non-linear searches: - -- **Example Fit**: A simple example of a non-linear search to remind us how it works. -- **Output To Hard-Disk**: Output results to hard-disk so they can be inspected and used to restart a crashed search. -- **Output Customization**: Customize the output of a non-linear search to hard-disk. -- **Unique Identifier**: Ensure results are output in unique folders, so tthey do not overwrite each other. -- **Iterations Per Update**: Control how often non-linear searches output results to hard-disk. -- **Parallelization**: Use parallel processing to speed up the sampling of parameter space. -- **Plots**: Perform non-linear search specific visualization using their in-built visualization tools. -- **Start Point**: Manually specify the start point of a non-linear search, or sample a specific region of parameter space. - -It then provides example code for using every search: - -- **Emcee (MCMC)**: The Emcee ensemble sampler MCMC. -- **Zeus (MCMC)**: The Zeus ensemble sampler MCMC. -- **DynestyDynamic (Nested Sampling)**: The Dynesty dynamic nested sampler. -- **DynestyStatic (Nested Sampling)**: The Dynesty static nested sampler. -- **LBFGS**: The L-BFGS scipy optimization. - -Example Fit ------------ - -An example of how to use a ``search`` to fit a model to data is given in other example scripts, but is shown below -for completeness. - -.. code-block:: python - - dataset_path = path.join("dataset", "example_1d", "gaussian_x1") - data = af.util.numpy_array_from_json(file_path=path.join(dataset_path, "data.json")) - noise_map = af.util.numpy_array_from_json( - file_path=path.join(dataset_path, "noise_map.json") - ) - - model = af.Model(af.ex.Gaussian) - - analysis = af.ex.Analysis(data=data, noise_map=noise_map) - -It is this line, where the command ``af.Emcee()`` can be swapped out for the examples provided throughout this -cookbook to use different non-linear searches. - -.. code-block:: python - - search = af.Emcee() - - result = search.fit(model=model, analysis=analysis) - -Output To Hard-Disk -------------------- - -By default, a non-linear search does not output its results to hard-disk and its results can only be inspected -in a Jupyter Notebook or Python script via the ``result`` object. - -However, the results of any non-linear search can be output to hard-disk by passing the ``name`` and / or ``path_prefix`` -attributes, which are used to name files and output the results to a folder on your hard-disk. - -The benefits of doing this include: - -- Inspecting results via folders on your computer is more efficient than using a Jupyter Notebook for multiple datasets. -- Results are output on-the-fly, making it possible to check that a fit is progressing as expected mid way through. -- Additional information about a fit (e.g. visualization) can be output. -- Unfinished runs can be resumed from where they left off if they are terminated. -- On high performance super computers results often must be output in this way. - -The code below shows how to enable outputting of results to hard-disk: - -.. code-block:: python - - search = af.Emcee( - path_prefix=path.join("folder_0", "folder_1"), - name="example_mcmc" - ) - - -These outputs are fully described in the scientific workflow example. - -Output Customization --------------------- - -For large model fitting problems outputs may use up a lot of hard-disk space, therefore full customization of the -outputs is supported. - -This is controlled by the ``output.yaml`` config file found in the ``config`` folder of the workspace. This file contains -a full description of all customization options. - -A few examples of the options available include: - -- Control over every file which is output to the ``files`` folder (e.g. ``model.json``, ``samples.csv``, etc.). - -- For the ``samples.csv`` file, all samples with a weight below a certain value can be automatically removed. - -- Customization of the ``samples_summary.json`` file, which summarises the results of the model-fit (e.g. the maximum - log likelihood model, the median PDF model and 3 sigma error). These results are computed using the full set of - samples, ensuring samples removal via a weight cut does not impact the results. - -In many use cases, the ``samples.csv`` takes up the significant majority of the hard-disk space, which for large-scale -model-fitting problems can exceed gigabytes and be prohibitive to the analysis. - -Careful customization of the ``output.yaml`` file enables a workflow where the ``samples.csv`` file is never output, -but all important information is output in the ``samples_summary.json`` file using the full samples to compute all -results to high numerical accuracy. - -Unique Identifier ------------------ - -Results are output to a folder which is a collection of random characters, which is the 'unique_identifier' of -the model-fit. This identifier is generated based on the model fitted and search used, such that an identical -combination of model and search generates the same identifier. - -This ensures that rerunning an identical fit will use the existing results to resume the model-fit. In contrast, if -you change the model or search, a new unique identifier will be generated, ensuring that the model-fit results are -output into a separate folder. - -A ``unique_tag`` can be input into a search, which customizes the unique identifier based on the string you provide. -For example, if you are performing many fits to different datasets, using an identical model and search, you may -wish to provide a unique tag for each dataset such that the model-fit results are output into a different folder. - -.. code-block:: python - - search = af.Emcee(unique_tag="example_tag") - -Iterations Per Update ---------------------- - -If results are output to hard-disk, this occurs every ``iterations_per_full_update`` number of iterations. - -For certain problems, you may want this value to be low, to inspect the results of the model-fit on a regular basis. -This is especially true if the time it takes for your non-linear search to perform an iteration by evaluating the -log likelihood is long (e.g. > 1s) and your model-fit often goes to incorrect solutions that you want to monitor. - -For other problems, you may want to increase this value, to avoid spending lots of time outputting the results to -hard-disk. This is especially true if the time it takes for your non-linear search to perform an iteration by -evaluating the log likelihood is fast (e.g. < 0.1s) and you are confident your model-fit will find the global -maximum solution given enough iterations. - -.. code-block:: python - - search = af.Emcee(iterations_per_full_update=1000) - -Parallelization ---------------- - -Many searches support parallelization using the Python ````multiprocessing```` module. - -This distributes the non-linear search analysis over multiple CPU's, speeding up the run-time roughly by the number -of CPUs used. - -To enable parallelization, input a ``number_of_cores`` greater than 1. You should aim not to exceed the number of -physical cores in your computer, as using more cores than exist may actually slow down the non-linear search. - -.. code-block:: python - - search = af.Emcee(number_of_cores=4) - -Plots ------ - -Every non-linear search supported by **PyAutoFit** has a dedicated ``plotter`` class that allows the results of the -model-fit to be plotted and inspected. - -This uses that search's in-built visualization libraries, which are fully described in the ``plot`` package of the -workspace. - -For example, ``Emcee`` has a corresponding ``EmceePlotter``, which is used as follows. - -Checkout the ``plot`` package for a complete description of the plots that can be made for a given search. - -.. code-block:: python - - samples = result.samples - - plotter = aplt.MCMCPlotter(samples=samples) - - plotter.corner( - bins=20, - range=None, - color="k", - hist_bin_factor=1, - smooth=None, - smooth1d=None, - label_kwargs=None, - titles=None, - show_titles=False, - title_fmt=".2f", - title_kwargs=None, - truths=None, - truth_color="#4682b4", - scale_hist=False, - quantiles=None, - verbose=False, - fig=None, - max_n_ticks=5, - top_ticks=False, - use_math_text=False, - reverse=False, - labelpad=0.0, - hist_kwargs=None, - group="posterior", - var_names=None, - filter_vars=None, - coords=None, - divergences=False, - divergences_kwargs=None, - labeller=None, - ) - - -The Python library ``GetDist ``_ can also be used to create plots of the -results. - -This is described in the ``plot`` package of the workspace. - -Start Point ------------ - -For maximum likelihood estimator (MLE) and Markov Chain Monte Carlo (MCMC) non-linear searches, parameter space -sampling is built around having a "location" in parameter space. - -This could simply be the parameters of the current maximum likelihood model in an MLE fit, or the locations of many -walkers in parameter space (e.g. MCMC). - -For many model-fitting problems, we may have an expectation of where correct solutions lie in parameter space and -therefore want our non-linear search to start near that location of parameter space. Alternatively, we may want to -sample a specific region of parameter space, to determine what solutions look like there. - -The start-point API allows us to do this, by manually specifying the start-point of an MLE fit or the start-point of -the walkers in an MCMC fit. Because nested sampling draws from priors, it cannot use the start-point API. - -We now define the start point of certain parameters in the model as follows. - -.. code-block:: python - - initializer = af.InitializerParamBounds( - { - model.centre: (49.0, 51.0), - model.normalization: (4.0, 6.0), - model.sigma: (1.0, 2.0), - } - ) - - -Similar behaviour can be achieved by customizing the priors of a model-fit. We could place ``GaussianPrior``'s -centred on the regions of parameter space we want to sample, or we could place tight ``UniformPrior``'s on regions -of parameter space we believe the correct answer lies. - -The downside of using priors is that our priors have a direct influence on the parameters we infer and the size -of the inferred parameter errors. By using priors to control the location of our model-fit, we therefore risk -inferring a non-representative model. - -For users more familiar with statistical inference, adjusting ones priors in the way described above leads to -changes in the posterior, which therefore impacts the model inferred. - -Emcee (MCMC) ------------- - -The Emcee sampler is a Markov Chain Monte Carlo (MCMC) Ensemble sampler. It is a Python implementation of the -``Goodman & Weare ``_ affine-invariant ensemble MCMC sampler. - -Information about Emcee can be found at the following links: - -- https://github.com/dfm/emcee -- https://emcee.readthedocs.io/en/stable/ - -The following workspace example shows examples of fitting data with Emcee and plotting the results. - -- ``autofit_workspace/notebooks/searches/mcmc/Emcee.ipynb`` -- ``autofit_workspace/notebooks/plot/EmceePlotter.ipynb`` - -The following code shows how to use Emcee with all available options. - -.. code-block:: python - - search = af.Emcee( - nwalkers=30, - nsteps=1000, - initializer=af.InitializerBall(lower_limit=0.49, upper_limit=0.51), - auto_correlation_settings=af.AutoCorrelationsSettings( - check_for_convergence=True, - check_size=100, - required_length=50, - change_threshold=0.01, - ), - ) - -Zeus (MCMC) ------------ - -The Zeus sampler is a Markov Chain Monte Carlo (MCMC) Ensemble sampler. - -Information about Zeus can be found at the following links: - -- https://github.com/minaskar/zeus -- https://zeus-mcmc.readthedocs.io/en/latest/ - -.. code-block:: python - - search = af.Zeus( - nwalkers=30, - nsteps=1001, - initializer=af.InitializerBall(lower_limit=0.49, upper_limit=0.51), - auto_correlation_settings=af.AutoCorrelationsSettings( - check_for_convergence=True, - check_size=100, - required_length=50, - change_threshold=0.01, - ), - tune=False, - tolerance=0.05, - patience=5, - maxsteps=10000, - mu=1.0, - maxiter=10000, - vectorize=False, - check_walkers=True, - shuffle_ensemble=True, - light_mode=False, - ) - -DynestyDynamic (Nested Sampling) --------------------------------- - -The DynestyDynamic sampler is a Dynamic Nested Sampling algorithm. It is a Python implementation of the -``Speagle ``_ algorithm. - -Information about Dynesty can be found at the following links: - -- https://github.com/joshspeagle/dynesty -- https://dynesty.readthedocs.io/en/latest/ - -.. code-block:: python - - search = af.DynestyDynamic( - nlive=50, - bound="multi", - sample="auto", - bootstrap=None, - enlarge=None, - update_interval=None, - walks=25, - facc=0.5, - slices=5, - fmove=0.9, - max_move=100, - ) - -DynestyStatic (Nested Sampling) -------------------------------- - -The DynestyStatic sampler is a Static Nested Sampling algorithm. It is a Python implementation of the -``Speagle ``_ algorithm. - -Information about Dynesty can be found at the following links: - -- https://github.com/joshspeagle/dynesty -- https://dynesty.readthedocs.io/en/latest/ - -.. code-block:: python - - search = af.DynestyStatic( - nlive=50, - bound="multi", - sample="auto", - bootstrap=None, - enlarge=None, - update_interval=None, - walks=25, - facc=0.5, - slices=5, - fmove=0.9, - max_move=100, - ) - -LBFGS ------ - -The LBFGS sampler is a Local Optimization algorithm. It is a Python implementation of the scipy.optimize.lbfgs -algorithm. - -Information about the L-BFGS method can be found at the following links: - -- https://docs.scipy.org/doc/scipy/reference/optimize.minimize-lbfgsb.html - -.. code-block:: python - - search = af.LBFGS( - tol=None, - disp=None, - maxcor=10, - ftol=2.220446049250313e-09, - gtol=1e-05, - eps=1e-08, - maxfun=15000, - maxiter=15000, - iprint=-1, - maxls=20, - ) diff --git a/docs/features/graphical.md b/docs/features/graphical.md new file mode 100644 index 000000000..23f3169e7 --- /dev/null +++ b/docs/features/graphical.md @@ -0,0 +1,194 @@ +(graphical)= + +# Graphical Models + +Throughout most examples, we compose a model and fit it to a single dataset. For simple model-fitting tasks this is +sufficient, however it is common for one to have multiple datasets and a desire to fit them simultaneously with a +unified model. + +This is what graphical models enable. Here, we will show how to build a graphical model that fits multiple datasets +with **PyAutoFit**. + +Using graphical models, **PyAutoFit** can compose and fit models that have 'local' parameters specific to each individual +dataset and higher-level model components that fit 'global' parameters. These higher level parameters will have +conditional dependencies with the local parameters. + +The major selling point of **PyAutoFit**'s graphical modeling framework is the high level of customization it offers, +whereby: + +- Specific `Analysis` classes can be defined for fitting differnent local models to different datasets. +- Each pairing of a local model-fit to data can be given its own non-linear search. +- Graphical model networks of any topology can be defined and fitted. + +In this example, we demonstrate the API for composing and fitting a graphical model to multiple-datasets, using the +simple example of fitting noisy 1D Gaussians. + +We begin by loading noisy 1D data containing 3 Gaussian's. + +```bash +total_gaussians = 3 + +dataset_path = path.join("dataset", "example_1d") + +data_list = [] +noise_map_list = [] + +for dataset_index in range(total_gaussians): + + dataset_name = f"dataset_{dataset_index}" + + dataset_path = path.join( + "dataset", "example_1d", "gaussian_x1__low_snr", dataset_name + ) + + data = af.util.numpy_array_from_json(file_path=path.join(dataset_path, "data.json")) + noise_map = af.util.numpy_array_from_json( + file_path=path.join(dataset_path, "noise_map.json") + ) + + data_list.append(data) + noise_map_list.append(noise_map) +``` + +This is what our three Gaussians look like: + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/features/images/gaussian_x1_1__low_snr.png +:alt: Alternative text +:width: 600 +``` + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/features/images/gaussian_x1_2__low_snr.png +:alt: Alternative text +:width: 600 +``` + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/features/images/gaussian_x1_3__low_snr.png +:alt: Alternative text +:width: 600 +``` + +They are much lower signal-to-noise than the Gaussian's in other examples. Graphical models extract a lot more information +from lower quantity datasets, something we demonstrate explic in the [HowToFit lectures on graphical models](https://github.com/PyAutoLabs/HowToFit/blob/main/notebooks/chapter_3_graphical_models). + +For each dataset we now create a corresponding `Analysis` class. By associating each dataset with an `Analysis` +class we are therefore associating it with a unique `log_likelihood_function`. If our dataset had many different +formats (e.g. images) it would be straight forward to write customized `Analysis` classes for each dataset. + +```bash +analysis_list = [] + +for data, noise_map in zip(data_list, noise_map_list): + + analysis = Analysis(data=data, noise_map=noise_map) + + analysis_list.append(analysis) +``` + +We now compose the graphical model we will fit using the `Model` and `Collection` objects. We begin by setting up a +shared prior for their `centre` using a single `GaussianPrior`. This is passed to a unique `Model` for +each `Gaussian` and means that all three `Gaussian`'s are fitted wih the same value of `centre`. That is, we have +defined our graphical model to have a shared value of `centre` when it fits each dataset. + +```bash +centre_shared_prior = af.GaussianPrior(mean=50.0, sigma=30.0) +``` + +We now set up three `Model` objects, each of which contain a `Gaussian` that is used to fit each of the +datasets we loaded above. Because all three of these `Model`'s use the `centre_shared_prior` the dimensionality of +parameter space is N=7, corresponding to three `Gaussians` with local parameters (`normalization` and `sigma`) and +a global parameter value of `centre`. + +```bash +model_list = [] + +for model_index in range(len(data_list)): + + gaussian = af.Model(p.Gaussian) + + gaussian.centre = centre_shared_prior # This prior is used by all 3 Gaussians! + gaussian.normalization = af.LogUniformPrior(lower_limit=1e-6, upper_limit=1e6) + gaussian.sigma = af.UniformPrior(lower_limit=0.0, upper_limit=25.0) + + model_list.append(gaussian) +``` + +To build our graphical model which fits multiple datasets, we simply pair each model-component to each `Analysis` +class, so that **PyAutoFit** knows that: + +- `gaussian_0` fits `data_0` via `analysis_0`. +- `gaussian_1` fits `data_1` via `analysis_1`. +- `gaussian_2` fits `data_2` via `analysis_2`. + +The point where a `Model` and `Analysis` class meet is called a `AnalysisFactor`. + +This term is used to denote that we are composing a 'factor graph'. A factor defines a node on this graph where we have +some data, a model, and we fit the two together. The 'links' between these different factors then define the global +model we are fitting **and** the datasets used to fit it. + +```bash +analysis_factor_list = [] + +for model, analysis in zip(model_list, analysis_list): + + analysis_factor = g.AnalysisFactor(prior_model=model, analysis=analysis) + + analysis_factor_list.append(analysis_factor) +``` + +We combine our `AnalysisFactor`'s into one, to compose the factor graph. + +```bash +factor_graph = g.FactorGraphModel(*analysis_factor_list) +``` + +So, what does our factor graph looks like? Unfortunately, we haven't yet build visualization of this into **PyAutoFit**, +so you'll have to make do with a description for now. + +The factor graph above is made up of two components: + +- **Nodes**: these are points on the graph where we have a unique set of data and a model that is made up of a subset of + +our overall graphical model. This is effectively the `AnalysisFactor` objects we created above. + +- **Links**: these define the model components and parameters that are shared across different nodes and thus retain the + +same values when fitting different datasets. + +We can now choose a non-linear search and fit the factor graph. + +```bash +search = af.DynestyStatic() + +result = search.fit( + model=factor_graph.global_prior_model, + analysis=factor_graph +) +``` + +This will fit the N=7 dimension parameter space where every Gaussian has a shared centre! + +This is all expanded upon in the [HowToFit chapter on graphical models](https://github.com/PyAutoLabs/HowToFit/blob/main/notebooks/chapter_3_graphical_models), where we will give a +more detailed description of why this approach to model-fitting extracts a lot more information than fitting each +dataset one-by-one. + +## Expectation Propagation + +For large datasets, a graphical model may have hundreds, thousands, or *hundreds of thousands* of parameters. The +high dimensionality of such a parameter space can make it inefficient or impossible to fit the model. + +Fitting high dimensionality graphical models in **PyAutoFit** can use an Expectation Propagation (EP) framework to +make scaling up feasible. This framework fits every dataset individually and pass messages throughout the graph to +inform every fit the expected +values of each parameter. + +The following paper describes the EP framework in formal Bayesian notation: + + + +## Hierarchical Models + +A specific type of graphical model is a hierarchical model, where the shared parameter(s) of a graph are assumed +to be drawn from a common parent distribution. Fitting these datasets simultanoeusly enables better estimate +of this global distribution. + +Hierarchical models can also be scaled up to large datasets via Expectation Propagation. diff --git a/docs/features/graphical.rst b/docs/features/graphical.rst deleted file mode 100644 index 1f96d0910..000000000 --- a/docs/features/graphical.rst +++ /dev/null @@ -1,195 +0,0 @@ -.. _graphical: - -Graphical Models -================ - -Throughout most examples, we compose a model and fit it to a single dataset. For simple model-fitting tasks this is -sufficient, however it is common for one to have multiple datasets and a desire to fit them simultaneously with a -unified model. - -This is what graphical models enable. Here, we will show how to build a graphical model that fits multiple datasets -with **PyAutoFit**. - -Using graphical models, **PyAutoFit** can compose and fit models that have 'local' parameters specific to each individual -dataset and higher-level model components that fit 'global' parameters. These higher level parameters will have -conditional dependencies with the local parameters. - -The major selling point of **PyAutoFit**'s graphical modeling framework is the high level of customization it offers, -whereby: - -- Specific ``Analysis`` classes can be defined for fitting differnent local models to different datasets. -- Each pairing of a local model-fit to data can be given its own non-linear search. -- Graphical model networks of any topology can be defined and fitted. - -In this example, we demonstrate the API for composing and fitting a graphical model to multiple-datasets, using the -simple example of fitting noisy 1D Gaussians. - -We begin by loading noisy 1D data containing 3 Gaussian's. - -.. code-block:: bash - - total_gaussians = 3 - - dataset_path = path.join("dataset", "example_1d") - - data_list = [] - noise_map_list = [] - - for dataset_index in range(total_gaussians): - - dataset_name = f"dataset_{dataset_index}" - - dataset_path = path.join( - "dataset", "example_1d", "gaussian_x1__low_snr", dataset_name - ) - - data = af.util.numpy_array_from_json(file_path=path.join(dataset_path, "data.json")) - noise_map = af.util.numpy_array_from_json( - file_path=path.join(dataset_path, "noise_map.json") - ) - - data_list.append(data) - noise_map_list.append(noise_map) - - -This is what our three Gaussians look like: - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/features/images/gaussian_x1_1__low_snr.png - :width: 600 - :alt: Alternative text - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/features/images/gaussian_x1_2__low_snr.png - :width: 600 - :alt: Alternative text - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/features/images/gaussian_x1_3__low_snr.png - :width: 600 - :alt: Alternative text - -They are much lower signal-to-noise than the Gaussian's in other examples. Graphical models extract a lot more information -from lower quantity datasets, something we demonstrate explic in the `HowToFit lectures on graphical models -`_. - -For each dataset we now create a corresponding ``Analysis`` class. By associating each dataset with an ``Analysis`` -class we are therefore associating it with a unique ``log_likelihood_function``. If our dataset had many different -formats (e.g. images) it would be straight forward to write customized ``Analysis`` classes for each dataset. - -.. code-block:: bash - - analysis_list = [] - - for data, noise_map in zip(data_list, noise_map_list): - - analysis = Analysis(data=data, noise_map=noise_map) - - analysis_list.append(analysis) - -We now compose the graphical model we will fit using the ``Model`` and ``Collection`` objects. We begin by setting up a -shared prior for their ``centre`` using a single ``GaussianPrior``. This is passed to a unique ``Model`` for -each ``Gaussian`` and means that all three ``Gaussian``'s are fitted wih the same value of ``centre``. That is, we have -defined our graphical model to have a shared value of ``centre`` when it fits each dataset. - -.. code-block:: bash - - centre_shared_prior = af.GaussianPrior(mean=50.0, sigma=30.0) - -We now set up three ``Model`` objects, each of which contain a ``Gaussian`` that is used to fit each of the -datasets we loaded above. Because all three of these ``Model``'s use the ``centre_shared_prior`` the dimensionality of -parameter space is N=7, corresponding to three ``Gaussians`` with local parameters (``normalization`` and ``sigma``) and -a global parameter value of ``centre``. - -.. code-block:: bash - - model_list = [] - - for model_index in range(len(data_list)): - - gaussian = af.Model(p.Gaussian) - - gaussian.centre = centre_shared_prior # This prior is used by all 3 Gaussians! - gaussian.normalization = af.LogUniformPrior(lower_limit=1e-6, upper_limit=1e6) - gaussian.sigma = af.UniformPrior(lower_limit=0.0, upper_limit=25.0) - - model_list.append(gaussian) - -To build our graphical model which fits multiple datasets, we simply pair each model-component to each ``Analysis`` -class, so that **PyAutoFit** knows that: - -- ``gaussian_0`` fits ``data_0`` via ``analysis_0``. -- ``gaussian_1`` fits ``data_1`` via ``analysis_1``. -- ``gaussian_2`` fits ``data_2`` via ``analysis_2``. - -The point where a ``Model`` and ``Analysis`` class meet is called a ``AnalysisFactor``. - -This term is used to denote that we are composing a 'factor graph'. A factor defines a node on this graph where we have -some data, a model, and we fit the two together. The 'links' between these different factors then define the global -model we are fitting **and** the datasets used to fit it. - -.. code-block:: bash - - analysis_factor_list = [] - - for model, analysis in zip(model_list, analysis_list): - - analysis_factor = g.AnalysisFactor(prior_model=model, analysis=analysis) - - analysis_factor_list.append(analysis_factor) - -We combine our ``AnalysisFactor``'s into one, to compose the factor graph. - -.. code-block:: bash - - factor_graph = g.FactorGraphModel(*analysis_factor_list) - -So, what does our factor graph looks like? Unfortunately, we haven't yet build visualization of this into **PyAutoFit**, -so you'll have to make do with a description for now. - -The factor graph above is made up of two components: - -- **Nodes**: these are points on the graph where we have a unique set of data and a model that is made up of a subset of -our overall graphical model. This is effectively the `AnalysisFactor` objects we created above. - -- **Links**: these define the model components and parameters that are shared across different nodes and thus retain the -same values when fitting different datasets. - -We can now choose a non-linear search and fit the factor graph. - -.. code-block:: bash - - search = af.DynestyStatic() - - result = search.fit( - model=factor_graph.global_prior_model, - analysis=factor_graph - ) - -This will fit the N=7 dimension parameter space where every Gaussian has a shared centre! - -This is all expanded upon in the `HowToFit chapter on graphical models -`_, where we will give a -more detailed description of why this approach to model-fitting extracts a lot more information than fitting each -dataset one-by-one. - -Expectation Propagation ------------------------ - -For large datasets, a graphical model may have hundreds, thousands, or *hundreds of thousands* of parameters. The -high dimensionality of such a parameter space can make it inefficient or impossible to fit the model. - -Fitting high dimensionality graphical models in **PyAutoFit** can use an Expectation Propagation (EP) framework to -make scaling up feasible. This framework fits every dataset individually and pass messages throughout the graph to -inform every fit the expected -values of each parameter. - -The following paper describes the EP framework in formal Bayesian notation: - -https://arxiv.org/pdf/1412.4869.pdf - -Hierarchical Models -------------------- - -A specific type of graphical model is a hierarchical model, where the shared parameter(s) of a graph are assumed -to be drawn from a common parent distribution. Fitting these datasets simultanoeusly enables better estimate -of this global distribution. - -Hierarchical models can also be scaled up to large datasets via Expectation Propagation. \ No newline at end of file diff --git a/docs/features/interpolate.md b/docs/features/interpolate.md new file mode 100644 index 000000000..d84526d13 --- /dev/null +++ b/docs/features/interpolate.md @@ -0,0 +1,212 @@ +(interpolate)= + +# Model Interpolation + +It is common to fit a model to many similar datasets, where it is anticipated that one or more model parameters vary +smoothly across the datasets. + +For example, the datasets may be taken at different times, where the signal in the data and therefore model parameters +vary smoothly as a function of time. Alternatively, the datasets may be taken at different wavelengths, with the signal +varying smoothly as a function of wavelength. + +In any of these cases, it may be desireable to fit the datasets one-by-one and then interpolate the results in order +to determine the most likely model parameters at any point in time (or at any wavelength). + +This example illustrates model interpolation functionality in **PyAutoFit** using the example of fitting 3 noisy +1D Gaussians, where these data are assumed to have been taken at 3 different times. The `centre` of each `Gaussian` +varies smoothly over time. The interpolation is therefore used to estimate the `centre` of each `Gaussian` at any time +outside of the times the data were observed. + +## Data + +We illustrate model interpolation using 3 noisy 1D Gaussian datasets taken at 3 different times, where the `centre` of +each `Gaussian` varies smoothly over time. + +The datasets are taken at 3 times, t=0, t=1 and t=2. + +```python +total_datasets = 3 + +data_list = [] +noise_map_list = [] +time_list = [] + +for time in range(3): + dataset_name = f"time_{time}" + + dataset_prefix_path = Path("dataset/example_1d/gaussian_x1_variable") + dataset_path = dataset_prefix_path / dataset_name + + data = af.util.numpy_array_from_json(file_path=dataset_path / "data.json") + noise_map = af.util.numpy_array_from_json(file_path=dataset_path / "noise_map.json") + + data_list.append(data) + noise_map_list.append(noise_map) + time_list.append(time) +``` + +Visual comparison of the datasets shows that the `centre` of each `Gaussian` varies smoothly over time, with it moving +from pixel 40 at t=0 to pixel 60 at t=2. + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/features/images/hi.png +:alt: Alternative text +:width: 600 +``` + +## Fit + +We now fit each of the 3 datasets. + +The fits are performed in a for loop, with the docstrings inside the loop explaining the code. + +The interpolate at the end of the fits uses the maximum log likelihood model of each fit, which we store in a list. + +```python +ml_instances_list = [] + +for data, noise_map, time in zip(data_list, noise_map_list, time_list): + + """ + __Analysis__ + + For each dataset we create an ``Analysis`` class, which includes the ``log_likelihood_function`` we fit the data with. + """ + analysis = af.ex.Analysis(data=data, noise_map=noise_map) + + """ + __Time__ + + The model composed below has an input not seen in other examples, the parameter ``time``. + + This is the time that the simulated data was acquired and is not a free parameter in the fit. + + For interpolation it plays a crucial role, as the model is interpolated to the time of every dataset input + into the model below. If the ``time`` input were missing, interpolation could not be performed. + + Over the iterations of the for loop, the ``time`` input will therefore be the values 0.0, 1.0 and 2.0. + + __Model__ + + We now compose our model, which is a single ``Gaussian``. + + The ``centre`` of the ``Gaussian`` is a free parameter with a ``UniformPrior`` that ranges between 0.0 and 100.0. + + We expect the inferred ``centre`` inferred from the fit to each dataset to vary smoothly as a function of time. + """ + model = af.Collection( + gaussian=af.Model(af.ex.Gaussian), + time=time + ) + + """ + __Search__ + + The model is fitted to the data using the nested sampling algorithm + Dynesty (https://dynesty.readthedocs.io/en/latest/). + """ + search = af.DynestyStatic( + path_prefix=path.join("interpolate"), + name=f"time_{time}", + nlive=100, + ) + + """ + __Model-Fit__ + + We can now begin the model-fit by passing the model and analysis object to the search, which performs a non-linear + search to find which models fit the data with the highest likelihood. + """ + result = search.fit(model=model, analysis=analysis) + + """ + __Instances__ + + Interpolation uses the maximum log likelihood model of each fit to build an interpolation model of the model as a + function of time. + + We therefore store the maximum log likelihood model of every fit in a list, which is used below. + """ + ml_instances_list.append(result.instance) +``` + +## Interpolation + +Now all fits are complete, we use the `ml_instances_list` to build an interpolation model of the model as a function +of time. + +This is performed using the `LinearInterpolator` object, which interpolates the model parameters as a function of +time linearly between the values computed by the model-fits above. + +More advanced interpolation schemes are available and described in the `interpolation.py` example. + +```python +interpolator = af.LinearInterpolator(instances=ml_instances_list) +``` + +The model can be interpolated to any time, for example time=1.5. + +This returns a new `instance` of the model, as an instance of the `Gaussian` object, where the parameters are computed +by interpolating between the values computed above. + +```python +instance = interpolator[interpolator.time == 1.5] +``` + +The `centre` of the `Gaussian` at time 1.5 is between the value inferred for the first and second fits taken +at times 1.0 and 2.0. + +This is a `centre` close to a value of 55.0. + +```python +print(f"Gaussian centre of fit 1 (t = 1): {ml_instances_list[0].gaussian.centre}") +print(f"Gaussian centre of fit 2 (t = 2): {ml_instances_list[1].gaussian.centre}") + +print(f"Gaussian centre interpolated at t = 1.5 {instance.gaussian.centre}") +``` + +## Serialisation + +The interpolator and model can be serialized to a .json file using **PyAutoConf**'s dedicated serialization methods. + +This means an interpolator can easily be loaded into other scripts. + +```python +from autoconf.dictable import output_to_json, from_json + +json_file = path.join(dataset_prefix_path, "interpolator.json") + +output_to_json(obj=interpolator, file_path=json_file) + +interpolator = from_json(file_path=json_file) +``` + +## Database + +It may be inconvenient to fit all the models in a single Python script (e.g. the model-fits take a long time and you +are fitting many datasets). + +PyAutoFit's allows you to store the results of model-fits from hard-disk. + +Database functionality then allows you to load the results of the fit above, set up the interpolator and perform the +interpolation. + +If you are not familiar with the database API, you should checkout the `cookbook/database.ipynb` example. + +```python +from autofit.aggregator.aggregator import Aggregator + +agg = Aggregator.from_directory( + directory=path.join("output", "interpolate"), completed_only=False +) + +ml_instances_list = [samps.max_log_likelihood() for samps in agg.values("samples")] + +interpolator = af.LinearInterpolator(instances=ml_instances_list) + +instance = interpolator[interpolator.time == 1.5] + +print(f"Gaussian centre of fit 1 (t = 1): {ml_instances_list[0].gaussian.centre}") +print(f"Gaussian centre of fit 2 (t = 2): {ml_instances_list[1].gaussian.centre}") + +print(f"Gaussian centre interpolated at t = 1.5 {instance.gaussian.centre}") +``` diff --git a/docs/features/interpolate.rst b/docs/features/interpolate.rst deleted file mode 100644 index c412422b7..000000000 --- a/docs/features/interpolate.rst +++ /dev/null @@ -1,217 +0,0 @@ -.. _interpolate: - -Model Interpolation -=================== - -It is common to fit a model to many similar datasets, where it is anticipated that one or more model parameters vary -smoothly across the datasets. - -For example, the datasets may be taken at different times, where the signal in the data and therefore model parameters -vary smoothly as a function of time. Alternatively, the datasets may be taken at different wavelengths, with the signal -varying smoothly as a function of wavelength. - -In any of these cases, it may be desireable to fit the datasets one-by-one and then interpolate the results in order -to determine the most likely model parameters at any point in time (or at any wavelength). - -This example illustrates model interpolation functionality in **PyAutoFit** using the example of fitting 3 noisy -1D Gaussians, where these data are assumed to have been taken at 3 different times. The ``centre`` of each ``Gaussian`` -varies smoothly over time. The interpolation is therefore used to estimate the ``centre`` of each ``Gaussian`` at any time -outside of the times the data were observed. - -Data ----- - -We illustrate model interpolation using 3 noisy 1D Gaussian datasets taken at 3 different times, where the ``centre`` of -each ``Gaussian`` varies smoothly over time. - -The datasets are taken at 3 times, t=0, t=1 and t=2. - -.. code-block:: python - - total_datasets = 3 - - data_list = [] - noise_map_list = [] - time_list = [] - - for time in range(3): - dataset_name = f"time_{time}" - - dataset_prefix_path = Path("dataset/example_1d/gaussian_x1_variable") - dataset_path = dataset_prefix_path / dataset_name - - data = af.util.numpy_array_from_json(file_path=dataset_path / "data.json") - noise_map = af.util.numpy_array_from_json(file_path=dataset_path / "noise_map.json") - - data_list.append(data) - noise_map_list.append(noise_map) - time_list.append(time) - -Visual comparison of the datasets shows that the ``centre`` of each ``Gaussian`` varies smoothly over time, with it moving -from pixel 40 at t=0 to pixel 60 at t=2. - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/features/images/hi.png - :width: 600 - :alt: Alternative text - -Fit ---- - -We now fit each of the 3 datasets. - -The fits are performed in a for loop, with the docstrings inside the loop explaining the code. - -The interpolate at the end of the fits uses the maximum log likelihood model of each fit, which we store in a list. - -.. code-block:: python - - ml_instances_list = [] - - for data, noise_map, time in zip(data_list, noise_map_list, time_list): - - """ - __Analysis__ - - For each dataset we create an ``Analysis`` class, which includes the ``log_likelihood_function`` we fit the data with. - """ - analysis = af.ex.Analysis(data=data, noise_map=noise_map) - - """ - __Time__ - - The model composed below has an input not seen in other examples, the parameter ``time``. - - This is the time that the simulated data was acquired and is not a free parameter in the fit. - - For interpolation it plays a crucial role, as the model is interpolated to the time of every dataset input - into the model below. If the ``time`` input were missing, interpolation could not be performed. - - Over the iterations of the for loop, the ``time`` input will therefore be the values 0.0, 1.0 and 2.0. - - __Model__ - - We now compose our model, which is a single ``Gaussian``. - - The ``centre`` of the ``Gaussian`` is a free parameter with a ``UniformPrior`` that ranges between 0.0 and 100.0. - - We expect the inferred ``centre`` inferred from the fit to each dataset to vary smoothly as a function of time. - """ - model = af.Collection( - gaussian=af.Model(af.ex.Gaussian), - time=time - ) - - """ - __Search__ - - The model is fitted to the data using the nested sampling algorithm - Dynesty (https://dynesty.readthedocs.io/en/latest/). - """ - search = af.DynestyStatic( - path_prefix=path.join("interpolate"), - name=f"time_{time}", - nlive=100, - ) - - """ - __Model-Fit__ - - We can now begin the model-fit by passing the model and analysis object to the search, which performs a non-linear - search to find which models fit the data with the highest likelihood. - """ - result = search.fit(model=model, analysis=analysis) - - """ - __Instances__ - - Interpolation uses the maximum log likelihood model of each fit to build an interpolation model of the model as a - function of time. - - We therefore store the maximum log likelihood model of every fit in a list, which is used below. - """ - ml_instances_list.append(result.instance) - -Interpolation -------------- - -Now all fits are complete, we use the ``ml_instances_list`` to build an interpolation model of the model as a function -of time. - -This is performed using the ``LinearInterpolator`` object, which interpolates the model parameters as a function of -time linearly between the values computed by the model-fits above. - -More advanced interpolation schemes are available and described in the ``interpolation.py`` example. - -.. code-block:: python - - interpolator = af.LinearInterpolator(instances=ml_instances_list) - -The model can be interpolated to any time, for example time=1.5. - -This returns a new ``instance`` of the model, as an instance of the ``Gaussian`` object, where the parameters are computed -by interpolating between the values computed above. - -.. code-block:: python - - instance = interpolator[interpolator.time == 1.5] - -The ``centre`` of the ``Gaussian`` at time 1.5 is between the value inferred for the first and second fits taken -at times 1.0 and 2.0. - -This is a ``centre`` close to a value of 55.0. - -.. code-block:: python - - print(f"Gaussian centre of fit 1 (t = 1): {ml_instances_list[0].gaussian.centre}") - print(f"Gaussian centre of fit 2 (t = 2): {ml_instances_list[1].gaussian.centre}") - - print(f"Gaussian centre interpolated at t = 1.5 {instance.gaussian.centre}") - -Serialisation -------------- - -The interpolator and model can be serialized to a .json file using **PyAutoConf**'s dedicated serialization methods. - -This means an interpolator can easily be loaded into other scripts. - -.. code-block:: python - - from autoconf.dictable import output_to_json, from_json - - json_file = path.join(dataset_prefix_path, "interpolator.json") - - output_to_json(obj=interpolator, file_path=json_file) - - interpolator = from_json(file_path=json_file) - -Database --------- - -It may be inconvenient to fit all the models in a single Python script (e.g. the model-fits take a long time and you -are fitting many datasets). - -PyAutoFit's allows you to store the results of model-fits from hard-disk. - -Database functionality then allows you to load the results of the fit above, set up the interpolator and perform the -interpolation. - -If you are not familiar with the database API, you should checkout the ``cookbook/database.ipynb`` example. - -.. code-block:: python - - from autofit.aggregator.aggregator import Aggregator - - agg = Aggregator.from_directory( - directory=path.join("output", "interpolate"), completed_only=False - ) - - ml_instances_list = [samps.max_log_likelihood() for samps in agg.values("samples")] - - interpolator = af.LinearInterpolator(instances=ml_instances_list) - - instance = interpolator[interpolator.time == 1.5] - - print(f"Gaussian centre of fit 1 (t = 1): {ml_instances_list[0].gaussian.centre}") - print(f"Gaussian centre of fit 2 (t = 2): {ml_instances_list[1].gaussian.centre}") - - print(f"Gaussian centre interpolated at t = 1.5 {instance.gaussian.centre}") \ No newline at end of file diff --git a/docs/features/search_chaining.md b/docs/features/search_chaining.md new file mode 100644 index 000000000..96bfd26d6 --- /dev/null +++ b/docs/features/search_chaining.md @@ -0,0 +1,217 @@ +(search-chaining)= + +# Search Chaining + +To perform a model-fit, we typically compose one model and fit it to our data using one non-linear search. + +Search chaining fits many different models to a dataset using a chained sequence of non-linear searches. Initial +fits are performed using simplified model parameterizations and faster non-linear fitting techniques. The results of +these simplified fits can then be used to initialize fits using a higher dimensionality model with more detailed +non-linear search. + +To fit highly complex models our aim is therefore to **granularize** the fitting procedure into a series +of **bite-sized** searches which are faster and more reliable than fitting the more complex model straight away. + +Our ability to construct chained non-linear searches that perform model fitting more accurately and efficiently relies +on our **domain specific knowledge** of the model fitting task. For example, we may know that our dataset contains +multiple features that can be accurately fitted separately before performing a joint fit, or that certain parameters +share minimal covariance such that allowing us to fix them to certain values in earlier model-fits. + +We may also know tricks that can speed up the fitting of the initial model, for example reducing the size of the data +or changing making a likelihood evaluation faster (most likely at the expense of the quality of the fit itself). By +using chained searches speed-ups can be relaxed towards the end of the model-fitting sequence when we want the most +precise, most accurate model that best fits the dataset available. + +## Data + +In this example we demonstrate search chaining using the example data where there are two `Gaussians` that are visibly +split: + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/features/images/gaussian_x2_split.png +:alt: Alternative text +:width: 600 +``` + +## Approach + +Instead of fitting them simultaneously using a single non-linear search consisting of N=6 parameters, we break +this model-fit into a chained of three searches where: + +1. The first model fits just the left `Gaussian` where N=3. +2. The second model fits just the right `Gaussian` where again N=3. +3. The final model is fitted with both `Gaussians` where N=6. Crucially, the results of the first two searches are used to initialize the search and tell it the highest likelihood regions of parameter space. + +By initially fitting parameter spaces of reduced complexity we can achieve a more efficient and reliable model-fitting +procedure. + +## Search 1 + +To fit the left `Gaussian`, our first `analysis` receive only half data removing the right `Gaussian`. Note that +this give a speed-up in log likelihood evaluation. + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/features/images/gaussian_x2_left.png +:alt: Alternative text +:width: 600 +``` + +We first compose the model, which only represents the left hand `Gaussian`. + +```python +model_1 = af.Collection(gaussian_left=af.ex.Gaussian) +``` + +```python +print(model_1.info) +``` + +The `info` attribute shows the model in a readable format. + +This gives the following output: + +```bash +gaussian_left + centre UniformPrior, lower_limit = 0.0, upper_limit = 100.0 + normalization LogUniformPrior, lower_limit = 1e-06, upper_limit = 1000000.0 + sigma UniformPrior, lower_limit = 0.0, upper_limit = 25.0 +``` + +We now create a search to fit this data. Given the simplicity of the model, we can use a low number of live points +to achieve a fast model-fit (had we fitted the more complex model right away we could not of done this). + +```python +analysis_1 = af.ex.Analysis(data=data[0:50], noise_map=noise_map[0:50]) + +search_1 = af.DynestyStatic( + name="search[1]__left_gaussian", + path_prefix=path.join("features", "search_chaining"), + nlive=30, +) + +result_1 = search_1.fit(model=model_1, analysis=analysis_1) +``` + +By plotting the result we can see we have fitted the left `Gaussian` reasonably well. + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/features/images/gaussian_x2_left_fit.png +:alt: Alternative text +:width: 600 +``` + +## Search 2 + +We now repeat the above process for the right `Gaussian`. + +We could remove the data on the left like we did the `Gaussian` above. However, we are instead going to fit the full +dataset. To fit the left Gaussian we use the maximum log likelihood model of the model inferred in search 1. + +For search chaining, **PyAutoFit** has many convenient methods for passing the results of a search to a subsequence +search. Below, we achieve this by passing the result of the search above as an `instance`. + +```python +model_2 = af.Collection( + gaussian_left=result_1.instance.gaussian_left, gaussian_right=af.ex.Gaussian +) +``` + +The `info` attribute shows the model, including how parameters and priors were passed from `result_1`. + +```python +print(model_2.info) +``` + +This gives the following output: + +```bash +gaussian_left + centre 25.43766022973362 + normalization 51.98717889043411 + sigma 12.99331932996352 +gaussian_right + centre UniformPrior, lower_limit = 0.0, upper_limit = 100.0 + normalization LogUniformPrior, lower_limit = 1e-06, upper_limit = 1000000.0 + sigma UniformPrior, lower_limit = 0.0, upper_limit = 25.0 +``` + +We now run our second Dynesty search to fit the right `Gaussian`. We can again exploit the simplicity of the model +and use a low number of live points to achieve a fast model-fit. + +```python +analysis_2 = af.ex.Analysis(data=data, noise_map=noise_map) + +search_2 = af.DynestyStatic( + name="search[2]__right_gaussian", + path_prefix=path.join("features", "search_chaining"), + nlive=30, +) + +result_2 = search_2.fit(model=model_2, analysis=analysis_2) +``` + +We can now see our model has successfully fitted both Gaussian's: + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/features/images/gaussian_x2_right_fit.png +:alt: Alternative text +:width: 600 +``` + +## Search 3 + +We now fit both `Gaussians`'s simultaneously, using the results of the previous two searches to initialize where +the non-linear searches parameter space. + +To pass the result in this way we use the command `result.model`, which in contrast to `result.instance` above passes +the parameters not as the maximum log likelihood values but as `GaussianPrior`'s that are fitted for by the +non-linear search. + +The `mean` and `sigma` value of each parmeter's `GaussianPrior` are set using the results of searches 1 and +2 to ensure our model-fit only searches the high likelihood regions of parameter space. + +```python +model_3 = af.Collection( + gaussian_left=result_1.model.gaussian_left, + gaussian_right=result_2.model.gaussian_right, +) +``` + +The `info` attribute shows the model, including how parameters and priors were passed from `result_1` and `result_2`. + +```python +print(model_3.info) +``` + +This gives the following output: + +```bash +gaussian_left + centre GaussianPrior, mean = 25.442897208320307, sigma = 20.0 + normalization GaussianPrior, mean = 51.98379634356712, sigma = 25.99189817178356 + sigma GaussianPrior, mean = 12.990448834848394, sigma = 6.495224417424197 +gaussian_right + centre GaussianPrior, mean = 75.052492251368, sigma = 20.0 + normalization GaussianPrior, mean = 48.757265879772476, sigma = 24.378632939886238 + sigma GaussianPrior, mean = 12.167662812557307, sigma = 6.083831406278653 +``` + +```python +analysis_3 = af.ex.Analysis(data=data, noise_map=noise_map) + +search_3 = af.DynestyStatic( + name="search[3]__both_gaussians", + path_prefix=path.join("features", "search_chaining"), + nlive=100, +) + +result_3 = search_3.fit(model=model_3, analysis=analysis_3) +``` + +We can now see our model has successfully fitted both Gaussians simultaneously: + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/features/images/gaussian_x2_fit.png +:alt: Alternative text +:width: 600 +``` + +## Wrap Up + +This fit used a technique called 'prior passing' to pass results from searches 1 and 2 to search 3. Full details of how +prior passing works can be found in the `search_chaining.ipynb` feature notebook. diff --git a/docs/features/search_chaining.rst b/docs/features/search_chaining.rst deleted file mode 100644 index cfb25f303..000000000 --- a/docs/features/search_chaining.rst +++ /dev/null @@ -1,221 +0,0 @@ -.. _search_chaining: - -Search Chaining -=============== - -To perform a model-fit, we typically compose one model and fit it to our data using one non-linear search. - -Search chaining fits many different models to a dataset using a chained sequence of non-linear searches. Initial -fits are performed using simplified model parameterizations and faster non-linear fitting techniques. The results of -these simplified fits can then be used to initialize fits using a higher dimensionality model with more detailed -non-linear search. - -To fit highly complex models our aim is therefore to **granularize** the fitting procedure into a series -of **bite-sized** searches which are faster and more reliable than fitting the more complex model straight away. - -Our ability to construct chained non-linear searches that perform model fitting more accurately and efficiently relies -on our **domain specific knowledge** of the model fitting task. For example, we may know that our dataset contains -multiple features that can be accurately fitted separately before performing a joint fit, or that certain parameters -share minimal covariance such that allowing us to fix them to certain values in earlier model-fits. - -We may also know tricks that can speed up the fitting of the initial model, for example reducing the size of the data -or changing making a likelihood evaluation faster (most likely at the expense of the quality of the fit itself). By -using chained searches speed-ups can be relaxed towards the end of the model-fitting sequence when we want the most -precise, most accurate model that best fits the dataset available. - -Data ----- - -In this example we demonstrate search chaining using the example data where there are two ``Gaussians`` that are visibly -split: - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/features/images/gaussian_x2_split.png - :width: 600 - :alt: Alternative text - -Approach --------- - -Instead of fitting them simultaneously using a single non-linear search consisting of N=6 parameters, we break -this model-fit into a chained of three searches where: - -1) The first model fits just the left ``Gaussian`` where N=3. -2) The second model fits just the right ``Gaussian`` where again N=3. -3) The final model is fitted with both ``Gaussians`` where N=6. Crucially, the results of the first two searches are used to initialize the search and tell it the highest likelihood regions of parameter space. - -By initially fitting parameter spaces of reduced complexity we can achieve a more efficient and reliable model-fitting -procedure. - -Search 1 --------- - -To fit the left ``Gaussian``, our first ``analysis`` receive only half data removing the right ``Gaussian``. Note that -this give a speed-up in log likelihood evaluation. - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/features/images/gaussian_x2_left.png - :width: 600 - :alt: Alternative text - -We first compose the model, which only represents the left hand `Gaussian`. - -.. code-block:: python - - model_1 = af.Collection(gaussian_left=af.ex.Gaussian) - -.. code-block:: python - - print(model_1.info) - -The `info` attribute shows the model in a readable format. - -This gives the following output: - -.. code-block:: bash - - gaussian_left - centre UniformPrior, lower_limit = 0.0, upper_limit = 100.0 - normalization LogUniformPrior, lower_limit = 1e-06, upper_limit = 1000000.0 - sigma UniformPrior, lower_limit = 0.0, upper_limit = 25.0 - -We now create a search to fit this data. Given the simplicity of the model, we can use a low number of live points -to achieve a fast model-fit (had we fitted the more complex model right away we could not of done this). - -.. code-block:: python - - analysis_1 = af.ex.Analysis(data=data[0:50], noise_map=noise_map[0:50]) - - search_1 = af.DynestyStatic( - name="search[1]__left_gaussian", - path_prefix=path.join("features", "search_chaining"), - nlive=30, - ) - - result_1 = search_1.fit(model=model_1, analysis=analysis_1) - -By plotting the result we can see we have fitted the left ``Gaussian`` reasonably well. - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/features/images/gaussian_x2_left_fit.png - :width: 600 - :alt: Alternative text - -Search 2 --------- - -We now repeat the above process for the right ``Gaussian``. - -We could remove the data on the left like we did the ``Gaussian`` above. However, we are instead going to fit the full -dataset. To fit the left Gaussian we use the maximum log likelihood model of the model inferred in search 1. - -For search chaining, **PyAutoFit** has many convenient methods for passing the results of a search to a subsequence -search. Below, we achieve this by passing the result of the search above as an ``instance``. - -.. code-block:: python - - model_2 = af.Collection( - gaussian_left=result_1.instance.gaussian_left, gaussian_right=af.ex.Gaussian - ) - -The `info` attribute shows the model, including how parameters and priors were passed from `result_1`. - -.. code-block:: python - - print(model_2.info) - -This gives the following output: - -.. code-block:: bash - - gaussian_left - centre 25.43766022973362 - normalization 51.98717889043411 - sigma 12.99331932996352 - gaussian_right - centre UniformPrior, lower_limit = 0.0, upper_limit = 100.0 - normalization LogUniformPrior, lower_limit = 1e-06, upper_limit = 1000000.0 - sigma UniformPrior, lower_limit = 0.0, upper_limit = 25.0 - -We now run our second Dynesty search to fit the right ``Gaussian``. We can again exploit the simplicity of the model -and use a low number of live points to achieve a fast model-fit. - -.. code-block:: python - - analysis_2 = af.ex.Analysis(data=data, noise_map=noise_map) - - search_2 = af.DynestyStatic( - name="search[2]__right_gaussian", - path_prefix=path.join("features", "search_chaining"), - nlive=30, - ) - - result_2 = search_2.fit(model=model_2, analysis=analysis_2) - -We can now see our model has successfully fitted both Gaussian's: - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/features/images/gaussian_x2_right_fit.png - :width: 600 - :alt: Alternative text - -Search 3 --------- - -We now fit both ``Gaussians``'s simultaneously, using the results of the previous two searches to initialize where -the non-linear searches parameter space. - -To pass the result in this way we use the command ``result.model``, which in contrast to ``result.instance`` above passes -the parameters not as the maximum log likelihood values but as ``GaussianPrior``'s that are fitted for by the -non-linear search. - -The ``mean`` and ``sigma`` value of each parmeter's ``GaussianPrior`` are set using the results of searches 1 and -2 to ensure our model-fit only searches the high likelihood regions of parameter space. - -.. code-block:: python - - model_3 = af.Collection( - gaussian_left=result_1.model.gaussian_left, - gaussian_right=result_2.model.gaussian_right, - ) - -The `info` attribute shows the model, including how parameters and priors were passed from `result_1` and `result_2`. - -.. code-block:: python - - print(model_3.info) - - -This gives the following output: - -.. code-block:: bash - - gaussian_left - centre GaussianPrior, mean = 25.442897208320307, sigma = 20.0 - normalization GaussianPrior, mean = 51.98379634356712, sigma = 25.99189817178356 - sigma GaussianPrior, mean = 12.990448834848394, sigma = 6.495224417424197 - gaussian_right - centre GaussianPrior, mean = 75.052492251368, sigma = 20.0 - normalization GaussianPrior, mean = 48.757265879772476, sigma = 24.378632939886238 - sigma GaussianPrior, mean = 12.167662812557307, sigma = 6.083831406278653 - - -.. code-block:: python - - analysis_3 = af.ex.Analysis(data=data, noise_map=noise_map) - - search_3 = af.DynestyStatic( - name="search[3]__both_gaussians", - path_prefix=path.join("features", "search_chaining"), - nlive=100, - ) - - result_3 = search_3.fit(model=model_3, analysis=analysis_3) - -We can now see our model has successfully fitted both Gaussians simultaneously: - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/features/images/gaussian_x2_fit.png - :width: 600 - :alt: Alternative text - -Wrap Up -------- - -This fit used a technique called 'prior passing' to pass results from searches 1 and 2 to search 3. Full details of how -prior passing works can be found in the ``search_chaining.ipynb`` feature notebook. \ No newline at end of file diff --git a/docs/features/search_grid_search.rst b/docs/features/search_grid_search.md similarity index 51% rename from docs/features/search_grid_search.rst rename to docs/features/search_grid_search.md index d94bab352..2373150d3 100644 --- a/docs/features/search_grid_search.rst +++ b/docs/features/search_grid_search.md @@ -1,133 +1,128 @@ -.. _search_grid_search: - -Search Grid-Search -================== - -A classic method to perform model-fitting is a grid search, where the parameters of a model are divided on to a grid of -values and the likelihood of each set of parameters on this grid is sampled. For low dimensionality problems this -simple approach can be sufficient to locate high likelihood solutions, however it scales poorly to higher dimensional -problems. - -**PyAutoFit** can perform a search grid search, which allows one to perform a grid-search over a subset of parameters -within a model, but use a non-linear search to fit for the other parameters. The parameters over which the grid-search -is performed are also included in the model fit and their values are simply confined to the boundaries of their grid -cell by setting these as the lower and upper limits of a ``UniformPrior``. - -The benefits of using a search grid search are: - -- For problems with complex and multi-model parameters spaces it can be difficult to robustly and efficiently perform model-fitting. If specific parameters are known to drive the multi-modality then sampling over a grid can ensure the parameter space of each individual model-fit is not multi-modal and therefore sampled more accurately and efficiently. - -- It can provide a goodness-of-fit measure (e.g. the Bayesian evidence) of many model-fits over the grid. This can provide additional insight into where the model does and does not fit the data well, in a way that a standard non-linear search does not. - -- The search grid search is embarrassingly parallel, and if sufficient computing facilities are available one can perform model-fitting faster in real-time than a single non-linear search. The **PyAutoFit** search grid search includes an option for parallel model-fitting via the Python ``multiprocessing`` module. - -Data ----- - -In this example we will demonstrate the search grid search feature, again using the example of fitting 1D Gaussian's -in noisy data. This 1D data includes a small feature to the right of the central ``Gaussian``, a second ``Gaussian`` -centred on pixel 70. - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/features/images/gaussian_x1_with_feature.png - :width: 600 - :alt: Alternative text - -Basic Fit ---------- - -Without the search grid search we can fit this data as normal, by composing and fitting a model -containing two ``Gaussians``'s. - -.. code-block:: bash - - model = af.Collection(gaussian_main=m.Gaussian, gaussian_feature=m.Gaussian) - - analysis = a.Analysis(data=data, noise_map=noise_map) - - search = af.DynestyStatic( - name="single_fit", - nlive=100, - ) - - result = search.fit(model=model, analysis=analysis) - -For test runs on my laptop it is 'hit or miss' whether the feature is fitted correctly. This is because although models -including the feature corresponds to the highest likelihood solutions, they occupy a small volume in parameter space -which the non linear search may miss. - -The image below shows a fit where we failed to detect the feature: - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/features/images/gaussian_x1_with_feature_fit_no_feature.png - :width: 600 - :alt: Alternative text - -Grid Search ------------ - -Lets now perform the search grid search using the ``SearchGridSearch`` object: - -.. code-block:: bash - - search = af.DynestyStatic( - name="grid_fit", - nlive=100, - ) - - grid_search = af.SearchGridSearch( - search=dynesty, - number_of_steps=5, - number_of_cores=1, - ) - - -We specified two new inputs to the ``SearchGridSearch``: - -``number_of_steps``: The number of steps in the grid search that are performed which is set to 5 below. Because the -prior on the parameter ``centre`` is a ``UniformPrior`` from 0.0 -> 100.0, this means the first grid search will -set the prior on the centre to be a ``UniformPrior`` from 0.0 -> 20.0. The second will run from 20.0 -> 40.0, the -third 40.0 -> 60.0, and so on. - -``number_of_cores``: The number of cores the grid search will parallelize the run over. If ``number_of_cores=1``, the -search is run in serial. For > 1 core, 1 core is reserved as a farmer, e.g., if ``number_of_cores=4`` then up to 3 -searches will be run in parallel. - -We can now run the grid search, where we specify the parameter over which the grid search is performed, in this case -the ``centre`` of the ``gaussian_feature`` in our model. - -.. code-block:: bash - - grid_search_result = grid_search.fit( - model=model, - analysis=analysis, - grid_priors=[model.gaussian_feature.centre] - ) - -Result ------- - -This returns a ``GridSearchResult``, which includes information on every model-fit performed on the grid. For example, -I can use it to print the ``log_evidence`` of all 5 model-fits. - -.. code-block:: bash - - print(grid_search_result.log_evidence_values) - -This shows a peak evidence value on the 4th cell of grid-search, where the ``UniformPrior`` on the ``centre`` ran from -60 -> 80 and therefore included the Gaussian feature. By plotting this model-fit we can see it has successfully -detected the feature. - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/features/images/gaussian_x1_with_feature_fit_feature.png - :width: 600 - :alt: Alternative text - -A multi-dimensional grid search can be easily performed by adding more parameters to the ``grid_priors`` input. - -The fit below belows performs a 5x5 grid search over the ``centres`` of both ``Gaussians`` - -.. code-block:: bash - - grid_search_result = grid_search.fit( - model=model, - analysis=analysis, - grid_priors=[model.gaussian_feature.centre, model.gaussian_main.centre] - ) \ No newline at end of file +(search-grid-search)= + +# Search Grid-Search + +A classic method to perform model-fitting is a grid search, where the parameters of a model are divided on to a grid of +values and the likelihood of each set of parameters on this grid is sampled. For low dimensionality problems this +simple approach can be sufficient to locate high likelihood solutions, however it scales poorly to higher dimensional +problems. + +**PyAutoFit** can perform a search grid search, which allows one to perform a grid-search over a subset of parameters +within a model, but use a non-linear search to fit for the other parameters. The parameters over which the grid-search +is performed are also included in the model fit and their values are simply confined to the boundaries of their grid +cell by setting these as the lower and upper limits of a `UniformPrior`. + +The benefits of using a search grid search are: + +- For problems with complex and multi-model parameters spaces it can be difficult to robustly and efficiently perform model-fitting. If specific parameters are known to drive the multi-modality then sampling over a grid can ensure the parameter space of each individual model-fit is not multi-modal and therefore sampled more accurately and efficiently. +- It can provide a goodness-of-fit measure (e.g. the Bayesian evidence) of many model-fits over the grid. This can provide additional insight into where the model does and does not fit the data well, in a way that a standard non-linear search does not. +- The search grid search is embarrassingly parallel, and if sufficient computing facilities are available one can perform model-fitting faster in real-time than a single non-linear search. The **PyAutoFit** search grid search includes an option for parallel model-fitting via the Python `multiprocessing` module. + +## Data + +In this example we will demonstrate the search grid search feature, again using the example of fitting 1D Gaussian's +in noisy data. This 1D data includes a small feature to the right of the central `Gaussian`, a second `Gaussian` +centred on pixel 70. + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/features/images/gaussian_x1_with_feature.png +:alt: Alternative text +:width: 600 +``` + +## Basic Fit + +Without the search grid search we can fit this data as normal, by composing and fitting a model +containing two `Gaussians`'s. + +```bash +model = af.Collection(gaussian_main=m.Gaussian, gaussian_feature=m.Gaussian) + +analysis = a.Analysis(data=data, noise_map=noise_map) + +search = af.DynestyStatic( + name="single_fit", + nlive=100, +) + +result = search.fit(model=model, analysis=analysis) +``` + +For test runs on my laptop it is 'hit or miss' whether the feature is fitted correctly. This is because although models +including the feature corresponds to the highest likelihood solutions, they occupy a small volume in parameter space +which the non linear search may miss. + +The image below shows a fit where we failed to detect the feature: + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/features/images/gaussian_x1_with_feature_fit_no_feature.png +:alt: Alternative text +:width: 600 +``` + +## Grid Search + +Lets now perform the search grid search using the `SearchGridSearch` object: + +```bash +search = af.DynestyStatic( + name="grid_fit", + nlive=100, +) + +grid_search = af.SearchGridSearch( + search=dynesty, + number_of_steps=5, + number_of_cores=1, +) +``` + +We specified two new inputs to the `SearchGridSearch`: + +`number_of_steps`: The number of steps in the grid search that are performed which is set to 5 below. Because the +prior on the parameter `centre` is a `UniformPrior` from 0.0 -> 100.0, this means the first grid search will +set the prior on the centre to be a `UniformPrior` from 0.0 -> 20.0. The second will run from 20.0 -> 40.0, the +third 40.0 -> 60.0, and so on. + +`number_of_cores`: The number of cores the grid search will parallelize the run over. If `number_of_cores=1`, the +search is run in serial. For > 1 core, 1 core is reserved as a farmer, e.g., if `number_of_cores=4` then up to 3 +searches will be run in parallel. + +We can now run the grid search, where we specify the parameter over which the grid search is performed, in this case +the `centre` of the `gaussian_feature` in our model. + +```bash +grid_search_result = grid_search.fit( + model=model, + analysis=analysis, + grid_priors=[model.gaussian_feature.centre] +) +``` + +## Result + +This returns a `GridSearchResult`, which includes information on every model-fit performed on the grid. For example, +I can use it to print the `log_evidence` of all 5 model-fits. + +```bash +print(grid_search_result.log_evidence_values) +``` + +This shows a peak evidence value on the 4th cell of grid-search, where the `UniformPrior` on the `centre` ran from +60 -> 80 and therefore included the Gaussian feature. By plotting this model-fit we can see it has successfully +detected the feature. + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/features/images/gaussian_x1_with_feature_fit_feature.png +:alt: Alternative text +:width: 600 +``` + +A multi-dimensional grid search can be easily performed by adding more parameters to the `grid_priors` input. + +The fit below belows performs a 5x5 grid search over the `centres` of both `Gaussians` + +```bash +grid_search_result = grid_search.fit( + model=model, + analysis=analysis, + grid_priors=[model.gaussian_feature.centre, model.gaussian_main.centre] +) +``` diff --git a/docs/features/sensitivity_mapping.md b/docs/features/sensitivity_mapping.md new file mode 100644 index 000000000..04a3884d1 --- /dev/null +++ b/docs/features/sensitivity_mapping.md @@ -0,0 +1,233 @@ +(sensitivity-mapping)= + +# Sensitivity Mapping + +Bayesian model comparison allows us to take a dataset, fit it with multiple models and use the Bayesian evidence to +quantify which model objectively gives the best-fit following the principles of Occam's Razor. + +However, a complex model may not be favoured by model comparison not because it is the 'wrong' model, but simply +because the dataset being fitted is not of a sufficient quality for the more complex model to be favoured. Sensitivity +mapping addresses what quality of data would be needed for the more complex model to be favoured. + +In order to do this, sensitivity mapping involves us writing a function that uses the model(s) to simulate a dataset. +We then use this function to simulate many datasets, for many different models, and fit each dataset using the same +model-fitting procedure we used to perform Bayesian model comparison. This allows us to infer how much of a Bayesian +evidence increase we should expect for datasets of varying quality and / or models with different parameters. + +## Data + +To illustrate sensitivity mapping we will again use the example of fitting 1D Gaussian's in noisy data. This 1D data +includes a small feature to the right of the central `Gaussian`, a second `Gaussian` centred on pixel 70. + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/features/images/gaussian_x1_with_feature.png +:alt: Alternative text +:width: 600 +``` + +## Model Comparison + +Before performing sensitivity mapping, we will quickly perform Bayesian model comparison on this data to get a sense +for whether the `Gaussian` feature is detectable and how much the Bayesian evidence increases when it is included in +the model. + +We therefore fit the data using two models, one where the model is a single `Gaussian`. + +```bash +model = af.Collection(gaussian_main=m.Gaussian) + +search = af.DynestyStatic( + path_prefix=path.join("features", "sensitivity_mapping", "single_gaussian"), + nlive=100, + iterations_per_full_update=500, +) + +result_single = search.fit(model=model, analysis=analysis) +``` + +For the second model it contains two `Gaussians`. To avoid slow model-fitting and more clearly pronounce the results of +model comparison, we restrict the centre of the `gaussian_feature` to its true centre of 70 and sigma value of 0.5. + +```bash +model = af.Collection(gaussian_main=m.Gaussian, gaussian_feature=m.Gaussian) +model.gaussian_feature.centre = 70.0 +model.gaussian_feature.sigma = 0.5 + +search = af.DynestyStatic( + path_prefix=path.join("features", "sensitivity_mapping", "two_gaussians"), + nlive=100, + iterations_per_full_update=500, +) + +result_multiple = search.fit(model=model, analysis=analysis) +``` + +We can now print the `log_evidence` of each fit and confirm the model with two `Gaussians` was preferred to the model +with just one `Gaussian`. + +```bash +print(result_single.samples.log_evidence) +print(result_multiple.samples.log_evidence) +``` + +On my laptop, the increase in Bayesian evidence for the more compelx model is ~30, which is significant. + +The model comparison above shows that in this dataset, the `Gaussian` feature was detectable and that it increased the +Bayesian evidence by ~25. Furthermore, the normalization of this `Gaussian` was ~0.3. + +A lower value of normalization makes the `Gaussian` fainter and harder to detect. We will demonstrate sensitivity mapping +by answering the following question, at what value of normalization does the `Gaussian` feature become undetectable and +not provide us with a noticeable increase in Bayesian evidence? + +## Base Model + +To begin, we define the `base_model` that we use to perform sensitivity mapping. This model is used to simulate every +dataset. It is also fitted to every simulated dataset without the extra model component below, to give us the Bayesian +evidence of the every simpler model to compare to the more complex model. + +The `base_model` corresponds to the `gaussian_main` above. + +```bash +base_model = af.Collection(gaussian_main=m.Gaussian) +``` + +## Perturbation Model + +We now define the `perturb_model`, which is the model component whose parameters we iterate over to perform +sensitivity mapping. Many instances of the `perturb_model` are created and used to simulate the many datasets +that we fit. However, it is only included in half of the model-fits corresponding to the more complex models whose +Bayesian evidence we compare to the simpler model-fits consisting of just the `base_model`. + +The `perturb_model` is therefore another `Gaussian` but now corresponds to the `gaussian_feature` above. + +By fitting both of these models to every simulated dataset, we will therefore infer the Bayesian evidence of every +model to every dataset. Sensitivity mapping therefore maps out for what values of `normalization` in the `gaussian_feature` +does the more complex model-fit provide higher values of Bayesian evidence than the simpler model-fit. We also fix the +values ot the `centre` and `sigma` of the `Gaussian` so we only map over its `normalization`. + +```bash +perturb_model = af.Model(m.Gaussian) +perturb_model.centre = 70.0 +perturb_model.sigma = 0.5 +perturb_model.normalization = af.UniformPrior(lower_limit=0.01, upper_limit=100.0) +``` + +## Simulation + +We are performing sensitivity mapping to determine how bright the `gaussian_feature` needs to be in order to be +detectable. However, every simulated dataset must include the `main_gaussian`, as its presence in the data will effect +the detectability of the `gaussian_feature`. + +We can pass the `main_gaussian` into the sensitivity mapping as the `simulation_instance`, meaning that it will be used +in the simulation of every dataset. For this example we use the inferred `main_gaussian` from one of the model-fits +performed above. + +```bash +simulation_instance = result_single.instance +``` + +We now write the `simulate_cls`, which takes the `instance` of our model (defined above) and uses it to +simulate a dataset which is subsequently fitted. + +Note that when this dataset is simulated, the quantity `instance.perturb` is used in the `simulate_cls`. +This is an instance of the `gaussian_feature`, and it is different every time the `simulate_cls` is called. + +In this example, this `instance.perturb` corresponds to different `gaussian_feature`'s with values of +`normalization` ranging over 0.01 -> 100.0, such that our simulated datasets correspond to a very faint and very bright +gaussian features. + +```bash +def __call__(instance, simulate_path): + + """ + Specify the number of pixels used to create the xvalues on which the 1D line of the profile is generated using and + thus defining the number of data-points in our data. + """ + pixels = 100 + xvalues = np.arange(pixels) + + """ + Evaluate the ``Gaussian`` and Exponential model instances at every xvalues to create their model profile and sum + them together to create the overall model profile. + + This print statement will show that, when you run ``Sensitivity`` below the values of the perturbation use fixed + values of ``centre=70`` and ``sigma=0.5``, whereas the normalization varies over the ``step_size`` based on its prior. + """ + + print(instance.perturb.centre) + print(instance.perturb.normalization) + print(instance.perturb.sigma) + + model_line = instance.gaussian_main.model_data_from(xvalues=xvalues) + instance.perturb.model_data_from(xvalues=xvalues) + + """Determine the noise (at a specified signal to noise level) in every pixel of our model profile.""" + signal_to_noise_ratio = 25.0 + noise = np.random.normal(0.0, 1.0 / signal_to_noise_ratio, pixels) + + """ + Add this noise to the model line to create the line data that is fitted, using the signal-to-noise ratio to compute + noise-map of our data which is required when evaluating the chi-squared value of the likelihood. + """ + data = model_line + noise + noise_map = (1.0 / signal_to_noise_ratio) * np.ones(pixels) + + return Imaging(data=data, noise_map=noise_map) +``` + +Here are what the two most extreme simulated datasets look like, corresponding to the highest and lowest normalization values + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/features/images/sensitivity_data_low.png +:alt: Alternative text +:width: 600 +``` + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/features/images/sensitivity_data_high.png +:alt: Alternative text +:width: 600 +``` + +## Summary + +We can now combine all of the objects created above and perform sensitivity mapping. The inputs to the `Sensitivity` +object below are: + +- `simulation_instance`: This is an instance of the model used to simulate every dataset that is fitted. In this example it contains an instance of the `gaussian_main` model component. +- `base_model`: This is the simpler model that is fitted to every simulated dataset, which in this example is composed of a single `Gaussian` called the `gaussian_main`. +- `perturb_model`: This is the extra model component that alongside the `base_model` is fitted to every simulated dataset, which in this example is composed of two `Gaussians` called the `gaussian_main` and `gaussian_feature`. +- `simulate_cls`: This is the function that uses the `instance` and many instances of the `perturb_model` to simulate many datasets that are fitted with the `base_model` and `base_model` + `perturb_model`. +- `step_size`: The size of steps over which the parameters in the `perturb_model` are iterated. In this example, normalization has a `LogUniformPrior` with lower limit 1e-4 and upper limit 1e2, therefore the `step_size` of 0.5 will simulate and fit just 2 datasets where the normalization is 1e-4 and 1e2. +- `number_of_cores`: The number of cores over which the sensitivity mapping is performed, enabling parallel processing. + +(Note that for brevity we have omitted a couple of extra inputs in this example, which can be found by going to the +full example script on the `autofit_workspace`). + +```bash +sensitivity = s.Sensitivity( + search=search, + simulation_instance=simulation_instance, + base_model=base_model, + perturb_model=perturb_model, + simulate_cls=simulate_cls, + analysis_class=Analysis, + step_size=0.5, + number_of_cores=2, +) + +sensitivity_result = sensitivity.run() +``` + +Here are what the fits to the two most extreme simulated datasets look like, for the models including the Gaussian +feature. + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/features/images/sensitivity_data_low_fit.png +:alt: Alternative text +:width: 600 +``` + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/features/images/sensitivity_data_high_fit.png +:alt: Alternative text +:width: 600 +``` + +The key point to note is that for every dataset, we now have a model-fit with and without the model `perturbation`. By +compairing the Bayesian evidence of every pair of fits for every value of `normalization` we are able to determine when +our model was sensitivity to the `Gaussian` feature and therefore could detect it! diff --git a/docs/features/sensitivity_mapping.rst b/docs/features/sensitivity_mapping.rst deleted file mode 100644 index 15f024b29..000000000 --- a/docs/features/sensitivity_mapping.rst +++ /dev/null @@ -1,241 +0,0 @@ -.. _sensitivity_mapping: - -Sensitivity Mapping -=================== - -Bayesian model comparison allows us to take a dataset, fit it with multiple models and use the Bayesian evidence to -quantify which model objectively gives the best-fit following the principles of Occam's Razor. - -However, a complex model may not be favoured by model comparison not because it is the 'wrong' model, but simply -because the dataset being fitted is not of a sufficient quality for the more complex model to be favoured. Sensitivity -mapping addresses what quality of data would be needed for the more complex model to be favoured. - -In order to do this, sensitivity mapping involves us writing a function that uses the model(s) to simulate a dataset. -We then use this function to simulate many datasets, for many different models, and fit each dataset using the same -model-fitting procedure we used to perform Bayesian model comparison. This allows us to infer how much of a Bayesian -evidence increase we should expect for datasets of varying quality and / or models with different parameters. - -Data ----- - -To illustrate sensitivity mapping we will again use the example of fitting 1D Gaussian's in noisy data. This 1D data -includes a small feature to the right of the central ``Gaussian``, a second ``Gaussian`` centred on pixel 70. - - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/features/images/gaussian_x1_with_feature.png - :width: 600 - :alt: Alternative text - -Model Comparison ----------------- - -Before performing sensitivity mapping, we will quickly perform Bayesian model comparison on this data to get a sense -for whether the ``Gaussian`` feature is detectable and how much the Bayesian evidence increases when it is included in -the model. - -We therefore fit the data using two models, one where the model is a single ``Gaussian``. - -.. code-block:: bash - - model = af.Collection(gaussian_main=m.Gaussian) - - search = af.DynestyStatic( - path_prefix=path.join("features", "sensitivity_mapping", "single_gaussian"), - nlive=100, - iterations_per_full_update=500, - ) - - result_single = search.fit(model=model, analysis=analysis) - -For the second model it contains two ``Gaussians``. To avoid slow model-fitting and more clearly pronounce the results of -model comparison, we restrict the centre of the ``gaussian_feature`` to its true centre of 70 and sigma value of 0.5. - -.. code-block:: bash - - model = af.Collection(gaussian_main=m.Gaussian, gaussian_feature=m.Gaussian) - model.gaussian_feature.centre = 70.0 - model.gaussian_feature.sigma = 0.5 - - search = af.DynestyStatic( - path_prefix=path.join("features", "sensitivity_mapping", "two_gaussians"), - nlive=100, - iterations_per_full_update=500, - ) - - result_multiple = search.fit(model=model, analysis=analysis) - -We can now print the ``log_evidence`` of each fit and confirm the model with two ``Gaussians`` was preferred to the model -with just one ``Gaussian``. - -.. code-block:: bash - - print(result_single.samples.log_evidence) - print(result_multiple.samples.log_evidence) - -On my laptop, the increase in Bayesian evidence for the more compelx model is ~30, which is significant. - -The model comparison above shows that in this dataset, the ``Gaussian`` feature was detectable and that it increased the -Bayesian evidence by ~25. Furthermore, the normalization of this ``Gaussian`` was ~0.3. - -A lower value of normalization makes the ``Gaussian`` fainter and harder to detect. We will demonstrate sensitivity mapping -by answering the following question, at what value of normalization does the ``Gaussian`` feature become undetectable and -not provide us with a noticeable increase in Bayesian evidence? - -Base Model ----------- - -To begin, we define the ``base_model`` that we use to perform sensitivity mapping. This model is used to simulate every -dataset. It is also fitted to every simulated dataset without the extra model component below, to give us the Bayesian -evidence of the every simpler model to compare to the more complex model. - -The ``base_model`` corresponds to the ``gaussian_main`` above. - -.. code-block:: bash - - base_model = af.Collection(gaussian_main=m.Gaussian) - -Perturbation Model ------------------- - -We now define the ``perturb_model``, which is the model component whose parameters we iterate over to perform -sensitivity mapping. Many instances of the ``perturb_model`` are created and used to simulate the many datasets -that we fit. However, it is only included in half of the model-fits corresponding to the more complex models whose -Bayesian evidence we compare to the simpler model-fits consisting of just the ``base_model``. - -The ``perturb_model`` is therefore another ``Gaussian`` but now corresponds to the ``gaussian_feature`` above. - -By fitting both of these models to every simulated dataset, we will therefore infer the Bayesian evidence of every -model to every dataset. Sensitivity mapping therefore maps out for what values of ``normalization`` in the ``gaussian_feature`` -does the more complex model-fit provide higher values of Bayesian evidence than the simpler model-fit. We also fix the -values ot the ``centre`` and ``sigma`` of the ``Gaussian`` so we only map over its ``normalization``. - -.. code-block:: bash - - perturb_model = af.Model(m.Gaussian) - perturb_model.centre = 70.0 - perturb_model.sigma = 0.5 - perturb_model.normalization = af.UniformPrior(lower_limit=0.01, upper_limit=100.0) - -Simulation ----------- - -We are performing sensitivity mapping to determine how bright the ``gaussian_feature`` needs to be in order to be -detectable. However, every simulated dataset must include the ``main_gaussian``, as its presence in the data will effect -the detectability of the ``gaussian_feature``. - -We can pass the ``main_gaussian`` into the sensitivity mapping as the ``simulation_instance``, meaning that it will be used -in the simulation of every dataset. For this example we use the inferred ``main_gaussian`` from one of the model-fits -performed above. - -.. code-block:: bash - - simulation_instance = result_single.instance - -We now write the ``simulate_cls``, which takes the ``instance`` of our model (defined above) and uses it to -simulate a dataset which is subsequently fitted. - -Note that when this dataset is simulated, the quantity ``instance.perturb`` is used in the ``simulate_cls``. -This is an instance of the ``gaussian_feature``, and it is different every time the ``simulate_cls`` is called. - -In this example, this ``instance.perturb`` corresponds to different ``gaussian_feature``'s with values of -``normalization`` ranging over 0.01 -> 100.0, such that our simulated datasets correspond to a very faint and very bright -gaussian features. - -.. code-block:: bash - - def __call__(instance, simulate_path): - - """ - Specify the number of pixels used to create the xvalues on which the 1D line of the profile is generated using and - thus defining the number of data-points in our data. - """ - pixels = 100 - xvalues = np.arange(pixels) - - """ - Evaluate the ``Gaussian`` and Exponential model instances at every xvalues to create their model profile and sum - them together to create the overall model profile. - - This print statement will show that, when you run ``Sensitivity`` below the values of the perturbation use fixed - values of ``centre=70`` and ``sigma=0.5``, whereas the normalization varies over the ``step_size`` based on its prior. - """ - - print(instance.perturb.centre) - print(instance.perturb.normalization) - print(instance.perturb.sigma) - - model_line = instance.gaussian_main.model_data_from(xvalues=xvalues) + instance.perturb.model_data_from(xvalues=xvalues) - - """Determine the noise (at a specified signal to noise level) in every pixel of our model profile.""" - signal_to_noise_ratio = 25.0 - noise = np.random.normal(0.0, 1.0 / signal_to_noise_ratio, pixels) - - """ - Add this noise to the model line to create the line data that is fitted, using the signal-to-noise ratio to compute - noise-map of our data which is required when evaluating the chi-squared value of the likelihood. - """ - data = model_line + noise - noise_map = (1.0 / signal_to_noise_ratio) * np.ones(pixels) - - return Imaging(data=data, noise_map=noise_map) - -Here are what the two most extreme simulated datasets look like, corresponding to the highest and lowest normalization values - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/features/images/sensitivity_data_low.png - :width: 600 - :alt: Alternative text - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/features/images/sensitivity_data_high.png - :width: 600 - :alt: Alternative text - -Summary -------- - -We can now combine all of the objects created above and perform sensitivity mapping. The inputs to the ``Sensitivity`` -object below are: - -- ``simulation_instance``: This is an instance of the model used to simulate every dataset that is fitted. In this example it contains an instance of the ``gaussian_main`` model component. - -- ``base_model``: This is the simpler model that is fitted to every simulated dataset, which in this example is composed of a single ``Gaussian`` called the ``gaussian_main``. - -- ``perturb_model``: This is the extra model component that alongside the ``base_model`` is fitted to every simulated dataset, which in this example is composed of two ``Gaussians`` called the ``gaussian_main`` and ``gaussian_feature``. - -- ``simulate_cls``: This is the function that uses the ``instance`` and many instances of the ``perturb_model`` to simulate many datasets that are fitted with the ``base_model`` and ``base_model`` + ``perturb_model``. - -- ``step_size``: The size of steps over which the parameters in the ``perturb_model`` are iterated. In this example, normalization has a ``LogUniformPrior`` with lower limit 1e-4 and upper limit 1e2, therefore the ``step_size`` of 0.5 will simulate and fit just 2 datasets where the normalization is 1e-4 and 1e2. - -- ``number_of_cores``: The number of cores over which the sensitivity mapping is performed, enabling parallel processing. - -(Note that for brevity we have omitted a couple of extra inputs in this example, which can be found by going to the -full example script on the ``autofit_workspace``). - -.. code-block:: bash - - sensitivity = s.Sensitivity( - search=search, - simulation_instance=simulation_instance, - base_model=base_model, - perturb_model=perturb_model, - simulate_cls=simulate_cls, - analysis_class=Analysis, - step_size=0.5, - number_of_cores=2, - ) - - sensitivity_result = sensitivity.run() - -Here are what the fits to the two most extreme simulated datasets look like, for the models including the Gaussian -feature. - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/features/images/sensitivity_data_low_fit.png - :width: 600 - :alt: Alternative text - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/features/images/sensitivity_data_high_fit.png - :width: 600 - :alt: Alternative text - -The key point to note is that for every dataset, we now have a model-fit with and without the model ``perturbation``. By -compairing the Bayesian evidence of every pair of fits for every value of ``normalization`` we are able to determine when -our model was sensitivity to the ``Gaussian`` feature and therefore could detect it! \ No newline at end of file diff --git a/docs/general/citations.md b/docs/general/citations.md new file mode 100644 index 000000000..962e2d0e0 --- /dev/null +++ b/docs/general/citations.md @@ -0,0 +1,16 @@ +(references)= + +# Citations & References + +The bibtex entries for **PyAutoFit** and its affiliated software packages can be found +[here](https://github.com/rhayes777/PyAutoFit/blob/main/files/citations.bib), with example text for citing **PyAutoFit** +in [.tex format here](https://github.com/rhayes777/PyAutoFit/blob/main/files/citation.tex) format here and +[.md format here](https://github.com/rhayes777/PyAutoFit/blob/main/files/citations.md). As shown in the examples, we +would greatly appreciate it if you mention **PyAutoFit** by name and include a link to our GitHub page! + +**PyAutoFit** is published in the [Journal of Open Source Software](https://joss.theoj.org/papers/10.21105/joss.02550#) and its +entry in the above .bib file is under the key `pyautofit`. + +## Dynesty + +If you used the nested sampling algorithm Dynesty, please follow the citation instructions [on the dynesty readthedocs](https://dynesty.readthedocs.io/en/latest/references.html). diff --git a/docs/general/citations.rst b/docs/general/citations.rst deleted file mode 100644 index 725f35cff..000000000 --- a/docs/general/citations.rst +++ /dev/null @@ -1,18 +0,0 @@ -.. _references: - -Citations & References -====================== - -The bibtex entries for **PyAutoFit** and its affiliated software packages can be found -`here `_, with example text for citing **PyAutoFit** -in `.tex format here `_ format here and -`.md format here `_. As shown in the examples, we -would greatly appreciate it if you mention **PyAutoFit** by name and include a link to our GitHub page! - -**PyAutoFit** is published in the `Journal of Open Source Software `_ and its -entry in the above .bib file is under the key ``pyautofit``. - -Dynesty -------- - -If you used the nested sampling algorithm Dynesty, please follow the citation instructions `on the dynesty readthedocs `_. \ No newline at end of file diff --git a/docs/general/configs.md b/docs/general/configs.md new file mode 100644 index 000000000..2ab63be81 --- /dev/null +++ b/docs/general/configs.md @@ -0,0 +1,24 @@ +# Configs + +**PyAutoFit** uses a number of configuration files that customize the default behaviour of the non-linear searches, +visualization and other aspects of **PyAutoFit**. + +Descriptions of every configuration file and their input parameters are provided in the `README.rst` in +the [config directory of the workspace](https://github.com/Jammy2211/autofit_workspace/tree/release/config) + +## Setup + +By default, **PyAutoFit** looks for the config files in a `config` folder in the current working directory, which is +why we run autofit scripts from the `autofit_workspace` directory. + +The configuration path can also be set manually in a script using the project **PyAutoConf** and the following +command (the path to the `output` folder where the results of a non-linear search are stored is also set below): + +```bash +from autoconf import conf + +conf.instance.push( + new_path="path/to/config", + output_path=f"path/to/output" +) +``` diff --git a/docs/general/configs.rst b/docs/general/configs.rst deleted file mode 100644 index 5f4900cc9..000000000 --- a/docs/general/configs.rst +++ /dev/null @@ -1,27 +0,0 @@ -Configs -======= - -**PyAutoFit** uses a number of configuration files that customize the default behaviour of the non-linear searches, -visualization and other aspects of **PyAutoFit**. - -Descriptions of every configuration file and their input parameters are provided in the ``README.rst`` in -the `config directory of the workspace `_ - - -Setup ------ - -By default, **PyAutoFit** looks for the config files in a ``config`` folder in the current working directory, which is -why we run autofit scripts from the ``autofit_workspace`` directory. - -The configuration path can also be set manually in a script using the project **PyAutoConf** and the following -command (the path to the ``output`` folder where the results of a non-linear search are stored is also set below): - -.. code-block:: bash - - from autoconf import conf - - conf.instance.push( - new_path="path/to/config", - output_path=f"path/to/output" - ) diff --git a/docs/general/credits.md b/docs/general/credits.md new file mode 100644 index 000000000..9aa05f330 --- /dev/null +++ b/docs/general/credits.md @@ -0,0 +1,11 @@ +(credits)= + +# Credits + +**Developers:** + +[Richard Hayes](https://github.com/rhayes777) - Lead developer + +[James Nightingale](https://github.com/Jammy2211) - Lead developer + +[Matthew Griffiths](https://github.com/matthewghgriffiths) - Graphical models guru diff --git a/docs/general/credits.rst b/docs/general/credits.rst deleted file mode 100644 index 171c4ed20..000000000 --- a/docs/general/credits.rst +++ /dev/null @@ -1,12 +0,0 @@ -.. _credits: - -Credits -------- - -**Developers:** - -`Richard Hayes `_ - Lead developer - -`James Nightingale `_ - Lead developer - -`Matthew Griffiths `_ - Graphical models guru \ No newline at end of file diff --git a/docs/general/roadmap.rst b/docs/general/roadmap.md similarity index 66% rename from docs/general/roadmap.rst rename to docs/general/roadmap.md index 2f8db976f..3d7f6b63f 100644 --- a/docs/general/roadmap.rst +++ b/docs/general/roadmap.md @@ -1,26 +1,25 @@ -.. _roadmap: - -Road Map -======== - -**PyAutoFit** is in active development and the road-map of features currently planned in the short and long term are -listed and described below: - -**JAX:** - -Supported for autodiff is nearly implemented, including JAX fits. - -**Non-Linear Searches:** - -We are always striving to add new non-linear searches to **PyAutoFit*. In the short term, we aim to provide a wrapper to the many method available in the ``scipy.optimize`` library with support for outputting results to hard-disk. - -If you would like to see a non-linear search implemented in **PyAutoFit** please `raise an issue on GitHub `_! - -**Approximate Bayesian Computation** - -Approximate Bayesian Computational (ABC) allows for one to infer parameter values for likelihood functions that are -intractable, by simulating many datasets and extracting from them a summary statistic that is compared to the -observed dataset. - -ABC in **PyAutoFit** will be closely tied to the Database tools, ensuring that the simulation, fitting and extraction -of summary statistics can be efficiently scaled up to extremely large datasets. \ No newline at end of file +(roadmap)= + +# Road Map + +**PyAutoFit** is in active development and the road-map of features currently planned in the short and long term are +listed and described below: + +**JAX:** + +Supported for autodiff is nearly implemented, including JAX fits. + +**Non-Linear Searches:** + +We are always striving to add new non-linear searches to PyAutoFit\*. In the short term, we aim to provide a wrapper to the many method available in the `scipy.optimize` library with support for outputting results to hard-disk. + +If you would like to see a non-linear search implemented in **PyAutoFit** please [raise an issue on GitHub](https://github.com/rhayes777/PyAutoFit/issues)! + +**Approximate Bayesian Computation** + +Approximate Bayesian Computational (ABC) allows for one to infer parameter values for likelihood functions that are +intractable, by simulating many datasets and extracting from them a summary statistic that is compared to the +observed dataset. + +ABC in **PyAutoFit** will be closely tied to the Database tools, ensuring that the simulation, fitting and extraction +of summary statistics can be efficiently scaled up to extremely large datasets. diff --git a/docs/general/software.rst b/docs/general/software.md similarity index 50% rename from docs/general/software.rst rename to docs/general/software.md index a811656e7..e17b0d6d9 100644 --- a/docs/general/software.rst +++ b/docs/general/software.md @@ -1,15 +1,14 @@ -.. _software: - -Software --------- - -The following software projects use **PyAutoFit**: - -`PyAutoLens `_ - -Astronomy software for modeling Strong Gravitational Lenses. - -`PyAutoGalaxy `_ - -Astronomy software for modeling galaxy light profiles and dynamics. - -`PyAutoCTI `_ - -Software for modeling Charge Transfer Inefficiency induced by radiation damage to CCDs. \ No newline at end of file +(software)= + +# Software + +The following software projects use **PyAutoFit**: + +[PyAutoLens](https://github.com/Jammy2211/PyAutoLens) - +Astronomy software for modeling Strong Gravitational Lenses. + +[PyAutoGalaxy](https://github.com/Jammy2211/PyAutoGalaxy) - +Astronomy software for modeling galaxy light profiles and dynamics. + +[PyAutoCTI](https://github.com/Jammy2211/PyAutoCTI) - +Software for modeling Charge Transfer Inefficiency induced by radiation damage to CCDs. diff --git a/docs/general/workspace.md b/docs/general/workspace.md new file mode 100644 index 000000000..1dd9d48b1 --- /dev/null +++ b/docs/general/workspace.md @@ -0,0 +1,48 @@ +(workspace)= + +# Workspace Tour + +You should have downloaded and configured the [autofit_workspace](https://github.com/Jammy2211/autofit_workspace) +when you installed **PyAutoFit**. If you didn't, checkout the +[installation instructions](https://pyautofit.readthedocs.io/en/latest/general/installation.html#installation-with-pip) +for how to downloaded and configure the workspace. + +The `README.rst` files distributed throughout the workspace describe every folder and file, and specify if +examples are for beginner or advanced users. + +New users should begin by checking out the following parts of the workspace. + +## HowToFit + +The **HowToFit** lecture series is a collection of Jupyter notebooks describing how to build a **PyAutoFit** model +fitting project and giving illustrations of different statistical methods and techniques. + +HowToFit now lives in its own standalone repository at [PyAutoLabs/HowToFit](https://github.com/PyAutoLabs/HowToFit). +Clone or browse the repo for a full description of the lectures and the notebooks for every chapter. + +## Scripts / Notebooks + +There are numerous example describing how perform model-fitting with **PyAutoFit** and providing an overview of its +advanced model-fitting features. All examples are provided as Python scripts and Jupyter notebooks. + +Descriptions of every configuration file and their input parameters are provided in the `README.rst` in +the [config directory of the workspace](https://github.com/Jammy2211/autofit_workspace/tree/release/config) + +## Config + +Here, you'll find the configuration files used by **PyAutoFit** which customize: + +> - The default settings used by every non-linear search. +> - Example priors and notation configs which associate model-component with model-fitting. +> - The `general.ini` config which customizes other aspects of **PyAutoFit**. + +Checkout the [configuration](https://pyautofit.readthedocs.io/en/latest/general/installation.html#installation-with-pip) +section of the `readthedocs` for a complete description of every configuration file. + +## Dataset + +This folder stores the example dataset's used in examples in the workspace. + +## Output + +The folder where the model-fitting results of a non-linear search are stored. diff --git a/docs/general/workspace.rst b/docs/general/workspace.rst deleted file mode 100644 index b6aaf7cef..000000000 --- a/docs/general/workspace.rst +++ /dev/null @@ -1,54 +0,0 @@ -.. _workspace: - -Workspace Tour -============== - -You should have downloaded and configured the `autofit_workspace `_ -when you installed **PyAutoFit**. If you didn't, checkout the -`installation instructions `_ -for how to downloaded and configure the workspace. - -The ``README.rst`` files distributed throughout the workspace describe every folder and file, and specify if -examples are for beginner or advanced users. - -New users should begin by checking out the following parts of the workspace. - -HowToFit --------- - -The **HowToFit** lecture series is a collection of Jupyter notebooks describing how to build a **PyAutoFit** model -fitting project and giving illustrations of different statistical methods and techniques. - -HowToFit now lives in its own standalone repository at `PyAutoLabs/HowToFit `_. -Clone or browse the repo for a full description of the lectures and the notebooks for every chapter. - -Scripts / Notebooks -------------------- - -There are numerous example describing how perform model-fitting with **PyAutoFit** and providing an overview of its -advanced model-fitting features. All examples are provided as Python scripts and Jupyter notebooks. - -Descriptions of every configuration file and their input parameters are provided in the ``README.rst`` in -the `config directory of the workspace `_ - -Config ------- - -Here, you'll find the configuration files used by **PyAutoFit** which customize: - - - The default settings used by every non-linear search. - - Example priors and notation configs which associate model-component with model-fitting. - - The ``general.ini`` config which customizes other aspects of **PyAutoFit**. - -Checkout the `configuration `_ -section of the ``readthedocs`` for a complete description of every configuration file. - -Dataset -------- - -This folder stores the example dataset's used in examples in the workspace. - -Output ------- - -The folder where the model-fitting results of a non-linear search are stored. \ No newline at end of file diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 000000000..f24fc3cc0 --- /dev/null +++ b/docs/index.md @@ -0,0 +1,218 @@ +# PyAutoFit + +**PyAutoFit** is a Python based probabilistic programming language for model fitting and Bayesian inference +of large datasets. + +The basic **PyAutoFit** API allows us a user to quickly compose a probabilistic model and fit it to data via a +log likelihood function, using a range of non-linear search algorithms (e.g. MCMC, nested sampling). + +Users can then set up **PyAutoFit** scientific workflow, which enables streamlined modeling of small +datasets with tools to scale up to large datasets. + +**PyAutoFit** supports advanced statistical methods, most +notably [a big data framework for Bayesian hierarchical analysis](https://pyautofit.readthedocs.io/en/latest/features/graphical.html). + +## Getting Started + +The following links are useful for new starters: + +- [The PyAutoFit readthedocs](https://pyautofit.readthedocs.io/en/latest), which includes an [installation guide](https://pyautofit.readthedocs.io/en/latest/installation/overview.html) and an overview of **PyAutoFit**'s core features. +- [The introduction Jupyter Notebook on Colab](https://colab.research.google.com/github/PyAutoLabs/autofit_workspace/blob/2026.4.13.6/notebooks/overview/overview_1_the_basics.ipynb), where you can try **PyAutoFit** in a web browser (without installation). +- [The autofit_workspace GitHub repository](https://github.com/Jammy2211/autofit_workspace), which includes example scripts demonstrating **PyAutoFit**'s features. +- [The standalone HowToFit repository](https://github.com/PyAutoLabs/HowToFit), a series of Jupyter notebook lectures which give new users a step-by-step introduction to **PyAutoFit**. + +## Support + +Support for installation issues, help with Fit modeling and using **PyAutoFit** is available by +[raising an issue on the GitHub issues page](https://github.com/rhayes777/PyAutoFit/issues). + +We also offer support on the **PyAutoFit** [Slack channel](https://pyautoFit.slack.com/), where we also provide the +latest updates on **PyAutoFit**. Slack is invitation-only, so if you'd like to join send +an [email](https://github.com/Jammy2211) requesting an invite. + +## HowToFit + +For users less familiar with Bayesian inference and scientific analysis you may wish to read through +the **HowToFits** lectures. These teach you the basic principles of Bayesian inference, with the +content pitched at undergraduate level and above. + +The lectures are available in the [standalone HowToFit repository](https://github.com/PyAutoLabs/HowToFit). + +## API Overview + +To illustrate the **PyAutoFit** API, we use an illustrative toy model of fitting a one-dimensional Gaussian to +noisy 1D data. Here's the `data` (black) and the model (red) we'll fit: + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/files/toy_model_fit.png +:width: 400 +``` + +We define our model, a 1D Gaussian by writing a Python class using the format below: + +```python +class Gaussian: + + def __init__( + self, + centre=0.0, # <- PyAutoFit recognises these + normalization=0.1, # <- constructor arguments are + sigma=0.01, # <- the Gaussian's parameters. + ): + self.centre = centre + self.normalization = normalization + self.sigma = sigma + + """ + An instance of the Gaussian class will be available during model fitting. + + This method will be used to fit the model to data and compute a likelihood. + """ + + def model_data_from(self, xvalues): + + transformed_xvalues = xvalues - self.centre + + return (self.normalization / (self.sigma * (2.0 * np.pi) ** 0.5)) * \ + np.exp(-0.5 * (transformed_xvalues / self.sigma) ** 2.0) +``` + +**PyAutoFit** recognises that this Gaussian may be treated as a model component whose parameters can be fitted for via +a non-linear search like [emcee](https://github.com/dfm/emcee). + +To fit this Gaussian to the `data` we create an Analysis object, which gives **PyAutoFit** the `data` and a +`log_likelihood_function` describing how to fit the `data` with the model: + +```python +class Analysis(af.Analysis): + + def __init__(self, data, noise_map): + + self.data = data + self.noise_map = noise_map + + def log_likelihood_function(self, instance): + + """ + The 'instance' that comes into this method is an instance of the Gaussian class + above, with the parameters set to values chosen by the non-linear search. + """ + + print("Gaussian Instance:") + print("Centre = ", instance.centre) + print("normalization = ", instance.normalization) + print("Sigma = ", instance.sigma) + + """ + We fit the ``data`` with the Gaussian instance, using its + "model_data_from" function to create the model data. + """ + + xvalues = np.arange(self.data.shape[0]) + + model_data = instance.model_data_from(xvalues=xvalues) + residual_map = self.data - model_data + chi_squared_map = (residual_map / self.noise_map) ** 2.0 + log_likelihood = -0.5 * sum(chi_squared_map) + + return log_likelihood +``` + +We can now fit our model to the `data` using a non-linear search: + +```python +model = af.Model(Gaussian) + +analysis = Analysis(data=data, noise_map=noise_map) + +emcee = af.Emcee(nwalkers=50, nsteps=2000) + +result = emcee.fit(model=model, analysis=analysis) +``` + +The `result` contains information on the model-fit, for example the parameter samples, maximum log likelihood +model and marginalized probability density functions. + +```{toctree} +:caption: 'Overview:' +:hidden: true +:maxdepth: 1 + +overview/the_basics +overview/scientific_workflow +overview/statistical_methods +``` + +```{toctree} +:caption: 'Cookbooks:' +:hidden: true +:maxdepth: 1 + +cookbooks/model +cookbooks/analysis +cookbooks/search +cookbooks/result +cookbooks/samples +cookbooks/configs +cookbooks/multiple_datasets +cookbooks/multi_level_model +``` + +```{toctree} +:caption: 'Features:' +:hidden: true +:maxdepth: 1 + +features/graphical +features/interpolate +features/search_chaining +features/search_grid_search +features/sensitivity_mapping +``` + +```{toctree} +:caption: 'Installation:' +:hidden: true +:maxdepth: 1 + +installation/overview +installation/conda +installation/pip +installation/source +installation/troubleshooting +``` + +```{toctree} +:caption: 'General:' +:hidden: true +:maxdepth: 1 + +general/workspace +general/configs +general/roadmap +general/software +general/citations +general/credits +``` + +```{toctree} +:caption: 'Science Examples:' +:hidden: true +:maxdepth: 1 + +science_examples/astronomy +``` + +```{toctree} +:caption: 'API Reference:' +:hidden: true +:maxdepth: 1 + +api/model +api/priors +api/analysis +api/searches +api/plot +api/samples +api/database +api/source +``` diff --git a/docs/index.rst b/docs/index.rst deleted file mode 100644 index e253d77ea..000000000 --- a/docs/index.rst +++ /dev/null @@ -1,219 +0,0 @@ -PyAutoFit -========= - -**PyAutoFit** is a Python based probabilistic programming language for model fitting and Bayesian inference -of large datasets. - -The basic **PyAutoFit** API allows us a user to quickly compose a probabilistic model and fit it to data via a -log likelihood function, using a range of non-linear search algorithms (e.g. MCMC, nested sampling). - -Users can then set up **PyAutoFit** scientific workflow, which enables streamlined modeling of small -datasets with tools to scale up to large datasets. - -**PyAutoFit** supports advanced statistical methods, most -notably `a big data framework for Bayesian hierarchical analysis `_. - -Getting Started ---------------- - -The following links are useful for new starters: - -- `The PyAutoFit readthedocs `_, which includes an `installation guide `_ and an overview of **PyAutoFit**'s core features. - -- `The introduction Jupyter Notebook on Colab `_, where you can try **PyAutoFit** in a web browser (without installation). - -- `The autofit_workspace GitHub repository `_, which includes example scripts demonstrating **PyAutoFit**'s features. - -- `The standalone HowToFit repository `_, a series of Jupyter notebook lectures which give new users a step-by-step introduction to **PyAutoFit**. - -Support -------- - -Support for installation issues, help with Fit modeling and using **PyAutoFit** is available by -`raising an issue on the GitHub issues page `_. - -We also offer support on the **PyAutoFit** `Slack channel `_, where we also provide the -latest updates on **PyAutoFit**. Slack is invitation-only, so if you'd like to join send -an `email `_ requesting an invite. - -HowToFit --------- - -For users less familiar with Bayesian inference and scientific analysis you may wish to read through -the **HowToFits** lectures. These teach you the basic principles of Bayesian inference, with the -content pitched at undergraduate level and above. - -The lectures are available in the `standalone HowToFit repository `_. - -API Overview ------------- - -To illustrate the **PyAutoFit** API, we use an illustrative toy model of fitting a one-dimensional Gaussian to -noisy 1D data. Here's the ``data`` (black) and the model (red) we'll fit: - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/files/toy_model_fit.png - :width: 400 - -We define our model, a 1D Gaussian by writing a Python class using the format below: - -.. code-block:: python - - class Gaussian: - - def __init__( - self, - centre=0.0, # <- PyAutoFit recognises these - normalization=0.1, # <- constructor arguments are - sigma=0.01, # <- the Gaussian's parameters. - ): - self.centre = centre - self.normalization = normalization - self.sigma = sigma - - """ - An instance of the Gaussian class will be available during model fitting. - - This method will be used to fit the model to data and compute a likelihood. - """ - - def model_data_from(self, xvalues): - - transformed_xvalues = xvalues - self.centre - - return (self.normalization / (self.sigma * (2.0 * np.pi) ** 0.5)) * \ - np.exp(-0.5 * (transformed_xvalues / self.sigma) ** 2.0) - -**PyAutoFit** recognises that this Gaussian may be treated as a model component whose parameters can be fitted for via -a non-linear search like `emcee `_. - -To fit this Gaussian to the ``data`` we create an Analysis object, which gives **PyAutoFit** the ``data`` and a -``log_likelihood_function`` describing how to fit the ``data`` with the model: - -.. code-block:: python - - class Analysis(af.Analysis): - - def __init__(self, data, noise_map): - - self.data = data - self.noise_map = noise_map - - def log_likelihood_function(self, instance): - - """ - The 'instance' that comes into this method is an instance of the Gaussian class - above, with the parameters set to values chosen by the non-linear search. - """ - - print("Gaussian Instance:") - print("Centre = ", instance.centre) - print("normalization = ", instance.normalization) - print("Sigma = ", instance.sigma) - - """ - We fit the ``data`` with the Gaussian instance, using its - "model_data_from" function to create the model data. - """ - - xvalues = np.arange(self.data.shape[0]) - - model_data = instance.model_data_from(xvalues=xvalues) - residual_map = self.data - model_data - chi_squared_map = (residual_map / self.noise_map) ** 2.0 - log_likelihood = -0.5 * sum(chi_squared_map) - - return log_likelihood - -We can now fit our model to the ``data`` using a non-linear search: - -.. code-block:: python - - model = af.Model(Gaussian) - - analysis = Analysis(data=data, noise_map=noise_map) - - emcee = af.Emcee(nwalkers=50, nsteps=2000) - - result = emcee.fit(model=model, analysis=analysis) - -The ``result`` contains information on the model-fit, for example the parameter samples, maximum log likelihood -model and marginalized probability density functions. - -.. toctree:: - :caption: Overview: - :maxdepth: 1 - :hidden: - - overview/the_basics - overview/scientific_workflow - overview/statistical_methods - -.. toctree:: - :caption: Cookbooks: - :maxdepth: 1 - :hidden: - - cookbooks/model - cookbooks/analysis - cookbooks/search - cookbooks/result - cookbooks/samples - cookbooks/configs - cookbooks/multiple_datasets - cookbooks/multi_level_model - -.. toctree:: - :caption: Features: - :maxdepth: 1 - :hidden: - - features/graphical - features/interpolate - features/search_chaining - features/search_grid_search - features/sensitivity_mapping - -.. toctree:: - :caption: Installation: - :maxdepth: 1 - :hidden: - - installation/overview - installation/conda - installation/pip - installation/source - installation/troubleshooting - -.. toctree:: - :caption: General: - :maxdepth: 1 - :hidden: - - general/workspace - general/configs - general/roadmap - general/software - general/citations - general/credits - -.. toctree:: - :caption: Science Examples: - :maxdepth: 1 - :hidden: - - science_examples/astronomy - -.. toctree:: - :caption: API Reference: - :maxdepth: 1 - :hidden: - - api/model - api/priors - api/analysis - api/searches - api/plot - api/samples - api/database - api/source - diff --git a/docs/installation/conda.md b/docs/installation/conda.md new file mode 100644 index 000000000..3d8d940e7 --- /dev/null +++ b/docs/installation/conda.md @@ -0,0 +1,43 @@ +(conda)= + +# Installation with conda + +Installation via a conda environment circumvents compatibility issues when installing certain libraries. This guide +assumes you have a working installation of [conda](https://conda.io/miniconda.html). + +First, create a conda environment (we name is `autofit` to signify it is for the **PyAutoFit** install). + +The command below creates this environment with some of the bigger package requirements, the rest will be installed +with **PyAutoFit** via pip: + +```bash +conda create -n autofit numpy scipy +``` + +Activate the conda environment (you will have to do this every time you want to run **PyAutoFit**): + +```bash +conda activate autofit +``` + +The latest version of **PyAutoFit** is installed via pip as follows (specifying the version as shown below ensures +the installation has clean dependencies): + +```bash +pip install autofit +``` + +Next, clone the `autofit_workspace` (the line `--depth 1` clones only the most recent branch on +the `autofit_workspace`, reducing the download size): + +```bash +cd /path/on/your/computer/you/want/to/put/the/autofit_workspace +git clone https://github.com/Jammy2211/autofit_workspace --depth 1 +cd autofit_workspace +``` + +Run the `welcome.py` script to get started! + +```bash +python3 welcome.py +``` diff --git a/docs/installation/conda.rst b/docs/installation/conda.rst deleted file mode 100644 index 6fefea8d7..000000000 --- a/docs/installation/conda.rst +++ /dev/null @@ -1,44 +0,0 @@ -.. _conda: - -Installation with conda -======================= - -Installation via a conda environment circumvents compatibility issues when installing certain libraries. This guide -assumes you have a working installation of `conda `_. - -First, create a conda environment (we name is ``autofit`` to signify it is for the **PyAutoFit** install). - -The command below creates this environment with some of the bigger package requirements, the rest will be installed -with **PyAutoFit** via pip: - -.. code-block:: bash - - conda create -n autofit numpy scipy - -Activate the conda environment (you will have to do this every time you want to run **PyAutoFit**): - -.. code-block:: bash - - conda activate autofit - -The latest version of **PyAutoFit** is installed via pip as follows (specifying the version as shown below ensures -the installation has clean dependencies): - -.. code-block:: bash - - pip install autofit - -Next, clone the ``autofit_workspace`` (the line ``--depth 1`` clones only the most recent branch on -the ``autofit_workspace``, reducing the download size): - -.. code-block:: bash - - cd /path/on/your/computer/you/want/to/put/the/autofit_workspace - git clone https://github.com/Jammy2211/autofit_workspace --depth 1 - cd autofit_workspace - -Run the `welcome.py` script to get started! - -.. code-block:: bash - - python3 welcome.py \ No newline at end of file diff --git a/docs/installation/overview.md b/docs/installation/overview.md new file mode 100644 index 000000000..e89a4f3b1 --- /dev/null +++ b/docs/installation/overview.md @@ -0,0 +1,41 @@ +(overview)= + +# Overview + +**PyAutoFit** requires Python 3.12 - 3.13 and supports the Linux, MacOS and Windows operating systems. + +**PyAutoFit** can be installed via the Python distribution [Anaconda](https://www.anaconda.com/) or using +[Pypi](https://pypi.org/) to `pip install` **PyAutoFit** into your Python distribution. + +We recommend Anaconda as it manages the installation of many major libraries used by **PyAutoFit** (e.g. numpy, scipy, +matplotlib, etc.) making installation more straight forward. + +The installation guide for both approaches can be found at: + +- [Anaconda installation guide](https://pyautofit.readthedocs.io/en/latest/installation/conda.html) +- [PyPI installation guide](https://pyautofit.readthedocs.io/en/latest/installation/pip.html) + +Users who wish to build **PyAutoFit** from source (e.g. via a `git clone`) should follow +our [building from source installation guide](https://pyautofit.readthedocs.io/en/latest/installation/source.html). + +## Known Issues + +There are currently no known issues with installing **PyAutoFit**. + +## Dependencies + +**PyAutoConf** + +**dynesty** + +**emcee** + +**astropy** + +**corner.py** + +**matplotlib** + +**numpy** + +**scipy** diff --git a/docs/installation/overview.rst b/docs/installation/overview.rst deleted file mode 100644 index 6e3ec7c42..000000000 --- a/docs/installation/overview.rst +++ /dev/null @@ -1,45 +0,0 @@ -.. _overview: - -Overview -======== - -**PyAutoFit** requires Python 3.12 - 3.13 and supports the Linux, MacOS and Windows operating systems. - -**PyAutoFit** can be installed via the Python distribution `Anaconda `_ or using -`Pypi `_ to ``pip install`` **PyAutoFit** into your Python distribution. - -We recommend Anaconda as it manages the installation of many major libraries used by **PyAutoFit** (e.g. numpy, scipy, -matplotlib, etc.) making installation more straight forward. - -The installation guide for both approaches can be found at: - -- `Anaconda installation guide `_ - -- `PyPI installation guide `_ - -Users who wish to build **PyAutoFit** from source (e.g. via a ``git clone``) should follow -our `building from source installation guide `_. - -Known Issues ------------- - -There are currently no known issues with installing **PyAutoFit**. - -Dependencies ------------- - -**PyAutoConf** https://github.com/rhayes777/PyAutoConf - -**dynesty** https://github.com/joshspeagle/dynesty - -**emcee** https://github.com/dfm/emcee - -**astropy** https://www.astropy.org/ - -**corner.py** https://github.com/dfm/corner.py - -**matplotlib** https://matplotlib.org/ - -**numpy** https://numpy.org/ - -**scipy** https://www.scipy.org/ \ No newline at end of file diff --git a/docs/installation/pip.md b/docs/installation/pip.md new file mode 100644 index 000000000..cd5e056c3 --- /dev/null +++ b/docs/installation/pip.md @@ -0,0 +1,54 @@ +(pip)= + +# Installation with pip + +:::{note} +**PyAutoFit** requires **Python 3.12 or later**. If you are on Python +3.9, 3.10, or 3.11, `pip install autofit` will fail with a "no matching +distribution" error. Upgrade Python to 3.12+ before installing. +::: + +We strongly recommend that you install **PyAutoFit** in a +[Python virtual environment](https://www.geeksforgeeks.org/python-virtual-environment/), with the link attached +describing what a virtual environment is and how to create one. + +The latest version of **PyAutoFit** is installed via pip as follows (specifying the version as shown below ensures +the installation has clean dependencies): + +```bash +pip install autofit +``` + +If this raises no errors **PyAutoFit** is installed! If there is an error check out +the [troubleshooting section](https://pyautofit.readthedocs.io/en/latest/installation/troubleshooting.html). + +Next, clone the `autofit_workspace` (the line `--depth 1` clones only the most recent branch on +the `autofit_workspace`, reducing the download size): + +```bash +cd /path/on/your/computer/you/want/to/put/the/autofit_workspace +git clone https://github.com/Jammy2211/autofit_workspace --depth 1 +cd autofit_workspace +``` + +Run the `welcome.py` script to get started! + +```bash +python3 welcome.py +``` + +## Legacy Python versions + +We dropped support for Python 3.9, 3.10, and 3.11 in release `2026.4.5.3` +(April 2026). Pre-`2026.4.5.3` releases on PyPI have been yanked, so they +will not install via the standard `pip install autofit` command. + +If you have an existing project that requires a pre-`2026.4.5.3` version, +you can still install it explicitly by pinning the version, e.g.: + +```bash +pip install autofit==2025.10.6.1 +``` + +Yanked releases remain available for explicit pins; only resolver-driven +fallback is blocked. diff --git a/docs/installation/pip.rst b/docs/installation/pip.rst deleted file mode 100644 index 3ecb3e9ae..000000000 --- a/docs/installation/pip.rst +++ /dev/null @@ -1,55 +0,0 @@ -.. _pip: - -Installation with pip -===================== - -.. note:: - **PyAutoFit** requires **Python 3.12 or later**. If you are on Python - 3.9, 3.10, or 3.11, ``pip install autofit`` will fail with a "no matching - distribution" error. Upgrade Python to 3.12+ before installing. - -We strongly recommend that you install **PyAutoFit** in a -`Python virtual environment `_, with the link attached -describing what a virtual environment is and how to create one. - -The latest version of **PyAutoFit** is installed via pip as follows (specifying the version as shown below ensures -the installation has clean dependencies): - -.. code-block:: bash - - pip install autofit - -If this raises no errors **PyAutoFit** is installed! If there is an error check out -the `troubleshooting section `_. - -Next, clone the ``autofit_workspace`` (the line ``--depth 1`` clones only the most recent branch on -the ``autofit_workspace``, reducing the download size): - -.. code-block:: bash - - cd /path/on/your/computer/you/want/to/put/the/autofit_workspace - git clone https://github.com/Jammy2211/autofit_workspace --depth 1 - cd autofit_workspace - -Run the ``welcome.py`` script to get started! - -.. code-block:: bash - - python3 welcome.py - -Legacy Python versions ----------------------- - -We dropped support for Python 3.9, 3.10, and 3.11 in release ``2026.4.5.3`` -(April 2026). Pre-``2026.4.5.3`` releases on PyPI have been yanked, so they -will not install via the standard ``pip install autofit`` command. - -If you have an existing project that requires a pre-``2026.4.5.3`` version, -you can still install it explicitly by pinning the version, e.g.: - -.. code-block:: bash - - pip install autofit==2025.10.6.1 - -Yanked releases remain available for explicit pins; only resolver-driven -fallback is blocked. \ No newline at end of file diff --git a/docs/installation/source.md b/docs/installation/source.md new file mode 100644 index 000000000..7e4c5ca3b --- /dev/null +++ b/docs/installation/source.md @@ -0,0 +1,45 @@ +(source)= + +# Building From Source + +Building from source means that you clone (or fork) the **PyAutoFit** GitHub repository and run **PyAutoFit** from +there. Unlike `conda` and `pip` this provides a build of the source code that you can edit and change, to +contribute the development **PyAutoFit** or experiment with yourself! + +First, clone (or fork) the **PyAutoFit** GitHub repository: + +```bash +git clone https://github.com/Jammy2211/PyAutoFit +``` + +Next, install the **PyAutoFit** dependencies via pip: + +```bash +pip install -r PyAutoFit/requirements.txt +``` + +If you are using a `conda` environment, add the source repository as follows: + +```bash +conda-develop PyAutoFit +``` + +Alternatively, if you are using a Python environment include the **PyAutoFit** source repository in your PYTHONPATH +(noting that you must replace the text `/path/to` with the path to the **PyAutoFit** directory on your computer): + +```bash +export PYTHONPATH=$PYTHONPATH:/path/to/PyAutoFit +``` + +For unit tests to pass you will also need the following optional requirements: + +```bash +pip install -r PyAutoFit/optional_requirements.txt +``` + +Finally, check the **PyAutoFit** unit tests run and pass (you may need to install pytest via `pip install pytest`): + +```bash +cd /path/to/PyAutoFit +python3 -m pytest +``` diff --git a/docs/installation/source.rst b/docs/installation/source.rst deleted file mode 100644 index dd3205d2c..000000000 --- a/docs/installation/source.rst +++ /dev/null @@ -1,46 +0,0 @@ -.. _source: - -Building From Source -==================== - -Building from source means that you clone (or fork) the **PyAutoFit** GitHub repository and run **PyAutoFit** from -there. Unlike ``conda`` and ``pip`` this provides a build of the source code that you can edit and change, to -contribute the development **PyAutoFit** or experiment with yourself! - -First, clone (or fork) the **PyAutoFit** GitHub repository: - -.. code-block:: bash - - git clone https://github.com/Jammy2211/PyAutoFit - -Next, install the **PyAutoFit** dependencies via pip: - -.. code-block:: bash - - pip install -r PyAutoFit/requirements.txt - -If you are using a ``conda`` environment, add the source repository as follows: - -.. code-block:: bash - - conda-develop PyAutoFit - -Alternatively, if you are using a Python environment include the **PyAutoFit** source repository in your PYTHONPATH -(noting that you must replace the text ``/path/to`` with the path to the **PyAutoFit** directory on your computer): - -.. code-block:: bash - - export PYTHONPATH=$PYTHONPATH:/path/to/PyAutoFit - -For unit tests to pass you will also need the following optional requirements: - -.. code-block:: bash - - pip install -r PyAutoFit/optional_requirements.txt - -Finally, check the **PyAutoFit** unit tests run and pass (you may need to install pytest via ``pip install pytest``): - -.. code-block:: bash - - cd /path/to/PyAutoFit - python3 -m pytest \ No newline at end of file diff --git a/docs/installation/troubleshooting.md b/docs/installation/troubleshooting.md new file mode 100644 index 000000000..232c9cef2 --- /dev/null +++ b/docs/installation/troubleshooting.md @@ -0,0 +1,29 @@ +(troubleshooting)= + +# Troubleshooting + +## Current Working Directory + +**PyAutoFit** scripts assume that the `autofit_workspace` directory is the Python working directory. This means +that, when you run an example script, you should run it from the `autofit_workspace` as follows: + +```bash +cd path/to/autofit_workspace (if you are not already in the autofit_workspace). +python3 scripts/overview/simple/fit.py +``` + +The reasons for this are so that **PyAutoFit** can: + +> - Load configuration settings from config files in the `autofit_workspace/config` folder. +> - Load example data from the `autofit_workspace/dataset` folder. +> - Output the results of models fits to your hard-disk to the `autofit/output` folder. +> - Import modules from the `autofit_workspace`, for example `from autofit_workspace.transdimensional import pipelines`. + +If you have any errors relating to importing modules, loading data or outputting results it is likely because you +are not running the script with the `autofit_workspace` as the working directory! + +## Support + +If you are still having issues with installation or using **PyAutoFit** in general, please raise an issue on the +[autofit_workspace issues page](https://github.com/Jammy2211/autofit_workspace/issues) with a description of the +problem and your system setup (operating system, Python version, etc.). diff --git a/docs/installation/troubleshooting.rst b/docs/installation/troubleshooting.rst deleted file mode 100644 index 3b2ca7b54..000000000 --- a/docs/installation/troubleshooting.rst +++ /dev/null @@ -1,32 +0,0 @@ -.. _troubleshooting: - -Troubleshooting -=============== - -Current Working Directory -------------------------- - -**PyAutoFit** scripts assume that the ``autofit_workspace`` directory is the Python working directory. This means -that, when you run an example script, you should run it from the ``autofit_workspace`` as follows: - -.. code-block:: bash - - cd path/to/autofit_workspace (if you are not already in the autofit_workspace). - python3 scripts/overview/simple/fit.py - -The reasons for this are so that **PyAutoFit** can: - - - Load configuration settings from config files in the ``autofit_workspace/config`` folder. - - Load example data from the ``autofit_workspace/dataset`` folder. - - Output the results of models fits to your hard-disk to the ``autofit/output`` folder. - - Import modules from the ``autofit_workspace``, for example ``from autofit_workspace.transdimensional import pipelines``. - -If you have any errors relating to importing modules, loading data or outputting results it is likely because you -are not running the script with the ``autofit_workspace`` as the working directory! - -Support -------- - -If you are still having issues with installation or using **PyAutoFit** in general, please raise an issue on the -`autofit_workspace issues page `_ with a description of the -problem and your system setup (operating system, Python version, etc.). \ No newline at end of file diff --git a/docs/overview/backup.md b/docs/overview/backup.md new file mode 100644 index 000000000..fb179fa37 --- /dev/null +++ b/docs/overview/backup.md @@ -0,0 +1,377 @@ +# Extending Models + +The model composition API is designed to make composing complex models, consisting of multiple components with many +free parameters, straightforward and scalable. + +To illustrate this, we will extend our model to include a second component, representing a symmetric 1D Exponential +profile, and fit it to data generated with both profiles. + +Lets begin by loading and plotting this data. + +```python +dataset_path = path.join("dataset", "example_1d", "gaussian_x1__exponential_x1") +data = af.util.numpy_array_from_json(file_path=path.join(dataset_path, "data.json")) +noise_map = af.util.numpy_array_from_json( + file_path=path.join(dataset_path, "noise_map.json") +) +xvalues = range(data.shape[0]) +plt.errorbar( + x=xvalues, y=data, yerr=noise_map, color="k", ecolor="k", elinewidth=1, capsize=2 +) +plt.show() +plt.close() +``` + +The data appear as follows: + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/images/data_2.png +:alt: Alternative text +:width: 600 +``` + +We define a Python class for the `Exponential` model component, exactly as we did for the `Gaussian` above. + +```python +class Exponential: + def __init__( + self, + centre=30.0, # <- **PyAutoFit** recognises these constructor arguments + normalization=1.0, # <- are the Exponentials``s model parameters. + rate=0.01, + ): + """ + Represents a symmetric 1D Exponential profile. + + Parameters + ---------- + centre + The x coordinate of the profile centre. + normalization + Overall normalization of the profile. + ratw + The decay rate controlling has fast the Exponential declines. + """ + + self.centre = centre + self.normalization = normalization + self.rate = rate + + def model_data_from(self, xvalues: np.ndarray): + """ + Returns the symmetric 1D Exponential on an input list of Cartesian + x coordinates. + + The input xvalues are translated to a coordinate system centred on + the Exponential, via its ``centre``. + + The output is referred to as the ``model_data`` to signify that it + is a representation of the data from the + model. + + Parameters + ---------- + xvalues + The x coordinates in the original reference frame of the data. + """ + + transformed_xvalues = np.subtract(xvalues, self.centre) + return self.normalization * np.multiply( + self.rate, np.exp(-1.0 * self.rate * abs(transformed_xvalues)) + ) +``` + +We can easily compose a model consisting of 1 `Gaussian` object and 1 `Exponential` object using the `af.Collection` +object: + +```python +model = af.Collection(gaussian=af.Model(Gaussian), exponential=af.Model(Exponential)) +``` + +A `Collection` behaves analogous to a `Model`, but it contains a multiple model components. + +We can see this by printing its `paths` attribute, where paths to all 6 free parameters via both model components +are shown. + +The paths have the entries `.gaussian.` and `.exponential.`, which correspond to the names we input into +the `af.Collection` above. + +```python +print(model.paths) +``` + +The output is as follows: + +```bash +[ + ('gaussian', 'centre'), + ('gaussian', 'normalization'), + ('gaussian', 'sigma'), + ('exponential', 'centre'), + ('exponential', 'normalization'), + ('exponential', 'rate') +] +``` + +We can use the paths to customize the priors of each parameter. + +```python +model.gaussian.centre = af.UniformPrior(lower_limit=0.0, upper_limit=100.0) +model.gaussian.normalization = af.UniformPrior(lower_limit=0.0, upper_limit=1e2) +model.gaussian.sigma = af.UniformPrior(lower_limit=0.0, upper_limit=30.0) +model.exponential.centre = af.UniformPrior(lower_limit=0.0, upper_limit=100.0) +model.exponential.normalization = af.UniformPrior(lower_limit=0.0, upper_limit=1e2) +model.exponential.rate = af.UniformPrior(lower_limit=0.0, upper_limit=10.0) +``` + +All of the information about the model created via the collection can be printed at once using its `info` attribute: + +```python +print(model.info) +``` + +The output appears as follows: + +```bash +Total Free Parameters = 6 +model Collection (N=6) + gaussian Gaussian (N=3) + exponential Exponential (N=3) + + gaussian + centre UniformPrior [13], lower_limit = 0.0, upper_limit = 100.0 + normalization UniformPrior [14], lower_limit = 0.0, upper_limit = 100.0 + sigma UniformPrior [15], lower_limit = 0.0, upper_limit = 30.0 + exponential + centre UniformPrior [16], lower_limit = 0.0, upper_limit = 100.0 + normalization UniformPrior [17], lower_limit = 0.0, upper_limit = 100.0 + rate UniformPrior [18], lower_limit = 0.0, upper_limit = 10.0 +``` + +A model instance can again be created by mapping an input `vector`, which now has 6 entries. + +```python +instance = model.instance_from_vector(vector=[0.1, 0.2, 0.3, 0.4, 0.5, 0.01]) +``` + +This `instance` contains each of the model components we defined above. + +The argument names input into the `Collection` define the attribute names of the `instance`: + +```python +print("Instance Parameters \n") +print("x (Gaussian) = ", instance.gaussian.centre) +print("normalization (Gaussian) = ", instance.gaussian.normalization) +print("sigma (Gaussian) = ", instance.gaussian.sigma) +print("x (Exponential) = ", instance.exponential.centre) +print("normalization (Exponential) = ", instance.exponential.normalization) +print("sigma (Exponential) = ", instance.exponential.rate) +``` + +The output appear as follows: + +```bash + +``` + +The `Analysis` class above assumed the `instance` contained only a single model-component. + +We update its `log_likelihood_function` to use both model components in the `instance` to fit the data. + +```python +class Analysis(af.Analysis): + def __init__(self, data: np.ndarray, noise_map: np.ndarray): + """ + The `Analysis` class acts as an interface between the data and + model in **PyAutoFit**. + + Its `log_likelihood_function` defines how the model is fitted to + the data and it is called many times by the non-linear search + fitting algorithm. + + In this example the `Analysis` `__init__` constructor only + contains the `data` and `noise-map`, but it can be easily + extended to include other quantities. + + Parameters + ---------- + data + A 1D numpy array containing the data (e.g. a noisy 1D signal) + fitted in the workspace examples. + noise_map + A 1D numpy array containing the noise values of the data, + used for computing the goodness of fit metric, the log likelihood. + """ + + super().__init__() + + self.data = data + self.noise_map = noise_map + + def log_likelihood_function(self, instance) -> float: + """ + Returns the log likelihood of a fit of a 1D Gaussian to the dataset. + + The data is fitted using an `instance` of multiple 1D profiles + (e.g. a `Gaussian`, `Exponential`) where + their `model_data_from` methods are called and summed + in order to create a model data representation that is fitted to the data. + """ + + """ + The `instance` that comes into this method is an instance of the + `Gaussian` and `Exponential` models above, which were created + via `af.Collection()`. + + It contains instances of every class we instantiated it with, where + each instance is named following the names given to the Collection, + which in this example is a `Gaussian` (with name `gaussian) and + Exponential (with name `exponential`). + + The parameter values are again chosen by the non-linear search, + based on where it thinks the high likelihood regions of parameter + space are. The lines of Python code are commented out below to + prevent excessive print statements. + + + # print("Gaussian Instance:") + # print("Centre = ", instance.gaussian.centre) + # print("Normalization = ", instance.gaussian.normalization) + # print("Sigma = ", instance.gaussian.sigma) + + # print("Exponential Instance:") + # print("Centre = ", instance.exponential.centre) + # print("Normalization = ", instance.exponential.normalization) + # print("Rate = ", instance.exponential.rate) + """ + """ + Get the range of x-values the data is defined on, to evaluate + the model of the Gaussian. + """ + xvalues = np.arange(self.data.shape[0]) + + """ + Internally, the `instance` variable is a list of all model + omponents pass to the `Collection` above. + + we can therefore iterate over them and use their + `model_data_from` methods to create the + summed overall model data. + """ + model_data = sum( + [ + profile_1d.model_data_from(xvalues=xvalues) + for profile_1d in instance + ] + ) + + """ + Fit the model gaussian line data to the observed data, computing the residuals, chi-squared and log likelihood. + """ + residual_map = self.data - model_data + chi_squared_map = (residual_map / self.noise_map) ** 2.0 + chi_squared = sum(chi_squared_map) + noise_normalization = np.sum(np.log(2 * np.pi * noise_map**2.0)) + log_likelihood = -0.5 * (chi_squared + noise_normalization) + + return log_likelihood +``` + +We can now fit this model to the data using the same API we did before. + +```python +analysis = Analysis(data=data, noise_map=noise_map) + +search = af.DynestyStatic( + nlive=100, + number_of_cores=1, +) + +result = search.fit(model=model, analysis=analysis) +``` + +The `info` attribute shows the result in a readable format, showing that all 6 free parameters were fitted for. + +```python +print(result.info) +``` + +The output appears as follows: + +```bash +Bayesian Evidence 144.86032973 +Maximum Log Likelihood 181.14287034 +Maximum Log Posterior 181.14287034 + +model Collection (N=6) + gaussian Gaussian (N=3) + exponential Exponential (N=3) + +Maximum Log Likelihood Model: + +gaussian + centre 50.223 + normalization 26.108 + sigma 9.710 +exponential + centre 50.057 + normalization 39.948 + rate 0.048 + + +Summary (3.0 sigma limits): + +gaussian + centre 50.27 (49.63, 50.88) + normalization 26.22 (21.37, 32.41) + sigma 9.75 (9.25, 10.27) +exponential + centre 50.04 (49.60, 50.50) + normalization 40.06 (37.60, 42.38) + rate 0.05 (0.04, 0.05) + + +Summary (1.0 sigma limits): + +gaussian + centre 50.27 (50.08, 50.49) + normalization 26.22 (24.33, 28.39) + sigma 9.75 (9.60, 9.90) +exponential + centre 50.04 (49.90, 50.18) + normalization 40.06 (39.20, 40.88) + rate 0.05 (0.05, 0.05) +``` + +We can again use the max log likelihood instance to visualize the model data of the best fit model compared to the +data. + +```python +instance = result.max_log_likelihood_instance + +model_gaussian = instance.gaussian.model_data_from( + xvalues=np.arange(data.shape[0]) +) +model_exponential = instance.exponential.model_data_from( + xvalues=np.arange(data.shape[0]) +) +model_data = model_gaussian + model_exponential + +plt.errorbar( + x=xvalues, y=data, yerr=noise_map, color="k", ecolor="k", elinewidth=1, capsize=2 +) +plt.plot(range(data.shape[0]), model_data, color="r") +plt.plot(range(data.shape[0]), model_gaussian, "--") +plt.plot(range(data.shape[0]), model_exponential, "--") +plt.title("Dynesty model fit to 1D Gaussian + Exponential dataset.") +plt.xlabel("x values of profile") +plt.ylabel("Profile normalization") +plt.show() +plt.close() +``` + +The plot appears as follows: + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/images/toy_model_fit.png +:alt: Alternative text +:width: 600 +``` diff --git a/docs/overview/backup.rst b/docs/overview/backup.rst deleted file mode 100644 index b680bc235..000000000 --- a/docs/overview/backup.rst +++ /dev/null @@ -1,380 +0,0 @@ - -Extending Models ----------------- - -The model composition API is designed to make composing complex models, consisting of multiple components with many -free parameters, straightforward and scalable. - -To illustrate this, we will extend our model to include a second component, representing a symmetric 1D Exponential -profile, and fit it to data generated with both profiles. - -Lets begin by loading and plotting this data. - -.. code-block:: python - - dataset_path = path.join("dataset", "example_1d", "gaussian_x1__exponential_x1") - data = af.util.numpy_array_from_json(file_path=path.join(dataset_path, "data.json")) - noise_map = af.util.numpy_array_from_json( - file_path=path.join(dataset_path, "noise_map.json") - ) - xvalues = range(data.shape[0]) - plt.errorbar( - x=xvalues, y=data, yerr=noise_map, color="k", ecolor="k", elinewidth=1, capsize=2 - ) - plt.show() - plt.close() - -The data appear as follows: - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/images/data_2.png - :width: 600 - :alt: Alternative text - -We define a Python class for the ``Exponential`` model component, exactly as we did for the ``Gaussian`` above. - -.. code-block:: python - - class Exponential: - def __init__( - self, - centre=30.0, # <- **PyAutoFit** recognises these constructor arguments - normalization=1.0, # <- are the Exponentials``s model parameters. - rate=0.01, - ): - """ - Represents a symmetric 1D Exponential profile. - - Parameters - ---------- - centre - The x coordinate of the profile centre. - normalization - Overall normalization of the profile. - ratw - The decay rate controlling has fast the Exponential declines. - """ - - self.centre = centre - self.normalization = normalization - self.rate = rate - - def model_data_from(self, xvalues: np.ndarray): - """ - Returns the symmetric 1D Exponential on an input list of Cartesian - x coordinates. - - The input xvalues are translated to a coordinate system centred on - the Exponential, via its ``centre``. - - The output is referred to as the ``model_data`` to signify that it - is a representation of the data from the - model. - - Parameters - ---------- - xvalues - The x coordinates in the original reference frame of the data. - """ - - transformed_xvalues = np.subtract(xvalues, self.centre) - return self.normalization * np.multiply( - self.rate, np.exp(-1.0 * self.rate * abs(transformed_xvalues)) - ) - - -We can easily compose a model consisting of 1 ``Gaussian`` object and 1 ``Exponential`` object using the ``af.Collection`` -object: - -.. code-block:: python - - model = af.Collection(gaussian=af.Model(Gaussian), exponential=af.Model(Exponential)) - -A ``Collection`` behaves analogous to a ``Model``, but it contains a multiple model components. - -We can see this by printing its ``paths`` attribute, where paths to all 6 free parameters via both model components -are shown. - -The paths have the entries ``.gaussian.`` and ``.exponential.``, which correspond to the names we input into -the ``af.Collection`` above. - -.. code-block:: python - - print(model.paths) - -The output is as follows: - -.. code-block:: bash - - [ - ('gaussian', 'centre'), - ('gaussian', 'normalization'), - ('gaussian', 'sigma'), - ('exponential', 'centre'), - ('exponential', 'normalization'), - ('exponential', 'rate') - ] - -We can use the paths to customize the priors of each parameter. - -.. code-block:: python - - model.gaussian.centre = af.UniformPrior(lower_limit=0.0, upper_limit=100.0) - model.gaussian.normalization = af.UniformPrior(lower_limit=0.0, upper_limit=1e2) - model.gaussian.sigma = af.UniformPrior(lower_limit=0.0, upper_limit=30.0) - model.exponential.centre = af.UniformPrior(lower_limit=0.0, upper_limit=100.0) - model.exponential.normalization = af.UniformPrior(lower_limit=0.0, upper_limit=1e2) - model.exponential.rate = af.UniformPrior(lower_limit=0.0, upper_limit=10.0) - -All of the information about the model created via the collection can be printed at once using its ``info`` attribute: - -.. code-block:: python - - print(model.info) - -The output appears as follows: - -.. code-block:: bash - - Total Free Parameters = 6 - model Collection (N=6) - gaussian Gaussian (N=3) - exponential Exponential (N=3) - - gaussian - centre UniformPrior [13], lower_limit = 0.0, upper_limit = 100.0 - normalization UniformPrior [14], lower_limit = 0.0, upper_limit = 100.0 - sigma UniformPrior [15], lower_limit = 0.0, upper_limit = 30.0 - exponential - centre UniformPrior [16], lower_limit = 0.0, upper_limit = 100.0 - normalization UniformPrior [17], lower_limit = 0.0, upper_limit = 100.0 - rate UniformPrior [18], lower_limit = 0.0, upper_limit = 10.0 - - -A model instance can again be created by mapping an input ``vector``, which now has 6 entries. - -.. code-block:: python - - instance = model.instance_from_vector(vector=[0.1, 0.2, 0.3, 0.4, 0.5, 0.01]) - -This ``instance`` contains each of the model components we defined above. - -The argument names input into the ``Collection`` define the attribute names of the ``instance``: - -.. code-block:: python - - print("Instance Parameters \n") - print("x (Gaussian) = ", instance.gaussian.centre) - print("normalization (Gaussian) = ", instance.gaussian.normalization) - print("sigma (Gaussian) = ", instance.gaussian.sigma) - print("x (Exponential) = ", instance.exponential.centre) - print("normalization (Exponential) = ", instance.exponential.normalization) - print("sigma (Exponential) = ", instance.exponential.rate) - -The output appear as follows: - -.. code-block:: bash - -The ``Analysis`` class above assumed the ``instance`` contained only a single model-component. - -We update its ``log_likelihood_function`` to use both model components in the ``instance`` to fit the data. - -.. code-block:: python - - class Analysis(af.Analysis): - def __init__(self, data: np.ndarray, noise_map: np.ndarray): - """ - The `Analysis` class acts as an interface between the data and - model in **PyAutoFit**. - - Its `log_likelihood_function` defines how the model is fitted to - the data and it is called many times by the non-linear search - fitting algorithm. - - In this example the `Analysis` `__init__` constructor only - contains the `data` and `noise-map`, but it can be easily - extended to include other quantities. - - Parameters - ---------- - data - A 1D numpy array containing the data (e.g. a noisy 1D signal) - fitted in the workspace examples. - noise_map - A 1D numpy array containing the noise values of the data, - used for computing the goodness of fit metric, the log likelihood. - """ - - super().__init__() - - self.data = data - self.noise_map = noise_map - - def log_likelihood_function(self, instance) -> float: - """ - Returns the log likelihood of a fit of a 1D Gaussian to the dataset. - - The data is fitted using an `instance` of multiple 1D profiles - (e.g. a `Gaussian`, `Exponential`) where - their `model_data_from` methods are called and summed - in order to create a model data representation that is fitted to the data. - """ - - """ - The `instance` that comes into this method is an instance of the - `Gaussian` and `Exponential` models above, which were created - via `af.Collection()`. - - It contains instances of every class we instantiated it with, where - each instance is named following the names given to the Collection, - which in this example is a `Gaussian` (with name `gaussian) and - Exponential (with name `exponential`). - - The parameter values are again chosen by the non-linear search, - based on where it thinks the high likelihood regions of parameter - space are. The lines of Python code are commented out below to - prevent excessive print statements. - - - # print("Gaussian Instance:") - # print("Centre = ", instance.gaussian.centre) - # print("Normalization = ", instance.gaussian.normalization) - # print("Sigma = ", instance.gaussian.sigma) - - # print("Exponential Instance:") - # print("Centre = ", instance.exponential.centre) - # print("Normalization = ", instance.exponential.normalization) - # print("Rate = ", instance.exponential.rate) - """ - """ - Get the range of x-values the data is defined on, to evaluate - the model of the Gaussian. - """ - xvalues = np.arange(self.data.shape[0]) - - """ - Internally, the `instance` variable is a list of all model - omponents pass to the `Collection` above. - - we can therefore iterate over them and use their - `model_data_from` methods to create the - summed overall model data. - """ - model_data = sum( - [ - profile_1d.model_data_from(xvalues=xvalues) - for profile_1d in instance - ] - ) - - """ - Fit the model gaussian line data to the observed data, computing the residuals, chi-squared and log likelihood. - """ - residual_map = self.data - model_data - chi_squared_map = (residual_map / self.noise_map) ** 2.0 - chi_squared = sum(chi_squared_map) - noise_normalization = np.sum(np.log(2 * np.pi * noise_map**2.0)) - log_likelihood = -0.5 * (chi_squared + noise_normalization) - - return log_likelihood - - - -We can now fit this model to the data using the same API we did before. - -.. code-block:: python - - analysis = Analysis(data=data, noise_map=noise_map) - - search = af.DynestyStatic( - nlive=100, - number_of_cores=1, - ) - - result = search.fit(model=model, analysis=analysis) - - -The ``info`` attribute shows the result in a readable format, showing that all 6 free parameters were fitted for. - -.. code-block:: python - - print(result.info) - -The output appears as follows: - -.. code-block:: bash - - Bayesian Evidence 144.86032973 - Maximum Log Likelihood 181.14287034 - Maximum Log Posterior 181.14287034 - - model Collection (N=6) - gaussian Gaussian (N=3) - exponential Exponential (N=3) - - Maximum Log Likelihood Model: - - gaussian - centre 50.223 - normalization 26.108 - sigma 9.710 - exponential - centre 50.057 - normalization 39.948 - rate 0.048 - - - Summary (3.0 sigma limits): - - gaussian - centre 50.27 (49.63, 50.88) - normalization 26.22 (21.37, 32.41) - sigma 9.75 (9.25, 10.27) - exponential - centre 50.04 (49.60, 50.50) - normalization 40.06 (37.60, 42.38) - rate 0.05 (0.04, 0.05) - - - Summary (1.0 sigma limits): - - gaussian - centre 50.27 (50.08, 50.49) - normalization 26.22 (24.33, 28.39) - sigma 9.75 (9.60, 9.90) - exponential - centre 50.04 (49.90, 50.18) - normalization 40.06 (39.20, 40.88) - rate 0.05 (0.05, 0.05) - -We can again use the max log likelihood instance to visualize the model data of the best fit model compared to the -data. - -.. code-block:: python - - instance = result.max_log_likelihood_instance - - model_gaussian = instance.gaussian.model_data_from( - xvalues=np.arange(data.shape[0]) - ) - model_exponential = instance.exponential.model_data_from( - xvalues=np.arange(data.shape[0]) - ) - model_data = model_gaussian + model_exponential - - plt.errorbar( - x=xvalues, y=data, yerr=noise_map, color="k", ecolor="k", elinewidth=1, capsize=2 - ) - plt.plot(range(data.shape[0]), model_data, color="r") - plt.plot(range(data.shape[0]), model_gaussian, "--") - plt.plot(range(data.shape[0]), model_exponential, "--") - plt.title("Dynesty model fit to 1D Gaussian + Exponential dataset.") - plt.xlabel("x values of profile") - plt.ylabel("Profile normalization") - plt.show() - plt.close() - -The plot appears as follows: - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/images/toy_model_fit.png - :width: 600 - :alt: Alternative text \ No newline at end of file diff --git a/docs/overview/scientific_workflow.md b/docs/overview/scientific_workflow.md new file mode 100644 index 000000000..06f18aec4 --- /dev/null +++ b/docs/overview/scientific_workflow.md @@ -0,0 +1,608 @@ +(scientific-workflow)= + +# Scientific Workflow + +A scientific workflow comprises the tasks you perform to conduct a scientific study. This includes fitting models to +datasets, interpreting the results, and gaining insights into your scientific problem. + +Different problems require different scientific workflows, depending on factors such as model complexity, dataset size, +and computational run times. For example, some problems involve fitting a single dataset with many models to gain +scientific insights, while others involve fitting thousands of datasets with a single model for large-scale studies. + +The **PyAutoFit** API is flexible, customizable, and extensible, enabling users to develop scientific workflows +tailored to their specific problems. + +This overview covers the key features of **PyAutoFit** that support the development of effective scientific workflows: + +- **On The Fly**: Display results immediately (e.g., in Jupyter notebooks) to provide instant feedback for adapting your workflow. +- **Hard Disk Output**: Output results to hard disk with high customization, allowing quick and detailed inspection of fits to many datasets. +- **Visualization**: Generate model-specific visualizations to create custom plots that streamline result inspection. +- **Loading Results**: Load results from the hard disk to inspect and interpret the outcomes of a model fit. +- **Result Customization**: Customize the returned results to simplify scientific interpretation. +- **Model Composition**: Extensible model composition makes it easy to fit many models with different parameterizations and assumptions. +- **Searches**: Support for various non-linear searches (e.g., nested sampling, MCMC), including gradient based fitting using JAX, to find the right method for your problem. +- **Configs**: Configuration files that set default model, fitting, and visualization behaviors, streamlining model fitting. +- **Database**: Store results in a relational SQLite3 database, enabling efficient management of large modeling results. +- **Scaling Up**: Guidance on scaling up your scientific workflow from small to large datasets. + +## On The Fly + +:::{note} +The on-the-fly feature described below is not implemented yet, we are working on it currently. +The best way to get on-the-fly output is to output to hard-disk, which is described in the next section. +This feature is fully implemented and provides on-the-fly output of results to hard-disk. +::: + +When a model fit is running, information about the fit is displayed at user-specified intervals. + +The frequency of this on-the-fly output is controlled by a search's `iterations_per_full_update` parameter, which +specifies how often this information is output. The example code below outputs on-the-fly information every 1000 iterations: + +```python +search = af.DynestyStatic( + iterations_per_full_update=1000 +) +``` + +In a Jupyter notebook, the default behavior is for this information to appear in the cell being run and to include: + +- Text displaying the maximum likelihood model inferred so far and related information. +- A visual showing how the search has sampled parameter space so far, providing intuition on how the search is performing. + +Here is an image of how this looks: + +!\[Example On-the-Fly Output\](path/to/image.png) + +The most valuable on-the-fly output is often specific to the model and dataset you are fitting. For instance, it +might be a `matplotlib` subplot showing the maximum likelihood model's fit to the dataset, complete with residuals +and other diagnostic information. + +The on-the-fly output can be fully customized by extending the `on_the_fly_output` method of the `Analysis` +class being used to fit the model. + +The example below shows how this is done for the simple case of fitting a 1D Gaussian profile: + +```python +class Analysis(af.Analysis): + def __init__(self, data: np.ndarray, noise_map: np.ndarray): + """ + Example Analysis class illustrating how to customize the on-the-fly output of a model-fit. + """ + super().__init__() + + self.data = data + self.noise_map = noise_map + + def on_the_fly_output(self, instance): + """ + During a model-fit, the `on_the_fly_output` method is called throughout the non-linear search. + + The `instance` passed into the method is maximum log likelihood solution obtained by the model-fit so far and it can be + used to provide on-the-fly output showing how the model-fit is going. + """ + xvalues = np.arange(analysis.data.shape[0]) + + model_data = instance.model_data_from(xvalues=xvalues) + + """ + The visualizer now outputs images of the best-fit results to hard-disk (checkout `visualizer.py`). + """ + import matplotlib.pyplot as plt + + plt.errorbar( + x=xvalues, + y=self.data, + yerr=self.noise_map, + color="k", + ecolor="k", + elinewidth=1, + capsize=2, + ) + plt.plot(xvalues, model_data, color="r") + plt.title("Maximum Likelihood Fit") + plt.xlabel("x value of profile") + plt.ylabel("Profile Normalization") + plt.show() # By using `plt.show()` the plot will be displayed in the Jupyter notebook. +``` + +Here's how the visuals appear in a Jupyter Notebook: + +!\[Example On-the-Fly Output\](path/to/image.png) + +In the early stages of setting up a scientific workflow, on-the-fly output is invaluable. It provides immediate +feedback on how your model fitting is performing, which is often crucial at the beginning of a project when things +might not be going well. It also encourages you to prioritize visualizing your fit and diagnosing whether the process +is working correctly. + +We highly recommend users starting a new model-fitting problem begin by setting up on-the-fly output! + +## Hard Disk Output + +By default, a non-linear search does not save its results to the hard disk; the results can only be inspected in a Jupyter Notebook or Python script via the returned `result`. + +However, you can enable the output of non-linear search results to the hard disk by specifying the `name` and/or `path_prefix` attributes. These attributes determine how files are named and where results are saved on your hard disk. + +Benefits of saving results to the hard disk include: + +- More efficient inspection of results for multiple datasets compared to using a Jupyter Notebook. +- Results are saved on-the-fly, allowing you to check the progress of a fit midway. +- Additional information about a fit, such as visualizations, can be saved (see below). +- Unfinished runs can be resumed from where they left off if they are terminated. +- On high-performance supercomputers, results often need to be saved in this manner. + +Here's how to enable the output of results to the hard disk: + +```python +search = af.Emcee( + path_prefix=path.join("folder_0", "folder_1"), + name="my_search_name" +) +``` + +The screenshot below shows the output folder where all output is enabled: + +```{image} https://raw.githubusercontent.com/Jammy2211/PyAutoFit/main/docs/overview/image/output_example.png +:alt: Alternative text +:width: 400 +``` + +Let's break down the output folder generated by **PyAutoFit**: + +- **Unique Identifier**: Results are saved in a folder named with a unique identifier composed of random characters. This identifier is automatically generated based on the specific model fit. For scientific workflows involving numerous model fits, this ensures that each fit is uniquely identified without requiring manual updates to output paths. +- **Info Files**: These files contain valuable information about the fit. For instance, `model.info` provides the complete model composition used in the fit, while `search.summary` details how long the search has been running and other relevant search-specific information. +- **Files Folder**: Within the output folder, the `files` directory contains detailed information saved as `.json` files. For example, `model.json` stores the model configuration used in the fit. This enables researchers to revisit the results later and review how the fit was performed. + +**PyAutoFit** offers extensive tools for customizing hard-disk output. This includes using configuration files to control what information is saved, which helps manage disk space utilization. Additionally, specific `.json` files tailored to different models can be utilized for more detailed output. + +For many scientific workflows, having detailed output for each fit is crucial for thorough inspection and accurate +interpretation of results. However, in scenarios where the volume of output data might overwhelm users or impede +scientific study, this feature can be easily disabled by omitting the `name` or `path prefix` when initiating the search. + +## Visualization + +When search hard-disk output is enabled in **PyAutoFit**, the visualization of model fits can also be saved directly +to disk. This capability is crucial for many scientific workflows as it allows for quick and effective assessment of +fit quality. + +To accomplish this, you can customize the `Visualizer` object of an `Analysis` class with a custom `Visualizer` class. +This custom class is responsible for generating and saving visual representations of the model fits. By leveraging +this approach, scientists can efficiently visualize and analyze the outcomes of model fitting processes. + +```python +class Visualizer(af.Visualizer): + + @staticmethod + def visualize_before_fit( + analysis, + paths: af.DirectoryPaths, + model: af.AbstractPriorModel + ): + """ + Before a model-fit, the `visualize_before_fit` method is called to perform visualization. + + The function receives as input an instance of the `Analysis` class which is being used to perform the fit, + which is used to perform the visualization (e.g. it contains the data and noise map which are plotted). + + This can output visualization of quantities which do not change during the model-fit, for example the + data and noise-map. + + The `paths` object contains the path to the folder where the visualization should be output, which is determined + by the non-linear search `name` and other inputs. + """ + + import matplotlib.pyplot as plt + + xvalues = np.arange(self.data.shape[0]) + + plt.errorbar( + x=xvalues, + y=analysis.data, + yerr=analysis.noise_map, + color="k", + ecolor="k", + elinewidth=1, + capsize=2, + ) + plt.title("Maximum Likelihood Fit") + plt.xlabel("x value of profile") + plt.ylabel("Profile Normalization") + plt.savefig(path.join(paths.image_path, f"data.png")) + plt.clf() + + @staticmethod + def visualize( + analysis, + paths: af.DirectoryPaths, + instance, + during_analysis + ): + """ + During a model-fit, the `visualize` method is called throughout the non-linear search. + + The function receives as input an instance of the `Analysis` class which is being used to perform the fit, + which is used to perform the visualization (e.g. it generates the model data which is plotted). + + The `instance` passed into the visualize method is maximum log likelihood solution obtained by the model-fit + so far and it can be used to provide on-the-fly images showing how the model-fit is going. + + The `paths` object contains the path to the folder where the visualization should be output, which is determined + by the non-linear search `name` and other inputs. + """ + xvalues = np.arange(analysis.data.shape[0]) + + model_data = instance.model_data_from(xvalues=xvalues) + residual_map = analysis.data - model_data + + """ + The visualizer now outputs images of the best-fit results to hard-disk (checkout `visualizer.py`). + """ + import matplotlib.pyplot as plt + + plt.errorbar( + x=xvalues, + y=analysis.data, + yerr=analysis.noise_map, + color="k", + ecolor="k", + elinewidth=1, + capsize=2, + ) + plt.plot(xvalues, model_data, color="r") + plt.title("Maximum Likelihood Fit") + plt.xlabel("x value of profile") + plt.ylabel("Profile Normalization") + plt.savefig(path.join(paths.image_path, f"model_fit.png")) + plt.clf() + + plt.errorbar( + x=xvalues, + y=residual_map, + yerr=analysis.noise_map, + color="k", + ecolor="k", + elinewidth=1, + capsize=2, + ) + plt.title("Residuals of Maximum Likelihood Fit") + plt.xlabel("x value of profile") + plt.ylabel("Residual") + plt.savefig(path.join(paths.image_path, f"model_fit.png")) + plt.clf() +``` + +The `Analysis` class is defined following the same API as before, but now with its `Visualizer` class attribute +overwritten with the `Visualizer` class above. + +```python +class Analysis(af.Analysis): + + """ + This over-write means the `Visualizer` class is used for visualization throughout the model-fit. + + This `VisualizerExample` object is in the `autofit.example.visualize` module and is used to customize the + plots output during the model-fit. + + It has been extended with visualize methods that output visuals specific to the fitting of `1D` data. + """ + Visualizer = Visualizer + + def __init__(self, data, noise_map): + """ + An Analysis class which illustrates visualization. + """ + super().__init__() + + self.data = data + self.noise_map = noise_map + + def log_likelihood_function(self, instance): + """ + The `log_likelihood_function` is identical to the example above + """ + xvalues = np.arange(self.data.shape[0]) + + model_data = instance.model_data_from(xvalues=xvalues) + residual_map = self.data - model_data + chi_squared_map = (residual_map / self.noise_map) ** 2.0 + chi_squared = sum(chi_squared_map) + noise_normalization = np.sum(np.log(2 * np.pi * noise_map**2.0)) + log_likelihood = -0.5 * (chi_squared + noise_normalization) + + return log_likelihood +``` + +Visualization of the results of the non-linear search, for example the "Probability Density +Function", are also automatically output during the model-fit on the fly. + +## Loading Results + +In your scientific workflow, you'll likely conduct numerous model fits, each generating outputs stored in individual +folders on your hard disk. + +To efficiently work with these results in Python scripts or Jupyter notebooks, **PyAutoFit** provides +the `aggregator` API. This tool simplifies the process of loading results from hard disk into Python variables. +By pointing the aggregator at the folder containing your results, it automatically loads all relevant information +from each model fit. + +This capability streamlines the workflow by enabling easy manipulation and inspection of model-fit results directly +within your Python environment. It's particularly useful for managing and analyzing large-scale studies where +handling multiple model fits and their associated outputs is essential. + +```python +from autofit.aggregator.aggregator import Aggregator + +agg = Aggregator.from_directory( + directory=path.join("result_folder"), +) +``` + +The `values` method is used to specify the information that is loaded from the hard-disk, for example the +`samples` of the model-fit. + +The for loop below iterates over all results in the folder passed to the aggregator above. + +```python +for samples in agg.values("samples"): + print(samples.parameter_lists[0]) +``` + +Result loading uses Python generators to ensure that memory use is minimized, meaning that even when loading +thousands of results from hard-disk the memory use of your machine is not exceeded. + +The [result cookbook](https://pyautofit.readthedocs.io/en/latest/cookbooks/model.html) gives a full run-through of +the tools that allow results to be loaded and inspected. + +## Result Customization + +The `Result` object is returned by a non-linear search after running the following code: + +```python +result = search.fit(model=model, analysis=analysis) +``` + +An effective scientific workflow ensures that this object contains all information a user needs to quickly inspect +the quality of a model-fit and undertake scientific interpretation. + +The result can be can be customized to include additional information about the model-fit that is specific to your +model-fitting problem. + +For example, for fitting 1D profiles, the `Result` could include the maximum log likelihood model 1D data: + +```python +print(result.max_log_likelihood_model_data_1d) +``` + +To do this we use the custom result API, where we first define a custom `Result` class which includes the +property `max_log_likelihood_model_data_1d`: + +```python +class ResultExample(af.Result): + + @property + def max_log_likelihood_model_data_1d(self) -> np.ndarray: + """ + Returns the maximum log likelihood model's 1D model data. + + This is an example of how we can pass the `Analysis` class a custom `Result` object and extend this result + object with new properties that are specific to the model-fit we are performing. + """ + xvalues = np.arange(self.analysis.data.shape[0]) + + return self.instance.model_data_from(xvalues=xvalues) +``` + +The custom result has access to the analysis class, meaning that we can use any of its methods or properties to +compute custom result properties. + +To make it so that the `ResultExample` object above is returned by the search we overwrite the `Result` class attribute +of the `Analysis` and define a `make_result` object describing what we want it to contain: + +```python +class Analysis(af.Analysis): + + """ + This overwrite means the `ResultExample` class is returned after the model-fit. + """ + Result = ResultExample + + def __init__(self, data, noise_map): + """ + An Analysis class which illustrates custom results. + """ + super().__init__() + + self.data = data + self.noise_map = noise_map + + def log_likelihood_function(self, instance): + """ + The `log_likelihood_function` is identical to the example above + """ + xvalues = np.arange(self.data.shape[0]) + + model_data = instance.model_data_from(xvalues=xvalues) + residual_map = self.data - model_data + chi_squared_map = (residual_map / self.noise_map) ** 2.0 + chi_squared = sum(chi_squared_map) + noise_normalization = np.sum(np.log(2 * np.pi * noise_map**2.0)) + log_likelihood = -0.5 * (chi_squared + noise_normalization) + + return log_likelihood + + def make_result( + self, + samples_summary: af.SamplesSummary, + paths: af.AbstractPaths, + samples: Optional[af.SamplesPDF] = None, + search_internal: Optional[object] = None, + analysis: Optional[object] = None, + ) -> Result: + """ + Returns the `Result` of the non-linear search after it is completed. + + The result type is defined as a class variable in the `Analysis` class (see top of code under the python code + `class Analysis(af.Analysis)`. + + The result can be manually overwritten by a user to return a user-defined result object, which can be extended + with additional methods and attribute specific to the model-fit. + + This example class does example this, whereby the analysis result has been overwritten with the `ResultExample` + class, which contains a property `max_log_likelihood_model_data_1d` that returns the model data of the + best-fit model. This API means you can customize your result object to include whatever attributes you want + and therefore make a result object specific to your model-fit and model-fitting problem. + + The `Result` object you return can be customized to include: + + - The samples summary, which contains the maximum log likelihood instance and median PDF model. + + - The paths of the search, which are used for loading the samples and search internal below when a search + is resumed. + + - The samples of the non-linear search (e.g. MCMC chains) also stored in `samples.csv`. + + - The non-linear search used for the fit in its internal representation, which is used for resuming a search + and making bespoke visualization using the search's internal results. + + - The analysis used to fit the model (default disabled to save memory, but option may be useful for certain + projects). + + Parameters + ---------- + samples_summary + The summary of the samples of the non-linear search, which include the maximum log likelihood instance and + median PDF model. + paths + An object describing the paths for saving data (e.g. hard-disk directories or entries in sqlite database). + samples + The samples of the non-linear search, for example the chains of an MCMC run. + search_internal + The internal representation of the non-linear search used to perform the model-fit. + analysis + The analysis used to fit the model. + + Returns + ------- + Result + The result of the non-linear search, which is defined as a class variable in the `Analysis` class. + """ + return self.Result( + samples_summary=samples_summary, + paths=paths, + samples=samples, + search_internal=search_internal, + analysis=self + ) +``` + +Result customization has full support for **latent variables**, which are parameters that are not sampled by the non-linear +search but are computed from the sampled parameters. + +They are often integral to assessing and interpreting the results of a model-fit, as they present information +on the model in a different way to the sampled parameters. + +The [result cookbook](https://pyautofit.readthedocs.io/en/latest/cookbooks/result.html) gives a full run-through of +all the different ways the result can be customized. + +## Model Composition + +In many scientific workflows, there's often a need to construct and fit a variety of different models. This +could range from making minor adjustments to a model's parameters to handling complex models with thousands of parameters and multiple components. + +For simpler scenarios, adjustments might include: + +- **Parameter Assignment**: Setting specific values for certain parameters or linking parameters together so they share the same value. +- **Parameter Assertions**: Imposing constraints on model parameters, such as requiring one parameter to be greater than another. +- **Model Arithmetic**: Defining relationships between parameters using arithmetic operations, such as defining a linear relationship like `y = mx + c`, where `m` and `c` are model parameters. + +In more intricate cases, models might involve numerous parameters and complex compositions of multiple model components. + +**PyAutoFit** offers a sophisticated model composition API designed to handle these complexities. It provides +tools for constructing elaborate models using lists of Python classes, NumPy arrays and hierarchical structures of Python classes. + +For a detailed exploration of these capabilities, you can refer to +the [model cookbook](https://pyautofit.readthedocs.io/en/latest/cookbooks/model.html), which provides comprehensive +guidance on using the model composition API. This resource covers everything from basic parameter assignments to +constructing complex models with hierarchical structures. + +## Searches + +Different model-fitting problems often require different approaches to fitting the model effectively. + +The choice of the most suitable search method depends on several factors: + +- **Model Dimensions**: How many parameters constitute the model and its non-linear parameter space? +- **Model Complexity**: Different models exhibit varying degrees of parameter degeneracy, which necessitates different non-linear search techniques. +- **Run Times**: How efficiently can the likelihood function be evaluated and the model-fit performed? +- **Gradients**: If your likelihood function is differentiable, leveraging JAX and using a search that exploits gradient information can be advantageous. + +**PyAutoFit** provides support for a wide range of non-linear searches, ensuring that users can select the method +best suited to their specific problem. + +During the initial stages of setting up your scientific workflow, it's beneficial to experiment with different +searches. This process helps identify which methods reliably infer maximum likelihood fits to the data and assess +their efficiency in terms of computational time. + +For a comprehensive exploration of available search methods and customization options, refer to +the [search cookbook](https://pyautofit.readthedocs.io/en/latest/cookbooks/search.html). This resource covers +detailed guides on all non-linear searches supported by PyAutoFit and provides insights into how to tailor them to your needs. + +:::{note} +There are currently no documentation guiding reads on what search might be appropriate for their problem and how to +profile and experiment with different methods. Writing such documentation is on the to do list and will appear +in the future. However, you can make progress now simply using visuals output by PyAutoFit and the search.summary\` file. +::: + +## Configs + +As you refine your scientific workflow, you'll often find yourself repeatedly setting up models with identical priors +and using the same non-linear search configurations. This repetition can result in lengthy Python scripts with +redundant inputs. + +To streamline this process, configuration files can be utilized to define default values. This approach eliminates +the need to specify identical prior inputs and search settings in every script, leading to more concise and +readable Python code. Moreover, it reduces the cognitive load associated with performing model-fitting tasks. + +For a comprehensive guide on setting up and utilizing configuration files effectively, refer +to the [configs cookbook](https://pyautofit.readthedocs.io/en/latest/cookbooks/configs.html). This resource provides +detailed instructions on configuring and optimizing your PyAutoFit workflow through the use of configuration files. + +## Database + +By default, model-fitting results are written to folders on hard-disk, which is straightforward for navigating and +manual inspection. However, this approach becomes impractical for large datasets or extensive scientific workflows, +where manually checking each result can be time-consuming. + +To address this challenge, all results can be stored in an sqlite3 relational database. This enables loading results +directly into Jupyter notebooks or Python scripts for inspection, analysis, and interpretation. The database +supports advanced querying capabilities, allowing users to retrieve specific model-fits based on criteria such +as the fitted model or dataset. + +For a comprehensive guide on using the database functionality within PyAutoFit, refer to +the `database cookbook `. This resource +provides detailed instructions on leveraging the database to manage and analyze model-fitting results efficiently. + +## Scaling Up + +Regardless of your final scientific objective, it's crucial to consider scalability in your scientific workflow and +ensure it remains flexible to accommodate varying scales of complexity. + +Initially, scientific studies often begin with a small number of datasets (e.g., tens of datasets). During this phase, +researchers iteratively refine their models and gain insights through trial and error. This involves fitting numerous +models to datasets and manually inspecting results to evaluate model performance. A flexible workflow is essential +here, allowing rapid iteration and outputting results in a format that facilitates quick inspection and interpretation. + +As the study progresses, researchers may scale up to larger datasets (e.g., thousands of datasets). Manual inspection +of individual results becomes impractical, necessitating a more automated approach to model fitting and interpretation. +Additionally, analyses may transition to high-performance computing environments, requiring output formats suitable for these setups. + +**PyAutoFit** is designed to enable the development of effective scientific workflows for both small and large datasets. + +## Wrap Up + +This overview has provided a comprehensive guide to the key features of **PyAutoFit** that support the development of +effective scientific workflows. By leveraging these tools, researchers can tailor their workflows to specific problems, +streamline model fitting, and gain valuable insights into their scientific studies. + +The final aspect of core functionality, described in the next overview, is the wide variety of statistical +inference methods available in **PyAutoFit**. These methods include graphical models, hierarchical models, +Bayesian model comparison and many more. diff --git a/docs/overview/scientific_workflow.rst b/docs/overview/scientific_workflow.rst deleted file mode 100644 index 7836526af..000000000 --- a/docs/overview/scientific_workflow.rst +++ /dev/null @@ -1,621 +0,0 @@ -.. _scientific_workflow: - -Scientific Workflow -=================== - -A scientific workflow comprises the tasks you perform to conduct a scientific study. This includes fitting models to -datasets, interpreting the results, and gaining insights into your scientific problem. - -Different problems require different scientific workflows, depending on factors such as model complexity, dataset size, -and computational run times. For example, some problems involve fitting a single dataset with many models to gain -scientific insights, while others involve fitting thousands of datasets with a single model for large-scale studies. - -The **PyAutoFit** API is flexible, customizable, and extensible, enabling users to develop scientific workflows -tailored to their specific problems. - -This overview covers the key features of **PyAutoFit** that support the development of effective scientific workflows: - -- **On The Fly**: Display results immediately (e.g., in Jupyter notebooks) to provide instant feedback for adapting your workflow. -- **Hard Disk Output**: Output results to hard disk with high customization, allowing quick and detailed inspection of fits to many datasets. -- **Visualization**: Generate model-specific visualizations to create custom plots that streamline result inspection. -- **Loading Results**: Load results from the hard disk to inspect and interpret the outcomes of a model fit. -- **Result Customization**: Customize the returned results to simplify scientific interpretation. -- **Model Composition**: Extensible model composition makes it easy to fit many models with different parameterizations and assumptions. -- **Searches**: Support for various non-linear searches (e.g., nested sampling, MCMC), including gradient based fitting using JAX, to find the right method for your problem. -- **Configs**: Configuration files that set default model, fitting, and visualization behaviors, streamlining model fitting. -- **Database**: Store results in a relational SQLite3 database, enabling efficient management of large modeling results. -- **Scaling Up**: Guidance on scaling up your scientific workflow from small to large datasets. - -On The Fly ----------- - -.. note:: - - The on-the-fly feature described below is not implemented yet, we are working on it currently. - The best way to get on-the-fly output is to output to hard-disk, which is described in the next section. - This feature is fully implemented and provides on-the-fly output of results to hard-disk. - -When a model fit is running, information about the fit is displayed at user-specified intervals. - -The frequency of this on-the-fly output is controlled by a search's `iterations_per_full_update` parameter, which -specifies how often this information is output. The example code below outputs on-the-fly information every 1000 iterations: - -.. code-block:: python - - search = af.DynestyStatic( - iterations_per_full_update=1000 - ) - -In a Jupyter notebook, the default behavior is for this information to appear in the cell being run and to include: - -- Text displaying the maximum likelihood model inferred so far and related information. -- A visual showing how the search has sampled parameter space so far, providing intuition on how the search is performing. - -Here is an image of how this looks: - -![Example On-the-Fly Output](path/to/image.png) - -The most valuable on-the-fly output is often specific to the model and dataset you are fitting. For instance, it -might be a ``matplotlib`` subplot showing the maximum likelihood model's fit to the dataset, complete with residuals -and other diagnostic information. - -The on-the-fly output can be fully customized by extending the ``on_the_fly_output`` method of the ``Analysis`` -class being used to fit the model. - -The example below shows how this is done for the simple case of fitting a 1D Gaussian profile: - -.. code-block:: python - - class Analysis(af.Analysis): - def __init__(self, data: np.ndarray, noise_map: np.ndarray): - """ - Example Analysis class illustrating how to customize the on-the-fly output of a model-fit. - """ - super().__init__() - - self.data = data - self.noise_map = noise_map - - def on_the_fly_output(self, instance): - """ - During a model-fit, the `on_the_fly_output` method is called throughout the non-linear search. - - The `instance` passed into the method is maximum log likelihood solution obtained by the model-fit so far and it can be - used to provide on-the-fly output showing how the model-fit is going. - """ - xvalues = np.arange(analysis.data.shape[0]) - - model_data = instance.model_data_from(xvalues=xvalues) - - """ - The visualizer now outputs images of the best-fit results to hard-disk (checkout `visualizer.py`). - """ - import matplotlib.pyplot as plt - - plt.errorbar( - x=xvalues, - y=self.data, - yerr=self.noise_map, - color="k", - ecolor="k", - elinewidth=1, - capsize=2, - ) - plt.plot(xvalues, model_data, color="r") - plt.title("Maximum Likelihood Fit") - plt.xlabel("x value of profile") - plt.ylabel("Profile Normalization") - plt.show() # By using `plt.show()` the plot will be displayed in the Jupyter notebook. - -Here's how the visuals appear in a Jupyter Notebook: - -![Example On-the-Fly Output](path/to/image.png) - -In the early stages of setting up a scientific workflow, on-the-fly output is invaluable. It provides immediate -feedback on how your model fitting is performing, which is often crucial at the beginning of a project when things -might not be going well. It also encourages you to prioritize visualizing your fit and diagnosing whether the process -is working correctly. - -We highly recommend users starting a new model-fitting problem begin by setting up on-the-fly output! - -Hard Disk Output ----------------- - -By default, a non-linear search does not save its results to the hard disk; the results can only be inspected in a Jupyter Notebook or Python script via the returned `result`. - -However, you can enable the output of non-linear search results to the hard disk by specifying the `name` and/or `path_prefix` attributes. These attributes determine how files are named and where results are saved on your hard disk. - -Benefits of saving results to the hard disk include: - -- More efficient inspection of results for multiple datasets compared to using a Jupyter Notebook. -- Results are saved on-the-fly, allowing you to check the progress of a fit midway. -- Additional information about a fit, such as visualizations, can be saved (see below). -- Unfinished runs can be resumed from where they left off if they are terminated. -- On high-performance supercomputers, results often need to be saved in this manner. - -Here's how to enable the output of results to the hard disk: - -.. code-block:: python - - search = af.Emcee( - path_prefix=path.join("folder_0", "folder_1"), - name="my_search_name" - ) - -The screenshot below shows the output folder where all output is enabled: - -.. image:: https://raw.githubusercontent.com/Jammy2211/PyAutoFit/main/docs/overview/image/output_example.png - :width: 400 - :alt: Alternative text - -Let's break down the output folder generated by **PyAutoFit**: - -- **Unique Identifier**: Results are saved in a folder named with a unique identifier composed of random characters. This identifier is automatically generated based on the specific model fit. For scientific workflows involving numerous model fits, this ensures that each fit is uniquely identified without requiring manual updates to output paths. - -- **Info Files**: These files contain valuable information about the fit. For instance, `model.info` provides the complete model composition used in the fit, while `search.summary` details how long the search has been running and other relevant search-specific information. - -- **Files Folder**: Within the output folder, the `files` directory contains detailed information saved as `.json` files. For example, `model.json` stores the model configuration used in the fit. This enables researchers to revisit the results later and review how the fit was performed. - -**PyAutoFit** offers extensive tools for customizing hard-disk output. This includes using configuration files to control what information is saved, which helps manage disk space utilization. Additionally, specific `.json` files tailored to different models can be utilized for more detailed output. - -For many scientific workflows, having detailed output for each fit is crucial for thorough inspection and accurate -interpretation of results. However, in scenarios where the volume of output data might overwhelm users or impede -scientific study, this feature can be easily disabled by omitting the `name` or `path prefix` when initiating the search. - -Visualization -------------- - -When search hard-disk output is enabled in **PyAutoFit**, the visualization of model fits can also be saved directly -to disk. This capability is crucial for many scientific workflows as it allows for quick and effective assessment of -fit quality. - -To accomplish this, you can customize the `Visualizer` object of an `Analysis` class with a custom `Visualizer` class. -This custom class is responsible for generating and saving visual representations of the model fits. By leveraging -this approach, scientists can efficiently visualize and analyze the outcomes of model fitting processes. - -.. code-block:: python - - class Visualizer(af.Visualizer): - - @staticmethod - def visualize_before_fit( - analysis, - paths: af.DirectoryPaths, - model: af.AbstractPriorModel - ): - """ - Before a model-fit, the `visualize_before_fit` method is called to perform visualization. - - The function receives as input an instance of the `Analysis` class which is being used to perform the fit, - which is used to perform the visualization (e.g. it contains the data and noise map which are plotted). - - This can output visualization of quantities which do not change during the model-fit, for example the - data and noise-map. - - The `paths` object contains the path to the folder where the visualization should be output, which is determined - by the non-linear search `name` and other inputs. - """ - - import matplotlib.pyplot as plt - - xvalues = np.arange(self.data.shape[0]) - - plt.errorbar( - x=xvalues, - y=analysis.data, - yerr=analysis.noise_map, - color="k", - ecolor="k", - elinewidth=1, - capsize=2, - ) - plt.title("Maximum Likelihood Fit") - plt.xlabel("x value of profile") - plt.ylabel("Profile Normalization") - plt.savefig(path.join(paths.image_path, f"data.png")) - plt.clf() - - @staticmethod - def visualize( - analysis, - paths: af.DirectoryPaths, - instance, - during_analysis - ): - """ - During a model-fit, the `visualize` method is called throughout the non-linear search. - - The function receives as input an instance of the `Analysis` class which is being used to perform the fit, - which is used to perform the visualization (e.g. it generates the model data which is plotted). - - The `instance` passed into the visualize method is maximum log likelihood solution obtained by the model-fit - so far and it can be used to provide on-the-fly images showing how the model-fit is going. - - The `paths` object contains the path to the folder where the visualization should be output, which is determined - by the non-linear search `name` and other inputs. - """ - xvalues = np.arange(analysis.data.shape[0]) - - model_data = instance.model_data_from(xvalues=xvalues) - residual_map = analysis.data - model_data - - """ - The visualizer now outputs images of the best-fit results to hard-disk (checkout `visualizer.py`). - """ - import matplotlib.pyplot as plt - - plt.errorbar( - x=xvalues, - y=analysis.data, - yerr=analysis.noise_map, - color="k", - ecolor="k", - elinewidth=1, - capsize=2, - ) - plt.plot(xvalues, model_data, color="r") - plt.title("Maximum Likelihood Fit") - plt.xlabel("x value of profile") - plt.ylabel("Profile Normalization") - plt.savefig(path.join(paths.image_path, f"model_fit.png")) - plt.clf() - - plt.errorbar( - x=xvalues, - y=residual_map, - yerr=analysis.noise_map, - color="k", - ecolor="k", - elinewidth=1, - capsize=2, - ) - plt.title("Residuals of Maximum Likelihood Fit") - plt.xlabel("x value of profile") - plt.ylabel("Residual") - plt.savefig(path.join(paths.image_path, f"model_fit.png")) - plt.clf() - -The ``Analysis`` class is defined following the same API as before, but now with its `Visualizer` class attribute -overwritten with the ``Visualizer`` class above. - -.. code-block:: python - - class Analysis(af.Analysis): - - """ - This over-write means the `Visualizer` class is used for visualization throughout the model-fit. - - This `VisualizerExample` object is in the `autofit.example.visualize` module and is used to customize the - plots output during the model-fit. - - It has been extended with visualize methods that output visuals specific to the fitting of `1D` data. - """ - Visualizer = Visualizer - - def __init__(self, data, noise_map): - """ - An Analysis class which illustrates visualization. - """ - super().__init__() - - self.data = data - self.noise_map = noise_map - - def log_likelihood_function(self, instance): - """ - The `log_likelihood_function` is identical to the example above - """ - xvalues = np.arange(self.data.shape[0]) - - model_data = instance.model_data_from(xvalues=xvalues) - residual_map = self.data - model_data - chi_squared_map = (residual_map / self.noise_map) ** 2.0 - chi_squared = sum(chi_squared_map) - noise_normalization = np.sum(np.log(2 * np.pi * noise_map**2.0)) - log_likelihood = -0.5 * (chi_squared + noise_normalization) - - return log_likelihood - -Visualization of the results of the non-linear search, for example the "Probability Density -Function", are also automatically output during the model-fit on the fly. - -Loading Results ---------------- - -In your scientific workflow, you'll likely conduct numerous model fits, each generating outputs stored in individual -folders on your hard disk. - -To efficiently work with these results in Python scripts or Jupyter notebooks, **PyAutoFit** provides -the `aggregator` API. This tool simplifies the process of loading results from hard disk into Python variables. -By pointing the aggregator at the folder containing your results, it automatically loads all relevant information -from each model fit. - -This capability streamlines the workflow by enabling easy manipulation and inspection of model-fit results directly -within your Python environment. It's particularly useful for managing and analyzing large-scale studies where -handling multiple model fits and their associated outputs is essential. - -.. code-block:: python - - from autofit.aggregator.aggregator import Aggregator - - agg = Aggregator.from_directory( - directory=path.join("result_folder"), - ) - -The ``values`` method is used to specify the information that is loaded from the hard-disk, for example the -``samples`` of the model-fit. - -The for loop below iterates over all results in the folder passed to the aggregator above. - -.. code-block:: python - - for samples in agg.values("samples"): - print(samples.parameter_lists[0]) - -Result loading uses Python generators to ensure that memory use is minimized, meaning that even when loading -thousands of results from hard-disk the memory use of your machine is not exceeded. - -The `result cookbook `_ gives a full run-through of -the tools that allow results to be loaded and inspected. - -Result Customization --------------------- - -The ``Result`` object is returned by a non-linear search after running the following code: - -.. code-block:: python - - result = search.fit(model=model, analysis=analysis) - -An effective scientific workflow ensures that this object contains all information a user needs to quickly inspect -the quality of a model-fit and undertake scientific interpretation. - -The result can be can be customized to include additional information about the model-fit that is specific to your -model-fitting problem. - -For example, for fitting 1D profiles, the ``Result`` could include the maximum log likelihood model 1D data: - -.. code-block:: python - - print(result.max_log_likelihood_model_data_1d) - -To do this we use the custom result API, where we first define a custom ``Result`` class which includes the -property ``max_log_likelihood_model_data_1d``: - -.. code-block:: python - - class ResultExample(af.Result): - - @property - def max_log_likelihood_model_data_1d(self) -> np.ndarray: - """ - Returns the maximum log likelihood model's 1D model data. - - This is an example of how we can pass the `Analysis` class a custom `Result` object and extend this result - object with new properties that are specific to the model-fit we are performing. - """ - xvalues = np.arange(self.analysis.data.shape[0]) - - return self.instance.model_data_from(xvalues=xvalues) - -The custom result has access to the analysis class, meaning that we can use any of its methods or properties to -compute custom result properties. - -To make it so that the ``ResultExample`` object above is returned by the search we overwrite the ``Result`` class attribute -of the ``Analysis`` and define a ``make_result`` object describing what we want it to contain: - -.. code-block:: python - - class Analysis(af.Analysis): - - """ - This overwrite means the `ResultExample` class is returned after the model-fit. - """ - Result = ResultExample - - def __init__(self, data, noise_map): - """ - An Analysis class which illustrates custom results. - """ - super().__init__() - - self.data = data - self.noise_map = noise_map - - def log_likelihood_function(self, instance): - """ - The `log_likelihood_function` is identical to the example above - """ - xvalues = np.arange(self.data.shape[0]) - - model_data = instance.model_data_from(xvalues=xvalues) - residual_map = self.data - model_data - chi_squared_map = (residual_map / self.noise_map) ** 2.0 - chi_squared = sum(chi_squared_map) - noise_normalization = np.sum(np.log(2 * np.pi * noise_map**2.0)) - log_likelihood = -0.5 * (chi_squared + noise_normalization) - - return log_likelihood - - def make_result( - self, - samples_summary: af.SamplesSummary, - paths: af.AbstractPaths, - samples: Optional[af.SamplesPDF] = None, - search_internal: Optional[object] = None, - analysis: Optional[object] = None, - ) -> Result: - """ - Returns the `Result` of the non-linear search after it is completed. - - The result type is defined as a class variable in the `Analysis` class (see top of code under the python code - `class Analysis(af.Analysis)`. - - The result can be manually overwritten by a user to return a user-defined result object, which can be extended - with additional methods and attribute specific to the model-fit. - - This example class does example this, whereby the analysis result has been overwritten with the `ResultExample` - class, which contains a property `max_log_likelihood_model_data_1d` that returns the model data of the - best-fit model. This API means you can customize your result object to include whatever attributes you want - and therefore make a result object specific to your model-fit and model-fitting problem. - - The `Result` object you return can be customized to include: - - - The samples summary, which contains the maximum log likelihood instance and median PDF model. - - - The paths of the search, which are used for loading the samples and search internal below when a search - is resumed. - - - The samples of the non-linear search (e.g. MCMC chains) also stored in `samples.csv`. - - - The non-linear search used for the fit in its internal representation, which is used for resuming a search - and making bespoke visualization using the search's internal results. - - - The analysis used to fit the model (default disabled to save memory, but option may be useful for certain - projects). - - Parameters - ---------- - samples_summary - The summary of the samples of the non-linear search, which include the maximum log likelihood instance and - median PDF model. - paths - An object describing the paths for saving data (e.g. hard-disk directories or entries in sqlite database). - samples - The samples of the non-linear search, for example the chains of an MCMC run. - search_internal - The internal representation of the non-linear search used to perform the model-fit. - analysis - The analysis used to fit the model. - - Returns - ------- - Result - The result of the non-linear search, which is defined as a class variable in the `Analysis` class. - """ - return self.Result( - samples_summary=samples_summary, - paths=paths, - samples=samples, - search_internal=search_internal, - analysis=self - ) - -Result customization has full support for **latent variables**, which are parameters that are not sampled by the non-linear -search but are computed from the sampled parameters. - -They are often integral to assessing and interpreting the results of a model-fit, as they present information -on the model in a different way to the sampled parameters. - -The `result cookbook `_ gives a full run-through of -all the different ways the result can be customized. - -Model Composition ------------------ - -In many scientific workflows, there's often a need to construct and fit a variety of different models. This -could range from making minor adjustments to a model's parameters to handling complex models with thousands of parameters and multiple components. - -For simpler scenarios, adjustments might include: - -- **Parameter Assignment**: Setting specific values for certain parameters or linking parameters together so they share the same value. -- **Parameter Assertions**: Imposing constraints on model parameters, such as requiring one parameter to be greater than another. -- **Model Arithmetic**: Defining relationships between parameters using arithmetic operations, such as defining a linear relationship like `y = mx + c`, where `m` and `c` are model parameters. - -In more intricate cases, models might involve numerous parameters and complex compositions of multiple model components. - -**PyAutoFit** offers a sophisticated model composition API designed to handle these complexities. It provides -tools for constructing elaborate models using lists of Python classes, NumPy arrays and hierarchical structures of Python classes. - -For a detailed exploration of these capabilities, you can refer to -the `model cookbook `_, which provides comprehensive -guidance on using the model composition API. This resource covers everything from basic parameter assignments to -constructing complex models with hierarchical structures. - -Searches --------- - -Different model-fitting problems often require different approaches to fitting the model effectively. - -The choice of the most suitable search method depends on several factors: - -- **Model Dimensions**: How many parameters constitute the model and its non-linear parameter space? -- **Model Complexity**: Different models exhibit varying degrees of parameter degeneracy, which necessitates different non-linear search techniques. -- **Run Times**: How efficiently can the likelihood function be evaluated and the model-fit performed? -- **Gradients**: If your likelihood function is differentiable, leveraging JAX and using a search that exploits gradient information can be advantageous. - -**PyAutoFit** provides support for a wide range of non-linear searches, ensuring that users can select the method -best suited to their specific problem. - -During the initial stages of setting up your scientific workflow, it's beneficial to experiment with different -searches. This process helps identify which methods reliably infer maximum likelihood fits to the data and assess -their efficiency in terms of computational time. - -For a comprehensive exploration of available search methods and customization options, refer to -the `search cookbook `_. This resource covers -detailed guides on all non-linear searches supported by PyAutoFit and provides insights into how to tailor them to your needs. - -.. note:: - - There are currently no documentation guiding reads on what search might be appropriate for their problem and how to - profile and experiment with different methods. Writing such documentation is on the to do list and will appear - in the future. However, you can make progress now simply using visuals output by PyAutoFit and the ``search.summary` file. - -Configs -------- - -As you refine your scientific workflow, you'll often find yourself repeatedly setting up models with identical priors -and using the same non-linear search configurations. This repetition can result in lengthy Python scripts with -redundant inputs. - -To streamline this process, configuration files can be utilized to define default values. This approach eliminates -the need to specify identical prior inputs and search settings in every script, leading to more concise and -readable Python code. Moreover, it reduces the cognitive load associated with performing model-fitting tasks. - -For a comprehensive guide on setting up and utilizing configuration files effectively, refer -to the `configs cookbook `_. This resource provides -detailed instructions on configuring and optimizing your PyAutoFit workflow through the use of configuration files. - -Database --------- - -By default, model-fitting results are written to folders on hard-disk, which is straightforward for navigating and -manual inspection. However, this approach becomes impractical for large datasets or extensive scientific workflows, -where manually checking each result can be time-consuming. - -To address this challenge, all results can be stored in an sqlite3 relational database. This enables loading results -directly into Jupyter notebooks or Python scripts for inspection, analysis, and interpretation. The database -supports advanced querying capabilities, allowing users to retrieve specific model-fits based on criteria such -as the fitted model or dataset. - -For a comprehensive guide on using the database functionality within PyAutoFit, refer to -the `database cookbook `. This resource -provides detailed instructions on leveraging the database to manage and analyze model-fitting results efficiently. - -Scaling Up ----------- - -Regardless of your final scientific objective, it's crucial to consider scalability in your scientific workflow and -ensure it remains flexible to accommodate varying scales of complexity. - -Initially, scientific studies often begin with a small number of datasets (e.g., tens of datasets). During this phase, -researchers iteratively refine their models and gain insights through trial and error. This involves fitting numerous -models to datasets and manually inspecting results to evaluate model performance. A flexible workflow is essential -here, allowing rapid iteration and outputting results in a format that facilitates quick inspection and interpretation. - -As the study progresses, researchers may scale up to larger datasets (e.g., thousands of datasets). Manual inspection -of individual results becomes impractical, necessitating a more automated approach to model fitting and interpretation. -Additionally, analyses may transition to high-performance computing environments, requiring output formats suitable for these setups. - -**PyAutoFit** is designed to enable the development of effective scientific workflows for both small and large datasets. - -Wrap Up -------- - -This overview has provided a comprehensive guide to the key features of **PyAutoFit** that support the development of -effective scientific workflows. By leveraging these tools, researchers can tailor their workflows to specific problems, -streamline model fitting, and gain valuable insights into their scientific studies. - -The final aspect of core functionality, described in the next overview, is the wide variety of statistical -inference methods available in **PyAutoFit**. These methods include graphical models, hierarchical models, -Bayesian model comparison and many more. \ No newline at end of file diff --git a/docs/overview/statistical_methods.rst b/docs/overview/statistical_methods.md similarity index 79% rename from docs/overview/statistical_methods.rst rename to docs/overview/statistical_methods.md index 336a89b35..285279b92 100644 --- a/docs/overview/statistical_methods.rst +++ b/docs/overview/statistical_methods.md @@ -1,115 +1,105 @@ -.. _statistical_methods: - -Statistical Methods -=================== - -**PyAutoFit** supports numerous statistical methods that allow for advanced Bayesian inference to be performed. - -Graphical Models ----------------- - -For inference problems consisting of many datasets, the model composition is often very complex. Model parameters -can depend on multiple datasets, and the datasets themselves may be interdependent. - -Graphical models concisely describe these model and dataset dependencies, allowing them to be fitted simultaneously. -This is achieved through a concise API and scientific workflow that ensures scalability to large datasets. - -A full description of using graphical models is given below: - -https://github.com/Jammy2211/autofit_workspace/blob/release/notebooks/features/graphical_models.ipynb - -Hierarchical Models -------------------- - -Hierarchical models are where multiple parameters in the model are assumed to be drawn from a common distribution. -The parameters of this parent distribution are themselves inferred from the data, enabling -more robust and informative model fitting. - -A full description of using hierarchical models is given below: - -https://github.com/PyAutoLabs/HowToFit/blob/main/notebooks/chapter_3_graphical_models/tutorial_4_hierachical_models.ipynb - -Model Comparison ----------------- - -Common questions when fitting a model to data are: what model should I use? How many parameters should the model have? -Is the model too complex or too simple? - -Model comparison answers to these questions. It amounts to composing and fitting many different models to the data -and comparing how well they fit the data. - -A full description of using model comparison is given below: - -https://github.com/Jammy2211/autofit_workspace/blob/release/notebooks/features/model_comparison.ipynb - -Interpolation -------------- - -It is common to fit a model to many similar datasets, where it is anticipated that one or more model parameters vary -smoothly across the datasets. - -For example, the datasets may be taken at different times, where the signal in the data and therefore model parameters -vary smoothly as a function of time. - -It may be desirable to fit the datasets one-by-one and then interpolate the results in order -to determine the most likely model parameters at any point in time. - -**PyAutoFit**'s interpolation feature allows for this, and a full description of its use is given below: - -https://github.com/Jammy2211/autofit_workspace/blob/release/notebooks/features/interpolate.ipynb - -Search Grid Search ------------------- - -A classic method to perform model-fitting is a grid search, where the parameters of a model are divided onto a grid of -values and the likelihood of each set of parameters on this grid is sampled. For low dimensionality problems this -simple approach can be sufficient to locate high likelihood solutions, however it scales poorly to higher dimensional -problems. - -**PyAutoFit** can perform a search grid search, which allows one to perform a grid-search over a subset of parameters -within a model, but use a non-linear search to fit for the other parameters. The parameters over which the grid-search -is performed are also included in the model fit and their values are simply confined to the boundaries of their grid -cell. - -This can help ensure robust results are inferred for complex models, and can remove multi modality in a parameter -space to further aid the fitting process. - -A full description of using search grid searches is given below: - -https://github.com/Jammy2211/autofit_workspace/blob/release/notebooks/features/search_grid_search.ipynb - -Search Chaining ---------------- - -To perform a model-fit, we typically compose one model and fit it to our data using one non-linear search. - -Search chaining fits many different models to a dataset using a chained sequence of non-linear searches. Initial -fits are performed using simplified models and faster non-linear fitting techniques. The results of these simplified -fits are then be used to initialize fits using a higher dimensionality model with a more detailed non-linear search. - -To fit highly complex models search chaining allows us to therefore to granularize the fitting procedure into a series -of **bite-sized** searches which are faster and more reliable than fitting the more complex model straight away. - -A full description of using search chaining is given below: - -https://github.com/Jammy2211/autofit_workspace/blob/release/notebooks/features/search_grid_search.ipynb - -Sensitivity Mapping -------------------- - -Bayesian model comparison allows us to take a dataset, fit it with multiple models and use the Bayesian evidence to -quantify which model objectively gives the best-fit following the principles of Occam's Razor. - -However, a complex model may not be favoured by model comparison not because it is the 'wrong' model, but simply -because the dataset being fitted is not of a sufficient quality for the more complex model to be favoured. Sensitivity -mapping addresses what quality of data would be needed for the more complex model to be favoured. - -In order to do this, sensitivity mapping involves us writing a function that uses the model(s) to simulate a dataset. -We then use this function to simulate many datasets, for different models, and fit each dataset to quantify -how much the change in the model led to a measurable change in the data. This is called computing the sensitivity. - -A full description of using sensitivity mapping is given below: - -https://github.com/Jammy2211/autofit_workspace/blob/release/notebooks/features/sensitivity_mapping.ipynb - - +(statistical-methods)= + +# Statistical Methods + +**PyAutoFit** supports numerous statistical methods that allow for advanced Bayesian inference to be performed. + +## Graphical Models + +For inference problems consisting of many datasets, the model composition is often very complex. Model parameters +can depend on multiple datasets, and the datasets themselves may be interdependent. + +Graphical models concisely describe these model and dataset dependencies, allowing them to be fitted simultaneously. +This is achieved through a concise API and scientific workflow that ensures scalability to large datasets. + +A full description of using graphical models is given below: + + + +## Hierarchical Models + +Hierarchical models are where multiple parameters in the model are assumed to be drawn from a common distribution. +The parameters of this parent distribution are themselves inferred from the data, enabling +more robust and informative model fitting. + +A full description of using hierarchical models is given below: + + + +## Model Comparison + +Common questions when fitting a model to data are: what model should I use? How many parameters should the model have? +Is the model too complex or too simple? + +Model comparison answers to these questions. It amounts to composing and fitting many different models to the data +and comparing how well they fit the data. + +A full description of using model comparison is given below: + + + +## Interpolation + +It is common to fit a model to many similar datasets, where it is anticipated that one or more model parameters vary +smoothly across the datasets. + +For example, the datasets may be taken at different times, where the signal in the data and therefore model parameters +vary smoothly as a function of time. + +It may be desirable to fit the datasets one-by-one and then interpolate the results in order +to determine the most likely model parameters at any point in time. + +**PyAutoFit**'s interpolation feature allows for this, and a full description of its use is given below: + + + +## Search Grid Search + +A classic method to perform model-fitting is a grid search, where the parameters of a model are divided onto a grid of +values and the likelihood of each set of parameters on this grid is sampled. For low dimensionality problems this +simple approach can be sufficient to locate high likelihood solutions, however it scales poorly to higher dimensional +problems. + +**PyAutoFit** can perform a search grid search, which allows one to perform a grid-search over a subset of parameters +within a model, but use a non-linear search to fit for the other parameters. The parameters over which the grid-search +is performed are also included in the model fit and their values are simply confined to the boundaries of their grid +cell. + +This can help ensure robust results are inferred for complex models, and can remove multi modality in a parameter +space to further aid the fitting process. + +A full description of using search grid searches is given below: + + + +## Search Chaining + +To perform a model-fit, we typically compose one model and fit it to our data using one non-linear search. + +Search chaining fits many different models to a dataset using a chained sequence of non-linear searches. Initial +fits are performed using simplified models and faster non-linear fitting techniques. The results of these simplified +fits are then be used to initialize fits using a higher dimensionality model with a more detailed non-linear search. + +To fit highly complex models search chaining allows us to therefore to granularize the fitting procedure into a series +of **bite-sized** searches which are faster and more reliable than fitting the more complex model straight away. + +A full description of using search chaining is given below: + + + +## Sensitivity Mapping + +Bayesian model comparison allows us to take a dataset, fit it with multiple models and use the Bayesian evidence to +quantify which model objectively gives the best-fit following the principles of Occam's Razor. + +However, a complex model may not be favoured by model comparison not because it is the 'wrong' model, but simply +because the dataset being fitted is not of a sufficient quality for the more complex model to be favoured. Sensitivity +mapping addresses what quality of data would be needed for the more complex model to be favoured. + +In order to do this, sensitivity mapping involves us writing a function that uses the model(s) to simulate a dataset. +We then use this function to simulate many datasets, for different models, and fit each dataset to quantify +how much the change in the model led to a measurable change in the data. This is called computing the sensitivity. + +A full description of using sensitivity mapping is given below: + + diff --git a/docs/overview/the_basics.md b/docs/overview/the_basics.md new file mode 100644 index 000000000..c57d1f322 --- /dev/null +++ b/docs/overview/the_basics.md @@ -0,0 +1,655 @@ +(the-basics)= + +# The Basics + +**PyAutoFit** is a Python based probabilistic programming language for model fitting and Bayesian inference +of large datasets. + +The basic **PyAutoFit** API allows us a user to quickly compose a probabilistic model and fit it to data via a +log likelihood function, using a range of non-linear search algorithms (e.g. MCMC, nested sampling). + +This overview gives a run through of: + +> - **Models**: Use Python classes to compose the model which is fitted to data. +> - **Instances**: Create instances of the model via its Python class. +> - **Analysis**: Define an `Analysis` class which includes the log likelihood function that fits the model to the data. +> - **Searches**: Choose an MCMC, nested sampling or maximum likelihood estimator non-linear search algorithm that fits the model to the data. +> - **Model Fit**: Fit the model to the data using the chosen non-linear search, with on-the-fly results and visualization. +> - **Results**: Use the results of the search to interpret and visualize the model fit. + +- **Samples**: Use the samples of the search to inspect the parameter samples and visualize the probability density function of the results. +- **Multiple Datasets**: Dedicated support for simultaneously fitting multiple datasets, enabling scalable analysis of large datasets. + +This overviews provides a high level of the basic API, with more advanced functionality described in the following +overviews and the **PyAutoFit** cookbooks. + +## Example + +To illustrate **PyAutoFit** we'll use the example modeling problem of fitting a 1D Gaussian profile to noisy data. + +To begin, lets import `autofit` (and `numpy`) using the convention below: + +```python +import autofit as af +import numpy as np +``` + +The example `data` with errors (black) is shown below: + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/images/data.png +:alt: Alternative text +:width: 600 +``` + +The 1D signal was generated using a 1D Gaussian profile of the form: + +$$ +g(x, I, \sigma) = \frac{N}{\sigma\sqrt{2\pi}} \exp{(-0.5 (x / \sigma)^2)} +$$ + +Where: + +> `x`: The x-axis coordinate where the `Gaussian` is evaluated. +> +> `N`: The overall normalization of the Gaussian. +> +> `sigma`: Describes the size of the Gaussian. + +Our modeling task is to fit the data with a 1D Gaussian and recover its parameters (`x`, `N`, `sigma`). + +## Model + +We therefore need to define a 1D Gaussian as a **PyAutoFit** model. + +We do this by writing it as the following Python class: + +```python +class Gaussian: + def __init__( + self, + centre=0.0, # <- PyAutoFit recognises these constructor arguments + normalization=0.1, # <- are the Gaussian`s model parameters. + sigma=0.01, + ): + """ + Represents a 1D `Gaussian` profile, which can be treated as a + PyAutoFit model-component whose free parameters (centre, + normalization and sigma) are fitted for by a non-linear search. + + Parameters + ---------- + centre + The x coordinate of the profile centre. + normalization + Overall normalization of the `Gaussian` profile. + sigma + The sigma value controlling the size of the Gaussian. + """ + self.centre = centre + self.normalization = normalization + self.sigma = sigma + + def model_data_from(self, xvalues: np.ndarray) -> np.ndarray: + """ + Returns the 1D Gaussian profile on a line of Cartesian x coordinates. + + The input xvalues are translated to a coordinate system centred on the + Gaussian, by subtracting its centre. + + The output is referred to as the `model_data` to signify that it is + a representation of the data from the model. + + Parameters + ---------- + xvalues + The x coordinates for which the Gaussian is evaluated. + """ + transformed_xvalues = xvalues - self.centre + + return np.multiply( + np.divide(self.normalization, self.sigma * np.sqrt(2.0 * np.pi)), + np.exp(-0.5 * np.square(np.divide(transformed_xvalues, self.sigma))), + ) +``` + +The **PyAutoFit** model above uses the following format: + +- The name of the class is the name of the model, in this case, "Gaussian". +- The input arguments of the constructor (the `__init__` method) are the parameters of the model, in this case `centre`, `normalization` and `sigma`. +- The default values of the input arguments define whether a parameter is a single-valued `float` or a multi-valued `tuple`. In this case, all 3 input parameters are floats. +- It includes functions associated with that model component, which are used when fitting the model to data. + +To compose a model using the `Gaussian` class above we use the `af.Model` object. + +```python +model = af.Model(Gaussian) +print("Model ``Gaussian`` object: \n") +print(model) +``` + +This gives the following output: + +```bash +Model `Gaussian` object: + +Gaussian (centre, UniformPrior [1], lower_limit = 0.0, upper_limit = 100.0), +(normalization, LogUniformPrior [2], lower_limit = 1e-06, upper_limit = 1000000.0), +(sigma, UniformPrior [3], lower_limit = 0.0, upper_limit = 25.0) +``` + +:::{note} +**PyAutoFit** supports the use of configuration files defining the default priors on every model parameter, which is +how the priors above were set. This allows the user to set up default priors in a consistent and concise way, but +with a high level of customization and extensibility. The use of config files to set up default behaviour is +described in the [configs cookbook](https://pyautofit.readthedocs.io/en/latest/cookbooks/configs.html). +::: + +The model has a total of 3 parameters: + +```python +print(model.total_free_parameters) +``` + +All model information is given by printing its `info` attribute: + +```python +print(model.info) +``` + +This gives the following output: + +```bash +Total Free Parameters = 3 + +model Gaussian (N=3) + +centre UniformPrior [1], lower_limit = 0.0, upper_limit = 100.0 +normalization LogUniformPrior [2], lower_limit = 1e-06, upper_limit = 1000000.0 +sigma UniformPrior [3], lower_limit = 0.0, upper_limit = 25.0 +``` + +The priors can be manually altered as follows: + +```python +model.centre = af.UniformPrior(lower_limit=0.0, upper_limit=100.0) +model.normalization = af.UniformPrior(lower_limit=0.0, upper_limit=1e2) +model.sigma = af.UniformPrior(lower_limit=0.0, upper_limit=30.0) +``` + +Printing the `model.info` displayed these updated priors. + +```python +print(model.info) +``` + +This gives the following output: + +```bash +Total Free Parameters = 3 + +model Gaussian (N=3) + +centre UniformPrior [4], lower_limit = 0.0, upper_limit = 100.0 +normalization UniformPrior [5], lower_limit = 0.0, upper_limit = 100.0 +sigma UniformPrior [6], lower_limit = 0.0, upper_limit = 30.0 +``` + +:::{note} +The example above uses the most basic PyAutoFit API to compose a simple model. The API is highly extensible and +can scale to models with thousands of parameters, complex hierarchies and relationships between parameters. +A complete overview is given in the [model cookbook](https://pyautofit.readthedocs.io/en/latest/cookbooks/model.html). +::: + +## Instances + +Instances of a **PyAutoFit** model (created via `af.Model`) can be generated by mapping an input `vector` of parameter +values to create an instance of the model's Python class. + +To define the input `vector` correctly, we need to know the order of parameters in the model. This information is +contained in the model's `paths` attribute. + +```python +print(model.paths) +``` + +This gives the following output: + +```bash +[('centre',), ('normalization',), ('sigma',)] +``` + +We input values for the three free parameters of our model in the order specified by the `paths` +attribute (i.e., `centre=30.0`, `normalization=2.0`, and `sigma=3.0`): + +```python +instance = model.instance_from_vector(vector=[30.0, 2.0, 3.0]) +``` + +This is an instance of the `Gaussian` class. + +```python +print("Model Instance: \n") +print(instance) +``` + +This gives the following output: + +```bash +Model Instance: + +<__main__.Gaussian object at 0x7f3e37cb1990> +``` + +It has the parameters of the `Gaussian` with the values input above. + +```python +print("Instance Parameters \n") +print("x = ", instance.centre) +print("normalization = ", instance.normalization) +print("sigma = ", instance.sigma) +``` + +This gives the following output: + +```bash +Instance Parameters + +x = 30.0 +normalization = 2.0 +sigma = 3.0 +``` + +We can use functions associated with the class, specifically the `model_data_from` function, to +create a realization of the `Gaussian` and plot it. + +```python +xvalues = np.arange(0.0, 100.0, 1.0) + +model_data = instance.model_data_from(xvalues=xvalues) + +plt.plot(xvalues, model_data, color="r") +plt.title("1D Gaussian Model Data.") +plt.xlabel("x values of profile") +plt.ylabel("Gaussian Value") +plt.show() +plt.clf() +``` + +Here is what the plot looks like: + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/images/model_gaussian.png +:alt: Alternative text +:width: 600 +``` + +:::{note} +Mapping models to instance of their Python classes is an integral part of the core **PyAutoFit** API. It enables +the advanced model composition and results management tools illustrated in the following overviews and cookbooks. +::: + +## Analysis + +We now tell **PyAutoFit** how to fit the model to the data. + +We define an `Analysis` class, which includes: + +- An `__init__` constructor that takes `data` and `noise_map` as inputs (this can be extended with additional elements necessary for fitting the model to the data). +- A `log_likelihood_function` that defines how to fit an `instance` of the model to the data and return a log likelihood value. + +Read the comments and docstrings of the `Analysis` class in detail for a full description of how the analysis works. + +```python +class Analysis(af.Analysis): + def __init__(self, data: np.ndarray, noise_map: np.ndarray): + """ + The ``Analysis`` class acts as an interface between the data and + model in **PyAutoFit**. + + Its ``log_likelihood_function`` defines how the model is fitted to + the data and it is called many times by the non-linear search fitting + algorithm. + + In this example the ``Analysis`` ``__init__`` constructor only contains + the ``data`` and ``noise-map``, but it can be easily extended to + include other quantities. + + Parameters + ---------- + data + A 1D numpy array containing the data (e.g. a noisy 1D signal) + fitted in the readthedocs and workspace examples. + noise_map + A 1D numpy array containing the noise values of the data, used + for computing the goodness of fit metric, the log likelihood. + """ + + super().__init__() + + self.data = data + self.noise_map = noise_map + + def log_likelihood_function(self, instance) -> float: + """ + Returns the log likelihood of a fit of a 1D Gaussian to the dataset. + + The data is fitted using an ``instance`` of the ``Gaussian`` class + where its ``model_data_from`` is called in order to + create a model data representation of the Gaussian that is fitted to the data. + """ + + """ + The ``instance`` that comes into this method is an instance of the ``Gaussian`` + model above, which was created via ``af.Model()``. + + The parameter values are chosen by the non-linear search and therefore are based + on where it has mapped out the high likelihood regions of parameter space are. + + The lines of Python code are commented out below to prevent excessive print + statements when we run the non-linear search, but feel free to uncomment + them and run the search to see the parameters of every instance + that it fits. + + # print("Gaussian Instance:") + # print("Centre = ", instance.centre) + # print("Normalization = ", instance.normalization) + # print("Sigma = ", instance.sigma) + """ + + """ + Get the range of x-values the data is defined on, to evaluate the model of the Gaussian. + """ + xvalues = np.arange(self.data.shape[0]) + + """ + Use these xvalues to create model data of our Gaussian. + """ + model_data = instance.model_data_from(xvalues=xvalues) + + """ + Fit the model gaussian line data to the observed data, computing the residuals, + chi-squared and log likelihood. + """ + residual_map = self.data - model_data + chi_squared_map = (residual_map / self.noise_map) ** 2.0 + chi_squared = sum(chi_squared_map) + noise_normalization = np.sum(np.log(2 * np.pi * self.noise_map**2.0)) + log_likelihood = -0.5 * (chi_squared + noise_normalization) + + return log_likelihood +``` + +Create an instance of the `Analysis` class by passing the `data` and `noise_map`. + +```python +analysis = Analysis(data=data, noise_map=noise_map) +``` + +:::{note} +The `Analysis` class shown above is the simplest example possible. The API is highly extensible and can include +model-specific output, visualization and latent variable calculations. A complete overview is given in the +[analysis cookbook](https://pyautofit.readthedocs.io/en/latest/cookbooks/analysis.html). +::: + +## Non Linear Search + +We now have a model ready to fit the data and an analysis class that performs this fit. + +Next, we need to select a fitting algorithm, known as a "non-linear search," to fit the model to the data. + +**PyAutoFit** supports various non-linear searches, which can be broadly categorized into three +types: MCMC (Markov Chain Monte Carlo), nested sampling, and maximum likelihood estimators. + +For this example, we will use the nested sampling algorithm called Dynesty. + +```python +search = af.DynestyStatic( + nlive=100, # Example how to customize the search settings +) +``` + +The default settings of the non-linear search are specified in the configuration files of **PyAutoFit**, just +like the default priors of the model components above. The ensures the basic API of your code is concise and +readable, but with the flexibility to customize the search to your specific model-fitting problem. + +:::{note} +PyAutoFit supports a wide range of non-linear searches, including detailed visualuzation, support for parallel +processing, and GPU and gradient based methods using the library JAX (). +A complete overview is given in the [searches cookbook](https://pyautofit.readthedocs.io/en/latest/cookbooks/search.html). +::: + +## Model Fit + +We begin the non-linear search by passing the model and analysis to its `fit` method. + +```python +print( + The non-linear search has begun running. + This Jupyter notebook cell with progress once the search + has completed - this could take a few minutes! +) + +result = search.fit(model=model, analysis=analysis) + +print("The search has finished run - you may now continue the notebook.") +``` + +## Result + +The result object returned by the fit provides information on the results of the non-linear search. + +The `info` attribute shows the result in a readable format. + +```python +print(result.info) +``` + +The output is as follows: + +```bash +Bayesian Evidence 167.54413502 +Maximum Log Likelihood 183.29775793 +Maximum Log Posterior 183.29775793 + +model Gaussian (N=3) + +Maximum Log Likelihood Model: + +centre 49.880 +normalization 24.802 +sigma 9.849 + + +Summary (3.0 sigma limits): + +centre 49.88 (49.51, 50.29) +normalization 24.80 (23.98, 25.67) +sigma 9.84 (9.47, 10.25) + + +Summary (1.0 sigma limits): + +centre 49.88 (49.75, 50.01) +normalization 24.80 (24.54, 25.11) +sigma 9.84 (9.73, 9.97) +``` + +Results are returned as instances of the model, as illustrated above in the instance section. + +For example, we can print the result's maximum likelihood instance. + +```python +print(result.max_log_likelihood_instance) + +print("\nModel-fit Max Log-likelihood Parameter Estimates: \n") +print("Centre = ", result.max_log_likelihood_instance.centre) +print("Normalization = ", result.max_log_likelihood_instance.normalization) +print("Sigma = ", result.max_log_likelihood_instance.sigma) +``` + +This gives the following output: + +```bash +Model-fit Max Log-likelihood Parameter Estimates: + +Centre = 49.87954357347897 +Normalization = 24.80227227310798 +Sigma = 9.84888033338011 +``` + +A benefit of the result being an instance is that we can use any of its methods to inspect the results. + +Below, we use the maximum likelihood instance to compare the maximum likelihood `Gaussian` to the data. + +```python +model_data = result.max_log_likelihood_instance.model_data_from( + xvalues=np.arange(data.shape[0]) +) + +plt.errorbar( + x=xvalues, y=data, yerr=noise_map, color="k", ecolor="k", elinewidth=1, capsize=2 +) +plt.plot(xvalues, model_data, color="r") +plt.title("Dynesty model fit to 1D Gaussian dataset.") +plt.xlabel("x values of profile") +plt.ylabel("Profile normalization") +plt.show() +plt.close() +``` + +The plot appears as follows: + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/images/toy_model_fit.png +:alt: Alternative text +:width: 600 +``` + +:::{note} +Result objects contain a wealth of information on the model-fit, including parameter and error estimates. They can +be extensively customized to include additional information specific to your scientific problem. A complete overview +is given in the [results cookbook](https://pyautofit.readthedocs.io/en/latest/cookbooks/result.html). +::: + +## Samples + +The results object also contains a `Samples` object, which contains all information on the non-linear search. + +This includes parameter samples, log likelihood values, posterior information and results internal to the specific +algorithm (e.g. the internal dynesty samples). + +Below we use the samples to plot the probability density function cornerplot of the results. + +```python +plotter = aplt.NestPlotter(samples=result.samples) +plotter.corner_anesthetic() +``` + +The plot appears as follows: + +```{image} https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/images/corner.png +:alt: Alternative text +:width: 600 +``` + +:::{note} +The [results cookbook](https://pyautofit.readthedocs.io/en/latest/cookbooks/result.html) also provides +a run through of the samples object API. +::: + +## Multiple Datasets + +Many model-fitting problems require multiple datasets to be fitted simultaneously in order to provide the best +constraints on the model. + +In **PyAutoFit**, all you have to do to fit multiple datasets is combine them with the model via `AnalysisFactor` +objects. + +```python +analysis_0 = Analysis(data=data, noise_map=noise_map) +analysis_1 = Analysis(data=data, noise_map=noise_map) + +analysis_list = [analysis_0, analysis_1] + +analysis_factor_list = [] + +for analysis in analysis_list: + + # The model can be customized here so that different model parameters are tied to each analysis. + model_analysis = model.copy() + + analysis_factor = af.AnalysisFactor(prior_model=model_analysis, analysis=analysis) + + analysis_factor_list.append(analysis_factor) +``` + +All `AnalysisFactor` objects are combined into a `FactorGraphModel`, which represents a global model fit to +multiple datasets using a graphical model structure. + +The key outcomes of this setup are: + +> - The individual log likelihoods from each `Analysis` object are summed to form the total log likelihood +> evaluated during the model-fitting process. +> - Results from all datasets are output to a unified directory, with subdirectories for visualizations +> from each analysis object, as defined by their `visualize` methods. + +This is a basic use of **PyAutoFit**'s graphical modeling capabilities, which support advanced hierarchical +and probabilistic modeling for large, multi-dataset analyses. + +To inspect the model, we print `factor_graph.global_prior_model.info`. + +```python +print(factor_graph.global_prior_model.info) +``` + +To fit multiple datasets, we pass the `FactorGraphModel` to a non-linear search. + +Unlike single-dataset fitting, we now pass the `factor_graph.global_prior_model` as the model and +the `factor_graph` itself as the analysis object. + +This structure enables simultaneous fitting of multiple datasets in a consistent and scalable way. + +```python +search = af.DynestyStatic( + nlive=100, +) + +result_list = search.fit(model=factor_graph.global_prior_model, analysis=factor_graph) +``` + +:::{note} +In the simple example above, instances of the same `Analysis` class (`analysis_0` and `analysis_1`) were +combined. However, different `Analysis` classes can also be combined. This is useful when fitting different +datasets that each require a unique `log_likelihood_function` to be fitted simultaneously. For more detailed +information and a dedicated API for customizing how the model changes across different datasets, refer to +the \[multiple datasets cookbook\](). +::: + +## Wrap Up + +This overview covers the basic functionality of **PyAutoFit** using a simple model, dataset, and model-fitting problem, +demonstrating the fundamental aspects of its API. + +By now, you should have a clear understanding of how to define and compose your own models, fit them to data using +a non-linear search, and interpret the results. + +The **PyAutoFit** API introduced here is highly extensible and customizable, making it adaptable to a wide range +of model-fitting problems. + +The next overview will delve into setting up a scientific workflow with **PyAutoFit**, utilizing its API to +optimize model-fitting efficiency and scalability for large datasets. This approach ensures that detailed scientific +interpretation of the results remains feasible and insightful. + +## Resources + +The [autofit_workspace:](https://github.com/Jammy2211/autofit_workspace/) repository on GitHub provides numerous +examples demonstrating more complex model-fitting tasks. + +This includes cookbooks, which provide a concise reference guide to the **PyAutoFit** API for advanced model-fitting: + +- \[Model Cookbook\](): Learn how to compose complex models using multiple Python classes, lists, dictionaries, NumPy arrays and customize their parameterization. +- \[Analysis Cookbook\](): Customize the analysis with model-specific output and visualization to gain deeper insights into your model fits. +- \[Searches Cookbook\](): Choose from a variety of non-linear searches and customize their behavior. This includes options like outputting results to hard disk and parallelizing the search process. +- \[Results Cookbook\](): Explore the various results available from a fit, such as parameter estimates, error estimates, model comparison metrics, and customizable visualizations. +- \[Configs Cookbook\](): Customize default settings using configuration files. This allows you to set priors, search settings, visualization preferences, and more. +- \[Multiple Dataset Cookbook\](): Learn how to fit multiple datasets simultaneously by combining their analysis classes so that their log likelihoods are summed. + +These cookbooks provide detailed guides and examples to help you leverage the **PyAutoFit** API effectively for a wide range of model-fitting tasks. diff --git a/docs/overview/the_basics.rst b/docs/overview/the_basics.rst deleted file mode 100644 index 3bdd0fd66..000000000 --- a/docs/overview/the_basics.rst +++ /dev/null @@ -1,679 +0,0 @@ -.. _the_basics: - -The Basics -========== - -**PyAutoFit** is a Python based probabilistic programming language for model fitting and Bayesian inference -of large datasets. - -The basic **PyAutoFit** API allows us a user to quickly compose a probabilistic model and fit it to data via a -log likelihood function, using a range of non-linear search algorithms (e.g. MCMC, nested sampling). - -This overview gives a run through of: - - - **Models**: Use Python classes to compose the model which is fitted to data. - - **Instances**: Create instances of the model via its Python class. - - **Analysis**: Define an ``Analysis`` class which includes the log likelihood function that fits the model to the data. - - **Searches**: Choose an MCMC, nested sampling or maximum likelihood estimator non-linear search algorithm that fits the model to the data. - - **Model Fit**: Fit the model to the data using the chosen non-linear search, with on-the-fly results and visualization. - - **Results**: Use the results of the search to interpret and visualize the model fit. -- **Samples**: Use the samples of the search to inspect the parameter samples and visualize the probability density function of the results. -- **Multiple Datasets**: Dedicated support for simultaneously fitting multiple datasets, enabling scalable analysis of large datasets. - -This overviews provides a high level of the basic API, with more advanced functionality described in the following -overviews and the **PyAutoFit** cookbooks. - -Example -------- - -To illustrate **PyAutoFit** we'll use the example modeling problem of fitting a 1D Gaussian profile to noisy data. - -To begin, lets import ``autofit`` (and ``numpy``) using the convention below: - -.. code-block:: python - - import autofit as af - import numpy as np - - -The example ``data`` with errors (black) is shown below: - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/images/data.png - :width: 600 - :alt: Alternative text - -The 1D signal was generated using a 1D Gaussian profile of the form: - -.. math:: - - g(x, I, \sigma) = \frac{N}{\sigma\sqrt{2\pi}} \exp{(-0.5 (x / \sigma)^2)} - -Where: - - ``x``: The x-axis coordinate where the ``Gaussian`` is evaluated. - - ``N``: The overall normalization of the Gaussian. - - ``sigma``: Describes the size of the Gaussian. - -Our modeling task is to fit the data with a 1D Gaussian and recover its parameters (``x``, ``N``, ``sigma``). - -Model ------ - -We therefore need to define a 1D Gaussian as a **PyAutoFit** model. - -We do this by writing it as the following Python class: - -.. code-block:: python - - class Gaussian: - def __init__( - self, - centre=0.0, # <- PyAutoFit recognises these constructor arguments - normalization=0.1, # <- are the Gaussian`s model parameters. - sigma=0.01, - ): - """ - Represents a 1D `Gaussian` profile, which can be treated as a - PyAutoFit model-component whose free parameters (centre, - normalization and sigma) are fitted for by a non-linear search. - - Parameters - ---------- - centre - The x coordinate of the profile centre. - normalization - Overall normalization of the `Gaussian` profile. - sigma - The sigma value controlling the size of the Gaussian. - """ - self.centre = centre - self.normalization = normalization - self.sigma = sigma - - def model_data_from(self, xvalues: np.ndarray) -> np.ndarray: - """ - Returns the 1D Gaussian profile on a line of Cartesian x coordinates. - - The input xvalues are translated to a coordinate system centred on the - Gaussian, by subtracting its centre. - - The output is referred to as the `model_data` to signify that it is - a representation of the data from the model. - - Parameters - ---------- - xvalues - The x coordinates for which the Gaussian is evaluated. - """ - transformed_xvalues = xvalues - self.centre - - return np.multiply( - np.divide(self.normalization, self.sigma * np.sqrt(2.0 * np.pi)), - np.exp(-0.5 * np.square(np.divide(transformed_xvalues, self.sigma))), - ) - -The **PyAutoFit** model above uses the following format: - -- The name of the class is the name of the model, in this case, "Gaussian". - -- The input arguments of the constructor (the ``__init__`` method) are the parameters of the model, in this case ``centre``, ``normalization`` and ``sigma``. - -- The default values of the input arguments define whether a parameter is a single-valued ``float`` or a multi-valued ``tuple``. In this case, all 3 input parameters are floats. - -- It includes functions associated with that model component, which are used when fitting the model to data. - - -To compose a model using the ``Gaussian`` class above we use the ``af.Model`` object. - -.. code-block:: python - - model = af.Model(Gaussian) - print("Model ``Gaussian`` object: \n") - print(model) - -This gives the following output: - -.. code-block:: bash - - Model `Gaussian` object: - - Gaussian (centre, UniformPrior [1], lower_limit = 0.0, upper_limit = 100.0), - (normalization, LogUniformPrior [2], lower_limit = 1e-06, upper_limit = 1000000.0), - (sigma, UniformPrior [3], lower_limit = 0.0, upper_limit = 25.0) - -.. note:: - - **PyAutoFit** supports the use of configuration files defining the default priors on every model parameter, which is - how the priors above were set. This allows the user to set up default priors in a consistent and concise way, but - with a high level of customization and extensibility. The use of config files to set up default behaviour is - described in the `configs cookbook `_. - -The model has a total of 3 parameters: - -.. code-block:: python - - print(model.total_free_parameters) - -All model information is given by printing its ``info`` attribute: - -.. code-block:: python - - print(model.info) - -This gives the following output: - -.. code-block:: bash - - Total Free Parameters = 3 - - model Gaussian (N=3) - - centre UniformPrior [1], lower_limit = 0.0, upper_limit = 100.0 - normalization LogUniformPrior [2], lower_limit = 1e-06, upper_limit = 1000000.0 - sigma UniformPrior [3], lower_limit = 0.0, upper_limit = 25.0 - -The priors can be manually altered as follows: - -.. code-block:: python - - model.centre = af.UniformPrior(lower_limit=0.0, upper_limit=100.0) - model.normalization = af.UniformPrior(lower_limit=0.0, upper_limit=1e2) - model.sigma = af.UniformPrior(lower_limit=0.0, upper_limit=30.0) - -Printing the ``model.info`` displayed these updated priors. - -.. code-block:: python - - print(model.info) - -This gives the following output: - -.. code-block:: bash - - Total Free Parameters = 3 - - model Gaussian (N=3) - - centre UniformPrior [4], lower_limit = 0.0, upper_limit = 100.0 - normalization UniformPrior [5], lower_limit = 0.0, upper_limit = 100.0 - sigma UniformPrior [6], lower_limit = 0.0, upper_limit = 30.0 - -.. note:: - - The example above uses the most basic PyAutoFit API to compose a simple model. The API is highly extensible and - can scale to models with thousands of parameters, complex hierarchies and relationships between parameters. - A complete overview is given in the `model cookbook `_. - -Instances ---------- - -Instances of a **PyAutoFit** model (created via `af.Model`) can be generated by mapping an input `vector` of parameter -values to create an instance of the model's Python class. - -To define the input `vector` correctly, we need to know the order of parameters in the model. This information is -contained in the model's `paths` attribute. - -.. code-block:: python - - print(model.paths) - -This gives the following output: - -.. code-block:: bash - - [('centre',), ('normalization',), ('sigma',)] - -We input values for the three free parameters of our model in the order specified by the `paths` -attribute (i.e., `centre=30.0`, `normalization=2.0`, and `sigma=3.0`): - -.. code-block:: python - - instance = model.instance_from_vector(vector=[30.0, 2.0, 3.0]) - -This is an instance of the ``Gaussian`` class. - -.. code-block:: python - - print("Model Instance: \n") - print(instance) - -This gives the following output: - -.. code-block:: bash - - Model Instance: - - <__main__.Gaussian object at 0x7f3e37cb1990> - -It has the parameters of the ``Gaussian`` with the values input above. - -.. code-block:: python - - print("Instance Parameters \n") - print("x = ", instance.centre) - print("normalization = ", instance.normalization) - print("sigma = ", instance.sigma) - -This gives the following output: - -.. code-block:: bash - - Instance Parameters - - x = 30.0 - normalization = 2.0 - sigma = 3.0 - -We can use functions associated with the class, specifically the ``model_data_from`` function, to -create a realization of the ``Gaussian`` and plot it. - -.. code-block:: python - - xvalues = np.arange(0.0, 100.0, 1.0) - - model_data = instance.model_data_from(xvalues=xvalues) - - plt.plot(xvalues, model_data, color="r") - plt.title("1D Gaussian Model Data.") - plt.xlabel("x values of profile") - plt.ylabel("Gaussian Value") - plt.show() - plt.clf() - -Here is what the plot looks like: - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/images/model_gaussian.png - :width: 600 - :alt: Alternative text - -.. note:: - - Mapping models to instance of their Python classes is an integral part of the core **PyAutoFit** API. It enables - the advanced model composition and results management tools illustrated in the following overviews and cookbooks. - -Analysis --------- - -We now tell **PyAutoFit** how to fit the model to the data. - -We define an ``Analysis`` class, which includes: - -- An ``__init__`` constructor that takes ``data`` and ``noise_map`` as inputs (this can be extended with additional elements necessary for fitting the model to the data). - -- A ``log_likelihood_function`` that defines how to fit an ``instance`` of the model to the data and return a log likelihood value. - -Read the comments and docstrings of the ``Analysis`` class in detail for a full description of how the analysis works. - -.. code-block:: python - - class Analysis(af.Analysis): - def __init__(self, data: np.ndarray, noise_map: np.ndarray): - """ - The ``Analysis`` class acts as an interface between the data and - model in **PyAutoFit**. - - Its ``log_likelihood_function`` defines how the model is fitted to - the data and it is called many times by the non-linear search fitting - algorithm. - - In this example the ``Analysis`` ``__init__`` constructor only contains - the ``data`` and ``noise-map``, but it can be easily extended to - include other quantities. - - Parameters - ---------- - data - A 1D numpy array containing the data (e.g. a noisy 1D signal) - fitted in the readthedocs and workspace examples. - noise_map - A 1D numpy array containing the noise values of the data, used - for computing the goodness of fit metric, the log likelihood. - """ - - super().__init__() - - self.data = data - self.noise_map = noise_map - - def log_likelihood_function(self, instance) -> float: - """ - Returns the log likelihood of a fit of a 1D Gaussian to the dataset. - - The data is fitted using an ``instance`` of the ``Gaussian`` class - where its ``model_data_from`` is called in order to - create a model data representation of the Gaussian that is fitted to the data. - """ - - """ - The ``instance`` that comes into this method is an instance of the ``Gaussian`` - model above, which was created via ``af.Model()``. - - The parameter values are chosen by the non-linear search and therefore are based - on where it has mapped out the high likelihood regions of parameter space are. - - The lines of Python code are commented out below to prevent excessive print - statements when we run the non-linear search, but feel free to uncomment - them and run the search to see the parameters of every instance - that it fits. - - # print("Gaussian Instance:") - # print("Centre = ", instance.centre) - # print("Normalization = ", instance.normalization) - # print("Sigma = ", instance.sigma) - """ - - """ - Get the range of x-values the data is defined on, to evaluate the model of the Gaussian. - """ - xvalues = np.arange(self.data.shape[0]) - - """ - Use these xvalues to create model data of our Gaussian. - """ - model_data = instance.model_data_from(xvalues=xvalues) - - """ - Fit the model gaussian line data to the observed data, computing the residuals, - chi-squared and log likelihood. - """ - residual_map = self.data - model_data - chi_squared_map = (residual_map / self.noise_map) ** 2.0 - chi_squared = sum(chi_squared_map) - noise_normalization = np.sum(np.log(2 * np.pi * self.noise_map**2.0)) - log_likelihood = -0.5 * (chi_squared + noise_normalization) - - return log_likelihood - -Create an instance of the ``Analysis`` class by passing the ``data`` and ``noise_map``. - -.. code-block:: python - - analysis = Analysis(data=data, noise_map=noise_map) - -.. note:: - - The `Analysis` class shown above is the simplest example possible. The API is highly extensible and can include - model-specific output, visualization and latent variable calculations. A complete overview is given in the - `analysis cookbook `_. - -Non Linear Search ------------------ - -We now have a model ready to fit the data and an analysis class that performs this fit. - -Next, we need to select a fitting algorithm, known as a "non-linear search," to fit the model to the data. - -**PyAutoFit** supports various non-linear searches, which can be broadly categorized into three -types: MCMC (Markov Chain Monte Carlo), nested sampling, and maximum likelihood estimators. - -For this example, we will use the nested sampling algorithm called Dynesty. - -.. code-block:: python - - search = af.DynestyStatic( - nlive=100, # Example how to customize the search settings - ) - - -The default settings of the non-linear search are specified in the configuration files of **PyAutoFit**, just -like the default priors of the model components above. The ensures the basic API of your code is concise and -readable, but with the flexibility to customize the search to your specific model-fitting problem. - -.. note:: - - PyAutoFit supports a wide range of non-linear searches, including detailed visualuzation, support for parallel - processing, and GPU and gradient based methods using the library JAX (https://jax.readthedocs.io/en/latest/). - A complete overview is given in the `searches cookbook `_. - -Model Fit ---------- - -We begin the non-linear search by passing the model and analysis to its ``fit`` method. - -.. code-block:: python - - print( - The non-linear search has begun running. - This Jupyter notebook cell with progress once the search - has completed - this could take a few minutes! - ) - - result = search.fit(model=model, analysis=analysis) - - print("The search has finished run - you may now continue the notebook.") - - -Result ------- - -The result object returned by the fit provides information on the results of the non-linear search. - -The ``info`` attribute shows the result in a readable format. - -.. code-block:: python - - print(result.info) - -The output is as follows: - -.. code-block:: bash - - Bayesian Evidence 167.54413502 - Maximum Log Likelihood 183.29775793 - Maximum Log Posterior 183.29775793 - - model Gaussian (N=3) - - Maximum Log Likelihood Model: - - centre 49.880 - normalization 24.802 - sigma 9.849 - - - Summary (3.0 sigma limits): - - centre 49.88 (49.51, 50.29) - normalization 24.80 (23.98, 25.67) - sigma 9.84 (9.47, 10.25) - - - Summary (1.0 sigma limits): - - centre 49.88 (49.75, 50.01) - normalization 24.80 (24.54, 25.11) - sigma 9.84 (9.73, 9.97) - -Results are returned as instances of the model, as illustrated above in the instance section. - -For example, we can print the result's maximum likelihood instance. - -.. code-block:: python - - print(result.max_log_likelihood_instance) - - print("\nModel-fit Max Log-likelihood Parameter Estimates: \n") - print("Centre = ", result.max_log_likelihood_instance.centre) - print("Normalization = ", result.max_log_likelihood_instance.normalization) - print("Sigma = ", result.max_log_likelihood_instance.sigma) - -This gives the following output: - -.. code-block:: bash - - Model-fit Max Log-likelihood Parameter Estimates: - - Centre = 49.87954357347897 - Normalization = 24.80227227310798 - Sigma = 9.84888033338011 - -A benefit of the result being an instance is that we can use any of its methods to inspect the results. - -Below, we use the maximum likelihood instance to compare the maximum likelihood ``Gaussian`` to the data. - -.. code-block:: python - - model_data = result.max_log_likelihood_instance.model_data_from( - xvalues=np.arange(data.shape[0]) - ) - - plt.errorbar( - x=xvalues, y=data, yerr=noise_map, color="k", ecolor="k", elinewidth=1, capsize=2 - ) - plt.plot(xvalues, model_data, color="r") - plt.title("Dynesty model fit to 1D Gaussian dataset.") - plt.xlabel("x values of profile") - plt.ylabel("Profile normalization") - plt.show() - plt.close() - -The plot appears as follows: - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/images/toy_model_fit.png - :width: 600 - :alt: Alternative text - -.. note:: - - Result objects contain a wealth of information on the model-fit, including parameter and error estimates. They can - be extensively customized to include additional information specific to your scientific problem. A complete overview - is given in the `results cookbook `_. - -Samples -------- - -The results object also contains a ``Samples`` object, which contains all information on the non-linear search. - -This includes parameter samples, log likelihood values, posterior information and results internal to the specific -algorithm (e.g. the internal dynesty samples). - -Below we use the samples to plot the probability density function cornerplot of the results. - -.. code-block:: python - - plotter = aplt.NestPlotter(samples=result.samples) - plotter.corner_anesthetic() - -The plot appears as follows: - -.. image:: https://raw.githubusercontent.com/rhayes777/PyAutoFit/main/docs/images/corner.png - :width: 600 - :alt: Alternative text - -.. note:: - - The `results cookbook `_ also provides - a run through of the samples object API. - -Multiple Datasets ------------------ - -Many model-fitting problems require multiple datasets to be fitted simultaneously in order to provide the best -constraints on the model. - -In **PyAutoFit**, all you have to do to fit multiple datasets is combine them with the model via ``AnalysisFactor`` -objects. - -.. code-block:: python - - analysis_0 = Analysis(data=data, noise_map=noise_map) - analysis_1 = Analysis(data=data, noise_map=noise_map) - - analysis_list = [analysis_0, analysis_1] - - analysis_factor_list = [] - - for analysis in analysis_list: - - # The model can be customized here so that different model parameters are tied to each analysis. - model_analysis = model.copy() - - analysis_factor = af.AnalysisFactor(prior_model=model_analysis, analysis=analysis) - - analysis_factor_list.append(analysis_factor) - -All ``AnalysisFactor`` objects are combined into a ``FactorGraphModel``, which represents a global model fit to -multiple datasets using a graphical model structure. - -The key outcomes of this setup are: - - - The individual log likelihoods from each ``Analysis`` object are summed to form the total log likelihood - evaluated during the model-fitting process. - - - Results from all datasets are output to a unified directory, with subdirectories for visualizations - from each analysis object, as defined by their ``visualize`` methods. - -This is a basic use of **PyAutoFit**'s graphical modeling capabilities, which support advanced hierarchical -and probabilistic modeling for large, multi-dataset analyses. - -To inspect the model, we print ``factor_graph.global_prior_model.info``. - -.. code-block:: python - - print(factor_graph.global_prior_model.info) - -To fit multiple datasets, we pass the ``FactorGraphModel`` to a non-linear search. - -Unlike single-dataset fitting, we now pass the ``factor_graph.global_prior_model`` as the model and -the ``factor_graph`` itself as the analysis object. - -This structure enables simultaneous fitting of multiple datasets in a consistent and scalable way. - -.. code-block:: python - - search = af.DynestyStatic( - nlive=100, - ) - - result_list = search.fit(model=factor_graph.global_prior_model, analysis=factor_graph) - -.. note:: - - In the simple example above, instances of the same ``Analysis`` class (``analysis_0`` and ``analysis_1``) were - combined. However, different ``Analysis`` classes can also be combined. This is useful when fitting different - datasets that each require a unique ``log_likelihood_function`` to be fitted simultaneously. For more detailed - information and a dedicated API for customizing how the model changes across different datasets, refer to - the [multiple datasets cookbook](https://pyautofit.readthedocs.io/en/latest/cookbooks/multiple_datasets.html). - -Wrap Up -------- - -This overview covers the basic functionality of **PyAutoFit** using a simple model, dataset, and model-fitting problem, -demonstrating the fundamental aspects of its API. - -By now, you should have a clear understanding of how to define and compose your own models, fit them to data using -a non-linear search, and interpret the results. - -The **PyAutoFit** API introduced here is highly extensible and customizable, making it adaptable to a wide range -of model-fitting problems. - -The next overview will delve into setting up a scientific workflow with **PyAutoFit**, utilizing its API to -optimize model-fitting efficiency and scalability for large datasets. This approach ensures that detailed scientific -interpretation of the results remains feasible and insightful. - -Resources ---------- - -The `autofit_workspace: `_ repository on GitHub provides numerous -examples demonstrating more complex model-fitting tasks. - -This includes cookbooks, which provide a concise reference guide to the **PyAutoFit** API for advanced model-fitting: - -- [Model Cookbook](https://pyautofit.readthedocs.io/en/latest/cookbooks/model.html): Learn how to compose complex models using multiple Python classes, lists, dictionaries, NumPy arrays and customize their parameterization. - -- [Analysis Cookbook](https://pyautofit.readthedocs.io/en/latest/cookbooks/search.html): Customize the analysis with model-specific output and visualization to gain deeper insights into your model fits. - -- [Searches Cookbook](https://pyautofit.readthedocs.io/en/latest/cookbooks/analysis.html): Choose from a variety of non-linear searches and customize their behavior. This includes options like outputting results to hard disk and parallelizing the search process. - -- [Results Cookbook](https://pyautofit.readthedocs.io/en/latest/cookbooks/result.html): Explore the various results available from a fit, such as parameter estimates, error estimates, model comparison metrics, and customizable visualizations. - -- [Configs Cookbook](https://pyautofit.readthedocs.io/en/latest/cookbooks/configs.html): Customize default settings using configuration files. This allows you to set priors, search settings, visualization preferences, and more. - -- [Multiple Dataset Cookbook](https://pyautofit.readthedocs.io/en/latest/cookbooks/multiple_datasets.html): Learn how to fit multiple datasets simultaneously by combining their analysis classes so that their log likelihoods are summed. - -These cookbooks provide detailed guides and examples to help you leverage the **PyAutoFit** API effectively for a wide range of model-fitting tasks. - - - diff --git a/docs/science_examples/astronomy.md b/docs/science_examples/astronomy.md new file mode 100644 index 000000000..ecde1848b --- /dev/null +++ b/docs/science_examples/astronomy.md @@ -0,0 +1,325 @@ +(astronomy)= + +# Astronomy + +This example illustrates model-component and fitting for an Astronomy science case, based are the phenomena +of strong gravitational lensing. This is the science case that sparked the development of **PyAutoFit** as a spin +off of our astronomy software [PyAutoLens](https://github.com/Jammy2211/PyAutoLens). + +The schematic below depicts a strong gravitational lens: + +```{image} https://raw.githubusercontent.com/Jammy2211/PyAutoLens/main/docs/overview/images/overview_1_lensing/schematic.jpg +:alt: Alternative text +:width: 600 +``` + +**Credit: F. Courbin, S. G. Djorgovski, G. Meylan, et al., Caltech / EPFL / WMKO** + + +A strong gravitational lens is a system consisting of multiple galaxy's down the light-of-sight to earth. To model +a strong lens, we ray-trace the traversal of light throughout the Universe so as to fit it to imaging data of a strong +lens. The amount light is deflected by is defined by the distances between each galaxy, which is called their redshift. + +## Multi-Level Models + +We therefore need a model which contains separate model-components for every galaxy, and where each galaxy contains +separate model-components describing its light and mass. A multi-level representation of this model is as follows: + +```{image} https://github.com/rhayes777/PyAutoFit/blob/main/docs/overview/image/lens_model.png?raw=true +:alt: Alternative text +:width: 600 +``` + +The image above shows that we need a model consisting of individual model-components for: + +> 1. The lens galaxy's *light* and *mass*. +> 2. The source galaxy's *light*. + +We also need each galaxy to be a **model-component** itself and for each of them to contain an additional parameter, +its `redshift`. The galaxies can then be combined into an overall model for the strong lens system. + +## Model Example + +To model the light of a galaxy, we define a `LightProfile` as a Python class, which behaves in the same way as +the `Gaussian` used in other **PyAutoFit** tutorials: + +```python +class LightProfile: + + def __init__( + self, + centre: typing.Tuple[float, float] = (0.0, 0.0), + normalization: float = 0.1, + radius: float = 0.6, + ): + """ + A light profile used in Astronomy to represent the surface brightness distribution of galaxies. + + Parameters + ---------- + centre + The (y,x) coordinates of the profile centre. + normalization + Overall normalization normalisation of the light profile. + radius + The circular radius containing half the light of this profile. + """ + + self.centre = centre + self.normalization = normalization + self.effective_radius = effective_radius + + def image_from_grid(self, grid: np.ndarray) -> np.ndarray: + """This function creates an image of the light profile, which is used in strong lens model-fitting""" + ... +``` + +We have omitted the code that creates the image from the light profile as we want to focus purely on multi-level model +composition with **PyAutoFit**. + +We also define a `MassProfile`: + +```python +class MassProfile: + def __init__( + self, + centre: typing.Tuple[float, float] = (0.0, 0.0), + mass: float = 1.0, + ): + """ + A mass profile used in Astronomy to represent the mass distribution of galaxies. + + Parameters + ---------- + centre + The (y,x) coordinates of the profile centre. + mass + The mass normalization of the profile. + """ + + self.centre = centre + self.mass = mass + + def deflections_from_grid(self, grid: np.ndarray) -> np.ndarray: + """This function describes the deflection of light due to the mass, which is used in strong lens model-fitting""" + ... +``` + +We have again omitted the code which computes how this mass profile deflects the path of light. + +We now define a `Galaxy` object, which contains instances of light and mass profiles and its redshift (e.g. distance +from Earth): + +```python +class Galaxy: + + def __init__( + self, + redshift: float, + light_profile_list: Optional[List] = None, + mass_profile_list: Optional[List] = None, + ): + """ + A galaxy, which contains light and mass profiles at a specified redshift. + + Parameters + ---------- + redshift + The redshift of the galaxy. + light_profile_list + A list of the galaxy's light profiles. + mass_profile_list + A list of the galaxy's mass profiles. + """ + + self.redshift = redshift + self.light_profile_list = light_profile_list + self.mass_profile_list = mass_profile_list + + def image_from_grid(self, grid: np.ndarray) -> np.ndarray: + """Returns the image of all light profiles.""" + ... + + def deflections_from_grid(self, grid: np.ndarray) -> np.ndarray: + """Returns the deflection angles of all mass profiles.""" + ... +``` + +If we were not composing a model, the code below shows how one would create an instance of the foreground lens galaxy, +which in the image above contains a light and mass profile: + +```python +light = LightProfile(centre=(0.0, 0.0), normalization=10.0, radius=2.0) +mass = MassProfile(centre=(0.0, 0.0), mass=0.5) + +lens = Galaxy(redshift=0.5, light_profile_list=[light], mass_profile_list=[mass]) +``` + +The code creates instances of the `LightProfile` and `MassProfile` classes and uses them to create an +instance of the `Galaxy` class. This uses a **hierarchy of Python classes**. + +## Multi-level Model + +We can compose a multi-level model using this same hierarchy of classes, using the `Model` and `Collection` objects. + +Lets first create a model of the lens galaxy: + +```python +light = af.Model(LightProfile) +mass = af.Model(MassProfile) + +lens = af.Model( + cls=Galaxy, + redshift=0.5, + light_profile_list=[light], + mass_profile_list=[mass] +) +``` + +Lets consider what the code above is doing: + +1. We use a `Model` to create the overall model component. The `cls` input is the `Galaxy` class, therefore the overall model that is created is a `Galaxy`. +2. **PyAutoFit** next inspects whether the key word argument inputs to the `Model` match any of the `__init__` constructor arguments of the `Galaxy` class. This determine if these inputs are to be composed as **model sub-components** of the overall `Galaxy` model. +3. **PyAutoFit** matches the `light_profile_list` and `mass_profile_list` inputs, noting they are passed as separate lists containing `Model`'s of the `LightProfile` and `MassProfile` classes. They are both created as sub-components of the overall `Galaxy` model. +4. It also matches the `redshift` input, making it a fixed value of 0.5 for the model and not treating it as a free parameter. + +We can confirm this by printing the `prior_count` of the lens, and noting it is 7 (4 parameters for +the `LightProfile` and 3 for the `MassProfile`). + +```python +print(lens.prior_count) +print(lens.light_profile_list[0].prior_count) +print(lens.mass_profile_list[0].prior_count) +``` + +The `lens` behaves exactly like the model-components we are used to previously. For example, we can unpack its +individual parameters to customize the model, where below we: + +> 1. Align the light profile centre and mass profile centre. +> 2. Customize the prior on the light profile `one`. +> 3. Fix the `one` of the mass profile to 0.8. + +```python +lens.light_profile_list[0].centre = lens.mass_profile_list[0].centre +lens.light_profile_list[0].one = af.UniformPrior(lower_limit=0.7, upper_limit=0.9) +lens.mass_profile_list[0].one = 0.8 +``` + +We can now create a model of our source galaxy using the same API. + +```python +source = af.Model( + astro.Galaxy, + redshift=1.0, + light_profile_list=[af.Model(astro.lp.LightProfile)] +) +``` + +We can now create our overall strong lens model, using a `Collection` in the same way we have seen previously. + +```python +model = af.Collection(galaxies=af.Collection(lens=lens, source=source)) +``` + +The model contains both galaxies in the strong lens, alongside all of their light and mass profiles. + +For every iteration of the non-linear search **PyAutoFit** generates an instance of this model, where all of the +`LightProfile`, `MassProfile` and `Galaxy` parameters of the are determined via their priors. + +An example instance is show below: + +```python +instance = model.instance_from_prior_medians() + +print("Strong Lens Model Instance:") +print("Lens Galaxy = ", instance.galaxies.lens) +print("Lens Galaxy Light = ", instance.galaxies.lens.light_profile_list) +print("Lens Galaxy Light Centre = ", instance.galaxies.lens.light_profile_list[0].centre) +print("Lens Galaxy Mass Centre = ", instance.galaxies.lens.mass_profile_list[0].centre) +print("Source Galaxy = ", instance.galaxies.source) +``` + +This model can therefore be used in a **PyAutoFit** `Analysis` class and `log_likelihood_function`. + +## Extensibility + +This example highlights how multi-level models can make certain model-fitting problem fully extensible. For example: + +> 1. A `Galaxy` class can be created using any combination of light and mass profiles. Although this was not shown + +explicitly in this example, this is because it implements their `image_from_grid` and `deflections_from_grid` methods +as the sum of individual profiles. + +> 2. The overall strong lens model can contain any number of `Galaxy`'s, as these methods and their redshifts are used + +to implement the lensing calculations in the `Analysis` class and `log_likelihood_function`. + +Thus, for problems of this nature, we can design and write code in a way that fully utilizes **PyAutoFit**'s multi-level +modeling features to compose and fits models of arbitrary complexity and dimensionality. + +To illustrate this further, consider the following dataset which is called a **strong lens galaxy cluster**: + +```{image} https://github.com/rhayes777/PyAutoFit/blob/main/docs/overview/image/cluster_example.jpg?raw=true +:alt: Alternative text +:width: 600 +``` + +For this strong lens, there are many tens of strong lens galaxies as well as multiple background source galaxies. +However, despite it being a significantly more complex system than the single-galaxy strong lens we modeled above, +our use of graphical models ensures that we can model such datasets without any additional code development, for +example: + +```python +lens_0 = af.Model( + Galaxy, + redshift=0.5, + light_profile_list=[af.Model(LightProfile)], + mass_profile_list=[af.Model(MassProfile)] +) + +lens_1 = af.Model( + Galaxy, + redshift=0.5, + light_profile_list=[af.Model(LightProfile)], + mass_profile_list=[af.Model(MassProfile)] +) + +source_0 = af.Model( + astro.Galaxy, + redshift=1.0, + light_profile_list=[af.Model(LightProfile)] +) + +# ... repeat for desired model complexity ... + +model = af.Collection( + galaxies=af.Collection( + lens_0=lens_0, + lens_1=lens_1, + source_0=source_0, + # ... repeat for desired model complexity ... + ) +) +``` + +Here is an illustration of this model's graph: + +```{image} https://github.com/rhayes777/PyAutoFit/blob/main/docs/overview/image/lens_model_cluster.png?raw=true +:alt: Alternative text +:width: 600 +``` + +**PyAutoFit** therefore gives us full control over the composition and customization of high dimensional graphical +models. + +## Wrap-Up + +An example project on the **autofit_workspace** shows how to use **PyAutoFit** to set up code which fits strong +lensing data, using **multi-level model composition**. + +If you'd like to perform the fit shown in this script, checkout the +[simple examples](https://github.com/Jammy2211/autofit_workspace/tree/main/notebooks/overview/simplee) on the +`autofit_workspace`. We detail how **PyAutoFit** works in the first 3 tutorials of +the [HowToFit lecture series](https://github.com/PyAutoLabs/HowToFit). + + diff --git a/docs/science_examples/astronomy.rst b/docs/science_examples/astronomy.rst deleted file mode 100644 index 24d7644d4..000000000 --- a/docs/science_examples/astronomy.rst +++ /dev/null @@ -1,328 +0,0 @@ -.. _astronomy: - -Astronomy -========= - -This example illustrates model-component and fitting for an Astronomy science case, based are the phenomena -of strong gravitational lensing. This is the science case that sparked the development of **PyAutoFit** as a spin -off of our astronomy software `PyAutoLens `_. - -The schematic below depicts a strong gravitational lens: - -.. image:: https://raw.githubusercontent.com/Jammy2211/PyAutoLens/main/docs/overview/images/overview_1_lensing/schematic.jpg - :width: 600 - :alt: Alternative text - -**Credit: F. Courbin, S. G. Djorgovski, G. Meylan, et al., Caltech / EPFL / WMKO** -https://www.astro.caltech.edu/~george/qsolens/ - -A strong gravitational lens is a system consisting of multiple galaxy's down the light-of-sight to earth. To model -a strong lens, we ray-trace the traversal of light throughout the Universe so as to fit it to imaging data of a strong -lens. The amount light is deflected by is defined by the distances between each galaxy, which is called their redshift. - -Multi-Level Models ------------------- - -We therefore need a model which contains separate model-components for every galaxy, and where each galaxy contains -separate model-components describing its light and mass. A multi-level representation of this model is as follows: - -.. image:: https://github.com/rhayes777/PyAutoFit/blob/main/docs/overview/image/lens_model.png?raw=true - :width: 600 - :alt: Alternative text - -The image above shows that we need a model consisting of individual model-components for: - - 1) The lens galaxy's *light* and *mass*. - 2) The source galaxy's *light*. - -We also need each galaxy to be a **model-component** itself and for each of them to contain an additional parameter, -its ``redshift``. The galaxies can then be combined into an overall model for the strong lens system. - -Model Example -------------- - -To model the light of a galaxy, we define a ``LightProfile`` as a Python class, which behaves in the same way as -the ``Gaussian`` used in other **PyAutoFit** tutorials: - -.. code-block:: python - - class LightProfile: - - def __init__( - self, - centre: typing.Tuple[float, float] = (0.0, 0.0), - normalization: float = 0.1, - radius: float = 0.6, - ): - """ - A light profile used in Astronomy to represent the surface brightness distribution of galaxies. - - Parameters - ---------- - centre - The (y,x) coordinates of the profile centre. - normalization - Overall normalization normalisation of the light profile. - radius - The circular radius containing half the light of this profile. - """ - - self.centre = centre - self.normalization = normalization - self.effective_radius = effective_radius - - def image_from_grid(self, grid: np.ndarray) -> np.ndarray: - """This function creates an image of the light profile, which is used in strong lens model-fitting""" - ... - -We have omitted the code that creates the image from the light profile as we want to focus purely on multi-level model -composition with **PyAutoFit**. - -We also define a ``MassProfile``: - -.. code-block:: python - - class MassProfile: - def __init__( - self, - centre: typing.Tuple[float, float] = (0.0, 0.0), - mass: float = 1.0, - ): - """ - A mass profile used in Astronomy to represent the mass distribution of galaxies. - - Parameters - ---------- - centre - The (y,x) coordinates of the profile centre. - mass - The mass normalization of the profile. - """ - - self.centre = centre - self.mass = mass - - def deflections_from_grid(self, grid: np.ndarray) -> np.ndarray: - """This function describes the deflection of light due to the mass, which is used in strong lens model-fitting""" - ... - -We have again omitted the code which computes how this mass profile deflects the path of light. - -We now define a ``Galaxy`` object, which contains instances of light and mass profiles and its redshift (e.g. distance -from Earth): - -.. code-block:: python - - class Galaxy: - - def __init__( - self, - redshift: float, - light_profile_list: Optional[List] = None, - mass_profile_list: Optional[List] = None, - ): - """ - A galaxy, which contains light and mass profiles at a specified redshift. - - Parameters - ---------- - redshift - The redshift of the galaxy. - light_profile_list - A list of the galaxy's light profiles. - mass_profile_list - A list of the galaxy's mass profiles. - """ - - self.redshift = redshift - self.light_profile_list = light_profile_list - self.mass_profile_list = mass_profile_list - - def image_from_grid(self, grid: np.ndarray) -> np.ndarray: - """Returns the image of all light profiles.""" - ... - - def deflections_from_grid(self, grid: np.ndarray) -> np.ndarray: - """Returns the deflection angles of all mass profiles.""" - ... - -If we were not composing a model, the code below shows how one would create an instance of the foreground lens galaxy, -which in the image above contains a light and mass profile: - -.. code-block:: python - - light = LightProfile(centre=(0.0, 0.0), normalization=10.0, radius=2.0) - mass = MassProfile(centre=(0.0, 0.0), mass=0.5) - - lens = Galaxy(redshift=0.5, light_profile_list=[light], mass_profile_list=[mass]) - -The code creates instances of the ``LightProfile`` and ``MassProfile`` classes and uses them to create an -instance of the ``Galaxy`` class. This uses a **hierarchy of Python classes**. - -Multi-level Model ------------------ - -We can compose a multi-level model using this same hierarchy of classes, using the ``Model`` and ``Collection`` objects. - -Lets first create a model of the lens galaxy: - -.. code-block:: python - - light = af.Model(LightProfile) - mass = af.Model(MassProfile) - - lens = af.Model( - cls=Galaxy, - redshift=0.5, - light_profile_list=[light], - mass_profile_list=[mass] - ) - -Lets consider what the code above is doing: - -1) We use a ``Model`` to create the overall model component. The ``cls`` input is the ``Galaxy`` class, therefore the overall model that is created is a ``Galaxy``. - -2) **PyAutoFit** next inspects whether the key word argument inputs to the ``Model`` match any of the ``__init__`` constructor arguments of the ``Galaxy`` class. This determine if these inputs are to be composed as **model sub-components** of the overall ``Galaxy`` model. - -3) **PyAutoFit** matches the ``light_profile_list`` and ``mass_profile_list`` inputs, noting they are passed as separate lists containing ``Model``'s of the ``LightProfile`` and ``MassProfile`` classes. They are both created as sub-components of the overall ``Galaxy`` model. - -4) It also matches the ``redshift`` input, making it a fixed value of 0.5 for the model and not treating it as a free parameter. - -We can confirm this by printing the ``prior_count`` of the lens, and noting it is 7 (4 parameters for -the ``LightProfile`` and 3 for the ``MassProfile``). - -.. code-block:: python - - print(lens.prior_count) - print(lens.light_profile_list[0].prior_count) - print(lens.mass_profile_list[0].prior_count) - -The ``lens`` behaves exactly like the model-components we are used to previously. For example, we can unpack its -individual parameters to customize the model, where below we: - - 1) Align the light profile centre and mass profile centre. - 2) Customize the prior on the light profile ``one``. - 3) Fix the ``one`` of the mass profile to 0.8. - -.. code-block:: python - - lens.light_profile_list[0].centre = lens.mass_profile_list[0].centre - lens.light_profile_list[0].one = af.UniformPrior(lower_limit=0.7, upper_limit=0.9) - lens.mass_profile_list[0].one = 0.8 - -We can now create a model of our source galaxy using the same API. - -.. code-block:: python - - source = af.Model( - astro.Galaxy, - redshift=1.0, - light_profile_list=[af.Model(astro.lp.LightProfile)] - ) - -We can now create our overall strong lens model, using a ``Collection`` in the same way we have seen previously. - -.. code-block:: python - - model = af.Collection(galaxies=af.Collection(lens=lens, source=source)) - -The model contains both galaxies in the strong lens, alongside all of their light and mass profiles. - -For every iteration of the non-linear search **PyAutoFit** generates an instance of this model, where all of the -``LightProfile``, ``MassProfile`` and ``Galaxy`` parameters of the are determined via their priors. - -An example instance is show below: - -.. code-block:: python - - instance = model.instance_from_prior_medians() - - print("Strong Lens Model Instance:") - print("Lens Galaxy = ", instance.galaxies.lens) - print("Lens Galaxy Light = ", instance.galaxies.lens.light_profile_list) - print("Lens Galaxy Light Centre = ", instance.galaxies.lens.light_profile_list[0].centre) - print("Lens Galaxy Mass Centre = ", instance.galaxies.lens.mass_profile_list[0].centre) - print("Source Galaxy = ", instance.galaxies.source) - -This model can therefore be used in a **PyAutoFit** ``Analysis`` class and ``log_likelihood_function``. - -Extensibility -------------- - -This example highlights how multi-level models can make certain model-fitting problem fully extensible. For example: - - 1) A ``Galaxy`` class can be created using any combination of light and mass profiles. Although this was not shown -explicitly in this example, this is because it implements their ``image_from_grid`` and ``deflections_from_grid`` methods -as the sum of individual profiles. - - 2) The overall strong lens model can contain any number of ``Galaxy``'s, as these methods and their redshifts are used -to implement the lensing calculations in the ``Analysis`` class and ``log_likelihood_function``. - -Thus, for problems of this nature, we can design and write code in a way that fully utilizes **PyAutoFit**'s multi-level -modeling features to compose and fits models of arbitrary complexity and dimensionality. - -To illustrate this further, consider the following dataset which is called a **strong lens galaxy cluster**: - -.. image:: https://github.com/rhayes777/PyAutoFit/blob/main/docs/overview/image/cluster_example.jpg?raw=true - :width: 600 - :alt: Alternative text - -For this strong lens, there are many tens of strong lens galaxies as well as multiple background source galaxies. -However, despite it being a significantly more complex system than the single-galaxy strong lens we modeled above, -our use of graphical models ensures that we can model such datasets without any additional code development, for -example: - -.. code-block:: python - - lens_0 = af.Model( - Galaxy, - redshift=0.5, - light_profile_list=[af.Model(LightProfile)], - mass_profile_list=[af.Model(MassProfile)] - ) - - lens_1 = af.Model( - Galaxy, - redshift=0.5, - light_profile_list=[af.Model(LightProfile)], - mass_profile_list=[af.Model(MassProfile)] - ) - - source_0 = af.Model( - astro.Galaxy, - redshift=1.0, - light_profile_list=[af.Model(LightProfile)] - ) - - # ... repeat for desired model complexity ... - - model = af.Collection( - galaxies=af.Collection( - lens_0=lens_0, - lens_1=lens_1, - source_0=source_0, - # ... repeat for desired model complexity ... - ) - ) - -Here is an illustration of this model's graph: - -.. image:: https://github.com/rhayes777/PyAutoFit/blob/main/docs/overview/image/lens_model_cluster.png?raw=true - :width: 600 - :alt: Alternative text - -**PyAutoFit** therefore gives us full control over the composition and customization of high dimensional graphical -models. - -Wrap-Up -------- - -An example project on the **autofit_workspace** shows how to use **PyAutoFit** to set up code which fits strong -lensing data, using **multi-level model composition**. - -If you'd like to perform the fit shown in this script, checkout the -`simple examples `_ on the -``autofit_workspace``. We detail how **PyAutoFit** works in the first 3 tutorials of -the `HowToFit lecture series `_. - -https://github.com/Jammy2211/autofit_workspace/tree/release/projects/astro \ No newline at end of file