451 python upgrade

[dhvanildesai]

Resolves #451

Upgrading to python 3.13.14 instead of 3.14 to avoid major Django changes. I have also added sparclclient and ppxf as required packages, which are being used for fetching host galaxy spectra and spectral fitting.

Also updated Dockerfile to install gfortran first before any pip installs because otherwise fsps has build dependency issues.

[manning-ncsa]

Although unit tests pass using bash run/blastctl ci up, running "full_dev" profile at commit 64edbd2 fails to complete a transient workflow, erroring at the Local Host SED Fitting task (see log below). This is probably a failure of our unit tests to capture tasks in the workflow.

We should take this opportunity to update the Python dependencies defined in requirements.txt. I will remove the version specs (aside from pinning Django v5) and see how close to the latest for these packages I can reach, then I'll push my updates. @dhvanildesai standby.

[2026-07-21 16:52:08,608: ERROR/ForkPoolWorker-27] Task Local Host SED Fitting[3e4ed282-498c-42c3-a8cd-e5d60df41ed7] raised unexpected: ModuleNotFoundError("No module named 'arviz.data'")
Traceback (most recent call last):
  File "/usr/local/lib/python3.13/site-packages/celery/app/trace.py", line 585, in trace_task
    R = retval = fun(*args, **kwargs)
                 ~~~^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/celery/app/trace.py", line 858, in __protected_call__
    return self.run(*args, **kwargs)
           ~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/app/host/transient_tasks.py", line 1534, in local_host_sed_fitting
    LocalHostSEDFitting(transient_name).run_process()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/app/host/transient_tasks.py", line 209, in run_process
    status_message = self._run_process(transient)
  File "/app/host/transient_tasks.py", line 1117, in _run_process
    status_message = super()._run_process(
        transient, aperture_type="local", mode=mode
    )
  File "/app/host/transient_tasks.py", line 1037, in _run_process
    posterior, errflag = fit_model(
                         ~~~~~~~~~^
        observations,
        ^^^^^^^^^^^^^
    ...<3 lines>...
        fit_type=aperture_type,
        ^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/app/host/prospector.py", line 753, in fit_model
    from host.SBI.run_sbi_blast import fit_sbi_pp
  File "/app/host/SBI/run_sbi_blast.py", line 13, in <module>
    from sbi import inference as Inference
  File "/usr/local/lib/python3.13/site-packages/sbi/inference/__init__.py", line 16, in <module>
    from sbi.inference.base import (  # noqa: F401
    ...<4 lines>...
    )
  File "/usr/local/lib/python3.13/site-packages/sbi/inference/base.py", line 19, in <module>
    from sbi.inference.posteriors.base_posterior import NeuralPosterior
  File "/usr/local/lib/python3.13/site-packages/sbi/inference/posteriors/__init__.py", line 3, in <module>
    from sbi.inference.posteriors.mcmc_posterior import MCMCPosterior
  File "/usr/local/lib/python3.13/site-packages/sbi/inference/posteriors/mcmc_posterior.py", line 11, in <module>
    from arviz.data import InferenceData
ModuleNotFoundError: No module named 'arviz.data'

[manning-ncsa]

I am also seeing SyntaxWarnings related to escape sequences that were apparently introduced in Python 3.12. These might be resolved by making these instances raw string literals r''

app-1           | "/mnt/data/.initializing_db" not found. Running database initialization script...
app-1           | /app/host/models.py:343: SyntaxWarning: invalid escape sequence '\s'
app-1           |   transmission_curve = pd.read_csv(curve_name, sep="\s+", header=None)  # noqa
app-1           | /app/host/models.py:366: SyntaxWarning: invalid escape sequence '\s'
app-1           |   corr_model = pd.read_csv(corr_model_name, sep="\s+", header=None)  # noqa
app-1           | /app/host/cutouts.py:246: SyntaxWarning: invalid escape sequence '\s'
app-1           |   filename_table = pd.read_csv(r.raw, sep="\s+")["filename"]
app-1           | /app/host/cutouts.py:615: SyntaxWarning: invalid escape sequence '\/'
app-1           |   regex = "<dt>run<\/dt>.*<dd>.*<\/dd>"
app-1           | /app/host/prospector.py:295: SyntaxWarning: invalid escape sequence '\S'
app-1           |   log10(\Sum_i M_i) to values of M_i.  j=0 is the most recent bin in lookback
app-1           | /app/host/prospector.py:562: SyntaxWarning: invalid escape sequence '\S'
app-1           |   log10(\Sum_i M_i) to values of M_i.  j=0 is the most recent bin in lookback
app-1           | /app/host/views.py:546: SyntaxWarning: invalid escape sequence '\o'
app-1           |   "{\\rm log}_{10}(M_{\\ast}/M_{\odot})\,",  # noqa
app-1           | /app/host/views.py:549: SyntaxWarning: invalid escape sequence '\ '
app-1           |   "{\\rm stellar\ age}",  # noqa
app-1           | /app/host/views.py:550: SyntaxWarning: invalid escape sequence '\o'
app-1           |   "{\\rm log}_{10}(Z_{\\ast}/Z_{\odot})",  # noqa
app-1           | /app/host/views.py:551: SyntaxWarning: invalid escape sequence '\o'
app-1           |   "{\\rm log}_{10}(Z_{gas}/Z_{\odot})\,",  # noqa
app-1           | /app/host/views.py:553: SyntaxWarning: invalid escape sequence '\d'
app-1           |   "\delta",  # noqa
app-1           | /app/host/views.py:557: SyntaxWarning: invalid escape sequence '\g'
app-1           |   "{\\rm log}_{10}(\gamma_e)\,",  # noqa
app-1           | /app/host/views.py:558: SyntaxWarning: invalid escape sequence '\,'
app-1           |   "{\\rm log}_{10}(f_{AGN})\,",  # noqa
app-1           | /app/host/views.py:559: SyntaxWarning: invalid escape sequence '\,'
app-1           |   "{\\rm log}_{10}(\\tau_{AGN})\,"  # noqa

[manning-ncsa]

As of commit de80835, unit tests pass, but the workflow fails at SED fitting.

celery-worker-1  | [2026-07-21 19:13:27,527: ERROR/ForkPoolWorker-9] Task Local Host SED Fitting[b87c5ae2-16b7-4132-9174-99700013e5ab] raised unexpected: AttributeError("'Flow' object has no attribute 'condition_shape'")
celery-worker-1  | Traceback (most recent call last):
celery-worker-1  |   File "/usr/local/lib/python3.13/site-packages/celery/app/trace.py", line 585, in trace_task
celery-worker-1  |     R = retval = fun(*args, **kwargs)
celery-worker-1  |                  ~~~^^^^^^^^^^^^^^^^^
celery-worker-1  |   File "/usr/local/lib/python3.13/site-packages/celery/app/trace.py", line 858, in __protected_call__
celery-worker-1  |     return self.run(*args, **kwargs)
celery-worker-1  |            ~~~~~~~~^^^^^^^^^^^^^^^^^
celery-worker-1  |   File "/app/host/transient_tasks.py", line 1534, in local_host_sed_fitting
celery-worker-1  |     LocalHostSEDFitting(transient_name).run_process()
celery-worker-1  |     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
celery-worker-1  |   File "/app/host/transient_tasks.py", line 209, in run_process
celery-worker-1  |     status_message = self._run_process(transient)
celery-worker-1  |   File "/app/host/transient_tasks.py", line 1117, in _run_process
celery-worker-1  |     status_message = super()._run_process(
celery-worker-1  |         transient, aperture_type="local", mode=mode
celery-worker-1  |     )
celery-worker-1  |   File "/app/host/transient_tasks.py", line 1037, in _run_process
celery-worker-1  |     posterior, errflag = fit_model(
celery-worker-1  |                          ~~~~~~~~~^
celery-worker-1  |         observations,
celery-worker-1  |         ^^^^^^^^^^^^^
celery-worker-1  |     ...<3 lines>...
celery-worker-1  |         fit_type=aperture_type,
celery-worker-1  |         ^^^^^^^^^^^^^^^^^^^^^^^
celery-worker-1  |     )
celery-worker-1  |     ^
celery-worker-1  |   File "/app/host/prospector.py", line 755, in fit_model
celery-worker-1  |     output, errflag = fit_sbi_pp(observations, fit_type=fit_type)
celery-worker-1  |                       ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
celery-worker-1  |   File "/app/host/SBI/run_sbi_blast.py", line 281, in fit_sbi_pp
celery-worker-1  |     chain, obs, flags = sbi_pp.sbi_pp(
celery-worker-1  |                         ~~~~~~~~~~~~~^
celery-worker-1  |         obs=obs, run_params=run_params, sbi_params=sbi_params
celery-worker-1  |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
celery-worker-1  |     )
celery-worker-1  |     ^
celery-worker-1  |   File "/app/host/SBI/sbi_pp.py", line 896, in sbi_pp
celery-worker-1  |     ) = sbi_missing_and_noisy(obs=obs, run_params=run_params, sbi_params=sbi_params)
celery-worker-1  |         ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
celery-worker-1  |   File "/app/host/SBI/sbi_pp.py", line 658, in sbi_missing_and_noisy
celery-worker-1  |     noiseless_theta = hatp_x_y.sample(
celery-worker-1  |         (npost,),
celery-worker-1  |         x=torch.as_tensor(samp_y_guess).to(device),
celery-worker-1  |         show_progress_bars=False,
celery-worker-1  |     )
celery-worker-1  |   File "/usr/local/lib/python3.13/site-packages/sbi/inference/posteriors/direct_posterior.py", line 171, in sample
celery-worker-1  |     x, event_shape=self.posterior_estimator.condition_shape
celery-worker-1  |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
celery-worker-1  |   File "/usr/local/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1967, in __getattr__
celery-worker-1  |     raise AttributeError(
celery-worker-1  |         f"'{type(self).__name__}' object has no attribute '{name}'"
celery-worker-1  |     )
celery-worker-1  | AttributeError: 'Flow' object has no attribute 'condition_shape'

[manning-ncsa]

For future reference, arviz must be less than v1.0.0 due to a backwards incompatibility that broke sbi. This appears to be fixed as of the sbi v0.26.0, but specifying v0.26 causes other errors when running the workflow tasks.

[manning-ncsa]

Unit tests pass at commit 1a5b3ae.

@dhvanildesai Would you fix those SyntaxWarnings?

Also, before we merge this, I want to reprocess one of the initial transients like 2026dgt and compare the exported data to v1.13.1.

[manning-ncsa]

@djones1040 As of commit 1a5b3ae, the computed data for 2026dix is significantly different to my untrained eye. Take a look at the attached export files and see what you think:

2026dix.new.json
2026dix.original.json

[djones1040]

@djones1040 As of commit 1a5b3ae, the computed data for 2026dix is significantly different to my untrained eye. Take a look at the attached export files and see what you think:

2026dix.new.json 2026dix.original.json

what differences are you seeing? Took only a quick glance but looks to me like photometry is the same, just things are in a different order (had to match up the filter PKs). Aperture orientation changed but -63 is the same as +297 so all good there (this is the photutils update probably). SED parameters are largely the same, and this is done via monte carlo sampling so we get some changes iteration to iteration.

[dhvanildesai]

@dhvanildesai Would you fix those SyntaxWarnings?

SyntaxWarnings fixed as of commit 0cc2433

[manning-ncsa]

Took only a quick glance but looks to me like photometry is the same, just things are in a different order (had to match up the filter PKs).

Since they look similar enough to you to be considered "the same", then we will proceed with the merge.

[manning-ncsa]

Thanks for noticing and fixing the Bokeh issue. I had already merged this branch into omnibus but neglected to explicitly state that here. I will cherry-pick this commit into that branch and it should be fine.

I will move your second changelog entry from the "fixed" list into the "changed" list, because this was something you caused/discovered and fixed during the development cycle. In other words, it was never an actual "bug" that afflicted a public release.