Refactor of calibration / optimization workflows#64
Open
rcarson3 wants to merge 1 commit into
Open
Conversation
The older workflow/optimization workflow was starting to show its age and was no longer really useable with the v0.9 refactor of ExaConstit. Additionally, it wasn't super flexible for newer features one might be interested in or making backends easier to use. So, I threw Claude Opus at the issue because why not... In reality, my bandwidth is way more limited than I would like to re-engineer everything from scratch. It in many ways way over engineered things. However, I would say the new framework should make be more extensive to even codes outside of ExaConstit. So, one should be able to sub in a diffferent code and its input and output without too much extra effort in theory... Part of this in the super extensive test suite... I think there's something like 300 or 400 tests total... New abstractions, a much nicer backend aspect of things so that it's easier to work with like mpi or flux. The flux backend is now much nicer than previously in that it provides async status updates and makes it easier to see how jobs are progressing. Additionally, we can track whether or not each job has finished or not so during restarts we can either start at the beginning of gene or only run the ones that didn't finish. A number of different smoothers are also available to make it easier to interp from experimental data to simulation data in a sane manner. Next, the problem, case setup, objective aspect have all been abstracted out into their own thing to make it easier to compose things together / build something up. Outside of that, we also have the ability to do like SQL databases to archive the results and then query those. Finally, all of the above were common workflow aspects and don't in any way tie into the actual optimizer being used. So, the NSGA-III aspect of things lives in a completely seperate set of directories. It is now its own thing with more abstracted out set of functions that should hopefully make it ultimately easier to navigate and try to parse out what everything is doing. The final portion of code is in many ways easier to reason but further clean-up is likely needed to make things even simpler... Additionally, a few nicety files have been added for the NSGA-III stuff which can regenerate a logbook if things were wiped out but the main SQL database still lives. Also, a simple script can let you do CLI views of how the job is progressing for the different objectives. There are also a number of markdown files that should make it easier to navigate all of this / also keep what ever agentic coding tool happy / able to understand what's going on. Lastly, I've got a very simple example that largely replicates what the old optimization script was doing but this time leverages the newer framework and also shows off how to build things up for your own needs. It has a lot of comments in it so hopefully it should be not too hard to understand what's going on. The test cases are based on just the ExaConstit/test/voce_full.toml test cases but I had one case at 1e-3 1/s and the other at 1e-1 1/s.
66b611c to
0ac18f8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
[Claude / Codex drove a lot of these refactors]
The older workflow/optimization workflow was starting to show its age and was no longer really useable with the v0.9 refactor of ExaConstit. Additionally, it wasn't super flexible for newer features one might be interested in or making backends easier to use. So, I threw Claude Opus at the issue because why not...
In reality, my bandwidth is way more limited than I would like to re-engineer everything from scratch. It in many ways way over engineered things. However, I would say the new framework should make be more extensive to even codes outside of ExaConstit. So, one should be able to sub in a diffferent code and its input and output without too much extra effort in theory... Part of this in the super extensive test suite... I think there's something like 300 or 400 tests total...
New abstractions, a much nicer backend aspect of things so that it's easier to work with like mpi or flux. The flux backend is now much nicer than previously in that it provides async status updates and makes it easier to see how jobs are progressing. Additionally, we can track whether or not each job has finished or not so during restarts we can either start at the beginning of gene or only run the ones that didn't finish.
A number of different smoothers are also available to make it easier to interp from experimental data to simulation data in a sane manner.
Next, the problem, case setup, objective aspect have all been abstracted out into their own thing to make it easier to compose things together / build something up.
Outside of that, we also have the ability to do like SQL databases to archive the results and then query those.
Finally, all of the above were common workflow aspects and don't in any way tie into the actual optimizer being used. So, the NSGA-III aspect of things lives in a completely seperate set of directories. It is now its own thing with more abstracted out set of functions that should hopefully make it ultimately easier to navigate and try to parse out what everything is doing. The final portion of code is in many ways easier to reason but further clean-up is likely needed to make things even simpler...
Additionally, a few nicety files have been added for the NSGA-III stuff which can regenerate a logbook if things were wiped out but the main SQL database still lives. Also, a simple script can let you do CLI views of how the job is progressing for the different objectives.
There are also a number of markdown files that should make it easier to navigate all of this / also keep what ever agentic coding tool happy / able to understand what's going on.
Lastly, I've got a very simple example that largely replicates what the old optimization script was doing but this time leverages the newer framework and also shows off how to build things up for your own needs. It has a lot of comments in it so hopefully it should be not too hard to understand what's going on. The test cases are based on just the ExaConstit/test/voce_full.toml test cases but I had one case at 1e-3 1/s and the other at 1e-1 1/s.