Feature/182 sigimax migration - #360
Open
ThomasMalletCodra wants to merge 51 commits into
Open
Conversation
…SigimaX DataLab already imports these widgets from sigimax.widgets throughout the code base and tests; the datalab/widgets copies were orphaned duplicates carrying dead config call sites. - Delete datalab/widgets: signalbaseline, signalcursor, signaldeltax, signalpeak, imagebackground, fitdialog, logviewer, fileviewer, filedialog, wizard, warningerror, h5browser. - Fix stale docstring reference in gui/processor/signal.py (datalab.widgets.signalpeak -> sigimax.widgets.signalpeak). status.py is kept (still used as dl_status alongside the SigimaX one) and will be handled in a dedicated merge.
PLUGIN_ERROR_COLOR resolved at runtime via the package __getattr__ delegation, but static analyzers cannot follow it, raising a false "No name 'PLUGIN_ERROR_COLOR' in module 'datalab.config'" warning at its import site (datalab/widgets/status.py). Add it to the explicit re-export list and __all__ so it is a documented public symbol and the warning disappears.
The DataLab application context manager duplicated the SigimaX one almost verbatim; the only DataLab-specific behaviour is the session-start INI reload. - datalab_app_context is now a thin wrapper: it reloads persisted settings (Conf.reload_from_ini()) then delegates to sigimax_app_context, which reads DataLab's active configuration (name, version, log paths) via get_conf(). - Remove the duplicated ~90-line body and 5 orphaned helpers (close_widgets_and_quit, get_log_contents, initialize_log_file, remove_empty_log_file, QAPP_INSTANCE); they were only used by the old body. - The datalab_app_context name is kept, so all callers (including datalab_test_app_context) are unchanged. Validated: backbone + macro app tests pass (56); ruff clean.
The signal/image object adapters duplicated SigimaX almost entirely; the only DataLab addition is geometry-result rendering, which depends on adapters_metadata (intentionally kept out of SigimaX). - New objects/adapters.py provides SignalObjPlotPyAdapter and ImageObjPlotPyAdapter inheriting the SigimaX adapters (SGMXSignalObjPlotPyAdapter / SGMXImageObjPlotPyAdapter) and overriding iterate_metadata_shape_items to render GeometryAdapter metadata via GeometryPlotPyAdapter, through the shared helper _iterate_geometry_result_items. - Removes the base/signal/image duplicates (~500 lines); scalar.py (geometry/table result adapters) stays DataLab-specific; factories.py and the package __init__ are repointed to objects.adapters. Requires the SigimaX ErrorBar update_item fix. Validated: backbone + geometry and plot result rendering (60 tests) pass; ruff clean.
coordutils, annotations, the PlotPy base helpers and the ROI base/image adapters were byte-for-byte duplicates of SigimaX (verified via git diff --no-index; only import paths differed). Import them from SigimaX instead. - Delete datalab/adapters_plotpy: coordutils.py, annotations.py, base.py, roi/base.py, roi/image.py. - Repoint importers to sigimax.adapters_plotpy: package __init__, converters, factories, roi/factory, roi/signal, objects/scalar, gui/processor/base and the coordutils unit test. The DataLab-specific roi/signal adapters (curve-clipped ROI fills) stay local and are still returned by the factories. DataLab's adapters_plotpy now holds only DataLab-specific code (curve-clip ROI, geometry/table result adapters, and the assembly factories). Validated: backbone, coordutils, ROI (signal + image), geometry and roieditor tests (71) pass; ruff clean; pylint 10/10.
…UIT var
After the qthelpers migration, the Qt exec-loop/quit decision lives in
`sigimax_app_context`, which reads `sigimax.env.execenv.do_not_quit`
(`SIGIMAX_DO_NOT_QUIT`). But `run_datalab_in_background` set DataLab's own
`DATALAB_DO_NOT_QUIT`, so SigimaX never saw the flag: in unattended mode the
background DataLab process auto-quit ~6s after startup (QTimer close-and-quit),
right after its XML-RPC server had come up. The remote client then failed to
connect ("XML-RPC server is not responding").
Source the DO_NOT_QUIT env var name from `sigimax.env.SGMXExecEnv` so the flag
set for the background app is the one SigimaX actually reads. UNATTENDED_ENV was
already shared through guidata and unaffected.
…I backend `rpc_server_port` is an Optional[int]. When the option was absent from the INI, `_load_field` called `conf.get(section, key, default=None)`, which makes guidata `UserConfig` re-persist the default via `set(None)`. Since the backend infers the type from a previously stored value (an int port), it ran `int(None)` and raised `TypeError`. This propagated uncaught through `RemoteClient.connect()` (which only retries on ConnectionRefusedError/OSError), breaking remote-control connection whenever the port had been cleared/unassigned. Read the stored value only when the option exists, otherwise assign the raw default directly — mirroring `_save_field`, which clears None-valued options instead of persisting them.
The XML-RPC server port is a runtime IPC value shared across processes through the INI file (written by the running instance, read by remote clients). It was part of the flat options container, so any bulk save_options_to_ini — triggered by an unrelated option change such as window geometry persisted on close — rewrote it. Concurrent/closing DataLab instances (and every GUI test main window) thus clobbered the live server's port, making remote-control tests connect to a dead port intermittently (flaky test_call_method in the full suite, passing in isolation). Exclude rpc_server_port (RUNTIME_FIELDS) from the bulk save while still loading it, and persist it authoritatively via a dedicated single-key write (save_runtime_option) from xmlrpc_server_started — restoring the pre-migration single-key persistence semantics.
Move the shared legacy configuration backend and generic HDF5 helpers to SigimaX, and remove their duplicated DataLab copies. - Repoint DataLab configuration consumers to `sigimax.utils.conf`. - Repoint generic HDF5 imports to `sigimax.h5`. - Keep `datalab.h5.native` for DataLab-specific workspace serialization. - Update affected tests and type-checking imports. - Remove the duplicated `utils/conf.py`, `h5/common.py`, `h5/generic.py`, and `h5/utils.py` modules. Validated with 41 focused configuration and HDF5 tests, Ruff, and a search for dangling references.
Make DLExecEnv inherit the generic execution environment behavior from SigimaX instead of duplicating mode handling, serialization, context management, demo mode, and output helpers. Keep DataLab-specific behavior locally: - XML-RPC port argument and environment variable - DataLab catcher-test environment variable - DataLab command-line parsing - HDF5 file and browser arguments - DataLab configuration reset and version handling - DataLab-specific debug logging Also make clearing the XML-RPC port remove its environment variable instead of storing the literal string "None". Ruff and direct environment-contract validation pass. The existing execenv_unit.py test still reports its pre-existing screenshot_path fixture inconsistency.
Make DLExecEnv inherit generic environment handling from SGMXExecEnv while keeping DataLab-specific command-line and XML-RPC behavior local. - Preserve DataLab HDF5 and HDF5 browser arguments. - Preserve DataLab configuration reset and version handling. - Preserve the XML-RPC port and catcher-test environment variables. - Remove duplicated mode, context, serialization, demo, and output handling. - Remove the XML-RPC environment variable when its value is set to None. - Make execenv tests ignore the harness-managed screenshot path during parent/subprocess comparisons. This keeps command-line parsing under DataLab's control and avoids conflicts with the SigimaX environment singleton. Validated with Ruff and datalab/tests/backbone/execenv_unit.py: 2 tests passed.
Replace DataLab's duplicated generic Qt helper implementations with a thin compatibility facade over sigimax.utils.qthelpers. Keep only DataLab-specific behavior locally: - application context with INI reload - processor error decorator - localized screenshot naming and destination - desktop path/folder helpers - DataLab menu button icon Move plugin error handling to its SigimaX owner and update plugin tests to patch the owning module. This removes 394 net lines while preserving DataLab's public qthelpers API. Validated with Ruff, Pylint (10/10), 55 backbone tests, and 12 plugin/GUI/HDF5 integration tests.
Make DLMainWindow inherit from SGMXMainWindow and initialize DataLab-specific state and services through the protected pre-setup lifecycle hook. Keep DataLab panels, plugins, XML-RPC, Web API, native HDF5, menus, and actions local while delegating generic shell behavior to SigimaX: - geometry and window-state persistence - toolbar and dock construction - memory-state handling - close/raise control methods - shared Qt signals Preserve DataLab's enhanced color-mode behavior and public control API. This removes 139 net lines from DataLab's main window. Validated with Ruff, Pylint (10/10), and 82 backbone, plugin, HDF5, embedded, and XML-RPC tests.
Derive DLMainWindow from SGMXMainWindow and delegate generic shell behavior: - window geometry and state persistence - toolbar and dock construction - memory and modified-state handling - console creation - standard HDF5 and quit actions - release and crash-log checks - close, show, and close-event lifecycle - shared Qt signals and control helpers Keep DataLab-specific panels, plugins, XML-RPC, Web API, native HDF5, menus, actions, console content, tour, and shutdown cleanup in protected overrides. This removes 190 net lines from DataLab's main window. Validated with Ruff, Pylint (10/10), and 86 backbone, plugin, HDF5, embedded, and XML-RPC tests.
Delegate the generic DataLab window creation lifecycle to the typed SigimaX factory: - splash screen creation and resource resolution - window construction, sizing, and display - maximized-state handling - initial HDF5 loading Keep DataLab-specific startup behavior local: - initial signal and image objects - HDF5 browser command-line input - concurrent-instance protection - XML-RPC/network configuration warning and lock handling Disable generic splash progress messages to preserve DataLab's existing image-only splash appearance. Validated with Ruff, Pylint (10/10), launcher/config/instance tests, HDF5 and object workflows, and real XML-RPC control tests.
- keep DataLab_v1.ini available for DataLab 1.2 downgrades - store typed DataLab 1.3 settings in DataLab_v1_typed.ini - migrate legacy settings atomically on first startup - preserve plugin paths and specialized DataSet values - prevent raw JSON values from being replaced by optional defaults - add explicit typed/legacy backend selection for development and tests
- isolate plugin and XML-RPC settings from the user profile - snapshot and restore the typed configuration around background tests - clear stale runtime XML-RPC ports before launching child processes - propagate dynamic ports to Sigima clients - clean up orphaned DataLab processes after each test
* [UPDATE] : datalab/config/config.py:
- initialize the shared DataLabOptions instance after DataSet classes
- prevent circular imports when spawned workers restore typed options
* [UPDATE] : datalab/tests/conftest.py:
- clear inherited typed configuration before importing DataLab
- prevent user options and XML-RPC ports from contaminating tests
* [UPDATE] : datalab/config/config.py:
- make user configuration initialization explicit
- preserve the active persistence mode when resetting configuration
- remove implicit INI reloads and environment option synchronization
- keep legacy-to-typed migration and runtime defaults deterministic
* [UPDATE] : datalab/config/config_options.py:
- persist option changes through the SigimaX option_changed hook
- restore defaults and initialization state without environment snapshots
* [UPDATE] : datalab/config/config_persistence.py:
- adapt typed INI conversion to the simplified option API
- prevent runtime option writes when persistence is disabled
* [UPDATE] : datalab/config/optionfields.py:
- restore lazy DataSet deserialization from DataLab 1.2.1
- discard invalid persisted DataSet values and use defaults
* [UPDATE] : datalab/config/__init__.py:
- export explicit initialization and in-memory reset helpers
* [UPDATE] : datalab/app.py:
- support explicit user configuration loading and option overrides
- accept a runtime XML-RPC port for programmatic launches
* [UPDATE] : datalab/control/remote.py:
- bind the XML-RPC server to an explicitly requested runtime port
- initialize configuration through the explicit lifecycle
* [UPDATE] : datalab/utils/qthelpers.py:
- remove implicit INI reloads from the Qt application context
* [UPDATE] : datalab/tests/conftest.py:
- reset the shared configuration before every test
- disable XML-RPC by default and clean background processes after tests
* [UPDATE] : datalab/tests/__init__.py:
- launch background DataLab applications with multiprocessing.Process
- pass startup options and XML-RPC port explicitly
- centralize graceful and forced process cleanup
* [ADD] : datalab/data/tests/config/DataLab_v1.ini:
- add an anonymized DataLab 1.2.1 legacy configuration fixture
* [UPDATE] : datalab/tests/backbone/config_persistence_unit_test.py:
- migrate the immutable legacy fixture into a temporary typed INI
- verify typed reload and legacy source preservation
* [UPDATE] : datalab/tests/backbone/config_unit_test.py:
- exercise configuration behavior with a temporary typed backend
* [UPDATE] : datalab/tests/backbone/config_options_unit_test.py:
- remove obsolete environment synchronization assertions
- verify INI persistence and complete default-state restoration
* [UPDATE] : datalab/tests/backbone/optionfields_unit_test.py:
- cover lazy and invalid DataSet restoration behavior
Restart the shared processor pool after runtime plugin discovery so spawned workers inherit updated plugin paths and can import plugin functions during task unpickling. Add reload lifecycle and runtime-plugin spawn regressions.
Align HDF5 API docstrings with the historical comma-separated selector.
Remove PLOTPY_DEFAULTS and the ROI_*/MARKER_* color constants, which were byte-for-byte identical to SigimaXOptions.get_plotpy_defaults(), and rely on the SigimaX implementation instead. This also fixes stale theme colors being re-applied after a color mode change. Also remove dead helpers (APP_PATH, DATETIME_FORMAT, local is_frozen, get_mod_source_dir, get_old_log_fname), derive APP_NAME/CONF_VERSION from DataLabOptions, move PLOTPY_CONF.set_application() into initialize(), and drop the __getattr__ delegation from the datalab.config facade. Assisted-by: Claude Opus 5
Rename the get_raw/set_raw accessors to to_storage/from_storage and drop the DataLabOptions to_dict, from_dict and reset_to_defaults overrides, now correctly handled by the SigimaX base container. Requires SigimaX with the OptionField storage protocol. Assisted-by: Claude Opus 5
The INI layer dispatched on concrete field types in four places to decide how many keys a field occupies, whether its value needed percent-escaping and how to encode it. Derive all of this from storage_suffixes, storage_escape and to_storage/from_storage instead, so adding an option type no longer requires touching the persistence layer. Loading now registers each field's raw default with UserConfig.set_default rather than passing it as get(default=...), which used to write the default back to the INI file; font, config-path and datetime keys are no longer pre-written on first startup. A stored value that a field cannot restore is removed from the INI instead of breaking every later startup. Add FormatStringOptionField for strftime-style options, replacing the hardcoded datetime field list. Assisted-by: Claude Opus 5
…data Replace the `INI_KEY_OVERRIDES` and `RUNTIME_FIELDS` lookup tables by the `storage_key` and `runtime` attributes now carried by the option fields themselves: `get_ini_location` reads `field.storage_key` before falling back to the `ai_`/`macro_` prefix rule, and `save_options_to_ini` skips fields declaring `runtime=True`. Rename `NON_PERSISTED` to `EXPECTED_UNCATEGORIZED` and fix the module docstring: the actual rule is that an option without a category is not persisted, the constant being only the allowlist of expected such options. Assisted-by: Claude Opus 5
DataLabOptions no longer replaces inherited fields whose type differed: traceback_log_path, faulthandler_log_path and base_dir now come from SigimaX with the right field types. FontOptionField and FormatStringOptionField are imported from sigimax.config as well. Only DataSetOptionField remains local, as a thin class_module compatibility subclass. No change to the on-disk INI format. Assisted-by: Claude Opus 5
…on field shim DataLabShapeParam used to hide `label`, `readonly` and `private` by calling `set_prop` on the inherited items. guidata shares its DataItem instances between a DataSet and its subclasses, so this mutated `plotpy.styles.ShapeParam` globally: the title, read-only and private fields disappeared from every shape parameters dialog in the application, not just from the settings dialog. The three items are now redeclared in the subclass instead. DataLabShapeParam is also exported from the `datalab.config` package and keeps `datalab.config` as its serialized module path, so annotation styles persisted by DataLab <= 1.2 are still restored instead of being silently reset. The `datalab.config.optionfields` compatibility module is removed: its only remaining class remapped that module path, which is no longer needed. DataSetOptionField now comes from sigimax.config like the other field types. Assisted-by: Claude Opus 5
DataLabOptions used to call back into config_persistence through four lazy, same-function imports (option_changed, is_option_initialized, snapshot_option_context_state, restore_option_context_state), each guarded by a "avoid a cycle" comment and a pylint disable. The options model effectively depended on the INI persistence layer, just deferred to call time. config_persistence now defines a narrow OptionStore protocol (load_all, save_all, save(name), has(name), remove(name)), implemented by IniOptionStore. DataLabOptions only depends on this protocol (imported under TYPE_CHECKING) through attach_store()/detach_store(), replacing the set_ini_persist_enabled(bool) flag. The lifecycle functions in config.py (initialize, reset, reset_to_defaults) attach/detach an IniOptionStore instead of toggling the flag. As a side effect, option_changed now persists only the changed field (store.save(name)) instead of rewriting every persisted option on each set(); each field already serializes its own state independently, so this is behavior-preserving. No change to the on-disk INI format or its public API surface. Assisted-by: Claude Opus 5
…ersistence Replace the bool|None _INITIALIZED_WITH_USER_CONFIG flag with a single explicit _MODE: None | "user" | "defaults" state. initialize(), reset() and reset_to_defaults() derive their behavior from this one variable instead of combining it with the load_user_config parameter at each call site. datalab.gui.settings no longer imports get_ini_location from config_persistence to match Settings-dialog DataSet fields against option names. DataLabOptions now exposes get_field_ui_key(name), the same short-key derivation (storage_key, then ai_/macro_ prefix stripping) previously duplicated inside config_persistence.get_ini_location, which becomes a thin wrapper around it. The Settings UI no longer depends on the INI persistence layer for this. Assisted-by: Claude Opus 5
…ce/core The package was two large modules (config.py, config_options.py, config_persistence.py) mixing several concerns: application identity and resource paths, the option model, INI persistence, and the initialization lifecycle. Split along those lines into four single-responsibility modules with an explicit, cycle-free dependency order: - appinfo.py: app name/paths, translation/resource setup, and the plugin-path helpers that don't touch the shared Conf singleton. No internal dependency. - options.py (renamed from config_options.py): the DataLabOptions container. Depends on appinfo (APP_NAME is now sourced from there instead of being duplicated). - persistence.py (renamed from config_persistence.py): the INI<->options mapping, plus DataLabUserConfig/_LegacyConfigReader/ atomic_save_configuration/migrate_legacy_configuration, moved here from config.py now that appinfo.py exists as a leaf module (this move would have reintroduced the cycle removed by the OptionStore change otherwise). - core.py (renamed from config.py): the initialization lifecycle, the shared Conf singleton, and the PlotPy default instances tied to it. The public facade (datalab.config) is unchanged for application code: every name it exposed before is still exposed, from its new location. A handful of white-box unit tests that reached into config.py directly (not through the facade, to exercise internals with no public API) now reach into core.py instead. No change to the on-disk INI format or the public API surface. Assisted-by: Claude Opus 5
Export save_runtime_option from datalab.config: the runtime-option persistence mechanism (fields declared runtime=True, like rpc_server_port) is meant to be reusable by any future owner of such a field, not just a one-off. gui/main.py now imports it as a normal top-level facade import instead of a local import - it was never motivated by an import cycle (unlike the two neighboring local imports in the same file, which do carry an explicit cyclic-import disable). Remove from the facade the 1.2->1.3 migration/versioning internals that no DataLab module ever imports outside of their own dedicated unit tests: CONF_VERSION, DataLabUserConfig, migrate_legacy_configuration, migrate_legacy_plugin_paths, get_config_app_name, get_legacy_config_filename, get_typed_config_filename. The migration itself stays reachable through core.initialize(), which remains public and is all application code needs. The two affected tests now import these internals directly from persistence.py/appinfo.py, consistent with how other white-box tests already reach into config submodules for symbols with no reason to be public. DataLabShapeParam stays exported: guidata resolves it by module/class name at JSON deserialization time, so it needs to remain reachable as datalab.config.DataLabShapeParam regardless of direct import usage. No change to the on-disk INI format or the migration behavior itself. Assisted-by: Claude Opus 5
…sable Audit of every `import-outside-toplevel` / `cyclic-import` disable in DataLab application code - datalab/env.py, sigimax/env.py: document why the config import is deferred (heavy PlotPy/Qt stack, not a circular import) Assisted-by: Claude Opus 5
Regenerate scripts/datalab_menus.txt, which had drifted from the current menu tree (signal marker table creation, special value replacement, peak position extraction, peak detection, calibration entry reordering).
Add "sigimax" to the packages reinstalled by scripts/reinstall_dev.py so local SigimaX changes are picked up by DataLab instead of a frozen copy shipped in site-packages.
…code Remove code that was copied verbatim from SGMXMainWindow during the SigimaX migration and is now inherited unchanged: * take_screenshot: strictly identical to the base implementation * __check_h5file: replaced by the inherited _check_h5file static method * save_to_h5_file: reduced to a thin override that only preserves the @remote_controlled decorator * TEST_SEGFAULT_ERROR: re-exported from sigimax.config instead of being redefined from the environment * logview_error / logview_app_test: duplicates of the SigimaX test suite No functional change: main window layout, docks, toolbars, status bar and menu tree are byte-identical before and after. Assisted-by: Claude Opus 5
Replace the manual dock bookkeeping by the base _add_dockwidget() API: self.docks is no longer reassigned wholesale in __add_signal_image_panels, and the three chained tabifyDockWidget() calls are expressed with the tabify_with argument instead of relying on previously stored docks. Signal and image views keep being registered under their panel, as the tab widget looks docks up by panel, using the new key argument. Docks now use stable object names (signal_view, image_view, macro_panel, ai_assistant) instead of their translated titles, so that the layout is no longer lost when the application language changes. Default layout, dock areas and tabified groups are unchanged. A dock layout customised with a previous version is reset once. Assisted-by: Claude Opus 5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #182