Set ExpRun's filePathRoot to be the container's file root#724
Merged
Conversation
…fb_copy_vs_mv_pano_files
…bKey/targetedms into 23.3_fb_copy_vs_mv_pano_files
labkey-jeckels
approved these changes
May 22, 2023
vagisha
added a commit
to LabKey/MacCossLabModules
that referenced
this pull request
May 25, 2023
…fileUrls. This should not be required anymore due to LabKey/targetedms#724. Set filePathRoot on the copied expRun to be the target container's file root. Log error if the data file path is unexpected, i.e. it does not contain "Run<runid>"
labkey-jeckels
added a commit
to LabKey/MacCossLabModules
that referenced
this pull request
Jul 1, 2023
* - PanoramaPublicFileImporter logs to the job log, and throws an exception if any of the datafileurls could not be fixed. - PanoramaPublicSymlinkManager.moveAndSymLinkDirectory takes a Logger parameter so the log output can go to the job log - Added a PanoramaPublicMetadataImporter. I moved some of the code out of CopyExperimentFinalTask into this class. This creates a row in the panoramapublic.experimentannotations table. It runs before PanoramaPublicFileImporter so that if there is an error, e.g. datafileurls cannot be fixed, the container can be deleted to move files back to the source container. - Updated test - import a document into a subfolder of the container file root. * - Fire symlink update events only when file / container being moved / renamed / deleted is in the Panorama Public project. We don't expect folders in other projects to contain symlink targets. - When handling folder rename (ContainerListener.propertyChange), pass the full paths of the old and renamed containers instead of just the folder names. Otherwise, it can lead to updating all symlinks that have the old folder name in the path. - When deleting a folder, use ExperimentAnnotationsManager.getExperimentIncludesContainer(c) to lookup the experiment. This method will return the experiment that contains runs from the folder even if it is a subfolder of the folder where the experiment was created. - When an experiment folder in Panorama Public is deleted, move the files back to next highest experiment version if one exists. Otherwise, move the files back to the source folder. * Rework datafile alignment * Scope datafile url to correct container * Removed PanoramaPublicFileWriter. * Limit the number of containers to look at when updating symlinks. This should only include the source container in the submitter's project as well as any containers with older versions of the data on Panorama Public. * Remove code to lookup runs in the source container when aligning datafileUrls. This should not be required anymore due to LabKey/targetedms#724. Set filePathRoot on the copied expRun to be the target container's file root. Log error if the data file path is unexpected, i.e. it does not contain "Run<runid>" Co-authored-by: vagisha <vagisha@gmail.com> Co-authored-by: Josh Eckels <jeckels@labkey.com> Co-authored-by: labkey-sweta <swetaj@labkey.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rationale
Skyline documents are typically uploaded and imported from the container's file root when the import is driven by Skyline. However, it is possible to manually upload and import documents from a subfolder of the container's file root. When these runs are exported as part of folder export, the exported directory path does not include the subfolder. For example, for a run imported from
@files/SkylineFilesthe contents of the export folder look like this:This does not include the
SkylineFilessubfolder, and makes it difficult to adjust the dataFileUrls on ExpData in the folder import part of the Panorama Public pipeline. Setting ExpRun's filePathRoot to always be the container's file root will ensure that the exported run files are at the same relative path as they are in the source container:Related Pull Requests