Skip to content

295 contentchanged for layers and pages#298

Merged
thehabes merged 23 commits into
developmentfrom
295-contentchanged-for-layers-and-pages
Aug 21, 2025
Merged

295 contentchanged for layers and pages#298
thehabes merged 23 commits into
developmentfrom
295-contentchanged-for-layers-and-pages

Conversation

@cubap
Copy link
Copy Markdown
Member

@cubap cubap commented Aug 7, 2025

No description provided.

cubap and others added 9 commits May 12, 2025 14:35
* update metadata (#171)

* update metadata

* modify route name and db.update

* update db.update dependent

* cleanup

---------

Co-authored-by: Bryan Haberberger <bryan.j.haberberger@slu.edu>

* quickfix

* cleanup

* modify db.update to receive one param {data, collection}

* nodiff

* restore action(data, collection) structure

* undiff

* Changed Collections Parameter for Save()

* Removing /:id put Call

* Removing the Limit from express.json()

* Removing the Limit from express.json()

* getting started with Vault (#190)

* getting started with Vault

* newer Vault

* tests

* removing redundant code

* proper tests passing

* sample Vault

* Bryan's refusal to .jsonld makes this not work as expected

* loading resources

This is incomplete by design. We need to ask Vault to add any resources we want resolved

* expanding the Manifest a bit

* touch up for merge

* touch up for merge

---------

Co-authored-by: Bryan Haberberger <bryan.j.haberberger@slu.edu>

* Current Project IIIF manifest Creation (#187)

* Current Project IIIF manifest Creation

* Refactored the exportManifest() and endpoint name

* Changed the exportManifest() function

* TPEN ID error handled

* endpoint to move the manifest.json to TPEN-Static-Dev

* Changed @id to id

* Getting all Ids

* Console Clean up

* Added env variables

* Adding Imports

* Adding Imports

* Deleting Ids

* Updating Logic Added

* User not a member cannot change the manifest

* Removing project2

* Adding comments to functions

* add hotkeys service (#184)

* add hotkeys service

* hotkey endpoints

* aggregate hotkeys during project retrieval

* specify hotkey fields to include

* cleanup

* Update Hotkeys.js

* Update ProjectFactory.mjs

* Return hotkeys as an Array of Strings

* aligning with Class changes

* remove create, since .save is not acting correctly

* cleanup and drop .post

* tests restored

no Jest here, just checking exists.

* tests and sinon upgrade

* no db tests directly

* Update exists_unit.test.mjs

* putting post back in...

* adding create back with safety

* adding upsert to accomodate bad errors

* Update Hotkeys.js

* uncatch to let errors through

* expect the  errors to come back

* switch to jest tests

---------

Co-authored-by: cubap <cubap@slu.edu>

* hotifx

* hotfix for symbols.

* delete enabled

* Create API.md

* collaborators and users

* add markdown reader

* package for markdown

* Update API.md

* Update API.md

* touch

* ah codes

* proxy for internal use (#201)

* Endpoint to save changes for the new layer (#199)

* Adding endpoint to save changes from the layer

* Adding New Layer

* No Empty Label and no Annotations

* Updated new Layer

* Adding Items to partOf

* Changing id convention

* Removing updating layer

* Annotation Change

* Adding Delete Endpoint

* not sure...

This type of thing?

* Label Change

* Added Layer Class

* Adding rerum ids to delete and add

* Changing tests

* Update exists.test.mjs

* Changing LayerLabel

* Changes AnnotationCollection Structure

* Update Pages API

* Updating partOf Ids in case of any change

* Adding Layer Metadata Label Change

* example results as a base for comments, delete these files before merge.

* Adding AddLayer Commenting Rest of the APIs

* Added DeleteLayer Back

* Deleting json files

* Updates to the comments

* Making Project to Layer Changes

* Renaming layerAnnotationCollection

* Removing UpdateOne() changes

* Removing UpdateOne() changes

* Removing UpdateOne() changes

* Error Handled for no ProjectID or LayerID

* Clear Code

* send() instead of json()

---------

Co-authored-by: cubap <cubap@slu.edu>
Co-authored-by: Bryan Haberberger <bryan.j.haberberger@slu.edu>

* Create sample.env

* Update sample.env

GitHub per @mepripri

* Update CODEOWNERS

* dev it and hotkeys

* 188 epic middleware to upgrade imported manifests (#209)

* bring in vault

improve Project building from Manifest import

* fixes #206

* add singleton vault as utility

fix #208

* cleanup logs

* 139 factor out type type dependencies (#211)

* determine data type by content

* Removing type dependencies

- Took controller and type out of controller
- Added a function to detect the data types and assign the correct collection

* matching tests to code move

* align with main

* Update driver.mjs (#217)

* 188 epic middleware to upgrade imported manifests (#218)

* bring in vault

improve Project building from Manifest import

* fixes #206

* add singleton vault as utility

fix #208

* cleanup logs

* rename redone

* This is Vault now

* test objects don't validate

* hulk smash 👊🏽

* _sub is no longer missing on invitees

* add temp sub to new user

* Update cd_dev.yaml

* Update cd_dev.yaml

* stop if things are missing

* adding verify on main

* Removing fileSystem from Github API (#214)

* Removing fileSystem from Github API

* Update ProjectFactory.mjs

---------

Co-authored-by: Bryan Haberberger <bryan.j.haberberger@slu.edu>

* 220 services for bug reporting and feedback (#221)

* file dump init

* services for feedback

* /feedback/feedback is not a good route

* Update cd_dev.yaml

* stop if things are missing

* adding verify on main

* test results

* added docs and ES6 conversion

* what the Golden AI hell is adding imports from my docs?

* cicd (#222)

* Update cd_dev.yaml

* stop if things are missing

* adding verify on main

* Update ci_dev.yaml (#219)

* Update ci_dev.yaml

* Update cd_prod.yaml

* Update ci_prod.yaml

* 220 services for bug reporting and feedback (#224)

* file dump init

* services for feedback

* /feedback/feedback is not a good route

* Update cd_dev.yaml

* stop if things are missing

* adding verify on main

* test results

* added docs and ES6 conversion

* what the Golden AI hell is adding imports from my docs?

* explicitly adding CORS

* API call to Update Profile (#223)

* API call to Update Profile

* existingEmail and existingName

* Changes to comments

* Update User.mjs

* Save AnnotationCollection, Pages and Annotations to RERUM (#215)

* saveCollection to RERUM

* Adding Tinypen to Create RERUM Object

* Update exists.test.mjs

* Update exists.test.mjs

* Update Page.mjs

* Update cd_dev.yaml

* stop if things are missing

* adding verify on main

* starting some adjustments

* better the tests

* headed home

* multiple ways to extract the data

* retest with suggestions

* layer/page halos

* percolating deletes

* setting up routes

---------

Co-authored-by: Patrick Cuba <cubap@slu.edu>

* Update index.mjs

* Update index.mjs

* Update index.mjs

* Update index.mjs

* Update index.mjs

* Import TPEN28 (#226)

* Update index.mjs

* Update index.mjs

* Update index.mjs

* Update index.mjs

* Update index.mjs

* Test restoration (#229)

* Update cd_dev.yaml

* stop if things are missing

* adding verify on main

* cleanup missing properties, changed method names

* just this route

* id shouldn't be optional here.

out of date test files dropped

* This should never break. What's up?

* bad merge

* Update exists_unit.test.mjs

* test is ugly

The page router needs a projectId as well to actually work

* Update end_to_end_unit.test.mjs

These two cannot work without a corresponding project, so it will need to be rewritten

* nested in router now

* Update exists.test.mjs

* un-mjs

* Refactor all .mjs files to .js and update imports. Closes #194 (#228)

* Update cd_dev.yaml

* stop if things are missing

* adding verify on main

* Update cd_dev.yaml

* stop if things are missing

* adding verify on main

* Update ci_dev.yaml (#219)

* Update ci_dev.yaml

* Update cd_prod.yaml

* Update ci_prod.yaml

* Update index.mjs

* Update index.mjs

* Update index.mjs

* Update index.mjs

* Refactor all .mjs files to .js and update imports. Closes #194

* un-mjs

* npm update

* Hey I heard you like tests, so I put tests in your tests

* mjs > js

* no mjs, bad mjs

* fine

* habesroxx

* how bout now

* hide, Jest is coming

* runner love

* Update package-lock.json

* jest no like to run

* null != undefined

* default not defaulting

* fixes "id is not defined"

shoulda wrote test for this

* out of scope, out of effs

---------

Co-authored-by: Priyal Patel <mepripri0712@gmail.com>

* Update package-lock.json

* Using UID to get User Projects

* Update index.mjs

* no label is fine for Pages

* Origin Fetch

* SetHeader Origin

* Update index.js

* Getting User Projects (#237)

* Update index.mjs

* Update index.mjs

* Update index.mjs

* Update index.mjs

* Update index.mjs

* Using UID to get User Projects

* Update index.mjs

* Origin Fetch

* SetHeader Origin

* Update index.js

* Update index.js

* Update index.js

* Localhost URLS Validating (#242)

* Update index.js

* Update index.js

* Update validateURL.js

---------

Co-authored-by: Patrick Cuba <cubap@slu.edu>

* 231 create overwrite layer (#239)

* passing through the projectID

* some tests

* support page

* path in Page class generator

* no label is fine for Pages

* updating layers

* AI generated tests

* Update API.md

* oh auth.

* how'd we miss this?

* old tests

* adjust for tests

* default exports for tests

* wrangling AI tests

Jest mock is a nightmare.

* just skip to move on

* Update Layer.js

* Update index.js

* merged mess unwrap

* dummy

* Update Project.js

* handle labels throughout

* unerring

* update layer organized

* return changes

* avoid hard crash

* prevent crash on a page 404

* full id

After I PUT a new label (this was successful) the "id" on the layer the db obj does not the the prefix and is just the hash.

* you get it

* vaildate all singular changes

* greedier try-catch

* Squashed commit of the following:

commit 44b75cf
Author: Priyal Patel <52342511+mepripri@users.noreply.github.com>
Date:   Wed May 7 10:13:21 2025 -0500

    Localhost URLS Validating (#242)

    * Update index.js

    * Update index.js

    * Update validateURL.js

    ---------

    Co-authored-by: Patrick Cuba <cubap@slu.edu>

commit 032ef45
Merge: f977234 52edaab
Author: cubap <cubap@slu.edu>
Date:   Mon May 5 10:40:24 2025 -0500

    Merge branch 'development' into import-tpen28

commit 52edaab
Author: Priyal Patel <52342511+mepripri@users.noreply.github.com>
Date:   Fri May 2 09:36:27 2025 -0500

    Getting User Projects (#237)

    * Update index.mjs

    * Update index.mjs

    * Update index.mjs

    * Update index.mjs

    * Update index.mjs

    * Using UID to get User Projects

    * Update index.mjs

    * Origin Fetch

    * SetHeader Origin

    * Update index.js

    * Update index.js

    * Update index.js

commit f977234
Author: Priyal Patel <mepripri0712@gmail.com>
Date:   Thu May 1 13:14:16 2025 -0500

    Update index.js

commit dde7c30
Author: Priyal Patel <mepripri0712@gmail.com>
Date:   Thu May 1 13:05:02 2025 -0500

    SetHeader Origin

commit d2ec198
Author: Priyal Patel <mepripri0712@gmail.com>
Date:   Thu May 1 12:47:32 2025 -0500

    Origin Fetch

commit 83fac3a
Merge: 6e79a8d 2e5bbbd
Author: Priyal Patel <mepripri0712@gmail.com>
Date:   Thu May 1 09:31:12 2025 -0500

    Merge branch 'development' into import-tpen28

commit 2e5bbbd
Merge: 443d0a2 9f102fb
Author: Patrick Cuba <cubap@slu.edu>
Date:   Wed Apr 30 22:38:43 2025 -0500

    Merge branch 'development' of https://github.com/CenterForDigitalHumanities/TPEN-services into development

commit 443d0a2
Author: cubap <cubap@slu.edu>
Date:   Wed Apr 30 15:23:02 2025 -0500

    no label is fine for Pages

commit 6e79a8d
Merge: ac0182f 9f102fb
Author: Priyal Patel <mepripri0712@gmail.com>
Date:   Wed Apr 30 12:06:38 2025 -0500

    Merge branch 'development' into import-tpen28

commit ac0182f
Author: Priyal Patel <mepripri0712@gmail.com>
Date:   Wed Apr 30 11:51:40 2025 -0500

    Update index.mjs

commit 673a5c5
Author: Priyal Patel <mepripri0712@gmail.com>
Date:   Tue Apr 29 17:22:08 2025 -0500

    Using UID to get User Projects

commit afe664e
Author: Priyal Patel <mepripri0712@gmail.com>
Date:   Mon Apr 28 10:16:27 2025 -0500

    Update index.mjs

commit 6bf9c9a
Author: Priyal Patel <mepripri0712@gmail.com>
Date:   Fri Apr 25 14:56:11 2025 -0500

    Update index.mjs

commit 0906084
Author: Priyal Patel <mepripri0712@gmail.com>
Date:   Fri Apr 25 14:50:54 2025 -0500

    Update index.mjs

commit 5dd077e
Author: Priyal Patel <mepripri0712@gmail.com>
Date:   Fri Apr 25 14:33:53 2025 -0500

    Update index.mjs

commit e9971bc
Author: Priyal Patel <mepripri0712@gmail.com>
Date:   Fri Apr 25 12:18:30 2025 -0500

    Update index.mjs

* 422 if no pages are there.

* Update index.js

* Update index.js

* API entries

* typo

---------

Co-authored-by: Bryan Haberberger <bryan.j.haberberger@slu.edu>

* remove unused file

---------

Co-authored-by: Onoja Victor <111019083+git-voo@users.noreply.github.com>
Co-authored-by: Bryan Haberberger <bryan.j.haberberger@slu.edu>
Co-authored-by: Onoja <onoja.jsdev@gmail.com>
Co-authored-by: mepripri <mepripri0712@gmail.com>
Co-authored-by: Priyal Patel <52342511+mepripri@users.noreply.github.com>
Refactored updatePageAndProject to simplify logic and remove the contentChanged parameter. Improved validation for update requests in page/index.js and streamlined layer lookup. Added default type to Line update output. Minor formatting and consistency improvements in shared.js.
@cubap
Copy link
Copy Markdown
Member Author

cubap commented Aug 7, 2025

This pull request introduces several improvements and bug fixes focused on the handling and updating of Page and Layer objects, as well as enhancements to reference management and request validation. The most important changes are grouped below by theme.

Reference management and data consistency

  • Added a new upgradeReferences utility function to standardize and upgrade object references (such as partOf) to use the RERUMIDPREFIX format, and integrated its use in the Page class constructor. [1] [2] [3]
  • Ensured the type property defaults to "Annotation" in the Line class when not explicitly set, improving data consistency.

Page and Layer update logic

  • Refactored the updatePageAndProject function to remove the contentChanged parameter, streamline the update process, and ensure the correct layer and page are updated in the project. Now, the function always updates the appropriate layer and project if the page has a RERUM ID.
  • Improved the Page class's update method to better detect when updates are necessary by checking for the presence of items or objects with a body property.

API and validation improvements

  • Enhanced the page update API endpoint to more robustly validate incoming update data, ensuring it is a non-empty object, and improved error handling for missing or invalid layers. [1] [2]
  • Updated the URL validation utility to properly capture and ignore caught errors, preventing unhandled exceptions.

Minor bug fixes and cleanup

  • Removed unnecessary checks and streamlined logic in the page update endpoint, including redundant checks for pageContentChanged and improved property assignment. [1] [2] [3]
  • Fixed a bug in getLayerContainingPage to safely handle projects that may not have a data property.

@cubap
Copy link
Copy Markdown
Member Author

cubap commented Aug 11, 2025

flowchart TD
    A([Start]) --> B{Project exists?}
    B -- No --> E1[Throw Error: Must know project to update Layer]
    B -- Yes --> C{Layer is provided?}
    C -- No --> E2[Throw Error: Layer must be provided to update]
    C -- Yes --> D{UserId exists?}
    D -- No --> E3[Throw Error: Must know user id to update layer]
    D -- Yes --> F{Layer creator exists?}
    F -- No --> G[Set layer creator using fetchUserAgent with userId]
    F -- Yes --> H[Find layer index in project layers]
    G --> H
    H --> I[Update layer and store as updatedLayer]
    I --> J{Layer id in project equals updatedLayer id?}
    J -- Yes --> M[Replace project layer at index with updatedLayer]
    J -- No --> K[Initialize pageOverwrites array]
    K --> L[Loop through each page in updatedLayer]
    L --> L1[Set page partOf id to updatedLayer id]
    L1 --> L2{Page id starts with RERUMIDPREFIX?}
    L2 -- No --> L3[Continue to next page]
    L2 -- Yes --> L4[Find old page in RERUM using tiny]
    L4 --> L5{Old page found?}
    L5 -- No --> E4[Throw Error: Page not found in RERUM]
    L5 -- Yes --> L6[Update old page partOf to reference updatedLayer]
    L6 --> L7[Push overwrite promise to pageOverwrites]
    L3 --> N[Replace project layer at index with updatedLayer]
    L7 --> N
    M --> O[Await all pageOverwrites and catch errors]
    N --> O
    O --> P[Update project]
    P --> Q([End])
Loading

cubap added 4 commits August 11, 2025 12:24
Replaces separate database controllers with a single dbDriver instance for 'tiny' operations. Updates all references to use the new databaseTiny object, simplifying and unifying database access throughout shared.js.
@cubap cubap marked this pull request as ready for review August 11, 2025 21:00
Copy link
Copy Markdown
Member

@thehabes thehabes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately this makes updatePageAndProject() non-functional as experienced through the /annotator interface and the http://localhost:4000/components/manage-layers/index.html component POC.

Originally it threw a 500
Cannot update Page. Its Layer was not found.

This was a failure due to a naïve falsey checker, which I put in a commit for.

After that, I received a different 500
User ID is required to fetch user agent

Gotta stop chasing the rabbit. This needs attention because it has breaking changes. I cannot use the /annotator interface or reorder pages with http://localhost:4000/components/manage-layers/index.html anymore.

thehabes and others added 4 commits August 15, 2025 10:28
Adds logic to assign the line's creator property to the user's agent if it is undefined when processing items. This ensures that the creator information is consistently set for new or updated lines.
Copy link
Copy Markdown
Member

@thehabes thehabes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went through this using TPEN Interfaces to test the scenarios where we do or do not introduce content/creators into the data flow. Data remained temp or was upgraded as expected. The creator property accompanies RERUM data.

Temp ids resolve as expected, and devstore.rerum.io ids resolve as expected.

@thehabes thehabes merged commit 73988d2 into development Aug 21, 2025
3 checks passed
@thehabes thehabes deleted the 295-contentchanged-for-layers-and-pages branch September 12, 2025 19:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants