Skip to content

Remove special unique constraints ReducedDatums - #1624

Open
jchate6 wants to merge 1 commit into
devfrom
1622-dataservice-target-creation-issue
Open

Remove special unique constraints ReducedDatums#1624
jchate6 wants to merge 1 commit into
devfrom
1622-dataservice-target-creation-issue

Conversation

@jchate6

@jchate6 jchate6 commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

The unique constraints we used to use were necessary because validating on the "value" field wasn't possible. This is no longer the case.

@jchate6 jchate6 linked an issue Jul 17, 2026 that may be closed by this pull request
1 task
@jchate6
jchate6 requested a review from Fingel July 17, 2026 23:13
@jchate6 jchate6 changed the title no longer need special unique constraints for duplicate data because … Remove special unique constraints ReducedDatums Jul 17, 2026
@jchate6

jchate6 commented Jul 17, 2026

Copy link
Copy Markdown
Contributor Author

@Fingel Without these constraints, I think we need to add some check in the API loop to make sure we aren't ingesting identical data.

I'm not sure where that should be... serializers.ReducedDatumSerializer.create? models.try_parse_reduced_datum?

Maybe we just need to include more fields in the constraints, rather than remove them, but this feels like it can be solved by a get_or_create somewhere rather than a constraint, since we have actual known fields we can test against now.

Suggestions?

@jchate6 jchate6 moved this to Needs Review in TOM Toolkit Jul 17, 2026
@Fingel

Fingel commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

I'm always going to prefer database constraints to (potentially) faulty business logic. A constraint doesn't prevent a get_or_create() from working either, they aren't mutually exclusive.

I think what actually happened here is the use case of duplicate reductions wasn't taken into account. I think Rachel was correct when she said that a version number should be tracked and added to the constraint, or alternatively one could use an upsert pattern to either update the incoming datum, or create a new one if it doesn't exist.

I don't think removing DB constraints entirely is a good idea.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Needs Review

Development

Successfully merging this pull request may close these issues.

Dataservice target creation issue

2 participants