Changing a feature flag `defaultVariant` doesn't dynamically update services while demo is running

[flands]

Bug Report

Which version of the demo you are using? 1.10.0

Symptom

Changing a feature flag defaultVariant while the demo application is running doesn't dynamically update services

What is the expected behavior?

Changing a feature flag defaultVariant to on while demo is running dynamically updates services

What is the actual behavior?

Changing a feature flag defaultVariant to on while demo is running is a noop.

Reproduce

Per https://opentelemetry.io/docs/demo/feature-flags/, To enable a flag, change the defaultVariant value in the config file for a given flag to “on”, while the demo application is running edit src/flagd/demo.flagd.json and change a defaultVariant to on

Additional Context

Perhaps this is expected behavior. If it is, then the documentation should be updated to state that if the demo application is running and a feature flag defaultVariant is changed then you need to restart the flagd service (make restart service=flagd) or restart the demo application for it to take effect.

I can make the documentation change if this is the recommended approach.

[julianocosta89]

Hey @flands I've actually tested that on docker and it works fine.
Any Feature Flag in specific you have tested?

I know that for K8s, that may be different as we are not using the flagd operator, but for Docker it worked fine.

[flands]

@julianocosta89 Interesting! I'm on Darwin (ARM-based) running the latest Docker desktop. I tried the first two feature flags multiple times (for example productCatalogFailure) and the only way I can get it to work is to restart the flagd service.

[beeme1mr]

Hey @flands, do you see an event in the flagd logs when you make a change in the config? Flagd uses fsnotify to detect changes. Perhaps your setup is triggering an event that we're currently not reacting to.

[beeme1mr]

We're considering more robust file watching approaches.

open-feature/flagd#1344

[flands]

It starts with the logs shown below and after changing the JSON file no new logs are recorded

2024-07-03T21:39:32.527Z	info	cmd/start.go:107	flagd version: v0.10.2 (d58fe3c3ac67843571d8fdc7d04b75996444befd), built at: 2024-05-10	{"component": "start"}
2024-07-03T21:39:32.528Z	info	flag-sync/sync_service.go:54	starting flag sync service on port 8015	{"component": "FlagSyncService"}
2024-07-03T21:39:32.530Z	info	file/filepath_sync.go:45	Starting filepath sync notifier	{"component": "sync", "sync": "filepath"}
2024-07-03T21:39:32.531Z	info	ofrep/ofrep_service.go:56	ofrep service listening at 8016	{"component": "OFREPService"}
2024-07-03T21:39:32.531Z	info	flag-evaluation/connect_service.go:243	metrics and probes listening at 8014	{"component": "service"}
2024-07-03T21:39:32.531Z	info	flag-evaluation/connect_service.go:223	Flag IResolver listening at [::]:8013	{"component": "service"}
2024-07-03T21:39:32.532Z	info	file/filepath_sync.go:74	watching filepath: ./etc/flagd/demo.flagd.json	{"component": "sync", "sync": "filepath"}

[Starefossen]

I had the same experience (docker/macos/arm64/colima). To reliably switch the feature toggles both flagd and the service that uses the flag had to be restarted.

vim src/flagd/demo.flagd.json
docker-compose restart flagd
docker-compose restart productcatalogservice

[Starefossen]

flagd v0.11.2 appears to be working better and reloading the file on disk properly whenever it changes.

[beeme1mr]

Yes, version 0.11.2 added more sophisticated file-watching capabilities. You can see the PR here if you're interested in the details.

@Starefossen, thanks for bumping the version in the demo.