Refuse to publish books with unknown ISO639-3 codes in `Language` metadata

[benoit74]

The CMS must refuse to publish books with unknown ISO639-3 codes in Language metadata.

mul special value must also be forbidden (even if part of the ISO639-3), since the specification says that we must list all languages used in this metadata.

This will obviously change over time as ISO639-3 norm gets updated. We should hence rely on a widely popular and regularly updated library / database for this check.

We might need to patch this library/database from time to time when it gets outdated as the norm evolves but the library/database is not yet updated.

This issue comes from openzim/overview#51 (comment) discussion

[benoit74]

Question: should the CMS regularly audit existing ZIMs to find the ones which became invalid?

[rgaudin]

Question: should the CMS regularly audit existing ZIMs to find the ones which became invalid?

Interesting question about the maintenance role of the CMS!

[kelson42]

Question: should the CMS regularly audit existing ZIMs to find the ones which became invalid?

We need an audit log, just to identify - for example - the ZIM files which are "old".

But this is actually only one of dozen of rules to check ZIM files against.

This check should happen once at publishing time. Therefore they should be extracted at that time from the ZIM file.

The set of publishing rules is going to move over time. Therefore, to be correct, we will need to reassess already published ZIM file.

But it would be IMHO costly (both in term of software/architecture and runtime) to run this against the ZIM files themselves. As a consequence, we have to copy all appropriate metadata in the CMS DB at publishing time.

Audit data can be then quickly be computed... on demand or schedule based is then pretty much a detail.