Haskell Relational Records (HRR) - initial sprint

[mgmeier]

This sprint's focus is the evaluation of HRR as a DB abstraction. The result should implement the domain API as per specs for Tenant, User and Role data types, encompassing the following:

• CreateRole, DeleteRole, AssignRole, RemoveRole, GetRole
• CreateUser, UpdateUser, ActivateUser, DeactivateUser, GetUser
• CreateTenant, UpdateTenant, ActivateTenant, DeactivateTenant, GetTenant

Furthermore, the result should serve as an example how to implement and/or deal with the following topics in HRR:

• DB model setup and migrations in HRR
• Mapping to and from Haskell data types
• Mapping of special types to and from Haskell: PostgreSQL enums and arrays, JSONB
• Expressive power of JOINs in HRR (e.g. fetch a list of Users along with their Roles)
• Treatment of housekeeping data in HRR (timestamp columns, transaction log)

Time estimate: 20 hours

[mgmeier]

@saurabhnanda The domain API is now up and running as described in the PR, minus the topics still open (enums, arrays, jsonb).

This would be a good time for you to review the code, comment some design choices, look at the overall resulting workflow etc. etc. so that I can adjust the code to your feedback, which will be highly appreciated.

[saurabhnanda]

@mgmeier ran into the very first (expected) issue. The HRR TemplateHaskell needs to connect to PG in the compilation step itself. How do I configure the PG credentials?

[saurabhnanda]

@mgmeier never mind. Figured it out. Btw, hitting a new compile error. Let's discuss it on Gitter.

[saurabhnanda]

• Does the PG text or varchar always map to Haskell's String? What's the easiest way to make it map to Data.Text?
• What does Relation x y really mean? You can use relationalQuery to convert Relation x y -> Query x y where Query x y = Query {untypeQuery :: String}. The type parameters x y in the entire chain seem to be phantom types. So, is Relation x y basically some form of SQL represented in a Haskell-friendly AST (or DSL)?
• Consequently, what's the difference between SimpleQuery x y and Relation x y?
• Has audit logging (transaction logging) been completed?

[saurabhnanda]

Is it a good idea to conflate a no-op with an update to NULL? eg. say this column were initially non-nullable, and thus, had the type Maybe Int32. And we had some call-sites where this field was being set as Nothing, assuming that it would map to an sql NULL. Then we made this column nullable, thus changing its type to Maybe (Maybe Int32). The call sites would still type-check, but they would result in completely unexpected behaviour.

What are your thoughts on the following: Nullable (Maybe Int32)

[mgmeier]

Maybe a is the HRR derivation of a nullable type a on the db.

The conflation problem you adressed can actually not occur: The outer Maybe signals the presence vs. absence of a value for the update. So, no value assignment whatsoever takes place in the update query for a value of Nothing. The inner Maybe can then be set to Just Nothing to update with sql NULL. Thus, the call-sites that never update some field (Nothing) stay correct, the ones updating a column that has been made nullable fail to type-check, as it should be.

You're right however that the mechanism is not quite clear at first glance. I've changed that by replacing the outer Maybe with a different, more expressive type.

[saurabhnanda]

Just wondering if it is possible to avoid this boilerplate if we're fine with having the same data-structure/type for, both, reading from the DB and writing to the DB?

[mgmeier]

HRR derived types have all strict fields; an "identity projection" of such a type would require us to assign a (non-existent) primary key value where present to construct the record -- which will then clash on the DB when we run the insert. For tables without PK, this is absolutely possible; I've done so for the join-through table users_roles, it looks like this:

assignRole :: Insert UsersRoles
assignRole = typedInsert (tableOf usersRoles) (defineDirectPi [0, 1])

[saurabhnanda]

Can be replaced with Data.Default?

[mgmeier]

yes. done.

[saurabhnanda]

how do we handle the case where certain columns need to be protected from "blind" updates? eg. password column? While it can be inserted initially, it should not be updatable later via a variadic update? One should be forced to call a special function for changing the password, which would also trigger a side-effect of notifying the user by email?

[mgmeier]

the solution would be to remove that field from the variadic update type completely.
for those updates which require special attention / a special code-path (you mentioned side effects), a specialized, explicit Update relation and corresponding domain API function would be in order.

[saurabhnanda]

Need some comments on what exactly is happening here? How is the current timestamp actually getting assigned?

[saurabhnanda]

Also, how is createdAt being handled?

[mgmeier]

The helper type for this update is type TimestampedUpdate = Update (DBTime, PKey) . This means, the generated query will have two placeholders, one for the timestamp, one for the primary key to update. The other updated values are rendered in the query as literals. So when rendering the query DSL, we get something like:
UPDATE users SET first_name='Foo', last_name='Bar', updatedAt=? WHERE id=?, where substitution of these placeholders happens one step later.

In HRR, these type parameters for e.g. an Update a have to be realized via the placeholder syntax. It's also possible to apply some datatype argument first, and use the literal inside the query, e.g. DBTime -> PKey -> Update (). One can decide on the context which is preferable.

Either way, getting a current timestamp from the system clock is done in the DBInterface, which in turn then runs the update query.

[mgmeier]

createdAt timestamps are always left to the DB, where they have default value current_timestamp in the schema.

[saurabhnanda]

Possible to discuss this on gitter chat?

[saurabhnanda]

Can you please give a usage example of this function? Is this just a DSL/AST representation of an inner-join between roles and users via the user_roles join-through table, with an unapplied user argument? What is the type of the unapplied user argument? Is it the PKey of the User table or the actual User record? How can you tell which arguments are unapplied from the function signature?

When this query is executed, what does it evaluate to? A list of Roles?

How would one write the following -- get all users along with each user's role(s), where the user has been created after a given timestamp AND the user's email ID belongs to a given domain (say, gmail.com or vacationlabs.com)?

Can I compose this Relation with another wheres clause specified outside this function? Can I compose this with another JOIN, say with tenants table?

[mgmeier]

I've changed HRR default mapping of PostgreSQL type text to Haskell's Data.Text. Such a custom mapping can easily be specified with HRR's db driver and is done in the module DefineTable.hs.

[mgmeier]

JSONB and ENUM: investigation results

This is the relation used by the HRR driver to get postgres type info; those will be the types considered for generating a corresponding attribute in the derived Haskell type (in pseudo-code, taken from module Database.Relational.Schema.PostgreSQL):

(select * from pg_type)
    wheres $ att ! Attr.atttypid'    .=. typ ! Type.oid'
    wheres $ typ ! Type.typtype'     .=. value 'b'  -- 'b': base type only

    wheres $ typ ! Type.typcategory' `in'` values [ 'B' -- Boolean types
                                                  , 'D' -- Date/time types
                                                  , 'I' -- Network Address types
                                                  , 'N' -- Numeric types
                                                  , 'S' -- String types
                                                  , 'T' -- typespan types
                                                  ]

We can see, JSONB is not amongst them as it is of category 'U' (user-defined).
Also, enums are not considered for mapping, they're of category 'E'; also, they're not a base-type, but an 'e' - enum type.

Conclusion: The HRR library would have to be patched accordingly for HRR to even consider
generating a datatype derivation for those categories in Haskell.

[saurabhnanda]

Do you think this is easy to patch in HRR (JSONB and enums)?

On 18 Nov 2016 9:50 pm, "M. G. Meier" notifications@github.com wrote:

JSONB and ENUM: investigation results

This is the relation used by the HRR driver to get postgres type info;
those will be the types considered for a generating a corresponding
attribute in the derived Haskell type (in pseudo-code, taken from module
Database.Relational.Schema.PostgreSQL):

(select * from pg_type)
wheres $ att ! Attr.atttypid' .=. typ ! Type.oid'
wheres $ typ ! Type.typtype' .=. value 'b' -- 'b': base type only

wheres $ typ ! Type.typcategory' `in'` values [ 'B' -- Boolean types
                                              , 'D' -- Date/time types
                                              , 'I' -- Network Address types
                                              , 'N' -- Numeric types
                                              , 'S' -- String types
                                              , 'T' -- typespan types
                                              ]

We can see, JSONB is not amongst them as it is of category 'U'
(user-defined).
Also, enums are not considered for mapping, they're of category 'E'.

Conlusion: The HRR library has to be patched accordingly for HRR to even
consider
generating a datatype derivation for those categories in Haskell.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#50 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABu0bDV2S0BeVoeuTFW4sCZIpviWHE0ks5q_dBjgaJpZM4KqFnU
.

[mgmeier]

Audit logging works fine for now; the logic for creating the JSON diff is in the application layer (Haskell), not on the DB.
NB. for the time being, log entries are texts. When I patch HRR for a derivation of jsonb, that'll change. Patching shouldn't be too complicated, as I see it.

[mgmeier]

Don't forget to create DB from scratch (schema.sql) and possibly completely eliminate .stack-work/, since I've replaced the source repo for HRR with the fork where my patch is applied.