Reimplement MGLStyleValue atop NSExpression

[1ec5]

MGLStyleValue should be reimplemented as a subclass of or wrapper around NSExpression, and MGLStyleLayer layout and paint properties should accept NSExpressions instead of MGLStyleValues.

Especially once we introduce support for arbitrary arithmetic and conditional expressions in style values (mapbox/mapbox-gl-js#4077 and possibly mapbox/mapbox-gl-js#4079), maintaining a custom vocabulary and type system around these values will become a major burden. Though it’s nice that the current MGLStyleValue system is strongly typed, taking better advantage of type inference in Swift, our API surface area and the learning curve will increase quite a bit as we essentially try to reinvent a statically-typed NSExpression.

NSExpression performs runtime type checking for initialization and static type checking for inspection. For maximum developer comfort, it features a format string mini-language and a series of strongly-typed getters that are applicable to certain kinds of expressions. MGLVectorStyleLayer takes advantage of all these features by representing filters as NSPredicates, which consist of NSExpressions. As with NSPredicate, developers are already familiar with NSExpression and its format string syntax, so there won’t be much of a learning curve at all.

The one thing NSExpression doesn’t provide is the concept of a style function. It does provide a rather awkward extensibility mechanism; we could extend NSExpression to understand functions and provide a more ergonomic, strongly typed API for working with functions. For constant values and expressions, the developer would use NSExpression directly. #5970 (comment) has a sketch of the proposed API.

My understanding is that mapbox/mapbox-gl-js#4077 would require a style specification version bump, which would precipitate an SDK major version bump, so we probably don’t need to worry so much about backwards compatibility for this refactoring.

/cc @boundsj @incanus @lucaswoj @jfirebaugh

[lucaswoj]

The one thing NSExpression doesn’t provide is the concept of a style function.

Are you referring to categorical/interval/exponential style functions or the general idea of a function?

[1ec5]

In this ticket, I’m referring to category/interval/exponential style functions, of the sort already supported by the style JSON format. We would need to extend the NSExpression system to support that concept. All the other ideas that have been floated, like conditionals and arithmetic, are already supported out of the box. NSExpression could also be extended with arbitrary selectors (aka programmatic callbacks), but that’s outside the scope of this feature proposal.

[1ec5]

mapbox/mapbox-gl-js#4715 (review) outlines how an expression syntax proposed for the style JSON file format would translate to NSExpression.

[1ec5]

The expression feature is coming along in mapbox/mapbox-gl-js#4777 and mapbox/mapbox-gl-js#4841. This is a good time to start designing the iOS/macOS equivalent so we know where the impedance mismatches will be.

To restate the points made above, the most natural representation for style JSON expressions would be NSExpression. NSExpression has the benefit of being widely known among iOS developers: as the core of the NSPredicate syntax, it’s used for tasks ranging from filtering an array to searching Core Data or Spotlight. It also takes a pragmatic approach, eschewing compile-time type safety in favor of a compact, memorable format string syntax (with a more structured syntax as a little-used alternative).

The downside is that the iOS and macOS SDKs will diverge syntactically from GL JS, even moreso than the Android SDK. However, the alternatives don’t lend themselves to concision or expressiveness in a library written in Objective-C.

To illustrate why we should use NSExpression instead of hewing to the style JSON syntax, consider the following constraints:

• A JSON-like syntax based on array and dictionary literals would still entail many syntax differences between JSON and Objective-C, such as all the @s. In Swift, all the expressions in draft of expressions examples doc mapbox-gl-js#4836 would have such complex inferred types that they’d fail to compile or take inordinately long to compile.
• A fully object-oriented API like the existing MGLStyleValue API doesn’t scale well. Even though MGLStyleValue provides only a tiny fraction of the expressiveness of the proposed expression syntax, it requires a lot of typing and parameterizing, making it onerous to deviate from example code. By comparison, our implementation of style filters as predicates (Add predicate property to MGLStyleLayer #5973) largely avoided these problems.
• The Android SDK’s approach of nested functions for filters is similar to the structured syntax for NSExpression. Unfortunately, in Objective-C, it would still be a lot of typing and parameterizing, because Objective-C lacks namespaces and strenuously avoids abbreviations. A variation based on standalone C functions could be more compact but would feel foreign and unapproachable. A syntax based on C macros could be just as compact as the style JSON format, but there would be severe limits to its expressiveness. (For example, you wouldn’t be able to nest one macro invocation inside another.)

By committing to NSExpression, we’re signing up to do a lot of translation work, and we may need to perform gymnastics at times to reconcile two very different syntaxes with different feature sets. To the extent possible, I think we should prioritize the following goals:

• Idiomatic: An iOS developer should be able to write basic expressions without consulting example code. Where NSExpression offers a built-in, usable syntax, we should strive to use it instead of inventing our own syntax. This may require some on-the-fly rewriting of expressions.
• Approachable: Some of the less frequented corners of the NSExpression language can be intimidating, particularly the function syntax. We should offer method-level conveniences that keep developers away from those arcane syntaxes.
• Correct: The translation between style JSON expressions and NSExpression should be lossless, even if rewriting means that an expression won’t literally round-trip. We’ll need good unit test coverage like what we have for the filter-predicate conversion code.

I wouldn’t say the following are non-goals, but they’re definitely lower priorities:

• Type-safe: MGLStyleValue suffers from its rigid, compile-time type-safety. It’s a pain to write anything more complex than a constant style value. To the extent the style specification and mbgl allow, we should instead rely on runtime type checking, which enables us to use format strings. In particular, mandatory type annotation would be contrary to both Objective-C (which practices weak typing where convenient) and Swift (where type annotations are often omitted due to type inference).
• Consistency with style JSON: The constraints above all but prevent us from exposing a syntax that closely hews to the style JSON format. If this becomes a problem, such as for React Native Mapbox GL integration, we could implement some JSON conversion functions as a convenience.

Here are some areas that, at this early stage, already look like they’ll be a bit messy:

• Feature property access and type coercion: The CAST() function will probably be news to most iOS developers who are otherwise familiar with expressions.
• Switch statements: We’ll need to rewrite them as nested if/then/else statements.
• String concatenation: Oddly, NSExpression doesn’t come with a built-in function for concatenation, so you have to use the FUNCTION(####, "stringByAppendingString:", ####) syntax. We can provide a convenience method that wraps this function, but we can’t simplify the format string syntax at all.

/cc @anandthakker

[1ec5]

The question came up the other day whether we would consider implementing our own expression system, instead of using NSExpression, if the style JSON expression format ends up dominated by concepts that are difficult to express in NSExpression’s format string syntax. I think the answer is no for the following reasons:

• NSExpression is the language’s answer for expressions. Just as we wouldn’t implement our own NSString or NSArray replacement, we shouldn’t get into the business of defining our own version of this language primitive.
• NSExpression comes with a mature parser. (This is incidentally the only part of NSPredicate we use for filters.) It’s a significant amount of work to reliably and safely parse format strings. We can’t simply reuse the NSString format string functionality, because we wouldn’t want (for example) an NSArray or NSSet to be flattened into whatever their -description methods return, which would be incompatible with our expression language.
• Other than a few basic syntaxes, NSExpression is already at the far end of what the majority of iOS developers would find usable. If style expressions require additional complexity that NSExpression can’t express well, that’s a sign we need to rethink style expressions. Otherwise, we’re dooming this feature to be used only in a few Mapbox-designed styles and never to be used in runtime styling.

[1ec5]

Here are some areas that, at this early stage, already look like they’ll be a bit messy

From a cursory glance at #9439, another impedance mismatch will come from the fact that mbgl::style::expression::type::Array expects an explicit element type. This requirement appears to have been introduced to facilitate stronger type checking at the C++ level, whereas there’s less desire for strong type checking at the Objective-C level.

Objective-C supports annotating an NSArray’s element type via lightweight generics, but it’s lightweight in the sense that only the compiler ever sees the generic parameter; it’s inaccessible to us at runtime. Our options would be:

• Iterate over all the elements of the array to determine the elements’ types. The developer would be able to use the familiar format string syntax, and they can specify or omit the type when creating the NSArray, in keeping with the opt-in nature of Objective-C type checking.
• Implement our own array syntax via the klunky FUNCTION() syntax.

I’d prefer the first approach. Hopefully the arrays passed into an expression will tend to be small, so determining the element type would incur only a negligible performance hit.