Pointer Reform

[andrewrk]

EDIT

• Latest iteration of the proposal: Pointer Reform #770 (comment)
• Progress: Pointer Reform #770 (comment)

---

& only used for address-of, no longer designates a pointer type. Necessary because of #588

• ^ pointer to exactly 1 thing.
• [*] pointer to a block of memory of unknown length
• [*]null pointer to block of memory, null-terminated (or 0 terminated for integers). proposal: type for null terminated pointer #265
• [] pointer to a block of memory with runtime known length. status quo slices.
• []null pointer to a block of memory with runtime known length, with a null/0 at ptr[len]
• [N] pointer to a block of memory with comptime known length
• [N]null pointer to a block of memory with comptime known length, and a null/0 at ptr[N]

All of them support pointer indexing and slicing except ^. Only [*] supports pointer arithmetic. All of them implicitly cast to [*]. []null and [N]null implicitly cast to [*]null.

• &ptr[x] and &foo always gives a ^.
• ptr[x..y] with comptime known x and y gives a [N].
• array[x..] gives a [N].

new array syntax

var array: 4*i32 = undefined;

Now it is clear whether you should do &array or &array[0]. Don't use &array. If you want a [N]T, e.g. a pointer with comptime known length, use array[0..]. If the function wants to access more than one element, you'll do this. Otherwise, &array[0], will give ^T, which would trigger a compile error if the array was length 0, and only this element can be accessed via this pointer.

This paves the way for #733
See also #386
See also #568

[Ilariel]

If possible I would like to have some other symbol/keyword for:

^ pointer to exactly 1 thing.

This is because ^ isn't ergonomic on all keyboard layouts. On Windows with some international/non-English keyboard layouts you have to type it twice to get ^^ and then you have to remove the extra every time you write it. See this superuser question
Sure it is used in bitwise xor, but you tend to have more pointers in your program than bitwise xors.

[andrewrk]

That's good to know. Do you have a suggestion for what other symbol to use?

[Ilariel]

To be honest I have only few possibly reasonable ideas as most commonly used or reasonable sigils have been used for something in Zig already.

1. $ -symbol is widely used in many programming languages. However this would a "foreign" sigil to learn as it is not used for this purpose in other languages.
2. ref/ptr -keyword, keyword bloat, takes a common variable name. However might be friendly for the reader (ref T)/(ptr T)
3. Change the current & operator to keyword/builtin (@)address_of and then use & as a sigil.

[Hejsil]

Is the new array syntax flexibly like multiply? Can I do these?

var array1: i32*4 = undefined;
var array2: 4*4*i32 = undefined;
var array3: 4*(4*i32) = undefined;

Or is it restricted to <comptime_int>*<type>? Aka, I would have to write array2 like this:

var array2: (4*4)*i32 = undefined;

[andrewrk]

@Hejsil great questions. I think that new array syntax is no good because of this. But we have to make something different than [N]T to distinguish from pointers.

[thejoshwolfe]

4*i32 is no good, because the * operator elsewhere doesn't change the type of something. Even ** turns arrays with a child type into another array with the same child type; it only changes the size. We need a way to turn a scalar into an array, which is not like any infix operator in the language.

Perhaps array[4]i32 where array is a keyword? Having a [4]i32 somewhere in there fits nicely with [4]i32 being a pointer to such.

I expect people will mistakenly declare their structs with [4]i32 instead of whatever the actual array syntax ends up being. Then they'll initialize the pointer to undefined thinking they initialized the elements to undefined, and then begin assigning into the elements, which will cause undefined behavior at runtime. Seems like a footgun.

[raulgrell]

I agree with @Ilariel, my keyboard is one of these (Portuguese) - If you press the ^ key once, followed by a vowel you get, for example, â. But with a consonant, you get for example, ^w. But tapping twice and erasing can break your typing flow - It's been surprisingly hard to build muscle memory for it. Other characters like this include the tilde ~ and the backtick/grave accent. The caret/circumflex accent ^ requires shift to be pressed and is on the same key as the tilde. The backtick/grave is on the same key as the very similar looking acute accent. I'm not sure about other keyboard layouts but these are very uncomfortable characters for the Portuguese.

I also liked @Ilariel's 2nd and 3rd suggestions. I don't dislike the idea of a ref/ptr keyword, but I find &?&&?T more readable than ref ? ref ref ? T for types. I think I'd prefer to keep the & as a reference type and instead use a builtin or an operator like # or $ for address-of.

As for the array syntax, would N[]T be possible? An array is a pointer to a block of memory with runtime known length just like status quo slices, so it's conceptually consistent at least.

const a: 3[]u32 = 3[]u32{1, 2, 3};
const a = 3[]u32{1, 2, 3};

const b = (2*2)[]u32{1, 2, 3, 4};

@thejoshwolfe made a good point about * not changing something's type, and ** changing something's size. But changing an array of one size into an array of another size is also changing the type of the array. Why not use ** itself?

const a: 3**u32 = 3**u32{1, 2, 3};
const a = 3**u32{1, 2, 3};

const b = (2*2)**u32{1, 2, 3, 4};

const c = a ++ b;

It communicates that you're creating a value type which is the result of putting N units of that value type together. If you allow ** and ++ to operate on scalars, the following could create an anonymous tuple.

const T = 4**u32 ++ bool 

const t : T = 4**u32{1,2,3,4} ++ true;
const t = 4**u32{1,2,3,4} ++ true;

[thejoshwolfe]

We can't use ** for making arrays out of scalars, because types are comptime values.

const State = 256**u8; // formerly known as [256]u8
const States = 4 ** State; // is this [1024]u8 or [4][256]u8?

[raulgrell]

I thought the both the ++ and ** operator were already only available with comptime values. In the example you gave, perhaps:

const State = 256**u8; // formerly known as [256]u8
const States = 4**State; // This would be [4][256]u8, ie (4 **(256**u8))
const States = (4 * State.len) ** State.child_type  // this is [1024]u8

Though I can see how this isn't ideal.

EDIT

Just to consolidating my 2 cents to the pointer-to discussion after a bit of thought.

In C you have & as the address-of operator, and pointers are declared with a *
In C++ you have std::addressof, and references are declared with a &

In Zig, the 'pointer to exactly one thing' is closer to a C++ reference than a C pointer, so it would make sense to stay close to their interface, and use a builtin @addressof function.

[tgschultz]

So it is documented and can be discussed here, we came up a possible solution in IRC:

• * pointer to exactly 1 thing.
• [*] pointer to a block of memory of unknown length
• [*]null pointer to block of memory, null-terminated (or 0 terminated for integers). proposal: type for null terminated pointer #265
• [] pointer to a block of memory with runtime known length. status quo slices.
• []null pointer to a block of memory with runtime known length, with a null/0 at ptr[len]
• [*N] pointer to a block of memory with comptime known length
• [*N]null pointer to a block of memory with comptime known length, and a null/0 at ptr[N]
• [N] block of memory (an array).

Advantages:

• * is a familiar symbol for "pointer".
• * doesn't have the keyboard issues ^ does.
• *,[*],[*N] are consistent.
• all blocks use [ ]
• array syntax doesn't change

Disadvantages:

• Slice syntax doesn't contain * even though it points to data 🤷‍♂️
• * in [*N] is ambiguous with * as a dereference operator

So if we went this route we'd need a new dereferencing operator. Personally I favor that anyway since prefix * is already a bit weird.

var z = x**y;  //x * (*y)
var a = *s.m.v; //dereferences "v", not s

One thought I had, and I realize this is a bit strange but hear me out, is postfix .. My reasoning is that . already sort of dereferences implicitly when used with structs.

var s = MyStruct{.v = 10};
var sp = &s;
var v = sp.s; 

so all we'd be doing is extending this property to non-structs, really.

var s = i32(25);
var sp = &s;
var v = sp.;  //dot dereferences sp
//...
var p = Point{.x = 0, .y = 10};
var pp = &&p;
var x = p..x;  //currently (*p).x, 

Advantages:

• No new symbols
• Consistent with current usage of symbol

Disadvantages:

• Subtle, might be easy to miss
• Unfamiliar to users of other languages
• Confusing when used with range operators .. and ...

So then we'd need new range operators too. -> is available unless it is resurrected for return types.

Other options for deref:

• Pre or postfix ^ which has noted issues with some keyboards and would be ambiguous with xor in the postfix case.
• Pre or postfix $, which is not unprecedented but is admittedly kind of ugly. introduces a new symbol. could confuse people used to scripting languages.
• Postfix [0], which is very C and unambiguous, but does make the pointer look like a block even though it isn't.
• Postfix .0, which looks really strange and kind of implies some kind of indexing.
• Postfix >, unambiguous as far as I can tell, easy to spot. Everyone seems to hate it though.

[thejoshwolfe]

[*N] pointer to a block of memory with comptime known length

Why not *[N]? That already means a pointer to a block of memory with comptime known length according to the rest of the proposal, and it doesn't have the ambiguity with *N.

[*N]null pointer to a block of memory with comptime known length, and a null/0 at ptr[N]

Instead of this concept, introduce [N]null which is an array of comptime known length with a null/0 at arr[N]. Then we just do *[N]null for a pointer to it.

Other options for deref:

Zig's grammar depends on knowing if we are at the end of an expression or in the middle of an operator. This means we can't have postfix operators that are identical to infix operators. Here's an example of the ambiguity using ^ as proposed for postfix pointer deref:

// this is ambiguous
const a = b^(1);

// b is a pointer to a function (or double pointer to a function),
// which is being called and given the parameter 1.
const a = (b^)(1);

// b is some integer being xor'ed with 1.
const a = b ^ 1;

We absolutely cannot have ambiguity between infix and postfix operators. This means ^, *, and > can't be used as postfix operators.

There is no problem with ambiguity of infix and prefix operators though, such as with * for deref and multiplation, - for negative and subtraction, & for address of and bitwise and, etc. The ability to distinguish between prefix and infix operators is what we get for having the above limitation with postfix and infix. And since it's so important to have - and ( as infix and postfix operators, the tradeoff to allow prefix/infix instead of postfix/infix is a no-brainer.

One thought I had, and I realize this is a bit strange but hear me out, is postfix ..

This actually does not suffer too horribly from the above ambiguity concern, because . isn't really an infix operator. After a ., you have to have an identifier, and an identifier can't ever be a postfix or infix operator. That being said, it still looks pretty horrible.

a.0 doesn't seem as bad, because at least there's something after the dot. This eliminates the problems with .. as you pointed out. But a.0 is bad because it begs the question: can I do a.1? It looks like the 0 is actually a zero, when it's really more of a keyword. 0 doesn't function like that in any other syntactic context, so this seems pretty bad too.

Personally I favor that anyway since prefix * is already a bit weird.

I'm not too concerned about prefix vs postfix, since this language (and pretty much every other language) has operator precedences that will get weird sometimes like this. Instead of moving operators around, I would propose requiring parentheses sometimes, if the confusion is bad enough. See #114.

The biggest problem I have with * meaning "pointer to" and * meaning "deref pointer" is that those are opposite directions. C has this problem, and I think it's one of the reasons children find pointers hard to learn in C (that and lack of respectable compile errors). If * means "pointer to", then we do need a new "pointer deref" operator, but it could be prefix or postfix. We could use ^ for prefix deref, or even >. There's a lot of space for new prefix operators, but less so with postfix/infix operators.

[tjpalmer]

Ignoring the syntax, I am really happy with the idea of separating pointer-to-1 from pointer-to-unknown-quantity and so on. (Sad that caret doesn't work easily.)

[raulgrell]

a.0 doesn't seem as bad, because at least there's something after the dot. This eliminates the problems with .. as you pointed out. But a.0 is bad because it begs the question: can I do a.1? It looks like the 0 is actually a zero, when it's really more of a keyword. 0 doesn't function like that in any other syntactic context, so this seems pretty bad too.

We could have a postfix .&. Borrowing @tgschultz's example:

var s = i32(25);
var sp = &s;
var v = sp.&;  // .& dereferences sp

var p = Tuple {.x = s, .y = sp, .z = &sp};
var pp = &&p;
var x: i32 = pp.&.x;  //currently (*pp).x, 
var y: i32 = pp.&.y.&; // currently *(*pp).y
var z. i32 = pp.&.z.&.&; // currently **(*p).z

It doesn't imply indexing, no ambiguities with range syntax, analogous to fields in structs: get value at field vs get value at address.