Wrong magic methods sequence with `??` operator

[nicolas-grekas]

Description

When ?? is used to check a property using magic methods, PHP unconditionnaly calls both __isset and __get even when __isset did actually set the property.

Reproducer: https://3v4l.org/cDlNW

This script:

#[AllowDynamicProperties]
class A
{
    public function __get($n)
    {
        echo __FUNCTION__, "\n";

        return $this->$n;
    }

    public function __isset($n)
    {
        echo __FUNCTION__, "\n";

        $this->$n = 123;

        return true;
    }
}

$a = new A;

echo $a->foo ?? 234;

echoes this:

__isset
__get
123

while it should echo this:

__isset
123

PHP Version

All since 7.0.6

Operating System

No response

[damianwadley]

__isset only confirms whether or not the "property" exists. That doesn't have to mean that the property exists and is not null.
So the call to __get is still necessary.

edit: Ah, you mean how the property gets created between the two calls.

[iluuu1994]

If you return false from __isset that's what happens. With true, the concrete value must be fetched. Edit: It seems you're asking that the existence of the property must be repeated after isset. I don't think isset should be used as a mechanism to create properties...

[nicolas-grekas]

Now seeing your edit @iluuu1994. But first of all yes the property should be fetched, but since __isset does set the property, the fetch shouldn't get through __get

I don't think isset should be used as a mechanism to create properties...

Nothing prevents it and that's actually the only way to properly implement lazy object so yes, it can :)

[iluuu1994]

I don't think there's a good reason to add another check. Can't you just return the property you create in __isset in __get? Adding a check slows down the 99+% case where __isset doesn't create a property.

[nicolas-grekas]

?? operation with magic methods is rare so that perf shouldn't be an issue. I also expect the call to be fast since it's just a missing hashmap check. And yes, this behavior prevents writing correctly behaving lazy objects - we cannot make them truly transparent.

[Girgias]

I must agree with @iluuu1994 here, __isset() shouldn't be used to assign properties, and adding a check is not something I am in favour of.

[nicolas-grekas]

I'm sorry but this doesn't make sense to me. The fact that __isset can be used to initialize properties is irrelevant to the issue. Actually, it's a critical capability that is leveraged since more than a decade to implement lazy objects. You might not be familiar with how those are implemented but the general idea is simple: all properties are unset when the object is created, and magic methods are added by inheritance to defer initialization to the moment when any property is accessed - that being via any magic methods, __isset included. There are critical piece of infrastructure that rely on lazy objects - many based on Doctrine entities and/or when using lazy services (eg Symfony DI container but not only it).

The current behavior just breaks the semantics of magic methods. They are supposed to be called only when a property isn't accessible, and the call to __get is just missing the check to guard this behavior.

Can you please expand on why you think things behave as expected? What's the argument to not fix this?

[bwoebi]

Currently __isset and __get are called as a pair: if (!exists()) { if (__isset()) return __get() else return null }.

Your proposal is to do if (!exists()) { if (!__isset()) return null; } if (!exists()) { return __get(); }.

This behaviour comes from the initial bugfix for PHP 7, it was probably done that way to just make it work, and maybe not the most fully thought out behaviour, but it's how it works today.

I don't think you should argue it would break critical pieces of infrastructure, otherwise they'd be long broken, since PHP 7 that is. @nicolas-grekas

I do however think, that from a theoretical perspective __get() should not be called, if it exists, period. It would be a break in long-standing behaviour though and as such only go into PHP 8.4, if this is changed.

I also don't think performance is a major concern in this case, as the relative overhead of __isset() and __get() calls must be far bigger than a simple undef check (for declared props) or a hash_exists (for dynamic props).

[iluuu1994]

Patch

diff --git a/Zend/tests/coalesce_is_create_property.phpt b/Zend/tests/coalesce_is_create_property.phpt
new file mode 100644
index 0000000000..10927b6289
--- /dev/null
+++ b/Zend/tests/coalesce_is_create_property.phpt
@@ -0,0 +1,22 @@
+--TEST--
+Coalesce IS fetch should repeat property existence check after __isset
+--FILE--
+<?php
+#[AllowDynamicProperties]
+class A {
+    public function __isset($prop) {
+        echo __FUNCTION__, "\n";
+        $this->$prop = 123;
+        return true;
+    }
+    public function __get($prop) {
+        throw new Exception('Unreachable');
+    }
+}
+
+$a = new A;
+echo $a->foo ?? 234;
+?>
+--EXPECT--
+__isset
+123
diff --git a/Zend/zend_object_handlers.c b/Zend/zend_object_handlers.c
index 6a22156d18..d2b34ff9bf 100644
--- a/Zend/zend_object_handlers.c
+++ b/Zend/zend_object_handlers.c
@@ -589,6 +589,36 @@ ZEND_API uint32_t *zend_get_property_guard(zend_object *zobj, zend_string *membe
 }
 /* }}} */
 
+static zval *zend_std_read_dynamic_property(zend_object *zobj, zend_string *name, void **cache_slot, uintptr_t property_offset)
+{
+	if (EXPECTED(zobj->properties != NULL)) {
+		if (!IS_UNKNOWN_DYNAMIC_PROPERTY_OFFSET(property_offset)) {
+			uintptr_t idx = ZEND_DECODE_DYN_PROP_OFFSET(property_offset);
+
+			if (EXPECTED(idx < zobj->properties->nNumUsed * sizeof(Bucket))) {
+				Bucket *p = (Bucket*)((char*)zobj->properties->arData + idx);
+
+				if (EXPECTED(p->key == name) ||
+					(EXPECTED(p->h == ZSTR_H(name)) &&
+						EXPECTED(p->key != NULL) &&
+						EXPECTED(zend_string_equal_content(p->key, name)))) {
+					return &p->val;
+				}
+			}
+			CACHE_PTR_EX(cache_slot + 1, (void*)ZEND_DYNAMIC_PROPERTY_OFFSET);
+		}
+		zval *retval = zend_hash_find(zobj->properties, name);
+		if (EXPECTED(retval)) {
+			if (cache_slot) {
+				uintptr_t idx = (char*)retval - (char*)zobj->properties->arData;
+				CACHE_PTR_EX(cache_slot + 1, (void*)ZEND_ENCODE_DYN_PROP_OFFSET(idx));
+			}
+			return retval;
+		}
+	}
+	return NULL;
+}
+
 ZEND_API zval *zend_std_read_property(zend_object *zobj, zend_string *name, int type, void **cache_slot, zval *rv) /* {{{ */
 {
 	zval *retval;
@@ -638,31 +668,9 @@ ZEND_API zval *zend_std_read_property(zend_object *zobj, zend_string *name, int
 			goto uninit_error;
 		}
 	} else if (EXPECTED(IS_DYNAMIC_PROPERTY_OFFSET(property_offset))) {
-		if (EXPECTED(zobj->properties != NULL)) {
-			if (!IS_UNKNOWN_DYNAMIC_PROPERTY_OFFSET(property_offset)) {
-				uintptr_t idx = ZEND_DECODE_DYN_PROP_OFFSET(property_offset);
-
-				if (EXPECTED(idx < zobj->properties->nNumUsed * sizeof(Bucket))) {
-					Bucket *p = (Bucket*)((char*)zobj->properties->arData + idx);
-
-					if (EXPECTED(p->key == name) ||
-				       (EXPECTED(p->h == ZSTR_H(name)) &&
-				        EXPECTED(p->key != NULL) &&
-				        EXPECTED(zend_string_equal_content(p->key, name)))) {
-						retval = &p->val;
-						goto exit;
-					}
-				}
-				CACHE_PTR_EX(cache_slot + 1, (void*)ZEND_DYNAMIC_PROPERTY_OFFSET);
-			}
-			retval = zend_hash_find(zobj->properties, name);
-			if (EXPECTED(retval)) {
-				if (cache_slot) {
-					uintptr_t idx = (char*)retval - (char*)zobj->properties->arData;
-					CACHE_PTR_EX(cache_slot + 1, (void*)ZEND_ENCODE_DYN_PROP_OFFSET(idx));
-				}
-				goto exit;
-			}
+		retval = zend_std_read_dynamic_property(zobj, name, cache_slot, property_offset);
+		if (retval) {
+			goto exit;
 		}
 	} else if (UNEXPECTED(EG(exception))) {
 		retval = &EG(uninitialized_zval);
@@ -693,6 +701,13 @@ ZEND_API zval *zend_std_read_property(zend_object *zobj, zend_string *name, int
 			}
 
 			zval_ptr_dtor(&tmp_result);
+
+			retval = zend_std_read_dynamic_property(zobj, name, cache_slot, property_offset);
+			if (retval) {
+				OBJ_RELEASE(zobj);
+				goto exit;
+			}
+
 			if (zobj->ce->__get && !((*guard) & IN_GET)) {
 				goto call_getter;
 			}

I agree with @bwoebi in that this is more of a missing feature than a bug. The "property fetch" is currently considered an atomic operation. zend_std_read_property does not actually consist of an "isset" and "get" phase. The properties value is fetched in the same step. Because there's no such equivalent in userland, we're forced to call both __isset and __get.

Anyway, returning false from __isset after creating a property is somewhat of a contradiction. We can act as if the property isn't there and call the ?? fallback (which normal properties cannot do, because they don't call __isset), or use the property anyway (which feels like it's going against what the user said). I went for the former, which is also what @bwoebi has expressed in his pseudo-code.