Fix EventSeries starting count discrepancy

[dgrisonnet]

What type of PR is this?

/kind bug

What this PR does / why we need it:

The kube-apiserver validation expects the Count of an EventSeries to be at least 2, otherwise, it rejects the Event. There was a discrepancy between the client and the server since the client was initializing an EventSeries to a count of 1.

According to the original KEP, the first event emitted should have an EventSeries set to nil and the second isomorphic event should have an EventSeries with a count of 2. Thus, we should match the behavior defined by the KEP and update the client.

Also, as an effort to make the old clients compatible with the servers, we should allow Events with an EventSeries count of 1 to prevent any unexpected rejections.

Special notes for your reviewer:

I stumbled upon this after finding some weird Server rejected event logs in the kube-scheduler. The error in the logs was in two parts:

'Event "apiserver-7bc6866f9c-2vk6r.17123d9ef072db8c" is invalid: [series.count: Invalid value: "": should be at least 2, eventTime: Invalid value: 2022-09-06 10:09:25.301316 +0000 UTC: field is immutable]' (will not retry!)

One is complaining about the series.count value being lower than 2 and the other about the eventTime value. The latter was recently fixed as part of #111928, but the count issue is still present.

Since series.count returning an empty string was a bit weird I looked into the audit logs to see what was the query sent to the apiserver. I found the following patch request body:

  "requestObject": {
     "series": {
       "count": 1,
       "lastObservedTime": "2022-09-08T19:35:25.972325Z"
     }
   }

Looking at the code, it seems that the empty string was hard coded... https://github.com/kubernetes/kubernetes/blob/master/pkg/apis/core/validation/events.go#L123

So the actual issue was a discrepancy between the client emitting the event and the server validating it.

Looking at the KEP, it seems that the server is correct: https://github.com/kubernetes/enhancements/tree/master/keps/sig-instrumentation/383-new-event-api-ga-graduation#short-examples

Does this PR introduce a user-facing change?

Update the Event series starting count when emitting isomorphic events from 1 to 2.

/cc @liggitt

[k8s-triage-robot]

Unknown CLA label state. Rechecking for CLA labels.

Send feedback to sig-contributor-experience at kubernetes/community.

/check-cla
/easycla

[aojea]

/lgtm
/test pull-kubernetes-e2e-gce-ubuntu-containerd
unrelated failure

/hold

for Jordan review

[liggitt]

this needs a test (probably an integration test) that exercises this reporter in a way that would have caught the failure

e.g. walk an event through creation, introduction of a series, and incrementing of the series, and verify the resulting event API object gets updated and looks correct at each stage

[aojea]

/lgtm cancel
Damien ping me in slack if you need help with the test

[leilajal]

/triage accepted

[dgrisonnet]

I added an integration test that is failing because it is hitting the data race reported in #92500, I'll send another PR to fix it.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

• Mark this PR as fresh with /remove-lifecycle stale
• Close this PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[dgrisonnet]

/test pull-kubernetes-integration
/remove-lifecycle stale

[dgrisonnet]

@liggitt can you please have another look at this PR? I added the integration test as you requested.

/cc @wojtek-t

[liggitt]

I'll defer review of the test to @aojea, the client update lgtm

[wojtek-t]

/lgtm
/approve

Thanks!

/hold cancel

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dgrisonnet, wojtek-t

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• staging/src/k8s.io/client-go/tools/events/OWNERS [dgrisonnet,wojtek-t]
• test/OWNERS [wojtek-t]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

LGTM label has been added.

Details

Git tree hash: 17053a94ac2111c582248d2a47337f09d81ee790

[alculquicondor]

Can we backport this? Do we still have a chance for 1.24?

[alculquicondor]

oh, the deadline is today https://groups.google.com/a/kubernetes.io/g/dev/c/A_hz52m4GUA/m/pWo7tJ0_BAAJ?utm_medium=email&utm_source=footer