'databricks jobs list' is unusably slow on large workspace

[lennartkats-db]

databricks jobs list --log-level debug currently does paginated jobs/list request, but since there is a lot of time between requests the command takes incredibly long on large workspaces such as the "dogfood" testing workspace.

[pietern]

As of a couple of days ago it uses token-based pagination which is about ~2x faster.

Follow-ups include:

• Increase the number of entries returned per page
• In the SDK, return an iterator for list functions such that the CLI can produce a streaming output

[andrewnester]

This PR to Go SDK should address the issue
databricks/databricks-sdk-go#543

[andrewnester]

This has been addressed with this PR

#1202